TimeGate is a serious game environment that allows groups of people to interactively explore a virtually reconstructed historical site or landscape.  It uses large-screen projection of a real-time visualisation of a 3D virtual reconstruction of a site or landscape, combined with a gesture-based natural interaction, allowing to navigate through this virtual world and manipulate objects in this virtual world.  The natural interaction of TimeGate is being implemented through arm gestures that are being detected by a Kinect2 camera.

Ename 1290 virtual walk

TimeGate is a virtual reality experience for groups, based upon natural interaction

The navigation of TimeGate consists of a forward movement (right arm extended forward) and left or right rotation.  The forward speed depends on the amount of extension of the arm. One can also look up and down (right arm up or down).  The gestures are recorded with several people, male and female, adults and children and fine-tuned in a gesture editor.


TimeGate recording of gestures at our offices

In this way, the TimeGate system has an excellent interaction for all kinds of users, even for users in a wheelchair.  The TimeGate system can be used as an educational game (with small auditory stories linked to the objects) or as a virtual tour (with a guide showing you around in the virtual reconstruction of the site).


Selecting an object by pointing at the object with two hands

The movement can take place along a predefined path (visualised as a white line) or can be free within an enclosed space.  In this way, the navigation through a large site or landscape is being optimised, so that no time is lost in exploring parts of the site that do not contribute to the understanding of the site (when making a virtual tour) or the story (when playing the educational game).


TimeGate allows you to explore vast areas by following a predefined path

In the new version of TimeGate, one can choose to enter in specific buildings or enclosed spaces, simply by selecting the door.  Objects and doors that can be selected are visible by magic sparkles emanating from it.  You select objects by pointing at the object with both arms.


Just select the door of the pub to enter

In such enclosed spaces, you can roam around freely. When you run against the furniture, it will be pushed aside or even tip over.  All interactive objects (that sparkle) can be picked up for manipulation. Once selected, they will float in front of you at arms lenght.


All sparkling objects can be picked up and have a story

You can look at the object that you have picked up from all sides by moving your two hands like holding a ball.  While you do this, you can listen at the story of the object if  you play it as an educational game (i.e. if the story sound channel is turned on).  If used as a virtual tour, there will be no story but the guide can talk about the object instead.


Objects that have been selected float in front of you and tell their story

When you play it as an educational game, there is no guidance on which object is important and holds the clue to the developing story, so you need to do some exploration.  If used as a virtual tour, the guide can select any object that is useful to support the explanation of the building and show it from all sides.


Take objects that might be useful and leave the room by walking out…

Every object that you can pick up, can be taken with you (make a gesture with both hands to put it in your pocket) or be put back (make the gesture with both hands to put it back).  Along the game, you will need some objects, so think carefully what to take with you. Some objects are too big or too heavy to take with you, but they nevertheless tell a story that can provide you with useful information for accomplishing the goal of the game.  You leave by just walking out and continue following the predefined path.


and continue following the path to explore the rest of the site…


to play 17th century skittles in the courtyard of the Swan Inn…


or fly over the village and the abbey and land anywhere

The new version also allows to fly around as a bird over the village and the abbey, and play skittles, a 17th century version of bowling. Object do have physics.  Other interactions will be added such as opening a book and browsing through it.

The TimeGate system and its possible applications have been registered at the BOIP in May 2016.