Project Natal and jumping the Fourth Wall

Video Games loom over other forms of entertainment in a significant way, they are much less bound by the fourth wall – the distance between the actors and the audience. Though most video games have grandiose themes - where players typically command gargantuan armies and civilizations, create cities, wield cutting edge weaponry, perform cool stunts beyond the Matrix, or rock the bejesus out of thousands of screaming fans – their success largely depends on how much can they engage and immerse the player. Video game designers employ many techniques to this end – from improving the realism of animation effects, to using atmospheric sounds, to incorporating engaging, emotional storylines with top notch voice acting (like David Hayter’s gravelly Solid Snake in the Metal Gear series).

The Wii commercialized a revolutionary idea of engaging the audience, letting them control the

character on screen through physical gestures instead of mashing buttons on a controller. However, players still depended on a “Wii-mote” to communicate with the console, and hence the range of motions was limited to simplistic arm gestures. Microsoft’s Project Natal is a bold new step forward in immersing the player physically into the game world.

The technology behind the Natal relies on a simple monochrome camera to capture a player’s motions. Though not as advanced as the motion-capture cameras used by films, it claims to reflect most player actions within 10 milliseconds and to recognize any object introduced before the camera within 160 ms. A 3d projector captures the player’s body motion, without any needs for sensors or physical attachments, The motion is compared to a database of human physiology to determine the movement and translate that onto the screen action. IGN has posted an interesting video presentation from the Consumer Electronics Show which can be seen here.

The implications of the technology are immense. It could be an engaging, fun solution to couch potatos’ weight woes and bring fitness into the living room. It could be employed to operate computers and other devices via gestures. It is a step forward in technology’s transition towards gesture and voice driven operations, reminiscent of Pranav Mistry’s exciting TED talk. Gaming is in for exciting times ahead.