Microsoft IllumiRoom turns Kinect on it’s head; may be the future of gaming


While Apple seems to have stopped experimenting and looks more focused on litigating competitors out of the market, Microsoft is silently innovating. Exhibit 1: IllumiRoom, a proof-of-concept technology that immersively projects the game environment into your physical space using the Kinect sensor and a projector.

The concept is simple: the Kinect camera surveys your physical space to ‘see’ the layout and the objects that are out there, and a projector extends elements from the game environment – such as the road in a 3D racing game or snowflakes falling in a FPS environment, in order to bring these elements out into your living room in real time “without any need to custom pre-process the graphics” in the game itself (which is to say, to have it work with any existing game).

We’ve long believed that Kinect, a fairly simple technology, is going to be extremely important in the future of gaming and/or human/machine interaction. Despite the fact that some hardcore gamers think of it as a novelty, we can imagine the Kinect devices of the future having multiple placements and not only in front of the user, and functioning not just as cameras or sensors reading the environment, but as a system that is projecting information outward as well.

Go here for more information on IllumiRoom.

  • Fred Thompson

    Very interesting. However, why scan the room if not to modify the graphics which will then be projected? Their statement doesn’t make sense. They can’t change the shape of the projector’s lens so they MUST modify the graphics. It also doesn’t make sense to claim this can work with existing games unless those games already have support for the projector. Maybe Xbox games are built to a standard that requires rendering detail beyond the edges of the screen already. If so, thoes details are already being calculated. If not, there is no calculation-free way of creating those extra details. To illustrate, look at the part of the video which shows building outlines and such. That data has to be created somewhere. Similarly, light weapons which extend beyond the screen and are in motion MUST be calculated in real time to match the screen. Something’s not correct in the MS description.

    This might be a great way to augment video, too. It moves beyond screen edge lighting for ambience. Even if all that is projected is a genre-specific frame around the display.

    Wonder how it could be used for remote control or robotics or virtual presence. One of the really cool M$ research projects from a few years ago was high-resolution in the center of maps and lower resolution framing the area. Think of a top-down view of a city where all the individual outlines of buildings create “noise” that distracts from the area of interest but is also needed for context. IIRC, the guy who worked on the Win7 volume mixer design was also part of that project.

    • Samer Kurdi

      The direct quote is “Our system uses the appearance and the geometry of the room (captured by Kinect) to adapt the projected visuals in real-time without any need to custom pre-process the graphics”.

      Perhaps I misunderstood what is meant by “without any need to custom pre-process the graphics”, which I took to mean that it performed its magic without customizing the game graphic resources, while they meant something else. What I do know, however, is that the video above represents the game experience and was not altered.

      • Fred Thompson

        I was commenting about their words. Maybe they mean the game engine doesn’t adjust the graphics to the room but that another software layer does that, similar to the way textures are wrapped about a 3D surface. That’s the only thing which makes sense. It looks great. The game is still in the screen “window” but it should be easier to mentally blend into the game.

  • Hughie

    This + 3D viewing. That is all.

  • Someone

    “Apple seems to have stopped experimenting and looks more focused on litigating competitors out of the market.”

    That’ll be because all they’ve ever done is steal others’ ideas anyway – From PARC’s WIMP GUI to BSD, they’ve never had an original idea in their existence

    • Samer Kurdi

      Hahah I can’t believe it took so long for someone to react to that one! I dunno if I agree with what you said 100%, but I do not intend to be caught on record as defending Apple.