Skip to content

Microsoft granted patent for 3D holodeck | Ars Technica

September 11, 2012

Microsoft patent shows Holodeck-style, full-room “immersive display”

From Ars Technica by – 9/11/12

mspatent 640x391 520x317 Microsofts patent for a 3D holodeck is all sorts of (awesome) Ray Bradbury fanfiction

The patent describes this figure as showing how the device could “project a peripheral image in a 360-degree field around [the] environmental display.” Source: Microsft / USPTO

A newly published patent shows Microsoft potentially planning to take console gaming beyond the barriers of the TV screen by projecting a “peripheral image” around a room, providing a 360-degree view of a virtual scene.

Microsoft’s patent for an “immersive display experience” was published by the US Patent Office last week after being filed back in early 2011. It describes a standard video game system with a connected “environmental display” capable of projecting a panoramic image that “appears to surround the user.”

Such a projector wouldn’t replace the central TV display used in current consoles, but it would provide a “peripheral image” that would “serve as an extension” of that primary display. The purpose, of course, is to extend the gaming environment outside of the TV screen, so a player could, for instance, “turn around and observe an enemy sneaking up from behind.”

The display device as described is integrated tightly with a depth-sensing camera system (read: Kinect) that could even be housed in the same casing as the environmental projector, according to the patent. This device could be a standard two-camera, structured-light-sensing system like the current Kinect, or a more sophisticated model that could include “multiple image capture devices” to “stitch a panoramic image from a plurality of captured images” pointed in all directions around the room.

Regardless of the form, the depth-sensing camera described in the patent aids the environmental projector by sensing the layout and topography of the room. This allows the projector to provide color and distortion correction, so the projected images look correct even when cast against different walls and pieces of furniture.

In addition, the environmental display could use player position information provided by the camera system to “adjust an apparent perspective of the peripheral image display,” making sure the environment continues to look natural as the player moves around the room. The patent even describes creating a “shielded” region in the projected image where the player is standing, so light doesn’t shine directly in a player’s eyes as they look in the direction of the projector.

Projected images in the system described by the patent “may be displayed at a lower resolution than the primary image,” but the document suggests this would not “adversely affect [the] user experience.” The projected images wouldn’t necessarily just sit flat against the wall, either—they could work with stereoscopic 3D glasses for an added sense of depth and realism, according to the patent. That last part gels somewhat with the “Fortaleza glasses” concept for a heads-up, augmented reality display mentioned in Microsoft documents leaked in June.

This figure from the patent shows the projector detecting a user’s position and “shielding” that area from the projected image. Source: Microsoft / USPTO

Holodeck Version 0.5?

The system described in this patent may never turn into a full-fledged product, but this kind of environmental system could be a real revolution in the way we play and think about video games. One of the main factors limiting the current potential of motion controls, as I see it, is the need to constantly stay facing the TV screen in order to know what’s going on in the game. This leads to a lot of games where leaning and shuffling to the side serves as a relatively poor substitute for actually turning your body or walking parallel to the screen.

With a game projected on all four walls, however, you’d be free to really walk around the entire room and look in any direction without missing out on what’s going on. Imagine a new version of Dance Central that projected actual Arthur Murray-style footprints on the floor, or an adventure game where you actually had to explore the room to find hidden treasures, or a first-person shooter where you physically turn toward a muzzle flash projected on the wall to your right, rather than simply twisting in response to a red flash on the right edge of the screen.

If this kind of environmental display worked as advertised, it could obviate the need for head-mounted virtual reality displays like the Oculus Rift, which use sophisticated head-tracking to simulate an environment that surrounds you. If the image of the game world is actually surrounding you, there’s no need to calculate which sliver of that world to show the player based on where they’re looking.

That said, I am a bit concerned about how well a single 360-degree projector can actually transform a room. Unless the projector described here was mounted on the ceiling, I don’t see how it would avoid the significant problem of image-blocking shadows, not just from the player but from assorted furniture and detritus in the room. Color and image-distortion correction is a nice idea, but if a piece of the scene is supposed to appear in the back corner of the room, having it instead appear on the Lay-Z-Boy sitting in front of that corner is going to be a bit disconcerting.

Still, it’s interesting to see Microsoft’s R&D department thinking about ways to extend games past the black borders of our flat-screen TVs in a potentially revolutionary way.

via Microsoft patent shows Holodeck-style, full-room “immersive display” | Ars Technica.

Advertisements
2 Comments
  1. tony koorlander permalink

    There’s nothing 3d in this application..no more than Cinerama at home. Big company filing ethereal claim to the future with no real advance over present technologies and indeed copying extant installations. It’s a moneymaking scam.

  2. Hi Tony – Thank for your comment. I will defer to your judgment on this considering your background. I just thought it was interesting “grist to the mill” for the 3D futurists out there.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: