While virtual reality games are built from the ground up to provide freedom of movement — they wouldn’t be very fun otherwise — the same can’t be said of VR video. There’s always a level of disappointment when viewing VR video because you’re just viewing it — you can turn your head around to look in all directions, but if you see something cool off in the distance, you can’t walk over to it. That might change with a new professional camera Lytro is developing. Called the Lytro Immerge, it takes Lytro’s existing light detection technology and uses it to not only take 360° video, but recreate a dynamic scene that you can interact with in VR.
Immerge is using the same technology Lytro implemented in their Illum camera last year. That camera took still images, but used a sensor that captured what Lytro called megarays, as opposed to megapixels — the camera would record entire rays of light, including three-dimensional depth information. Because light reflects off all surfaces, this three-dimensional depth information could be used to track exactly how far away objects were from the camera, allowing photographers to adjust focus after shooting.
In the Immerge, this idea has been applied to video. By creating a massive spherical camera, Lytro has been able to create an entire light field — the bits of information that make up a normal photo or video are two-dimensional in nature, made up of light as it hits the camera’s lens. Lytro’s light field is, genuinely, an entire environment recorded, with data accounting for exactly where each object is within a scene. Using that information, video taken using the Immerge can be processed and made into virtual reality experiences you can actually move around in. Instead of flying over a volcano attached to a virtual helicopter and moving your head to look around, you can actually walk into that volcano. It’s a massive difference that has the potential to make virtual reality genuinely engaging.
In addition to the Immerge, Lytro is developing their own light field editing software, which will be compatible with the major VR platforms of the future, like Oculus Rift, HTC Vive, and Sony’s Project Morpheus. Lytro will also be bundling in an entire server to store the data recorded by Immerge — I’m not going to speculate on exactly how much storage even a few-seconds long scene would require, but I’m willing to bet there’s no SD card in existence that could hold it.
Immerge is still in development, but word is it should be available to rent or buy sometime early next year, with CEO Jason Rosenthal suggesting that it will cost in the hundreds of thousands of dollars. Needless to say, this thing is for the big-budget gaming and movie studios out there, meaning that this technology could shape the future of AAA gaming and Hollywood blockbusters alike. That’ll be even more significant as Lytro continues to develop the technology — they’re currently working on integrating CGI into real world scenes, speeding up the rendering of 3D models, and making it possible to refocus VR video according to where you shift your gaze.