Three Reasons Why 3D TV and Movies Will Help Augmented Reality

While we augmented reality aficionados would like to believe that AR has hit its stride, the nascent technology is no where near the level of 3D movies and TVs.  This all may change in the future, but for now the 3D movement far outweighs AR.

This isn’t so bad as I believe, and will try to explain, how the change to 3D TVs and movies will help augmented reality:

1) Augmented reality is just 3D unhinged from a screen

The first and most obvious reason is that augmented reality by its nature exists in a three dimensional space (though in its current iteration we often see 2D sprites hovering in the air.)  So products like 3D movies, TVs, and games will help drive interest in bringing an immersive 3D experience like augmented reality to consumers, as opposed to the 3D view within a flat screen that current 3D offers.  Why be stuck with a screen when you can enhance the whole space around you?

Image from Skooal on Flicker –

2) Why not augmented plays?

I’m having a hard time imagining what an augmented reality movie would look like.  It seems extraneous to add that space in the theater to the story telling medium because that space is a part of your life rather than the story in the screen.

Plays on the other hand could benefit greatly from augmented reality.  What 3D is doing for the movies, I could see AR doing for plays.  Theater typically breaks the fourth wall during its performances, letting the audience in the secret or involving them, even if its just through thunderous applause.

So what if every seat had AR glasses (they could be a little bulky for a two hour experience right?) and the players interacted with this 3D immersive medium?  The type of material presented in that format could be wildly expanded and new forms of storytelling could emerge.

(Picture from Armida!)

3) 3D Glasses –> AR Glasses

Let’s be real.  Ten years ago, someone with a PDA (Personal Digital Assistant for those that don’t remember the 90s) was considered pretty nerdy.  Wearing one on your belt was the equivalent of the pocket protector.

Flash forward to 2010 and one of the big concerns for AR glasses is the style.  Does anyone remember the early versions of the Blackberry?  I felt like I was wearing a frisbee on my hip or that I had a side-holster with a six shooter in it.  If it’s functional and not too bulky (i.e. – neck ache) then it’ll be a hit.

The 3D glasses we start buying for our TVs and games will help bridge that fashion gap.  Who cares what you look like when you’re used to wearing them at home in front of the TV?  Just convince Lady Gaga to wear a pair of bulky AR glasses as a fashion statement and Vuzix will be trying to make them bigger (which brings me to the thought that Lady Gaga is probably just dying to get freaky with some AR.)

What the Movie Avatar Can Teach Augmented Reality

The biggest news about the movie Avatar has been the 3D experience and the way its blown the doors off the previous records. The movie has garnered huge success because it pushed the boundaries of technology and told an interesting story.

I loved the movie and the way 3D helped give more perspective to the enviroment. My own Star Trek loving mother didn’t even realize the Na’vi were CGI. She thought they were people in blue suits (really… I’m not joking.) And though storytelling will become important to later advanced augmented reality applications, it’s not what I wanted to point out.

James Cameron is part art-dude and part tech-geek. He waited for years for the technology to ripen enough to do the movie the way he wanted. One of the innovations that he created for the movie was the Fusion camera for the live-action sequences. Normally, scenes are filmed before a green screen and then the CGI is added afterwards. The actors play a game of make-believe and the director has to guess at how the enviroment will unfold around them. CGI movies tend view flatly because the emotions are added later by the special effects guys and not the actors on the scene. Cameron has changed all that.

The Fusion camera system is an augmented reality viewport into the CGI world. When Cameron was filming the actors, he was able to direct them and see the results. When he looks through his camera, he can see them interacting with the world Pandora as the nine foot Na’vi and help them tell the story. The camera itself wasn’t even a real camera in the sense that it filmed the action. The camera allowed Cameron to see the action being recorded by multiple sensors and cameras.  Once the action was recorded, he could go back and reshoot the action from a different perspective, even with the actors gone.

Facial expression was another hurdle they had to jump to make the movie work. So they added little cameras hanging on people’s heads to capture their range of facial expressions and then tweaked algorithms to get them to react correctly.  Even now we can pull off this trick.

Together these systems are similar to an immersive augmented reality world. While we don’t have the HMDs, complete camera access and processing power to pull off the world of Pandora now, time and continued improvement will make lesser versions possible.

If you look at the Fusion camera system, the camera is essentially the HMD display, albeit a large and bulky one. Multiple cameras, RFIDs and tracking markers help the computer understand the world, and complex and powerful computers put all the pieces together. I can only imagine that this system could be turned into a mind-blowing game in an empty warehouse with the proper HMDs.

Essentially, the movie Avatar teaches us that augmented reality has sky-high practical possibilities. All the components of his Fusion system can be ported to the commercial world (not now, but in three or four years) and used to make complex and believable environments overlaid our own world.

In the future, you too can be a nine-foot tall blue Na’vi and you won’t even have to have your soul sucked through a fiber-optic tree.