What the Movie Avatar Can Teach Augmented Reality

The biggest news about the movie Avatar has been the 3D experience and the way its blown the doors off the previous records. The movie has garnered huge success because it pushed the boundaries of technology and told an interesting story.

I loved the movie and the way 3D helped give more perspective to the enviroment. My own Star Trek loving mother didn’t even realize the Na’vi were CGI. She thought they were people in blue suits (really… I’m not joking.) And though storytelling will become important to later advanced augmented reality applications, it’s not what I wanted to point out.

James Cameron is part art-dude and part tech-geek. He waited for years for the technology to ripen enough to do the movie the way he wanted. One of the innovations that he created for the movie was the Fusion camera for the live-action sequences. Normally, scenes are filmed before a green screen and then the CGI is added afterwards. The actors play a game of make-believe and the director has to guess at how the enviroment will unfold around them. CGI movies tend view flatly because the emotions are added later by the special effects guys and not the actors on the scene. Cameron has changed all that.

The Fusion camera system is an augmented reality viewport into the CGI world. When Cameron was filming the actors, he was able to direct them and see the results. When he looks through his camera, he can see them interacting with the world Pandora as the nine foot Na’vi and help them tell the story. The camera itself wasn’t even a real camera in the sense that it filmed the action. The camera allowed Cameron to see the action being recorded by multiple sensors and cameras.  Once the action was recorded, he could go back and reshoot the action from a different perspective, even with the actors gone.

Facial expression was another hurdle they had to jump to make the movie work. So they added little cameras hanging on people’s heads to capture their range of facial expressions and then tweaked algorithms to get them to react correctly.  Even now we can pull off this trick.

Together these systems are similar to an immersive augmented reality world. While we don’t have the HMDs, complete camera access and processing power to pull off the world of Pandora now, time and continued improvement will make lesser versions possible.

If you look at the Fusion camera system, the camera is essentially the HMD display, albeit a large and bulky one. Multiple cameras, RFIDs and tracking markers help the computer understand the world, and complex and powerful computers put all the pieces together. I can only imagine that this system could be turned into a mind-blowing game in an empty warehouse with the proper HMDs.

Essentially, the movie Avatar teaches us that augmented reality has sky-high practical possibilities. All the components of his Fusion system can be ported to the commercial world (not now, but in three or four years) and used to make complex and believable environments overlaid our own world.

In the future, you too can be a nine-foot tall blue Na’vi and you won’t even have to have your soul sucked through a fiber-optic tree.

Insights Into Augmented Reality from Total Immersion

Total Immersion leads the augmented reality industry in total projects (around 125 last year and they’re expecting over 250 in 2010.)  They’ve successfully created world-wide campaigns like Coke Zero and the Avatar i-Tag game line.  So when they talk about augmented reality, I want to make sure I’m taking notes.  Iriny Kuznetsova from 2Nova interviewed Nicolas Bapst about the company and their current activities.  The interview was short, but had a few interesting insights.

Total Immersion has done work for the military in creating augmented reality solutions that put simulated objects on the battlefield.  This is a much cheaper alternative to war-gaming with real equipment.  Hopefully this encourages the military to fund more see-through AR HMDs. 

Total Immersion expects that AR mobile marketing will be the new trend in the coming year and shows off a brief demonstration.  They’re converting their PC software to mobile to take advantage of the smartphone growth.  I found Nicolas’ observation about how augmented reality marketing applications give you direct access to your customers interesting.  By moving people from static newspapers to the computer (and especially the smartphone), then they can find out exactly who is interested in their product and then leverage social media to spread the word.  Nicolas explains they doubled time on websites by adding augmented reality content.  I’m curious if this increase will sustain as the novelty of augmented reality wears off. 

Nothing game breaking here, but worth a few minutes if you’re not familiar with the company.