The Augmented Reality Ballet

The Ballet Font Project combines ballet with augmented reality.  I assume they’ll have big screens showing the combined real time dance with the foot drawn fonts.  The project is a rare cross breed between geekdom and high-end culture.

Used watch batteries and infrared LEDs to create a 2D motion tracking system. We taped these little devices to ballet dancers and had then perform moves which formed letters, which will be used in a headline font called “Ligne”.

This video shows a few letters being performed where we’ve used to the tracking data to overlay FX. The stroke width is controlled by the speed of movement. We’ll be doing a live demonstration at the Armory in Portland, Oregon on Thursday, July 1, 2010. Participants will be able to use the tracking devices to do real time augmented reality.

This project is the brainchild of Weiden + Kennedy’s WK12. Oregon Ballet Theatre contributed the choreography and dancing talent. I developed three versions of software called “Chireo” (chirography + choreography). “capture” did the initial motion and video capture. “augment” let us clean up the data and export the font and rendered videos. “live” is the real time motion tracking / augmented reality software we’ll be using at the demo. Todd Greco here at Fashionbuddha helped on the visual FX in the “augment” and “live” versions.

Robots Need Augmented Reality Too

The object recognition portion of augmented reality is a little like that hand-held label printer that you got when you were a kid and then went crazy putting tags on everything in your room.  Did you really need to put a tag on your table that said, “Table”?  Nah.  But it felt good doing it.

High-end object recognition (and I’m including facial) is really a key component to ubiquitous AR.  Well, and those pesky glasses, but we won’t talk about them today.

So back to object recognition.  For our computers to understand the world enough to create seamless reality interfaces, they’re going to have to understand what a chair is, where it is when they see it and what it’s used for.  This understanding will be useful for us humans, but it will be even more useful for robotics in the future.

With easy access to information, labeled in a computer friendly way, robots can learn to use our environment better than before.  And I’m not even talking about high-end robotics either.  A couple of cameras on a Roomba could help it know when to vacuum the floor and when to stay put because a party is going on.  We use unattended vehicles to transfer parts around our Toyota plants.  Allowing these simple vehicles to know when a box has been left in the way and to quietly move around would make them work better.

And who knows, maybe in the far-flung future when Turing level robots become possible, they’ll educate themselves on the wider world by taking long journeys and absorbing the trash-tags left by their human overlords.

And for fun, here’s a picture and video of a robot.

Kickball AR App

I guess I feel compelled to post this app because of the World Cup.  It’s a cute idea worth a few minutes of play time, but after that, I’m not sure what the appeal would be.  Personally, I’m ready for the next phase of smartphone AR which requires more awareness of objects and the world. Floating augmented objects just isn’t tickling my fancy anymore.

Digital Bullets: Augmented Reality – Financially Viable Or A Gimmicky Technology Driven Fad?

Lavolta asked a number of CEOs the above question: Augmented Reality – Financially viable or gimmicky technology driven fad?  If you’re a long time reader of Games Alfresco, I don’t suppose I’ll need to expound on my views.  But I would like to hear what your thoughts are about the comments made by the various CEOs (about fifteen.)

TV Trickery with Augmented Reality

We’re a long way from fooling the man on the street, but augmented reality may soon play trickery on your TV screens.  This video montage combining scenes from movies like The Running Man and Wag the Dog; and bits of real-time AR from the last few years makes for a convincing argument about the future of this new medium.

I’m partial to such obscuring of reality as it speaks to my science fiction interests.  Whenever I see a video like this is makes me think of one of my favorite authors, Philip K. Dick.  And while this argument is probably twenty years too soon, the ubiquitous use of AR may eventually enchant the populous with its devious wares.  Though you may scoff at such influences, think long and hard about how current technologies and techniques subject the masses to keep the absence of reason as their masters.

But like I said, that argument is twenty-years too soon, or twenty-years too late, if you consider the wealth of propaganda techniques which by and large are more insidious in their use.  Of course, the real danger is not that sophisticated techniques will be developed to hide or alter the truth, its that they will know everything about you so that they can tailor their obfuscation to maximum effect.  But I digress.

Augmented Reality’s Console Upgrade

I’ve been watching the news from E3 with keen interest.  First off, I’m a gamer, if you can’t tell.  But I’m also wildly curious how augmented reality will shape the future of gaming.  Now I haven’t seen any games that strictly use AR, but the direction consoles are taking shows how AR is shaping them.

Microsoft’s Kinect

The formally named Project Natal previews have shown how the dual camera system would allow for gesture based gaming.  The concept doesn’t change with the new name, but we have more information about it.  One of the cooler parts of the system doesn’t even deal with gaming.  Gestures and words will now access the interface like Minority Report.

While we’re a long way from invisible AR systems that travel with us and connect us to the world in new and unique ways, the Kinect system certain raises the bar on how machines can see the world.  The AR smartphone gives us a tiny magic window into the data driven world, while the AR console brings the magic to our living room.

The announced games for Kinect are pretty limited to a racing game, an adventure game involving white water rafting, sports, and dancing.  The only brand name was a Star Wars game that seems like a perfect fit for Kinect and a virtual pet game Kinectimals that gives us the only true augmented reality experience like the PS3 EyePet. The system is due out in November, just in time for the holidays.


Playstation Move and EyeToy

PlayStation is rightly packaging the Move controller along with the EyeToy (for promotional as much as technical reasons.)  While it doesn’t quite compare to the gesture based Kinect, it is a step forward.  However, if I were buying a system for its AR inclusion, the Kinect wins hands down (pun intended.)

First Image Processing iPhone AR App in AppStore

Now I can’t guarantee that it’s the first, but it’s a start.  The Faces app comes from Pixelshed and is available in the AppStore right now.

Make fun of your friends by putting new faces on their heads and share these images via email and Facebook.

Point your iPhone towards your friend’s face and a magical overlays will appear on the screen. Change the overlay by swiping left and right on the screen.

Faces is the very first app that takes advantage of cutting edge Augmented Reality (AR) technology on your iPhone.

Five Reasons Why the iPhone 4 Was Made for Augmented Reality

The last year we’ve had quite the love/hate relationship with Steve Jobs and his beloved iPhone.  Like a pimple-faced love struck boy with a handful of daisies in our hand, we felt the fools when the OS3.1 stood us up at the local Burger Barn.  I mean, we weren’t asking to go steady, just a date and maybe a kiss on the cheek in the form of video access API.  Afterwards, we crossed out Steve Job’s picture in our yearbook and shot spit wads at him in hallway when we got the chance.

Now it looks like the new iPhone 4 and iOS4, will make us love sick again.  Though this time we’re a little older and not as pimple laded as last year.  Augmented reality is growing up and we have more to offer, and in return, so does the new iPhone and OS as Steve Jobs announced the details on it yesterday.

1) Video Access API

Clearly this is the number one based on the long and fruitless flirtation during 2009.  With this change, the iPhone can finally become a real AR smartphone using video feed instead of lame picture frame workarounds.

2) Dual facing cameras

We thought the iPad might have it but we picked the wrong Apple product.  Dual facing cameras can turn the iPhone into the same weird tool that the webcam is, replacing our faces with cows, Iron Man, or Transformers.  I hope developers can learn to be more creative than that.

3) Apple A4 Processor

Playing with video requires more power.  This new chip will raise the ceiling on potential applications.  ‘Nuff said.

4) Gyroscope

This one caught me by surprise, and a pleasant one at that.  With the gyroscope, the iPhone can now understand the world without having to actual see anything.  With the accelerometer it can essentially sense gravity, giving the iPhone an easy way to tell where the floor is.  This should greatly improve those shooter games that left zombies and fires floating in free space.  If I were a developer, I would be quite excited about this addition.

5) 5 Megapixel HD Camera

While this isn’t better than the Droid eight-megapixel and only matches the Nexus One five-megapixel, it improves from the previous number of three.  While this isn’t a huge upgrade, it does help our vision system see what’s going on.

A few months ago, I had pretty much decided not to get an iPhone when my wife’s contract was up, but with these changes I think the iPhone is back in the hunt.  As new demos of AR apps start hitting YouTube, I think we’ll all get a better sense of how good the new iPhone is going to be for augmented reality.  I for one, hope we grow to have a wonderful relationship with full benefits, if you know what I mean.

Paint the Town Red with iRiS

The iRiS (Intuitive/iPhone Remote Interaction System) allows you to paint a multi-media facade on your favorite building.  Assuming it’s hooked up with a psychedelic projector.   But hey, we’re talking art here, not convenience.

The gamer in me wants it to be a game of Tetris on that building, but I’m sure that’s the most obvious way to utilize it.

The beauty of this is that its projected luminance can be seen by anyone, regardless if they have a smartphone or not.  But those with one get a remote controlled Lite-Bright building to play upon, making the others envious.  It’s easy to imagine this kind of art in an AR space and modifiable by anyone, though not everyone has an artist’s eye.  To each his own.

[Via Interactive Media Blog]

Three Reasons Why 3D TV and Movies Will Help Augmented Reality

While we augmented reality aficionados would like to believe that AR has hit its stride, the nascent technology is no where near the level of 3D movies and TVs.  This all may change in the future, but for now the 3D movement far outweighs AR.

This isn’t so bad as I believe, and will try to explain, how the change to 3D TVs and movies will help augmented reality:

1) Augmented reality is just 3D unhinged from a screen

The first and most obvious reason is that augmented reality by its nature exists in a three dimensional space (though in its current iteration we often see 2D sprites hovering in the air.)  So products like 3D movies, TVs, and games will help drive interest in bringing an immersive 3D experience like augmented reality to consumers, as opposed to the 3D view within a flat screen that current 3D offers.  Why be stuck with a screen when you can enhance the whole space around you?

Image from Skooal on Flicker – http://www.flickr.com/photos/skooal/322346446/

2) Why not augmented plays?

I’m having a hard time imagining what an augmented reality movie would look like.  It seems extraneous to add that space in the theater to the story telling medium because that space is a part of your life rather than the story in the screen.

Plays on the other hand could benefit greatly from augmented reality.  What 3D is doing for the movies, I could see AR doing for plays.  Theater typically breaks the fourth wall during its performances, letting the audience in the secret or involving them, even if its just through thunderous applause.

So what if every seat had AR glasses (they could be a little bulky for a two hour experience right?) and the players interacted with this 3D immersive medium?  The type of material presented in that format could be wildly expanded and new forms of storytelling could emerge.

(Picture from Armida!)

3) 3D Glasses –> AR Glasses

Let’s be real.  Ten years ago, someone with a PDA (Personal Digital Assistant for those that don’t remember the 90s) was considered pretty nerdy.  Wearing one on your belt was the equivalent of the pocket protector.

Flash forward to 2010 and one of the big concerns for AR glasses is the style.  Does anyone remember the early versions of the Blackberry?  I felt like I was wearing a frisbee on my hip or that I had a side-holster with a six shooter in it.  If it’s functional and not too bulky (i.e. – neck ache) then it’ll be a hit.

The 3D glasses we start buying for our TVs and games will help bridge that fashion gap.  Who cares what you look like when you’re used to wearing them at home in front of the TV?  Just convince Lady Gaga to wear a pair of bulky AR glasses as a fashion statement and Vuzix will be trying to make them bigger (which brings me to the thought that Lady Gaga is probably just dying to get freaky with some AR.)