Top 10 Augmented Reality Projects by the Music Industry

Augmented Reality is like music to my ears (couldn’t resist the pun…)

The music industry is known more for stifling technology innovation than adopting it (cough*Napster *cough) which makes it especially gratifying to see recent attempts to harness augmented reality for music lovers (and/or nerds).

Here are my Top 10 augmented reality projects created by the music industry:

10) StudioB, Adobe and John Mayer “AR video clip”

“Studio B teamed with Adobe and John Mayer to create the worlds first Augmented Reality music video. Studio B shot and keyed the green screen footage using the new Apple ProRes 444 codec. I’m holding up a piece of paper with the designated marker (in this case a broken heart), and the video of John Mayer follows it around the screen, and even tilts in 3D space!”

It’s impressive to see a major music figure dedicating his time for an AR app – but is this significantly more interesting than just watching this 3D video on your PC?

9) Video for German Band Saint Aside

Here’s a “me too” music video in Germany for the band Saint Aside:

“Germany’s first Augmented Reality music video of the band Saint Aside. Das erste deutsche Augmented Reality Musikvideo der Band Saint Aside. Powered by AUGMEDIA dem Virtual Reality Marketing Kreativ-Netzwerk aus Deutschland, Leipzig.”

8) Doritos and Rihanna deliver AR experience

Can’t really see the AR interaction in this video, but hearing Rihana say “Augmented Reality” is worth it.

“To view “Who’s That Chick” by Rihanna in full, simply buy a bag of Doritos Late Night and got to
Using special packs of Doritos Late Night chips and a web cam, you can unlock an augmented reality experience with Rihanna.”

Triggering the experience from a consumer package is cool (no more printed markers!)

7) Kit-Kat and Scouting For Girls music

“Scouting For Girls – Augmented Reality with Kit Kat. If you open up a Kit Kat, do you see the augmented reality of Scouting for girls? Do the She’s So Lovely Stars really play in there? Join Scouting For Girls here to find out. Or, visit

Yet another music video triggered from a consumer package (Kit-Kat) – this time it’s more mindful to the new medium.

6) Eminem augmented reality competition by Polydor Records

“To take part all you need is a printer and a webcam – then, using the graffiti tools on the entry page, customize your very own 3D rotating Eminem “E” logo.”

“…enter the amazing Eminem augmented reality competition and you could win the trip of a lifetime to visit Em’s home town of Detroit”

Although not an AR music experience per se (more of a graffiti experience)- it’s definitely encouraging fans to interact with their idol in a new way. Talk about high profile motivation.

5) 10th Dimension create your own music video, TakingDog Studio

(Start watching at minute 2:03)

“Now you can create your own Augmented Reality real time interactive music videos for two of the songs from this project. Go to to try it out for yourself.”

Anything that involves some creation beats “screen watching”.

4) Music mixer with 5Gum

“Truly interactive Augmented Reality Application. Log on to to record your mix.”

Making music with an augmented reality interface is a cause for celebration.

3) Aphex twin overlay on Audience faces

“At this weekend London Electronic Festival (LED) Aphex Twin used live facial recognition to map the audience and overlay images of his own, trademark distorted face. Mesmerising and disturbing in equal measure! “

This is one of these things that had to be done. And they did it – beautiful!

2) Lost Valentinos customizable video clip with AR

“Innovative Sydney band Lost Valentinos and label etcetc have pushed an emerging technology to the very limit by launching the world’s first truly interactive augmented reality music video project. The project allows fans to record their own customised music videos for the single ‘Nightmoves’ by capturing band members performing the track in the fan’s own world. From lead singer Nik performing the song atop a user’s shoulders, to the whole band playing the song at the base of the Eiffel Tower – it’s a project limited only by the imagination.”

Record your own video clip with AR? Can’t get much better than that!

1) Black Eyed Peas 360

The most impressive use of AR in the music industry so far and the reason behind this post:

“App review: The New ‘BEP360‘ App for iPad, iPhone and iPod touch Takes Fans Inside ‘The Time (Dirty Bit)’ with 360-Degree Motion Control and Augmented Reality app Developed by Peas Front Man”

Edo Segal, from Futurity and a good friend – is behind this initiative that attempts to redefine how we experience music.


Didn’t make the list:

  • Voodoo Experience by Zehnder with Layar Augmented Reality browser (meant to enhance the concert experience – not to deliver a new way to experience music which is the focus of this post)
  • Exile on Your Street – an AR app for Rolling Stones fans – built on Layar – it enables you to leave Rolling Stones posters at your current location.
    Each poster can be customized with a track from Exile on Main Street and a personal message (no visual reference found)
  • N-Dubz, Against All Odds record cover triggering AR by Total Immersion (couldn’t find a video)
  • Album by hip-hop act Sinik, “Ballon d’Or,” include images giving access to ‘augmented reality’ features, by Total Immersion (couldn’t find video)
  • Sean Kingston: printing a special icon in the CD booklet for his album, which gave users access to the special AR section of his website. They could then sing along into their webcams to Sean’s Fire Burning track, with an animated version of the star appearing to dance around them. There were sharing elements here, too – the resulting videos could be posted on social networks or shared on the main site. (via Music Ally)
  • Lynx Augmented Reality Groupies on your desk (promoted a deodorant)
  • AR in French Music Video by Toxic Avenger (not really AR)

Ok, that’s cool. But grab any music industry executive that gets it, and you’ll hear that the true value of AR doesn’t lie in helping promote music (the industry is pretty good at marketing music as is), but in creating new experiences, new products which will lead to new revenue streams (think Guitar Hero). The sober realization is that it’s unlikely people will buy more music than they already do – but they are likely to want to experience music in new ways, and through new channels.

What will it look like?

How would you like to experience your favorite band in the real world (AR style)?

Additional references for AR in the Music Industry:

Live from GDC: Augmented Reality at the Game Developers Conference – by the Numbers

Reporting live from The Game Developer Conference in San Francisco, with my Augmented Reality filter on – here are the stats:

0 (Zero)

Number of mentions of Augmented Reality during Sid Meier’s keynote – the game design legend behind “Civilization”
Truth be told –  also missing from the keynote were mobile games, social games, casual games, and virtual goods…


Number of mentions of Augmented Reality in 10 Microtalks by 10 of the most revered visionaries in the game industry
Jesse Schell touched on the impact of games on reality (AKA Gamepocalypse) as an epilogue to his mind blowing Dice Talk


Number of mentions by investors on a panel about how to get funded
Brought up as example of games that will help developers differentiate themselves. The advice: build real life RPGs!


Mentions of Augmented Reality  during Facebook’s Social Games Summit Keynote
Gareth Davis touted augmented reality twice as the next generation of mobile social games.


Number of press articles about augmented reality during the GDC week


Number of GDC sessions focused on Augmented Reality
(on a personal rant – my submission for a talk titled “Put a Spell: Post Mortem of the first augmented reality learning game for the iPhone” was shamefully rejected!)


Number of Augmented Reality game demos on the exhibition floor
  • Sony EyePet and Move
  • Vuzix presented a game by Ohan Oda – Columbia University (video below shows a similar game)
  • AR Drone CES sensation by Parrot and Int13;
  • Nestle Cereal box as controller by 3DVia-Dassault Systemes;
(Metaio skipped the show this year and is betting on SxSW with ScavengAR)


Number of AR capable devices showcased at the event
iPhone, Android, DSi, Sony EyeToy. Plus Windows Phone 7, and Xbox Natal – promised to be released before the end of 2010 (NVidia mobile AR demo was MIA – missing In Action)


The least number of back-room meetings focusing on augmented reality which took place at GDC
(or in other words – meetings I was part of…)


Percentage of game developers familiar with the concept of augmented reality
(based on my anecdotal survey)

∞ (infinity)

Amount of inspiration at the event for designing augmented reality games


So how does GDC 2010 compare with last year’s Tiny Spark of Augmented Reality?
In a nut shell: Augmented Reality made progress in mind share – but not yet in real impact on the game industry.
Wanted: Game Designers to build Augmented Reality Games!

ISMAR 2009: Sketch and Shape Recognition Preview From Ben Gurion University

ISMAR 2009 the world’s best augmented reality event starts in 3 days!

If you are still contemplating whether to go – check out what you might be missing on our preview post.

The folks from the Visual Media Lab at Ben Gurion University in collaboration with HIT Lab NZ are preparing a real treat for ISMAR 2009 participants.

Sketch recognition (already covered in our previous post) is a major break away from “ugly” markers or NFT  (tracking natural 2d images). It is the dawn of user generated content for Augmented Reality, and an intuitive new interaction approach for changing the CONTENT overlaid on a marker. Big wow.

In-Place 3D Sketching

But the team lead by Nate Hagbi and Oriel Bergig (with support from Jihad El-Sana and Mark Billinghurst) is just warming up…In the next video Nate shows how any sketch you draw on a paper (or even on your hand!) can be tracked.

So are you telling me I won’t need to print stuff every time I want to play with augmented reality?
-That’s right! Hug a tree and save some ink!

Shape Recognition and Pose Estimation

But wait, there is more!

Nate says this demo already runs on an iPhone.

And to prove it, he is willing to share the code used to access the live video on iPhone 3.0.
(note: this code accesses a private API on the iPhone SDK)

Ready for the BIG NEWS?

For the first time ever, the core code necessary for real augmented reality “(real” here means precise alignment of graphics overlaid on real life objects) on iPhone 3.0 is available to the public.

To get access to the source code – send us an email.

May a thousand augmented reality apps bloom!

ISMAR 2009: Tracking a City Model – Preview From Graz University

Only 1 week to go for ISMAR 2009, the world’s best Augmented Reality (AR) event.

Here is one more reason to go to the event.

This stunning “Jakomini” demo from Graz University – the masters of handheld Augmented Reality – shows a 3D city model being tracked on a “natural feature” surface (or in plain language – a regular bird’s view image of a city)


What handheld was used for this demo?

(My guess is it’s Nvidia’s Tegra)

What’s behind the mysterious Jakomini name?

(Jakomini is the 6th District of Graz and the most populous)

What’s hidden in Jakomini?

(I guess we’ll all find out during ISMAR…)

Need more reasons to come to ISMAR?

Check out these previews.

Reblog this post [with Zemanta]

Augmented Reality Cookies Are Sweeter

Two things that I like about the Augmented Reality community are the spirit of creativity and the spirit of sharing.

From the Tellart website

This letter from Mike Clare embodies these 2 traits:

This summer I worked on a number AR projects while interning at Tellart in Providence RI.  On my last day I thought I would bring in something to show my appreciation to my co-workers for having me as their intern.  What better way than batch of home made AR marker cookies that would display Tellart’s logo?

Vodpod videos no longer available.

In addition, Mike has prepared a tutorial so that others could try making them at home.

That’s creativity and sharing in the power of 2 (cookies and recipe).

What a cool guy!

But Mike isn’t the first…check out our 2008 Top 5 Holiday AR greeting cards

Red Bull Gives You Augmented Reality Wings and Saves Magazines with Print 2.0

They gave you wings, extreme sports races, and Flugtag – and now they want to save print magazines with the bold concept: Print 2.0.


Guess which one of these is a true fact?
1) In 2006, more than 3 billion cans of Red Bull were sold in over 130 countries

2) Red Bull publishes a printed magazine (2 million copies per issue)

3) Red Bull is an Austrian company

Answer: all of the above!
Can’t say which fact is more shocking, but they certainly explain why Red Bull decided to partner with Imagination (an Austrian company) to create a webcam augmented reality experience where:

This magazine sings, dances, flies and even scores a touchdown…

The cover and multiple pages (any page with the Bull’s eye) can be activated by pointing to a webcam thanks to Imagination’s natural feature tracking software.

Try it yourself.

If you don’t have the printed magazine – don’t worry – you can download and print at home.

Or just watch it here…

The magazine editor dives into more autophilia:


It’s not often that a magazine can call itself revolutionary,  but we’re delighted to say this one can.

This very issue of The Red Bulletin takes us from print  to Print 2.0, thanks to the incorporation of some nifty  software known as ‘augmented reality’…the  fun stuff is this: simply by holding the mag up to a computer you can take it ‘beyond the page’ and into the world-wide web. So, for example…[the cover] will link through to a video package explaining exactly how augmented reality can enhance your reading experience in a way you almost certainly never imagined, with music, film, animations and more.
Then turn to page 5 and Red Bull Air Race ace  Paul Bonhomme will give you an ‘as live’ introduction to  the magazine and the world of augmented reality. Head to our Now and Next pages, find the story about Black Gold  on page 20 and do the same again with the mag. Lo, you’ll
find the band’s latest video on the website. Clever, eh?
Further in, you can read about Burcu Cetinkaya and Cicek Güney – the girls putting the glam into rallying – then link to exclusive interviews with them and videos of them driving flat-out… and crashing!
And we’re not done yet, no way. Our Reggie Bush cover story, on page 48, combines with an exclusive mini-movie  of Reggie at home, as he talks to correspondent Jan Cremer, while page 62 will take you right into the pocket-rocket  world of the Red Bull Rookie motorbike racers.
No other magazine has ever tried anything like this,  and we have plenty more ideas for the future. But for now,  just get your magazine and computer primed and prepare  to be amazed…

…And Daniel Wagner certainly shows his skills in the video above…

ISMAR 2009: Sneak Preview of Demos at the World’s Best Augmented reality Event

ISMAR 2009, the world’s best augmented reality (AR) event is just 19 days away.

Check out these 8 reasons to attend ISMAR 2009.

If you are into AR – you got to register!

If not – check out some of these upcoming demos – and you’ll become an AR fan instantly.

Here is a sneak preview of some of the demos planned for the event:
Augmented Earth (GA Tech)

6DOF Object Detection (Graz university)

Room design (unknown)

CD and book recognition system (Nokia research Center)


Put a Spell – Learn to Spell with Augmented Reality (presented by Ogmento, developed by Arballoon)

PTAM on iPhone (Georg Klein – Oxford University)

Multiple object recognition – PTAM extension (Robert Castle)

Will Metaio show their Augmented Cards? (Metaio)

Goblin XNA: Infrastructure for Augmented Reality (Columbia University)

Image Space – Social Media Sharing (Nokia)

YVision – AR framework  (YDreams)

Vodpod videos no longer available.

Embedded AR (HitLAB NZ)

Android AR (HitLAB NZ)

Tracking a City Model (Graz University)

Jakomini (Graz University)

Shape Recognition and Pose Estimation (HIT Lab NZ and VML)

In-Place 3D Sketching (HIT Lab NZ and VML)

What will you show at ISMAR?

GDC 2009: First Augmented Reality Demo at a Game Developer Conference

As soon as the exhibition floor opened today at the Game Developer Conference (GDC) in San Francisco, I rushed to visit the Vuzix-Metaio booth to witness the first consumer-oriented augmented reality booth ever to exhibit at GDC.

I was not disappointed. Just watch this clip.

Stefan Misslinger (Metaio) did a fantastic job  acting it out, don’t you think?

Vuzix also had on the display the upcoming-anticipated-long-awaited-soon-to-be-released 920AV AR glasses. It wasn’t in working condition, but Vuzix promised it will be released this fall.

When the event kicked off, I offered to serve as your eyes and ears; Peter Milford took the offer and expressed his curiosity about the latest from Vuzix. This one is for you Peter: yours truly posing with the goods.


Now the key question is:

how would you rank the look of these specs on a dork-to-cool scale?

Augmented Reality Game in Primetime TV Show

To complete the trilogy covering the Japanese foray into Augmented Reality, here’s a revival of a TV show featuring ARis the Augmented Reality Geisha.

The game creator is a super star; the new scenarios are staggering; give it up to the unwavering, hanky-panky, poke-her doll – ARis the augmented!

The first adult AR game is garnering excitement in Japanese pop culture.

Is ARis really at the center of a this TV game show?

What’s the big showdown about?

Who won this “boss” battle?

Japanese Augmented Reality Punch Continues

via augmented times, via Sein Blog, thanks to google translator…here are the AR news from Japan.

Scene Recognition Engine (SR Engine) demonstrates its ability to recognize buildings, pull relevant information, and display it live – on an iPhone. And it works outdoors – alfresco.

Looks like a marriage between Tonchidot’s Sekai Camera and Mobilizy’s Wikitude.

Interested in a look under the hood?

How does it work?
(loose translation from the creator’s speech in Japanese):

Stores in the city are stored in a database and mapped and tagged with information from brochures and videos. Based on the “scene” caught by the Camera, the app pulls information from its database and tags the actual image, the AR-free sensor networks. It uses GPS, angle variation and image matching, to recognize the targets.

So, I guess it only works when you observe the target from a certain perspective.

Why develop it for the iPhone?

IPhone’s CPU is [not] bad, folks like it…

Who’s behind it?

According to the blog it’s the man with the big name:

Sunday Star Geumchon

have you seen this man?

Who are you Sunday Star?