Live from ISMAR 2009: Starts Today with Innovation Workshops – Mobile Augmented Reality

ISMAR 2009 is starting Today. Woo-hoo!
Looks like a true conference. Beyond the strong presence of researchers – many start ups, game developers, artists, press. The augmented reality industry is forming in front of our eyes.

Multiple interesting innovation workshops are on the agenda for today. I am joining the Mobile Magic Wand workshop.

Christine Perey kicks off the workshop with a dry definition of what Mobile AR: is a catch all Mobile AR apps, services, hardware, content  for consumers, professionals, prosumers, both indoor and outdoor.
What does a capable mobile device need to pack in order to play in the AR world?CPU 600 mhz, camera, GPS, compass, and some more goodies depending on the target application…

Today’s mobile AR users: more than a thousand and less than 1 million (90%+ on Android and iPhones)
These are early adopters, avid users of social networks, mostly located in western Europe, north America and south east Asia.

Who’s in the audience? Mobile network operators and mostly content and application developers. Dirk from Layar confirms they have shared 1000 developers keys. This is starting to look like a solid community.

Perey presents  usage scenarios that will be tackled by 4 of the AR companies on the workshop (Layar, Mobilizy, Across Air, Metaio):

1) Entertaining in teen town

2) College fresh on campus

3) Assisting a tourist

Peter Meier CTO of Metaio is first on stage (self described Dinozaurs in the AR space) starting with an introduction to Junaio: take pictures of locations and post in 3D in a physical location that are stored on a Metaio server – so that both mobile and home users could enjoy.

He bravely demos it live, posting a 3D object at the end of the room. Peter: “It’s always a hard one…we rely on a bad GPS signal…”

Peter then switches phones and starts playing a zombie game, slashing zombies hidden among the audience “I hope you are all over 18 because there is some explicit content here”…

Anyone can create Junaio enabled games using web APIs. It’s for programmers and non-programmers, it will includs marker tracking and markerless tracking.

Peter wraps up with a display of his excitement about the public awareness to AR: ” I have been explaining AR for so long now, that I am so happy with recent developments in the space.”

Markus Tripp from Mobilizy, joined the founder Philipp in the very early days of this start up (currently 10 employees).

He starts addressing the first use case: coaching a college freshman, by touting Mobilizy’s proposed markup language – ARML. It makes it easy to customize the content for the aforementioned freshman (writing XML). He describes the developer API and the architecture behind the scenes that enables the content development and user experience (including search, bookmarks, filters , etc.)

The SDK is free for developers (developers have to pay when they publish an app based on Wikitude).

Next is David Murphy from Nokia. David starts with the news: “There is more to life than Android and iPhone…”

Nokia has been in the space for a while and the underlying OS Symbian is being open sourced.
N97 is a great device for augmented reality scenarios. But there are many moe cheaper phones that can do AR.

David dives into a technical recipe of Nokia’s elaborate capabilities to address the scenarios covering: Camera. display, sensors and network.  Important considerarions include location acquisition API (compensating for inaccurate GPS signal with existing POIs), user interface (has to be flexible), networking (multiple standard protocols), and even translation to other languages support. On screen augmentation: multiple options are available such as OpenGL (though Nokia phones have no 3D acceleration in hardware.)

David wraps up with a caution: “be realistic with your applications – accuracy will be iffy…so aim to annotate building size objects. And don’t forget AR services are very power hungry – the battery is finite afterall…;)”

Last on the roundup is Chetan Damani from AcrossAir.

Established 2008, a free AR browser (planned to be released in December.) not as advanced as some of the other players so far.
Across Air browser will be completely 3D with angles and dimension. Keeping a simple user interface.

The use case: learning entertainment app. The proposal is about Dinosaurs AR with 3D animation.
AccrossAir has put together a massive back-end infrastructure that could be used for this scenario. The tool for developing the content seem well defined. AcrossAir is always there to give a hand…

Shows a video of 3D dinosaurs walking around you in a park!
(result of about 8 hours of work)

Chetan wraps up with success factors to consider for the app such as keep the UI simple, take advantage of the features, keep data dynamic (web access), and make sure the location is valuable to the app – and don’t forget to promote your feed and notify users. How much can you make out of it: you could charge $1.99 to ativate feed and rev share for advertising network, affiliate programs.

Augmented Reality in 2012

After a quick break “fluid adjustments” Jay Wright, responsible for business development at Qualcomm, comes on stage to talk about the vision of Augmented Reality for 2012. Exciting.

Jay is responsible for the commercialization of AR at Qualcomm. That’s music to my ears!

He starts with a statement: Wireless will bridge digital and the physical worlds. What deos AR have to do with it? it’s a new UI paradigm. The links get richer and richer the smarter the devices are (make sense coming from a mobile phone chip manufacturer.)

Jay sees the future of mobile AR as moving from a compass based (Wikitude, Layar) to vision-based approaches.

Some of the existing barriers for mobile AR such as limited computational power and power consumption will have to be addressed by phone manufacturers.: more MIPS (like with SnapDragon that offers 1GHz!), programmable GPUs, multimedia accelerations, and integrated chipsets (helps optimize power consumptions) , new classes of sensors with dedicated processors, new displays with higher resolution that work better in sunlight (OLED). Jay even hints at including advance gesture recognition into hardware.

Head-mounted displays – we are all waiting for see-thru displays which are aparently very difficult to achieve with today’s technology, and we need to think about new interactive and intuitive UI paradygms.

Location services accuracy will improve by fusing multiple location engines.

Batter life – AR is a very hungry for battery power, which are only advancing 8% a year. Manufacturers are looking at new simpler ways to charge devices (by dropping the wires).

Cameras – we’ll need wider field of view,fast and efficient programmatic retrieval of image frames, moving into HD etc.

Additional considerations for the future: operating systems will introduce AR specific capabilities, ubquitious WWAN broadband, smartphone peripherals, peer to peer capabilities (e.g. for games), and location aids.

Standards bodies: OGC, AR Commons (in Japan), Khronos for rendering standards (OpenGL, WebGL, OpenCL, Collada) and for image signatures: ISO (JPEG. JBIG)

Jay wraps up with a quick look out of AR platforms: they will consist (like in many instances in the past) of player and content, supported by authoring tools.

Christine touches on domains that will become infused with AR capabilities such as navigation, education, games, commerce, social media…

How will the total industry revenue in 2012 will be broken up?  AR developers for technologies and tools, corporations spending on products and services, and eventually end users will pay for premium apps and services.

Now a switch to Social Augmented Reality (AR 2.0) presented by Tobias Hollerer (along with Mark Billinghurst and Dieter Schmalstieg)

Content for AR browsers will include public sources, premium content authored by agencies, and crowd sourcing. To get a sense of the potential – in the last few weeks Tobias tracked about 3000-5000 twits a day about Augmented Reality.

Business models that could work for AR 2.0 (similar to web 2.0): ads, affiliate, pay per view for premium content.

Ways to publish content: stand alone apps (not scalable), AR browsers (is now very popular), AR widgets (more modular).

Dieter picks up the discussion and dives into user created content for social augmented reality (building on the success of  web 2.0): bottom up process of rating and tagging information by the crowds.

AR infrastructure will consist of big content providers and personal content providers.

Authoring : today is mostly happening on the desktop. In situ authoring will become more prevalent and efficient at capturing real time, location-specific information.

Dieter ends with a tongue in cheek: Google cars will not go into your bed room (to create street view like images) – so there will be a need for individuals to capture content indoors.

Mark Billinghurst comes on stage at lunch time and proposes a social experiment…”if you are hungry – go eat. If not – stay with me and listen up…”
Nobody leaves…is nobody hungry or is Mark that good?

How is the AR experience affected with AR 2.0?

AR 2.0 stages will follow the stages in social media:
1. Establishing online profiles
2. Functionality – platform for social interactivity
3. Colonization – single identity
4. Context
5. Social commerce

How do you deal with thousands of twits in your area?
The answer is of course through information filtering.

Designing for individuals is very different than designing for crowds.
Mentioning an example of Carlo Ratti from MIT – which tracks location of people over a city scale.

Mark wraps up with a bunch of remaining questions that the industry will have to tackle…

Now, off to lunch.

Stay tuned for exciting breakout sessions after lunch.

Weekly Linkfest

It’s time again for the weekly linkfest. Let’s see what else happened this week in the world of augmented reality:

In the realm of mobile browsers:

  • SREngine has a new video out showing its slick user interface.
  • Hong Kong’s Leisure Guide – yet another AR browser (YAARB™)
  • Metaio announces the coming release of Junaio, but doesn’t give much information about what Junaio really is.

And in other news:

  • Total Immersion now enters the world of soccer trading cards, at least in Italy.
  • You can get pregnant without the morning sickness via the wonders of Flash based AR.
  • The Virtec project is a thesis project using AR to teach about Aztec culture by means of a book and a head mounted display.

This week’s video revisits one of this blog’s reoccurring themes, the Transformers (previously here and here). This application comes with the Transformers coming DVD, or as this trailer puts it “Prepare for Augmented Reality! An exclusive 3D Holographic Experience!”:

Have a nice week!

Metaio Announcing Mobile Augmented Reality Platform – Junaio

junaio_screen

Metaio, Inc. Will Launch Its Mobile Augmented Reality Platform junaio On November 2nd San Francisco, September 18th 2009 – Today, metaio officially announced the launch date of junaio, its mobile augmented reality platform. On November 2nd, the leading company in augmented reality will release a first version exclusively for the iPhone. More features will be released soon after, including capabilities on the Android and Symbian platforms. junaio will change the way we create, access and share information. By combining innovative online and mobile technologies, junaio will allow users to see location-based content through the display of a mobile device. Users can leave traces, messages or objects and visually interact with their friends or anyone else in the world. Already existing web services can be enhanced and completely new ways of interaction can be created. Whether it is social networks, multimedia content or game concepts – virtually anything can be embedded in the real world and connected to a certain place. “The possibilities are endless, we are taking the Internet outside to the real world,” says Thomas Alt, Chief Executive Officer of metaio. Seeing location-based multimedia content through the display of your mobile device is only one part of the story. “Mobile augmented reality is all about the user´s orientation. But to deliver a really useful and robust application, you have to be user oriented,” says Peter Meier, Chief Technology Officer. metaio is defining a new dimension in mobile augmented reality through incorporation of features that will allow better usability and social interaction. junaio is the result of more than six years application development in augmented reality and months of research and usability tests for mobile applications. So get ready for the ultimate Outernet experience!

For more information and updates, please refer to: www.junaio.com

Weekly Linkfest

Hope you didn’t miss the weekly linkfest’s early edition, published yesterday, covering some of the best articles, posts and talks that were published during the week. Here are some more interesting bits from around the AR ecosystems making news this week:

And finally, this week video comes from Hongik University of South Korea. It shows a project named “Will be”, created in 2004 (and presented in ISMAR05), which is the augmented reality take on a story board. It’s quite nice, though some of the features could have been more accessible if they were implemented via standard GUI, rather than ARUI:

Have a nice week!

Weekly Linkfest

Last week I published a poll, asking how do you define yourself – are you an engineer, an artist or maybe an Entrepreneur? As of writing this post, 75 readers have answered the poll, 34 of them (45%) identified themselves as engineers. I thought there would be more artists among you (15%), and was surprised by the percent of entrepreneurs (15%). The poll is still open, so you can still cast a vote.
Moving on to the weekly linkfest (it gets bigger every week!) –

  • Metaio blitzed the airwaves with two podcasts – Noora Guldemond (head of sales and marketing) interviews here and Peter Meier, Metaio’s CTO is giving an interview here. Sadly, I haven’t found the time last week to hear them, but I plan to do so in the next few days.
  • And it was a good week for SPRXMobile (Layar) as well. Aparently, Layar comes preinstalled on Samsung’s new Android phone, they were featured on The Financial Times, and things are only going to get better, since they are holding their first Layar event.
  • TweetWorld is Gamaray’s attempt to have an augmented tweeter application, joinning the ranks of Layar and TwittAround.
  • The BBC – Mobile phones get cyborg vision: “Not only could this form of rich, intuitive and easy to grasp data be the next killer app for the mobile, some see it changing our world view forever.
  • ReadWriteWeb – Augmented Reality: A Human Interface for Ambient Intelligence: “Augmented reality (or AR) is fast becoming as ubiquitous a term as Web 2.0. The field is getting noisier by the day, and AR as a field of research now has to co-exist with its status as an industry buzzword
  • A short introduction to programming AR applications for the Android OS.
  • Is this the first augmented shirt on Threadless?
  • CrashCorp demos a rudimentry AR application for the iPhone.
  • YDreams and Zugara join the AR consortium (can I join too?)
  • And Zugara (covered previously here) also launched what must be the second augmented reality game on Facebook (since last week Total Immersion had the first), CannonBallz (video). Just four years ago, we would have called this kind of games “Eye Toy” like, but today we have new buzzwords. Still, it is a well produced game.

Our weekly video is of a game created by Circ.us, to promote Chris Angel’s new show, “The five lives of Chris Angel”. Since it’s a puzzle game, and this summer turned me into a brain-dead blogger, I haven’t tried it myself to give an educated review. You, on other hand, can play it here, or just watch the embedded video below:

As always, have a nice week!

Weekly Linkfest

Here are just some of the things that happened this week in the realm of Augmented Reality, which I didn’t have the time to write a whole post about. It’s going to be a long post, you may want to prepare a snack before you go ahead.

This week’s video comes from Metaio, which launched an ambitious initiative named “metaio World“. There haven’t given much information about it, but here’s a short quote –

You can view, create, upload, modify, navigate, share, rate or play games with real 3D content anywhere in the real world. Add your 3D and post interactive elements, your favourite photos, twitter messages or anything you can imagine.

An ambitious project deserves an ambitious video, but can they deliver?

Have a nice week!

Weekly Linkfest

And yet another week ends, full with exciting AR news. Some news items were put aside in order to make place to more urgent reports. Luckily, the weekly linkfest is here to mend things up.

This week’s video is of a little application you might have heard about, called tweet TwittAround.
It enables you to see tweets overlaid on top of the video input coming from your iPhone 3GS. Tweets are rendered according to the location they are coming from. Interestingly, this project comes from the same guy, Michael Zoellner, that is behind some other cool AR applications.

Have a nice week!

Open Letter to Apple: Let us Augment Reality with the iPhone!

A letter sent to Apple Developer Relations.

Dear Apple,

We are a collection of augmented reality (AR) enthusiasts and professionals (from business and academia), who have been working on a multitude of AR apps for the iPhone. These apps are poised to change the way people interact with the real world.

But here is the rub: we are currently unable to publish these apps on the app store because the iPhone SDK lacks public APIs for manipulating live video.

We are asking Apple to provide a public API to access live video in real time, on the iPhone.
We will be happy to offer additional technical details.

The impact of augmented reality (AR) on our lives could be as significant as the introduction of the PC.
In 10 years, we believe augmented reality will change the way everyone experiences travel, design, training, personal productivity, health care, entertainment, games, art, and advertising (videos).

Looking back just a few years, AR pioneers had to hack a slew of components into ridiculously large backpacks and HUDs, and be confined to rigged environments. Nowadays, it comes in friendly, affordable packages and the iPhone is one of the first devices to have it all – except for a public API.

The battle to determine the winning device has already begun; a public API to access live video will give the iPhone a lucrative ticket to compete.
We believe Apple has a window of opportunity of about 3 months before developers start looking elsewhere. If Apple decides to publish the API in that time frame – in the next 10 years, everyone might be using the iPhone as the preferred device to interact with the real world.

Here is how augmented reality could open up new opportunities for the iPhone this year:

Arf (Georgia Tech)

a virtual pet you take anywhere

ARghhhh (Georgia Tech)

first person table-top action game

Sekai Camera (Tonchidot)

AirTag the real world

Kweekies (int13)

a portal to creatures in a parallel world

Layar (SPRXmobile)

Browse the world with an AR browserDetails

Artoolkit for the iPhone (Artoolworks)

the most popular AR kit now on the iPhone

StudierStube ES (Imagination, Graz TU)

the only AR engine designed for mobile devices, now on iPhoneDetails

PTAM on the iPhone (Oxford University)

next generation AR tracking with no markers or images

Wikitude (Mobilizy)

a travel guide that “tells you what you see”

Virtual Santa (Metaio)

interactive Christmas application using the augmented reality

Augmented Reality Sightseeing (Fraunhofer IGD)

Historic photographs overlaid on your field of view while strolling in a street

These are apps that are practically ready to go. There is a whole bunch of apps and games that are just waiting for the API to be available.

…And Apple, we know you can’t share your plans…so please surprise us soon!

Many many thanks for your consideration –
Sincerely,

Signed:
Michael Gervautz – Managing Director Imagination GesmbH
Robert Rice – CEO Neogence
Georg Klein – PhD PTAM creator from Oxford University
Stephane Cocquereaumont –  President & Lead Developer Int13 (Kweekies)
Maarten Lens-FitzGerald – Founder & Partner SPRXmobile, developer of Layar
Ori Inbar – Author of GamesAlfresco.com and CEO and founder – Ogmento (formerly Pookatak Games)
Philippe Breuss – Lead developer, Mobilizy
Philip R. Lamb – CTO, Artoolworks
Noora Guldemond – Metaio
Takahito Iguchi – CEO, Tonchidot
Blair MacIntyre – Associate Professor, Georgia Institute of Technology

Bruno Uzzan – CEO, Total Immersion
Michael Zoellner
Fraunhofer IGD
Andrea Carignano – CEO,  Seac02

If you are developing an AR app for the iPhone and wish to join this effort – just let us know.

AR, no Programming Knowledge Needed

Do you want to play with AR, but too afraid you don’t have the required programming knowledge needed? Now you have two applications to play with from the comfort of your own home.

First is Metaio Unifeye Design. Available freely in an (almost fully featured) demo version, this tool is far from being a toy. You can create elaborate scenarios of marker based and marker-less image based augmented reality without writing a single line of code. You can test your creations with your web-camera or using a prerecorded video clip of a marker. There’s even a tool for creating new markers.

However, such flexibility does come with a price. I had a hard time working with the user interface beyond the basic functions, and I’m quite sure I only scratched the surface of what’s possible with this tool. That’s why I had this application installed for about a month now, but I was hesitant about writing anything about it.
Metaio claims that video tutorials are coming shortly, and I think they are very much needed (and I’m a programmer in my spare time :).
In more somber news, you can’t use your own 3d models, or export your scenarios as a stand alone application. You probably have to pay for those features.

Now, if Metaio Unifeye Design was a bit too complex for you, the next application is truly augmented reality for beginners. It’s called Atomic, and it wraps ARToolKit to give you the basic functionality of augmenting markers with VRML 3d objects when you examine them using your webcam. The application source is available to download as well, so eager programmers may add more features to it in the future. Another advantage of this tool is that it allows you to import and use your own models. However, don’t expect to amaze any AR veteran using it.

Once again:
Unifeye Design
Atomic (Atomic via Rising Wisely)
Happy Augmenting!

GDC 2009: First Augmented Reality Demo at a Game Developer Conference

As soon as the exhibition floor opened today at the Game Developer Conference (GDC) in San Francisco, I rushed to visit the Vuzix-Metaio booth to witness the first consumer-oriented augmented reality booth ever to exhibit at GDC.

I was not disappointed. Just watch this clip.

Stefan Misslinger (Metaio) did a fantastic job  acting it out, don’t you think?

Vuzix also had on the display the upcoming-anticipated-long-awaited-soon-to-be-released 920AV AR glasses. It wasn’t in working condition, but Vuzix promised it will be released this fall.

When the event kicked off, I offered to serve as your eyes and ears; Peter Milford took the offer and expressed his curiosity about the latest from Vuzix. This one is for you Peter: yours truly posing with the goods.

vuzix920v

Now the key question is:

how would you rank the look of these specs on a dork-to-cool scale?