Open Letter to Apple: Let us Augment Reality with the iPhone!

Posted on July 2, 2009 by Ori Inbar

A letter sent to Apple Developer Relations.

Dear Apple,

We are a collection of augmented reality (AR) enthusiasts and professionals (from business and academia), who have been working on a multitude of AR apps for the iPhone. These apps are poised to change the way people interact with the real world.

But here is the rub: we are currently unable to publish these apps on the app store because the iPhone SDK lacks public APIs for manipulating live video.

We are asking Apple to provide a public API to access live video in real time, on the iPhone.
We will be happy to offer additional technical details.

The impact of augmented reality (AR) on our lives could be as significant as the introduction of the PC.
In 10 years, we believe augmented reality will change the way everyone experiences travel, design, training, personal productivity, health care, entertainment, games, art, and advertising (videos).

Looking back just a few years, AR pioneers had to hack a slew of components into ridiculously large backpacks and HUDs, and be confined to rigged environments. Nowadays, it comes in friendly, affordable packages and the iPhone is one of the first devices to have it all – except for a public API.

The battle to determine the winning device has already begun; a public API to access live video will give the iPhone a lucrative ticket to compete.
We believe Apple has a window of opportunity of about 3 months before developers start looking elsewhere. If Apple decides to publish the API in that time frame – in the next 10 years, everyone might be using the iPhone as the preferred device to interact with the real world.

Here is how augmented reality could open up new opportunities for the iPhone this year:

Arf (Georgia Tech)

a virtual pet you take anywhere

ARghhhh (Georgia Tech)

first person table-top action game

Sekai Camera (Tonchidot)

AirTag the real world

Kweekies (int13)

a portal to creatures in a parallel world

Layar (SPRXmobile)

Browse the world with an AR browser – Details

Artoolkit for the iPhone (Artoolworks)

the most popular AR kit now on the iPhone

StudierStube ES (Imagination, Graz TU)

the only AR engine designed for mobile devices, now on iPhone – Details

PTAM on the iPhone (Oxford University)

next generation AR tracking with no markers or images

Wikitude (Mobilizy)

a travel guide that “tells you what you see”

Virtual Santa (Metaio)

interactive Christmas application using the augmented reality

Augmented Reality Sightseeing (Fraunhofer IGD)

Historic photographs overlaid on your field of view while strolling in a street

These are apps that are practically ready to go. There is a whole bunch of apps and games that are just waiting for the API to be available.

…And Apple, we know you can’t share your plans…so please surprise us soon!

Many many thanks for your consideration –
Sincerely,

Signed:
Michael Gervautz – Managing Director Imagination GesmbH
Robert Rice – CEO Neogence
Georg Klein – PhD PTAM creator from Oxford University
Stephane Cocquereaumont –  President & Lead Developer Int13 (Kweekies)
Maarten Lens-FitzGerald – Founder & Partner SPRXmobile, developer of Layar
Ori Inbar – Author of GamesAlfresco.com and CEO and founder – Ogmento (formerly Pookatak Games)
Philippe Breuss – Lead developer, Mobilizy
Philip R. Lamb – CTO, Artoolworks
Noora Guldemond – Metaio
Takahito Iguchi – CEO, Tonchidot
Blair MacIntyre – Associate Professor, Georgia Institute of Technology

Bruno Uzzan – CEO, Total Immersion
Michael Zoellner – Fraunhofer IGD
Andrea Carignano – CEO,  Seac02

If you are developing an AR app for the iPhone and wish to join this effort – just let us know.

Filed under: AR Devices | Tagged: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , | 171 Comments »

Wrap up 2008: Your Greatest Augmented Reality Moments

Posted on December 27, 2008 by Ori Inbar

Top 10 AR milestones in 2008 was one of the most popular posts this year. What came out of it was even more gratifying: a multitude of reflections, impressions, and thoughts I received about your own AR moments, including some last minute finds.

Here is an anecdotal collection of your greatest AR moments in 2008:

1. The Most fundamental AR milestone in 2008

Oriel Bergig: During 2008 we have seen some major advances in the field of Augmented Reality. Porting AR technology to mobile devices and especially cellular phones creates an opportunity to reach millions of users. For several years, the biggest AR labs and companies have made huge steps in this direction. In 2008 these efforts have started to show results. Pose estimation has been upgraded with the StbTracker release in the end of 2007. Research focusing on better user experience, and in particular on making mobile AR technology accessible to people with no special training, is being conducted by the best minds of the HitLabNZ. During one of the top covered events of the year, CES2008, Intel’s CEO Paul Otellini demonstrated Total Immersion’s technology enabling mobile AR experiences such as urban guidance. To wrap-up, the 2008 most fundamental milestone would be: AR technology is closing up fast on the mass user market.

Charles Woodward: The greatest milestone? Commercial breakthroughs by Metaio and Total  Immersion.

Thomas Wrobel: Wikitude I think. Seems the first released, useful, AR software. Runner up to the AR Geisha doll…

2) The best AR device of the year

Oriel: Since 2008 would most be remembered for its advances in mobile AR technology, the AR device of the year is the mobile phone. Nokia has released the Navigator phone that includes a GPS and an accelerometer, which make a valuable addition. The N95 has been demonstrated as well in many more contexts as a good choice for AR applications.  The next AR device of the year would be the Nokia N97 and of course the iPhone with its huge global success. iPhone feets very well AR applications and a successful attempt to port ARToolKit to iPhone has already been made by ARToolWorks. Appealing applications are next to come but only after the iPhone OS has better support for real time video acquirement.

Charles: Best device? iPhone, and/or Nokia 6210…

Thomas: hmz…tricky. I personally think hardware is still rather lackluster, and I have had little experience with some of the most recently released stuff.
I guess probably the iPhone + G1 devices…while far from ideal, they are at least getting location-aware services, and “barcode scanning” style product information into public hands.

Eric Rice shares what gets him excited about a video comparing between PS2 Eyetoy and PS3 Eye.

Vodpod videos no longer available.

3) Best AR Demo

Oriel: The best demo of 2008 is the demo that will be remembered by most people a decade from now. The demo that reached most of the people in the world is most likely Intel’s CEO Paul Otellini keynote talk during CES2008.

Charles: Haunted Book, Cherrer et al at ISMAR2008 – just beatiful!
(click Interaction on the left menu bar and then Haunted House.)

Thomas: LevelHead [by Julian Oliver], I think. Although this pet demo [ARf] is also nice;
(that may be because I want my own desuke though :p)

Vodpod videos no longer available.

4) Person of the AR year

Charles: Georg Klein – leads the way in anything he touches [single handedly won ISMAR 2008 tracking competition]

Thomas: There’s been so much development by so many individuals and companies I don’t know one specific person.

5) The most significant AR deal of 2008

Charles: Beijing Olympics fake fireworks. About the viewers of the Olympics openning ceremony:”What they did not realise was that what they were watching was in fact computer graphics, meticulously created over a period of months and inserted into the coverage electronically at exactly the right moment. ”

Thomas: Not sure about AR deals as such, but Total Immersion getting offices in the US is a good sign for the company and AR in general.

6) A [Predictable?] disappointment

Gizmondo won’t be coming out this year after all…The Nordik Link has the scoop.

7. Last minute find: A Surprising Simplicity in AR

Anyone can build 3D models with Google’s Sketchup. With the AR Media plugin from Inglobe – anyone can bring it into an augmented reality scene. ArchDaily tried it here.

Thanks for contributors and especially: Charles Woodward, Oriel Bergig, and the always there AR enthusiast: Thomas Wrobel

Bonus: Blair MacIntyre shares his greatest AR moments in 2009 in his blog.

Filed under: AR Events | Tagged: , , , , , , , , , , , , , , | 1 Comment »

The Making of ARf: Me, My Dog and i-Phone

Posted on December 4, 2008 by Ori Inbar

Blair MacIntyre sent me a nice proof of concept of an augmented reality virtual pet running on an iPhone.

So I thought, why not write about “the making of ARf”?

Shot a couple of questions to Blair and he conveniently turned it into a well structured interview. Thanks Blair!

Here it is for your edutainment.

games alfresco: Hey Blair, I’d like to write about ARf in my blog.

Blair: Great! :)

games alfresco: Is there anything beyond the video that I could share?

Blair: We (my student Kimberly Spreen, really) did this relatively quickly.  She figured out how to get video [on an iphone], and we’d been thinking about doing a virtual pet game for quite a while, so we decided to implement some of the ideas to test out the iPhone.

games alfresco: Could you share a description of the current features?

Blair: Right now, you can interact via the touch screen, and by moving the markers.  Kim did a nice little implementation of multi-marker tracking where you can just add new markers as you feel like and don’t need to preconfigure the multi-marker layout.  You can interact with the dog by touching it (touch its nose and it jumps up to lick, its tail and it chases it, rub its back and it rolls over to let you rub its tummy) or by touching the ground to send it somewhere.  If it gets near its water it drinks, near the other dog it plays, or near a smudge (that you put on the ground by rubbing the ground) it sniffs it (alas, the smudge looks like a little “pile”, which works, but wasn’t the intent).

games alfresco: Plans for a full game?

Blair: This is a project we’ve been thinking about for a few years, going back to our “Dart the Dog” project that we did in Director.  The goal is to explore what it means to let everyone have a virtual pet they can take with them, and interact with through different interfaces (desktop, handheld, handheld AR, etc).  Most importantly, we want the location (bedroom, living room, work, bus, bar, etc) and activity (sound level, light levels, etc) and presence of other pets to impact how the pet develops.

To handle the development, we are talking to some folks at an AI company, who are creating an engine for doing creature AI based on reinforcement learning.  They hope to have something we can use next year.  If we can get that, we will be able to really have pets that grown, change, evolve, etc.

A few company’s who are funding us are interested in this, so I hope we can devote some energy to it next year.  We’ll probably target a few platforms, but obviously the iPhone has a lot of appeal.  From a research perspective, I’m interested in it because there is the potential to release a research game and (with permission of the people who download it, of course), collect a lot of usage data.  Ironically, since the create AI engine is server based, I don’t know if we could handle a big success and provide the AI service to everyone who gets the game, but I’ll worry about that it we ever get there.

games alfresco: Can you share more details about the software? Is it a Jailbroken iPhone?

Blair: Official iPhone SDK, unhacked phones.  I have no interest in working with jailbroken phones;  the appeal of the iPhone is the potential for mass distribution to support broad evaluation and feedback.

Obviously, we have hacked the API to get at the camera, so we can’t release this until Apple creates an official API.

We are using StbTracker for tracking.  The rest of the software was written by us.

games alfresco: Cool. Thanks for showing us “under the hood” of ARf.

For a doggie game, the name ARf works nicely in English.

It could get weird when translated into:

  • Spanish – jau, jau
  • Afrikaans – blaf
  • Albanian – ham, ham
  • Arabic – how how
  • Armenian – haf, haf
  • Basque –  zaunk-zaunk
  • Bulgarian –  jaff, jaff
  • Catalan – bau, bau
  • Chinese, Cantonese – wow, wow
  • Chinese, Mandarin – wang, wang
  • Croatian – vau, vau
  • Danish – vov, vov
  • Dutch – waf, waf;
  • Esperanto – boj, boj
  • French – ouaf, ouaf
  • German – wuff, wuff;
  • Greek – ghav, ghav
  • Hebrew – hav, hav
  • Hindi – bho, bho
  • Icelandic – voff, voff
  • Indonesian – guk, guk
  • Irish – amh-amh
  • Japanese – wan, wan
  • Korean – mung mung
  • Latvian – vau, vau
  • Persian – vogh, vogh
  • Portuguese – béu-béu
  • Russian – gav, gav
  • Serbian – av, av
  • Slovenian – hov, hov
  • Thai – hoang, hoang

Filed under: AR Demos, AR Games | Tagged: , , , , , | 9 Comments »