Simply put – it offers a new API that allows to overlay graphics on a live video stream.
This is great news for Augmented reality (AR) browser applications such as Layar and Wikitude (for more examples see right side bar). Although similar applications have sneaked into the app store before – I am sure it’s great to finally be legit on the store.
However, 3.1 provides only semi support for AR apps.
When you want to align graphics with real world things (like markers or other objects) – you need to be able to analyze the live video stream off of the iPhone camera. This functionality is only available as a private API on the iPhone (even with 3.1). It’s technically possible, but Apple may not allow such an app on the App store.
By the way, when I say “support” I do not mean by any means that Apple provides AR capabilities on the iPhone. Some folks in the press got that impression from the flood of headlines announcing support for AR on the iPhone…
To clarify, AR companies still need to do the heavy lifting such as AR detection and tracking; bearing the ripe fruits of many years of research and development.
So AR is not yet totally legit on the iPhone. But we are getting closer.
Augmented Reality has hit the iPhone app store. And we didn’t even have to wait for OS 3.1.
Presselite apps have been updated with a new Augmented Reality functionality called Your New Eye. This only works with the latest iPhone 3GS (compass needed). If you don’t believe it – check out the quick demo here :
In the words of co-founder Antoine Morcos:
Basically, with this application, you are able to see all the metro and bus stations around you (<1km), and also a lot of Points of Interest (Restaurants, etc.) via Augmented Reality, thanks to the camera.
London Bus is available on AppStore. (More details – screenshots, etc.)
Métro Paris for iPhone is also available here.
How come Apple approved an app that uses a private API ?
Does it really matter as long as it did?
Has Apple opened up?
Will it signal a flood of AR apps on the iPhone app store?
It may treasure what we’ve all been waiting for. The elusive API. The holy access to live video on the iPhone.
We will never know if the Open Letter to Apple had any dent on Apple’s decision to introduce the new APIs – but for a moment – we are blissful.
By tomorrow we’ll know for sure if it works. What ever the outcome, at least we made a lot of friends and discovered a swarm of AR developers eager to bring the augmented reality experience to the masses.
Thank you all for the overwhelming response!
And let me challenge you:
The first to confirm the above theory will be indicted to
a new identification application apparently under development by Apple that would help identifying objects in a user’s surroundings so that their iPhone can present additional information about the identified objects.
The patent describes scenarios such as:
“the portable electronic device can allow the user to select a mode based on the types of objects that the user wants to identify. Based on the selected mode, the portable electronic device can adjust parameters used for searching an identification database. For example, if the user selects to identify an object in a “MUSEUM” mode, the portable electronic device can search the identification database for objects that are commonly found in a museum. In some embodiments, the portable electronic device can determine the location of the user to help identify an object. For example, if the user is determined to be in Las Vegas and the portable electronic device is set to a “RESTAURANT” mode, the device can limit the search of the identification database to restaurants in Las Vegas.”
As much as it’s encouraging to see Apple’s interest in this domain, it sounds awfully similar to augmented reality research published over the past 10 years.
Moreover, it actually describes the functionality behind existing AR browsing applications already in the market such as Layar, Mobilizy’s World Browser, Tonchidot’s Sekai Camera, Nru, and more!
And how about augmenting Museum experiences? Has anyone at Apple read our roundup of AR museum experiences?
Is there anything new in this patent? Can Apple defend it against previously published AR work? What do you think?
We are a collection of augmented reality (AR) enthusiasts and professionals (from business and academia), who have been working on a multitude of AR apps for the iPhone. These apps are poised to change the way people interact with the real world.
But here is the rub: we are currently unable to publish these apps on the app store because the iPhone SDK lacks public APIs for manipulating live video.
We are asking Apple to provide a public API to access live video in real time, on the iPhone.
We will be happy to offer additional technical details.
The impact of augmented reality (AR) on our lives could be as significant as the introduction of the PC.
In 10 years, we believe augmented reality will change the way everyone experiences travel, design, training, personal productivity, health care, entertainment, games, art, and advertising (videos).
Looking back just a few years, AR pioneers had to hack a slew of components into ridiculously large backpacks and HUDs, and be confined to rigged environments. Nowadays, it comes in friendly, affordable packages and the iPhone is one of the first devices to have it all – except for a public API.
The battle to determine the winning device has already begun; a public API to access live video will give the iPhone a lucrative ticket to compete.
We believe Apple has a window of opportunity of about 3 months before developers start looking elsewhere. If Apple decides to publish the API in that time frame – in the next 10 years, everyone might be using the iPhone as the preferred device to interact with the real world.
Here is how augmented reality could open up new opportunities for the iPhone this year:
Our inaugural post from early 2008: “Top 10 AR demos that will…” sparked huge interest. Since then, we have witnessed loads of AR games swarming the market.
Well, that may be an exaggeration – but the industry has certainly transitioned from delivering mere demos to actual games; from proof of concepts to commercial products; from “Yay” to “W00t!”
We have covered these AR games before, but Today is your chance to choose.
Vote for your all time favorite augmented reality games!
Our only rules for nomination:
1) It’s a fun game
2) It registers computer graphics on reality
3) It runs on commercial off the shelf hardware.
Here are the 18 nominees in chronological order (when first surfaced on the web):
go!
1. The Invisible Train
2004 – Graz University (PDA, Gizmondo)
2. Catapult
March 2006 – Gizmondo (Gizmondo)
3. Eye of Judgment
May 2006 – Sony (Sony EyeToy)
4. AR Tennis
June 2006 – Fanta/HIT Lab NZ (Nokia)
5. WizQubes
March 2007 – MXR
6. Level Head
October 2007 – Julian Oliver (webcam)
7. ARis
July 2008 – Geisha Entertainment (Webcam)
8. Kweekies
October 2008 – Int13 (Nokia, iPhone)
9. Ghostwire
October 2008 – A Different Game (Nintendo DSi, Nokia)
10. Tower of Defense
December 2008 – Sergey Ten
11. Topps
March 2009 – Total Immersion (Webcam)
12. Scope
March 2009 – Frantz Lasorne (Goggles)
Vodpod videos no longer available.
13. Do the Dip
April 2009 – MacDonald’s (webcam)
14. ARhrrrr!
May 2009 – GA Tech and SCAD-Atlanta (Nvidia Tegra)
15. Candy Wars
May 2009 – GA Tech and SCAD-Atlanta(Gizmondo)
16. Art of Defense
May 2009 –GA Tech (Nokia)
17. RubberDuckzilla
May 2009 – Oasis (webcam)
18. InVizimals
June 2009 – Sony (PSP)
-*-*-*
Which are your favorites?
Share with your friends and find out their favorites! (share button at top right of page)
Robert Tercek Chairman of GDC Mobile kicks of the Mobile summit with “Welcome to the Next Level! ”
He speaks about the state of the mobile gaming world (looks much better than last year) and even spends a moment on our favorite topic: Virtual layers superimposed on a fictitious reality. He admits it has only been done for advertising so far.
Next is a session I have been looking forward to: “Why the iPhone just changed everything” by
Neil Young founder of Ngmoco :)
When it comes to mobile games, everything before the iPhone was lame. Mostly because of the carriers, the devices, and their usability.
With the iPhone a new Everything was born.
As a game machine it’s so much better than anything before.
4 major Changes:
1) The market – iPhone has more installed based than the Nintendo DS and Sony PSP. 165 new apps per day.
Short term problem: Clutter + pricing pressure.
Long term – awesome opportunity.
2) The games – Graphics are better. The experience is better.
And it’s always connected. new opportunities for social, multi players.
3) Game making – fantastic SDK and a much faster development cycle.
4) and Publishing.
There’s never been a better time to be an independent developer and to create a closer relationship with customers.
Then comes the Ngmoco plug: they help create hits by focusing on largest audience appeal, superior execution, and making sure it’s native to the iPhone
There other job is helping games get in front of as many people as possible.
The Ngmoco platform helps is that endevour: analytics, silent download of levels and episodes, promotions engine.
The session ends in high note: iPhone is current the top dog.
Take pictures of the cube’s faces with your iPhone and it will guide you through the shortest number of moves required to solve it.
Now, take it a step further and imagine the iPhone (or better yet – goggles) continuously watching your cube and cleverly guiding you on every move – for the fastest solution ever.
Wouldn’t it be cool?
Well, at least for Cube obsessed kids, it would.
In any case, I’d mark this app as an important milestone towards putting augmented reality to use for the good of mankind.
Today it teaches us how to solve the Rubik’s cube. Tomorrow it will teach us everything else.
Top 10 AR milestones in 2008 was one of the most popular posts this year. What came out of it was even more gratifying: a multitude of reflections, impressions, and thoughts I received about your own AR moments, including some last minute finds.
Here is an anecdotal collection of your greatest AR moments in 2008:
1. The Most fundamental AR milestone in 2008
Oriel Bergig: During 2008 we have seen some major advances in the field of Augmented Reality. Porting AR technology to mobile devices and especially cellular phones creates an opportunity to reach millions of users. For several years, the biggest AR labs and companies have made huge steps in this direction. In 2008 these efforts have started to show results. Pose estimation has been upgraded with the StbTracker release in the end of 2007. Research focusing on better user experience, and in particular on making mobile AR technology accessible to people with no special training, is being conducted by the best minds of the HitLabNZ. During one of the top covered events of the year, CES2008, Intel’s CEO Paul Otellini demonstrated Total Immersion’s technology enabling mobile AR experiences such as urban guidance. To wrap-up, the 2008 most fundamental milestone would be: AR technology is closing up fast on the mass user market.
Charles Woodward: The greatest milestone? Commercial breakthroughs by Metaio and Total Immersion.
Thomas Wrobel: Wikitude I think. Seems the first released, useful, AR software. Runner up to the AR Geisha doll…
2) The best AR device of the year
Oriel: Since 2008 would most be remembered for its advances in mobile AR technology, the AR device of the year is the mobile phone. Nokia has released the Navigator phone that includes a GPS and an accelerometer, which make a valuable addition. The N95 has been demonstrated as well in many more contexts as a good choice for AR applications. The next AR device of the year would be the Nokia N97 and of course the iPhone with its huge global success. iPhone feets very well AR applications and a successful attempt to port ARToolKit to iPhone has already been made by ARToolWorks. Appealing applications are next to come but only after the iPhone OS has better support for real time video acquirement.
Charles: Best device? iPhone, and/or Nokia 6210…
Thomas: hmz…tricky. I personally think hardware is still rather lackluster, and I have had little experience with some of the most recently released stuff.
I guess probably the iPhone + G1 devices…while far from ideal, they are at least getting location-aware services, and “barcode scanning” style product information into public hands.
Eric Rice shares what gets him excited about a video comparing between PS2 Eyetoy and PS3 Eye.
Vodpod videos no longer available.
3) Best AR Demo
Oriel: The best demo of 2008 is the demo that will be remembered by most people a decade from now. The demo that reached most of the people in the world is most likely Intel’s CEO Paul Otellini keynote talk during CES2008.
Charles: Haunted Book, Cherrer et al at ISMAR2008 – just beatiful!
(click Interaction on the left menu bar and then Haunted House.)
Thomas: LevelHead [by Julian Oliver], I think. Although this pet demo [ARf] is also nice; (that may be because I want my own desuke though :p)
Vodpod videos no longer available.
4) Person of the AR year
Charles: Georg Klein – leads the way in anything he touches [single handedly won ISMAR 2008 tracking competition]
Thomas: There’s been so much development by so many individuals and companies I don’t know one specific person.
5) The most significant AR deal of 2008
Charles: Beijing Olympics fake fireworks. About the viewers of the Olympics openning ceremony:”What they did not realise was that what they were watching was in fact computer graphics, meticulously created over a period of months and inserted into the coverage electronically at exactly the right moment. ”
Thomas: Not sure about AR deals as such, but Total Immersion getting offices in the US is a good sign for the company and AR in general.
6) A [Predictable?] disappointment
Gizmondo won’t be coming out this year after all…The Nordik Link has the scoop.
7. Last minute find: A Surprising Simplicity in AR
Anyone can build 3D models with Google’s Sketchup. With the AR Media plugin from Inglobe – anyone can bring it into an augmented reality scene. ArchDaily tried it here.
Thanks for contributors and especially: Charles Woodward, Oriel Bergig, and the always there AR enthusiast: Thomas Wrobel
Bonus: Blair MacIntyre shares his greatest AR moments in 2009 in his blog.
OK, how many augmented reality apps can you find on the iPhone app store?
None.
That is – none yet. Not until Apple makes public its live video API – because without live video – there is nothing really to augment.
So in the meantime, whom can we learn from?
Let’s take a look at select iPhone apps – with a special focus on kids. After all, we want the NEXT generation to get hooked on AR.
See how kids interact with iPhone apps…
See how screens are organized…
See the space required for finger touch…
See how much content can be crumbed into a screen…
See how to use the accelerometer…
See for yourself…
…and most importantly – enjoy!
Word Magic – learn to spell
Interactive book – Interactive book gone wild on the iPhone
Shapes, Colors, Sounds – that’ll keep the toddler busy for 5 minutes
Memory game – how wide is your finger?
A Fast Collection – how long do you play an iPhone app?
3 year old experience – playing through the eyes if a 3 year old
Apps for kids
iChalky – Clever use of accelerometer with a stick figure
Pinch ‘n Pop – there is a use for finger pinching besides zooming
Advanced editor – how many functions can you cram into an iPhone?
![Reblog this post [with Zemanta]](https://i0.wp.com/img.zemanta.com/reblog_e.png)
Games Alfresco 