Live from ISMAR ’08: Latest and Greatest on Augmented Reality Displays

Welcome back to ISMAR ’08; This is the second day and we are getting to the meaty topics.

Ozan Cakmakci is on stage and kicks off with walking through his paper: Optical free form surfaces in Off-Axis Head-Worn Display Design.

Ozan zooms through a quick history of optics and switches to a set of graphs and functions which you can review in his paper.

The conclusion is pretty clear though: Free form surfaces are useful in optical design to maximize performance in pupil size or field of view.

Questions such as who’s going to build it, when or how much it will cost – are left for guessing…

~~~

Next on stage is Sheng Liu from University of Arizona with the topic: An Optical See-Through Head Mounted Display with Addressable Focal Planes

Sheng talks about the stress on the eye in an AR situation where the eye has to accommodate real and virtual object and adjust the focus accordingly, and could cause headache to the viewer.

The solution is a variable-focal plane in a liquid lens.

Vari-focal with liquid lens for AR

Subjective tests result in a pretty good response from the participants. With the vari-focal plane in liquid lens, the human eye can accommodate change in focus from infinity to near focus and can be used for AR applications. This would even be improved in the future with upcoming improvements in liquid lens.

One of the members of the audience asks why not do this in software vs. hardware? Wouldn’t it be less expensive?

– Sheng claims the results are more accurate with the hardware approach.

To learn more about this, check out their website, the paper [link will be posted here], or contact sliu[at]optics.arizona.edu.

~~~

In the third leg of the “Displays” session Ernst Kruijff will speak about Vesp’R: design and evaluation of a handheld AR device.

UMPCs are a good starting point for AR displays – but tend to get bulky…

VAIO used for outdoor AR tracking at Oxford University

VAIO used for outdoor AR tracking at Oxford University

[I have analyzed this and other devices in my post: Top 10 AR devices]

Ernst will present an alternative design. The motivation for the research and the resulting paper was the lack of published knowledge  on this topic.

The team looked at a wide range of AR apps (such as Vidente an AR app for field workers) on different platforms and observed the common needs.

The need is simple: a lightweight device, with options for more controls, for long duration of use – indoors and outdoors.

UMPCs such as Vaio could are pretty heavy and become very tiring, especially when you hold it high.

Here is the result:

A solid case; velvety grip; controls are built into handles.
How good is it?
Based on a user attitude study – the new design is reasonable but not ideal…
When comparing with existing devices -some aspects were better and others not.

The conclusion is that although Vesp’R doubles the weight of a usual UMPC, it still provides improved ergonomics. But there is room for more research and improvements in this domain.

A member of the audience dares to ask: what if you used a much lighter device (such as a cell phone), would the results still be the same…?

Ernst is positive; just try to hold your hands straight ahead with no device at all – and you’ll feel the pain in a few minutes…

Stay tuned for the outdoor demo on Wednesday!

=================

From the ISMAR ’08 Program

  • Optical Free-Form Surfaces in Off-Axis Head-Worn Display Design
    Ozan Cakmakci, Sophie Vo, Simon Vogl, Rupert Spindelbalker, Alois Ferscha, Jannick Rolland
  • An Optical See-Through Head Mounted Display with Addressable Focal Planes
    Sheng Liu, Dewen Cheng, Hong Hua
  • Vesp’R: design and evaluation of a handheld AR device
    Eduardo Veas, Ernst Kruijff

ISMAR ’08 Live: Workshop on Industrial Augmented Reality: Needs and Solutions


Welcome to the first workshop of ISMAR 2008.

We are starting with the Industrial AR workshop.

Selim Benhimane introduces ISMAR Chair Ralf Rabaetje which introduces the first speaker Dr. Werner Schreiber from Volkswagen AG.

Ralf describes the main reason for VW to research in augmented reality: “we need to find new and better ways to develop, test and produce cars. And we need to make the process less expensive.”

VW is doing it as part of a government funded project dubbed AVILUS, in collaboration with major EU companies such as Airbus, Daimler and Siemens.

One of the improvements that can be achieved with AR is improved safety.

Werner shows various technologies that are being worked on, and slides of applications for improvement in designing and building cars. Example: applying labels for air bags in the language of the car’s destination. The error rate of the previous approach (using written lists) was improved dramatically with an AR system (with an HMD). Metaio provided elements of this solution.

Werner concludes with general requirements for these type of AR systems:

  • Keep it simple
  • Intuitive without special technology know how needs
  • Standard system
  • Universal system
  • Multi use in various industrial processes
  • Less than 30 min prep time
  • Economic
Question: How did workers react to these solutions?
– Some were skeptics, other were enthusiasts…you have to find tricks to make it easy to adapt to.
Q: Are you willing to take the risk of significanlty changing the process to include AR? Why not a sound system or monitors?
– adding the information in the field of view of the worker and reducing the cost – was worth it.
~~~
Second presenter is being introduced: Dr. Axel Hildebrand from Daimler heading an AR project; a perfect continuation of the previous talk. It will focus on how to deal with the maturity gap between needs and the current technology. Axel was formerly working on AR in the Fraunhofer institute.
We recognize that technology has to go through multiple stages until it’s ready for use in industrial systems, but we also know we have to start with such technologies early – to help them mature…we need to take some risk.
From Gartner’s hype cycle: in 2006 AR was a “technology trigger”. It was conspicuously missing in 2007 and then reappeared in 2008 – yet again as “technology trigger”.
At Daimler, the technology building blocks are: Data access, Interaction, displaer, visualization, tracking.
Example applications: Mobile picking objects. (collaborated with Metaio) will start a prototype with AR at the engine assembly line.
Current mobile devices are text driven usd for quality assurance.
With Symbian based devices added visuals in context to enrich the information workers have during picking objects.
Mobile Quality Assurance – a concept of using camera to visually test quality of products being produced.
Mixed Reality Ergonomics Situation – Use AR to improve posture of workers on assembly line. Using a mockup that simulates a car to test the posture of workers during certain tasks and then improving the procedures to improve the ergonomics.
Spatial AR for Automotive Design – projecting various scenarios (e.g. colors) on car models during design process.
Factory Shop floor approval – mobile AR device superimposes data from various sources about the shop floor to check the environment.
[skiping videos – what a shame…]
Thermal protection of the Overall Vehicle – superimposedata from simulations to that workers can readjust the engine [finally showing a video explaining the technique!]
Mixed reality Assembly

Axel summarizes: Need to havea step-wise approach. Convincing the business side, with high value projects – and then going further and applying to more projects.

Technologies still immature: indoor tracking, full HMD usage, AR visualization, Interaction. And we are still before the trough of disillusionment…

Some applications will require HMDs when the activity requires to be hands free. Others mobile devices are fine. And in other cases will need spatial AR.

[Coffee break]

Gudrun Klinker introduces Shinichi the next speaker Shinichi Aratani. He will talk about the current state of industrial MR/AR at Canon.

[colorful Japanese slides] Canon started working on MR in 1997 and focuses on 4 areas: Industry, Presentation, Art and Entertainment. Interior simulation of living room, Media art, etc. After 2001 moved to industrial use such as design evaluation, digital mockups, usability testing, etc. In order to achieve  value in MR applications  Canon assumed these requirements: real scale, intuitive visualization, intuitive operation.

Between 2000 and 2007 – reduced cost in development process and intend to continue reduction of cost. One example if simplified physical prototype. 84% of workers in a canon survey thought that MR applications improve effectiveness. Some noted that HMD can be mounted for no more than 15 min (beyond that it creates motion sickness). Another issue was narrow angle view where both hands can not be viewed.

Showing demonstration video held in Tokyo last week [

get link]: simulates how to maintain a canon printer. Concept HMD used is VH2007 with higher resolution, including a video camera.

Canon intends to use CAD to simulate actual operation. Input motion parameters, display analytical simulation on top of that simulated operation. Showing concept video of a lens mockup, superimposing motion parameters of a real product. [get link]. It ‘s a promising concept, but there are still issues with resolution and picture quality…

He then goes to describe the MR platform marker technology, sensor use, calibration tools, etc.

Future work: offer a common platform for MR. Details still fuzzy…

Areas of future focus: navigation, construction, art…here is such an example:

Tracking the motion of an instrument (Clarinet) for a Media Art project : super imposed graphics change based on the sound and movement – very amusing!

Canon sees major value in MR and continue to develop the platform and HMD.

~~~

Next speaker:  Benjamin Becker (EADS) European Aeronautic Defense and Space Company

From the advanced design and visualization team working on: industrial design for aircrafts (e.g. Airbus), cabin interior, seatings, lower deck crew test, catering, lavatories. Also working on visualization for Helicopters, etc.

Main AR project: Trackframe using Ubitrack tracking framework.

AR combines multiple technologies: rendering and visualization, wearable computing, tracking. Caveats: HMD, interaction and usability, local and global tracking for large area.

Example: Sales and marketing project – present concepts of improved cabins (e.g. adding a bar) to customers and solicit feedback (add coloring, and in the future provide haptics). Spatial AR: Projecting daylight or night like on the cabin ceiling to help passengers adjust to jet lag [get video].

Explaining additional examples from maintenance, manufacturing, factory planning.

Question: Are we getting aesthetically pleasing view?

-it’s not photorealistic, but it’s better than seeing the options in a textual list…

[Unfortunately, I’ll have to miss the afternoon sessions in this track – due to the parallel Handheld mobile AR session which I can’t afford to miss…]

========================

From ISMAR ’08 program.

Organizers: Selim Benhimane (TUM), Gudrun Klinker (TUM), Ralf Rabaetje (Volkswagen AG), Bruce H. Thomas (UniSA)

As the interest and the development of Augmented Reality (AR) is growing fast, it is important that, periodically, people from academia, research and industry sit together and discuss about what are the major limitations and results that were achieved recently. This workshop is the followup to the two successful one-day events that took place at ISMAR’05 in Vienna and at ISMAR’06 in Santa Barbara. The workshop will be split into four sessions of invited talks:

– Recent Advances in Tracking and Programming Frameworks for AR

– Requirements on AR Systems imposed by Industrial Applications

– Requirements on AR Systems imposed by Industrial Applications (continued)

– Recent Advances in Visualization and User Interfaces

There will be 9 speakers and each speaker will give 25-minute talk followed by a 5-minute questions and answers. An open discussion will take place at the end of the workshop in order to get the audience and the speakers discussing questions: What does Industry need from AR? What problems need to be solved for AR to work in Industry? What are the good target Industries for AR as it is seen in 2008?

Further information, including the full program and details of speakers, can be found on the Workshop website.

Live from ISMAR ’08 in Cambridge: Enjoy the Weather…

“Enjoy the weather” uttered sarcastically a kindhearted British witch (aka air hostess) while we were leaving the aircraft; surprisingly – we did in the first day. We were then promised this is accidental and surely the last day of summer. Splendid.

Venice? nope, Cambridge!

Venice, Italy? nope, Cambridge, UK!

I have landed in Cambridge, UK (where people go to augment their reality) and all I ever heard about it – is true: British meadow green, majestic 600 year old buildings, cosmopolitan young folks, fish cakes…a combination that gives this university city its unique aura; a great setting for the event starting tomorrow – reality only better, at ISMAR ’08.

St. Catherine College - can't ask for a nicer place to stay...

St. Catherine College - can't ask for a nicer place to stay...

For those who couldn’t make it, stay tuned for a live coverage of ISMAR ’08, the world’s best augmented reality event.

Featuring AR pioneers such as: Tom Drummond, Paul McIlroy, Mark Billinghurst, Blair MacIntyre, Daniel Wagner, Wayne Piekarski, Uli Bockholt (Fraunhofer IGD), Peter Meier, Mark A. Livingston, Diarmid Campbell, David Murray, Rolf R. Hainich, Oliver Bimber, Hideo Saito and many more —

— overing topics such as: Industrial augmented reality, hand-held augmented reality, displays, user studies, applications, layouts, demos, state-of-the-art AR, and don’t miss the highly anticipated tracking competition.

Welcome all speakers and attendees to the event, and don’t forget: look right first!

If you are at the event (or not) and want to chat, share thoughts, or ask questions – leave a comment here or send a message on facebook.

Augmented Reality Panel at Virtual Worlds Conference

For those of you who missed Virtual Worlds last week in LA, and therefore found it hard to attend the Augmented Reality Panel – David Orban was there for you.

Here are all 59 minutes of the discussion he recorded and uploaded. Thanks David!

Roo, the panel host, brilliantly captured a blow by blow of the discussion and included slides that were used to illustrate the panel. Kudos to Roo!

From the event program:

Augmented Reality: Virtual Interfaces to Tangible Spaces
Marc Goodman, Director, Alcatel-Lucent
Eric Rice, Producer, Slackstreet Studios
Blair MacIntyre, Associate Professor, School of Interactive Computing, Director, GVU Center Augmented Environments Lab, Georgia Institute of Technology
David Orban, Founder & Chief Evangelist, WideTag, Inc.
Andrew (Roo) Reynolds, Portfolio Executive for Social Media, BBC Vision (moderator)

Now repeat after me this tongue twister: I virtually participated in the Virtual Worlds panel about augmented reality and I really augmented my world.

Next, we are off to ISMAR ’08 September 15-18th in Cambridge, UK!

Unveiling Tonchidot: A Cool Parallel World – on the iPhone

Yesterday, at the TechCrunch 50 conference in San Francisco, a small start up from Tokyo stole the show; the name: Technidot, the cool product they unveil: Sekai Camera (World Camera) running on the iPhone.

In their own words, the Sekai Camera is:

…a real-world interface for the iPhone that connects real and virtual worlds, allowing anybody to create, experience and participate in both.

While at your favorite mall, point your iPhone camera at things around you such as food, toys, art, transit maps, and get detailed information about it. More fun for you, more business for retailers.

AirFilter takes care of the customized search to bring you the additional information, based on a database generated by advertizers.

And you’re not on your own; the Sekai Camera is also a social networking environment: leave messages to your friends – in space – so they can see it when they pass by.

Isn’t that a killer augmented reality application?

And Tonchidot’s take on Evolution could become the symbol of the AR revolution…

Here is the coverage of the demo by Anthony Ha on VentureBeat : including enthusiasts and skeptics.

This just in! More skeptics!

Blair MacIntyre finds it hard to believe it’s real. The TechCrunch panelists were amused and confused but blamed it on the language barrier.

What do you think? Is it real? or really augmented?

Filip, many thanks for the tip!

Will Wright: Augmented Reality – Way Forward For Mobile Gaming

Captured in an interview with Pocket Gamer (Thanks!) in London via Kotaku:

“I can imagine mobile platforms evolving…in that they interact with the world around us in a way that changes our perceptions in a really interesting way”, said Wright, “Games could increase our awareness of our immediate environment, rather than distract us from it”.

Will Wright is by far my idol in the gaming world. He’s the smartest and the funniest, and he single handedly invents new game genres (SimCity, The Sims, Spore).

If you wonder why or how, give yourself 20 minutes off and watch his speech at TED

 

Is Will joining the Augmented Reality revolution?

(Can you say “mega idol”?…)

I want my iPhone augmented too!

The buzz continues.

As the coolest gadget around, the iphone is raising a lot of interest in the augmented reality community. This was once again confirmed in an interview with Stéphane Cocquereaumont, President and Lead developer at Int13, a mobile games developer:

“The iPhone is in fact our main target, the next demo we’ll publish will be on this device, this is probably the best device to do mobile AR today, even if its camera is far from perfect.”

Stéphane conversed with me following a post of Int13’s demo on the augmented reality games facebook group titled: “Augmented Reality on mobile devices that just works” 

He added:

“Our AR library should work easily on devices such as Intel’s MID [which is coming out this summer – games alfresco]…but we’re more interested in Smartphones…”

Games alfresco: How central do you see augmented reality in your company’s future?

Stéphane: “We plan to continue working on our AR technology and improve it, we’re especially interested in markerless tracking.”

And here is what I really liked in this interview:

“But our main objective is to create cool games and sell them”

Int13 is among the first pure game developers to dive into the AR space.

“if our current AR projects reveal themselves to be commercially successful then AR could become central for us.”

Cautious optimism, always a good trait for a game developer.

Thanks Stéphane and France from Int13 for sharing your experiences and plans!

 

Don’t you want your iPhone augmented too?

How do you spell Augmented Reality in iPhonese?

Doesn’t it feel like every other day, a new augmented reality demo for the iPhone hits YouTube?

 

Here is a play by play of the sequence of events that led the iphone to the top of the AR devices list:

First, on June 20th, “Augmented Reality Navigation” on the iphone hits youtube…

 

Then the Holotext hoax (on the iPhone) confuses the already confused youtube audience…

 

Furthermore, on July 18th, Thom York shows up singing as a [tricked] hologram on the iphone…

 

Nonwistawnding, artoolworks surprises, on July 21st, with the release of arttoolkit 4.4 for iPhone… 

 

What will they do next?

The End Of Museums As We Know It

As the light at the end of the (summer vacation) tunnel is almost insight, let me ask you a reflective question:

Where do kids prefer to be on a summer day:

(a) Museums

(b) Theme parks

(c) Staying at home and playing video games

Any volunteers for (a) ?…

What if you could combine all three into one?

What if you could transform learning about cultures, art, science, history – into a fun experience for kids? What if museums were as much fun as outdoor adventures and video games combined?

This fantasy is becoming a reality thanks to efforts by pioneers around the world.

Here are my picks of the 4 5 best augmented reality tours that are reinventing museums:

1. Lifeplus in Pompei

Pompei Ladies in an afternoon promenade in front of your eyes

Breathing new life into the ruined streets of Pompei

Visitors stroll in the real streets of Pompei, while watching thru their glasses, virtual scenes of city natives living their lives as if it’s 79 AD, minutes before the eruption of Vesuvius.

This EU funded project was lead by MIRALab – university of Geneva in 2004. See more at Lifeplus.

2. DNP-Louvre Museum lab

A behind the scenes look at exhibits

A mobile device with live video, shows on the display virtual objects such as a balloon that guides visitors through the exhibits. The climax of this tour arrives (1:37) when shards of an antique Islamic platter are virtually reconstructed to create the real platter.

Kudos to Metaio who developed the experience for DNP-Louvre Museum Lab in Tokyo, though they should try trimming the bulky device…

See more at DNP-Louvre Museum lab in Tokyo

3. Mobile Augmented Reality Quest (MARQ) –         Expedition Schatzsuche

Treasure hunt in a museum (in Austria)

A team oriented game where museum visitors play the role of investigators required to solve 3d virtual puzzles surrounding exhibits. Successful completion of puzzles reveal further steps of the story.

Beyond the new type of interaction with museum exhibits, MARQ introduces multi user collaboration: collected virtual items can be shared between groups, and “guided tour replays” can be viewed at any time – on the Gizmondo (RIP) gaming device.

This novice approach to experiencing Museum exhibits was developed by the Graz University team in Vienna, led by Daniel Wagner and Dieter Schmalstieg. It was shot at the Kärner Landesmuseum in Klagenfurt/Carinthia.

See more about MARQ at Studierstube

And here is an older project from the same team: Enigma Rally at the Vienna Technical Museum.

4. Rome Reborn, now Augmented

Undo the barbarians

Rome Reborn is the largest computer simulation of an ancient city. Cool. But what Fraunhofer (Institute for Computer Graphics Research) has done with it is way cool: walk among the ruins of the Roman Forum and point your Vaio UMPC anywhere to see buildings being reconstructed.

See more about the technology used at InstantReality

5. Voices of Oakland

If a cemetery were a museum

Blair MacIntyre and his team at GA Tech have done the impossible: they have turned the Oakland cemetery in Atlanta to a visitor magnet – all thanks to an augmented reality tour which can be experienced on a cell phone. You have to see it to believe it.

Unfortunately, the cemetery was destroyed last year by a hurricane. So the students of subsequent years wont be able to keep playing with it.

6. Science Museum in Paris

Navigate Museums with AirTags

A new implementation by Tonchidot for La Villette Museum in Paris where visitors use AirTags provided by the Museum or by users to enrich their museum experience.
In the creators own words:

Sekai camera turns a museum into a “living” internet environment…The real world becomes “clickable”

7. Digital Binocular Station for Cultural Museums

A stationary Augmented Reality device developed by Mind Space Solutions. Because it is fixed to a single location, it allows the use photorealistic, cinema-quality visuals, and compensate for the lack of parallax by presenting everything in stereoscopic 3D.

=========================================

What are the 10 ingredients to augment a museum tour?

  1. A practical augmented reality device (avoid backpacks and bulky displays) with visual tracking software
  2. High quality 3D models of exhibits, and how they looked and behaved in the past (and future?)
  3. A combination of learning and fun with a really really good story
  4. Breath life into inanimate objects
  5. Sprinkle some sound when necessary
  6. Incinerate verbose plaques; say what you have to say in 2 to 5
  7. Indoor tours are great; outdoor tours are even better
  8. Multi user interaction and collaboration
  9. Location based services, including (fun) navigation instructions
  10. Did I mention it has to be fun?
…and don’t forget to send the kids home with a souvenir DVD: “my augmented tour at the museum”

Didn’t make the list…

The following efforts didn’t make the list, mostly because they forgot ingredient #10…

Have you seen other augmented museum tours? Share your experiences!

Want your own augmented reality Geisha?

Geisha Tokyo Entertainment Inc. announced a new augmented reality game. They call it Dennoh Figure ARis or Cyber Figure Alice (which only works in japanese thanks to the interchangeability of L and R in Japanese). Thanks Akki from Asiajin for bringing it to the western world. It is scheduled to hit Japan this fall.

Is it a breakthrough?
From a technical perspective we have seen similar implementations on the market – many covered in this blog’s demo category (see left sidebar). It’s based on the widely used marker AR technology (that is ‘widely’ within the narrow AR community).

What’s new about it is the theme chosen for this game: a high-school-graduate-in-training-geisha. This is a departure from the infant oriented magic books and wiz cubes, yet it doesn’t dwell with extreme practicality as the Fix-Your-Own-Car applications.

Enter the world of augmented reality for grown ups: here you can peep, poke (using the supplied cyber sticks), and peep at your own geisha.
The publisher describes her (according to Google’s translation of the product’s site) as “rich in the sensitivity of your desktop to make sure spectacular”. You may rephrase based on your own taste.

Now who will take it outdoors?

What if you could experience this non-intrusive interactivity with real people on the street?