Live from ISMAR ’08: Augmented Reality – What Users Are Saying

Everyone is back from lunch and the afternoon session is on: User studies in augmented reality.

First on stage is Benjamin Avery to talk (with an animated Australian accent) about User Evaluation of See-Through Vision for Mobile Outdoor Augmented Reality. This

The study took users outdoors in various scenarios to test the performance of AR vision using see through displays. Then, they compared it with a second group that watched the video through a desktop computer

[link to paper, videos, images to come]

The result demonstrates complex trade-off between AR and desktop visualizations. AR system provided increased accuracy in locating specific points in the scene. AR visualization was quite simple, beating the desktop in tracking and in better visualization.

Stay tuned for the demo (which was hauled all the way from Australia to Cambridge)!


Next on stage is Cindy Robertson from Georgia Tech (Honorable mention in ISMAR 2007) and she discusses An Evaluation of Graphical Context in Registered AR, Non-Registered AR, and Heads-Up Displays.

How are users affected when there are many registration errors  or in other words when when tracking is not perfect? Can the user handle it better if a graphics context is provided?

They tested it with a set of tasks encompassing placing virtual Lego blocks with groups using Registered AR, Non-Registered AR, and Heads-Up Displays.

Following an exhaustive analysis of the resulted data they uncovered the following insights:

  • Head movement and memorization increased performance
  • Head movement affected perceived mental workload and frustration
  • When you have graphics obstructing your view, and switching between it and real world is frustrating
  • HUD-visible case was surprisingly faster than the other cases. But people hated it…

Final conclusion: Registered outperformed both the non-registered AR and graphics displayed on a HUD. Non-registered AR does not offer any significant improvement.

Future plans are to test home-like scenarios and impose more complex tasks.


On stage Mark Livingston is getting ready to talk about The Effect of Registration Error on Tracking Distant Augmented Objects.

A basic assumption is that registration errors limits performance of users in AR. “We wanted to measure the sources (such errors are noise, latency, position and orientation error) and see the affect on the user – and then be able to write requirements for future systems.”

For this study, they used the nVisorST.

The tasks were trying to measure the users ability to understand behaviors and situational awareness in the AR application: following a target (car) when buildings stand in between.

Conclusions are straight forward though somewhat surprising:

  • Latency has significant effect on performance and response time – was the worse.
  • Noise was disliked but did not have significant impact on performance
  • Orientation error fifn’t have significant effect
  • Weather had significant impact on results: darker weather delivered improved performances. Brightness was a major distraction.


From the ISMAR Program

User Studies (from ISMAR ’08 program)

  • User Evaluation of See-Through Vision for Mobile Outdoor Augmented Reality
    Benjamin Avery, Bruce H. Thomas, Wayne Piekarski
  • An Evaluation of Graphical Context in Registered AR, Non-Registered AR, and Heads-Up Displays
    Cindy Robertson, Blair MacIntyre, Bruce Walker
  • The Effect of Registration Error on Tracking Distant Augmented Objects
    Mark A. Livingston, Zhuming Ai

ISMAR ’08 Live: Workshop on Industrial Augmented Reality: Needs and Solutions

Welcome to the first workshop of ISMAR 2008.

We are starting with the Industrial AR workshop.

Selim Benhimane introduces ISMAR Chair Ralf Rabaetje which introduces the first speaker Dr. Werner Schreiber from Volkswagen AG.

Ralf describes the main reason for VW to research in augmented reality: “we need to find new and better ways to develop, test and produce cars. And we need to make the process less expensive.”

VW is doing it as part of a government funded project dubbed AVILUS, in collaboration with major EU companies such as Airbus, Daimler and Siemens.

One of the improvements that can be achieved with AR is improved safety.

Werner shows various technologies that are being worked on, and slides of applications for improvement in designing and building cars. Example: applying labels for air bags in the language of the car’s destination. The error rate of the previous approach (using written lists) was improved dramatically with an AR system (with an HMD). Metaio provided elements of this solution.

Werner concludes with general requirements for these type of AR systems:

  • Keep it simple
  • Intuitive without special technology know how needs
  • Standard system
  • Universal system
  • Multi use in various industrial processes
  • Less than 30 min prep time
  • Economic
Question: How did workers react to these solutions?
– Some were skeptics, other were enthusiasts…you have to find tricks to make it easy to adapt to.
Q: Are you willing to take the risk of significanlty changing the process to include AR? Why not a sound system or monitors?
– adding the information in the field of view of the worker and reducing the cost – was worth it.
Second presenter is being introduced: Dr. Axel Hildebrand from Daimler heading an AR project; a perfect continuation of the previous talk. It will focus on how to deal with the maturity gap between needs and the current technology. Axel was formerly working on AR in the Fraunhofer institute.
We recognize that technology has to go through multiple stages until it’s ready for use in industrial systems, but we also know we have to start with such technologies early – to help them mature…we need to take some risk.
From Gartner’s hype cycle: in 2006 AR was a “technology trigger”. It was conspicuously missing in 2007 and then reappeared in 2008 – yet again as “technology trigger”.
At Daimler, the technology building blocks are: Data access, Interaction, displaer, visualization, tracking.
Example applications: Mobile picking objects. (collaborated with Metaio) will start a prototype with AR at the engine assembly line.
Current mobile devices are text driven usd for quality assurance.
With Symbian based devices added visuals in context to enrich the information workers have during picking objects.
Mobile Quality Assurance – a concept of using camera to visually test quality of products being produced.
Mixed Reality Ergonomics Situation – Use AR to improve posture of workers on assembly line. Using a mockup that simulates a car to test the posture of workers during certain tasks and then improving the procedures to improve the ergonomics.
Spatial AR for Automotive Design – projecting various scenarios (e.g. colors) on car models during design process.
Factory Shop floor approval – mobile AR device superimposes data from various sources about the shop floor to check the environment.
[skiping videos – what a shame…]
Thermal protection of the Overall Vehicle – superimposedata from simulations to that workers can readjust the engine [finally showing a video explaining the technique!]
Mixed reality Assembly

Axel summarizes: Need to havea step-wise approach. Convincing the business side, with high value projects – and then going further and applying to more projects.

Technologies still immature: indoor tracking, full HMD usage, AR visualization, Interaction. And we are still before the trough of disillusionment…

Some applications will require HMDs when the activity requires to be hands free. Others mobile devices are fine. And in other cases will need spatial AR.

[Coffee break]

Gudrun Klinker introduces Shinichi the next speaker Shinichi Aratani. He will talk about the current state of industrial MR/AR at Canon.

[colorful Japanese slides] Canon started working on MR in 1997 and focuses on 4 areas: Industry, Presentation, Art and Entertainment. Interior simulation of living room, Media art, etc. After 2001 moved to industrial use such as design evaluation, digital mockups, usability testing, etc. In order to achieve  value in MR applications  Canon assumed these requirements: real scale, intuitive visualization, intuitive operation.

Between 2000 and 2007 – reduced cost in development process and intend to continue reduction of cost. One example if simplified physical prototype. 84% of workers in a canon survey thought that MR applications improve effectiveness. Some noted that HMD can be mounted for no more than 15 min (beyond that it creates motion sickness). Another issue was narrow angle view where both hands can not be viewed.

Showing demonstration video held in Tokyo last week [

get link]: simulates how to maintain a canon printer. Concept HMD used is VH2007 with higher resolution, including a video camera.

Canon intends to use CAD to simulate actual operation. Input motion parameters, display analytical simulation on top of that simulated operation. Showing concept video of a lens mockup, superimposing motion parameters of a real product. [get link]. It ‘s a promising concept, but there are still issues with resolution and picture quality…

He then goes to describe the MR platform marker technology, sensor use, calibration tools, etc.

Future work: offer a common platform for MR. Details still fuzzy…

Areas of future focus: navigation, construction, art…here is such an example:

Tracking the motion of an instrument (Clarinet) for a Media Art project : super imposed graphics change based on the sound and movement – very amusing!

Canon sees major value in MR and continue to develop the platform and HMD.


Next speaker:  Benjamin Becker (EADS) European Aeronautic Defense and Space Company

From the advanced design and visualization team working on: industrial design for aircrafts (e.g. Airbus), cabin interior, seatings, lower deck crew test, catering, lavatories. Also working on visualization for Helicopters, etc.

Main AR project: Trackframe using Ubitrack tracking framework.

AR combines multiple technologies: rendering and visualization, wearable computing, tracking. Caveats: HMD, interaction and usability, local and global tracking for large area.

Example: Sales and marketing project – present concepts of improved cabins (e.g. adding a bar) to customers and solicit feedback (add coloring, and in the future provide haptics). Spatial AR: Projecting daylight or night like on the cabin ceiling to help passengers adjust to jet lag [get video].

Explaining additional examples from maintenance, manufacturing, factory planning.

Question: Are we getting aesthetically pleasing view?

-it’s not photorealistic, but it’s better than seeing the options in a textual list…

[Unfortunately, I’ll have to miss the afternoon sessions in this track – due to the parallel Handheld mobile AR session which I can’t afford to miss…]


From ISMAR ’08 program.

Organizers: Selim Benhimane (TUM), Gudrun Klinker (TUM), Ralf Rabaetje (Volkswagen AG), Bruce H. Thomas (UniSA)

As the interest and the development of Augmented Reality (AR) is growing fast, it is important that, periodically, people from academia, research and industry sit together and discuss about what are the major limitations and results that were achieved recently. This workshop is the followup to the two successful one-day events that took place at ISMAR’05 in Vienna and at ISMAR’06 in Santa Barbara. The workshop will be split into four sessions of invited talks:

– Recent Advances in Tracking and Programming Frameworks for AR

– Requirements on AR Systems imposed by Industrial Applications

– Requirements on AR Systems imposed by Industrial Applications (continued)

– Recent Advances in Visualization and User Interfaces

There will be 9 speakers and each speaker will give 25-minute talk followed by a 5-minute questions and answers. An open discussion will take place at the end of the workshop in order to get the audience and the speakers discussing questions: What does Industry need from AR? What problems need to be solved for AR to work in Industry? What are the good target Industries for AR as it is seen in 2008?

Further information, including the full program and details of speakers, can be found on the Workshop website.