Metaio Offers Flash Forward for Lego Buyers

From Metaio‘s press release”

LEGO, the Danish toy manufacturer will test launch its “DIGITAL BOX” in selected toyshops and LEGO® stores worldwide. This interactive  terminal will utilize innovative technology supplied by Metaio in the form of a software  program specially-developed for the LEGO Group by the Munich-based experts in augmented reality solutions. Together with a camera and display screen, the software  lets LEGO packaging reveal its contents fully-assembled within live 3D animated scenes.

lego_digital_box_showcase

The press release continues:

The partnership between Metaio and one of the largest toy manufacturers in the world is a truly  major milestone in the history of the company.

Indeed, Lego is associated with playful, innovative toys and will certainly expose the concept of augmented reality to many kids around the world.

Kudos to Metaio for a great splash at the onset of the AR year.

Now who’s going to post the first video of this experience?

Would You Kill for These Android Augmented Reality Apps?

Tom Spring from PC World put together a nice collection of Android apps for T-Mobile’s G1.

The lion’s share of these apps is compelling thanks to their novel use of camera, GPS, communication capabilities – or all combined.

Tom listed 15 Android “killer apps”. I pick 4 apps which could inspire the augmented reality world.

You will judge if they “kill”, Bill.

1. BreadCrumbz

First step towards user generated Geo tags

BreadCrumbz is a different kind of navigation application.

Navigate-by-Pictures: Navigate your route using pictures instead of a map (there’s also a map, if you like).

Users Create Routes: Easily record routes using your smartphone. Share them with your friends, share them with the world.

2. CitySlikkers

Attempting a real-world social network

City Slikkers is a Pervasive Game (alternatively Location Based Game) which takes place in the real-existing city. It is designed to connect a large number of players through-out the world and change the way the surroundings are seen. The central idea behind the concept is to give people the opportunity to symbolically interfere with the everyday urban environment and come into contact with previously unknown people.

3. ShopSavvy

Using the oldest Marker technology: barcode

ShopSavvy™ is a shopping assistant developed exclusively for Google’s Android mobile phone platform and is one of T-Mobile’s featured applications in their 2008 US and UK launch. Users can scan the bar code of any product using their phone’s built-in camera. ShopSavvy will then search for the best prices online and through the inventories of nearby, local stores using the phone’s built-in GPS. ShopSavvy won Google’s Android Developer Challenge and is available in Google’s Android Market.

4. Get a Life

Getting folks out of the house is a good thing

Locate your friends and family– LifeAware gives you the ability to create your own network of friends and family members and locate them. Unlike other location services, LifeAware will provide you with their last known location, providing insight even if their phone is off or out of range.

Establish safety zones for family members – Create geographical zones and setup notifications for when a family member enters or leaves the defined zone. Setup a zone to be notified when Johnny arrives or leaves school, or when a loved one arrives at their destination when taking a trip.

Tag locations for yourself or for sharing– Tag locations you visit, or find on the map and share with the members of your network.

Send locations– Send your current location or location of your choosing to friends in your network, or to any email address. Send the favorite meeting place to friend and have them meet you.

Now, you be the judge!

Live from ISMAR ’08: Is Augmented Reality at Work Better than Reality Itself ?

Bruce Thomas introduces the afternoon session at ISMAR ’08 focusing on user studies in industrial augmented reality.

First is Johannes Tuemler which will talk about Mobile Augmented Reality in Industrial Applications: Approaches for Solution of User-Related Issues.

The study looks at psychological and ergonomic factors in augmented reality usage and create a requirements catalog for mobile AR assistance systems in diverse scenarios. This was a collaboration with Volkswagen, Ergonomics department in Ott-von-Wolfsburg,  Perception Psychology from Weymar University, and Information technology by the Fraunhofer Institute.

The reference scenario chosen was “AR picking”, where subjects would work for a couple of hours of picking items from shelves using a mobile AR device. The users reported no rise of stress level with an AR system compared with no AR (except for some visual discomfort). Since the AR system was less than optimal, the research may point to the fact that with a better AR system the stress level of workers – compared with no AR system – could be reduced!

~~~

As a direct follow up to the first study, Bjoern Schwerdtfeger comes on stage to describe the results of an Order Picking with AR work.

Traditionally the system includes a print out with instructions of what items to pick from bins on shelving.

How can an AR system help improve the performance of such an activity?

Glasstron by Nomad

They looked at mulitple visualization options: Frame tunnel, Rings tunnel, and 3D Arrow.

The results showed that the frame visualization was more efficient than the arrow. It’s not clear whether the rings visualization is superior.

~~~

Final speaker for this session is Gerhard Schall from Graz University to discuss Virtual Redlining for Civil Engineering in Real Environments.

What is virtual redlining? Virtually annotation paper maps or 2d digital information systems (mostly for the utility sector). This process helps significantly in the workflows associated with network planning or inspection.

The process involved mapping of 2D geographical data with 3D models of buildings and underground infrastructure. The tool developed allows for collaboration, inspection, and annotation.

Results of the usage study confirms that the AR system has significant advantage in civil engineering – in this redlining scenario. The color coding was important, as well as the digital terrain model.

Question from the audience: where do you get the 3D modeling of the piping?

Answer: Some utility companies have started to map the underground infrastructure. But in most cases we create it based on 2D maps which is only an approximation.

And that concludes the Industrial user studies session. See you next at the last session of the event: Rendering and Scene Acquisition, leading to the grand finale with the award ceremony for the winner of the Tracking Competition.

=============

From ISMAR Program:

User studies in Industrial AR

  • Mobile Augmented Reality in Industrial Applications: Approaches for Solution of User-Related Issues
    Johannes Tuemler, Ruediger Mecke, Michael Schenk, Anke Huckauf, Fabian Doil, Georg Paul, Eberhard A. Pfister, Irina Boeckelmann, Anja Roggentin
  • Supporting Order Picking with AR
    Bjoern Schwerdtfeger, Gudrun Klinker
  • Virtual Redlining for Civil Engineering in Real Environments
    Gerhard Schall, Erick Mendez, Dieter Schmalstieg

Live from ISMAR ’08: Latest and Greatest in Augmented Reality Applications

It’s getting late in the second day of ISMAR ’08 and things are heating up…the current session is about my favorite topic: Augmented Reality applications.

Unfortunately, I missed the first talk (had a brilliant interview with Mark Bullinghurst) by Raphael Grasset about the Design of a Mixed-Reality Book: Is It Still a Real Book?

I will do my best to catch up.

Next, Tsutomu Miyashita and Peter Meier (Metaio) are on stage to present an exciting project that games alfresco covered in our Museum roundup: An Augmented Reality Museum Guide a result of a partnership between Louvre-DNP Museum lab and Metaio.

Miyashita introduces the project and describes the two main principles of this application are Works appreciation and guidance.

Peter describes the technology requirements:

  • guide the user through the exhibition and provide added value to the exhibitions
  • integrate with an audio guide service
  • no markers or large area trackin – only optical and mobile trackers

Technology used was Metaio’s Unifeye SDK, with a special program developed for the museum guide. Additional standard tools (such as Maia) were used for the modeling. All the 3d models were loaded on the mobile device. The location recognition was performed based on the approach introduced by Reitmayr and Drummond: Robust model based outdoor augmented reality (ISMAR 2006)

600 people experienced the “work appreciation” and 300 people the guidance application.

The visitors responses ranged from “what’s going on?” to “this is amazing!”.

In web terms, the AR application created a higher level of “stickiness”. Users came back to see the art work and many took pictures of the exhibits. The computer graphics definitely captured the attention of users. It especially appealed to young visitors.

The guidance application got high marks : ” I knew where I had to go”, but on the flip side, the device was too heavy…

In conclusion, in this broad exposure of augmented reality to a wide audience, the reaction was mostly positive. it was a “good” surprise from the new experience. Because this technology is so new to visitors, there is a need to keep making it more and more intuitive.

~~~

Third and last for this session is John Quarles discussing A Mixed Reality System for Enabling Collocated After Action Review (AAMVID)

Augmented reality is a great too for Training.

Case in point: Anesthesia education – keeping the patient asleep through anesthetic substance.

How cold we use AR to help educate the students on this task?

After action review is used in the military for ages: discussing after performing a task what happened? how did I do? what can I do better?

AR can provide two functions: review a fault test + provide directed instruction repetition.

With playback controls on a magic lens, the student can review her own actions, see the expert actions in the same situation, while viewing extra information about how the machine works (e.g. flow of liquids in tubes) – which is essentially real time abstract simulation of the machine.

The result of a study with testers showed that users prefer Expert Tutorial Mode which collocates expert log with realtime interaction.

Educators, on the other hand, can Identify trends in the class and modify the course accordingly.
Using “Gaze mapping” the educator can see where many students are pointing their magic lens and unearth an issue that requires a different teaching method. In addition, educators can see statistics of student interactions.

Did students prefer the “magic lens” or a desktop?

Desktop was good for personal review (afterward) which the Magic lens was better for external review.

The conclusion is that an after action review using AR works. Plus it’s a novel assessment tool for educators.

And the punch line: John Quarles would have killed to have such an After action review to help him practice for this talk…:-)

=====================

From ISMAR ’08 Program:

Applications

  • Design of a Mixed-Reality Book: Is It Still a Real Book?
    Raphael Grasset, Andreas Duenser, Mark Billinghurst
  • An Augmented Reality Museum Guide
    Tsutomu Miyashita, Peter Georg Meier, Tomoya Tachikawa, Stephanie Orlic, Tobias Eble, Volker Scholz, Andreas Gapel, Oliver Gerl, Stanimir Arnaudov, Sebastian Lieberknecht
  • A Mixed Reality System for Enabling Collocated After Action Review
    John Quarles, Samsun Lampotang, Ira Fischler, Paul Fishwick, Benjamin Lok

ISMAR ’08 Live: Workshop on Industrial Augmented Reality: Needs and Solutions


Welcome to the first workshop of ISMAR 2008.

We are starting with the Industrial AR workshop.

Selim Benhimane introduces ISMAR Chair Ralf Rabaetje which introduces the first speaker Dr. Werner Schreiber from Volkswagen AG.

Ralf describes the main reason for VW to research in augmented reality: “we need to find new and better ways to develop, test and produce cars. And we need to make the process less expensive.”

VW is doing it as part of a government funded project dubbed AVILUS, in collaboration with major EU companies such as Airbus, Daimler and Siemens.

One of the improvements that can be achieved with AR is improved safety.

Werner shows various technologies that are being worked on, and slides of applications for improvement in designing and building cars. Example: applying labels for air bags in the language of the car’s destination. The error rate of the previous approach (using written lists) was improved dramatically with an AR system (with an HMD). Metaio provided elements of this solution.

Werner concludes with general requirements for these type of AR systems:

  • Keep it simple
  • Intuitive without special technology know how needs
  • Standard system
  • Universal system
  • Multi use in various industrial processes
  • Less than 30 min prep time
  • Economic
Question: How did workers react to these solutions?
– Some were skeptics, other were enthusiasts…you have to find tricks to make it easy to adapt to.
Q: Are you willing to take the risk of significanlty changing the process to include AR? Why not a sound system or monitors?
– adding the information in the field of view of the worker and reducing the cost – was worth it.
~~~
Second presenter is being introduced: Dr. Axel Hildebrand from Daimler heading an AR project; a perfect continuation of the previous talk. It will focus on how to deal with the maturity gap between needs and the current technology. Axel was formerly working on AR in the Fraunhofer institute.
We recognize that technology has to go through multiple stages until it’s ready for use in industrial systems, but we also know we have to start with such technologies early – to help them mature…we need to take some risk.
From Gartner’s hype cycle: in 2006 AR was a “technology trigger”. It was conspicuously missing in 2007 and then reappeared in 2008 – yet again as “technology trigger”.
At Daimler, the technology building blocks are: Data access, Interaction, displaer, visualization, tracking.
Example applications: Mobile picking objects. (collaborated with Metaio) will start a prototype with AR at the engine assembly line.
Current mobile devices are text driven usd for quality assurance.
With Symbian based devices added visuals in context to enrich the information workers have during picking objects.
Mobile Quality Assurance – a concept of using camera to visually test quality of products being produced.
Mixed Reality Ergonomics Situation – Use AR to improve posture of workers on assembly line. Using a mockup that simulates a car to test the posture of workers during certain tasks and then improving the procedures to improve the ergonomics.
Spatial AR for Automotive Design – projecting various scenarios (e.g. colors) on car models during design process.
Factory Shop floor approval – mobile AR device superimposes data from various sources about the shop floor to check the environment.
[skiping videos – what a shame…]
Thermal protection of the Overall Vehicle – superimposedata from simulations to that workers can readjust the engine [finally showing a video explaining the technique!]
Mixed reality Assembly

Axel summarizes: Need to havea step-wise approach. Convincing the business side, with high value projects – and then going further and applying to more projects.

Technologies still immature: indoor tracking, full HMD usage, AR visualization, Interaction. And we are still before the trough of disillusionment…

Some applications will require HMDs when the activity requires to be hands free. Others mobile devices are fine. And in other cases will need spatial AR.

[Coffee break]

Gudrun Klinker introduces Shinichi the next speaker Shinichi Aratani. He will talk about the current state of industrial MR/AR at Canon.

[colorful Japanese slides] Canon started working on MR in 1997 and focuses on 4 areas: Industry, Presentation, Art and Entertainment. Interior simulation of living room, Media art, etc. After 2001 moved to industrial use such as design evaluation, digital mockups, usability testing, etc. In order to achieve  value in MR applications  Canon assumed these requirements: real scale, intuitive visualization, intuitive operation.

Between 2000 and 2007 – reduced cost in development process and intend to continue reduction of cost. One example if simplified physical prototype. 84% of workers in a canon survey thought that MR applications improve effectiveness. Some noted that HMD can be mounted for no more than 15 min (beyond that it creates motion sickness). Another issue was narrow angle view where both hands can not be viewed.

Showing demonstration video held in Tokyo last week [

get link]: simulates how to maintain a canon printer. Concept HMD used is VH2007 with higher resolution, including a video camera.

Canon intends to use CAD to simulate actual operation. Input motion parameters, display analytical simulation on top of that simulated operation. Showing concept video of a lens mockup, superimposing motion parameters of a real product. [get link]. It ‘s a promising concept, but there are still issues with resolution and picture quality…

He then goes to describe the MR platform marker technology, sensor use, calibration tools, etc.

Future work: offer a common platform for MR. Details still fuzzy…

Areas of future focus: navigation, construction, art…here is such an example:

Tracking the motion of an instrument (Clarinet) for a Media Art project : super imposed graphics change based on the sound and movement – very amusing!

Canon sees major value in MR and continue to develop the platform and HMD.

~~~

Next speaker:  Benjamin Becker (EADS) European Aeronautic Defense and Space Company

From the advanced design and visualization team working on: industrial design for aircrafts (e.g. Airbus), cabin interior, seatings, lower deck crew test, catering, lavatories. Also working on visualization for Helicopters, etc.

Main AR project: Trackframe using Ubitrack tracking framework.

AR combines multiple technologies: rendering and visualization, wearable computing, tracking. Caveats: HMD, interaction and usability, local and global tracking for large area.

Example: Sales and marketing project – present concepts of improved cabins (e.g. adding a bar) to customers and solicit feedback (add coloring, and in the future provide haptics). Spatial AR: Projecting daylight or night like on the cabin ceiling to help passengers adjust to jet lag [get video].

Explaining additional examples from maintenance, manufacturing, factory planning.

Question: Are we getting aesthetically pleasing view?

-it’s not photorealistic, but it’s better than seeing the options in a textual list…

[Unfortunately, I’ll have to miss the afternoon sessions in this track – due to the parallel Handheld mobile AR session which I can’t afford to miss…]

========================

From ISMAR ’08 program.

Organizers: Selim Benhimane (TUM), Gudrun Klinker (TUM), Ralf Rabaetje (Volkswagen AG), Bruce H. Thomas (UniSA)

As the interest and the development of Augmented Reality (AR) is growing fast, it is important that, periodically, people from academia, research and industry sit together and discuss about what are the major limitations and results that were achieved recently. This workshop is the followup to the two successful one-day events that took place at ISMAR’05 in Vienna and at ISMAR’06 in Santa Barbara. The workshop will be split into four sessions of invited talks:

– Recent Advances in Tracking and Programming Frameworks for AR

– Requirements on AR Systems imposed by Industrial Applications

– Requirements on AR Systems imposed by Industrial Applications (continued)

– Recent Advances in Visualization and User Interfaces

There will be 9 speakers and each speaker will give 25-minute talk followed by a 5-minute questions and answers. An open discussion will take place at the end of the workshop in order to get the audience and the speakers discussing questions: What does Industry need from AR? What problems need to be solved for AR to work in Industry? What are the good target Industries for AR as it is seen in 2008?

Further information, including the full program and details of speakers, can be found on the Workshop website.