Live From ISMAR 08: Augmented Reality Sensors and Sensor Fusion

Posted on September 18, 2008 by Ori Inbar

The last day of ISMAR ’08 is upon us, and the day opens by stimulating our senses with a session about sensors.

Gabriele Bleser starts this session with a talk about Using the marginalised particle filter for real-time visual-inertial sensor fusion

She starts by showing a short clip with an erratic camera motion that makes everyone dizzie…it actually proves an important capability that she studied which creates less jitter and less requirements imposed on the camera.

She explains the basics of particle filter and the use of inertial measurement.  In the past researchers studied standard particle filter. This is the first study using the a marginalised particle filter.

Testing using the new technique (non linear state space model with linear Gaussian substructure for real time visual inertial pose estimation) with 100 particles resulted in increased robustness against rapid motions.

To prove: Gabriele shows the rapid camera movements once again…

Well, we have to suffer now so that in the future users won’t have to. Kudos Gabriele.

~~~

Next is Daniel Pustka with Dynamic Gyroscope Fusion in Ubiquitous Tracking Environments. This is part of Gudrun Klinker’s journey towards Ubi-AR.

What you need for ubiquitous tracking is automatic discovery of tracking infrastructure, and shield applications from tracking details.

Gyroscopes are very interesting to use (low latency, high update rate, always available), but they have drawbacks (drift, only  for rotation) and are only usable when fused with other sensors.

Daniel and team have proved that the ubiquitous tracking tool set consisting of spatial relationship graphs and patterns is very useful to analyze tracking setups including gyroscopes. It allows a Ubitrack system to automatically infer occasions for gyroscope fusion in dynamically changing tracking situations.

~~~

Jeroen Hol presents Relative Pose Calibration of a Spherical Camera and an IMU

This study builds on the idea that by combining vision and inertial sensors  you get accurate real time position and orientation in a robust and fast motion, and this is very suitable for AR applications. However, calibration is the essential point for this to work.

An easy to use algorithm has been developed and yields results with real data.

Ron Azuma asks: When the image is captured in high motion does it create blur?

Jeroen answers that it can be addressed by changing some parameters.

~~~

Last for this session is Wee Teck Fong from NUS to discuss A Differential GPS Carrier Phase Technique for Precision Outdoor AR Tracking.

The solution that Fong presents provides good accuracy with low jitter, drift and low computational load – and no resolution ambiguities. It works well for outdoor AR apps. With just one GPS you get an accuracy of about 10 meters plus you get high jitter of the tracking. Differential GPS using 2 GPS receivers (low cost 25mm sized) improves the accuracy of tracking. Fong and team have taken it a steps further with an advanced computational model that delivers higher precision for outdoor AR tracking. Fong claims that with a more expensive receiver he can achieve a less than 1mm accuracy, but you can’t use this technique anywhere. An infrastructure of stationary GPS stations transmitting wirelessly could provide a wide constant coverage for this technique.

Fong concludes with a positive note regarding the upcoming European update to the GPS system dubbed Galileo (in 5 years) were things will get significantly better.

===============

From ISMAR ’08 Program

  • Using the marginalised particle filter for real-time visual-inertial sensor fusion
    Gabriele Bleser, Didier Stricker
  • Dynamic Gyroscope Fusion in Ubiquitous Tracking Environments
    Daniel Pustka, Gudrun Klinker
  • Relative Pose Calibration of a Spherical Camera and an IMU
    Jeroen Hol, Thomas Schoen, Fredrik Gustafsson
  • A Differential GPS Carrier Phase Technique for Precision Outdoor AR Tracking
    Wee Teck Fong, S. K. Ong, A. Y. C. Nee

Filed under: AR Engines, AR Events | Tagged: , , , , , , , , , , , , , | 1 Comment »

Live from ISMAR ’08: Tracking – Latest and Greatest in Augmented Reality

Posted on September 17, 2008 by Ori Inbar

After a quick liquid adjustment, and a coffee fix – we are back with the next session of ISMAR ’08, tackling a major topic in augmented reality: Tracking.

Youngmin Park is first on stage with Multiple 3D Object Tracking. His first demonstration is mind blowing. He shows an application that tracks multiple 3D objects, which have never been done before – and is quite essential for an AR application.

The approach combines the benefits of multiple approaches while avoiding their drawbacks:

  • Match input image against only a subset of keyframes
  • Track features lying on the visible objects over consecutive frames
  • Two sets of matches are combined to estimate the object 3d poses by propagating errors

Conclusion: Multiple objects are tracked in interactive frame rate and is not affected by the number of objects.

Don’t miss the demo.

~~~

Next two talks with Daniel Wagner from Graz university about his favorite topic Robust and Unobtrusive Marker Tracking on Mobile Phones.

Why AR on cell phones? there are more than a billion phones out there and everyone knows how to use them (which is unusual for new hardware).

A key argument, Daniel is making: Marker tracking and natural feature tracking are complementary. But we need a more robust tracking for phones, and create less obtrusive markers.

The goal: Less obtrusive markers. Here are 3 new marker designs:

The frame markers (the frame provides the marker while the inner area is used to present human readable information.

The split marker (somewhat inspired by Sony’s by the eye of judgment) we use barcode split, with a similar thinking to the frame marker.

A third marker is a Dot marker. It covers only 1% of the overall area (assuming it’s uniquely textured – such as a map).

Incremental tracking using optical flow:

These requirements are driven from industrial needs: “more beautiful markers” and of course making them more robust.

~~~

Daniel continues with the next discussion about Natural feature tracking on mobile phones.

Compared with marker tracking, natural feature tracking is less robust, more knowledge about the scene, more memory, better cameras, more computational load…

To make things worse, mobile phones have less memory, with less processing power (and no floating point computation), and a low camera resolution…

The result is that a high end cell phone runs x10 slower than a PC, and it’s not going to improve soon, because the battery power is limiting the advancement of this capabilities.

So what to do?

We looked at two approaches:

  • SIFT (one of the best object recognition engines – though slow) and –
  • Ferns (state of the art for fast pose tracking – but is very memory intensive)

So both approaches wont work for cell phones…

The solution: combine the best of both worlds into what they call: PhonySift (Modified SIFT for phones). And then complementing it with PhonyFern – detecting dominant orientation and predicting where the feature will be in the next frame.

Conclusion: both approaches did eventually work on mobile phones in an acceptable fashion. The combined strength made it work, and now both Fern and Sift work at similar speeds and memory usages.

================

From ISMAR ’08 Program:

  • Multiple 3D Object Tracking for Augmented Reality
    Youngmin Park, Vincent Lepetit, Woontack Woo
  • Robust and Unobtrusive Marker Tracking on Mobile Phones
    Daniel Wagner, Tobias Langlotz, Dieter Schmalstieg
  • Pose Tracking from Natural Features on Mobile Phones
    Daniel Wagner, Gerhard Reitmayr, Alessandro Mulloni, Tom Drummond, Dieter Schmalstieg

Filed under: AR Engines, AR Events | Tagged: , , , , , , , , , , , , | 1 Comment »