Meet the “Six Sense” Device: Augmented Reality MIT style

This week at TED, on the very stage where Bill Gates unleashed mosquitos into the audience to make a point about the need to cure Malaria, an MIT researcher, Patty Maes, unveiled a “six sense” device.

Maes demonstrated a portable device constructed out of commercial of the shelf products such as a web camera, pocket projector, and a cell phone.

What kind of “six sense” feats can it achieve?

Yahoo Tech captured the new ways to interact with the world made possible with this device:

  • turn any surface into a touch-screen for computing, controlled by simple hand gestures
  • take photographs by framing a scene with your hands
  • project a watch face by creating a circle on your wrist with your finger
  • recognize items on store shelves and provide personalized recommendations
  • look at an airplane ticket and know whether the flight is on time
  • project information about a book while browsing at a store
  • recognize articles in newspapers, retrieve the latest related stories or video from the Internet and play them on pages

Augmented reality enthusiasts would immediately recognize these fantastic ideas. Whether you use cell phones, goggles, or a projectors to view the added information – it’s a whole new way to interact with the world.

Now we have to wait patiently until TED uploads the video. ***update*** see videos below.

Last year cellphones took center stage in spearheading augmented reality into the main stream. Out of the blue comes this spatial augmented reality cobbled-together-device and takes the spot light.

Oliver Bimber is not surprised. He’s been leading that school of thought for a while and even wrote a book about it: Spatial Augmented Reality: Merging Real and Virtual Worlds

So what’s the total?

“Six sense” device: $300

Interacting with the world in a totally new way: priceless…

ETA: 2019


Andy Baio just tipped me off that Wired posted up these videos – thanks Andy!

Vodpod videos no longer available. Vodpod videos no longer available.

I Had A MID Night Dream

The US celebrated Martin Luther King’s day last week, which above all reminds us to keep dreaming – sometimes dreams do come true.

I had a dream too…and in my dream, an amazing Mobile Internet Device (MID) was released for our augmented reality experiences.

(See a list of existing MIDs)

my ar device

Here is a first take at defining the dream MID for augmented reality (2009-2010 time frame):

  • Manufacturer – a credible leader, with a friendly content distribution channel
  • Price – Ideally sub $200. Initially not more than $400.
  • CPU – Dual core 1.3 Mhz, with a Floating Point Unit, SIMD extensions
  • GPU – integrated with performance similar to TI’s OMAP3 and NVidia’s Tegra (the competition!)
  • Screen – 4.5 Inch, Min 800×480 resolution, Multitouch, and a very bright screen
  • Camera – A GOOD CAMERA with a quality lens, video recording at 320×240 or preferably 640×480 (VGA) at 30fps at a good quality (noise, contrast, colors, etc) even under low lighting. Zoom and auto focus a bonus. Front camera – bonus.
  • Low latency for getting the the camera image to the CPU/GPU and in turn to the display
  • Zero-latency video output from the device for a head-worn display (digital or analog)
  • Low-latency inputs for external sensors (such as a tracker on the head-worn display) and cameras (on the head-worn display).
  • GOOD graphics drivers, Open GL 2.0 (unlike the current Intel OpenGL drivers on Atom which are almost a show stopper for many projects…)
  • Device size – roughly 130x70x12mm (so that there’s little margin around the screen)
  • Weight – less than 200g
  • OS – The best Mobile Linux out there, with C/C++ based SDK and a good emulator. Also as an alternative: Win Mobile support (better dev tools)
  • Buttons – Very few. QWERTY keyboard is a nice to have.
  • Connectivity – 3G/GSM, WIFI, Bluetooth
  • Sensors – A-GPS, accelerometer, 3DOF Gyro sensors
  • 3-axis compass
  • Storage – 8G and expandable
  • Memory – 1G RAM
  • Battery – Min. 3 hours while in full use of camera and network
  • Extensibility – video out for an HMD, USB port on it.
  • Openness – open source…

So what do you think?

This spec was actually a swift response to a challenge presented by Intel’s Ashley McCorkle.

Many thanks for the contribution by Daniel-Good camera!-Wagner, Steven-don’t forget latency!-Feiner , Bruce-a couple of extras-Thomas, and Charles-Very bright screen-Woodward.

In ISMAR 2009 in Orlando, we are planning to organize a round table discussion for this very purpose. Would you be interested in participating?


The experts and enthusiasts are weighing in, and as it usually is in reality (as opposed to dreams) remind us that we need to consider trade-offs.

Charles for example says he would trade off battery time for a lighter device. He also suggests that for professional use – a higher price ($1000 range) for a higher quality device would be reasonable.

Ab Fab Lab in a Street Near You

While we are on the topic of touristic applications, labs, a travel site, has found a new way to tackle the problem of interfacing with an augmented reality world – for tourists. It is uncovered in an article on FastCompany.

The application Nru (pronounced Near You) uses  the GPS, compass, sensors and other goodies available on Google’s G1 phone. As you point it to different directions while strolling on a London street – Nru will display signals about your surrounding attractions; the usual suspects include restaurants, movies, shows.

Now, here comes the interesting part: the user interface. Hold it parallel to the ground and it displays a radar like view of your touristic targets. Hold it vertically and it transforms into a purple-black “heat” sensor highlighting worthy targets in your front.

To get more info about a selected target use the “old” gesture: just touch it.

They claim to pull information from a number of sources including Qype and fonefood. Not surprisingly – both are London focused information services.

Some will argue that since it doesn’t overlay (register) the signals on top of what’s in your field of view – it’s not a pure augmented reality implementation, but rather a location-based app.

But the British accent certainly masks that thought and adds a certain Je-ne-sais-quoi to the demo. Absolutely fabulous.

Nru is now available on the android marketplace – but only for UK customers. Top bollocks.

The One Eyed Man is King

In a land where what you see is what you get, those who can see more – even with one eye – are kings.

Kijin Shin from Yanko design believes in it and makes a point with this interesting concept design. He calls it the “Third Eye concept designed is for travelers.”

See something interesting? Just place the Third Eye up to your eyes like a monocle and the device pulls all relevant historical, travel, shopping, and tourist information.

Augmented reality lends itself well to touristic applications. When people explore new places – extra (augmented) information in context is highly sought after.

Multiple concepts focused on tourism have been thought of and implemented before: Museums apps, Wikitude, YDreams Sightseeing, Archeoguide, and many others.

What’s interesting in this one – is the form factor and the user interface.

Sometimes you have to reduce features (one eye only) to achieve simplicity. That has the potential to drive massive adoption.

If you are into the pros and cons, check out the interesting discussion on the site featuring the usual supporters vs. skeptics. One commenter compared it to the Celestron SkyScout:

We’ll see if Kijin’s design raises interest among hardware manufacturers. By then we’ll realize if the one eyed man becomes king – or whether he’ll be facing a land populated with 2 eyed specs.

So Many MIDs to Augment Your Reality at CES 2009

A quick look at all MIDs (Mobile Internet Devices) Intel is showcasing at CES 2009 (thanks to Truc and Warner from GottaBeMobile)

Vodpod videos no longer available.

Intel introduced the MID design concept only last year with just a handful of working devices. This year, they are definitely making a  big splash with too many to count.

Here’s Intel’s own Uday sneak-peaking a bag full of MIDs prior to the show

A notable MID, the Gigabyte m528, was released last month boasting great capabilities albeit a steep price ($750)

Stay tuned for Mid Moves – where four well-known technology bloggers will be taking 8 Intel-based Mobile Internet Devices on a fun and challenging 4-day tour starting on Jan 19th.

Now, which one will you use for your next augmented reality experience?

A Formidable Mobile Augmented Reality Device? Meet the HTC Touch HD

The new contender to become the mobile augmented reality device of choice is here: meet the HTC Touch HD.


PC World (Taiwan) hands on experience is pretty favorable.

With a far better screen (800×480 compared with iPhone’s 480×320) and a higher rez camera (5MP compared with iPhones 2MP) and its ability to record video (unlike the iPhone’s), it looks very promising.

The only big caveat is the high price tag ($776). It’s now available in Taiwan but poised to hit the EU and US by year end.

It will not have the screen size and the power of a MID – but it fits in a pocket, which is a plus for most of us.

As a result, the HTC Touch HD takes the place of the elusive Meizu M8 (still not out) on my top 5 mobile Devices for high-end AR.

Here is the updated round up.


Which one do you like best?

How to Get the Next Generation Hooked on Augmented Reality – Today: Part II

In my previous post “How to Get the Next Generation Hooked on Augmented Reality – Today” we explored the value of mobile educational games.

Most of these games were built for PDAs relying on a GPS, but did not include real time visuals of the real world (AR Tracking).

These PDAs are now obsolete.

In order to make it appealing for Kids, we’ll have to put in their hands something more trendy; iPhone, G1, Nokia N85, or a Mobile Internet Device (MID) come to mind.

But here’s the rub: will you give your toddler your precious smartphone? Your iPhone (God forbid) ?

What if electronics manufacturers raise to the occasion and create dedicated mobile devices for education?

Here are the Mobile Learning Devices already in progress:

One remarkable and noble project already in flight is project Inkwell.

The project’s ambitious goal is to create technology standards for the K-12 industry including defining the specifications for an Inkwell learning device. The design is by IDEO Spark.

It does not have a camera yet. But once mobile learning games break free – I am sure Inkwell will update its specs to include a camera.

Two other companies take a more commercially oriented approach (read: practical) introducing education oriented mobile devices (not yet AR enabled) such as 
VTech’s Create-A-Story

or LeapFrog’s Leapster and Didj.

These are less expensive devices that target a smaller niche.

Will these dedicated mobile learning devices be able to take a bite from the 800 pound gorillas in mobile gaming: Nintendo DS and Sony PSP ?

Will these devices drive the next generation’s Augmented Learning experience?

In my post about the new Nintendo DSi, I highlight the innovation and track record that has characterized Nintendo over the years. They will certainly fight the recent attempts in mobile learning devices with all their might.

Or will the iPhones and iClones of the world, with their massive adoption and cool factor, rule the mobile learning market after all?

What do you think?