Tasbeeraman – An Augmented Reality Hand Movement Detection Game

No licensing issues here, really…

I applaud the robust hand tracking, but could the game be something new rather than a Pac Man rip-off?  Come on Total Immersion.  You guys are better than this.

Learn more on the Total Immersion blog.

Weekly Linkfest

Kind of slow week, but we still have some tasty links for you:

As for this week’s video we’ve got a little fun game from Total Immersion’s Partner C4M to promote French pasta brand Lustucru. Unlike previous TI games, this one is iPhone based, though there’s still a webcam version. It’s called Germaine et les Martiens:

Have a great week!

More Augmented Reality X-Ray Systems

Augmented reality will make superheros of us all.  A preview of what you can expect for ISMAR2010 from Magic Vision Lab.

Sandor, C., Cunningham, A., Dey, A., and Mattila, V.
An Augmented Reality X-Ray System based on Visual Saliency
To appear in: Proceedings of IEEE International Symposium on Mixed and Augmented Reality, Seoul, Korea, October, 2010.

In the past, several systems have been presented that enable users to view occluded points of interest using Augmented Reality X-ray visualizations. It is challenging to design a visualization that provides correct occlusions between occluder and occluded objects while maximizing legibility. We have previously published an Augmented Reality X-ray visualization that renders edges of the occluder region over the occluded region to facilitate correct occlusions while providing foreground context. While this approach is simple and works in a wide range of situations, it provides only minimal context of the occluder object.

In this paper, we present the background, design, and implementation of our novel visualization technique that aims at providing users with richer context of the occluder object. While our previous visualization only employed one salient feature (edges) to determine which parts of the occluder to display, our novel visualization technique is an initial attempt to explore the design space of employing multiple salient features for this task. The prototype presented in this paper employs three additional salient features: hue, luminosity, and motion.

We have conducted two evaluations with human participants to investigate the benefits and limitations of our prototype compared to our previous system. The first evaluation showed that although our novel visualization provides a richer context of the occluder object, it does not impede users to select objects in the occluded area; but, it also indicated problems in our prototype. In the second evaluation, we have investigated these problems through an online survey with systematically varied occluder and occluded scenes, focussing on the qualitative aspects of our visualizations. The results were encouraging, but pointed out that our novel visualization needs a higher level of adaptiveness.

Upcoming Qualcomm Developer Challenge

This fall, Qualcomm is going to host a $200,000 Augmented Reality Developer Challenge.  The details for the challenge haven’t been released yet, but the total amount of prize money should have would-be developers planning their products as we speak.  The challenge will start when they release the Qualcomm AR SDK.  More details can be found here.

ANTICIPATED TIMELINE

  • Fall 2010 – AR SDK available for download. Challenge commences.
  • Early November, 2010 – Submission site opens. Developers may submit apps.
  • January 7, 2011 – Submission site closes.
  • Mobile World Congress 2011 – Winners announced and showcased at Qualcomm events.

Weekly Linkfest

Flying drones, electronic musician and virtual girlfriends, all in this week’s linkfest:

For this week’s video we’ve got yet another futuristic vision, this time by German designer Björn Matthes. In his diploma project, Araproject, Matthes adds an energetic urban twist to augmented reality:

Have a beautiful week, happy Eid-ul-Fitr and Rosh Hashanah.

Samsung Galaxy Tab – When AR Becomes a Selling Point, We Won

Does this ad mark the transformation of augmented reality from being a term used by the tech-elite to one used by the commons?

Gravity Jack Indoor Office and SDK

Gravity Jack software firm has created an AR indoor office.  I like the idea of populating an indoor space with augments.  Personally, I wouldn’t use it for work.  I’d rather dork up my writing space at home and show off my favorite LOLcats on the wall next to my computer.  Each to his own, I guess.

Besides giving me an excuse to put a NASDAQ ticker over the latest Inspirational Poster, Gravity Jack is developing an SDK for their indoor system.

Features of the SiREAL World SDK in development now:

IPS – Indoor positioning system, using a patent pending method the phones running our software create a mesh that allows us to position the phone exactly in a Cartesian space indoors. It is almost like walking around with your phone acting like an RFID tag but with coordinates. We also use patented vision techniques to make your phone become the symbol thereby avoiding symbol recognition.

Instant Mapping – Utilizing a combination of scene recognition and real time edge detection we can build the real world scene into a 3d wireframe model on the fly. This allows for actually placing 3d objects in the scene and even behind real world objects because we have depth built into our model. AR products today just overlay things on the same front layer. Our technology allows us to place objects in the actual 3d plain.

GeoLocation and Proximity Alert – Allows a user to receive push notifications when they are near a SiREAL object and for storage of GeoTags for placing objects in the real world.

Asset management server – Our asset management server allows you to upload 3d assets or triggers, symbols associated with those assets and our object recognition will return the correct asset to your application.

Stickyness – While not a technical feature – what we have discovered is that by creating augmented reality applications and an engine that allows for true, non gimmick, life enhancing AR; users keep coming back! AR isn’t just a gimmick with us.

In summary, we fully support existing technology like symbol recognition, GPS tagging, but have new ways for AR to work indoors, in the dark, socially and we can create stickyness in our AR applications. Not only that, our tech understands what it is looking at and gives depth to scenes – something currently done with expensive dual camera setups.

A New Device for Augmented Reality Games Was Born

Finally, the iPod Touch has a camera!

Today, Steve Jobs announced at the Apple Music Event, the new incarnation of the legendary iPod product line.

One of my personal highlights was of course the new iteration of the iPod Touch. Faster (Apple A4 processors – like in the iPhone), with the new Game Center(iOS 4.1), the largest games market (1.5 billion games and apps have been downloaded to the Touch), and now – can do Augmented Reality.

Jobs bragged the iPod games outsells Nintendo DS and PSP games – combined!

There you have it – the perfect new platform for augmented reality games (as long as you have WIFI…)

Online Image Learning – The Next Big Leap in Mobile AR?

Mobile, image recognition based, augmented reality is very cool, as evident from the Popcode’s demos we posted yesterday. However, creation of a model used by the mobile phone to recognize a new image still requires a desktop, hindering realtime creation and sharing of AR content.

Thanks to the work of researchers from the Korean Gwangju Institute of Science and Technology and the Swiss EPFL, this needn’t be the case anymore. In a paper titled “Point-and-Shoot for Ubiquitous Tagging on Mobile Phones” accepted to ISMAR 2010, they present a method to scan surfaces and create “recognition-models” by using your phone (no data is sent to a remote server).

You don’t even need to take the perfect straight-on picture. As the video below shows, this means you can augment hard to reach surfaces. Best of all, you can share those models with your friends.

A little bit more detail over Wonwoo Lee’s blog.

3 Reasons Why Games Are The Killer App For Augmented Reality

We all are great believers in the potential of Augmented Reality (AR). Within the next 10 years, we say, it will totally change the way we interact with the world.

But what industry will be the first to bring AR to the masses?

When you co-found a company that focuses on Augmented Reality games – and especially – when it secures an investment from a venture capital firm – you have some deeper explaining to do.

Here are 3 key (business) reasons why games will be the first industry to spearhead augmented reality into the mainstream:

Reason #1 Games are the killer app for emerging technologies

Games Accelerate the Adoption of New Technologies by the Mainstream

Games have a history of pushing the envelop of new technologies and bringing new inventions to the masses: computer graphics, memory, silicon, screens, 3D animation, interactive story telling, immersion…and the list goes on and on. What made GPUs popular and drove its development? Games! First mainstream handheld electronic device? Games! When PCs were first introduced to consumers how were they marketed? As more capable than video game consoles…

Nolan Bushnell (the legendary founder of Atari) wrote back in 1996:

“WHIMSY and fun are often the precursors to powerful tools that are used later for more serious applications. A project at inception might not be useful because of lack of infrastructure or incomplete refinements. The automobile and airplane were considered toys before they were reliable and safe. The computer game has filled a similar role in being the incubator for many innovations that drive the usefulness of the computer.”

Was the first PC a video game console?

Gaming has done it many times in the past – now it could accelerate the adoption of Augmented Reality (AR) by the mainstream. And it has another advantage…

Game Design Can Overcome Technology Limitations And Deliver a Powerful Experience

In the best games, the pleasure manifests itself in the players imagination – not necessarily on the screen.
Skeptics will say AR is not ready for the mainstream yet: It’s not fast enough; it doesn’t work in bad lighting conditions; hardware has a long way to go – and you know what – they may be right. However, in games – clever game design can help overcome technical limitations and provide a fun experience. This luxury is not available for many types of real world apps, definitely not military or healthcare apps – which deal with life or death situations.

In games, you can focus the player on the highlights of the technology, and dodge the stumbling blocks. Or you can take advantage of a technical drawback and present it as a challenge of the gameplay  – which makes the game even more captivating. In the words of Raph Koster, a veteran game designer: “Creativity, is largely about finding solutions within a known problem space.”

For example: recognizing an object takes too long? build it into the suspense of the game. Tracking gets out of whack when the camera moves too fast? encourage players to keep it steady as a special skill required for succeeding in the game. Tell a good story; use non AR elements to create a more rounded experience; use AR as a highlight that takes the game to a never-before-seen level.

Game design is a strong reason why games will be first to drive AR into the hands of consumers.

But is there a market for these AR gaming experiences?

Reason #2 Games is a big business going through transformation

500 Million Digital Natives Will Spend 10,000 Hours Playing Games Before the Age of 21

A recent statistic (heard at TED by Jane McGonigal quoting a research by Carnegie Mellon University): “500 Million gamers Will Spend 10,000 Hours Playing Games Before the Age of 21” (and in a decade another billion gamers will be added.) That’s a lot of people, spending a lot of time with your product. Plus they are early adopters which makes them a likely audience to want to try your product.

AR devices are already in our pockets

What’s the barrier for entry? Can digital natives afford the hardware required to experience AR games?

With the invasion of the iphone, Android phones, and other smart phones into our pockets, all of a sudden, we carry with us at all times, the ingredients required for an augmented reality experience (nice screen, video camera, adequate processing power, communication, GPS, and other sensors). Many digital natives already have an AR capable device…in their pockets.

(31% of U.S. teens want iPhones And 14% already own one and Android phones are catching up.)

By the way, how many americans currently play on mobile devices? 42%!

OK, so many people are spending a lot of time playing games and have the hardware to play AR games, but what’s the revenue potential of the game industry?

Gaming is a $67 Billion Growth Industry Going Through Transformation

Gaming is huge. Bigger than music, bigger than movie box office. And yet, it is going through a major disruption.

On the one hand revenues by major publishers such as EA and Activision, are growing, but on the other hand they are struggling with profitability.

One of the key drivers is the massive transformation from retail distribution to digital distribution: the iphone, online games, and social games – are bypassing the old retail distribution model which charged $30-$60 per title and are reaching directly to consumers with a race to the bottom in terms of pricing.

Another major disruption is the rise of new game genres – most notably social games. Companies which as recent as 3 years ago were unknowns, are now being gobbled for huge sums:  $300M for Playfish (acquired by EA), $763M for Playdom (acquired by Disney), Slide (acquired by Google), Kongregate acquired by GameStop, Zynga is estimated to be worth $3B and can hardly be acquired…

Zynga's Farmville

Social games and casual games are reaching new audiences that previously were out of reach for game developers (survey shows that dominant age group playing social games such as Farmville is women 35-55).

Jesse Schell, instructor of entertainment technology at Carnegie Mellon University  says: “There are games now for pretty much every age, every demographic. More and more women are going online. It comes down to everybody is playing games. Games are just evolving like species in order to fit into every little niche of our lives.”

In parallel to the change in game genres and audiences there is also a massive change in business models. If up until recently games were selling for $30-$50 a pop and required a trip to the store which limited its potential audience – nowadays a new game is a click-to-download away and more and more games are available for free.

Which leads to the next point – a fresh business model that was made for games.

Free2Play + Micro-transactions is a Business Model  that Works Best in Games

web 2.0 has brought about the concept of Freemium business models which help reach massive audiences and drive light speed growth in revenue. Free2Play + micro-transactions is taking it further. Games are much more engaging than any other form of apps and drive the consumption of digital goods. Tying purchase to the compulsion loop in games has proven to be very effective. In the words of ngmoco’s Neil Young – it’s the new “Quarter Sink” from the arcade games era.

Of course, when using micro-transactions in games it’s important not to detract from the experience. Farmville by Zynga is a good example: impatience is the driving factor behind the appeal of micro transactions. And thus the experience is in no way impacted by those with patience and resourcefulness. As long as you avoid the potential pitfalls with micro transactions as articulated by Kevin Miller this business model is a fantastic revenue stream for games. Especially on the iphone where a 99 cent for a virtual weapon needed in the game is just a tap away. Worldwide sales of virtual items are expected to reach $7 billion by 2015, according to online games research firm DFC Intelligence.

Where else (except music) do you find such levels of revenue from digital goods?

Games Are a Key Tool For Marketers to Reach Their Audiences and Bolster Brands

Games are such a powerful medium for engaging audiences, that it is becoming a key tool for marketers to reach their customers and strengthen their brands. Take GE’s Smart Grid interactive campaign featuring augmented reality –  it got more than a million YouTube hits – an order of magnitude higher than equivalent traditional online campaigns. The Advergaming industry had revenues in excess of $3 billion on mobile, iPhone, and social networks platforms. And this is accelerating because digital natives expect to interact with the world through games. New target age groups and demographics are playing games which are more immersive than other types of advertising.

Games have just surpassed email as the #2 online time killer for americans. Guess what’s the first time killer? social networks. And what do people do on social networks? 40% of the time is spent playing games…

Adam Dole (a Design researcher) wrote in his paper : “Games engage consumers and build value around products and services, creating a powerful competitive advantage. The highest level of consumer engagement and brand loyalty comes from positive experience. By reflecting the way people want to act, a product or service more effectively changes attitude and behavior.”

Couldn’t have said it better.

So, games are big, and will get bigger. But how does it relate to AR Games which are played in reality?

Reason #3 Games are Getting Physical

For Digital Natives Gaming has become the primary metaphor for interacting with the world

The next generation (aka digital natives) has evolved to expect a very different way to interact with the world, and it’s largely influenced by games and the internet. They expect everything they interact with – to behave like games: provide challenge and reward loops, fellowship, discovery, narrative, expression. These are game pleasures (loosely taken from Marc LeBlanc’s list) that digital natives are used to – and they seek them in anything they do – in their real lives.

So if games escape computers and consoles and penetrate the real world, how will it impact our lives?

Games have the power to transform our real lives

Really?  Just checkout the Fun Theory Guys’ work. This team is hard at work proving that by injecting fun into daily routines you can change people’s behavior . When presented with multiple options (stairs or escalator) people are more inclined to chose the option that is more fun (playing piano while going up the stairs.)

Gabe Zichermann – which has been touting the funware theory says: “anything can become more fun if you bake games into it. The human brain is attracted to fun. Games can improve the outcome of every aspect in life. Points, rewards, immersion…”

In sales, it has already been used successfully: leaderboards, scoring and badges are a common aspect of sales people in many sectors.

Jane McGonigal takes it further in her Ted Talk and claims: “Games create urgent optimism and blissful productivity.” How could we harness this power to make the world a better place?

Jane McGonigal

Speaking of Jane, she has a track record of applying game mechanics to non game environments: she designed game features in ebay.com to reward sellers for high throughput combined with high satisfaction, by awarding them badged and literally unlocking levels.

Here are more examples listed by Adam Dole (a Design researcher) in his paper about Gaming for Behavior Change:

Mint.com, a financial management tool that leverages gaming principles to successfully capture and expand its market.

When Toyota began visualizing fuel consumption for drivers in their Prius models, they created a “fuel economy game.”

RecycleBank is a web-based service designed to promote recycling. Families accrue points based on the weekly amount of materials recycled; these points can be redeemed for discounts at over 1500 national businesses.

Byron and Leighton in their book: Total Engagement – show how to use games and virtual worlds to change the way people work and businesses compete.

But are websites the most optimal interface to “gamify”  our lives?

Why not insert these same mechanics into our field of view?  Into our visible reality? Into our daily routine?

Augmented Reality and games mechanics is a match made in heaven to reshape our real lives.

But here’s the caveat: when I first gave my 12 year old daughter an AR game to test – she asked: will I have to move…?

Are kids really interested in moving while playing? Will gamers be leaving the comfy couch and beloved controller in exchange for playing in the real world?

Games are Already Moving (Back) into The Real World

Digital natives were born into the digital revolution; they may be shocked by the following revelation: before video games – games were actually played in the real world. Since the dawn of man – playing games was how humans learned new skills. When civilization started 10,000 years ago people have already been playing board games and team games. In 1972 with the introduction of Pong – games have started a transition away from the real world and into the digital realm…

The latest trends in the gaming world (across consoles, social games, and location based games) point to games moving back into the real world: interfaces are becoming more intuitive – losing the traditional game controllers in favor of gestures. This has been aggressively persued in the game console segment: first by the phenomenal success of the Nintendo Wii which opened up new markets for gaming beyond hard core gamers. Sony Eyetoy and Sony Move are taking it a step further with a camera based system that tracks the movement of gamers. Microsoft  Kinect for Xbox (planned to be introduced this holiday season) will not require any controller whatsoever – the player’s body becomes the controller.

In social games, players interact with real life friends instead of imaginary avatars such as DragonSlayer85 which, in a way, is making the games more linked to the real world. In addition, Zynga, the leader in social games, expanded its super successful Farmville game to interact with real world objects. Starting this summer, consumers will be able to purchase specially marked products to receive a redemption code that can be used for a new, limited-edition virtual good in one of three Zynga games. For instance, you can buy a real Big Gulp from 7-Eleven and get a virtual version as well.

Location-based games are emerging and attracting massive audiences. Games such as MyTown which let you buy and own your favorite real-life locations, and collect rent when other people check-in to your shops – has amassed more than 2 million users in several months (more users than Foursquare has!), and has recently introduced Product Check-ins – registering interaction with an actual product by typing in a code.

Facebook, a phenomenal growth engine for social games, is also getting closer to the real world with the recently announced Facebook Places.

Clearly, there is an appetite for gamers to play in the real world. This transition from a made up virtual world that takes place on a screen – to the real world – is already happening. Augmented Reality has the power to accelerate this momentum and along the way introduce a new type of experience to massive audiences.

The Opportunity

Now here’s the big question: if games can advance AR forward, have a huge market potential, and have the power to impact our real lives for the better  – how do you explain the following fact:

The top 18 of 20 paid iphone apps of all time have been games and entertainment (2 are music and 16 are games) and yet, there are only a handful of (mediocre) AR games available on the app store?

How come?

The underlying reason is that good AR games are hard to make. Since the technology is admittedly in its early stages, developing AR games require in house expertise in AR, which are hard to find. Even if you do posses these rare expertise in house – it must be merged with game design skills to take advantage of the technology and create a fun and sustainable game experience.

Building successful AR Games relies on nurturing multiple disciplines and carefully fusing them together – with lots of passion. This has never been done – until now.

Fusion...

This gap between the attractiveness of games in general and the lack of AR games for sale – points to a huge opportunity.

Someone, somewhere is currently working on a killer Augmented Reality game that will completely change the way we look at games, and will catapult Augmented Reality to mainstream awareness. I believe it will be unveiled in the coming months.

Are you it?

Do you have what it takes to make AR Games that reach the mainstream?