Tomorrow, I’ll be at the IEEE VR 2010 conference in Boston. Monday is dedicated to a series of augmented reality presentations.
One of the most interesting one is:
In-Place Sketching for Content Authoring in Augmented Reality Games
By the all star team from Ben Gurion University (Israel) and HIT Lab (New Zealand):
- Nate Hagbi
- Raphaël Grasset
- Oriel Bergig
- Mark Billinghurst
- Jihad El-Sana
When it comes to AR games – we are all still searching for “Pong” a simple game that will captivate millions of players and kickoff this new genre.
One of the challenges in many AR games, is the reliance on printouts of ugly markers.
Plus many games use the markers as controllers which is a bit awkward (especially to a bystander).
Sketching offers an alternative for a more natural user interface.
Sketching is more natural than drawing with a mouse on a PC, even more intuitive that a touch screen. That’s still the first thing that kids are taught in school.
It’s not necessarily a better interface – but it’s an alternative that offers a very intuitive interaction, and enriched the player’s experience. I believe it could create a whole new genre of games.
In place sketching in AR games has huge potential in gaming – but many questions arise:
- What’s the design space for such a game?
- What are the tools to be used?
- How do you understand what the player meant in a sketch?
- What’s the flow of interaction?
- How do you track it?
What’s “In-place AR”? It’s when the augmented content is extracted from the real world (an illustration, an image , a sketch, or a real life object)
Here is the sequence of research efforts leading to this:
- Extraction and augmentation – A new way for storing and distributing augmented reality content.(ISMAR 2008)
- Hand-sketched physical experiments – using isometric drawings and a physics engines (ISMAR 2009)
- 3D registration using natural shapes – best paper award at ISMAR 2009
Here are 2 game prototypes the team created called AR Gardener and Sketch-Chaser. It is played on a regular white board.
Draw symbols on the white board and 3D content is pulled from a database of objects to appeas in an Augmented Reality (AR) scene.
The sketch determines what object to create, its location, scale, and rotation.
The outer line sketched here defines the game anchor and is served for tracking; in this game it becomes a brown ridge.
Simple symbols drawn generate a couple of benches, a cabin, and in the spirit of the playground theme – rockers, and swings.
Virtual elements could also be created based on a real life object such as a leaf; here it is used to create a patch of grass using the color and shape of the leaf (and no, it can’t recognize that’s a leaf, or 3D object whatsoever)
The color of the marker could define the type of virtual object created: For example, blue represents water. Other objects that are put in it will sink.
In the second game you basically create an obstacle course for a car chase.
It’s a “catch the flag” or tag game. The winner is whoever has the flag for the most time.
First you draw, then play.
Once again, the continuous brown line represents a ridge and bounds the game.
A small circle with a dot in it represents the starting point for the cars.
A flag becomes the flag to capture. A simple square creates a building, etc.
The player adds more ridges to make it more challenging. Adds blue to generate a little pond (which also indicates a different physical trait to this area)
Then – graphics are generated, the players grab their beloved controllers and the battle begins!
This research represents an opportunity for a whole new kind of game experience that could make kids play more in the real world.
Many questions still remain, such as how do you recognize in a sketch what the player really means without requiring her to be an artist or an architect. Or where does the sketch fit in the game play? Before, after or during?
Now, it’s up to game designers to figure out what sketching techniques work best, what’s fun, what’s interesting, and what’s just a doodle.
Who want’s to design an sketch-based augmented reality a game?
ISMAR 2009 the world’s best augmented reality event starts in 3 days!
If you are still contemplating whether to go – check out what you might be missing on our preview post.
The folks from the Visual Media Lab at Ben Gurion University in collaboration with HIT Lab NZ are preparing a real treat for ISMAR 2009 participants.
Sketch recognition (already covered in our previous post) is a major break away from “ugly” markers or NFT (tracking natural 2d images). It is the dawn of user generated content for Augmented Reality, and an intuitive new interaction approach for changing the CONTENT overlaid on a marker. Big wow.
In-Place 3D Sketching
But the team lead by Nate Hagbi and Oriel Bergig (with support from Jihad El-Sana and Mark Billinghurst) is just warming up…In the next video Nate shows how any sketch you draw on a paper (or even on your hand!) can be tracked.
So are you telling me I won’t need to print stuff every time I want to play with augmented reality?
-That’s right! Hug a tree and save some ink!
Shape Recognition and Pose Estimation
But wait, there is more!
Nate says this demo already runs on an iPhone.
And to prove it, he is willing to share the code used to access the live video on iPhone 3.0.
(note: this code accesses a private API on the iPhone SDK)
Ready for the BIG NEWS?
For the first time ever, the core code necessary for real augmented reality “(real” here means precise alignment of graphics overlaid on real life objects) on iPhone 3.0 is available to the public.
To get access to the source code – send us an email.
May a thousand augmented reality apps bloom!
Special Message from Mark Billinghurst: Introducing FLARManager – Can Building AR Apps in Flash Be Easier?
July 1st 2009 Press Release
ARToolworks Releases Commercial License for FLARManager
ARToolworks is very pleased to announce that it is able to offer commercial licenses for the popular FLARManager software. FLARManager is a software framework developed by Eric Socolofsky that makes building FLARToolKit Flash based Augmented Reality applications easier.
FLARManager decouples the marker-tracking functionality from Papervision3D, and provides a more robust event-based system for managing marker addition, update, and removal. It supports detection and management of multiple patterns, and multiple markers of a given pattern.
Most importantly, FLARManager sits on top of FLARToolKit and makes it much faster and easier to develop flash based AR applications, typically half the time or less of developing a straight FLARToolKit application.
Philip Lamb, CTO of ARToolworks, says “We are delighted to be able to provide commercial license for this outstanding tool. This will enable FLARToolKit developers to build Flash AR applications quicker than ever before, and is the perfect compliment to our existing product line.”
FLARManager will continue to be freely available under a GPL license from http://www.facebook.com/l/;http://transmote.com/flar/, but ARToolworks has the exclusive rights to sell commercial licenses to those companies that do not want to share the source code of their applications as required by the GPL license.
The developer of FLARManager, Eric Socolofsky, says, “I’m excited to be able to offer FLARManager to both the commercial and experimental community. FLARManager began as an effort to bring FLARToolkit to a wider audience, and this commercial license will help to expand the reach of augmented reality and new interfaces to the web.”
For a limited time, ARToolworks is selling FLARManager for a reduced price of only $295 USD for a single product license, and also selling a discounted bundle of FLARToolKit and FLARManager licenses together. FLARToolKit is required to use FLARManager.
Please contact firstname.lastname@example.org for more details.
…in 10-15 years everyone will use Augmented Reality to experience the world in a more meaningful way.
Our collective mission:
…nurture a healthy industry that will drive the adoption sooner than later.
So where do we start?
…by educating the youngest “digital natives”.
That generation is ripe and eager to try new experiences that speak their language. And that same generation will carry the AR movement to its glory.
The challenge is how to give them something they like, and at the same time offer value to those who hold the buying power – their parents, guardians, or teachers.
Tech savvy parents and teachers tend to recognize the value of PCs and video games in educating their kids – but they hate the isolation resulting in too many hours in front of the screen.
These games use social dynamics and real world contexts to enhance game play…and can create compelling educational and engaging environments for learners…help develop 21 century skills…tackle complex problems…and acquire information in just-in-time fashion”
Eric doesn’t stop at arguing, he actually does what he preaches. Together with colleagues at MIT Teacher education program & the Education Arcade and in collaboration with Madison-Wisconsin and Harvard, they developed multiple mobile games (see below) – and experimented and improved them – with kids.
And they’re not alone. Researchers around the world have studied this huge opportunity and wrote about it extensively.
Future Lab in the UK is passionate about transforming the way people learn, and develop new approaches to learning for the 21st century (see games below).
Mike Adams ranted in 2004 about the prospects and dangers of augmented reality games in his passionate The Top Ten Technologies: #3 Augmented Reality
Cathy Cavanaugh wrote the essay “Augmented Reality Gaming in Education for Engaged Learning” as the fifth chapter of a massive hand book dubbed Effective Electronic Gaming in Education. (You can get it for $695.00 at Information Science Reference.)
Cavanaugh explores a (surprisingly large) number of educational games developed in the last 4 years:
Most were designed to teach concepts in scientiﬁc systems, and the remaining AR games focus on the difﬁcult-to-master, ill- deﬁned domains of communication, managing data collected in the ﬁeld, problem solving, and understanding cultural and historic foundations of a region.
Based on that list, here is an (alphabetical) culmination of mobile educational games in recent history:
Big Fish Little Fish (MIT)
Concepts including predator-prey dynamics, over ﬁshing, biodiversity, evolution for school-age children.
Groups of students use handheld devices while physically interacting with each other to simulate ﬁsh feeding behavior.
Charles River City (MIT)
Outdoor GPS-based Augmented Reality game for teenagers. Players team up as experts including scientists, public health experts, and environmental specialists to analyze and solve an outbreak of illness coinciding with a major event in the Boston Metro Area.
Create-a-Scape (Future Lab)
Mediascapes are a powerful way of engaging with the world around us. Using PDAs they offer new opportunities to explore and interact with the landscape in exciting and varied ways.
Eduventure Middle Rhine (Institute for Knowledge Media)
Learning the cultural history of the Middle Rhine Valley for adults. Learners alternate between problem solving using video of the castle setting and problem exploration using mobile devices in the real castle.
Environmental Detectives (MIT)
Collaborative understanding of scientiﬁc and social aspects of threats to the environment and public health for adults. Participants role-play as teams of scientists investigating contaminated water using networked handheld devices in a ﬁeld setting.
Epidemic Menace (Fraunhofer Institute)
Collaborative problem solving and experiences with learning arts for adults. Teams assume the roles of medical experts to battle a threatening virus using gaming and communication devices in a room and outdoors.
Support for ﬁeld-based learning of children ages 9-11. Groups of children respond to scenarios in the ﬁeld using a portable data collection and communication device.
Live Long and Prosper (MIT)
Concepts including genetics and experimental design for school-age children. Groups of students use handheld devices while physically interacting with each other to simulate the genetic actions of reproduction.
Mobi Mission (Future Lab)
Communication and reﬂection activities for teenagers.
Groups of students write verbal missions and respond to the missions of others using cell phones.
Mystery @ the Museum (MIT)
Collaborative thinking skills for adults and youngsters. Teams consisting of a Biologist, a Technologist and a Detective must work together to solve a crime at the Museum of Science.
Newtoon (Future Lab)
Physics principles for adolescents. Students use mobile phones and Web sites to play, create, and share games that demonstrate physics principles.
Outbreak @ MIT (MIT)
Experience with the complexities of responding to an avian ﬂu outbreak, for young adults.
Players are brought in to investigate a potential epidemic on campus with hand-held networked Pocket PCs.
Savannah (Future Lab)
The science of living things interacting within an ecosystem, for ages 11-12. Children, acting as lions, navigate the savannah using mobile handheld devices.
Sugar and Spice (MIT)
Concepts including population economics and mathematics for school-age children. Groups of students use handheld devices while physically interacting with each other to simulate interactions between populations and resources
Concepts including epidemics, scientiﬁc method, population growth for school-age children. Groups of students use handheld devices while physically interacting with each other to simulate the spread of disease
So what’s next?
These old games have built-in educational value, they strive to be more fun than traditional classroom lessons, and most importantly – they achieve it while detaching children from the screen.
However, none of these games has really made it to the mass market.
In order to break into the mainstream, games will have to be
- more visual (see what you mean),
- more intuitive (touchscreen and accelerometers – drop the Pocket PC look & feel),
- more ubiquitous (play anywhere, anytime),
- and they will have to run on devices that look more like an iPhone than a Newton.
Devices for education is in fact the main topic for the second part of this post.
Stay tuned. Or better yet – tell us what you think.
Filed under: AR Education, AR Games | Tagged: Augmented Reality Gaming in Education for Engaged Learn, Big Fish Little Fish, Cathy Cavanaugh, Charles River City, Create-a-Scape, duventure Middle Rhine, Education Arcade, Effective Electronic Gaming in Education, Environmental Detectives, Epidemic Menace, Eric Klopfer, Fraunhofer Institute, Future Lab, HandLeR, HARP, Institute for Knowledge Media, Live Long and Prosper, Mark Billinghurst, Mike Adams, Mobi Mission, Mystery @ the Museum, Newtoon, Outbreak @ MIT, Savannah, Sugar and Spice, University of Birmingham, Virus | 8 Comments »