Glass Explore and Explorer


Google I/O kicked off today with not much fanfare around Glass. From a pure awareness stand point, Glass is the best thing that happened to Augmented Reality since the iPhone. And as a champion of the Augmented Reality industry from way back in 2007 – I am an avid supporter.

But Glass Explorers* make me angry (*users of the Google Glass prototype.)

I am not angry at Explorers because they love to walk on the street with Glass so that passerbys stop and ask them about it (although passerbys just want to take selfies with Glass)

Glass Selfie2

I am not angry at Explorers because they love getting into bars just to be denied service. Nor am I angry because they drive cars with Glass just to annoy highway patrol officers.

And you know what, I am not even angry at their eagerness to pay an exuberant amount of money to be testers in the most expensive beta program ever.

All that doesn’t bother me so much.

As Jon Stewart says : Intolerance shouldn’t be tolerated.

You know why I am angry at Glass Explorers?

Because they totally mistake the purpose of wearing.

In the Daily Show’s “Glass Half Empty” segment a Glass Explorer explains: “it’s basically a cell phone on your face.”


Daily show wearing Glass

“Make calls, get email, surf the internet…accessibility to everything on your cell phone” but now “right there on your eye”.

This is a bad case of skeuomorphism. Arrrgh!


Skeuomorphism: a dial phone on a touch screen!?

Can’t escape the comparison to Dumb and Dumber.

The “Glass Half Empty” Explorer argues: “With Glass you maintain in the here and now…”

So far – that’s brilliant. With Augmented Reality you Play in the Now.

But then he continues: “when I check messages I am looking in your general direction – I am not distracted.”

Just when I thought you couldn’t possibly be any more explorer. Or dumber.

Dumb and dumberer

My friend (and I mean it from the bottom of my heart), if you are reading a text message while talking to me – you ARE distracted. And looking in my GENERAL direction is like farting in my general direction.

Maybe these are just run-of-the-mill explorers regurgitating talking points.

So I asked a [very] senior [and very smart] member of the Google Glass team what’s the most compelling Glass app he’s seen so far. He didn’t flinch when answered: “Texting.”


This makes me mad!!!

The second Law of Augmented Reality design clearly states “Augmented Reality must not distract from reality”.

Second law of AR design

If it does distract you – it ventures into virtual reality which is an escape from the real world. The fundamental purpose of Augmented reality is to make you more aware of the real world and make things around you more interactive. Because in an interactive world everything you do is more engaging, productive, and fun.

The Simpsons’ Days of Future Future episode warns us about the consequences of not paying attention to the real world:


An incident that brought my anger to a head: A senior member of the Glass team which recently participated in a Glass Class at AWE 2014 didn’t agree to be video-taped or mentioned by name while at the same time was wearing Glass and [could have] recorded us all…


When I calm down, I’ll show what I consider good uses of Augmented Reality.

In the meantime check out over a hundred videos from AWE 2014 – the world’s largest event focused on Augmented Reality, Wearables, and the Internet of Things.

Guest Post: Presence – The Powerful Sensation Making Virtual Reality Ready for Primetime

Evolving the Human Machine Interface Part III

The concept of Presence in Virtual Reality (VR) has been gaining popularity over the past year, particularly within the gaming community. With consumer VR devices in development from Oculus, Sony, and more than likely Microsoft, Presence has become the metric by which we evaluate all VR experiences. But Presence is difficult to describe to someone who has never tried VR. “It’s like I was actually there. It made me feel like what I was seeing was actually happening to me, as though I was experiencing it for real,” is how one colleague described the experience.

Presence in VR triggers the same physical and emotional responses one would normally associate with a real world situation, it is the wonderfully magical experience of VR. But how is Presence achieved? While many research studies have provided a variety of subjective descriptions for Presence, there seem to be 3 common variables that affect tele or virtual Presence most:

1.  Sensory input: at minimum the ability to display spatial awareness

2.  Control: the ability to modify one’s view and interact with the environment

3.  Cognition: our individual ability to process and respond to sensory input

Because the nature of VR isolates the user from real world visual input, if the device’s sensory input and control are inadequate or missing then the effect of Presence fails, the results of which are oftentimes met will ill side effects: You feel sick to your stomach!

Sensory Input

For those who have tried VR, at some point or another you’ve felt queasy. That point which the experience turns from wonder to whoa, has been an unfortunate side effect throughout the development of VR. As Michael Abrash presented at Valve’s Steam Dev days, the hurdles needed to overcome VR sickness and achieve Presence are within reach. In the video below, Michael expertly summarizes the technical hurdles to achieve a believable sense of Presence in VR.

“What VR Could, Should, and Almost Certainly Will Be within Two Years” Steam Dev Days 2014

Michael Abrash, Valve Softwrae


To achieve a minimum level of Presence, head tracking is used to display the VR image to match the users own head position and orientation. While this helps to create the sense of spatial awareness within VR, it still doesn’t make someone’s experience truly “Present.” To do that, we need to add the representation of ourselves, either through an avatar or our own body image. Viewing our physical presence in VR, known as body awareness, creates an instant sense of scale and helps to ground the user within the experience. In the video sample below, Untold Games is creating body awareness through avatar control.

“Loading Human, true body awareness” Untold Games 2014

Without body awareness, VR can feel more like an out-of-body experience. Everything “looks” real and the user has spatial awareness, but the user’s body and movements are not reflected therefore the user does not feel actually present. Combining body awareness with VR’s spatial awareness creates a strong bond between the user and the experience.


The third perimeter of Presence is us. The feeling of Presence in VR is directly influenced by our personal ability to process and react to environmental changes in the real world. It’s likely that many of us will not have the same reactions to the experiences within VR. If you get sick riding in cars easily, then VR motion will give you the same sensation. If you’re afraid of heights, fire, spiders, etc. you’re going to have the same strong reactions and feelings in VR. Our individual real life experience influences our perception and reactions to VR. This can lead to some interesting situations, in particular with gaming. For example one player may be relatively unaffected by a situation or challenge, while another may be strongly affected.

Obviously the conditions of Presence are perceptual only. In most cases we’re not at the same physical risk in virtual environments as we would be in real life. But our own cognition coupled with VR’s ability to create Presence is why VR is such a popular field for everything from gaming and entertainment to therapy and rehabilitation.

Once we start to overcome these technical hurdles and provide a basic level of Presence, we next need to understand what it will ultimately enable. What does Presence provide for us in an experience other than merely perceiving the experience as real-like? We’ll explore that idea in the next segment, and try to understand where Presence will have the most impact.

Guest Post: Harnessing the Power of Human Vision

Harnessing the Power of Human Vision

By Mike Nichols, VP Content and Applications at SoftKinetic

For some time now, we’ve been in the midst of a transition away from computing on a single screen. Advances in technology, combined with the accessibility of the touch-based Human Machine Interface (HMI) have enabled mobile computing to explode. This trend will undoubtedly continue to evolve as we segue into more wearable Augmented Reality (AR) and Virtual Reality (VR) technologies. While AR and VR may provide substantially different experiences to their flat screen contemporaries, both AR and VR face similar issues of usability. Specifically, how and what are the most accessible ways to interact using these devices?

The history of HMI development for both AR and VR has iterated along similar paths of using physical controllers to provide user navigation. Although the use of physical controls has been a necessity in the past, if they remain the primary input, these tethered devices will only serve as shackles that prevent AR and VR from reaching full potential as wearable devices. While physical control devices can and do undoubtedly add a feeling of immersion to an experience, in particular with gaming, you would no more want a smart-phone that was only controllable via a special glove, then you would want to control your smart-glass through a tethered controller. As the technology for AR and VR continues to evolve it will eventually need embedded visual and audio sensors to support the HMI. In particular, visual sensors to support a full suite of human interactions that will integrate with our daily activities in a more natural and seamless way than our mobile devices do today.

In Depth

A depth sensor is the single most transformative technology for AR and VR displays because it is able to see the environment as 3-dimensional data, much like you or I do with our own eyes. It’s the key piece of technology that provides us with the building blocks needed to interact with our environments – virtual or otherwise. The depth sensor allows us to reach out and manipulate virtual objects and UI by tracking our hands and fingers

A scenes’ depth information can be used for surface and object detection, then overlaid with graphics displayed relative to any surface at the correct perspective to our heads’ position and angle. Depth recognition combined with AR and VR presents a profound change from the way we receive and interact with our 2D digital sources today. To simulate this effect, the video below shows an example of how a process known as projection mapping can transform even simple white cards in astonishing ways.

“Box” Bot & Dolly

It’s not hard to imagine how AR combined with depth can be used to transform our view of the world around us. To not only augment our world view with information, but even transform live entertainment such as theater, concerts, sporting events, even photography and more.

Take a more common example like navigation. Today, when we use our smart phones or GPS devices to navigate, our brain has to translate the 2D information on the screen into the real world. Transference of information from one context to another is a learned activity and often confusing for many people. We’ve all missed a turn from time-to-time and blamed the GPS for confusing directions. In contrast, when navigating with depth-enabled AR glasses the path will be displayed as if being projected into the environment, not overlaid on a flat simulated screen. Displaying projected graphics mapped to our environment creates more context aware interactions, and becomes easier to parse relevant information based on distance and view angle.

Bridge the gap

As we look to the future of AR and VR they will both certainly require new approaches to enable an accessible HMI. But that won’t happen overnight. With commercialized VR products from the likes of Oculus, Sony and more coming soon we’ll have to support an interactive bridge to a new HMI through existing controllers. Both Sony and Microsoft already offer depth cameras for their systems that support depth recognition and human tracking. The new Oculus development kit includes a camera for tracking head position.

We’re going to learn a lot about what interactions work well and those that do not over the next few years. With technology advances still a ways off to make commercial AR glass feasible as a mass market option, it’s even more important to learn from VR. Everything done to make VR more accessible will make AR better.

Stay tuned for our next guest post, where we’ll take a closer look at how depth will provide a deeper and more connected experience.

Guest Post: Evolving the Human Machine Interface

How the World Is Finally Ready For Virtual and Augmented Reality

By Mike Nichols, VP, Content and Applications at SoftKinetic

The year is 1979 and Richard Bolt, a student at MIT, demonstrates a program that enables the control of a graphic interface by combining both speech and gesture recognition. As the video of his thesis below demonstrates, Richard points at a projected screen image and issues a variety of verbal commands like “put that there”, to control the placement of images within a graphical interface in what he calls a “natural user modality”.

“Put-That-There”: Voice and Gesture at the Graphics Interface
Richard A. Bolt, Architecture Machine Group
Massachusetts Institute of Technology – under contract with the Cybernetics Technology Division of the Defense Advanced Research Projects Agency, 1979.

What Bolt demonstrated in 1979 was the first natural user interface. A simple pointing gesture combined with a verbal command, while an innate task in human communication, was and still is difficult for machines to understand correctly. It would take another 30 years for a consumer product to appear that might just fulfill that vision.

A new direction

In the years following Richard’s research, technology would advance to offer another choice to improve the Human Machine Interface (HMI). By the mid 80’s the mouse, a pointing device for 2D screen navigation, had evolved to provide an accurate, cost effective, and convenient method for navigating a graphical interface. Popularized by Apple’s Lisa and Macintosh computers, and supported by the largest software developer Microsoft, the mouse would become the primary input for computer navigation over the next 20 years.


“The Macintosh uses an experimental pointing device called a ‘mouse’. There is no evidence that people want to use these things.”

San Francisco Examiner, John C. Dvorak – image provided by…


In 2007, technology advancements helped Apple once again popularize an equally controversial device, the iPhone. With its touch sensitive screen and gesture recognition, the touch interface in all its forms has now become the dominant form of HMI.

The rebirth of natural gesture

Although seemingly dormant throughout the 80’s and 90’s, research continued to refine a variety of methods for depth and gesture recognition. In 2003 Sony released the Eye Toy for use with the PlayStation2. The Eye Toy enabled Augmented Reality (AR) experiences and could track simple body motions. Then in 2005 Nintendo premiered a new console, the Wii, which used infrared in combination with handheld controllers to detect hand motions for video games. The Wii controllers, with their improved precision over Sony’s Eye Toy, proved wildly successful and set the stage for the next evolution in natural gesture.

In 2009 Microsoft announced the Kinect for Xbox 360, with its ability to read human motions to control our games and media user interface (UI), without the aid of physical controllers.

What Richard Bolt had demonstrated some 30+ years prior was finally within grasp. Since the premier of Kinect we’ve seen more progress in the development of computer vision and recognition technologies than in the previous 35 years combined. Products like the Asus Xtion, Creative Senz3D, and Leap Motion have inspired an energetic global community of developers to create countless experiences across a broad spectrum of use cases.

The future’s so bright

To this day, Richard’s research speaks to the core of what natural gesture technology aims to achieve, that “natural user modality”. While advances in HMI have continued to iterate and improve over time, the medium for our visual interaction has remained relatively intact: the screen. Navigation of our modern UI has been forced to work within the limits of the 2D screen. With the emergence of AR and VR, our traditional forms of HMI do not provide the same accessible input as the mouse and touch interfaces of the past. Our HMI must evolve to allow users the ability to interact to the scene and not the screen.

CES 2014, Road to VR, SoftKinetic premiers hand and finger tracking for VR.


Next, we’ll explore how sensors, not controllers, will provide the “natural user modality” that will propel AR and VR to become more pervasive than mobile is today. The answer, it seems, may be right in front of us…we just need to reach out and grab it.


4 years at ARNY: Augmented Reality meetup celebrates 1500 members, 200 demos, new startups.

I founded the Augmented Reality New York meetup (ARNY) exactly 4 years ago as a labor of love, and it developed a life of it’s own: attracting nearly 1500 members, introducing 200 Augmented Reality demos, helping advance AR in NYC, creating partnerships, helping AR enthusiasts find jobs, and spurring some fantastic AR startups.

How did we celebrate last night’s ARNY?

With a fantastic collections of speakers and demos from all over the world: Israel, Canada, Columbia and New York City.

Huge shout out for our wonderful host Mark Skwarek at NYU Poly!

1) Brendan Scully – Metaio – first ever SLAM demo on Google glass from the Augmented Reality Company

2) Niv Borenstein – Kazooloo – A truly fun to play Augmented Reality game-toy combination

3) Keiichi Matsuda – Hyper-Reality – A new vision for the future with plenty of Augmented Reality goodness. Back him on Kickstarter!

4) Dhan Balachand – Founder and CEO, Sulon Technologies – a new head mounted console that will fully immerse players into games where the real world and the virtual world become one.

5) Ori Inbar – The latest and greatest from around the world of augmented reality

AR on Sony Playstation 4 featuring Augmented Reality on mainstream TV

MIT Tangible Interfaces: inForm – Interacting with a dynamic shape display

8Tree – fastCHECK is a revolutionary surface inspection system that is amazingly easy to use combining a laser scanner with a projector to help inspect aircrafts.

2013 DEMO Gods winner review – Pristine – Delivering the next generation of telemedicine and process control solutions at the point of care through Google Glass.

Moto X box opening AR experience – Augmenting paper popup for story telling

One Fat Sheep’s Hell Pizza Zombie game (New Zealand)

TWNKLS cool Augmented Reality SLAM demo  for maintenance and repair at Europort

Re+Public new AR mural app – resurrecting murals with Augmented Reality

Nikola Tesla app IndieGoGo launch by Brian Yetzer

Stay tuned for a full video of the entire event

RE+Public launching urban art Augmented Reality – reviving murals

Re+Public is (finally!) publicly launching its anticipated and innovative urban art augmented reality mobile app.

App Store: search: “republic”

Google Play: search “re+public heavy”


In addition to our projects at the Bowery Wall (NYC) and Wynwood Walls (MIA), the launch of our free app coincides with the video release of our most recent project, which is a collaboration with MOMO to create an interactive digital mural in St. Louis:

Click Thumbnail to View Video:

+ Note: if you are unable to visit these cities, you can still trigger the augmented reality experience from mural images on the Re+Public website. Click Here to view mural images.

While currently only available on mobile devices, the Re+Public app is a visionary initial step in the coming future of digitally augmented urban spaces that individuals will view and interact with through wearables. The Re+Public app will soon expand to include projects in more US locations and cities abroad.

A sincere thanks for your support and continuing to follow Re+Public. Please feel free to forward this email and help get the word out ;)

Additional Links:

Project Videos


Official Press Release


Re+Imagining Public Space

Los Angeles | New York City

Press Release: Sulon Technologies announces a New Head-Mounted Video Game Console that 1ups Traditional Consoles

Introducing GVX, an Xtreme Reality head-mounted gaming console that adapts the game to your surroundings, allowing you to Live The Game.

Richmond Hill-based game technology company Sulon Technologies, Inc. (Sulon) announces their revolutionary new product, GVX, a head-mounted gaming console that offers avid gamers the freedom to play anywhere, whether indoors or outdoors. Sulon’s proprietary technology involves applying advanced Augmented Reality (AR) and Virtual Reality (VR) technology to create the most realistic and immersive gaming experience available, enabling players to have a Star Trek ‘holodeck’ experience in their own living space and bringing to life traditional table top games in 3D on any flat surface.


Sulon’s solution to a tired console market is a brand new gaming device that introduces innovative gameplay by combining the benefits of console quality gaming on a mobile platform. Sulon is a team of experienced engineers, product development specialists, researchers and science fiction and gaming enthusiasts who have discovered and proven how to make any physical environment into a “holodeck” zone. “I’ve always been interested in new technologies and how society is quick to absorb them into their everyday lives” said Dhanushan Balachandreswaran, Founder and CEO of Sulon Technologies. “We are excited about creating technology that we originally thought to be fiction and turning it into a reality.”

The GVX system introduces the concept of Xtreme Reality (XR) defined as the one-to-one integration of the real world and the virtual world with the ability to scale the whole spectrum of AR to full VR. XR blurs the lines between the real and virtual worlds to actually place the player into their game by adapting their entire physical environment into the game world. GVX uses complex and adaptive algorithms, high end graphics processing, motion tracking and position tracking to achieve the XR experience. The system has the unique ability to map the actual environment as the physical characteristics and boundaries of the game environment. Unlike any other device available, it applies AR and VR to transform any space (even outdoors!) into a completely new game environment in real-time. It is a “wear and play” experience that expands a player’s gaming space from the area in front of their TVs and PCs to their entire home. GVX runs on the Android Operating System where applications can be developed quickly and games can span the entire spectrum of casual to hard core genres. XR games on GVX are classified as Active or Surface gaming.


Active games are adrenaline inducing and interactive, where the system conducts an accurate and rapid scan of the player’s entire environment to adapt it into the game world using sophisticated SLAM algorithms. GVX is also completely wireless with all devices communicating via Bluetooth or Wi-Fi, giving users complete freedom of movement. Players can now live their video game by physically exploring and interacting with the virtual environment. With Active gaming, space limitations are not an issue as the system is able to generate new graphics and scenarios to the same space multiple times during one gaming session, allowing for an endless number of new gaming experiences.

Surface games on the other hand are a creative throwback to traditional gaming where games are augmented in 3D onto any flat surface. Players can watch their cities actively grow, wage virtual wars against other players around the world or enjoy a tabletop game with friends and family. Surface gaming provides a new reach for social gameplay and a new avenue and meaning for social interaction. Family game nights can be made possible even when family members are not physically present.“GVX really gives game developers creative freedom to take advantage of all the functions and capabilities of the system when designing games.” said Dhanushan Balachandreswaran.

What also makes GVX a game-changer is that Active and Surface gaming is not mutually exclusive and can be combined to create innovative gaming experiences. Jumanji is a great example of how surface and active gaming can be combined. The game board is augmented in 3D on a flat surface (surface gaming aspect) and game events that occur would require players to interact with the virtual environment (active gaming aspect). In addition to the XR experience of Active and Surface gaming, GVX is also highly flexible and capable of playing existing games such as PC games, games specialized for stereo VR or mobile games like Angry Birds.

“The concept of mixed reality has been around for years but nobody has succeeded in making it real” said Jackie Zhang, Vice President of Research and Development at Sulon Technologies. “With the latest technologies and innovation, we have made this concept possible. It’s a disruptive product whose limitation is the bounds of your imagination.”

GVX also features a removable component (the GVX Player) that offers players a variety of gaming options. The GVX player can be used on its own to play existing mobile games from the Google Play store as well as connect to a HDTV and a Bluetooth controller for those who enjoy traditional gaming on their television. These gaming options on the GVX Player can be combined with the XR experience on GVX (i.e. Active and Surface gaming) to create a multitude of unique one-of-a-kind gaming experiences. A game could feature as many or as little of these gaming combinations. For example, a game could begin as a simple mobile game, but an in-game event may prompt the player to switch to Active gaming in order to fully experience the event. These flexible gaming options allow players to easily pick and choose their preferred gaming experiences.

GVX is also a consumer-friendly gaming device that eliminates many of the problems associated with adopting AR and VR technology. Surface games (AR application) on GVX are hassle-free and do not require physical markers. VR gaming on GVX is safe and does not induce motion sickness. GVX features an option where players can adjust the level of graphics opacity. Lowering the graphics opacity allows more of the player’s physical surroundings to appear in the game environment, ensuring safe gameplay and providing players who are not ready for a full virtual environment an alternative. The opacity option, as well as the wand controller’s one-to-one true motion tracking prevents motion sickness because the player’s movement in the physical world matches their movement in the game world.

“GVX is truly a game-changer in the gaming and mobile space. Never before has a gaming platform been able to successfully bring to life that Star Trek “holodeck” experience in a simple and straight-forward product, while freeing the user from the constraints of a wired environment,” comments Ken Wawrew, the Chief Operating Officer at Sulon Technologies.

GVX developer kits are available for pre-order on the Sulon Technologies website.

For more information, visit


Get every new post delivered to your Inbox.

Join 35 other followers