You don鈥檛 need special glasses to see an augmented view of the world. A new combined camera and computer can superimpose images over real-world objects without the need for a head-mounted display.
connects to a video projector to beam images and animations on to surrounding objects, essentially turning any surface into a screen 鈥� a technique called projection mapping. To do this, it scans the environment using depth sensors to map the shape of objects, then tailors its lighting effects to fit. 鈥淭he idea is to seamlessly merge the virtual world with the physical world, and to do it without wearing anything on your face,鈥� says Lightform CEO Brett Jones, whose firm came out of 鈥渟tealth mode鈥� this week.
Initial demos show a coffee shop鈥檚 price list materialising on a blank slate (pictured, top), squiggly lines dancing across a store window display, and a cactus undulating with decorative pulses of light.
Advertisement
Augmented reality has so far often been delivered through a wearable device: Microsoft鈥檚 HoloLens uses a headset, and the much-hyped Florida-based start-up Magic Leap is expected to launch its AR headset later this year. In February, Facebook鈥檚 Mark Zuckerberg hinted that the company is interested in developing AR eyewear alongside its Oculus Rift virtual reality headset. 鈥淭he goal is to make VR and AR what we all want it to be: glasses small enough to take anywhere,鈥� he .
But eliminating headsets would mean many people can share in an AR experience at once and without special preparation, and removes problems around the comfort, weight and power cables of wearable displays.
Projection mapping technology is usually used for large-scale, one-off events. Jones previously worked at Disney Imagineering, developing projections for theme parks, and led projects at Microsoft Research to create expansive projected gaming experiences.
AR for anyone
Lightform, based in San Francisco, aims to turn the technology into something anyone can use. The device is designed to work with existing projectors and comes with software that Jones says is as easy to use as Photoshop.
The size of the projection depends on the projector. 鈥淵ou could do your coffee mug using a tiny pico projector or you could do the side of a building using a really big projector,鈥� says Lightform鈥檚 design director Phil Reyneri. The camera periodically rescans the scene and recalibrates projections if things have moved, making it suitable for long-term installations, and you can control or modify the graphics through an app. The whole package will cost more than a depth sensor like Microsoft鈥檚 Kinect but less than a mid-range laptop when it starts shipping later this year, says Jones.
The mapping is not quite real-time 鈥� it takes about a minute to do a scan 鈥� and you can鈥檛 interact with the projected images, unlike with some systems that use haptic devices or motion tracking to give users the illusion of touching what they see. A by Texas-based , for example, uses computer vision to allow people to play air hockey using real objects as bats to hit a virtual puck, with the pitch markings projected on to a tabletop 鈥� though it only works on a flat surface.
Argo chief technologist Jared Ficklin imagines projected interfaces being used to control smart home devices alongside voice-recognition technology like Amazon鈥檚 Alexa. It could project a recipe on to a kitchen surface, for example.
Using light to augment reality is exciting, says at the Massachusetts Institute of Technology. Adding projection mapping means you can 鈥減aint with light鈥� and give real-world objects virtual textures that fool the eyes. But projected AR has its own drawbacks, he points out. Shadows can be a problem if anything gets between the projector and surface, and it does not work well in bright spaces.
If projected interfaces become integrated into the spaces we live and work in, they could usher in a new kind of ubiquitous computing, says Linder. But first they need to find really useful applications.
