Californian start-up Lightform has developed a new type of augmented reality that works without a headset. Design director Phil Reyneri explains the projection-based technology and how it can transform the built environment with light and texture.
What is Lightform?
Lightform is a device that adds computer vision to any video projector. It can 3D scan and project images or information onto objects, effectively turning anything into a screen. We see it as a more accessible alternative to head-mounted augmented reality displays, which are currently very expensive, only work for one person at a time, run out of battery, and can cause neck strain.
Projection mapping has historically been limited to niche art installations or big budget events. Lightform makes the technique much more accessible by simplifying the technical aspects. It also makes permanent installations possible, by automatically fine-tuning image alignment when anything moves. We bill it as the first end-to-end projection mapping solution, from 3D scanning, to content creation, to installation and management.
Who developed it?
Lightform’s founders did computer-vision related research internships for Microsoft, Disney and Adobe while completing their PhDs together at the University of Illinois Urbana-Champaign.
Much of their research involved using the Microsoft Kinect for projection mapping and IllumiRoom used projection to expand the peripheral bounds of a TV screen into the room itself.
My personal background is in more on the entertainment side, helping coordinate projections on Sydney Opera House and the Guggenheim in New York.
We’ve raised $2.6m from investors including Seven Seas and Lux, and have secured grants from the National Science Foundation. Our advisers also include the former chief scientist for Oculus. The product will be available for pre-order this summer.
How much will it cost?
That has yet to be announced. The device includes a camera and a computer, it will therefore cost more than a traditional camera but less than most laptop computers.
Is it possible to create an augmented reality environment using just one projector?
Yes, absolutely. Lightform will work with single or multiple projectors depending on size of the installation. As projectors continue to decrease in size and price, we foresee them becoming as ubiquitous as traditional lighting fixtures.
We see Lightform being used as a more dynamic form a digital signage, to highlight a specific piece of architecture, or to create visual experiences not possible with a conventional TV screens.
What are the key benefits for architects and designers?
We’re exploring the notion of traditional illumination becoming smarter and higher definition. Projection mapping enables the possibility of digital facades and digital architecture, blending digital information with physical structures. Lightform allows projection to understand the geometry it’s illuminating, and can automatically create effects specifically tailored for the form.
Laser and LED light sources are making projection more feasible for long-term installations. The hardware is smaller, brighter, more energy efficient, and lasts 10-30 times longer than traditional bulb-based projectors. Many can now operate maintenance free upwards of six years, which is in the ballpark of traditional screen-based digital signage.
What is the major advance in terms of the technology?
Projecting a rectangular image on a flat wall is easy. Projecting 3D effects precisely aligned to complex geometry is hard. It currently requires many different steps and professional software licenses. With Lightform, we’re creating an integrated tool that covers all the bases, from scanning, to content creation, to permanent installation.
Over time, image alignment and calibration can be a big challenge with current technologies. Even with the most rigid rigging, things still move. Whether it be projection optics heating up, the subtle fluctuations of a building, or gradual vibrations over time, keeping precise alignment over time is nearly impossible.
With projection mapping, you quickly start to notice when things are misaligned because it involves two independent components. It currently requires expensive software and high-end cameras, or a dedicated human that is on call for realignment.
Lightform enables what we call “set it and forget it”, so that once the projection installation is set up you don’t ever have to worry about it, it simply adjusts to changes and stays calibrated.
We’re exploring the notion of traditional illumination becoming smarter and higher definition. Projection mapping enables the possibility of digital facades and digital architecture, blending digital information with physical structures.– Phil Reyneri, Lightform
