Performance Architecture

Using landmark spaces as digital canvas, projection mapping is trompe l'oeil on a gigantic, pulsating scale. Where will it take us next?

Call it projection mapping, pixel mapping, video mapping, or even architecturally registered mapping. It’s known by all those names and more. But the new media format that has been the toast of YouTube—and live audiences worldwide—has broken a major visual barrier and is poised to knock down more walls.

Whether its canvas is a 13th century castle, an iconic museum, or even a half-kilometer-long string of grain silos, projection mapping has effectively released high-definition video from the rectangular prison of the display screen. It allows immersive 3D and even interactive experiences to be layered onto landmark buildings, complex interior spaces, and urban landscapes. And it offers promising possibilities for public art, urban planning, exhibition design, and branded environments.

Lighting, art, video, and beyond

While projection mapping has emerged as a unique media platform, its lineage is in outdoor show lighting and in Son et Lumiere (sound and light) shows mostly presented in Europe. It also has direct origins in digital art, including the groundbreaking work of artist/architect Pablo Valbuena, artists such as Tina Frank and Karsten Schmidt, and collectives such as Maxalot Gallery (Amsterdam) and Griduo (Istanbul).

In the past three years, it has taken off as a means to create jaw-dropping, reality-smashing 3D outdoor spectacle: trompe l’oeil on a gigantic, pulsating scale. The results have created blockbuster enthusiasm from the public actually attending these events, as well as the millions of fans who view the shows online.

"Projection mapping is definitely on the rise as a booming video projection trend," says George Tsintzouras, senior director of product management for Christie (Kitchener, Ont., Canada). “As a supplier of high-end video projection equipment, Christie has been involved in projection mapping projects for trade shows, product launches, and concerts and entertainment. And there is a growing creative community embracing it as a marketing and entertainment tool.”

Suspending disbelief

Essentially, projection mapping involves three steps: generating a 3D model of a given structure or space, creating animation or motion graphic content that conforms to it, and projecting the content onto the actual facade or space. The basic tools of this process include software such as Autodesk 3ds Max, Adobe After Effects, Cinema 4D, or Maya, as well as high-lumen video projectors, powerful sound systems, and media servers. 

By mapping and working within the exact architectural contours of a building or space, the creators of these projections can animate entire facades or even make them react to the environment in real time. Shapes and colors appear and morph, creatures fly or crawl across the space in 3D, weather changes in an instant—the entire space becomes a textured surface for storytelling.

The more geometrically complex the space, the more complex the mapping process—and the more interesting the result, says Anthony Rawson, associate with Australian architectural firm The Buchan Group.

Rawson describes projection mapping as a sophisticated melding of content and architecture, with both as equally important elements. "The more protrusions, columns, and sculptured front surfaces you have, the more likely you’ll have an interesting projection surface, albeit a challenging one,” notes Rawson. “The last thing you want is a plain, flat wall with no geometric variation.”

For client Rio Tinto Coal Australia, Buchan leveraged the iconic architectural features of the Customs House in Brisbane to create a virtual mine. Projection mapping offered Rio Tinto a unique and memorable way to tell its story to clients and employees.   

Given the size and level of realism that can be supported with 3D projection mapping, “You can really create incredible spectacles that allow audiences to momentarily suspend disbelief," says Jeff Grantz, principal of Materials & Methods (Boston), which works with developers, architects, and designers to explore creative integration of digital media within built environments.

Grantz co-produced FLASH:light, a May 2011 multimedia projection festival organized by Nuit Blanche NYC and the New Museum in SoHo. The festival featured several site-specific projection-mapped installations whose creators—more than 20 multimedia artists—liken their work to virtual graffiti. “They can leave their mark without making any marks on the building,” says Grantz.

Virtual and real benefits

Projection mapping has considerable appeal for media producers and clients.

It  allows for large-scale presentations that can easily and cost-effectively transform a building facade or space. It’s compelling enough to draw very large audiences. And it has wide appeal for branding, advertising, trade show presentations, museum installations, entertainment happenings, public art, and themed special events. Its ephemeral nature allows a building, landscape, or interior space to become a virtual agora, offering a temporary display format that can transform one space into many different realities.

Another major benefit is its ability to transform physical space in a non-invasive way: you don't have to temporarily or permanently alter a structure by hanging a screen or removing a sight-line interference. If you're not "touching" the structure, that provides more production freedom and projection opportunity, freeing building owners from worry about their buildings being physically changed or damaged during pre-production and event presentation. This is particularly helpful when landmark or historic legacy buildings are involved.

In an age when traditional advertising and branding are being re-evaluated, projection mapping has become a unique marketing and social media tool, creating huge flash audiences to witness these visual spectacles.

“It’s becoming a great social enabler, creating these communal viewing experiences that bring people together,” says Craig Hanna, chief creative officer for experience design and production firm Thinkwell (Burbank, Calif.). “And because it’s typically produced outdoors, it takes the idea of an entertainment experience and puts it in a public space.” While an indoor theater can hold a few thousand people at best, a projection mapping event in a large outdoor viewing space can reach 10,000 to 15,000 people in a single presentation. Thinkwell’s most recent projection mapping project was for a daily summer evening show at Universal Studios Orlando, which drew crowds of several thousand at a time.

And when the initial projection mapping experience goes viral—via individual camera phones as well as professional video documentation—the appeal multiplies. Suddenly the events become YouTube and Vimeo bytes shared by the planet, creating exponential impacts.

Total immersion

London-based studio Seeper has been at the forefront of projection mapping, part of the avante-garde creating content that has sparked the world’s imagination. As part of Ogilvy & Mather’s campaign for the Ford S-MAX minivan, Seeper “froze” the facade of the imposing Senate House in London’s Russell Square.

But coaxing the audience to look on wasn’t enough. As a virtual climber attempted to scale the 3D wall of ice, audience members broke off chunks of the ice using “laser cannons” provided as part of the show.

Seeper founder Evan Grant considers himself an “interactive engagement artist” and has begun exploring natural user interactions and ubiquitous computing to create multi-sensory experiences. "Our shows are interactive, like an art installation where the viewers are controlling the performance."

In its interactive projection mapping events (including Ford S-Max Ice Experience, Xbox Kinect Munich Launch, and the Google Chrome Interactive Projections), Seeper’s approach is “to create immersive and interactive environments that bring the viewer into the immediacy of the experience.”

With interactive tools such as 3D motion tracking, sound-responsive graphics, and multitouch devices, Seeper builds in features that allow audience members to impact and be a part of the performances.

Projecting the future

Beyond its current uses, the possibilities for projection mapping are far reaching, with some experts touting it as the bridge to fully realized augmented reality.

As projection mapping matures and becomes easier and more affordable to use, its application base will expand and its visibility in the public sphere will increase. Smaller, higher-lumen projectors will allow designers to place more units more discreetly than before and from longer distances, says Chris Manson, Thinkwell’s technical director. Improvements in real-time, adaptable, and intelligent warping and tracking will help place imagery in more ways and places. “And lower costs and better usability will give non-experts and those with smaller budgets the chance to create amazing work,” he adds.

Grantz is already working to use projection mapping as a tool for building developers to show off new building plans or revisions to clients. "Certainly on that level, the visual impact of such large-scale images goes a long way in ramping up enthusiasm for a new building or urban design when they see it at its intended 1:1 scale."

And he believes projection mapping can extend beyond festivals or temporary events to serve as a platform for public art. “Communities are often reluctant or apprehensive about permanent public art installations, whereas with projection art you are really just providing a digital canvas and inviting artists to present their work,” he notes.

Thinkwell’s Hanna sees projection mapping as an increasingly important media platform for corporate events and entertainment. "As HD video projectors become more powerful and less expensive, projection mapping will be more accessible as a useful visual format.” Most important, he adds, "as the medium matures, it will be less a visual effect gimmick and more a storytelling technique.”

He says projection mapping also has potential in urban planning, as a means to attract local communities to gather for ongoing outdoor presentations. LED-based versions of this concept are already alive and well in Las Vegas (Fremont Street Experience) and Dallas (Victory Park).

Creators also see projection mapping as a tool for placemaking and even exhibition design. Skinnable retail stores, branded environments, and event spaces won’t be far behind. 

Seeper’s Grant says interactivity is the key to projection mapping’s future. As audiences demand more active engagement with content in museums, entertainment facilities, stores, and other venues, the logical next step is real-time installations that can be manipulated and controlled by their users. “People will want to engage with these interactive performances as immersive, multi-sensory experiences.”

--By Louis M. Brill, segdDESIGN No. 32, 2011

Editor's note: Louis M. Brill is a journalist and consultant for high-tech entertainment and media communications.