(courtesy of TED, Photo: Ryan Lash)
Magic and Storytelling
interactive augmented reality performance

2012

Colloboration with:
Marco Tempest
checksum5

powered by vvvv.org


Header Photo: Ryan Lash
»Magic and Storytelling« is an interactive presentation about the history of storytelling augmented with virtual magic in real-time. It was produced and created by New York based cyber illusionist Marco Tempest in collaboration with onformative and checksum5. When Marco asked us to work on the art direction for »Magic and Storytelling« we were fascinated by the challenge of combining not only several new technologies, but also giving the whole presentation a homogenous and magical look. In order to achieve this we developed a variety of different particle systems. A combination of three different systems was used to compose the visual effects of each scenes. The »global magic dust«, which is based on and controlled by the Kinect depth image, the »hand magic dust«, that we placed particle emitters via hand-tracking onto Marcos hands for
to emphasize his movements, and the »custom magic dust«, which allowed us to picture very different elements, as for instance appearing party guests, typography or flying fireballs and bullets made out of particles. The main task was to bringt together a very long list of demanding interactive real-time effects – such as hand-tracking, face-tracking, face substitution, particle systems, elements in 3d space – with the exact course of a six minutes show, which was exactly timed by the given soundtrack. For the development of this graphics and computational intensive app, we worked together with checksum5, the developers of vvvv. Not to forget the support of Tebjan Halm and Enrico Viola for additional vvvv development and preliminary studies done in openFrameworks.
The aim in »Magic and Storytelling« was to find a magical and engaging way to enhance a live performance with interactive graphic. It is generated by a data flow system in real time and in reaction to the live elements.
Marco Tempest at TED2012
documentation video
Marco Tempest about »Magic and Storytelling«
The aim in »Magic and Storytelling« was to find a magical and engaging way to enhance a live performance with interactive graphics. The live component could be anything from a lecture to a performance. The augmentation layer is not pre-rendered. It is generated by a data flow system in real time and in reaction to the live elements. I think that this type of magic can help us to map out what that part of the future might feel like, where 3D projection and gestural sensing are intelligent and ubiquitous. The collaboration with onformative and checksum5 created the perfect synergies to explore my vision. vvvv proofed to be the ideal tool to develop this graphics and computational intensive app. Magic and illusion are a good sandbox for creativity. Adding the element of illusion to an existing technology allows us to create things that are otherwise not possible. An illusion takes a technology beyond its natural boundaries. Magic is a unique tool for pre-visualization.
By that I mean the illusion created is so convincing that the experience is almost indistinguishable from the reality. In time the advances in technology will turn the illusion into reality. But for now the illusion gives us a very good idea of what that technology might accomplish. Looking to the very near future, when every laptop or device has a 3D sensor built into it, imagine the entertainment and educational possibilities of a system like this, designed for face-to-face communication. Psychology and cognition, chemical and engineering, design and computer. These are all disciplines that are employed in magic. They are the tools of illusion but they are not liberally interfaced with other fields of research and that strikes me as a shame. For me the most progress is going to be made by embracing current and future technologies in the best way possible, by sharing and collaborating, and inventing new and seemingly impossible things.
The collaboration with onformative and checksum5 created the perfect synergies to explore my vision. Magic and illusion are a good sandbox for creativity. Adding the element of illusion to an existing technology allows us to create things that are otherwise not possible
images of TED2012

(courtesy of TED, Photo: Ryan Lash)

(courtesy of TED, Photo: Ryan Lash)

more images on flickr
making of