1
The Origins of Projection Mapping

1.1. Introduction

Figure 1.1. Evolution of the queries of the keyword "video mapping" in French on Google from 2004 to 2019 (source: Google Trends, 2019). For a color version of the figures in this book see, www.iste.co.uk/schmitt/image.zip

When did projection mapping start? More than 10 years separate this text from the emergence of the key word "video mapping" on Google, but this criterion alone is not enough to capture its origin and requires a more distant archaeology. The materials that make up this chapter come from a collection of significant historical elements during a series of expeditions in the reflector that is the archived history of ideas and works of art or technology up to the 20th Century. Presented with a desire to go beyond the passive collection, these facts are ordered in such a way that one can understand how projection mapping can be understood as an instrument of vision comparable by similarities and differences to many other devices that preceded it, but also as an original way of seeing that has been maintained over time.

1.2. Let's moonwalk! A short crossing through time

1.2.1. The emergence of the expressions "video mapping", "projection mapping", "spatial augmented reality" and "spatial correspondence" between the beginning of the 21st Century and the end of the 20th Century

By literally writing "video mapping" on the Google Trends trend analysis tool, we can see that queries for this keyword increase significantly from a point of origin located in 2008. This information highlights the recognition of the unity of a concept based on Web searches for it and would make it possible to clearly date the birth certificate of projection mapping. Google and the use of this word alone, however, do not reflect what precedes them: before 2008, this expression was not used or very rarely used, whether written as one or two words, with or without an accent, and with or without the word "projection" instead of "video". This initial year would therefore be sufficient if it was not very clear that the technical device and its spatial writing logic were already well identified in the 1990s by technologists or artists. The often cited "spatial correspondence" was first mentioned by Michael Naimark in 1984 in Los Angeles, while "spatial augmented reality" was first mentioned in 1998 in North Carolina by Ramesh Raskar.

Between these two publications, Paul Milgram proposed in 1994 in Toronto a taxonomy of "mixed realities", and the first patents identified for digital video projections that match volume surfaces were filed for Disney Inc. in 1991 and General Electric Co. in 1994. Between these two patents, artist Tony Oursler exhibited The Watching in 1992. Later, in 1999, John Underkoffler, the designer of Minority Report's famous interface, invented I/O bulb and Luminous room when the facade of Amiens Cathedral was painted. In 2004, choreographer Klaus Obermaier used infrared to divert the real-time video from the silhouette of a performer in the foreground of the stage and projected a video on his body that differed from the background. Johnny Chung Lee published his doctoral thesis on automated projector calibration the same year. The following year in 2005, Olivier Bimber planned to project onto a classical-age painting.

1.2.2. From 17th Century magic lanterns to ancient camera obscura

If we stick to the technical device and spatial writing logic, many artists, painters, visual artists, illustrators or directors have also used, since the beginning of the 20th Century, the unconventional image projected in relation to the reference that is cinema. They played with the image carriers and environments of their works in a way that projection mapping still offers today. Before them, in the 19th Century, it was travelling projectionists like Maximillian Skladanowsky who projected natural disasters, sometimes improvising projection spaces for storms, fires or earthquakes. These images are reminiscent of buildings that collapse with digital pixels. Savoyard lanternists brave the mountain slopes accompanied by marmots, monkeys, drums, or barrel organs like the contemporary off-road projection mapping projectionists do outside rooms.

At the end of the 18th Century, phantasmagorical spectacles such as those performed by Giuseppe Balsamo called the Count of Cagliostro, Johann Georg Schröpfer, Paul Philipsthal or Étienne-Gaspard Robert called Robertson used illusory projections of supernatural beings. They would double the projections of thunderous noises or the smell of burnt feathers. Between the 15th and 16th centuries, a magic lantern was designed by Leonardo da Vinci, but they were not made by Christiaan Huygens until 1659. Then they were carried around in 1664 by Thomas Walgenstein, at the time when anamorphoses were theorized. The lantern is widely disclosed by Johann Christoph Sturm and the Jesuit Athanasius Kircher. It was Milliet Dechales, in 1674, who produced a series of projected images approaching animation and, in 1698, Johann Christoph Weigel developed an overlay of images in the same way as a compositing of layers. This was shortly before the lantern was mounted on wheels and several were used to compose the same image, which today would be called stacking or shifting the image. Johannes Zahn, a Jesuit, also shows animated projections: he projects worms in a jar, the movements of a weather vane or those of a clock. Van Musschenbroek continued Zahn's work around 1730 by inventing animated plates that made it possible, for example, to turn the wings of a mill. As early as 1608, Cornelis Drebbel said he could change the appearance of his clothes and make giants appear.

In 1650, another Jesuit, Gabriel Magalhaens, reported descriptions of the use of oriental lanterns that he said were quite convincing. Chiang Khuei and Fang Chheng, in the 13th Century, made animated projections on smoke. Sun Kuang-hsien, in 930, or Shao Ong the magician, as early as 121 BC, did comparable tricks. In the past, they would turn flying dragons. In the 16th Century, in the West, Giambattista della Porta described in his Magie naturelle how to reveal volumetric light in chalk powder. He provides a remarkable drawing of it. Girolamo Cardano and Benvenuto Cellini projected images for shows. In 1420, Johannes de Fontana produced a drawing of what he called the lantern of fear in the Bellicorum instrumentorum liber. Jokes and mysteries are played in the Middle Ages with these thaumaturgical lanterns. Arnaud de Villeneuve, from 1290, would have used it. The camera obscura was known to Aristotle and Mozi in the 4th Century BC, and even to Apollonius of Tyana, the miracle worker, in the 1st Century.

Contemporary archaeologist Matt Gatton argues that the camera obscura may have been used as an archaic projector to animate the faces of statues during the Eleusian feasts in ancient Greece - no doubt long before the use of 16 mm projector film from 1969 onwards to project on the busts sculpted inside Disneyland's Haunted Mansion and on their facades and other crystal balls. Contemporary authors Jean-Jacques Lefrère and Bertrand David have even put forward the hypothesis, with supporting arguments, that figurative shadows were projected using tallow candles as early as the Palaeolithic era.

1.2.3. The screen as a material considered as a void: projection mapping in negative from the 15th Century onwards

As far as we can go in the history of light or shadow projection devices, pinpointing a beginning seems complex. Would it be necessary to have a kind of visual antinomy of what projection mapping is in order to be able to record its beginning by way of a contrast? One can find an antinomy of this type in an asserted disregard for the medium receiving the image, since it is the interplay with the latter that gives meaning here to the words mapping, correspondence or spatial. One of the most important key points of such disregard is undoubtedly the explicit verbalization of Leon Battista Alberti written in 1435 in De Pictura. He presents the painting board as an "open window". Mapping on the void that is the open interior of a window is a misdirection: no more composition according to the medium is possible as soon as it is expressed as non-existent.

If we agree to consider the artistic movement that began with the transition to the 20th Century of so-called modern art - or at least one of the most important parts of this movement - as a challenge to this Albertian representation as a visual screen form that has become hegemonic, the hypothesis is that projection mapping would only be one manifestation, among other types, of what happened as a result of this paradigm shift. Extension of the painting and access to new dimensions sought by painters, removal of the base in sculpture, total or synesthetic art, sublimation or enhancement of the material, background scenery becoming an actor in scenographies, etc., are all fine art studies whose origins can be traced to projection mapping.

1.2.4. How far back in history can we go?

Thus, the first image not to consider its medium as an "open window" is the one that took advantage of the formal specific...