Virtual Reality and Augmented Reality
Description
More details
Other editions
Additional editions
Persons
P. Guitton, Professor, University of Bordeaux, France.
G. Moreau, Professor, Ecole Centrale de Nantes, France.
Content
Preface xi
Introduction xv Bruno ARNALDI, Pascal GUITTON and Guillaume MOREAU
Chapter 1. New Applications 1 Bruno ARNALDI, Stéphane COTIN, Nadine COUTURE, Jean-Louis DAUTIN, Valérie GOURANTON, François GRUSON and Domitile LOURDEAUX
1.1. New industrial applications 1
1.1.1. Virtual reality in industry 1
1.1.2. Augmented reality and industrial applications 3
1.1.3. VR-AR for industrial renewal 4
1.1.4. And what about augmented reality? 12
1.2. Computer-assisted surgery 14
1.2.1. Introduction 14
1.2.2. Virtual reality and simulation for learning 16
1.2.3. Augmented reality and intervention planning 21
1.2.4. Augmented reality in surgery 26
1.2.5. Current conditions and future prospects 31
1.3. Sustainable cities 32
1.3.1. Mobility aids in an urban environment 33
1.3.2. Building and architecture 37
1.3.3. Cities and urbanism 41
1.3.4. Towards sustainable urban systems 46
1.4. Innovative, integrative and adaptive societies 48
1.4.1. Education 48
1.4.2. Arts and cultural heritage 54
1.4.3. Conclusion 60
1.5. Bibliography 61
Chapter 2. The Democratization of VR-AR 73 Sébastien KUNTZ, Richard KULPA and Jérôme ROYAN
2.1. New equipment 73
2.1.1. Introduction 73
2.1.2. Positioning and orientation devices 74
2.1.3. Restitution devices 82
2.1.4. Technological challenges and perspectives 100
2.1.5. Conclusions on new equipment 109
2.2. New software 111
2.2.1. Introduction 111
2.2.2. Developing 3D applications 113
2.2.3. Managing peripheral devices 116
2.2.4. Dedicated VR-AR software solutions 119
2.2.5. Conclusion 120
2.3. Bibliography 121
Chapter 3. Complexity and Scientific Challenges 123 Ferran ARGELAGUET SANZ, Bruno ARNALDI, Jean-Marie BURKHARDT, Géry CASIEZ, Stéphane DONIKIAN, Florian GOSSELIN, Xavier GRANIER, Patrick LE CALLET, Vincent LEPETIT, Maud MARCHAL, Guillaume MOREAU, Jérôme PERRET and Toinon VIGIER
3.1. Introduction: complexity 123
3.1.1. Physical model and detecting collisions 124
3.1.2. Populating 3D environments: single virtual human to a surging crowd 130
3.1.3. The difficulty of making 3D interaction natural 137
3.1.4. The difficulty of synthesizing haptic feedback 141
3.2. The real-virtual relationship in augmented reality 150
3.2.1. Acquisition and restitution equipment 151
3.2.2. Pose computation 152
3.2.3. Realistic rendering 156
3.3. Complexity and scientific challenges of 3D interaction 158
3.3.1. Introduction 158
3.3.2. Complexity and challenges surrounding the 3D interaction loop 158
3.3.3. Challenge 1: sensory-motor actions for interaction 159
3.3.4. Challenge 2: multisensory feedback 163
3.3.5. Challenge 3: users and perception 166
3.3.6. Conclusion 167
3.4. Visual perception 168
3.4.1. A glossary of terms related to unease, fatigue and physical discomfort 168
3.4.2. Display factors 173
3.4.3. Conclusion 179
3.5. Evaluation 179
3.5.1. Objectives and scope of this section 179
3.5.2. Evaluation: a complex problem 180
3.5.3. Evaluation using studies with human subjects 184
3.5.4. Drawbacks to overcome 193
3.5.5. Evolutions in measuring performance and behavior, characterizing participants 195
3.5.6. Conclusion and perspectives 200
3.6. Bibliography 201
Chapter 4. Towards VE that are More Closely Related to the Real World 217 Géry CASIEZ, Xavier GRANIER, Martin HACHET, Vincent LEPETIT, Guillaume MOREAU and Olivier NANNIPIERI
4.1. "Tough" scientific challenges for AR 218
4.1.1. Choosing a display device . 218
4.1.2. Spatial localization 221
4.2. Topics in AR that are rarely or never approached 223
4.2.1. Introduction 223
4.2.2. Hybridization through a screen or HMD 224
4.3. Spatial augmented reality 227
4.3.1. Hybridization of the real world and the virtual world 227
4.3.2. Current evolutions 228
4.4. Presence in augmented reality . 229
4.4.1. Is presence in reality the model for presence in virtual environments? 229
4.4.2. Mixed reality: an end to the real versus virtual binary? 231
4.4.3. From mixed reality to mixed presence 231
4.4.4. Augmented reality: a total environment 232
4.5. 3D interaction on tactile surfaces 233
4.5.1. 3D interaction 234
4.5.2. 3D interaction on tactile surfaces 236
4.6. Bibliography 240
Chapter 5. Scientific and Technical Prospects 247 Caroline BAILLARD, Philippe GUILLOTEL, Anatole LÉCUYER, Fabien LOTTE, Nicolas MOLLET, Jean-Marie NORMAND and Gaël SEYDOUX
5.1. The promised revolution in the field of entertainment 247
5.1.1. Introduction 247
5.1.2. Defining a new, polymorphic immersive medium 248
5.1.3. Promised experiences 251
5.1.4. Prospects 255
5.2. Brain-computer interfaces 258
5.2.1. Brain-computer interfaces: introduction and definitions 258
5.2.2. What BCIs cannot do 260
5.2.3. Working principle of BCIs . 261
5.2.4. Current applications of BCIs 263
5.2.5. The future of BCIs 268
5.3. Alternative perceptions in virtual reality 269
5.3.1. Introduction 269
5.3.2. Pseudo-sensory feedback 271
5.3.3. Alternative perception of movement 275
5.3.4. Altered perception of one's body 278
5.3.5. Conclusion 283
5.4. Bibliography 284
Chapter 6. The Challenges and Risks of Democratization of VR-AR 289 Philippe FUCHS
6.1. Introduction 289
6.2. Health and comfort problems 292
6.2.1. The different problems 292
6.2.2. Sensorimotor incoherences . 293
6.3. Solutions to avoid discomfort and unease 297
6.3.1. Presentation of the process . 297
6.3.2. Mitigation of the impact on visuo-vestibular incoherence 297
6.3.3. Removing visuo-vestibular incoherence by modifying the functioning of the interaction paradigm 298
6.3.4. Removing visuo-vestibular incoherence by modifying interfaces 299
6.3.5. Levels of difficulty in adapting 299
6.4. Conclusion 300
6.5. Bibliography 301
Conclusion 303 Bruno ARNALDI, Pascal GUITTON and Guillaume MOREAU
Postface 309 Bruno ARNALDI, Pascal GUITTON and Guillaume MOREAU
Glossary 315
List of Authors 317
Index 321
Introduction
It can have escaped no one that 2016 and 2017 often features in the media as "The Time" for virtual reality and augmented reality. It is no less obvious that in the field of technology, many and regular breakthroughs are announced, each more impressive than the last. In the face of this media clamor, it is useful to step back and take a pragmatic look at some historical facts and information:
- - The first of these is the fact (however difficult to accept) that virtual reality and augmented reality date back several decades and that there is a large international community working on these subjects. This work is being carried out both at the scientific level (research teams, discoveries, conferences, publication) and at the industrial level (companies, products, large-scale production). It is also useful to remember that many companies, technological or not, have been successfully using virtual reality and augmented reality technologies for many years now.
- - Many of these technological announcements talk about the design of "new" virtual reality headsets (e.g. HTC Vive, Oculus Rift) and augmented reality headsets (e.g. HoloLens). But the fact is that the invention of the first "visioheadset"1 dates back to almost 50 years, to Ivan Sutherland's seminal work [SUT 68].
- - Let us also note that these "visioheadsets" only represent a small part of the equipment used in virtual reality, whether for display (with projection systems, for example), motion-capture or interaction.
- - The concept and applications of virtual reality are described in the series Le traité de la réalité virtuelle (The Virtual Reality Treatise), an encyclopedic volume produced collectively by many French authors (both academics and voices from the industry), the breadth and scope of which remains unmatched even today. The different editions of this are:
- - the first edition in 2001 (Presses de l'Ecole des Mines), written by Philippe Fuchs, Guillaume Moreau and Jean-Paul Papin with 530 pages;
- - the second edition in 2003 (Presses de l'Ecole des Mines), edited by Philippe Fuchs and Guillaume Moreau with help from 18 contributors, running to 930 pages in 2 volumes;
- - the third edition in 2005 (Presses de l'Ecole des Mines), edited by Philippe Fuchs and Guillaume Moreau, with over 100 contributors, running to 2,200 pages in 5 volumes;
- - an English version "Virtual Reality: Concepts and Technologies", in 2011 (CRC Press), edited by Philippe Fuchs, Guillaume Moreau and Pascal Guitton with 432 pages.
- - Finally, we must mention the creation of the "Association Française de Réalité Virtuelle" (AFRV) or the French Virtual Reality Association, established in 2005. The association has made it possible to structure the community better by bringing together teachers and researchers from universities and research institutions as well as engineers working within companies. From 2005 onward, the AFRV has been organizing an annual conference that sees presentations, activities and exchanges among participants.
As can be seen from this overview, there are already several communities at the international level as well as a wealth of literature on the subject and anyone who wishes to establish a scientific and/or technological culture will benefit from referring to publications such as [FUC 16] (in French) or [LAV 17, SCH 16], to mention a few.
I.1. The origins of virtual reality
When we talk about historic references relating to virtual reality, we may commence by discussing Plato's Allegory of the Cave [PLA 07]. In Book VII of Plato's Republic, there is a detailed description of the experiences of several men chained in a cave, who can only perceive shadows (thrown against the walls of the cave) of what happens in the outside world. The notion of reality and perception through what is and what is perceived becomes the subject of analysis, in particular concerning the passage from one world to another.
A few centuries later, in 1420, the Italian engineer Giovani Fontana wrote a book, Bellicorum instrumentorum liber [FON 20], in which he describes a magic lantern capable of projecting images onto the walls of a room (see Figure I.1(a)). He proposed that this could be used to project the images of fantastic creatures. This mechanism brings to mind the large immersion system (CAVE) developed a few centuries later by Carolina Cruz-Neira et al. [CRU 92] at the University of Illinois.
Figure I.1. a) Diagram of Giovani Fontana's magic lantern, b) using the magic lantern. For a color version of this figure, see www.iste.co.uk/arnaldi/virtual.zip
In books that recount the history of VR, we often come across the (legitimate) controversy around the first appearance of the term "virtual reality". Some authors attribute it to Jaron Lanier, during a press conference in 1985, while others attribute it to Antonin Artaud, in his 1983 essay, Le théâtre et son double (published in English as "The Theatre and its Double") [ART 09].
Artaud was unarguably the inventor of this term, which he used in his collection of essays on Theatre and, more specifically, in the chapter titled Le théâtre alchimique ("The Alchemical Theatre"). It must be noted that in this volume, Artaud talks at length about reality and virtuality (these words being frequently used in the text). The precise citation where the term "virtual reality" appears is on page 75 of the 1985 edition, collection Folio/essais de Gallimard:
"All true alchemists know that the alchemical symbol is a mirage as the theater is a mirage. And this perpetual allusion to the materials and the principle of the theater found in almost all alchemical books should be understood as the expression of an identity (of which alchemists are extremely aware) existing between the world in which the characters, objects, images and in a general way all that constitutes the virtual reality of the theater develop and the purely fictitious and illusory world in which the symbols of alchemy are evolved".
Furthermore, a few pages earlier, he speaks about Plato's Allegory of the Cave.
However, it is clear that Jaron Lanier was the first person to use this term in the sense that it is used in this book, when he used the English term virtual reality. It is also useful to remember that there is a subtle difference between the English term virtual and the French word virtuel (see Chapter 1, Volume 1 of the Virtual Reality Treatise, edition 3). In English, the word means "acting as" or "almost a particular thing or quality". However, in French, the word indicates "potential", what is "possible" and what "does not come to pass". Linguistically speaking, the more appropriate French term would have been "réalité vicariante" - a reality that substitutes or replaces another.
Science-fiction writers, especially those writing in the "speculative fiction" genre (a genre which, as its name indicates, consists of imagining what our world could be like in the future) have also written books that integrate and/or imagine the VR-AR technologies we will discuss in this volume. The list of such books is quite long, and the four books presented here have been chosen simply for the impact they had. In chronological order, these are:
- - Vernor Vinge, in his 1981 novella True Names, introduced a cyberspace (without explicitly naming it thus), where a group of computer pirates use virtual reality immersion technology to fight against the government. He is also the creator of the concept of "singularity": that point in time when machines will be more intelligent than human beings;
- - William Gibson, in his 1984 novel Neuromancer, described a world of networks where virtual reality consoles allow a user to live out experiences in virtual worlds. Gibson "invented" the term cyberspace, which he described as "a consensual hallucination experienced daily by billions of legitimate operators". This concept of cyperspace spans different worlds: the digital world, the cybernetic world and the space in which we evolve;
- - Neal Stephenson, in his 1992 novel Snow Crash, introduced the concept of the metaverse (a virtual, thus fictional, world in which a community, represented by avatars, is evolving); a universe like the one in the online virtual world Second Life;
- - Ernest Cline, in his 2011 novel Ready Player One, offerred us a world where humanity lives in an enormous virtual social network to escape the slums in real life. This network also contains the key to riches, leading to a new kind of quest for the holy grail.
Literature is not the only field in which early references to virtual reality set up links between the real and the virtual. For example, we must mention the pioneering work of Morton Leonard Heilig in the world of cinema. Following a project he had worked on since the 1950s, he patented the Sensorama system in 1962. This system allowed users to virtually navigate an urban setting on a motorbike, in an immersive experience based on stereoscopic visualization, the sounds of the motorbike and by reproducing the vibration of the engine and the sensation of wind...
