Introduction

Spatial Computing comprises all software and hardware technologies that enable humans, virtual beings, or robots to move through real or virtual worlds, and includes Artificial Intelligence, Computer Vision, Augmented Reality (AR), VR, Sensor Technology, and Automated Vehicles.

Seven industry verticals will see transformational change due to Spatial Computing: Transportation; Technology, Media, and Telecommunications (TMT); Manufacturing; Retail; Healthcare; Finance; and Education.

These changes are what is driving strategy at many tech companies and the spending of billions of dollars of R&D investment. Already, products such as Microsoft's HoloLens AR headset have seen adoption in places from surgery rooms to military battlefields. Devices like this show this new computing paradigm, albeit in a package that's currently a little too bulky and expensive for more than a few of the hardiest early adopters. However, these early devices are what got us to be most excited by a future that will be here soon.

Our first experiences with HoloLens, and other devices like it, showed us such a fantastic world that we can predict that when this new world arrives in full, it will be far more important to human beings than the iPhone was.

We were shown virtual giant monsters crawling on skyscrapers by Metaio years ago near its Munich headquarters. As we stood in the snow, aiming a webcam tethered to a laptop at the building next door, the real building came alive thanks to radical new technology. What we saw at Metaio had a similar effect on Apple's CEO, Tim Cook. Soon, Apple had acquired Metaio and started down a path of developing Augmented Reality and including new sensors in its products, and the capabilities of this are just starting to be explored. Today's phones have cameras, processors, small 3D sensors, and connectivity far better than that early prototype had, and tomorrow's phones and soon, the glasses we wear will make today's phones seem similarly quaint.

In Israel, we saw new autonomous drones flying over the headquarters of Airobotics. These drones were designed to have no human hands touching them. Robots even changed memory cards and batteries. New Spatial Computing technology enabled both to "see" each other and sense the world around us in new ways. The drones were designed to fly along oil pipelines looking for problems, and others could fly around facilities that needed to be watched. Flying along fences and around parking lots, their Artificial Intelligence could identify things that would present security or other risks. These drones fly day or night and never complain or call in sick.

Focusing on just the technology, though, would have us miss what really is going to happen to the world because of these technologies. Our cities and countryside will reconfigure due to automation in transportation and supply chains as robot tech drives our cars, trucks, and robots rolling down sidewalks delivering products. We'll spend more time in virtual worlds and metaverses. More of our interfaces, whether they are the knobs on our watches, cars, doors, and other devices, will increasingly be virtualized. In fact, many things that used to be physical may be virtualized, including stores and educational lessons from chemistry experiments to dissection labs.

Computing will be everywhere, always listening, always ready to talk back, and once we start wearing Spatial Computing glasses, visual computing will always be ready to show us visualizations of everything from your new designs to human patterns in stores, on streets, and in factories and offices. Some call this "invisible computing" or "ambient computing," but to us, these systems that use your eyes, voice, hands, and even your body as a "controller" are part of Spatial Computing.

At the same time, all this new computing is joined by radically fast new wireless technology in the form of 5G. The promises of 5G are threefold. First, we'll have more than a gigabit of data if we have the highest bitrates, and even the lowest rates promise to give us more bandwidth than current LTE phones. Second, wireless will soon add almost no latency, which means that as soon as you do something, like throw a football in a virtual game, it'll happen, even if you have many players viewing the football in real time. Third, 5G supports many more devices per tower, which means you will be able to live stream even a Taylor Swift concert with tens of thousands of fans filling a stadium.

When you combine 5G with all the new things under the Spatial Computing umbrella, you get a big bang. All of a sudden, cars can work together, where one of the cars will be sending detailed 3D imaging of streets instantly to other cars behind it. New kinds of virtual games will be possible in the streets, where hundreds of people can play virtual football games in parks and other places. Crazy new virtual shopping malls appear where virtual celebrities show you around, and you can imagine new products directly in a virtual scan of your home and in other places as well.

A range of new capabilities will appear over the next few years in devices you wear over your eyes. There will be very light ones that are optimized for utility, showing you navigation, notifications, reminding you where you have left things, or nagging you to do some exercise or meditation to keep on top of your physical and mental health. There also will be heavier devices that will be more optimized for everything from detailed design or architecture work to entertainment and video game work. We can even imagine owning several different kinds of Spatial Computing devices, along with some smart contact lenses, that will let us go out on a date night without looking like we have any computing devices on at all.

We currently have VR devices that cost a few hundred dollars that are great for games and a few other things, like corporate training. On the more expensive side of the scale, we have devices that can be used by car designers, or even as flight simulators to train airline pilots. The expensive ones, though, will soon look as out of date as one of the first cell phones does today. By 2028, the computing inside will shrink to a fraction of the size of today's devices and the screens inside will be much sharper and capable of presenting virtual and augmented worlds to us that far exceed what we can experience today.

It is this next wave of devices that will usher in the paradigm shift in computing and in human living that we are discussing here. Already these changes are benefiting many enterprises, raising productivity. Inside many warehouses, hundreds of thousands of robots scurry about, moving products from trucks to packages. These new warehouses have evolved over the past decade and enable retailers to keep up with floods of new online orders that, back in 2000, were only dreamt of in futuristic books like this one.

The productivity gains will spread to many jobs. At Cleveland Clinic, surgeons are already using similar technology that shows them digital views from ultrasound, CAT scans, and other sensors. Like the warehouse worker who sees a blue line on the floor telling her how to find the product she's looking for, in this case, when a surgeon navigates to the right place to cut out a cancerous tumor, it lights up like a missile guidance system and tells the surgeon they are in the right place.

Other systems help workers "phone a friend" with new remote assistance features. This can help companies that have expensive machinery, or other work forces, including surgeons, architects, and engineers, save money. At some plants, the savings will be substantial. It took us 30 minutes to simply walk across the Boeing floor where it builds airliners. Asking someone for advice virtually might save someone an hour of walking just to come over and see your problem in a plant like that.

New devices let these remote helpers see what you are dealing with, and they can often show you visually what to do. Imagine trying to remove an engine while holding a phone or tablet in your hand. These systems, because they use wearable glasses, can let workers use both of their hands while talking and showing the remote assistant what is happening. The savings in downtime can be extreme. Imagine a problem that is causing a shutdown in a line at Ford. Every minute it is down, Ford loses about $50,000.

Even for salespeople and managers, the cost savings add up. A flight from San Francisco to Los Angeles usually costs about $700, including airfare, hotel, a decent meal, and an Uber/taxi ride or two. Increasingly, these meetings will be replaced by ones held in Virtual Reality. Two headsets are less than $800. In just a couple of virtual meetings, the headsets pay for themselves. Social VR from Facebook, Spatial, Microsoft, and others, are rapidly improving to make these virtual meetings almost as good as physical ones. The time and cost saved will add up to big numbers, and workers will be happier. Workforces will be less likely to pick up viruses from travelers too, which will also add up to big savings for corporations and reduced risks.

More lives could be saved, too. Mercedes-Benz had an Augmented Reality system built to show first responders how to cut apart a wrecked car. The app showed where fuel and electrical lines were so that firefighters working to free an accident victim wouldn't start a fire or electrocute themselves. This isn't the only example we have of this technology helping first responders, either through better training or by giving them various assistance on scene. One such system helps police gather evidence and then be able to recreate crime scenes for...