Chapter 2

Problem 1

From Touch to Feeling

Abstract

This Problem chapter, and the four Opportunity chapters that follow, explore and argue for mobile user experiences that truly recruit our human senses and abilities. Through the focus areas of food, fashion, fitness, and materials, we challenge you to reconsider apps that simply focus on glassy touch screens. Instead, we demonstrate how more engaging, multisensory, physical designs can enliven the user experience.

Keywords

touch; feeling; physicalityemotions What's the Problem? Digital interactions through mobiles are an increasingly prominent part of day-to-day lived experience. But what are they doing to the richness of this everyday life? Our starting point, in this first Problem, is to pause for a moment and think about the extent to which the smooth glass of our phones, which separates us from the digital world inside, numbs or dulls, rather than enlivens. As you'll see as you read on, this book is a celebration of what it is to be truly alive-to revel in the complexity, ambiguity, messiness, and stimulation the world provides. WHY SHOULD YOU TACKLE IT? If we look away from our interactions with gadgets, we see inspirations for what mobile experiences might be both now and in the future. We see a world of multisensory beings that taste, smell, see, and feel the world. Sometimes we are hit with a double espresso jolt of life-think of the pain of falling off a bike; other times we feel it much more subtly-as a gentle breeze brushes the hairs on the back of your neck. We live in a world where emotion is as important as efficiency. We also experience a world that we can shape and manipulate through an equally broad spectrum of actions: from demolishing a wall with a sledgehammer to creating beautiful origami with deft finger folds. Our challenge to you here, then, is to consider how these human skills can be put to better use, and inform the interaction designs we make both today and on the devices to come. KEY POINTS ? "Touch," as in "touch screen," is a limited design resource compared to what humans are capable of in terms of the ways we can sense, respond, and manipulate. ? We have been built for physical materials; digital materials currently lack many qualities to enable us to fully engage with them. ? When we think about the physical world, we are reminded that not every interaction is pleasant, calming, and joyful. Facing up to a spectrum of emotional responses can introduce new thinking to interaction design. ? Research labs and visionary designers have been exploring how to break through the glass to create digital experiences that engage better with these multisensory, emotional, and multimanipulator abilities.

Introduction

Have you ever walked into a glass door? It's a shocking, dazing experience. The shock comes from the sudden, unexpected impact-one moment you are striding, unhindered, the world seemingly visible to you; the next you are stopped suddenly by the unseen barrier. You reel, perhaps curse, and after a while continue onwards. Contrast the numbness from the glass door collision to having your senses fully stimulated. Think back to the last time you sunbathed on a beach. Lying there, you felt the warmth of the sun on your skin, noticing when comfortable heat began to turn to painful burn. Perhaps you dug your hands into the sand, rubbing the grains between your fingers, sensing the gritty texture. Despite your eyes being closed, you felt fully aware of the scene around you, listening to children splash close by, the gossipy chatter of neighbors anchored to beach chairs, the cawing of seagulls. You were alive in and alive to the place. Or, if you are a more active sort, that feeling of vitality comes when you do sport or exercise. If you are a runner you'll know that feeling, halfway through a long route, where your legs fill with lactic acid, your heart and lungs feel like they might rip from your chest, and your eyes water with the exertion. Painfully, exhilaratingly alive. Touch screens on smartphones have transformed mobile user experiences. They are widely seen as providing for diverse, deft manipulations: selections, pinches, swipes, magnifications, and so on. Rather than congratulating each other on the wonders of this technology, though, we want to provoke you. Design Challenge What is the danger of these glass lids? Are they actually blunting our senses? Search for Bret Victor brief rant Each time we prod the screen with our fingers, it is as if we are walking into a glass door. Every tap is a micro-moment that dazes us. Bret Victor, the well-known interaction designer and blogger, calls this interaction style "pictures under glass," worrying, as we do, about how the paradigm denies us a richer, sense-ful interaction. Modern glass doors and barriers have markings at eye level to warn the walker of danger (see Figure 2.1). What we'll be doing in this chapter is to act as the warning dots. We will highlight how impoverishing simple touch interaction is in comparison to real life, and point to alternatives. We pose this question: how might we build experiences that allow us to feel fully alive? To move, that is, from touch screens to technologies and designs that encourage us to use all our senses to feel and connect to the world.
Figure 2.1 Warning! Glass can blunt experience. Alternative perspective You might be thinking about objections to our focus here: surely mobile devices and their apps and services don't have to provide such an immersive set of experiences? After all, do we expect other tools (say a pencil or a pad of paper, or a cooking pot and a stirring spoon) to make us feel so alive? Surely, mobiles are just another object, and richer, visceral living can be left to other activities, like sunbathing or biking?

Built as bodies, built for materials

Ben is one of Matt's children. He's 12, and for a few years has been developing a wonderful talent in creating origami models (see Figure 2.2 for an example). Dexterously he turns small, simple pieces of paper into dragons, boxes, or frogs.
Figure 2.2 Ben's origami. We are built to manipulate, shape, and rearrange the world physically. Of course we use our hands, but we also "head" soccer balls, rush through leaves, kicking them into a storm with our feet, and deform beanbags with our backsides as we drop lazily down onto them. "With an entire body at your command, do you seriously think the Future Of Interaction should be a single finger?" Bret Victor Human-computer interaction researchers and designers have recognized the drive to get physical for decades. Beginning with the shift away from command-line interfaces-the blinking cursor on a screen that tells you nothing of the computer world beneath-to graphical user interfaces (GUIs), suddenly computers were turned from the foreign to the more familiar: there were pictures of documents, trashcans, and filing cabinets that the user could "pick up" and "move" using a mouse and keyboard. While lots of today's interactions are much like the early GUIs, the mouse being replaced by our fingers, we've also seen a move to more tangible and direct interfaces. For many of us, at the moment, we are most likely to encounter these in gaming consoles, where we might hold a controller and use it as a baseball bat or ski pole, or stand with our arm outstretched imagining we are holding a bowling ball. There are other specialized applications too, such as systems to train surgeons to perform intricate surgery, holding an instrument that provides realistic force feedback as the trainee probes a simulated abdomen. Back to Ben and another activity he loves to help with: cooking. We were finishing a soup making session, heating and stirring the final product together. Our tools were a pan and wooden spoon, and as he ladled the spoon through the liquid, it bubbled, occasionally hissing and spitting small flecks as it swirled over the hottest sides of its container. The soup was a highly responsive material, and one that engaged all our senses. We could see it; smell it; feel its texture-even our eyes watered as the vapors of garlic and onion hit their membranes. We are built for materials like this. Then, suddenly, there was panic. The pan spilt, and searingly hot soup hit Ben's legs. Wearing short trousers he was immediately in pain, the soup triggering the reflex to pull away. His skin started to blister, one material responding with urgency to the scalding temperature of the other. Running cold water over the burn, the skin began to cool, the dangerous heat soothed. Though pain is a highly discomforting experience, it's the body's way of protecting us from danger. The way Ben's body responded to the scalding soup and the cooling water starkly illustrates how we've evolved to cope with physical materials: we've been built for them. The Uncomfortable interactions box below considers how our ability to feel discomfort might expand our notions of user...