1
Processors for the Internet of Things

Pieter VAN DER WOLF1 and Yankin TANURHAN2

1Solutions Group, Synopsys, Inc., Eindhoven, The Netherlands

2Solutions Group, Synopsys, Inc., Mountain View, USA

The Internet of Things (IoT) enables a "smart world" in which many billions of devices communicate to provide advanced functionalities. More specifically, a broad variety of IoT edge devices that can "sense", "listen" and "see" is emerging to capture data for further processing and communication. Such devices have a broad range of compute requirements for implementing a mixture of control processing, digital signal processing (DSP), machine learning, security, etc. In this chapter, we analyze the compute requirements of IoT edge devices and discuss processor capabilities that support the efficient implementation of such devices. More specifically, we focus on IoT edge devices that demand low power consumption. We present concrete examples of versatile, configurable and extensible processors that provide capabilities for control processing, DSP (e.g. voice/audio processing, communications) and machine learning. The processor examples are complemented with benchmark data for illustrative IoT edge application functions.

1.1. Introduction

In recent years, computing paradigms have evolved significantly. One such paradigm is cloud computing, a centralized paradigm that aims to offer computing as a utility. Another complementary paradigm is edge computing, a decentralized paradigm that aims to offer smart compute capabilities at the edge of the network. A related term is the Internet of Things (IoT), which refers to large numbers of interconnected computing devices aimed at offering services to a great variety of applications.

In this chapter, we specifically focus on IoT edge devices: smart devices at the edge of the network that interact with the "real world". These devices acquire data from the environment using sensors. This data is subsequently processed locally on the IoT edge device and/or on computing devices in the network. For each application, a proper trade-off must be made about which functions to perform where based on the requirements for computing, bandwidth, latency, connectivity, security, reliability, etc.

The number of IoT edge devices is predicted to grow to tens of billions over the coming years. Some example IoT edge devices are:

• - smartphones and tablets;
• - smart doorbells with cameras, performing face detection for triggering an alert, accompanied by an image or video, on the owner's smartphone;
• - smart speakers with voice control, employing local speech recognition for a limited vocabulary of voice commands while relaying other speech data into the cloud for more advanced analysis;
• - smart sensing devices used in agriculture to monitor and control, for example, soil quality, crop yield and livestock, while sporadically communicating data over cellular connections using, for example, NB-IoT protocols for low power consumption.

Many IoT edge devices are battery-operated and demand an optimized implementation in order to enable a long battery life. Therefore, we must target low power consumption for functions that need to be performed in software locally on the IoT edge device. This, in turn, requires programmable processors that are optimized for executing these software functions efficiently, which is the topic of this chapter.

1.2. Versatile processors for low-power IoT edge devices

1.2.1. Control processing, DSP and machine learning

Low-power IoT edge devices typically perform a range of different functions locally on the device. They run a local application that controls the device, its sensors and other interfaces, such as a communications interface to the network and a user interface. For this purpose, a processor must have capabilities for efficient processing of control code, including low branch overheads, efficient interrupt handling, timers, efficient integration with peripherals, support for real-time kernels, etc.

Furthermore, IoT edge devices typically perform some processing on the data acquired through their sensors. These can be sensors to monitor physical phenomena, such as thermometers, gyroscopes, accelerometers and magnetometers. Let us consider, for example, a personal health device or the smart sensing devices used in agriculture, mentioned above. Data rates for this type of sensors are typically low. Microphones are another type of sensors that have higher data rates. For example, a 16 kHz sample rate is often used for voice data. Even higher data rates can be observed in IoT edge devices that use a camera, such as a smart doorbell performing face detection. Data rates for image and video data can vary largely, based on resolution and frame rates. Data rates of hundreds of MB/s are not unusual in high-end devices, but for more power-sensitive camera-based applications, much lower data rates can be observed.

The processing of sensor data typically involves digital signal processing (DSP) with functions such as filtering (e.g. FIR, correlation, biquad), transforms (e.g. FFT, DCT), and vector and matrix operations. Voice data can be processed by various DSP functions, including noise reduction and echo cancellation. In addition, the IoT edge device can perform encoding and/or decoding of voice or audio data. For example, consider an audio playback function on the device.

Communicating data involves further DSP functions. For example, some key functions in an NB-IoT protocol stack involve FFT, auto- and cross-correlations, and complex multiplications and convolutions. Furthermore, trigonometric functions such as sine and cosine must be performed. In addition, such protocol stacks perform convolutional coding, for example, Viterbi.

We conclude that the efficient processing of sensor data on an IoT edge device requires processors equipped with DSP capabilities. The relevant DSP capabilities are:

• - support for fixed-point data types and arithmetic, including fixed-point multiply-accumulate (MAC) instructions, wide accumulators, and efficient saturation and rounding;
• - support for floating-point data types and instructions, including fused multiply-add instructions;
• - advanced address generation for efficient memory access, including circular and bit-reversed addressing for DSP kernels such as FIR filters and FFTs;
• - zero-overhead loops;
• - support for complex data types and arithmetic, including complex multiply and MAC instructions;
• - support for vector or SIMD processing to enable increased efficiency by exploiting data parallelism;
• - efficient divide and square root operations;
• - high load/store bandwidth, as DSP functions can be memory-access intensive.

In addition to control processing and DSP, machine learning has recently emerged in various application areas as a technology for building IoT edge devices with advanced functionalities. Some illustrative examples are smart speakers, wearable activity trackers and smart doorbells. These devices apply machine learning technology that has been trained to recognize certain complex patterns (e.g. voice commands, human activity, faces) from data captured by one or more sensors (e.g. a microphone, a gyroscope, a camera). When such a pattern is recognized, the device can perform an appropriate action. For example, when the voice command "play music" is recognized, a smart speaker can initiate the playback of a song. In the following sections, we dig deeper into the requirements and processor capabilities for efficient machine learning in low-power IoT devices.

Integrated circuits for low-power IoT edge devices may use one or more processors for implementing the different types of processing. Multiple processors are required if a single processor cannot handle the complete software workload. A further reason for using multiple processors is that specialized processors can be used for the different types of processing. More specifically, different processors can be used for control processing, DSP and machine learning.

However, there are also good reasons to aim to reduce the number of processors. Lower cost is a key benefit, which is particularly relevant for low-cost IoT edge devices that are produced in high volumes. The use of fewer processors also reduces design complexity, as it simplifies the interconnect and memory subsystem required to integrate the processors. Furthermore, if multiple interacting functions are combined to be executed on a single processor, then this will limit data movements and reduce the software overhead for communication. An additional benefit for software developers is that a single tool chain can be used. To enable the flexible combination of functions, we need versatile processors that can efficiently execute different types of workloads, including control tasks, DSP and machine learning. Such processors are also referred to as DSP-enhanced RISC cores. They add a broad set of instructions for DSP and machine learning to a RISC core. If done well, the hardware overhead of these additions is small, for example, by sharing the register file and having unified functional units (e.g. a multiplier) for control processing, DSP and machine learning. Today, optimized DSP-enhanced RISC cores are...