Introduction

This book is dedicated to deep learning, which is a recent branch of a slightly older discipline: machine learning.1 Deep learning is particularly well suited to the analysis of complex data, such as images and natural language. For this reason, it is at the heart of many of the artificial intelligence applications that we will describe in this book. Although deep learning today relies almost exclusively on neural networks, we will first look at other machine learning methods, partly because of the concepts they share with neural networks and which it is important to understand in their generality, and partly to compare their results with those of deep learning methods. We will then be able to fully measure the effectiveness of deep learning methods in computer vision and automatic natural language processing problems. This is what the present book will do, recalling the theoretical foundations of these methods while showing how to implement them in concrete situations, with examples treated with the open source deep learning libraries of Python and mainly R, as indicated below. As we will see, the prodigious development of deep learning and artificial intelligence has been made possible by new theoretical concepts, by more powerful computing tools, but also by the possibility of using immense masses of various data, images, videos, audios, texts, traces on the Internet, signals from connected objects ... these big data will be very present in this book.

The Structure of the Book

Chapter 1 is an overview of deep learning and big data with their principles and applications in the main sectors of finance, insurance, industry, transport, medicine, and scientific research. A few pages are devoted to the main difficulties that can be encountered in processing data in machine learning and deep learning, particularly when it comes to big data. We must not neglect the IT risks inherent in the collection and storage, sometimes in a cloud, of large amounts of personal data. The news about certain social networks regularly reminds us of this. At the opposite end of the spectrum from their commercial vision of big data are open data, which closes the chapter.

Chapter 2 deals with concepts that data scientists must know when dealing with large volumes of data: parsimony in modeling, algorithmic complexity, parallel computing and its generalization, which is distributed computing. We devote a few pages to the MapReduce algorithm at the basis of distributed computing, its implementation in the Hadoop system, and to the database management systems, known as NoSQL and column-oriented, particularly adapted to big data. We will see that "analytical" applications such as machine learning have particular computing requirements that require specific solutions: Spark is one of them. We then review the hardware and software resources to be implemented, whether they are on the user's machine or in a cloud. We talk about the processors that enable deep learning computations to be accelerated, as well as the two most used open source software in statistics, machine learning, and deep learning: R and Python. A synoptic table compares the main machine learning methods implemented in R, Python (scikit-learn library) and Spark (MLlib). We also found it interesting to mention quantum computing, for which specific versions of algorithms are starting to be designed, notably in linear algebra, machine learning, optimization, and cryptography. The prospects of quantum computing are still distant but very promising, with the possibility of a considerable reduction in computing time.

Chapter 3 recalls some essential principles of machine learning and data science: the bias-variance dilemma in modeling, complexity reduction methods, optimization algorithms, such as gradient descent, Newton or Levenberg-Marquardt, ensemble (or aggregation) methods by random forests, Extra-Trees or boosting, and useful methods for big data, such as incremental algorithms and recommendation systems used by social networks and online commerce. Apart from these reminders, it is assumed that the reader is familiar with machine learning methods but, if required, a bibliography is given at the end of the book and notes are provided in each chapter for specific references.

Chapter 4 presents natural language processing methods. The principles of textual analysis are introduced, including segmentation into units or tokenization, part-of-speech tagging, named entity recognition, lemmatization, and other simplification operations that aim to reduce the volume of data and the complexity of the problem as much as possible while retaining the maximum amount of information, which is a constant concern in statistics and machine learning. We then describe the operations of vector representation of words, which go from the classical document-term matrix to the methods of word embedding, which started with Word2Vec, GloVe, and fastText, and the list of these is continuously growing. We speak of embedding because each word is associated with a point in a vector space of fairly small dimensions, of the order of a few hundred, i.e. much less than different terms, with the remarkable property that two semantically close words correspond to close points in the vector space, and that arithmetic operations in this vector space can lead to identities such as "King" - "Man" + "Woman" = "Queen". These vector embeddings preserve not only the proximity of words but also their relations. They are therefore an efficient way to transform documents for analysis, for example, to classify them into categories: spam or non-spam, type of message, subject of the complaint, etc. We also discuss topic modeling, which uses methods such as latent Dirichlet allocation to detect all the topics present in a corpus of documents. We present another current method of natural language processing, sentiment analysis, which seeks to detect the sentiments expressed in a text, either in a binary form of positive or negative sentiments, or in a more elaborate form of joy, fear, anger, etc. Neural methods applied to natural language processing are discussed in Chapter 9, after the one devoted to the principles of deep learning.

Chapter 5 shows how to analyze social networks, starting from the notions of graph theory and taking the example of Twitter. We are particularly interested in the so-called centrality and influence measures, as they are very important in social networks and web search engines. We are also interested in the detection of communities, which are the dense sub-graphs that can constitute a partition of the studied graph. The search for communities in a graph is an active field of research, in various domains (biology, sociology, marketing), because the vertices of a same community tend to have in common interesting properties. Some considerations are turned to the economic model of social networks and to digital advertising and what is called programmatic advertising.

Chapter 6 deals with the classical problem of recognizing handwritten digits on bank checks and postal codes on envelopes, among others. On a well-known dataset (MNIST), it compares the different machine learning methods previously discussed in the book: in particular penalized regression, random forests, gradient boosting, and support vector machines.

Chapter 7 is a long and important chapter on deep learning. It explains the principles of deep learning and the architecture of deep neural networks, especially convolutional and recurrent networks, which are those most widely used for computer vision and natural language processing today. The many features designed to optimize their performance are presented, such as pooling, normalization, dropout, and adaptive learning, with indications on how best to use them. We review the fundamental learning mechanism of neural networks, backpropagation, the difficulties encountered in its application to multilayer networks with the vanishing gradient phenomenon that led for a while to the "winter of artificial intelligence," and the solutions found in the last ten years by new ideas and increased computing power. Particular networks are described: autoencoders for data compression, and generative neural networks that are increasingly being developed to have artificial intelligence produce texts, images or music. Illustrations show the interest of deep learning for subjects ranging from object detection to strategy games.

Chapter 8 presents the application to computer vision of the methods seen in Chapter 7, using MXNet, Keras-TensorFlow, and PyTorch libraries. In particular, they are applied to three classical datasets: (1) the MNIST database already discussed in Chapter 6, which allows the performances of classical and deep learning methods to be compared; (2) the CIFAR-10 image database; and (3) a database of cat and dog pictures. We apply transfer learning. We sketch the question of the explicability of machine learning algorithms by applying the LIME method to images to find out on which parts of the image the model relies for its predictions. We show how to configure a computer with a Windows operating system to use its graphics processing unit (GPU) for deep learning computations, which are much faster on these graphics processors than on classical processors (CPUs). This configuration is not very simple and it is necessary to follow the different steps indicated. The chapter concludes with examples of cloud computing, using the Google Colab platform with a Jupyter notebook running Python code.

Chapter 9 returns to natural language...