Machine Learning and AI with Simple Python and Matlab Scripts
Description
A practical guide to AI applications for Simple Python and Matlab scripts
Machine Learning and AI with Simple Python and Matlab Scripts: Courseware for Non-computing Majors introduces basic concepts and principles of machine learning and artificial intelligence to help readers develop skills applicable to many popular topics in engineering and science. Step-by-step instructions for simple Python and Matlab scripts mimicking real-life applications will enter the readers into the magical world of AI, without requiring them to have advanced math and computational skills. The book is supported by instructor only lecture slides and sample exams with multiple-choice questions.
Machine Learning and AI with Simple Python and Matlab Scripts includes information on:
- Artificial neural networks applied to real-world problems such as algorithmic trading of financial assets, Alzheimer's disease prognosis
- Convolution neural networks for speech recognition and optical character recognition
- Recurrent neural networks for chatbots and natural language translators
- Typical AI tasks including flight control for autonomous drones, dietary menu planning, and route planning
- Advanced AI tasks including particle swarm optimization and differential and grammatical evolution as well as the current state of the art in AI tools
Machine Learning and AI with Simple Python and Matlab Scripts is an accessible, thorough, and practical learning resource for undergraduate and graduate students in engineering and science programs along with professionals in related industries seeking to expand their skill sets.
More details
Other editions
Additional editions
Person
M. Ümit Uyar is a Professor at the City College of the City University of New York, USA. Dr. Uyar is an IEEE Fellow, author, co-author and co-editor of seven books, holder of seven U.S. patents, and developer of AI and game theory-based algorithms for applications in topology control in mobile networks and personalized cancer treatment.
Content
About the Author xiii
Preface xv
Acknowledgments xvii
About the Companion Website xix
1 Introduction 1
1.1 Artificial Intelligence 1
1.2 A Historical Perspective 1
1.3 Principles of AI 2
1.4 Applications That Are Impossible Without AI 2
1.5 Organization of This Book 3
2 Artificial Neural Networks 7
2.1 Introduction 7
2.2 Applications of ANNs 7
2.3 Components of ANNs 8
2.3.1 Neurons 8
2.3.2 Sigmoid Activation Function 9
2.3.3 Rectilinear Activation Function 9
2.3.4 Weights of Synapses 10
2.4 Training an ANN 11
2.5 Forward Propagation 12
2.5.1 Forward Propagation from Input to Hidden Layer 13
2.6 Back Propagation 13
2.6.1 Back Propagation for a Neuron 13
2.6.2 Back Propagation - from Output to Hidden Layer 15
2.6.3 Back Propagation - from Hidden Layer to Input 16
2.7 Updating Weights 17
2.8 ANN with Input Bias 17
2.9 A Simple Algorithm for ANN Training 18
2.10 Computational Complexity of ANN Training 18
2.11 Normalization of ANN Inputs and Outputs 19
2.12 Concluding Remarks 20
2.13 Exercises for Chapter 2 20
3 ANNs for Optimized Prediction 23
3.1 Introduction 23
3.2 Selection of ANN Inputs 24
3.3 Selection of ANN Outputs 24
3.4 Construction of Hidden Layers 25
3.5 Case Study 1: Sleep-Study Example 25
3.5.1 Using Matrices for ANN Training 26
3.5.2 Forward Propagation 28
3.5.3 Back Propagation 28
3.5.4 Updating Weights 29
3.5.5 Forward Propagation with New Weights 29
3.5.6 Back Propagation with New Weights 30
3.5.7 Using Normalized Input and Output Values 31
3.5.8 Reducing Errors During Training 34
3.5.9 Implementation of Sleep-Study ANN in Python 34
3.5.10 Implementation of Sleep-Study ANN in Matlab 37
3.6 Case Study 2: Prediction of Bike Rentals 41
3.6.1 Python Script for Bike Rentals Using an ANN 41
3.6.2 Matlab Script for Bike Rentals Using an ANN 46
3.7 Concluding Remarks 48
3.8 Exercises for Chapter 3 48
4 ANNs for Financial Stock Trading 51
4.1 Introduction 51
4.2 Programs that Buy and Sell Stocks 51
4.3 Technical Indicators 51
4.3.1 Simple Moving Average 52
4.3.2 Momentum 53
4.3.3 Exponential Moving Average 54
4.3.4 Bollinger Bands 54
4.4 A Simple Algorithmic Trading Policy 55
4.5 A Simple ANN for Algorithmic Stock Trading 57
4.5.1 ANN Inputs and Outputs 57
4.5.2 ANN Architecture 58
4.6 Python Script for Stock Trading Using an ANN 59
4.7 Matlab Script for Stock Trading Using an ANN 63
4.8 Concluding Remarks 65
4.9 Exercises for Chapter 4 65
5 ANNs for Alzheimer's Disease Prognosis 67
5.1 Introduction 67
5.2 Alzheimer's Disease 67
5.3 A Simple ANN for AD Prognosis 68
5.4 Python Script for AD Prognosis Using an ANN 71
5.5 Matlab Script for AD Prognosis Using an ANN 75
5.6 Concluding Remarks 80
5.7 Exercises for Chapter 5 81
6 ANNs for Natural Language Processing 83
6.1 Introduction 83
6.2 Impact of Text Messages on Stock Markets 84
6.3 A Simple ANN for NLP 85
6.3.1 ANN Inputs and Outputs 85
6.3.2 Keywords 85
6.3.3 Formation of Training Data 86
6.3.4 ANN Architecture 88
6.4 Python Script for NLP Using an ANN 89
6.5 Matlab Script for NLP Using an ANN 92
6.6 Concluding Remarks 96
6.7 Exercises for Chapter 6 97
7 Convolutional Neural Networks 99
7.1 Introduction 99
7.1.1 Training CNNs 100
7.2 Variations of CNNs 101
7.3 Applications of CNNs 101
7.4 CNN Components 102
7.5 A Numerical Example of a CNN 102
7.6 Computational Cost of CNN Training 108
7.7 Concluding Remarks 112
7.8 Exercises for Chapter 7 112
8 CNNs for Optical Character Recognition 115
8.1 Introduction 115
8.2 A Simple CNN for OCR 115
8.3 Organization of Training and Reference Files 117
8.4 Python Script for OCR Using a CNN 119
8.5 Matlab Script for OCR Using a CNN 124
8.6 Concluding Remarks 130
8.7 Exercises for Chapter 8 130
9 CNNs for Speech Recognition 133
9.1 Introduction 133
9.2 A Simple CNN for Speech Recognition 134
9.3 Organization of Training and Reference Files 136
9.4 Python Script for Speech Recognition Using a CNN 138
9.5 Matlab Script for Speech Recognition Using a CNN 144
9.6 Concluding Remarks 150
9.7 Exercises for Chapter 9 150
10 Recurrent Neural Networks 151
10.1 Introduction 151
10.2 One-to-One Single RNN Cell 153
10.2.1 A Simple Alphabet and One-Hot Encoding 156
10.2.2 Forward and Back Propagation 157
10.3 A Numerical Example 158
10.4 Multiple Hidden Layers 163
10.5 Embedding Layer 165
10.5.1 Forward and Back Propagation with Embedding 167
10.5.2 A Numerical Example with Embedding 168
10.6 Concluding Remarks 172
10.7 Exercises for Chapter 10 172
11 RNNs for Chatbot Implementation 175
11.1 Introduction 175
11.2 Many-to-Many RNN Architecture 175
11.3 A Simple Chatbot 176
11.4 Python Script for a Chatbot Using an RNN 179
11.5 Matlab Script for a Chatbot Using an RNN 183
11.6 Concluding Remarks 188
11.7 Exercises for Chapter 11 189
12 RNNs with Attention 191
12.1 Introduction 191
12.2 One-to-One RNN Cell with Attention 191
12.3 Forward and Back Propagation 193
12.4 A Numerical Example 195
12.5 Embedding Layer 200
12.6 A Numerical Example with Embedding 202
12.7 Concluding Remarks 207
12.8 Exercises for Chapter 12 207
13 RNNs with Attention for Machine Translation 209
13.1 Introduction 209
13.2 Many-to-Many Architecture 210
13.3 Python Script for Machine Translation by an RNN-Att 211
13.4 Matlab Script for Machine Translation by an RNN-Att 216
13.5 Concluding Remarks 223
13.6 Exercises for Chapter 13 223
14 Genetic Algorithms 225
14.1 Introduction 225
14.2 Genetic Algorithm Elements 226
14.3 A Simple Algorithm for a GA 227
14.4 An Example of a GA 230
14.5 Convergence in GAs 231
14.6 Concluding Remarks 232
14.7 Exercises for Chapter 14 232
15 GAs for Dietary Menu Selection 235
15.1 Introduction 235
15.2 Definition of the KP 236
15.3 A Simple Algorithm for the KP 238
15.4 Variations of the KP 239
15.5 GAs for KP Solution 240
15.6 Python Script for Dietary Menu Selection Using a GA 242
15.7 Matlab Script for Dietary Menu Selection Using a GA 245
15.8 Concluding Remarks 248
15.9 Exercises for Chapter 15 248
16 GAs for Drone Flight Control 251
16.1 Introduction 251
16.2 UAV Swarms 251
16.3 UAV Flight Control 252
16.4 A Simple GA for UAV Flight Control 253
16.4.1 Virtual Force-Based Fitness Function 254
16.4.2 FGA Progression 255
16.4.3 Chromosome for FGA 257
16.5 Python Script for UAV Flight Control Using a GA 260
16.6 Matlab Script for UAV Flight Control Using a GA 264
16.7 Concluding Remarks 270
16.8 Exercises for Chapter 16 271
17 GAs for Route Optimization 273
17.1 Introduction 273
17.2 Definition of the TSP 274
17.3 A Simple Algorithm for the TSP 276
17.4 Variations of the TSP 277
17.5 GA Solution for the TSP 277
17.6 Python Script for Route Optimization Using a GA 279
17.7 Matlab Script for Route Optimization Using a GA 284
17.8 Concluding Remarks 287
17.9 Exercises for Chapter 17 289
18 Evolutionary Methods 291
18.1 Introduction 291
18.2 Particle Swarm Optimization 291
18.2.1 Applications of PSO 292
18.2.2 PSO Operation 293
18.2.3 Remarks for PSO 298
18.3 Differential Evolution 298
18.3.1 Different Versions of DE 299
18.3.2 Applications of DE 299
18.3.3 A Simple Algorithm for DE 299
18.3.4 Numerical Example: Maximum of sinc by DE 302
18.3.5 Remarks for DE 305
18.4 Grammatical Evolution 306
18.4.1 A Simple Algorithm for GE 306
18.4.2 Definition of GE 307
18.4.3 A Simple GA to Implement GE 314
18.4.4 Remarks on GE 315
Appendix A ANNs with Bias 317
A.1 Introduction 317
A.2 Training with Bias Input 317
A.3 Forward Propagation 318
A.3.1 Forward Propagation from Input to Hidden Layer 319
A.3.2 Neuron Back Propagation with Bias Input 319
Appendix B Sleep Study ANN with Bias 321
B.1 Inclusion of Bias Term in ANN 321
B.1.1 Inclusion of Bias in Matrices 321
B.1.2 Forward Propagation with Biases 322
Appendix C Back Propagation in a CNN 327
Appendix D Back Propagation Through Time in an RNN 331
D.1 Back Propagation in an RNN 331
D.2 Embedding Layer 335
Appendix E Back Propagation Through Time in an RNN with Attention 337
E.1 Back Propagation in an RNN-Att 337
E.2 Embedding Layer 340
Bibliography 343
Index 353
1
Introduction
1.1 Artificial Intelligence
Artificial intelligence (AI) has become one of the most exciting and active fields of research in recent years. AI attempts to develop software and computing devices that are able to perform tasks which have been associated only with thinking beings for centuries. Mimicking intelligence, creativity, deduction capability and ability to learn from experience are some of the directions that AI undertakes. AI's recent ubiquitous presence in everyone's life makes this technology an integral part of daily activities that is often taken for granted. Most people treat AI tools as black boxes (i.e. devices whose outputs are observable but whose internal details are unknown by users) which spit out solutions to make problems go away. A reader of this book will quickly appreciate that there is no magic behind the way AI operates. By understanding the mechanisms employed in many popular AI techniques, correct solutions are easily obtained for complicated tasks. Through hands-on programming projects, the reader will be able to grasp how easily many otherwise unsolvable problems can be handled using AI.
1.2 A Historical Perspective
The question of whether a human-created device can be intelligent goes back to the middle of the twentieth century when the so-called Turing test was proposed to examine whether or not a machine is capable of thinking [1, 2]. The Turing test assumes that a human interrogator asks questions to both another human and a computer through an impediment that prevents identification of the responders. The proposition of the Turing test is that if the interrogator cannot distinguish which answers come from whom, then the interrogated human and the machine have the same intelligence. The Turing test has been used ever since to judge whether a computer can think. The term artificial intelligence was coined in 1955, six years after the Turing test was introduced, as the science and engineering for making intelligent machines together with the Turing test are considered as the beginning of modern AI [3].
The path from initial theoretical AI research to its realistic applications has progressed through many successful breakthroughs. One of the prominent milestones for engineers was marked by the introduction of the mobile robot Shaky in 1966 [4]. Shaky was controlled by so-called intelligent algorithms that analysed its surroundings to carry out a plan for an intended goal. On another path, it has long been believed that being able to effectively play a chess game is a good indicator for the effectiveness of AI. IBM developed a chess-playing computer called Deep Blue that defeated the chess grandmaster Garry Kasparov, the chess champion of the world at that time, in a historic match in 1997 [5]. Recent developments in self-driving cars would not have been possible without AI-based algorithms, which are essential for replicating the complex decision processes of a human driver [6]. In 2015, car maker Tesla marked one of the greatest milestones by announcing fully automated self-driving vehicles. By equipping its cars with AI software for interpreting and understanding the visual world, Tesla showed that robust path planning and real-time decision making were within reach [7]. These are only a few of the highlights marking recent developments in AI. It is widely believed that the importance of AI will only grow in years to come.
1.3 Principles of AI
An AI can be classified by the areas for which it is designed to find solutions. Currently, these areas include natural language processing, computer vision, automatic programming, robotics and intelligent data retrieval systems. However, grouping AI by the computational concepts employed for solving problems is a more accurate and better-suited classification for the readers of this book.
Searching is one of the computational methods that can be used for various AI tasks such as reasoning (e.g. finding inference roles), planning (e.g. search through goals and sub-goals) and moving (e.g. exploring a surrounding space by robots). Common AI tools to provide intelligent decisions in situations with incomplete or uncertain information can involve engaging Bayesian models, probabilistic algorithms and approaches derived from decision theory. Evolutionary algorithms are problem solving methods that attempt to replicate evolutionary operations of biological organisms in searching through a solution space. They are often used in AI to learn from previous experiences and to find solutions in large and complex settings. Learning principles of AI can be implemented using artificial neural networks and its variants, including convolutional neural networks and recurrent neural networks (RNNs), which allow for a system to be trained on a known set of data and for subsequent prediction of outcomes from new unknown events. Generative AI refers to a class of artificial intelligence systems that are designed to create new content, such as text, images, audio or code, by learning patterns from existing data. Unlike traditional AI, which often focuses on recognizing patterns or making predictions, generative AI models use those patterns to produce novel outputs that resemble the input data but are not mere copies. These systems are typically built using techniques like deep learning and neural networks, making them capable of generating realistic and creative content in various domains. Examples of generative AI methods include RNNs, long short term memory (LSTM) [8] and transformers [9].
1.4 Applications That Are Impossible Without AI
Starting from the early days of the twenty-first century, AI has been used in countless applications ranging from solving a trivial task of suggesting emojis in text messages [10] to design of flight-control software for commercial aeroplanes [11]. Some of the most prominent uses of AI are in the areas of medical diagnosis, image processing, control of autonomous vehicles and prediction of real-life events. In many similar tasks, an abundance of data (e.g. camera input, radar reading, proximity sensor) has to be processed in real time, something at which AI is especially good.
Applications of AI can save lives, literally. In the healthcare field, high-risk patients can be successfully monitored remotely with AI-based voice assistants. Automated physician assistance systems guided by AI methods are capable of generating questions to best diagnose a patient [12] and in finding an optimal and personalized treatment for them [13,14]. During surgery, an AI-based system can guide a surgeon's scalpel so that soft tissue damage is minimized and the resulting data collected during the procedures are quickly interpreted in clinical context [15].
AI can control an autonomous vehicle in places when a human operator cannot or does not want to operate. Self-driving cars [16], unmanned ground vehicles, drones and underwater vehicles [17-19] are only a few examples where the advantages of AI are directly visible.
Predicting the future may be one of the most powerful desires for many of us. Although AI cannot find answers to important existential questions, it can be helpful in guessing the probabilities of many future events [20]. For example, in an attempt to prepare for earthquakes, a combination of evolutionary algorithms, swarm intelligence and artificial neural networks is being used to forecast possible future seismic events. The natural language processing methods of AI can be employed in economics to estimate upcoming changes in trading markets [21] and in judicial applications to predict the outcome of a court ruling based on court proceedings [22].
1.5 Organization of This Book
This book is a beginner's guide to exciting and modern applications of AI techniques. The reader will be introduced to the basic concepts and principles of AI in order to develop skills readily applicable to real engineering projects. Through step-by-step instructions, the reader will learn how to implement typical AI tasks including control of autonomous drones, speech and character recognition, natural language processing, dietary menu planning, optimal selections for project management tasks and maximizing profits in algorithmic stock trading. We believe that all readers, regardless of their area of expertise, will enjoy an empowerment stemming from seeing relatively simple AI procedures used to tackle otherwise formidable problems.
This book is written with a diverse spectrum of readers in mind: engineers, scientists, economists and all backgrounds of students ranging from engineering to liberal arts. It welcomes all readers who would like to enter the realm of AI-based problem solving as it has become an essential skill for professionals in the twenty-first century.
In Chapter 2, the general concepts of an artificial neural network (ANN) architecture and its training process are introduced. As one of the most popular tools in AI in handling tasks that mimic events recorded in large amounts of data, ANN-based AI tools dominate the applications...
