1
A Comprehensive Review of Various Machine Learning Techniques

Pooja Pathak* and Parul Choudhary┼

Dept. of Computer Engineering and Applications, GLA University, Mathura, India

Abstract

The creation of an intelligent system that works like a human is due to Artificial intelligence (AI). It can be broadly classified into four techniques: machine learning, machine vision, automation and Robotics and natural language processing. These domains can learn from data provided, identify the hidden pattern and make decisions with human intervention. There are three types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. Thus, to reduce the risk factor while decision making, machine learning techniques are more beneficial. The benefit of machine learning is that it can do the work automatically, once it learns what to do. Therefore, in this work, we discuss the theory behind machine learning techniques and the tasks they perform such as classification, regression, clustering, etc. We also provide a review of the state of the art of several machine learning algorithms like Naive Bayes, random forest, K-Means, SVM, etc., in detail.

Keywords: Machine learning, classification, regression, recognition, clustering, etc.

1.1 Introduction

Machine learning spans IT, statistics, probability, AI, psychology, neurobiology, and other fields. Machine learning solves problems by creating a model that accurately represents a dataset. Teaching computers to mimic the human brain has advanced machine learning and expanded the field of statistics to include statistical computational theories of learning processes.

Machine learning is the subdomain of artificial intelligence. There are two subsets of AI - machine learning and deep learning. Machine learning can effectively work on small datasets and it takes less time for training, while deep learning can work on large datasets and it takes more time for training. Machine learning has three types-supervised, unsupervised and reinforcement learning. Supervised learning algorithms such as neural networks are worked with labelled datasets. Unsupervised learning algorithms such as clustering, etc., are worked with unlabeled datasets. Machine learning algorithms are grouped by desired outcome.

Supervised learning: Algorithms map inputs to outputs. The classification problem is a standard supervised learning task. The learner must learn a function that maps a vector into one of several classes by looking at input-output examples. Unsupervised learning models unlabeled inputs. Semi-supervised learning combines labelled and unlabeled examples to create a classifier. Reinforcement learning teaches an algorithm how to act based on data. Every action affects the environment, which guides the learning algorithm. Transduction is similar to supervised learning but does not explicitly construct a function. Instead, it predicts new outputs based on training inputs, training outputs, and new inputs. The algorithm learns its own inductive bias from past experience.

Machine learning algorithms are divided into supervised and unsupervised groups. Classes are predetermined in supervised algorithms. These classes are created as a finite set by humans, which labels a segment of data. Machine learning algorithms find patterns and build math models. These models are evaluated based on their predictive ability and data variance [1].

In this chapter, we review some techniques of machine learning algorithms and elaborate on them.

1.1.1 Random Forest

To realise AI, machine learning is the most powerful technique. There are several algorithms in machine learning, out of which random forest is considered as a group classification technique algorithm known for its effectiveness and simplicity. The Proximities, Out-of-bag error, Variable Importance Measure, etc., are the important features of it. An algorithm based on ensemble learning and belonging to supervised learning can be used for regression and classification. The classifier can be called a "Decision Tree Classifier" from which it chooses the best tree as the final classification tree via voting. Note that as the number of trees increases in a forest, it gives high accuracy and prevents overfitting problems. Random forest algorithm is chosen because it takes less training time, it runs efficiently for a large dataset to predict output with highest accuracy and it maintain accuracy even if a big proportion of data is missing. Figure 1.1 shows a diagram of this algorithm.

Working: It is divided into two phases: first, it combines N-decision trees to create a random forest. And second, it predicts each tree that was created in the first step.

Step 1: First, choose k data points at random from the training dataset.

Step 2: Decision trees are to be built with selected subsets.

Step 3: Then choose any number of trees which we need to build and repeat steps 1 and 2. The random forest algorithm is used in four major sectors: medicine, banking, marketing, and land use. Although it is sometimes mentioned that this algorithm is for regression and classification, that is not true; random forest is good for regression only [2].

1.1.2 Decision Tree

Decision tree [3] represents a classifier expressed as a recursive partition of the instance space. The decision tree is a distributed tree with a root node and no incoming edges.

All of the other nodes have exactly one incoming edge. Internal or test nodes have outgoing edges. The rest are leaves. Each test node in a decision tree divides the instance space into two or more sub-spaces based on input values. In the simplest case, each test considers a single attribute, such that the instance space is portioned according to the attribute's value. In case of numeric attributes, the condition refers to a range. Each leaf is assigned the best target class. The leaf may hold a probability vector that indicates the probability of the target attribute having a certain value. Navigating from the tree's root to the leaf classifies instances based on tests along the way. Figure 1.2 shows a basic decision tree. Each node's branch is labelled with the attribute it tests. Given this classifier, an analyst can predict the customer and understand their behaviour [4].

Figure 1.1 Architecture of random forest.

Figure 1.2 Decision tree.

1.1.3 Support Vector Machine

SVM kernel is the name of a function that transforms low-dimensional space to high-dimensional space. Non-linear separation problems can be solved using this kernel.

The main advantage of SVM is that it is very effective in a high dimension case.

SVM is used for regression as well as classification but primarily it is best for classification purpose; the main purpose is to find distinct input features by hyperplane. If there are two input features then there is only one line and if there are three input features then hyperplane is 2-D plane. The draw is, when the number of features exceeds by three then it becomes difficult to imagine [5].

An extreme vector is chosen by SVM which helps to create hyperplane. Vectors which are nearer to hyperplane are called support vectors; thus, the algorithm is known as support vector machine, illustrated in Figure 1.3.

Figure 1.3 Support vector model.

Example: SVM can be understood with an example.

A classification of dog and cat images can be easily done by SVM. In this case, first we train the model and then we apply testing with a creature. Cat and dog are two data points which are distinguished by one hyperplane. If there are extreme cases of cat, then it will classify cat; otherwise dog is to be classified.

There are two types of SVM kernel-linear and non-linear.

In linear SVM, dataset is classified into two classes using one hyperplane; such data is called linearly separable data. While in non-linear, dataset is not classified in one hyperplane; this data is called non-linear data.

1.1.4 Naive Bayes

Naive Bayes is based on Bayes theorem in classification technique. It is one of the methods in supervised learning algorithms and statistical method. But it is used for both clustering and classification depending on conditional probabilities that happen [6].

Let us assume one probabilistic model which allows to capture uncertainty by determining the probabilities of an outcome. The main reason for using Naive Bayes is that it can solve predictive problems; also, it evaluates learning algorithms. It obtains practical algorithms and can merge all observed data [7].

Figure 1.4 Naive Bayes.

Let's consider a general probability distribution of two values R(a1, a2). Using Bayes rule, without loss of generality we get this equation in Figure 1.4.

Considering a general probability distribution of two values R(a1,a2). We obtain without any loss of generality an equation [8].

1.1.5 K-Means Clustering

It is one of the clustering techniques in unsupervised learning algorithm. It classifies number of clusters from given dataset; and for each cluster, k centers are defined [9] as shown in Figure 1.5. This k centers are placed in a calculated way due to various locations which cause different results. Hence the better way is to place each cluster away from the...