Fundamentals of Machine Learning

Contents
Preface
1 Symbolic Classification and Nearest Neighbour Classification
1.1 Symbolic Classification
1.2 Nearest Neighbour Classification
2 Separating Planes and Linear Programming
2.1 Finding a Separating Hyperplane
2.2 Testing for feasibility of linear constraints
2.3 Linear Programming
MATLAB example
2.4 Conclusion
3 Separating Margins and Quadratic Programming
3.1 Quadratic Programming
3.2 Maximum Margin Separator Planes
3.3 Slack Variables
4 Dualization and Support Vectors
4.1 Duals of Linear Programs
4.2 Duals of Quadratic Programs
4.3 Support Vectors
5 Lagrange Multipliers and Duality
5.1 Multidimensional functions
5.2 Support Vector Expansion
5.3 Support Vector Expansion with Slack Variables
6 Kernel Functions
6.1 Feature Spaces
6.2 Feature Spaces and Quadratic Programming
6.3 Kernel Matrix and Mercer's Theorem
6.4 Proof of Mercer's Theorem
Step 1 - Definitions and Prerequisites
Step 2 - Designing the right Hilbert Space
Step 3 - The reproducing property
7 The SMO Algorithm
7.1 Overview and Principles
7.2 Optimisation Step
7.3 Simplified SMO
8 Regression
8.1 Slack Variables
8.2 Duality, Kernels and Regression
8.3 Deriving the Dual form of the QP for Regression
9 Perceptrons, Neural Networks and Genetic Algorithms
9.1 Perceptrons
Perceptron-Algorithm
Perceptron-Lemma and Convergence
Perceptrons and Linear Feasibility Testing
9.2 Neural Networks
Forward Propagation
Training and Error Backpropagation
9.3 Genetic Algorithms
9.4 Conclusion
10 Bayesian Regression
10.1 Bayesian Learning
10.2 Probabilistic Linear Regression
10.3 Gaussian Process Models
10.4 GP model with measurement noise
Optimization of hyperparameters
Covariance functions
10.5 Multi-Task Gaussian Process (MTGP) Models
11 Bayesian Networks
Propagation of probabilities in causal networks
Appendix - Linear Programming
A.1 Solving LP0 problems
A.2 Schematic representation of the iteration steps
A.3 Transition from LP0 to LP
A.4 Computing time and complexity issues
References
Index