The Business of Machine Learning

Chapter 1: What is ML: Why the hype right now. (30 pages)

a) Conversation with a Machine Learning Expert

b) Where's ML being used today?

i) Spam checking, spell check and grammar

ii) Siri, search engines, music selection (Spotify.)

c) Short history of Machine learning dating back to the 1950s

d) What is AI and what's its relationship to ML

e) Why Machine Learning is Hot Right now

i) Storage is more accessible than ever

ii) Access to compute, especially GPUS, is higher than ever

iii) New algorithms are being created every day

iv) New tooling making things more accessible

f) How is Machine Learning done?

i) What's inside a model?

ii) What's a feature?

g) How do data scientists think about feature extraction?

i) This requires domain expertise so a data scientist in the financial space, for example, wouldn't necessarily be effective in machine translation tasks or obstacle avoidance.

h) What are some of the new tools that will help folks access machine learning

Chapter 2: What is DL, how does it differ, why now? (20 pages)

a) Conversation involving Deep Learning

b) Deep learning is a branch of machine learning.

i) Machines do their own feature extraction

ii) Deep learning has turned intractable problems to tractable

c) What's made this possible at this point?

i) Compute, especially GPUs, are more accessible than ever

ii) New Math: Back propagation and Gradient Descent

d) Constructing a deep network

i) Training a model

ii) Using activation functions

iii) batch normalization.

iv) What is dropout?

v) Choosing an optimization function: SGD and Beyond

vi) Evaluating a model

vii) What is a SoftMax?

Chapter 3: Things that kind of look like AI and solve amazing problems but really aren't... (20 pages)

a) Conversation between a programmer and an ML expert on choosing the right tool for the job

b) Why are these things not actually Machine Learning?

i) Who programmed the rules?

ii) Deterministic verses Probabilistic results

c) Ways to solve problems that look like machine learning

i) Expert systems - rule based systems including state machines

(1) When to use an expert system

(2) Pitfalls and drawbacks of an expert system

ii) Convex optimization - set of techniques for deterministically finding the optimal resourcing

(1) When to use convex optimization

(2) Pitfalls and drawbacks to convex optimization

iii) Time-series Forecasting - using past data to predict the future, taking into account seasonal effects and short/long-term trends

(1) When to use time-series forecasting

(2) Pitfalls and drawbacks to using time-series forecasting

iv) Dynamic programming - cleverly breaking problems down, solving the easier smaller ones, and storing their solutions

(1) When to use dynamic programming

(2) Pitfalls and drawbacks of dynamic programming

Chapter 4: What sort of problems can you /should you solve with ML? DL? (20 pages)

a) Conversation with data scientist around selecting ML tools

b) What's ML really good at?

i) Discussion of problems where ML has helped

c) What's ML not good at?

i) A walk through a few problems where ML doesn't do very well

ii) Not enough data

iii) Curse of dimensionality

d) Recognizing a machine learning problem

i) Filtering out problems that can be solved by methods in Chapter 3

ii) Phrasing your problem as an ML problem

e) When is an ML problem actually a Deep Learning problem? How to know when this answer has changed (things are moving fast!)

Chapter 5: ML: Dealing with Data (featurization) (20 pages)

a) Conversation with a data scientist about featurization of data.

b) How do determine what data you need to collect

c) How to store this data

i) Data store

ii) Formats

d) How a data scientist works with data

i) Data cleaning

ii) Labeling

iii) Featurization

e) Potential pitfalls

i) Feature skew and Heteroskedasticity

ii) Label skew

iii) Interdependence of features

iv) Outlier detection

v) Data sparsity

vi) Missing values

Chapter 6: ML Under Supervision: Regression and Classification (20 pages)

a) The primary two methods of solving problems with classic machine learning

i) Classification turns features into a single decision such as a yes or no or into different buckets. E.G. Will it rain tomorrow?

ii) Regression turns features into numeric values. E.G. What's the temperature likely to be tomorrow?

Chapter 7: Unsupervised and Semi-supervised ML (20 pages)

a) Conversation with a data scientist about what a computer can do without any direction

b) What unsupervised learning can do

i) Working with unlabeled data

ii) Clustering and finding patterns in data

iii) Anomaly Detection

iv) How to recognize an unsupervised learning problem

c) How semi-supervised learning can help augment supervised learning

i) Combines labeled and unlabeled data

d) Moving from unsupervised to supervised

i) Creating labels from the clusters

Chapter 8: Deep Learning: On Images (CNNs) (20 pages)

a) Conversation with a data scientist about image processing

b) Short history of image recognition

i) How featurization of an image has traditionally been done

c) How the introduction of deep learning has accelerated the field

d) How does Deep Learning work with Images

i) Creating image features

ii) What is a convolution

iii) what is a convolutional neural network(CNN)

iv) what do width, stride and padding mean to a CNN?

Chapter 9: Deep Learning: On Text and Sound (RNNs) (20 pages)

a) Conversation with a data scientist about text and sound processing

b) Processing sequential data is far different than processing an image

i) Sequencing matters

ii) Looking at sequencing

(1) Letters

(2) Words

(3) Phoneme

c) Recurrent Neural Networks

i) Semantic mapping through of sequential data

ii) Vanishing and exploding gradient problems

iii) LSTMs and GRUs

iv) Attention-based methods

d) How can this be used?

i) Learn to caption images

ii) Speak like a celebrity

iii) Find information from long documents that would otherwise remain hidden with more traditional ML methods.

Chapter 10: Deep Learning: Self-Xing Y's, or Deep Reinforcement Learning (20 pages)

a) Conversation with a data scientist about reinforcement learning

b) Reinforcement Learning

i) Short history of reinforcement learning

(1) old technique being given dramatic new life with Deep Learning.

c) Basics of Q-Learning

i) Defining intelligent Q-functions

d) High profile uses

i) defeat the best Go player in the world,

ii) drive cars

iii) fly planes on its own.

Chapter 11: The ML Process: Data Provenance, Model Versioning, Deployment, etc. (30 pages)

a) Conversation with a data scientist on the life cycle of a machine learning project.

b) Traditional Software Lifecycle verses Machine Learning Software Lifecycle

i) Accounting for the data

(1) Where the data is coming from

(2) How it's changed over the time

ii) Versioning the model itself

c) Tracking quality of the model over time

i) Accuracy over time

d) Deployment of the model

i) Where does the model live?

(1) Server

(2) On a device

(3) Hybrid

ii) Hierarchical models

(1) Stacked degrees of precision

iii) Model compression/quantization

(1) Various techniques for doing this

(a) Normalization of values

(b) Giving up degrees of precision

e) Updating the model

i) Retraining

ii) Redeployment

Chapter 12: Advanced Topics: (20 pages)

a

b) VAEs (Variational Auto Encoders), GANs (Generative Adversarial Networks), Cyclegans?

c) Featurizers, x2vec

d) Ensembling and voting ensembles - multiple models who are ensembled (voting on the regression discussion in chapter 7)

e) Collaborative Filtering and Recommenders