Advances in Financial Machine Learning
1st Edition
Published on 23. January 2018
400 pages
978-1-119-48211-6 (ISBN)
Description
Learn to understand and implement the latest machine learning innovations to improve your investment performance
Machine learning (ML) is changing virtually every aspect of our lives. Today, ML algorithms accomplish tasks that - until recently - only expert humans could perform. And finance is ripe for disruptive innovations that will transform how the following generations understand money and invest.
In the book, readers will learn how to:
* Structure big data in a way that is amenable to ML algorithms
* Conduct research with ML algorithms on big data
* Use supercomputing methods and back test their discoveries while avoiding false positives
Advances in Financial Machine Learning addresses real life problems faced by practitioners every day, and explains scientifically sound solutions using math, supported by code and examples. Readers become active users who can test the proposed solutions in their individual setting.
Written by a recognized expert and portfolio manager, this book will equip investment professionals with the groundbreaking tools needed to succeed in modern finance.
Machine learning (ML) is changing virtually every aspect of our lives. Today, ML algorithms accomplish tasks that - until recently - only expert humans could perform. And finance is ripe for disruptive innovations that will transform how the following generations understand money and invest.
In the book, readers will learn how to:
* Structure big data in a way that is amenable to ML algorithms
* Conduct research with ML algorithms on big data
* Use supercomputing methods and back test their discoveries while avoiding false positives
Advances in Financial Machine Learning addresses real life problems faced by practitioners every day, and explains scientifically sound solutions using math, supported by code and examples. Readers become active users who can test the proposed solutions in their individual setting.
Written by a recognized expert and portfolio manager, this book will equip investment professionals with the groundbreaking tools needed to succeed in modern finance.
More details
Other editions
Additional editions
Marcos Lopez de Prado
Advances in Financial Machine Learning
Book
05/2018
1st Edition
Wiley
€52.50
Article not available at the moment
Content
1 - Cover [Seite 1]
2 - Title Page [Seite 9]
3 - Copyright [Seite 10]
4 - Contents [Seite 15]
5 - About the Author [Seite 27]
6 - Preamble [Seite 29]
6.1 - 1 Financial Machine Learning as a Distinct Subject [Seite 31]
6.1.1 - 1.1 Motivation [Seite 31]
6.1.2 - 1.2 The Main Reason Financial Machine Learning Projects Usually Fail [Seite 32]
6.1.2.1 - 1.2.1 The Sisyphus Paradigm [Seite 32]
6.1.2.2 - 1.2.2 The Meta-Strategy Paradigm [Seite 33]
6.1.3 - 1.3 Book Structure [Seite 34]
6.1.3.1 - 1.3.1 Structure by Production Chain [Seite 34]
6.1.3.2 - 1.3.2 Structure by Strategy Component [Seite 37]
6.1.3.3 - 1.3.3 Structure by Common Pitfall [Seite 40]
6.1.4 - 1.4 Target Audience [Seite 40]
6.1.5 - 1.5 Requisites [Seite 41]
6.1.6 - 1.6 FAQs [Seite 42]
6.1.7 - 1.7 Acknowledgments [Seite 46]
6.1.8 - Exercises [Seite 47]
6.1.9 - References [Seite 48]
6.1.10 - Bibliography [Seite 48]
7 - PART 1 Data Analysis [Seite 49]
7.1 - 2 Financial Data Structures [Seite 51]
7.1.1 - 2.1 Motivation [Seite 51]
7.1.2 - 2.2 Essential Types of Financial Data [Seite 51]
7.1.2.1 - 2.2.1 Fundamental Data [Seite 51]
7.1.2.2 - 2.2.2 Market Data [Seite 52]
7.1.2.3 - 2.2.3 Analytics [Seite 53]
7.1.2.4 - 2.2.4 Alternative Data [Seite 53]
7.1.3 - 2.3 Bars [Seite 53]
7.1.3.1 - 2.3.1 Standard Bars [Seite 54]
7.1.3.2 - 2.3.2 Information-Driven Bars [Seite 57]
7.1.4 - 2.4 Dealing with Multi-Product Series [Seite 60]
7.1.4.1 - 2.4.1 The ETF Trick [Seite 61]
7.1.4.2 - 2.4.2 PCA Weights [Seite 63]
7.1.4.3 - 2.4.3 Single Future Roll [Seite 64]
7.1.5 - 2.5 Sampling Features [Seite 66]
7.1.5.1 - 2.5.1 Sampling for Reduction [Seite 66]
7.1.5.2 - 2.5.2 Event-Based Sampling [Seite 66]
7.1.6 - Exercises [Seite 68]
7.1.7 - References [Seite 69]
7.2 - 3 Labeling [Seite 71]
7.2.1 - 3.1 Motivation [Seite 71]
7.2.2 - 3.2 The Fixed-Time Horizon Method [Seite 71]
7.2.3 - 3.3 Computing Dynamic Thresholds [Seite 72]
7.2.4 - 3.4 The Triple-Barrier Method [Seite 73]
7.2.5 - 3.5 Learning Side and Size [Seite 76]
7.2.6 - 3.6 Meta-Labeling [Seite 78]
7.2.7 - 3.7 How to Use Meta-Labeling [Seite 79]
7.2.8 - 3.8 The Quantamental Way [Seite 81]
7.2.9 - 3.9 Dropping Unnecessary Labels [Seite 82]
7.2.10 - Exercises [Seite 83]
7.2.11 - Bibliography [Seite 84]
7.3 - 4 Sample Weights [Seite 87]
7.3.1 - 4.1 Motivation [Seite 87]
7.3.2 - 4.2 Overlapping Outcomes [Seite 87]
7.3.3 - 4.3 Number of Concurrent Labels [Seite 88]
7.3.4 - 4.4 Average Uniqueness of a Label [Seite 89]
7.3.5 - 4.5 Bagging Classifiers and Uniqueness [Seite 90]
7.3.5.1 - 4.5.1 Sequential Bootstrap [Seite 91]
7.3.5.2 - 4.5.2 Implementation of Sequential Bootstrap [Seite 92]
7.3.5.3 - 4.5.3 A Numerical Example [Seite 93]
7.3.5.4 - 4.5.4 Monte Carlo Experiments [Seite 94]
7.3.6 - 4.6 Return Attribution [Seite 96]
7.3.7 - 4.7 Time Decay [Seite 98]
7.3.8 - 4.8 Class Weights [Seite 99]
7.3.9 - Exercises [Seite 100]
7.3.10 - References [Seite 101]
7.3.11 - Bibliography [Seite 101]
7.4 - 5 Fractionally Differentiated Features [Seite 103]
7.4.1 - 5.1 Motivation [Seite 103]
7.4.2 - 5.2 The Stationarity vs. Memory Dilemma [Seite 103]
7.4.3 - 5.3 Literature Review [Seite 104]
7.4.4 - 5.4 The Method [Seite 105]
7.4.4.1 - 5.4.1 Long Memory [Seite 105]
7.4.4.2 - 5.4.2 Iterative Estimation [Seite 106]
7.4.4.3 - 5.4.3 Convergence [Seite 108]
7.4.5 - 5.5 Implementation [Seite 108]
7.4.5.1 - 5.5.1 Expanding Window [Seite 108]
7.4.5.2 - 5.5.2 Fixed-Width Window Fracdiff [Seite 110]
7.4.6 - 5.6 Stationarity with Maximum Memory Preservation [Seite 112]
7.4.7 - 5.7 Conclusion [Seite 116]
7.4.8 - Exercises [Seite 116]
7.4.9 - References [Seite 117]
7.4.10 - Bibliography [Seite 117]
8 - PART 2 Modelling [Seite 119]
8.1 - 6 Ensemble Methods [Seite 121]
8.1.1 - 6.1 Motivation [Seite 121]
8.1.2 - 6.2 The Three Sources of Errors [Seite 121]
8.1.3 - 6.3 Bootstrap Aggregation [Seite 122]
8.1.3.1 - 6.3.1 Variance Reduction [Seite 122]
8.1.3.2 - 6.3.2 Improved Accuracy [Seite 124]
8.1.3.3 - 6.3.3 Observation Redundancy [Seite 125]
8.1.4 - 6.4 Random Forest [Seite 126]
8.1.5 - 6.5 Boosting [Seite 127]
8.1.6 - 6.6 Bagging vs. Boosting in Finance [Seite 128]
8.1.7 - 6.7 Bagging for Scalability [Seite 129]
8.1.8 - Exercises [Seite 129]
8.1.9 - References [Seite 130]
8.1.10 - Bibliography [Seite 130]
8.2 - 7 Cross-Validation in Finance [Seite 131]
8.2.1 - 7.1 Motivation [Seite 131]
8.2.2 - 7.2 The Goal of Cross-Validation [Seite 131]
8.2.3 - 7.3 Why K-Fold CV Fails in Finance [Seite 132]
8.2.4 - 7.4 A Solution: Purged K-Fold CV [Seite 133]
8.2.4.1 - 7.4.1 Purging the Training Set [Seite 133]
8.2.4.2 - 7.4.2 Embargo [Seite 135]
8.2.4.3 - 7.4.3 The Purged K-Fold Class [Seite 136]
8.2.5 - 7.5 Bugs in Sklearns Cross-Validation [Seite 137]
8.2.6 - Exercises [Seite 138]
8.2.7 - Bibliography [Seite 139]
8.3 - 8 Feature Importance [Seite 141]
8.3.1 - 8.1 Motivation [Seite 141]
8.3.2 - 8.2 The Importance of Feature Importance [Seite 141]
8.3.3 - 8.3 Feature Importance with Substitution Effects [Seite 142]
8.3.3.1 - 8.3.1 Mean Decrease Impurity [Seite 142]
8.3.3.2 - 8.3.2 Mean Decrease Accuracy [Seite 144]
8.3.4 - 8.4 Feature Importance without Substitution Effects [Seite 145]
8.3.4.1 - 8.4.1 Single Feature Importance [Seite 145]
8.3.4.2 - 8.4.2 Orthogonal Features [Seite 146]
8.3.5 - 8.5 Parallelized vs. Stacked Feature Importance [Seite 149]
8.3.6 - 8.6 Experiments with Synthetic Data [Seite 150]
8.3.7 - Exercises [Seite 155]
8.3.8 - References [Seite 155]
8.4 - 9 Hyper-Parameter Tuning with Cross-Validation [Seite 157]
8.4.1 - 9.1 Motivation [Seite 157]
8.4.2 - 9.2 Grid Search Cross-Validation [Seite 157]
8.4.3 - 9.3 Randomized Search Cross-Validation [Seite 159]
8.4.3.1 - 9.3.1 Log-Uniform Distribution [Seite 160]
8.4.4 - 9.4 Scoring and Hyper-parameter Tuning [Seite 162]
8.4.5 - Exercises [Seite 163]
8.4.6 - References [Seite 164]
8.4.7 - Bibliography [Seite 165]
9 - PART 3 Backtesting [Seite 167]
9.1 - 10 Bet Sizing [Seite 169]
9.1.1 - 10.1 Motivation [Seite 169]
9.1.2 - 10.2 Strategy-Independent Bet Sizing Approaches [Seite 169]
9.1.3 - 10.3 Bet Sizing from Predicted Probabilities [Seite 170]
9.1.4 - 10.4 Averaging Active Bets [Seite 172]
9.1.5 - 10.5 Size Discretization [Seite 172]
9.1.6 - 10.6 Dynamic Bet Sizes and Limit Prices [Seite 173]
9.1.7 - Exercises [Seite 176]
9.1.8 - References [Seite 177]
9.1.9 - Bibliography [Seite 177]
9.2 - 11 The Dangers of Backtesting [Seite 179]
9.2.1 - 11.1 Motivation [Seite 179]
9.2.2 - 11.2 Mission Impossible: The Flawless Backtest [Seite 179]
9.2.3 - 11.3 Even If Your Backtest Is Flawless, It Is Probably Wrong [Seite 180]
9.2.4 - 11.4 Backtesting Is Not a Research Tool [Seite 181]
9.2.5 - 11.5 A Few General Recommendations [Seite 181]
9.2.6 - 11.6 Strategy Selection [Seite 183]
9.2.7 - Exercises [Seite 186]
9.2.8 - References [Seite 186]
9.2.9 - Bibliography [Seite 187]
9.3 - 12 Backtesting through Cross-Validation [Seite 189]
9.3.1 - 12.1 Motivation [Seite 189]
9.3.2 - 12.2 The Walk-Forward Method [Seite 189]
9.3.2.1 - 12.2.1 Pitfalls of the Walk-Forward Method [Seite 190]
9.3.3 - 12.3 The Cross-Validation Method [Seite 190]
9.3.4 - 12.4 The Combinatorial Purged Cross-Validation Method [Seite 191]
9.3.4.1 - 12.4.1 Combinatorial Splits [Seite 192]
9.3.4.2 - 12.4.2 The Combinatorial Purged Cross-Validation Backtesting Algorithm [Seite 193]
9.3.4.3 - 12.4.3 A Few Examples [Seite 193]
9.3.5 - 12.5 How Combinatorial Purged Cross-Validation Addresses Backtest Overfitting [Seite 194]
9.3.6 - Exercises [Seite 195]
9.3.7 - References [Seite 196]
9.4 - 13 Backtesting on Synthetic Data [Seite 197]
9.4.1 - 13.1 Motivation [Seite 197]
9.4.2 - 13.2 Trading Rules [Seite 197]
9.4.3 - 13.3 The Problem [Seite 198]
9.4.4 - 13.4 Our Framework [Seite 200]
9.4.5 - 13.5 Numerical Determination of Optimal Trading Rules [Seite 201]
9.4.5.1 - 13.5.1 The Algorithm [Seite 201]
9.4.5.2 - 13.5.2 Implementation [Seite 202]
9.4.6 - 13.6 Experimental Results [Seite 204]
9.4.6.1 - 13.6.1 Cases with Zero Long-Run Equilibrium [Seite 205]
9.4.6.2 - 13.6.2 Cases with Positive Long-Run Equilibrium [Seite 208]
9.4.6.3 - 13.6.3 Cases with Negative Long-Run Equilibrium [Seite 210]
9.4.7 - 13.7 Conclusion [Seite 220]
9.4.8 - Exercises [Seite 220]
9.4.9 - References [Seite 221]
9.5 - 14 Backtest Statistics [Seite 223]
9.5.1 - 14.1 Motivation [Seite 223]
9.5.2 - 14.2 Types of Backtest Statistics [Seite 223]
9.5.3 - 14.3 General Characteristics [Seite 224]
9.5.4 - 14.4 Performance [Seite 226]
9.5.4.1 - 14.4.1 Time-Weighted Rate of Return [Seite 226]
9.5.5 - 14.5 Runs [Seite 227]
9.5.5.1 - 14.5.1 Returns Concentration [Seite 227]
9.5.5.2 - 14.5.2 Drawdown and Time under Water [Seite 229]
9.5.5.3 - 14.5.3 Runs Statistics for Performance Evaluation [Seite 229]
9.5.6 - 14.6 Implementation Shortfall [Seite 230]
9.5.7 - 14.7 Efficiency [Seite 231]
9.5.7.1 - 14.7.1 The Sharpe Ratio [Seite 231]
9.5.7.2 - 14.7.2 The Probabilistic Sharpe Ratio [Seite 231]
9.5.7.3 - 14.7.3 The Deflated Sharpe Ratio [Seite 232]
9.5.7.4 - 14.7.4 Efficiency Statistics [Seite 233]
9.5.8 - 14.8 Classification Scores [Seite 234]
9.5.9 - 14.9 Attribution [Seite 235]
9.5.10 - Exercises [Seite 236]
9.5.11 - References [Seite 237]
9.5.12 - Bibliography [Seite 237]
9.6 - 15 Understanding Strategy Risk [Seite 239]
9.6.1 - 15.1 Motivation [Seite 239]
9.6.2 - 15.2 Symmetric Payouts [Seite 239]
9.6.3 - 15.3 Asymmetric Payouts [Seite 241]
9.6.4 - 15.4 The Probability of Strategy Failure [Seite 244]
9.6.4.1 - 15.4.1 Algorithm [Seite 245]
9.6.4.2 - 15.4.2 Implementation [Seite 245]
9.6.5 - Exercises [Seite 247]
9.6.6 - References [Seite 248]
9.7 - 16 Machine Learning Asset Allocation [Seite 249]
9.7.1 - 16.1 Motivation [Seite 249]
9.7.2 - 16.2 The Problem with Convex Portfolio Optimization [Seite 249]
9.7.3 - 16.3 Markowitzs Curse [Seite 250]
9.7.4 - 16.4 From Geometric to Hierarchical Relationships [Seite 251]
9.7.4.1 - 16.4.1 Tree Clustering [Seite 252]
9.7.4.2 - 16.4.2 Quasi-Diagonalization [Seite 257]
9.7.4.3 - 16.4.3 Recursive Bisection [Seite 257]
9.7.5 - 16.5 A Numerical Example [Seite 259]
9.7.6 - 16.6 Out-of-Sample Monte Carlo Simulations [Seite 262]
9.7.7 - 16.7 Further Research [Seite 264]
9.7.8 - 16.8 Conclusion [Seite 266]
9.7.9 - APPENDICES [Seite 267]
9.7.9.1 - 16.A.1 Correlation-based Metric [Seite 267]
9.7.9.2 - 16.A.2 Inverse Variance Allocation [Seite 267]
9.7.9.3 - 16.A.3 Reproducing the Numerical Example [Seite 268]
9.7.9.4 - 16.A.4 Reproducing the Monte Carlo Experiment [Seite 270]
9.7.10 - Exercises [Seite 272]
9.7.11 - References [Seite 273]
10 - PART 4 Useful Financial Features [Seite 275]
10.1 - 17 Structural Breaks [Seite 277]
10.1.1 - 17.1 Motivation [Seite 277]
10.1.2 - 17.2 Types of Structural Break Tests [Seite 277]
10.1.3 - 17.3 CUSUM Tests [Seite 278]
10.1.3.1 - 17.3.1 Brown-Durbin-Evans CUSUM Test on Recursive Residuals [Seite 278]
10.1.3.2 - 17.3.2 Chu-Stinchcombe-White CUSUM Test on Levels [Seite 279]
10.1.4 - 17.4 Explosiveness Tests [Seite 279]
10.1.4.1 - 17.4.1 Chow-Type Dickey-Fuller Test [Seite 279]
10.1.4.2 - 17.4.2 Supremum Augmented Dickey-Fuller [Seite 280]
10.1.4.3 - 17.4.3 Sub- and Super-Martingale Tests [Seite 287]
10.1.5 - Exercises [Seite 289]
10.1.6 - References [Seite 289]
10.2 - 18 Entropy Features [Seite 291]
10.2.1 - 18.1 Motivation [Seite 291]
10.2.2 - 18.2 Shannons Entropy [Seite 291]
10.2.3 - 18.3 The Plug-in (or Maximum Likelihood) Estimator [Seite 292]
10.2.4 - 18.4 Lempel-Ziv Estimators [Seite 293]
10.2.5 - 18.5 Encoding Schemes [Seite 297]
10.2.5.1 - 18.5.1 Binary Encoding [Seite 298]
10.2.5.2 - 18.5.2 Quantile Encoding [Seite 298]
10.2.5.3 - 18.5.3 Sigma Encoding [Seite 298]
10.2.6 - 18.6 Entropy of a Gaussian Process [Seite 299]
10.2.7 - 18.7 Entropy and the Generalized Mean [Seite 299]
10.2.8 - 18.8 A Few Financial Applications of Entropy [Seite 303]
10.2.8.1 - 18.8.1 Market Efficiency [Seite 303]
10.2.8.2 - 18.8.2 Maximum Entropy Generation [Seite 303]
10.2.8.3 - 18.8.3 Portfolio Concentration [Seite 303]
10.2.8.4 - 18.8.4 Market Microstructure [Seite 304]
10.2.9 - Exercises [Seite 305]
10.2.10 - References [Seite 306]
10.2.11 - Bibliography [Seite 307]
10.3 - 19 Microstructural Features [Seite 309]
10.3.1 - 19.1 Motivation [Seite 309]
10.3.2 - 19.2 Review of the Literature [Seite 309]
10.3.3 - 19.3 First Generation: Price Sequences [Seite 310]
10.3.3.1 - 19.3.1 The Tick Rule [Seite 310]
10.3.3.2 - 19.3.2 The Roll Model [Seite 310]
10.3.3.3 - 19.3.3 High-Low Volatility Estimator [Seite 311]
10.3.3.4 - 19.3.4 Corwin and Schultz [Seite 312]
10.3.4 - 19.4 Second Generation: Strategic Trade Models [Seite 314]
10.3.4.1 - 19.4.1 Kyles Lambda [Seite 314]
10.3.4.2 - 19.4.2 Amihuds Lambda [Seite 316]
10.3.4.3 - 19.4.3 Hasbroucks Lambda [Seite 317]
10.3.5 - 19.5 Third Generation: Sequential Trade Models [Seite 318]
10.3.5.1 - 19.5.1 Probability of Information-based Trading [Seite 318]
10.3.5.2 - 19.5.2 Volume-Synchronized Probability of Informed Trading [Seite 320]
10.3.6 - 19.6 Additional Features from Microstructural Datasets [Seite 321]
10.3.6.1 - 19.6.1 Distibution of Order Sizes [Seite 321]
10.3.6.2 - 19.6.2 Cancellation Rates, Limit Orders, Market Orders [Seite 321]
10.3.6.3 - 19.6.3 Time-Weighted Average Price Execution Algorithms [Seite 322]
10.3.6.4 - 19.6.4 Options Markets [Seite 323]
10.3.6.5 - 19.6.5 Serial Correlation of Signed Order Flow [Seite 323]
10.3.7 - 19.7 What Is Microstructural Information? [Seite 323]
10.3.8 - Exercises [Seite 324]
10.3.9 - References [Seite 326]
11 - PART 5 High-Performance Computing Recipes [Seite 329]
11.1 - 20 Multiprocessing and Vectorization [Seite 331]
11.1.1 - 20.1 Motivation [Seite 331]
11.1.2 - 20.2 Vectorization Example [Seite 331]
11.1.3 - 20.3 Single-Thread vs. Multithreading vs. Multiprocessing [Seite 332]
11.1.4 - 20.4 Atoms and Molecules [Seite 334]
11.1.4.1 - 20.4.1 Linear Partitions [Seite 334]
11.1.4.2 - 20.4.2 Two-Nested Loops Partitions [Seite 335]
11.1.5 - 20.5 Multiprocessing Engines [Seite 337]
11.1.5.1 - 20.5.1 Preparing the Jobs [Seite 337]
11.1.5.2 - 20.5.2 Asynchronous Calls [Seite 339]
11.1.5.3 - 20.5.3 Unwrapping the Callback [Seite 340]
11.1.5.4 - 20.5.4 Pickle/Unpickle Objects [Seite 341]
11.1.5.5 - 20.5.5 Output Reduction [Seite 341]
11.1.6 - 20.6 Multiprocessing Example [Seite 343]
11.1.7 - Exercises [Seite 344]
11.1.8 - Reference [Seite 345]
11.1.9 - Bibliography [Seite 345]
11.2 - 21 Brute Force and Quantum Computers [Seite 347]
11.2.1 - 21.1 Motivation [Seite 347]
11.2.2 - 21.2 Combinatorial Optimization [Seite 347]
11.2.3 - 21.3 The Objective Function [Seite 348]
11.2.4 - 21.4 The Problem [Seite 349]
11.2.5 - 21.5 An Integer Optimization Approach [Seite 349]
11.2.5.1 - 21.5.1 Pigeonhole Partitions [Seite 349]
11.2.5.2 - 21.5.2 Feasible Static Solutions [Seite 351]
11.2.5.3 - 21.5.3 Evaluating Trajectories [Seite 351]
11.2.6 - 21.6 A Numerical Example [Seite 353]
11.2.6.1 - 21.6.1 Random Matrices [Seite 353]
11.2.6.2 - 21.6.2 Static Solution [Seite 354]
11.2.6.3 - 21.6.3 Dynamic Solution [Seite 355]
11.2.7 - Exercises [Seite 355]
11.2.8 - References [Seite 356]
11.3 - 22 High-Performance Computational Intelligence and Forecasting Technologies [Seite 357]
11.3.1 - 22.1 Motivation [Seite 357]
11.3.2 - 22.2 Regulatory Response to the Flash Crash of 2010 [Seite 357]
11.3.3 - 22.3 Background [Seite 358]
11.3.4 - 22.4 HPC Hardware [Seite 359]
11.3.5 - 22.5 HPC Software [Seite 363]
11.3.5.1 - 22.5.1 Message Passing Interface [Seite 363]
11.3.5.2 - 22.5.2 Hierarchical Data Format 5 [Seite 364]
11.3.5.3 - 22.5.3 In Situ Processing [Seite 364]
11.3.5.4 - 22.5.4 Convergence [Seite 365]
11.3.6 - 22.6 Use Cases [Seite 365]
11.3.6.1 - 22.6.1 Supernova Hunting [Seite 365]
11.3.6.2 - 22.6.2 Blobs in Fusion Plasma [Seite 366]
11.3.6.3 - 22.6.3 Intraday Peak Electricity Usage [Seite 368]
11.3.6.4 - 22.6.4 The Flash Crash of 2010 [Seite 369]
11.3.6.5 - 22.6.5 Volume-synchronized Probability of Informed Trading Calibration [Seite 374]
11.3.6.6 - 22.6.6 Revealing High Frequency Events with Non-uniform Fast Fourier Transform [Seite 375]
11.3.7 - 22.7 Summary and Call for Participation [Seite 377]
11.3.8 - 22.8 Acknowledgments [Seite 378]
11.3.9 - References [Seite 378]
12 - Index [Seite 381]
13 - EULA [Seite 395]
2 - Title Page [Seite 9]
3 - Copyright [Seite 10]
4 - Contents [Seite 15]
5 - About the Author [Seite 27]
6 - Preamble [Seite 29]
6.1 - 1 Financial Machine Learning as a Distinct Subject [Seite 31]
6.1.1 - 1.1 Motivation [Seite 31]
6.1.2 - 1.2 The Main Reason Financial Machine Learning Projects Usually Fail [Seite 32]
6.1.2.1 - 1.2.1 The Sisyphus Paradigm [Seite 32]
6.1.2.2 - 1.2.2 The Meta-Strategy Paradigm [Seite 33]
6.1.3 - 1.3 Book Structure [Seite 34]
6.1.3.1 - 1.3.1 Structure by Production Chain [Seite 34]
6.1.3.2 - 1.3.2 Structure by Strategy Component [Seite 37]
6.1.3.3 - 1.3.3 Structure by Common Pitfall [Seite 40]
6.1.4 - 1.4 Target Audience [Seite 40]
6.1.5 - 1.5 Requisites [Seite 41]
6.1.6 - 1.6 FAQs [Seite 42]
6.1.7 - 1.7 Acknowledgments [Seite 46]
6.1.8 - Exercises [Seite 47]
6.1.9 - References [Seite 48]
6.1.10 - Bibliography [Seite 48]
7 - PART 1 Data Analysis [Seite 49]
7.1 - 2 Financial Data Structures [Seite 51]
7.1.1 - 2.1 Motivation [Seite 51]
7.1.2 - 2.2 Essential Types of Financial Data [Seite 51]
7.1.2.1 - 2.2.1 Fundamental Data [Seite 51]
7.1.2.2 - 2.2.2 Market Data [Seite 52]
7.1.2.3 - 2.2.3 Analytics [Seite 53]
7.1.2.4 - 2.2.4 Alternative Data [Seite 53]
7.1.3 - 2.3 Bars [Seite 53]
7.1.3.1 - 2.3.1 Standard Bars [Seite 54]
7.1.3.2 - 2.3.2 Information-Driven Bars [Seite 57]
7.1.4 - 2.4 Dealing with Multi-Product Series [Seite 60]
7.1.4.1 - 2.4.1 The ETF Trick [Seite 61]
7.1.4.2 - 2.4.2 PCA Weights [Seite 63]
7.1.4.3 - 2.4.3 Single Future Roll [Seite 64]
7.1.5 - 2.5 Sampling Features [Seite 66]
7.1.5.1 - 2.5.1 Sampling for Reduction [Seite 66]
7.1.5.2 - 2.5.2 Event-Based Sampling [Seite 66]
7.1.6 - Exercises [Seite 68]
7.1.7 - References [Seite 69]
7.2 - 3 Labeling [Seite 71]
7.2.1 - 3.1 Motivation [Seite 71]
7.2.2 - 3.2 The Fixed-Time Horizon Method [Seite 71]
7.2.3 - 3.3 Computing Dynamic Thresholds [Seite 72]
7.2.4 - 3.4 The Triple-Barrier Method [Seite 73]
7.2.5 - 3.5 Learning Side and Size [Seite 76]
7.2.6 - 3.6 Meta-Labeling [Seite 78]
7.2.7 - 3.7 How to Use Meta-Labeling [Seite 79]
7.2.8 - 3.8 The Quantamental Way [Seite 81]
7.2.9 - 3.9 Dropping Unnecessary Labels [Seite 82]
7.2.10 - Exercises [Seite 83]
7.2.11 - Bibliography [Seite 84]
7.3 - 4 Sample Weights [Seite 87]
7.3.1 - 4.1 Motivation [Seite 87]
7.3.2 - 4.2 Overlapping Outcomes [Seite 87]
7.3.3 - 4.3 Number of Concurrent Labels [Seite 88]
7.3.4 - 4.4 Average Uniqueness of a Label [Seite 89]
7.3.5 - 4.5 Bagging Classifiers and Uniqueness [Seite 90]
7.3.5.1 - 4.5.1 Sequential Bootstrap [Seite 91]
7.3.5.2 - 4.5.2 Implementation of Sequential Bootstrap [Seite 92]
7.3.5.3 - 4.5.3 A Numerical Example [Seite 93]
7.3.5.4 - 4.5.4 Monte Carlo Experiments [Seite 94]
7.3.6 - 4.6 Return Attribution [Seite 96]
7.3.7 - 4.7 Time Decay [Seite 98]
7.3.8 - 4.8 Class Weights [Seite 99]
7.3.9 - Exercises [Seite 100]
7.3.10 - References [Seite 101]
7.3.11 - Bibliography [Seite 101]
7.4 - 5 Fractionally Differentiated Features [Seite 103]
7.4.1 - 5.1 Motivation [Seite 103]
7.4.2 - 5.2 The Stationarity vs. Memory Dilemma [Seite 103]
7.4.3 - 5.3 Literature Review [Seite 104]
7.4.4 - 5.4 The Method [Seite 105]
7.4.4.1 - 5.4.1 Long Memory [Seite 105]
7.4.4.2 - 5.4.2 Iterative Estimation [Seite 106]
7.4.4.3 - 5.4.3 Convergence [Seite 108]
7.4.5 - 5.5 Implementation [Seite 108]
7.4.5.1 - 5.5.1 Expanding Window [Seite 108]
7.4.5.2 - 5.5.2 Fixed-Width Window Fracdiff [Seite 110]
7.4.6 - 5.6 Stationarity with Maximum Memory Preservation [Seite 112]
7.4.7 - 5.7 Conclusion [Seite 116]
7.4.8 - Exercises [Seite 116]
7.4.9 - References [Seite 117]
7.4.10 - Bibliography [Seite 117]
8 - PART 2 Modelling [Seite 119]
8.1 - 6 Ensemble Methods [Seite 121]
8.1.1 - 6.1 Motivation [Seite 121]
8.1.2 - 6.2 The Three Sources of Errors [Seite 121]
8.1.3 - 6.3 Bootstrap Aggregation [Seite 122]
8.1.3.1 - 6.3.1 Variance Reduction [Seite 122]
8.1.3.2 - 6.3.2 Improved Accuracy [Seite 124]
8.1.3.3 - 6.3.3 Observation Redundancy [Seite 125]
8.1.4 - 6.4 Random Forest [Seite 126]
8.1.5 - 6.5 Boosting [Seite 127]
8.1.6 - 6.6 Bagging vs. Boosting in Finance [Seite 128]
8.1.7 - 6.7 Bagging for Scalability [Seite 129]
8.1.8 - Exercises [Seite 129]
8.1.9 - References [Seite 130]
8.1.10 - Bibliography [Seite 130]
8.2 - 7 Cross-Validation in Finance [Seite 131]
8.2.1 - 7.1 Motivation [Seite 131]
8.2.2 - 7.2 The Goal of Cross-Validation [Seite 131]
8.2.3 - 7.3 Why K-Fold CV Fails in Finance [Seite 132]
8.2.4 - 7.4 A Solution: Purged K-Fold CV [Seite 133]
8.2.4.1 - 7.4.1 Purging the Training Set [Seite 133]
8.2.4.2 - 7.4.2 Embargo [Seite 135]
8.2.4.3 - 7.4.3 The Purged K-Fold Class [Seite 136]
8.2.5 - 7.5 Bugs in Sklearns Cross-Validation [Seite 137]
8.2.6 - Exercises [Seite 138]
8.2.7 - Bibliography [Seite 139]
8.3 - 8 Feature Importance [Seite 141]
8.3.1 - 8.1 Motivation [Seite 141]
8.3.2 - 8.2 The Importance of Feature Importance [Seite 141]
8.3.3 - 8.3 Feature Importance with Substitution Effects [Seite 142]
8.3.3.1 - 8.3.1 Mean Decrease Impurity [Seite 142]
8.3.3.2 - 8.3.2 Mean Decrease Accuracy [Seite 144]
8.3.4 - 8.4 Feature Importance without Substitution Effects [Seite 145]
8.3.4.1 - 8.4.1 Single Feature Importance [Seite 145]
8.3.4.2 - 8.4.2 Orthogonal Features [Seite 146]
8.3.5 - 8.5 Parallelized vs. Stacked Feature Importance [Seite 149]
8.3.6 - 8.6 Experiments with Synthetic Data [Seite 150]
8.3.7 - Exercises [Seite 155]
8.3.8 - References [Seite 155]
8.4 - 9 Hyper-Parameter Tuning with Cross-Validation [Seite 157]
8.4.1 - 9.1 Motivation [Seite 157]
8.4.2 - 9.2 Grid Search Cross-Validation [Seite 157]
8.4.3 - 9.3 Randomized Search Cross-Validation [Seite 159]
8.4.3.1 - 9.3.1 Log-Uniform Distribution [Seite 160]
8.4.4 - 9.4 Scoring and Hyper-parameter Tuning [Seite 162]
8.4.5 - Exercises [Seite 163]
8.4.6 - References [Seite 164]
8.4.7 - Bibliography [Seite 165]
9 - PART 3 Backtesting [Seite 167]
9.1 - 10 Bet Sizing [Seite 169]
9.1.1 - 10.1 Motivation [Seite 169]
9.1.2 - 10.2 Strategy-Independent Bet Sizing Approaches [Seite 169]
9.1.3 - 10.3 Bet Sizing from Predicted Probabilities [Seite 170]
9.1.4 - 10.4 Averaging Active Bets [Seite 172]
9.1.5 - 10.5 Size Discretization [Seite 172]
9.1.6 - 10.6 Dynamic Bet Sizes and Limit Prices [Seite 173]
9.1.7 - Exercises [Seite 176]
9.1.8 - References [Seite 177]
9.1.9 - Bibliography [Seite 177]
9.2 - 11 The Dangers of Backtesting [Seite 179]
9.2.1 - 11.1 Motivation [Seite 179]
9.2.2 - 11.2 Mission Impossible: The Flawless Backtest [Seite 179]
9.2.3 - 11.3 Even If Your Backtest Is Flawless, It Is Probably Wrong [Seite 180]
9.2.4 - 11.4 Backtesting Is Not a Research Tool [Seite 181]
9.2.5 - 11.5 A Few General Recommendations [Seite 181]
9.2.6 - 11.6 Strategy Selection [Seite 183]
9.2.7 - Exercises [Seite 186]
9.2.8 - References [Seite 186]
9.2.9 - Bibliography [Seite 187]
9.3 - 12 Backtesting through Cross-Validation [Seite 189]
9.3.1 - 12.1 Motivation [Seite 189]
9.3.2 - 12.2 The Walk-Forward Method [Seite 189]
9.3.2.1 - 12.2.1 Pitfalls of the Walk-Forward Method [Seite 190]
9.3.3 - 12.3 The Cross-Validation Method [Seite 190]
9.3.4 - 12.4 The Combinatorial Purged Cross-Validation Method [Seite 191]
9.3.4.1 - 12.4.1 Combinatorial Splits [Seite 192]
9.3.4.2 - 12.4.2 The Combinatorial Purged Cross-Validation Backtesting Algorithm [Seite 193]
9.3.4.3 - 12.4.3 A Few Examples [Seite 193]
9.3.5 - 12.5 How Combinatorial Purged Cross-Validation Addresses Backtest Overfitting [Seite 194]
9.3.6 - Exercises [Seite 195]
9.3.7 - References [Seite 196]
9.4 - 13 Backtesting on Synthetic Data [Seite 197]
9.4.1 - 13.1 Motivation [Seite 197]
9.4.2 - 13.2 Trading Rules [Seite 197]
9.4.3 - 13.3 The Problem [Seite 198]
9.4.4 - 13.4 Our Framework [Seite 200]
9.4.5 - 13.5 Numerical Determination of Optimal Trading Rules [Seite 201]
9.4.5.1 - 13.5.1 The Algorithm [Seite 201]
9.4.5.2 - 13.5.2 Implementation [Seite 202]
9.4.6 - 13.6 Experimental Results [Seite 204]
9.4.6.1 - 13.6.1 Cases with Zero Long-Run Equilibrium [Seite 205]
9.4.6.2 - 13.6.2 Cases with Positive Long-Run Equilibrium [Seite 208]
9.4.6.3 - 13.6.3 Cases with Negative Long-Run Equilibrium [Seite 210]
9.4.7 - 13.7 Conclusion [Seite 220]
9.4.8 - Exercises [Seite 220]
9.4.9 - References [Seite 221]
9.5 - 14 Backtest Statistics [Seite 223]
9.5.1 - 14.1 Motivation [Seite 223]
9.5.2 - 14.2 Types of Backtest Statistics [Seite 223]
9.5.3 - 14.3 General Characteristics [Seite 224]
9.5.4 - 14.4 Performance [Seite 226]
9.5.4.1 - 14.4.1 Time-Weighted Rate of Return [Seite 226]
9.5.5 - 14.5 Runs [Seite 227]
9.5.5.1 - 14.5.1 Returns Concentration [Seite 227]
9.5.5.2 - 14.5.2 Drawdown and Time under Water [Seite 229]
9.5.5.3 - 14.5.3 Runs Statistics for Performance Evaluation [Seite 229]
9.5.6 - 14.6 Implementation Shortfall [Seite 230]
9.5.7 - 14.7 Efficiency [Seite 231]
9.5.7.1 - 14.7.1 The Sharpe Ratio [Seite 231]
9.5.7.2 - 14.7.2 The Probabilistic Sharpe Ratio [Seite 231]
9.5.7.3 - 14.7.3 The Deflated Sharpe Ratio [Seite 232]
9.5.7.4 - 14.7.4 Efficiency Statistics [Seite 233]
9.5.8 - 14.8 Classification Scores [Seite 234]
9.5.9 - 14.9 Attribution [Seite 235]
9.5.10 - Exercises [Seite 236]
9.5.11 - References [Seite 237]
9.5.12 - Bibliography [Seite 237]
9.6 - 15 Understanding Strategy Risk [Seite 239]
9.6.1 - 15.1 Motivation [Seite 239]
9.6.2 - 15.2 Symmetric Payouts [Seite 239]
9.6.3 - 15.3 Asymmetric Payouts [Seite 241]
9.6.4 - 15.4 The Probability of Strategy Failure [Seite 244]
9.6.4.1 - 15.4.1 Algorithm [Seite 245]
9.6.4.2 - 15.4.2 Implementation [Seite 245]
9.6.5 - Exercises [Seite 247]
9.6.6 - References [Seite 248]
9.7 - 16 Machine Learning Asset Allocation [Seite 249]
9.7.1 - 16.1 Motivation [Seite 249]
9.7.2 - 16.2 The Problem with Convex Portfolio Optimization [Seite 249]
9.7.3 - 16.3 Markowitzs Curse [Seite 250]
9.7.4 - 16.4 From Geometric to Hierarchical Relationships [Seite 251]
9.7.4.1 - 16.4.1 Tree Clustering [Seite 252]
9.7.4.2 - 16.4.2 Quasi-Diagonalization [Seite 257]
9.7.4.3 - 16.4.3 Recursive Bisection [Seite 257]
9.7.5 - 16.5 A Numerical Example [Seite 259]
9.7.6 - 16.6 Out-of-Sample Monte Carlo Simulations [Seite 262]
9.7.7 - 16.7 Further Research [Seite 264]
9.7.8 - 16.8 Conclusion [Seite 266]
9.7.9 - APPENDICES [Seite 267]
9.7.9.1 - 16.A.1 Correlation-based Metric [Seite 267]
9.7.9.2 - 16.A.2 Inverse Variance Allocation [Seite 267]
9.7.9.3 - 16.A.3 Reproducing the Numerical Example [Seite 268]
9.7.9.4 - 16.A.4 Reproducing the Monte Carlo Experiment [Seite 270]
9.7.10 - Exercises [Seite 272]
9.7.11 - References [Seite 273]
10 - PART 4 Useful Financial Features [Seite 275]
10.1 - 17 Structural Breaks [Seite 277]
10.1.1 - 17.1 Motivation [Seite 277]
10.1.2 - 17.2 Types of Structural Break Tests [Seite 277]
10.1.3 - 17.3 CUSUM Tests [Seite 278]
10.1.3.1 - 17.3.1 Brown-Durbin-Evans CUSUM Test on Recursive Residuals [Seite 278]
10.1.3.2 - 17.3.2 Chu-Stinchcombe-White CUSUM Test on Levels [Seite 279]
10.1.4 - 17.4 Explosiveness Tests [Seite 279]
10.1.4.1 - 17.4.1 Chow-Type Dickey-Fuller Test [Seite 279]
10.1.4.2 - 17.4.2 Supremum Augmented Dickey-Fuller [Seite 280]
10.1.4.3 - 17.4.3 Sub- and Super-Martingale Tests [Seite 287]
10.1.5 - Exercises [Seite 289]
10.1.6 - References [Seite 289]
10.2 - 18 Entropy Features [Seite 291]
10.2.1 - 18.1 Motivation [Seite 291]
10.2.2 - 18.2 Shannons Entropy [Seite 291]
10.2.3 - 18.3 The Plug-in (or Maximum Likelihood) Estimator [Seite 292]
10.2.4 - 18.4 Lempel-Ziv Estimators [Seite 293]
10.2.5 - 18.5 Encoding Schemes [Seite 297]
10.2.5.1 - 18.5.1 Binary Encoding [Seite 298]
10.2.5.2 - 18.5.2 Quantile Encoding [Seite 298]
10.2.5.3 - 18.5.3 Sigma Encoding [Seite 298]
10.2.6 - 18.6 Entropy of a Gaussian Process [Seite 299]
10.2.7 - 18.7 Entropy and the Generalized Mean [Seite 299]
10.2.8 - 18.8 A Few Financial Applications of Entropy [Seite 303]
10.2.8.1 - 18.8.1 Market Efficiency [Seite 303]
10.2.8.2 - 18.8.2 Maximum Entropy Generation [Seite 303]
10.2.8.3 - 18.8.3 Portfolio Concentration [Seite 303]
10.2.8.4 - 18.8.4 Market Microstructure [Seite 304]
10.2.9 - Exercises [Seite 305]
10.2.10 - References [Seite 306]
10.2.11 - Bibliography [Seite 307]
10.3 - 19 Microstructural Features [Seite 309]
10.3.1 - 19.1 Motivation [Seite 309]
10.3.2 - 19.2 Review of the Literature [Seite 309]
10.3.3 - 19.3 First Generation: Price Sequences [Seite 310]
10.3.3.1 - 19.3.1 The Tick Rule [Seite 310]
10.3.3.2 - 19.3.2 The Roll Model [Seite 310]
10.3.3.3 - 19.3.3 High-Low Volatility Estimator [Seite 311]
10.3.3.4 - 19.3.4 Corwin and Schultz [Seite 312]
10.3.4 - 19.4 Second Generation: Strategic Trade Models [Seite 314]
10.3.4.1 - 19.4.1 Kyles Lambda [Seite 314]
10.3.4.2 - 19.4.2 Amihuds Lambda [Seite 316]
10.3.4.3 - 19.4.3 Hasbroucks Lambda [Seite 317]
10.3.5 - 19.5 Third Generation: Sequential Trade Models [Seite 318]
10.3.5.1 - 19.5.1 Probability of Information-based Trading [Seite 318]
10.3.5.2 - 19.5.2 Volume-Synchronized Probability of Informed Trading [Seite 320]
10.3.6 - 19.6 Additional Features from Microstructural Datasets [Seite 321]
10.3.6.1 - 19.6.1 Distibution of Order Sizes [Seite 321]
10.3.6.2 - 19.6.2 Cancellation Rates, Limit Orders, Market Orders [Seite 321]
10.3.6.3 - 19.6.3 Time-Weighted Average Price Execution Algorithms [Seite 322]
10.3.6.4 - 19.6.4 Options Markets [Seite 323]
10.3.6.5 - 19.6.5 Serial Correlation of Signed Order Flow [Seite 323]
10.3.7 - 19.7 What Is Microstructural Information? [Seite 323]
10.3.8 - Exercises [Seite 324]
10.3.9 - References [Seite 326]
11 - PART 5 High-Performance Computing Recipes [Seite 329]
11.1 - 20 Multiprocessing and Vectorization [Seite 331]
11.1.1 - 20.1 Motivation [Seite 331]
11.1.2 - 20.2 Vectorization Example [Seite 331]
11.1.3 - 20.3 Single-Thread vs. Multithreading vs. Multiprocessing [Seite 332]
11.1.4 - 20.4 Atoms and Molecules [Seite 334]
11.1.4.1 - 20.4.1 Linear Partitions [Seite 334]
11.1.4.2 - 20.4.2 Two-Nested Loops Partitions [Seite 335]
11.1.5 - 20.5 Multiprocessing Engines [Seite 337]
11.1.5.1 - 20.5.1 Preparing the Jobs [Seite 337]
11.1.5.2 - 20.5.2 Asynchronous Calls [Seite 339]
11.1.5.3 - 20.5.3 Unwrapping the Callback [Seite 340]
11.1.5.4 - 20.5.4 Pickle/Unpickle Objects [Seite 341]
11.1.5.5 - 20.5.5 Output Reduction [Seite 341]
11.1.6 - 20.6 Multiprocessing Example [Seite 343]
11.1.7 - Exercises [Seite 344]
11.1.8 - Reference [Seite 345]
11.1.9 - Bibliography [Seite 345]
11.2 - 21 Brute Force and Quantum Computers [Seite 347]
11.2.1 - 21.1 Motivation [Seite 347]
11.2.2 - 21.2 Combinatorial Optimization [Seite 347]
11.2.3 - 21.3 The Objective Function [Seite 348]
11.2.4 - 21.4 The Problem [Seite 349]
11.2.5 - 21.5 An Integer Optimization Approach [Seite 349]
11.2.5.1 - 21.5.1 Pigeonhole Partitions [Seite 349]
11.2.5.2 - 21.5.2 Feasible Static Solutions [Seite 351]
11.2.5.3 - 21.5.3 Evaluating Trajectories [Seite 351]
11.2.6 - 21.6 A Numerical Example [Seite 353]
11.2.6.1 - 21.6.1 Random Matrices [Seite 353]
11.2.6.2 - 21.6.2 Static Solution [Seite 354]
11.2.6.3 - 21.6.3 Dynamic Solution [Seite 355]
11.2.7 - Exercises [Seite 355]
11.2.8 - References [Seite 356]
11.3 - 22 High-Performance Computational Intelligence and Forecasting Technologies [Seite 357]
11.3.1 - 22.1 Motivation [Seite 357]
11.3.2 - 22.2 Regulatory Response to the Flash Crash of 2010 [Seite 357]
11.3.3 - 22.3 Background [Seite 358]
11.3.4 - 22.4 HPC Hardware [Seite 359]
11.3.5 - 22.5 HPC Software [Seite 363]
11.3.5.1 - 22.5.1 Message Passing Interface [Seite 363]
11.3.5.2 - 22.5.2 Hierarchical Data Format 5 [Seite 364]
11.3.5.3 - 22.5.3 In Situ Processing [Seite 364]
11.3.5.4 - 22.5.4 Convergence [Seite 365]
11.3.6 - 22.6 Use Cases [Seite 365]
11.3.6.1 - 22.6.1 Supernova Hunting [Seite 365]
11.3.6.2 - 22.6.2 Blobs in Fusion Plasma [Seite 366]
11.3.6.3 - 22.6.3 Intraday Peak Electricity Usage [Seite 368]
11.3.6.4 - 22.6.4 The Flash Crash of 2010 [Seite 369]
11.3.6.5 - 22.6.5 Volume-synchronized Probability of Informed Trading Calibration [Seite 374]
11.3.6.6 - 22.6.6 Revealing High Frequency Events with Non-uniform Fast Fourier Transform [Seite 375]
11.3.7 - 22.7 Summary and Call for Participation [Seite 377]
11.3.8 - 22.8 Acknowledgments [Seite 378]
11.3.9 - References [Seite 378]
12 - Index [Seite 381]
13 - EULA [Seite 395]
