Deep Learning in Quantitative Finance
Description
Deep learning, that is, the use of deep neural networks, is now one of the hottest topics amongst quantitative analysts. Deep Learning in Quantitative Finance provides a comprehensive treatment of deep learning and describes a wide range of applications in mainstream quantitative finance. Inside, you'll find over ten chapters which apply deep learning to multiple use cases across quantitative finance. You'll also gain access to a companion site containing a set of Jupyter notebooks, developed by the author, that use Python to illustrate the examples in the text. Readers will be able to work through these examples directly.
This book is a complete resource on how deep learning is used in quantitative finance applications. It introduces the basics of neural networks, including feedforward networks, optimization, and training, before proceeding to cover more advanced topics. You'll also learn about the most important software frameworks. The book then proceeds to cover the very latest deep learning research in quantitative finance, including approximating derivative values, volatility models, credit curve mapping, generating realistic market data, and hedging. The book concludes with a look at the potential for quantum deep learning and the broader implications deep learning has for quantitative finance and quantitative analysts.
- Covers the basics of deep learning and neural networks, including feedforward networks, optimization and training, and regularization techniques
- Offers an understanding of more advanced topics like CNNs, RNNs, autoencoders, generative models including GANs and VAEs, and deep reinforcement learning
- Demonstrates deep learning application in quantitative finance through case studies and hands-on applications via the companion website
- Introduces the most important software frameworks for applying deep learning within finance
This book is perfect for anyone engaged with quantitative finance who wants to get involved in a subject that is clearly going to be hugely influential for the future of finance.
More details
Other editions
Additional editions
Person
ANDREW GREEN FIMA MINSTP BA MA MAST DPHIL is a Managing Director, and Lead Rates and XVA Quant at Scotiabank with over twenty-five years of experience in quantitative finance. He has previously held leadership roles in XVA modelling at Lloyds Banking Group and Barclays Capital. He is also the author of XVA: Credit, Funding and Capital Valuation Adjustments (Wiley, 2015). Andrew has worked on interest rate, credit, and equity derivative model development and implementation during his career.
Content
Acknowledgments xix
1 Introduction 3
1.1 What this book is about 3
1.2 The Rise of AI 5
1.3 The Promise of AI in Quantitative Finance 7
1.4 Practicalities 7
1.5 Reading this book 10
2 Feed Forward Neural Networks 13
2.1 Introducing Neural Networks 13
2.2 Regression and Classification 18
2.3 Activation Functions 27
2.4 The Universal Function Approximation Theorem 45
2.5 Conclusions 48
3 Training Neural Networks 49
3.1 Backpropagation and Adjoint Algorithmic Differentiation 50
3.2 Data Preparation and Scaling 53
3.3 Weight Initialization 57
3.4 The Choice of Loss Function 68
3.5 Optimization Algorithms 82
3.6 Common Training Problems 97
3.7 Batch Normalization 104
3.8 Evaluation and Validation 110
3.9 Sobolev Training Using Function Derivatives 124
3.10 Conclusions 131
4 Regularisation 133
4.1 Introduction Regularisation and Generalisation 133
4.2 Weight Decay 134
4.3 Early Stopping 137
4.4 Ensemble Methods and Dropout 138
4.5 Data Augmentation 146
4.6 Other Regularisation Methods 147
4.7 Conclusions Regularisation Strategy 149
5 Hyperparameter Optimization 151
5.1 Introduction 151
5.2 Manual 155
5.3 Grid Search 155
5.4 Random Search 158
5.5 Bayesian Optimization 159
5.6 Bandit-based 165
5.7 Population Based Training (PBT) 181
5.8 Conclusions 184
6 Convolutional Neural Networks 187
6.1 Introduction 187
6.2 Convolutions 188
6.3 Downsampling 203
6.4 Data Augmentation 206
6.5 Transfer Learning Using Pre-trained Networks 211
6.6 Visualising Features 213
6.7 Famous CNNs 223
6.8 Conclusions on CNNs 252
7 Sequence Models 255
7.1 Introducing Sequence Models 255
7.2 Recurrent Neural Networks 257
7.3 Neural Natural Language Processing 276
7.4 Conclusions on Sequence Models 322
8 Autoencoders 323
8.1 Introduction 323
8.2 Autoencoders and Singular-Valued Decomposition 325
8.3 Shallow and Deep Autoencoders 332
8.4 Regularized and Sparse Autoencoders 336
8.5 Denoising Autoencoders 339
8.6 Autoencoders and Generative Models 341
8.7 Conclusion 342
9 Generative Models 343
9.1 Introduction 343
9.2 Evaluating Generative Model Performance 345
9.3 Energy-based Models (EBMs) 348
9.4 Variational Autoencoders (VAEs) 383
9.5 Generative Adversarial Networks (GANs) 396
9.6 Latent Diffusion Models (LDMs) 491
9.7 Conclusions on Generative Models 493
10 Deep Reinforcement Learning 495
10.1 Introduction 495
10.2 Key Concepts in Reinforcement Learning 496
10.3 Markov Decision Processes (MDPs) and the Bellman Equations 506
10.4 Dynamic Programming and Policy Search 509
10.5 Monte Carlo Methods for RL 516
10.6 TD Learning 535
10.7 Deep Q Networks (DQNs) 546
10.8 Policy Gradient 561
10.9 Actor-Critic Methods 567
10.10 Conclusions 568
11 Derivative Valuation using Neural Networks 571
11.1 Introduction 571
11.2 Derivative Valuation using Neural Networks trained as Non-parametric Models 572
11.3 Derivative Valuation Function Approximation 584
12 High Dimensional PDE and BSDE Solvers 603
12.1 Introduction 603
12.2 Deep Galerkin Method (DGM) 604
12.3 Deep BSDE Solvers 619
12.4 Projection and Martingale Solvers 641
12.5 Deep Path Dependent PDEs (DPPDE) 642
12.6 Physics Informed Neural Networks (PINNs) 644
12.7 Deep Backward Dynamic Programming (DBDP) 646
12.8 Deep Splitting (DS) 647
12.9 Conclusions 649
13 Deep Monte Carlo and Optimal Stopping 651
13.1 Introduction 651
13.2 Deep Monte Carlo 653
13.3 Deep Optimal Stopping and Applications 685
13.4 Conclusion Deep Monte Carlo 703
14 Static Replication using Neural Networks 705
14.1 (Semi) Static Replication 705
14.2 Neural Static Replication 708
14.3 Conclusions on Neural Static Replication 716
15 Volatility Surfaces 717
15.1 Introduction 717
15.2 Volatility Surface Models 718
15.3 Deep Learning Volatility Surfaces 722
15.4 Deep Local Volatility 736
15.5 Conclusions 750
16 Model Calibration 751
16.1 Introduction 751
16.2 Model Calibration 752
16.3 Conclusion on Deep Calibration 767
17 XVA 769
17.1 Introduction 769
17.2 Credit Curve Mapping 771
17.3 Exposure Calculation using Neural Networks 784
17.4 Conclusions on Deep XVA 791
18 Generating Realistic Market Data 793
18.1 Introduction and Classical Methods 793
18.2 Motivation and Applications of Synthetic Financial Market Data 796
18.3 Time Series Generation 798
18.4 Generating Higher Dimensional Market Data Structures 864
18.5 Completing Market Data - imputing missing values 886
18.6 Conclusions Synthetic Market Data 888
19 Deep Hedging 893
19.1 Introduction 893
19.2 Approaches to Deep Hedging 894
19.3 Deep Hedging Examples 935
19.4 Conclusion 942
20 The Future Quant 957
20.1 Conclusion on Deep Learning 957
20.2 The Future of Quantitative Analytics 959
20.3 The Future Quant 960
20.4 A Final Word 960
CHAPTER 1
Introduction
furor est profecto, furor egredi ex eo et, tamquam interna eius cuncta plane iam nota sint, ita scrutari extera, quasi vero mensuram ullius rei possit agere qui sui nesciat, aut mens hominis videre quae mundus ipse non capiat.
It is madness, perfect madness, to go out of this world and to search for what is beyond it, as if one who is ignorant of his own dimensions could ascertain the measure of anything else, or as if the human mind could see what the world itself cannot contain.
Pliny the Elder, Natural History 2.1
1.1 WHAT THIS BOOK IS ABOUT
This book is about deep learning or, as it is more popularly known, artificial intelligence (AI), and its application to problems in traditional quantitative finance. It grew from a desire on my own part to learn as much as possible about the subject. Before 2017, I had extensive experience in quantitative finance and in the numerical algorithms typically employed in pricing models, particularly in the context of XVA, the subject of my first book XVA: Credit, Funding and Capital Valuation Adjustments (Green, 2015). XVA typically involves high-performance computing (HPC) and, in my particular case, experience with GPUs. I also had exposure to algorithmic differentiation. While I had limited previous experience in machine learning beyond simple regression models, deep learning, using GPUs and gradient-based learning, seemed like a natural fit for quantitative finance.
The first concrete application of deep learning in quantitative finance I explored was replicating a basket option as described in Ferguson and Green (2018). This basket option was chosen because it was of moderate dimension and, simultaneously, challenging because the replicated underlying model used a Monte Carlo simulation. The work aimed to demonstrate that an accurate replicating model could be trained to handle noisy input and that the performance during inference would exceed the original model once trained. It also allowed the impact of hyperparameter tuning to be presented. The experiment was successful; however, simple replication of a pricing function has proved to be a small subset of the capabilities of deep learning in quantitative finance.
What category of method is deep learning? Quantitative analysts have historically used numerical methods to construct models, such as partial differential equation (PDE) solvers and Monte Carlo simulations. Deep learning and, more broadly, machine learning are no different. Deep neural networks are approximation models with weights that are learned. A better description would be that the weights are optimised using gradient-based optimisers that are variants of stochastic gradient descent. Neural networks should be seen as another numerical method in the arsenal of such methods available to quantitative analysts and the broader community of scientists. Neural networks have strengths and weaknesses like any other method.
However, deep learning is not without controversy in quantitative finance as elsewhere. Like any new approach, it has its sceptics and supporters. The accompanying AI revolution and media hype have not been conducive to the wider adoption of deep learning in quantitative finance. Deep learning is connected with the search for artificial general intelligence (AGI) and the timeless quest for humans to create an intelligence like themselves. Media hype there has been, but the last decade has genuinely seen a revolution with the advent of tools like large language models (LLM) that are clearly transformative in many fields.
Deep learning is an empirical and practical discipline. While mathematical, it is usually limited to linear algebra and hence is relatively straightforward. Training a neural network model, however, is not always straightforward, and making advances requires practical experience. Neural networks are often criticised as 'black boxes' and this opacity can be extended to both the training stage and inference time. It is not always clear why results from a particular network configuration perform poorly or well. Hence, this book contains many examples coded in Python and presented in Jupyter Notebooks. The examples are presented 'warts and all'; many work well, but some do not. Working with deep neural networks means developing experience and intuition to make things work.
The book has two main blocks of chapters. The first block explores deep learning techniques in their original content. This means covering many topics that are not related to quantitative finance at all, such as image classification and image generation. This is justified as it is how I have personally explored the deep learning discipline and provides a firm foundation for using neural network models in any context. The second block of chapters explores deep learning in quantitative finance, building on the foundation of the earlier general chapters. Hence, the book aims to serve two purposes: firstly, to provide a practical introduction to a wide variety of neural network models and, secondly, to explore the application of those techniques in traditional quantitative finance. The selection of topics is personal, reflecting the areas I am interested in. I have made it reasonably comprehensive, but there are inevitably omissions. This is in no small part due to the explosive growth of AI research in the last decade and during the writing of this text.
1.2 THE RISE OF AI
Figure 1.1 presents a timeline of the main developments in AI since 1900. What is notable in this image is the fact that half the entries are from developments in the last 25 years, reflecting the rise of AI in this period. Quantitative finance has explored the use of neural networks since the 1980s. However, practical applications have only arisen since the re-emergence of deep learning in recent years, with a significant increase in interest after 2016. The key challenge this has presented in writing a book has been the pace of change. This has inevitably meant the contents of the book's conclusion are different than they were at the project's inception. For example, when starting to write about generative models, generative adversarial networks (GANs) were the pre-eminent models, but around 2022, diffusion-based models became the leaders and GANs were quickly left behind. This has necessitated an almost constant updating of the text at specific points to reflect ongoing developments.
FIGURE 1.1 A timeline of the history of AI.
Source: Tarjomyar on Wikimedia Commons. Licensed under CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0/deed.en. Printed versions have been converted to greyscale.
1.2.1 LLMs
One development that happened part way through this project was the arrival of highly capable LLMs like GPT-4 from OpenAI and Claude 3 from Anthropic. Writing assistants like Grammarly now embed generative AI tools within their workflow. I have used LLMs and Grammarly as writing assistants and as a coding copilot to accelerate the preparation of examples since the release of these tools in 2023. LLMs are helpful as productivity tools, a subject I will return to later in Chapter 20.
1.3 THE PROMISE OF AI IN QUANTITATIVE FINANCE
One driver for this book is the promise of neural networks in traditional quantitative finance. While many and varied examples of deep learning are covered in the later chapters, that promise is not yet fulfilled. There is limited evidence of many deep learning models in production applications at the time of writing in 2025. This reflects partly on the inertia of financial market participants, where more traditional techniques still hold sway. However, quants are steadily finding new ways to use neural networks, and initial scepticism is slowly being replaced by practical steps forward. There is more to do; hence, this work also advocates using deep learning models. Deep learning will never replace traditional methods, but it does complement them well.
1.4 PRACTICALITIES
As noted earlier, deep neural networks are often seen as 'black boxes' that cannot be understood. In reality, they are, at best, 'grey boxes' that can sometimes be characterised through exploring the features they learn, as can be seen from the various methods to visualise features in computer vision applications as described in Chapter 6. This is only possible with small models; it is impossible for LLMs with trillions of parameters. While there are essential theoretical results such as the universal function approximation theorem (Hornik et al., 1989, 1990; Cybenko, 1989; Hornik, 1991), they are limited in number. Advances in deep learning have typically come through experiment and data science. Experience gained through practice is invaluable in guiding research.
1.4.1 The Examples
The examples are available on the GitHub repo:
https://github.com/greandrew/DeepLearningBookExamples Each chapter with examples has its subdirectory, and the Python packages are grouped by chapter. Using Python virtual environments to manage the dependencies is possible, but I recommend using Docker containers. All examples are presented in Jupyter Notebooks or through Jupyter Lab. A small number of Python files are sometimes used to manage utilities. Some examples are computationally demanding, and a reasonably powerful consumer GPU card with NVIDIA® CUDAT capability is strongly recommended (NVIDIA Corporation, 2007).1 Most models have been trained using an NVIDIA RTXT3060...
