Hands-On AI Trading with Python, QuantConnect, and AWS
1st Edition
Published on 18. February 2025
Book
Hardback
416 pages
978-1-394-26843-6 (ISBN)
Description
Master the art of AI-driven algorithmic trading strategies through hands-on examples, in-depth insights, and step-by-step guidance
Hands-On AI Trading with Python, QuantConnect, and AWS explores real-world applications of AI technologies in algorithmic trading. It provides practical examples with complete code, allowing readers to understand and expand their AI toolbelt.
Unlike other books, this one focuses on designing actual trading strategies rather than setting up backtesting infrastructure. It utilizes QuantConnect, providing access to key market data from Algoseek and others. Examples are available on the book's GitHub repository, written in Python, and include performance tearsheets or research Jupyter notebooks.
The book starts with an overview of financial trading and QuantConnect's platform, organized by AI technology used:
Examples include constructing portfolios with regression models, predicting dividend yields, and safeguarding against market volatility using machine learning packages like SKLearn and MLFinLab.
Use principal component analysis to reduce model features, identify pairs for trading, and run statistical arbitrage with packages like LightGBM.
Predict market volatility regimes and allocate funds accordingly.
Predict daily returns of tech stocks using classifiers.
Forecast Forex pairs' future prices using Support Vector Machines and wavelets.
Predict trading day momentum or reversion risk using TensorFlow and temporal CNNs.
Apply large language models (LLMs) for stock research analysis, including prompt engineering and building RAG applications.
Perform sentiment analysis on real-time news feeds and train time-series forecasting models for portfolio optimization.
Better Hedging by Reinforcement Learning and AI: Implement reinforcement learning models for hedging options and derivatives with PyTorch.
AI for Risk Management and Optimization: Use corrective AI and conditional portfolio optimization techniques for risk management and capital allocation.
Written by domain experts, including Jiri Pik, Ernest Chan, Philip Sun, Vivek Singh, and Jared Broad, this book is essential for hedge fund professionals, traders, asset managers, and finance students. Integrate AI into your next algorithmic trading strategy with Hands-On AI Trading with Python, QuantConnect, and AWS.
Hands-On AI Trading with Python, QuantConnect, and AWS explores real-world applications of AI technologies in algorithmic trading. It provides practical examples with complete code, allowing readers to understand and expand their AI toolbelt.
Unlike other books, this one focuses on designing actual trading strategies rather than setting up backtesting infrastructure. It utilizes QuantConnect, providing access to key market data from Algoseek and others. Examples are available on the book's GitHub repository, written in Python, and include performance tearsheets or research Jupyter notebooks.
The book starts with an overview of financial trading and QuantConnect's platform, organized by AI technology used:
Examples include constructing portfolios with regression models, predicting dividend yields, and safeguarding against market volatility using machine learning packages like SKLearn and MLFinLab.
Use principal component analysis to reduce model features, identify pairs for trading, and run statistical arbitrage with packages like LightGBM.
Predict market volatility regimes and allocate funds accordingly.
Predict daily returns of tech stocks using classifiers.
Forecast Forex pairs' future prices using Support Vector Machines and wavelets.
Predict trading day momentum or reversion risk using TensorFlow and temporal CNNs.
Apply large language models (LLMs) for stock research analysis, including prompt engineering and building RAG applications.
Perform sentiment analysis on real-time news feeds and train time-series forecasting models for portfolio optimization.
Better Hedging by Reinforcement Learning and AI: Implement reinforcement learning models for hedging options and derivatives with PyTorch.
AI for Risk Management and Optimization: Use corrective AI and conditional portfolio optimization techniques for risk management and capital allocation.
Written by domain experts, including Jiri Pik, Ernest Chan, Philip Sun, Vivek Singh, and Jared Broad, this book is essential for hedge fund professionals, traders, asset managers, and finance students. Integrate AI into your next algorithmic trading strategy with Hands-On AI Trading with Python, QuantConnect, and AWS.
More details
Language
English
Place of publication
New York
United States
Target group
Professional and scholarly
Dimensions
Height: 256 mm
Width: 190 mm
Thickness: 32 mm
Weight
1044 gr
ISBN-13
978-1-394-26843-6 (9781394268436)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
Jiri Pik | Ernest P. Chan | Jared Broad
Hands-On AI Trading with Python, QuantConnect, and AWS
E-Book
01/2025
1st Edition
Wiley
€35.99
Available for download
Jiri Pik | Ernest P. Chan | Jared Broad
Hands-On AI Trading with Python, QuantConnect, and AWS
E-Book
01/2025
1st Edition
Wiley
€35.99
Available for download
Persons
JIRI PIK: Founder and CEO of RocketEdge.com. A software architect and cloud computing expert, Jiri Pik specializes in designing high-performance trading systems. He has decades of experience in financial technologies and has worked with some of the world's leading financial institutions, including Goldman Sachs and JPMorgan Chase.
ERNEST P. CHAN: A pioneer in applying machine learning to quantitative trading, Ernest P. Chan founded Predictnow.ai and QTS Capital Management. He is author of books such as Quantitative Trading and Machine Trading.
JARED BROAD: Founder and CEO of QuantConnect (TM), Jared Broad has empowered over 300,000 algorithmic traders worldwide with a platform that simplifies strategy design, backtesting, and live deployment.
PHILIP SUN: CEO and Co-founder of Adaptive Investment Solutions, LLC, and a seasoned quantitative fund manager, Philip Sun and his team focus on building state-of-the-art AI-driven risk management platform for wealth advisors and institutional investors.
VIVEK SINGH: A product leader at Amazon Web Services (AWS), Vivek Singh spearheads the development of large language models (LLMs) and Generative AI applications, bringing cutting-edge AI technologies to the trading domain.
ERNEST P. CHAN: A pioneer in applying machine learning to quantitative trading, Ernest P. Chan founded Predictnow.ai and QTS Capital Management. He is author of books such as Quantitative Trading and Machine Trading.
JARED BROAD: Founder and CEO of QuantConnect (TM), Jared Broad has empowered over 300,000 algorithmic traders worldwide with a platform that simplifies strategy design, backtesting, and live deployment.
PHILIP SUN: CEO and Co-founder of Adaptive Investment Solutions, LLC, and a seasoned quantitative fund manager, Philip Sun and his team focus on building state-of-the-art AI-driven risk management platform for wealth advisors and institutional investors.
VIVEK SINGH: A product leader at Amazon Web Services (AWS), Vivek Singh spearheads the development of large language models (LLMs) and Generative AI applications, bringing cutting-edge AI technologies to the trading domain.
Content
Biographies xiii
Preface: QuantConnect xv
Introduction xxiii
Part I Foundations of Capital Markets and Quantitative Trading 1
Chapter 1 Foundations of Capital Markets 3
Market Mechanics 3
Market Participants 4
Trading Is the "Play" 4
The Stage and Basic Rules of Trading-The Limit Order Book 4
Actors-Liquidity Trader, Market Maker, and
Informed Trader 5
Liquidity Trader 5
Market Maker 5
Informed Trader 6
AI Actors Wanted! 7
Data and Data Feeds 7
Custom and Alternative Data 9
Brokerages and Transaction Costs 10
Transaction Costs 11
Security Identifiers 13
Assets and Derivatives 15
US Equities 15
US Equity Options 19
Index Options 21
US Futures 21
Cryptocurrency 23
Chapter 2 Foundations of Quantitative Trading 25
Research Process 25
Research 25
Backtesting 26
Parameter Optimization 26
Paper and Live Trading 26
Testing and Debugging Tools 26
Debuggers 27
Logging 27
Charting 27
Object Store 28
Coding Process 28
Time and Look-ahead Bias 29
Look-ahead Bias 29
Market Hours and Scheduling 30
Strategy Styles 30
Trading Signals 31
Allocating Capital 31
Regimes and Portfolios of Strategies 32
Parameter Sensitivity Testing and Optimization 33
1. Remove 33
2. Replace 34
3. Reduce 34
Parameter Sensitivity Testing 34
Margin Modeling 35
Equities 35
Equity Options 36
Futures 37
Diversification and Asset Selection 37
Fundamental Asset Selection 38
ETF Constituents Asset Selection 39
Dollar-Volume Asset Selection 40
Universe Settings 40
Indicators and Other Data Transformations 41
Automatic Indicators 41
Manual Indicators 41
Indicator Warm Up 42
Storing Objects 42
Indicator Events 42
Sourcing Ideas 42
Hypothesis-driven Testing 43
Data Driven Investing 44
Quantpedia 44
QuantConnect Research and Strategy Explorer 45
Part II Foundations of AI and ML in Algorithmic Trading 47
Step-by-step Guide for AI-based Algorithmic Trading 48
Chapter 3 Step 1: Problem Definition 49
Chapter 4 Step 2: Dataset Preparation 53
Data Collection 53
Exploratory Data Analysis 53
Data Preprocessing 54
Handling Missing Data 55
Handling Outliers 58
Feature Engineering 61
Normalization and Standardization of Features 62
Transforming Time Series Features to Stationary 64
Identification of Cointegrated Time Series with Engle-Granger Test 70
Feature Selection 76
Correlation Analysis 76
Feature Importance Analysis 77
Auto-identification of Features 78
Dimensionality Reduction/Principal Component Analysis 80
Splitting of Dataset into Training, Testing, and Possibly Validation Sets 83
How to Split Your Data 83
Chapter 5 Step 3: Model Choice, Training, and Application 87
Regression 88
Linear Regression 89
Polynomial Regression 91
LASSO Regression 93
Ridge Regression 96
Markov Switching Dynamic Regression 99
Decision Tree Regression 103
Support Vector Machines Regression with
Wavelet Forecasting 105
Classification 110
Multiclass Random Forest Model 110
Logistic Regression 114
Hidden Markov Models 117
Gaussian Naive Bayes 119
Convolutional Neural Networks 122
Ranking 127
LGBRanker Ranking 127
Clustering 130
OPTICS Clustering 130
Language Models 132
OpenAI Language Model 132
Amazon Chronos Model 135
FinBERT Model 137
Part III Advanced Applications of AI in Trading and Risk Management 141
Getting Started with Source Code 141
Chapter 6 Applied Machine Learning 143
Example 1-ML Trend Scanning with MLFinlab 143
Example 2-Factor Preprocessing Techniques for Regime Detection 148
Example 3-Reversion vs. Trending: Strategy Selection by Classification 154
Example 4-Alpha by Hidden Markov Models 158
Example 5-FX SVM Wavelet Forecasting 170
Example 6-Dividend Harvesting Selection of
High-Yield Assets 176
Example 7-Effect of Positive-Negative Splits 181
Example 8-Stop Loss Based on Historical Volatility and Drawdown Recovery 185
Example 9-ML Trading Pairs Selection 197
Example 10-Stock Selection through Clustering
Fundamental Data 207
Example 11-Inverse Volatility Rank and Allocate to Future Contracts 214
Example 12-Trading Costs Optimization 221
Example 13-PCA Statistical Arbitrage Mean Reversion 228
Example 14-Temporal CNN Prediction 233
Example 15-Gaussian Classifier for Direction Prediction 242
Example 16-LLM Summarization of Tiingo News Articles 250
Example 17-Head Shoulders Pattern Matching with CNN 256
Example 18-Amazon Chronos Model 265
Example 19-FinBERT Model 272
Chapter 7 Better Hedging with Reinforcement Learning 281
Introduction 281
A New AI Trading Assistant 281
Continuous Hedging Is Not Required 282
Machine Learning Comes to the Rescue 283
A Simplified but Effective Reinforcement
Learning Approach 284
Overview of the Reinforcement Learning 285
Identification 285
Simulation 286
Ref inement Training on Actual Market Data 287
Testing and Implementation 287
Implementation on QuantConnect 288
Primary Research Notebook 289
The Policy Network 290
Model Functions 292
Fine-tuning with Market Data 296
Results 300
Conclusion 303
Chapter 8 AI for Risk Management and Optimization 305
What Is Corrective AI and Conditional
Parameter Optimization? 305
Feature Engineering 308
Applying Corrective AI to Daily Seasonal Forex Trading 312
What Is Conditional Parameter Optimization? 318
Applying Conditional Parameter Optimization to an ETF Strategy 319
Unconditional vs. Conditional Parameter Optimizations 320
Performance Comparisons 322
Conditional Portfolio Optimization 322
Regime Changes Obliterate Traditional Portfolio Optimization Methods 322
Learning to Optimize 324
Ranking Is Easier Than Predicting 325
The Fama-French Lineage 327
Comparison with Conventional Optimization Methods 327
Model Tactical Asset Allocation Portfolio 331
CPO Software-as-a-Service 333
Conclusion 340
Definitions of Spread_EMA & Spread_VAR 340
Chapter 9 Application of Large Language Models and Generative AI in Trading 341
Role of Generative AI in Creating Alpha 341
Selecting an LLM for Building a Generative AI Application 342
Prompt Engineering 344
Prompt Engineering in Practice 345
Addressing Model "Hallucination" 346
Question Answering Using a Retrieval Augmented Application in SageMaker Canvas 347
RAG Application Costs and Optimization Techniques 350
Testing Our Infrastructure 351
Summarization 356
Useful AI Platforms and Services 359
ChatGPT 359
Gemini 359
Bedrock 359
SageMaker 359
Q Business 360
References 361
Subject Index 363
Code Index 379
Preface: QuantConnect xv
Introduction xxiii
Part I Foundations of Capital Markets and Quantitative Trading 1
Chapter 1 Foundations of Capital Markets 3
Market Mechanics 3
Market Participants 4
Trading Is the "Play" 4
The Stage and Basic Rules of Trading-The Limit Order Book 4
Actors-Liquidity Trader, Market Maker, and
Informed Trader 5
Liquidity Trader 5
Market Maker 5
Informed Trader 6
AI Actors Wanted! 7
Data and Data Feeds 7
Custom and Alternative Data 9
Brokerages and Transaction Costs 10
Transaction Costs 11
Security Identifiers 13
Assets and Derivatives 15
US Equities 15
US Equity Options 19
Index Options 21
US Futures 21
Cryptocurrency 23
Chapter 2 Foundations of Quantitative Trading 25
Research Process 25
Research 25
Backtesting 26
Parameter Optimization 26
Paper and Live Trading 26
Testing and Debugging Tools 26
Debuggers 27
Logging 27
Charting 27
Object Store 28
Coding Process 28
Time and Look-ahead Bias 29
Look-ahead Bias 29
Market Hours and Scheduling 30
Strategy Styles 30
Trading Signals 31
Allocating Capital 31
Regimes and Portfolios of Strategies 32
Parameter Sensitivity Testing and Optimization 33
1. Remove 33
2. Replace 34
3. Reduce 34
Parameter Sensitivity Testing 34
Margin Modeling 35
Equities 35
Equity Options 36
Futures 37
Diversification and Asset Selection 37
Fundamental Asset Selection 38
ETF Constituents Asset Selection 39
Dollar-Volume Asset Selection 40
Universe Settings 40
Indicators and Other Data Transformations 41
Automatic Indicators 41
Manual Indicators 41
Indicator Warm Up 42
Storing Objects 42
Indicator Events 42
Sourcing Ideas 42
Hypothesis-driven Testing 43
Data Driven Investing 44
Quantpedia 44
QuantConnect Research and Strategy Explorer 45
Part II Foundations of AI and ML in Algorithmic Trading 47
Step-by-step Guide for AI-based Algorithmic Trading 48
Chapter 3 Step 1: Problem Definition 49
Chapter 4 Step 2: Dataset Preparation 53
Data Collection 53
Exploratory Data Analysis 53
Data Preprocessing 54
Handling Missing Data 55
Handling Outliers 58
Feature Engineering 61
Normalization and Standardization of Features 62
Transforming Time Series Features to Stationary 64
Identification of Cointegrated Time Series with Engle-Granger Test 70
Feature Selection 76
Correlation Analysis 76
Feature Importance Analysis 77
Auto-identification of Features 78
Dimensionality Reduction/Principal Component Analysis 80
Splitting of Dataset into Training, Testing, and Possibly Validation Sets 83
How to Split Your Data 83
Chapter 5 Step 3: Model Choice, Training, and Application 87
Regression 88
Linear Regression 89
Polynomial Regression 91
LASSO Regression 93
Ridge Regression 96
Markov Switching Dynamic Regression 99
Decision Tree Regression 103
Support Vector Machines Regression with
Wavelet Forecasting 105
Classification 110
Multiclass Random Forest Model 110
Logistic Regression 114
Hidden Markov Models 117
Gaussian Naive Bayes 119
Convolutional Neural Networks 122
Ranking 127
LGBRanker Ranking 127
Clustering 130
OPTICS Clustering 130
Language Models 132
OpenAI Language Model 132
Amazon Chronos Model 135
FinBERT Model 137
Part III Advanced Applications of AI in Trading and Risk Management 141
Getting Started with Source Code 141
Chapter 6 Applied Machine Learning 143
Example 1-ML Trend Scanning with MLFinlab 143
Example 2-Factor Preprocessing Techniques for Regime Detection 148
Example 3-Reversion vs. Trending: Strategy Selection by Classification 154
Example 4-Alpha by Hidden Markov Models 158
Example 5-FX SVM Wavelet Forecasting 170
Example 6-Dividend Harvesting Selection of
High-Yield Assets 176
Example 7-Effect of Positive-Negative Splits 181
Example 8-Stop Loss Based on Historical Volatility and Drawdown Recovery 185
Example 9-ML Trading Pairs Selection 197
Example 10-Stock Selection through Clustering
Fundamental Data 207
Example 11-Inverse Volatility Rank and Allocate to Future Contracts 214
Example 12-Trading Costs Optimization 221
Example 13-PCA Statistical Arbitrage Mean Reversion 228
Example 14-Temporal CNN Prediction 233
Example 15-Gaussian Classifier for Direction Prediction 242
Example 16-LLM Summarization of Tiingo News Articles 250
Example 17-Head Shoulders Pattern Matching with CNN 256
Example 18-Amazon Chronos Model 265
Example 19-FinBERT Model 272
Chapter 7 Better Hedging with Reinforcement Learning 281
Introduction 281
A New AI Trading Assistant 281
Continuous Hedging Is Not Required 282
Machine Learning Comes to the Rescue 283
A Simplified but Effective Reinforcement
Learning Approach 284
Overview of the Reinforcement Learning 285
Identification 285
Simulation 286
Ref inement Training on Actual Market Data 287
Testing and Implementation 287
Implementation on QuantConnect 288
Primary Research Notebook 289
The Policy Network 290
Model Functions 292
Fine-tuning with Market Data 296
Results 300
Conclusion 303
Chapter 8 AI for Risk Management and Optimization 305
What Is Corrective AI and Conditional
Parameter Optimization? 305
Feature Engineering 308
Applying Corrective AI to Daily Seasonal Forex Trading 312
What Is Conditional Parameter Optimization? 318
Applying Conditional Parameter Optimization to an ETF Strategy 319
Unconditional vs. Conditional Parameter Optimizations 320
Performance Comparisons 322
Conditional Portfolio Optimization 322
Regime Changes Obliterate Traditional Portfolio Optimization Methods 322
Learning to Optimize 324
Ranking Is Easier Than Predicting 325
The Fama-French Lineage 327
Comparison with Conventional Optimization Methods 327
Model Tactical Asset Allocation Portfolio 331
CPO Software-as-a-Service 333
Conclusion 340
Definitions of Spread_EMA & Spread_VAR 340
Chapter 9 Application of Large Language Models and Generative AI in Trading 341
Role of Generative AI in Creating Alpha 341
Selecting an LLM for Building a Generative AI Application 342
Prompt Engineering 344
Prompt Engineering in Practice 345
Addressing Model "Hallucination" 346
Question Answering Using a Retrieval Augmented Application in SageMaker Canvas 347
RAG Application Costs and Optimization Techniques 350
Testing Our Infrastructure 351
Summarization 356
Useful AI Platforms and Services 359
ChatGPT 359
Gemini 359
Bedrock 359
SageMaker 359
Q Business 360
References 361
Subject Index 363
Code Index 379