Hands-On AI Trading with Python, QuantConnect, and AWS

Biographies xiii

Preface: QuantConnect xv

Introduction xxiii

Part I Foundations of Capital Markets and Quantitative Trading 1

Chapter 1 Foundations of Capital Markets 3

Market Mechanics 3

Market Participants 4

Trading Is the “Play” 4

The Stage and Basic Rules of Trading—The Limit Order Book 4

Actors—Liquidity Trader, Market Maker, and

Informed Trader 5

Liquidity Trader 5

Market Maker 5

Informed Trader 6

AI Actors Wanted! 7

Data and Data Feeds 7

Custom and Alternative Data 9

Brokerages and Transaction Costs 10

Transaction Costs 11

Security Identifiers 13

Assets and Derivatives 15

US Equities 15

US Equity Options 19

Index Options 21

US Futures 21

Cryptocurrency 23

Chapter 2 Foundations of Quantitative Trading 25

Research Process 25

Research 25

Backtesting 26

Parameter Optimization 26

Paper and Live Trading 26

Testing and Debugging Tools 26

Debuggers 27

Logging 27

Charting 27

Object Store 28

Coding Process 28

Time and Look-ahead Bias 29

Look-ahead Bias 29

Market Hours and Scheduling 30

Strategy Styles 30

Trading Signals 31

Allocating Capital 31

Regimes and Portfolios of Strategies 32

Parameter Sensitivity Testing and Optimization 33

1. Remove 33

2. Replace 34

3. Reduce 34

Parameter Sensitivity Testing 34

Margin Modeling 35

Equities 35

Equity Options 36

Futures 37

Diversification and Asset Selection 37

Fundamental Asset Selection 38

ETF Constituents Asset Selection 39

Dollar-Volume Asset Selection 40

Universe Settings 40

Indicators and Other Data Transformations 41

Automatic Indicators 41

Manual Indicators 41

Indicator Warm Up 42

Storing Objects 42

Indicator Events 42

Sourcing Ideas 42

Hypothesis-driven Testing 43

Data Driven Investing 44

Quantpedia 44

QuantConnect Research and Strategy Explorer 45

Part II Foundations of AI and ML in Algorithmic Trading 47

Step-by-step Guide for AI-based Algorithmic Trading 48

Chapter 3 Step 1: Problem Definition 49

Chapter 4 Step 2: Dataset Preparation 53

Data Collection 53

Exploratory Data Analysis 53

Data Preprocessing 54

Handling Missing Data 55

Handling Outliers 58

Feature Engineering 61

Normalization and Standardization of Features 62

Transforming Time Series Features to Stationary 64

Identification of Cointegrated Time Series with Engle-Granger Test 70

Feature Selection 76

Correlation Analysis 76

Feature Importance Analysis 77

Auto-identification of Features 78

Dimensionality Reduction/Principal Component Analysis 80

Splitting of Dataset into Training, Testing, and Possibly Validation Sets 83

How to Split Your Data 83

Chapter 5 Step 3: Model Choice, Training, and Application 87

Regression 88

Linear Regression 89

Polynomial Regression 91

LASSO Regression 93

Ridge Regression 96

Markov Switching Dynamic Regression 99

Decision Tree Regression 103

Support Vector Machines Regression with

Wavelet Forecasting 105

Classification 110

Multiclass Random Forest Model 110

Logistic Regression 114

Hidden Markov Models 117

Gaussian Naive Bayes 119

Convolutional Neural Networks 122

Ranking 127

LGBRanker Ranking 127

Clustering 130

OPTICS Clustering 130

Language Models 132

OpenAI Language Model 132

Amazon Chronos Model 135

FinBERT Model 137

Part III Advanced Applications of AI in Trading and Risk Management 141

Getting Started with Source Code 141

Chapter 6 Applied Machine Learning 143

Example 1—ML Trend Scanning with MLFinlab 143

Example 2—Factor Preprocessing Techniques for Regime Detection 148

Example 3—Reversion vs. Trending: Strategy Selection by Classification 154

Example 4—Alpha by Hidden Markov Models 158

Example 5—FX SVM Wavelet Forecasting 170

Example 6—Dividend Harvesting Selection of

High-Yield Assets 176

Example 7—Effect of Positive-Negative Splits 181

Example 8—Stop Loss Based on Historical Volatility and Drawdown Recovery 185

Example 9—ML Trading Pairs Selection 197

Example 10—Stock Selection through Clustering

Fundamental Data 207

Example 11—Inverse Volatility Rank and Allocate to Future Contracts 214

Example 12—Trading Costs Optimization 221

Example 13—PCA Statistical Arbitrage Mean Reversion 228

Example 14—Temporal CNN Prediction 233

Example 15—Gaussian Classifier for Direction Prediction 242

Example 16—LLM Summarization of Tiingo News Articles 250

Example 17—Head Shoulders Pattern Matching with CNN 256

Example 18—Amazon Chronos Model 265

Example 19—FinBERT Model 272

Chapter 7 Better Hedging with Reinforcement Learning 281

Introduction 281

A New AI Trading Assistant 281

Continuous Hedging Is Not Required 282

Machine Learning Comes to the Rescue 283

A Simplified but Effective Reinforcement

Learning Approach 284

Overview of the Reinforcement Learning 285

Identification 285

Simulation 286

Ref inement Training on Actual Market Data 287

Testing and Implementation 287

Implementation on QuantConnect 288

Primary Research Notebook 289

The Policy Network 290

Model Functions 292

Fine-tuning with Market Data 296

Results 300

Conclusion 303

Chapter 8 AI for Risk Management and Optimization 305

What Is Corrective AI and Conditional

Parameter Optimization? 305

Feature Engineering 308

Applying Corrective AI to Daily Seasonal Forex Trading 312

What Is Conditional Parameter Optimization? 318

Applying Conditional Parameter Optimization to an ETF Strategy 319

Unconditional vs. Conditional Parameter Optimizations 320

Performance Comparisons 322

Conditional Portfolio Optimization 322

Regime Changes Obliterate Traditional Portfolio Optimization Methods 322

Learning to Optimize 324

Ranking Is Easier Than Predicting 325

The Fama-French Lineage 327

Comparison with Conventional Optimization Methods 327

Model Tactical Asset Allocation Portfolio 331

CPO Software-as-a-Service 333

Conclusion 340

Definitions of Spread_EMA & Spread_VAR 340

Chapter 9 Application of Large Language Models and Generative AI in Trading 341

Role of Generative AI in Creating Alpha 341

Selecting an LLM for Building a Generative AI Application 342

Prompt Engineering 344

Prompt Engineering in Practice 345

Addressing Model “Hallucination” 346

Question Answering Using a Retrieval Augmented Application in SageMaker Canvas 347

RAG Application Costs and Optimization Techniques 350

Testing Our Infrastructure 351

Summarization 356

Useful AI Platforms and Services 359

ChatGPT 359

Gemini 359

Bedrock 359

SageMaker 359

Q Business 360

References 361

Subject Index 363

Code Index 379