Deep Learning on Embedded Systems

Preface x

Acknowledgment xi

Biography xii

About the Companion Websitexiii

1 Introduction 1

1.1 Machine Learning to Deep Learning 1

1.2 Modern Embedded Systems 2

1.3 Deep Transfer Learning in Embedded System 4

1.4 Deep Learning Frameworks: An Overview 5

1.4.1 PyTorch 5

1.4.2 TensorFlow 6

1.4.3 Other Major Frameworks 6

1.5 Deep Learning and AI: Big Data and the Road Ahead 7

1.5.1 Advances in Computational Hardware 7

1.5.2 Adoptions in Industry and Business 8

1.5.3 Ethical, Legal, and Privacy Concerns 8

1.5.4 Future Challenges 9

2 Fundamentals of Deep Learning 15

2.1 Neural Networks Overview 15

2.2 Basic Concepts and Terminologies 15

2.2.1 Neurons and Layers 15

2.2.2 Role of Activation Functions 17

2.2.3 Learning Types 18

2.3 Training on a Network 18

2.3.1 Forward and Backward Propagation 18

2.3.2 Loss Functions 19

2.4 Gradient Descent Algorithm 19

2.5 Weight Initialization and Regularization 21

2.6 Hyperparameter Tuning 22

2.6.1 Architecture-based Hyperparameters 23

2.6.2 Training-based Hyperparameters 23

2.7 Overview of Common Hyperparameters 23

2.7.1 Kernel Size 23

2.7.2 Learning Rate 23

2.7.3 Weight Decay 24

2.7.4 Dropout 24

2.7.5 Batch, Iteration, and Epochs 24

2.8 Challenges and Best Practices for Efficient Tuning 25

3 Convolutional and Recurrent Neural Network 27

3.1 Introduction 27

3.2 Historical Background 27

3.3 Convolutional Neural Networks 28

3.3.1 CNN Framework 28

3.3.2 Convolutional Operations 28

3.3.3 Padding and Stride 30

3.3.4 Pooling Layers 31

3.3.5 Fully Connected Layers 32

3.3.6 Output Layers 32

3.3.7 Overall Architecture 32

3.4 Recurrent Neural Networks 33

3.4.1 RNN Framework 33

3.4.2 Basic Architectures 33

3.4.3 Backpropagation Through Time 34

3.4.4 Long Short-term Memory Networks 35

3.4.5 Gated Recurrent Units 35

3.4.6 Bidirectional RNNs 35

3.5 Applications 36

3.6 Conclusion 37

4 Deep Learning Using PyTorch 41

4.1 Introduction to PyTorch 41

4.2 Anaconda and PyTorch for Windows System 41

4.2.1 Anaconda Installation and Environment Setup 42

4.2.2 Setting up the PyTorch Framework 44

4.2.2.1 Visual Studio Professional 47

4.2.2.2 CUDA and cuDNN Installation 48

4.3 Other Essential Packages for Deep Learning 50

4.4 Introduction to Tensor 52

4.4.1 Defining a Tensor 52

4.4.2 Tensors in PyTorch 53

4.5 Basic Torch Operations in PyTorch 53

4.6 Gradient Calculation in PyTorch 55

4.7 Exercise Problems 57

5 Introduction to Jetson Nano and Setup 59

5.1 Introduction to Jetson Embedded Devices 59

5.1.1 Jetson Nano 60

5.1.2 Hardware and Power Requirements 61

5.2 Jetpack Installation 62

5.3 Direct Setup 65

5.3.1 Increase Root Partition Size 65

5.3.2 Other Settings 68

5.3.3 Wi-Fi Driver 70

5.4 Configure Visual Studio Code on Jetson 71

5.5 OpenCV and PyTorch in Jetson 75

5.6 Setting up Jetson Inference 76

5.7 OpenCV Library and Test Video Capture Functionality 79

5.8 Conclusion 81

5.9 Exercise Problems 81

6 Linux Terminal Overview 85

6.1 Introduction 85

6.2 Basic Terminal Commands and Syntax 86

6.3 Overview of File System 87

6.4 Navigating Files and Directories 87

6.5 Create, Edit, and Delete 90

6.5.1 Copy and Move Operations 92

6.6 Create and Execute Python Code from Terminal 93

6.7 Common Wildcard Characters 95

6.8 Find, View, and Get Information 96

6.9 Permission and Ownership 99

6.9.1 Permissions Using Octal and Symbolic Notations 100

6.9.2 Update Permission Using Operators 101

6.10 Install and Uninstall Packages Using “sudo” 104

6.11 Conclusion 105

7 Docker Engine Setup 107

7.1 Introduction to Docker Engine 107

7.2 Docker in Embedded Devices 107

7.3 Jetson Inference Docker 108

7.3.1 Download and Test Pre-trained Models 109

7.4 Using Host Files in Docker Environment 110

7.4.1 Create Executable Python 111

7.4.2 Attach Host Directory to Root System and Testing 113

7.5 Building a Docker Image 115

7.5.1 Runtime Image for MediaPipe Ecosystem 115

7.5.2 Install Docker Extension 116

7.5.3 Create Docker File and Build Image 116

7.6 Run Python Through Docker Container 118

7.7 Exercise Problems 119

8 Dataset Development 121

8.1 Introduction and Requirements 121

8.2 Types of Datasets 121

8.3 Manual Dataset Creation 122

8.3.1 Classification and Detection Dataset 122

8.3.2 LabelImg Setup 123

8.4 Automatic Image Collection Using Embedded Device 127

8.5 Automatic Data Labeling 128

8.6 Data Preprocessing and Cleaning 130

8.7 Exercise Problem 131

9 Training Model for Image Classification 133

9.1 Problem Statement 133

9.2 Default Configurations and Libraries 134

9.3 Setup Data Frame Using Annotations 134

9.4 Dataset Class and Methods 136

9.5 Data Loader and Model Configuration 137

9.6 Model Training 140

9.7 Testing and Inference 141

9.8 Fine-tuning 142

9.8.1 Using a Different Model 144

9.8.2 Using Different Optimizers and Schedulers 145

9.9 Application in Embedded System 146

9.10 Exercise Problems 147

10 Object Detection with Classification 149

10.1 Introduction 149

10.2 Import Modules and Libraries 149

10.3 Default Configurations and Random Seeds 151

10.4 Create Data Frame and Process Labels 151

10.5 Training and Validation of Transformers 152

10.6 Dataset Class and Methods 153

10.7 Data Loader and Classification Backbone 153

10.8 Training and Validation Approach 155

10.9 Run Multiple Epochs and Save the Best 156

10.10 Model Inference 157

10.10.1 Preprocessing Functions 158

10.10.2 Inference on Test Images 160

10.11 Multiple Object Detection with Classification 161

10.12 Model Inference for Multiple Object Detection 163

10.13 Conclusion 165

10.14 Exercise Problems 167

11 Deploy Deep Learning Models on Jetson Nano 169

11.1 Introduction 169

11.2 Pre-trained Models 169

11.3 Inference on an Image File 170

11.4 ONNX Model 172

11.4.1 Convert PyTorch Model to ONNX 172

11.5 Inference on Live Video Stream 173

11.6 Conclusion 175

11.7 Exercise Problem 176

12 Trained PyTorch Model: From Desktop PC to Jetson Nano 177

12.1 Introduction 177

12.2 Model Training on a PC 177

12.3 ONNX Model Inference 177

12.3.1 PyTorch’s.pt to ONNX Conversion 177

12.3.2 Image Classification Using ONNX 179

12.3.3 PyTorch’s.pth to ONNX Conversion 181

12.3.4 Object Detection Using ONNX 181

12.4 Conclusion 186

12.5 Exercise Problems 186

13 Setting up Raspberry Pi 187

13.1 Introduction to Raspberry Pi 187

13.2 Hardware and Power Requirements 188

13.2.1 Direct Setup 190

13.3 Operating System Setup 190

13.3.1 Flashing the OS Image 191

13.3.2 Setting up OS 194

13.4 Create Virtual Environment 197

13.5 PyTorch and OpenCV Installation 198

13.6 Other Essential Packages 201

13.7 Conclusion 206

13.8 Exercise Problem 207

14 Deploy Deep Learning Models on Raspberry Pi 209

14.1 Introduction 209

14.2 Face Detection and Recognition in Video Feeds 209

14.2.1 Face Detection Using HOG 209

14.2.2 Face Recognition Using Transfer Learning 211

14.3 Real-time Object Detection 214

14.3.1 Object Detection Using YoloV3 214

14.4 Real-time Classification 217

14.4.1 Classification Using PyTorch’s Quantized Model 217

14.5 Real-time Segmentation 219

14.5.1 Segmentation Using k-means Clustering 219

14.6 Exercise Problems 221

15 Trained PyTorch Model: From Desktop PC to Raspberry Pi 225

15.1 Introduction 225

15.2 Model Training on a Desktop PC 225

15.3 PyTorch’s.pth to ONNX 225

15.3.1 ONNX Runtime 225

15.3.2 Model Conversion in Raspberry Pi 5 226

15.4 ONNX Model Inference on Raspberry Pi 5 228

15.4.1 Import Libraries and Define Configurations 228

15.4.2 Image Preprocessing 228

15.4.3 Model Inference with Bounding Boxes 229

15.5 Conclusion 231

15.6 Exercise Problems 232

Index 235