List of Contributors xv
Preface xxi
Part I Spectrum Intelligence and Adaptive Resource Management 1
1 Machine Learning for Spectrum Access and Sharing 3
Kobi Cohen
1.1 Introduction 3
1.2 Online Learning Algorithms for Opportunistic Spectrum Access 4
1.2.1 The Network Model 4
1.2.2 Performance Measures of the Online Learning Algorithms 5
1.2.3 The Objective 6
1.2.4 Random and Deterministic Approaches 6
1.2.5 The Adaptive Sequencing Rules Approach 7
1.2.5.1 Structure of Transmission Epochs 7
1.2.5.2 Selection Rule under the ASR Algorithm 8
1.2.5.3 High-Level Pseudocode and Implementation Discussion 9
1.3 Learning Algorithms for Channel Allocation 9
1.3.1 The Network Model 10
1.3.2 Distributed Learning, Game-Theoretic, and Matching Approaches 11
1.3.3 Deep Reinforcement Learning for DSA 13
1.3.3.1 Background on Q-learning and Deep Reinforcement Learning (DRL): 13
1.3.4 Existing DRL-Based Methods for DSA 14
1.3.5 Deep Q-Learning for Spectrum Access (DQSA) Algorithm 15
1.3.5.1 Architecture of the DQN Used in the DQSA Algorithm 15
1.3.5.2 Training the DQN and Online Spectrum Access 16
1.3.5.3 Simulation Results 17
1.4 Conclusions 19
Acknowledgments 20
Bibliography 20
2 Reinforcement Learning for Resource Allocation in Cognitive Radio Networks 27
Andres Kwasinski, Wenbo Wang, and Fatemeh Shah Mohammadi
2.1 Use of Q-Learning for Cross-layer Resource Allocation 29
2.2 Deep Q-Learning and Resource Allocation 33
2.3 Cooperative Learning and Resource Allocation 36
2.4 Conclusions 42
Bibliography 43
3 Machine Learning for Spectrum Sharing in Millimeter-Wave Cellular Networks 45
Hadi Ghauch, Hossein Shokri-Ghadikolaei, Gabor Fodor, Carlo Fischione, and Mikael Skoglund
3.1 Background and Motivation 45
3.1.1 Review of Cellular Network Evolution 45
3.1.2 Millimeter-Wave and Large-Scale Antenna Systems 46
3.1.3 Review of Spectrum Sharing 47
3.1.4 Model-Based vs. Data-Driven Approaches 48
3.2 System Model and Problem Formulation 49
3.2.1 Models 49
3.2.1.1 Network Model 49
3.2.1.2 Association Model 49
3.2.1.3 Antenna and Channel Model 49
3.2.1.4 Beamforming and Coordination Models 50
3.2.1.5 Coordination Model 50
3.2.2 Problem Formulation 51
3.2.2.1 Rate Models 52
3.2.3 Model-based Approach 52
3.2.4 Data-driven Approach 53
3.3 Hybrid Solution Approach 54
3.3.1 Data-Driven Component 55
3.3.2 Model-Based Component 56
3.3.2.1 Illustrative Numerical Results 58
3.3.3 Practical Considerations 58
3.3.3.1 Implementing Training Frames 58
3.3.3.2 Initializations 59
3.3.3.3 Choice of the Penalty Matrix 59
3.4 Conclusions and Discussions 59
Appendix A Appendix for Chapter 3 61
A.1 Overview of Reinforcement Learning 61
Bibliography 61
4 Deep Learning-Based Coverage and Capacity Optimization 63
Andrei Marinescu, Zhiyuan Jiang, Sheng Zhou, Luiz A. DaSilva, and Zhisheng Niu
4.1 Introduction 63
4.2 Related Machine Learning Techniques for Autonomous Network Management 64
4.2.1 Reinforcement Learning and Neural Networks 64
4.2.2 Application to Mobile Networks 66
4.3 Data-Driven Base-Station Sleeping Operations by Deep Reinforcement Learning 67
4.3.1 Deep Reinforcement Learning Architecture 67
4.3.2 Deep Q-Learning Preliminary 68
4.3.3 Applications to BS Sleeping Control 68
4.3.3.1 Action-Wise Experience Replay 69
4.3.3.2 Adaptive Reward Scaling 70
4.3.3.3 Environment Models and Dyna Integration 70
4.3.3.4 DeepNap Algorithm Description 71
4.3.4 Experiments 71
4.3.4.1 Algorithm Comparisons 71
4.3.5 Summary 72
4.4 Dynamic Frequency Reuse through a Multi-Agent Neural Network Approach 72
4.4.1 Multi-Agent System Architecture 73
4.4.1.1 Cell Agent Architecture 75
4.4.2 Application to Fractional Frequency Reuse 75
4.4.3 Scenario Implementation 76
4.4.3.1 Cell Agent Neural Network 76
4.4.4 Evaluation 78
4.4.4.1 Neural Network Performance 78
4.4.4.2 Multi-Agent System Performance 79
4.4.5 Summary 81
4.5 Conclusions 81
Bibliography 82
5 Machine Learning for Optimal Resource Allocation 85
Marius Pesavento and Florian Bahlke
5.1 Introduction and Motivation 85
5.1.1 Network Capacity and Densification 86
5.1.2 Decentralized Resource Minimization 87
5.1.3 Overview 88
5.2 System Model 88
5.2.1 Heterogeneous Wireless Networks 88
5.2.2 Load Balancing 89
5.3 Resource Minimization Approaches 90
5.3.1 Optimized Allocation 91
5.3.2 Feature Selection and Training 91
5.3.3 Range Expansion Optimization 93
5.3.4 Range Expansion Classifier Training 94
5.3.5 Multi-Class Classification 94
5.4 Numerical Results 96
5.5 Concluding Remarks 99
Bibliography 100
6 Machine Learning in Energy Efficiency Optimization 105
Muhammad Ali Imran, Ana Flavia dos Reis, Glauber Brante, Paulo Valente Klaine, and Richard Demo Souza
6.1 Self-Organizing Wireless Networks 106
6.2 Traffic Prediction and Machine Learning 110
6.3 Cognitive Radio and Machine Learning 111
6.4 Future Trends and Challenges 112
6.4.1 Deep Learning 112
6.4.2 Positioning of Unmanned Aerial Vehicles 113
6.4.3 Learn-to-Optimize Approaches 113
6.4.4 Some Challenges 114
6.5 Conclusions 114
Bibliography 114
7 Deep Learning Based Traffic and Mobility Prediction 119
Honggang Zhang, Yuxiu Hua, Chujie Wang, Rongpeng Li, and Zhifeng Zhao
7.1 Introduction 119
7.2 Related Work 120
7.2.1 Traffic Prediction 120
7.2.2 Mobility Prediction 121
7.3 Mathematical Background 122
7.4 ANN-Based Models for Traffic and Mobility Prediction 124
7.4.1 ANN for Traffic Prediction 124
7.4.1.1 Long Short-Term Memory Network Solution 124
7.4.1.2 Random Connectivity Long Short-Term Memory Network Solution 125
7.4.2 ANN for Mobility Prediction 128
7.4.2.1 Basic LSTM Network for Mobility Prediction 128
7.4.2.2 Spatial-Information-Assisted LSTM-Based Framework of Individual Mobility Prediction 130
7.4.2.3 Spatial-Information-Assisted LSTM-Based Framework of Group Mobility Prediction 131
7.5 Conclusion 133
Bibliography 134
8 Machine Learning for Resource-Efficient Data Transfer in Mobile Crowdsensing 137
Benjamin Sliwa, Robert Falkenberg, and Christian Wietfeld
8.1 Mobile Crowdsensing 137
8.1.1 Applications and Requirements 138
8.1.2 Anticipatory Data Transmission 139
8.2 ML-Based Context-Aware Data Transmission 140
8.2.1 Groundwork: Channel-aware Transmission 140
8.2.2 Groundwork: Predictive CAT 142
8.2.3 ML-based CAT 144
8.2.4 ML-based pCAT 146
8.3 Methodology for Real-World Performance Evaluation 148
8.3.1 Evaluation Scenario 148
8.3.2 Power Consumption Analysis 148
8.4 Results of the Real-World Performance Evaluation 149
8.4.1 Statistical Properties of the Network Quality Indicators 149
8.4.2 Comparison of the Transmission Schemes 149
8.4.3 Summary 151
8.5 Conclusion 152
Acknowledgments 154
Bibliography 154
Part II Transmission Intelligence and Adaptive Baseband Processing 157
9 Machine Learning-Based Adaptive Modulation and Coding Design 159
Lin Zhang and Zhiqiang Wu
9.1 Introduction and Motivation 159
9.1.1 Overview of ML-Assisted AMC 160
9.1.2 MCS Schemes Specified by IEEE 802.11n 161
9.2 SL-Assisted AMC 162
9.2.1 k-NN-Assisted AMC 162
9.2.1.1 Algorithm for k-NN-Assisted AMC 163
9.2.2 Performance Analysis of k-NN-Assisted AMC System 164
9.2.3 SVM-Assisted AMC 166
9.2.3.1 SVM Algorithm 166
9.2.3.2 Simulation and Results 170
9.3 RL-Assisted AMC 172
9.3.1 Markov Decision Process 172
9.3.2 Solution for the Markov Decision 173
9.3.3 Actions, States, and Rewards 174
9.3.4 Performance Analysis and Simulations 175
9.4 Further Discussion and Conclusions 178
Bibliography 178
10 Machine Learning-Based Nonlinear MIMO Detector 181
Song-Nam Hong and Seonho Kim
10.1 Introduction 181
10.2 A Multihop MIMO Channel Model 182
10.3 Supervised-Learning-based MIMO Detector 184
10.3.1 Non-Parametric Learning 184
10.3.2 Parametric Learning 185
10.4 Low-Complexity SL (LCSL) Detector 188
10.5 Numerical Results 191
10.6 Conclusions 193
Bibliography 193
11 Adaptive Learning for Symbol Detection: A Reproducing Kernel Hilbert Space Approach 197
Daniyal Amir Awan, Renato Luis Garrido Cavalcante, Masahario Yukawa, and Slawomir Stanczak
11.1 Introduction 197
11.2 Preliminaries 198
11.2.1 Reproducing Kernel Hilbert Spaces 198
11.2.2 Sum Spaces of Reproducing Kernel Hilbert Spaces 199
11.3 System Model 200
11.3.1 Symbol Detection in Multiuser Environments 201
11.3.2 Detection of Complex-Valued Symbols in Real Hilbert Spaces 202
11.4 The Proposed Learning Algorithm 203
11.4.1 The Canonical Iteration 203
11.4.2 Practical Issues 204
11.4.3 Online Dictionary Learning 205
11.4.3.1 Dictionary for the Linear Component 206
11.4.3.2 Dictionary for the Gaussian Component 206
11.4.4 The Online Learning Algorithm 206
11.5 Simulation 207
11.6 Conclusion 208
Appendix A Derivation of the Sparsification Metric and the Projections onto the Subspace Spanned by the Nonlinear Dictionary 210
Bibliography 211
12 Machine Learning for Joint Channel Equalization and Signal Detection 213
Lin Zhang and Lie-Liang Yang
12.1 Introduction 213
12.2 Overview of Neural Network-Based Channel Equalization 214
12.2.1 Multilayer Perceptron-Based Equalizers 215
12.2.2 Functional Link Artificial Neutral Network-Based Equalizers 215
12.2.3 Radial Basis Function-Based Equalizers 216
12.2.4 Recurrent Neural Networks-Based Equalizers 216
12.2.5 Self-Constructing Recurrent Fuzzy Neural Network-Based Equalizers 217
12.2.6 Deep-Learning-Based Equalizers 217
12.2.7 Extreme Learning Machine-Based Equalizers 218
12.2.8 SVM- and GPR-Based Equalizers 218
12.3 Principles of Equalization and Detection 219
12.4 NN-Based Equalization and Detection 223
12.4.1 Multilayer Perceptron Model 223
12.4.1.1 Generalized Multilayer Perceptron Structure 224
12.4.1.2 Gradient Descent Algorithm 225
12.4.1.3 Forward and Backward Propagation 226
12.4.2 Deep-Learning Neural Network-Based Equalizers 227
12.4.2.1 System Model and Network Structure 227
12.4.2.2 Network Training 228
12.4.3 Convolutional Neural Network-Based Equalizers 229
12.4.4 Recurrent Neural Network-Based Equalizers 231
12.5 Performance of OFDM Systems With Neural Network-Based Equalization 232
12.5.1 System Model and Network Structure 232
12.5.2 DNN and CNN Network Structure 233
12.5.3 Offline Training and Online Deployment 234
12.5.4 Simulation Results and Analyses 235
12.6 Conclusions and Discussion 236
Bibliography 237
13 Neural Networks for Signal Intelligence: Theory and Practice 243
Jithin Jagannath, Nicholas Polosky, Anu Jagannath, Francesco Restuccia, and Tommaso Melodia
13.1 Introduction 243
13.2 Overview of Artificial Neural Networks 244
13.2.1 Feedforward Neural Networks 244
13.2.2 Convolutional Neural Networks 247
13.3 Neural Networks for Signal Intelligence 248
13.3.1 Modulation Classification 249
13.3.2 Wireless Interference Classification 252
13.4 Neural Networks for Spectrum Sensing 255
13.4.1 Existing Work 256
13.4.2 Background on System-on-Chip Computer Architecture 256
13.4.3 A Design Framework for Real-Time RF Deep Learning 257
13.4.3.1 High-Level Synthesis 257
13.4.3.2 Design Steps 258
13.5 Open Problems 259
13.5.1 Lack of Large-Scale Wireless Signal Datasets 259
13.5.2 Choice of I/Q Data Representation Format 259
13.5.3 Choice of Learning Model and Architecture 260
13.6 Conclusion 260
Bibliography 260
14 Channel Coding with Deep Learning: An Overview 265
Shugong Xu
14.1 Overview of Channel Coding and Deep Learning 265
14.1.1 Channel Coding 265
14.1.2 Deep Learning 266
14.2 DNNs for Channel Coding 268
14.2.1 Using DNNs to Decode Directly 269
14.2.2 Scaling DL Method 271
14.2.3 DNNs for Joint Equalization and Channel Decoding 272
14.2.4 A Unified Method to Decode Multiple Codes 274
14.2.5 Summary 276
14.3 CNNs for Decoding 277
14.3.1 Decoding by Eliminating Correlated Channel Noise 277
14.3.1.1 BP-CNN Reduces Decoding BER 279
14.3.1.2 Multiple Iterations Between CNN and BP Further Improve Performance 279
14.3.2 Summary 279
14.4 RNNs for Decoding 279
14.4.1 Using RNNs to Decode Sequential Codes 279
14.4.2 Improving the Standard BP Algorithm with RNNs 281
14.4.3 Summary 283
14.5 Conclusions 283
Bibliography 283
15 Deep Learning Techniques for Decoding Polar Codes 287
Warren J. Gross, Nghia Doan, Elie Ngomseu Mambou, and Seyyed Ali Hashemi
15.1 Motivation and Background 287
15.2 Decoding of Polar Codes: An Overview 289
15.2.1 Problem Formulation of Polar Codes 289
15.2.2 Successive-Cancellation Decoding 290
15.2.3 Successive-Cancellation List Decoding 291
15.2.4 Belief Propagation Decoding 291
15.3 DL-Based Decoding for Polar Codes 292
15.3.1 Off-the-Shelf DL Decoders for Polar Codes 292
15.3.2 DL-Aided Decoders for Polar Codes 293
15.3.2.1 Neural Belief Propagation Decoders 293
15.3.2.2 Joint Decoder and Noise Estimator 295
15.3.3 Evaluation 296
15.4 Conclusions 299
Bibliography 299
16 Neural Network-Based Wireless Channel Prediction 303
Wei Jiang, Hans Dieter Schotten, and Ji-ying Xiang
16.1 Introduction 303
16.2 Adaptive Transmission Systems 305
16.2.1 Transmit Antenna Selection 305
16.2.2 Opportunistic Relaying 306
16.3 The Impact of Outdated CSI 307
16.3.1 Modeling Outdated CSI 307
16.3.2 Performance Impact 308
16.4 Classical Channel Prediction 309
16.4.1 Autoregressive Models 310
16.4.2 Parametric Models 311
16.5 NN-Based Prediction Schemes 313
16.5.1 The RNN Architecture 313
16.5.2 Flat-Fading SISO Prediction 314
16.5.2.1 Channel Gain Prediction with a Complex-Valued RNN 314
16.5.2.2 Channel Gain Prediction with a Real-Valued RNN 315
16.5.2.3 Channel Envelope Prediction 315
16.5.2.4 Multi-Step Prediction 316
16.5.3 Flat-Fading MIMO Prediction 316
16.5.3.1 Channel Gain Prediction 317
16.5.3.2 Channel Envelope Prediction 317
16.5.4 Frequency-Selective MIMO Prediction 317
16.5.5 Prediction-Assisted MIMO-OFDM 319
16.5.6 Performance and Complexity 320
16.5.6.1 Computational Complexity 320
16.5.6.2 Performance 321
16.6 Summary 323
Bibliography 323
Part III Network Intelligence and Adaptive System Optimization 327
17 Machine Learning for Digital Front-End: a Comprehensive Overview 329
Pere L. Gilabert, David Lopez-Bueno, Thi Quynh Anh Pham, and Gabriel Montoro
17.1 Motivation and Background 329
17.2 Overview of CFR and DPD 331
17.2.1 Crest Factor Reduction Techniques 331
17.2.2 Power Amplifier Behavioral Modeling 334
17.2.3 Closed-Loop Digital Predistortion Linearization 335
17.2.4 Regularization 337
17.2.4.1 Ridge Regression or Tikhonov 2 Regularization 338
17.2.4.2 LASSO or 1 Regularization 339
17.2.4.3 Elastic Net 340
17.3 Dimensionality Reduction and ML 341
17.3.1 Introduction 341
17.3.2 Dimensionality Reduction Applied to DPD Linearization 343
17.3.3 Greedy Feature-Selection Algorithm: OMP 345
17.3.4 Principal Component Analysis 345
17.3.5 Partial Least Squares 348
17.4 Nonlinear Neural Network Approaches 350
17.4.1 Introduction to ANN Topologies 350
17.4.2 Design Considerations for Digital Linearization and RF Impairment Correction 353
17.4.2.1 ANN Architectures for Single-Antenna DPD 354
17.4.2.2 ANN Architectures for MIMO DPD, I/Q Imbalances, and DC Offset Correction 355
17.4.2.3 ANN Training and Parameter Extraction Procedure 357
17.4.2.4 Validation Methodologies and Key Performance Index 361
17.4.3 ANN for CFR: Design and Key Performance Index 364
17.4.3.1 SLM and PTS 364
17.4.3.2 Tone Injection 365
17.4.3.3 ACE 366
17.4.3.4 Clipping and Filtering 368
17.5 Support Vector Regression Approaches 368
17.6 Further Discussion and Conclusions 373
Bibliography 374
18 Neural Networks for Full-Duplex Radios: Self-Interference Cancellation 383
Alexios Balatsoukas-Stimming
18.1 Nonlinear Self-Interference Models 384
18.1.1 Nonlinear Self-Interference Model 385
18.2 Digital Self-Interference Cancellation 386
18.2.1 Linear Cancellation 386
18.2.2 Polynomial Nonlinear Cancellation 387
18.2.3 Neural Network Nonlinear Cancellation 387
18.2.4 Computational Complexity 389
18.2.4.1 Linear Cancellation 389
18.2.4.2 Polynomial Nonlinear Cancellation 390
18.2.4.3 Neural Network Nonlinear Cancellation 390
18.3 Experimental Results 391
18.3.1 Experimental Setup 391
18.3.2 Self-Interference Cancellation Results 391
18.3.3 Computational Complexity 392
18.4 Conclusions 393
18.4.1 Open Problems 394
Bibliography 395
19 Machine Learning for Context-Aware Cross-Layer Optimization 397
Yang Yang, Zening Liu, Shuang Zhao, Ziyu Shao, and Kunlun Wang
19.1 Introduction 397
19.2 System Model 399
19.3 Problem Formulation and Analytical Framework 402
19.3.1 Fog-Enabled Multi-Tier Operations Scheduling (FEMOS) Algorithm 403
19.3.2 Theoretical and Numerical Analysis 405
19.3.2.1 Theoretical Analysis 405
19.3.2.2 Numerical Analysis 406
19.4 Predictive Multi-tier Operations Scheduling (PMOS) Algorithm 409
19.4.1 System Model 409
19.4.2 Theoretical Analysis 411
19.4.3 Numerical Analysis 413
19.5 A Multi-tier Cost Model for User Scheduling in Fog Computing Networks 413
19.5.1 System Model and Problem Formulation 413
19.5.2 COUS Algorithm 416
19.5.3 Performance Evaluation 418
19.6 Conclusion 420
Bibliography 421
20 Physical-Layer Location Verification by Machine Learning 425
Stefano Tomasin, Alessandro Brighente, Francesco Formaggio, and Gabriele Ruvoletto
20.1 IRLV by Wireless Channel Features 427
20.1.1 Optimal Test 428
20.2 ML Classification for IRLV 428
20.2.1 Neural Networks 429
20.2.2 Support Vector Machines 430
20.2.3 ML Classification Optimality 431
20.3 Learning Phase Convergence 431
20.3.1 Fundamental Learning Theorem 431
20.3.2 Simulation Results 432
20.4 Experimental Results 433
20.5 Conclusions 437
Bibliography 437
21 Deep Multi-Agent Reinforcement Learning for Cooperative Edge Caching 439
M. Cenk Gursoy, Chen Zhong, and Senem Velipasalar
21.1 Introduction 439
21.2 System Model 441
21.2.1 Multi-Cell Network Model 441
21.2.2 Single-Cell Network Model with D2D Communication 442
21.2.3 Action Space 443
21.3 Problem Formulation 443
21.3.1 Cache Hit Rate 443
21.3.2 Transmission Delay 444
21.4 Deep Actor-Critic Framework for Content Caching 446
21.5 Application to the Multi-Cell Network 448
21.5.1 Experimental Settings 448
21.5.2 Simulation Setup 448
21.5.3 Simulation Results 449
21.5.3.1 Cache Hit Rate 449
21.5.3.2 Transmission Delay 450
21.5.3.3 Time-Varying Scenario 451
21.6 Application to the Single-Cell Network with D2D Communications 452
21.6.1 Experimental Settings 452
21.6.2 Simulation Setup 452
21.6.3 Simulation Results 453
21.6.3.1 Cache Hit Rate 453
21.6.3.2 Transmission Delay 454
21.7 Conclusion 454
Bibliography 455
Index 459
