About the Author xvii
Acknowledgements xix
About the Companion Website xxi
1 Introduction 1
1.1 Why Write This Book? 1
1.2 What Is in the Book? 2
1.3 How Should You Use This Book? 3
References 4
2 Video Coding and Video Quality 5
2.1 Introduction 5
2.2 An Overview of Video Coding 6
2.3 Inputs and Outputs 9
2.4 Structural Elements 15
2.5 Prediction 17
2.6 Transform and Quantisation 25
2.7 Bitstream Coding 27
2.8 The Coded Bitstream 30
2.9 Storing and Transmitting the Coded Bitstream 30
2.10 The Decoder 32
2.11 The Video Codec Model 33
2.12 Video Codec Performance 35
2.13 Conclusion 40
References 41
3 A History of Video Coding and Video Coding Standards 43
3.1 Introduction 43
3.2 The Foundations of Video Coding, 1950–1990 44
3.3 Video Coding Standards and Formats: 1990–2021 53
3.4 Comparing Video Coding Standards 65
3.5 Conclusions 66
References 66
4 Structures 69
4.1 Introduction 69
4.2 Coded Video: Sequence to Picture 72
4.3 Coded Video: Picture to Basic Unit 75
4.4 Coded Video: Basic Unit to Block 80
4.5 HEVC Coding Structures 82
4.6 Structures in Versatile Video Coding/H. 266 111
4.7 Conclusion 114
Reference 114
5 Intra Prediction 115
5.1 Introduction 115
5.2 The Intra Prediction Process 118
5.3 Intra Prediction Modes 118
5.4 Prediction Block Sizes 126
5.5 Signalling Intra Prediction Choices 128
5.6 Choosing a Prediction 129
5.7 HEVC Intra Prediction 130
5.8 VVC Intra Prediction 139
5.9 Conclusions 144
References 144
6 Inter Prediction 145
6.1 Introduction 145
6.2 Inter Prediction – the Basics 146
6.3 Forward, Backward and Biprediction 155
6.4 Inter Prediction Block Sizes 159
6.5 Motion Vectors 162
6.6 Sub-Pixel Interpolation 167
6.7 Reference Pictures 176
6.8 Signalling Inter Prediction Choices 186
6.9 Skip Mode 192
6.10 Loop Filter 193
6.11 When Inter Prediction Does Not Find a Good Match 194
6.12 HEVC Inter Prediction 197
6.13 Inter Prediction in VVC 219
6.14 Conclusions 222
References 222
7 Transform and Quantisation 223
7.1 Introduction 223
7.2 Residual Blocks 226
7.3 Block Transforms 228
7.4 Quantisation 246
7.5 Transform and Quantisation in Practice 253
7.6 HEVC Transform and Quantisation 254
7.7 Transform and Quantise in H.266 Versatile Video Coding 272
7.8 Conclusions 273
References 273
8 Entropy Coding 275
8.1 Introduction 275
8.2 Entropy Coding for Video Compression 278
8.3 Pre-processing 283
8.4 Probability Models and Context Adaptation 285
8.5 Variable-Length Coding 286
8.6 Arithmetic Coding 291
8.7 Binary Arithmetic Coding 294
8.8 Context-Adaptive Binary Arithmetic Coding (CABAC) 309
8.9 Entropy Coding in HEVC 314
8.10 Entropy Coding in H.266/VVC 334
8.11 Conclusion 334
References 335
9 Coded Video Filtering 337
9.1 Introduction 337
9.2 Filtering and Video Coding 339
9.3 Detecting and Correcting Video Coding Artefacts 341
9.4 HEVC In-Loop Filtering 347
9.5 VVC Filtering 354
9.6 Conclusions 355
References 355
10 Storing and Transporting Coded Video 357
10.1 Introduction 357
10.2 Storing and Delivering Coded Video 359
10.3 Coded Video File Formats 361
10.4 Transport of Coded Video 365
10.5 Video Rate Control 374
10.6 Error Handling 379
10.7 Conclusions 381
References 382
11 Implementation and Performance 383
11.1 Introduction 383
11.2 Implementing Video Codecs 384
11.3 Software Implementation 384
11.4 Hardware Implementation 388
11.5 Video Codec Performance 390
11.6 Getting Started with Experiments 397
11.7 Conclusion 405
References 405
12 Conclusions 407
12.1 What This Book Has and Has Not Covered 407
12.2 Where Is Video Coding Going Next? 408
12.3 Where Should You Go Next? 409
References 409
Glossary 411
Index 415
