Coding Video

About the Author xvii

Acknowledgements xix

About the Companion Website xxi

1 Introduction 1

1.1 Why Write This Book? 1

1.2 What Is in the Book? 2

1.3 How Should You Use This Book? 3

References 4

2 Video Coding and Video Quality 5

2.1 Introduction 5

2.2 An Overview of Video Coding 6

2.3 Inputs and Outputs 9

2.4 Structural Elements 15

2.5 Prediction 17

2.6 Transform and Quantisation 25

2.7 Bitstream Coding 27

2.8 The Coded Bitstream 30

2.9 Storing and Transmitting the Coded Bitstream 30

2.10 The Decoder 32

2.11 The Video Codec Model 33

2.12 Video Codec Performance 35

2.13 Conclusion 40

References 41

3 A History of Video Coding and Video Coding Standards 43

3.1 Introduction 43

3.2 The Foundations of Video Coding, 1950-1990 44

3.3 Video Coding Standards and Formats: 1990-2021 53

3.4 Comparing Video Coding Standards 65

3.5 Conclusions 66

References 66

4 Structures 69

4.1 Introduction 69

4.2 Coded Video: Sequence to Picture 72

4.3 Coded Video: Picture to Basic Unit 75

4.4 Coded Video: Basic Unit to Block 80

4.5 HEVC Coding Structures 82

4.6 Structures in Versatile Video Coding/H. 266 111

4.7 Conclusion 114

Reference 114

5 Intra Prediction 115

5.1 Introduction 115

5.2 The Intra Prediction Process 118

5.3 Intra Prediction Modes 118

5.4 Prediction Block Sizes 126

5.5 Signalling Intra Prediction Choices 128

5.6 Choosing a Prediction 129

5.7 HEVC Intra Prediction 130

5.8 VVC Intra Prediction 139

5.9 Conclusions 144

References 144

6 Inter Prediction 145

6.1 Introduction 145

6.2 Inter Prediction - the Basics 146

6.3 Forward, Backward and Biprediction 155

6.4 Inter Prediction Block Sizes 159

6.5 Motion Vectors 162

6.6 Sub-Pixel Interpolation 167

6.7 Reference Pictures 176

6.8 Signalling Inter Prediction Choices 186

6.9 Skip Mode 192

6.10 Loop Filter 193

6.11 When Inter Prediction Does Not Find a Good Match 194

6.12 HEVC Inter Prediction 197

6.13 Inter Prediction in VVC 219

6.14 Conclusions 222

References 222

7 Transform and Quantisation 223

7.1 Introduction 223

7.2 Residual Blocks 226

7.3 Block Transforms 228

7.4 Quantisation 246

7.5 Transform and Quantisation in Practice 253

7.6 HEVC Transform and Quantisation 254

7.7 Transform and Quantise in H.266 Versatile Video Coding 272

7.8 Conclusions 273

References 273

8 Entropy Coding 275

8.1 Introduction 275

8.2 Entropy Coding for Video Compression 278

8.3 Pre-processing 283

8.4 Probability Models and Context Adaptation 285

8.5 Variable-Length Coding 286

8.6 Arithmetic Coding 291

8.7 Binary Arithmetic Coding 294

8.8 Context-Adaptive Binary Arithmetic Coding (CABAC) 309

8.9 Entropy Coding in HEVC 314

8.10 Entropy Coding in H.266/VVC 334

8.11 Conclusion 334

References 335

9 Coded Video Filtering 337

9.1 Introduction 337

9.2 Filtering and Video Coding 339

9.3 Detecting and Correcting Video Coding Artefacts 341

9.4 HEVC In-Loop Filtering 347

9.5 VVC Filtering 354

9.6 Conclusions 355

References 355

10 Storing and Transporting Coded Video 357

10.1 Introduction 357

10.2 Storing and Delivering Coded Video 359

10.3 Coded Video File Formats 361

10.4 Transport of Coded Video 365

10.5 Video Rate Control 374

10.6 Error Handling 379

10.7 Conclusions 381

References 382

11 Implementation and Performance 383

11.1 Introduction 383

11.2 Implementing Video Codecs 384

11.3 Software Implementation 384

11.4 Hardware Implementation 388

11.5 Video Codec Performance 390

11.6 Getting Started with Experiments 397

11.7 Conclusion 405

References 405

12 Conclusions 407

12.1 What This Book Has and Has Not Covered 407

12.2 Where Is Video Coding Going Next? 408

12.3 Where Should You Go Next? 409

References 409

Glossary 411

Index 415