The Most Dangerous Codec in the World: Finding and Exploiting Vulnerabilities in H.264 Decoders
06-10, 10:30–11:00 (America/New_York), Grand Salon

Modern video encoding standards such as H.264 are a marvel of hidden complexity. But with hidden complexity comes hidden security risk. Decoding video today involves interacting with dedicated hardware accelerators and the proprietary, privileged software components used to drive them. The video decoder ecosystem is obscure, opaque, diverse, highly privileged, largely untested, and highly exposed -- a dangerous combination.

We introduce H26Forge, a framework that carefully crafts video files to expose edge cases in H.264 decoders. H26Forge’s key insight is operating on the syntax elements rather than on the encoded bitstring to build syntactically correct but semantically spec-non-compliant video files. These files cause H.264 decoders to find themselves in undefined states or unhandled errors.

We used H26Forge to uncover numerous vulnerabilities across the video decoder ecosystem, including kernel memory corruption bugs in iOS, memory corruption bugs in Firefox and VLC for Windows, and video accelerator and application processor kernel memory bugs in multiple Android devices. These bugs have been acknowledged by multiple vendors including Apple, Mozilla, and FFmpeg.

In this talk, we will share the internals of H.264 decoding pipelines and describe our experience developing and using H26Forge to find vulnerabilities. H26Forge and its related tools are fully open source and available to participants. We will also explore how participants can use H26Forge to test the security of H.264 decoders on the platforms they use.

See also: Presentation slides (4.2 MB)

Willy R. Vasquez is a PhD student at The University of Texas at Austin working on cryptography and systems security. His research currently focuses on the security of hardware video decoders and building out zero-knowledge proof technologies. His research interests lie in privacy, systems security, cryptosystems, and formal methods.