Abstract:

This presentation outlines the innovative software decoding techniques implemented in edge264, an H.264/AVC decoder designed for Progressive High and MVC 3D profiles. Developed by Thibault Raffaillac, edge264 is benchmarked as highly performant and extremely compact compared to established decoders like FFmpeg, libavc, and openh264. Benchmarks show it is up to 10% faster on Intel Macs and 4x faster on ARM platforms (Raspberry Pi 5) while having a significantly smaller code and binary footprint. The core of the talk details five technical innovations: an optimized Piston cached bitstream reader, on-the-fly SIMD unescaping for AVC bytestream parsing, multiarch SIMD programming using GCC vector extensions to maximize performance across Intel and ARM architectures, utilization of the Structure of Arrays (SoA) pattern for efficient memory management, and deferred error checking to minimize branches in critical parsing paths.

More Innovations in H.264/AVC Software Decoding

• Introduction to edge264 (0:00): edge264 is an H.264/AVC software decoder supporting Progressive High & MVC 3D profiles. It is multi-threaded and supports Intel (SSE2/SSSE3) and ARM64 (NEON) across Linux, Windows, and Mac.
• Benchmarks (2:00): edge264 is highly competitive in speed, showing 10% faster performance than FFmpeg on a 2015 Macbook (Intel) and 4x faster on the Raspberry Pi 5 (ARM). Crucially, its code size and binary size are dramatically smaller than FFmpeg, libavc, and openh264.
• Technique 1: Piston Cached Bitstream Reader (5:00): This technique optimizes the reading of variable-length codes using an int64[2] "piston cache" that minimizes dependencies and read/write operations compared to traditional methods.
• Technique 2: On-the-fly SIMD Unescaping (9:00): When parsing AVC bytestreams, delimiters (like 003 in MPEG-TS) must be escaped. edge264 avoids the typical fast pre-scan approach by using SIMD instructions at the cache refill stage to perform unescaping on the fly, leading to faster parsing.
• Technique 3: Multiarch SIMD Programming (14:00): The decoder uses GCC vector extensions alongside standard vector intrinsics to achieve maximum performance on both Intel (SSE) and ARM (NEON).
  • Takeaway: The development workflow emphasizes unit testing and aliasing NEON instructions with SSE (or using #if/#else fallback) but often requires specialized helper functions and even inline assembly to manage compiler limitations and force optimal instruction generation (e.g., for multiple-latency instructions like mla).
• Technique 4: Structure of Arrays (SoA) Pattern (22:00): Data structures are redesigned by "exploding" arrays of structs into independent arrays (e.g., frame_buffers, picture_order_counts, and flags are separate arrays).
  • Benefit: Enables the replacement of loops with fast vector operations and built-in functions (like __builtin_ctz for finding an empty frame buffer).
  • Tradeoff: Improves efficiency but can increase cache pollution and reduce code intuition.
• Technique 5: Deferred Error Checking (25:00): Traditional parsing uses immediate error checking via goto statements, which adds "noise" for the branch predictor. edge264 defers validation checks until the end of headers.
  • Takeaway: This approach significantly reduces conditional branches inside the core parsing code, improving branch prediction efficiency and overall speed, as exact error location is less crucial for viewers and media players.
• Roadmap (29:00): The immediate roadmap includes finishing multithreading, establishing fuzzying and Continuous Integration (CI), creating plugins for VLC/ffmpeg/GStreamer, and adding support for ARM32.