This presentation examines the critical concepts of binary compatibility (BC) and source compatibility (SC) within C++ library development, drawing heavily on decades of experience from the KDE and Qt projects. The talk clarifies the distinction between C++ API (Application Programming Interface) and ABI (Application Binary Interface), demonstrating how changes that maintain source compatibility (e.g., adding a default argument to a function) can simultaneously break binary compatibility, and vice versa (e.g., adding an overload).
Key challenges to maintaining BC are explored, particularly regarding name mangling, symbol exporting, and the implications of inline functions and data member layout. The discussion highlights techniques like the PIMPL idiom and reserved fields for class size stability, and proposes a "cheating" mechanism—the "removed API only" pattern—to manage API evolution while preserving the ABI for legacy applications. A core takeaway identifies virtual functions as the primary impediment to long-term ABI stability, as demonstrated by the transition between major library versions.
Summary for C++ Software Engineers:
0:00 Introduction and Scope: The talk, presented by Marc Mutz, focuses on maintaining binary and source compatibility (BC/SC) in large C++ user libraries, specifically discussing the challenges and solutions developed by Qt and KDE. It explicitly limits scope by excluding in-depth discussion of C++20 modules, static linking, and multi-compiler environments.
2:34 Motivating Example: BC vs. SC: The speaker presents a scenario where a library function is updated. Adding a defaulted argument to an existing function maintains source compatibility (SC) but breaks binary compatibility (BC) for dynamically linked applications because the function's mangled name changes. Conversely, adding an overload maintains BC but can break SC for code attempting to take the function's address.
8:20 Definition of Binary Compatibility (BC): BC is defined as a program dynamically linked against an older library version continuing to run without recompilation when updated to a newer version. It applies locally to specific types and functions within a shared library (DSO/DLL).
12:54 API vs. ABI Distinction: The API (Application Programming Interface) is what the C++ compiler sees, allowing for features like overloading and defaulted arguments. The ABI (Application Binary Interface) is what the linker sees, dealing in mangled names and fixed memory layouts. BC requires keeping the ABI stable even if the API changes.
16:44 Name Mangling and ABI Breaking Changes: Name mangling encodes a function's name and signature into a unique symbol for the linker. Changes that affect name mangling inputs (function name, scope, signature parameters) break ABI. Changes to non-inputs (default values, noexcept status, argument names) generally do not break ABI, though platform variations exist (e.g., MSVC on Windows).
21:17 Symbol Exporting and Inlines: On Unix, all external functions are typically exported by default, increasing the attack surface for BC issues. On Windows, explicit exporting (e.g., via __declspec(dllexport)) is required. Inline functions are typically compiled into the client executable (TU), allowing changes without affecting the library's BC. However, wholesale exporting of a class can export inline functions, making them part of the ABI.
25:35 The Golden Rule of Class Exporting: To ensure a stable vtable, polymorphic (virtual) classes should be exported wholesale. Non-polymorphic classes should be exported selectively (e.g., functions only, not inline members) to minimize ABI exposure.
31:36 PIMPL and Virtual Hooks: The PIMPL (Pointer to Implementation) idiom maintains BC by hiding private members behind an opaque pointer, allowing changes to internal data without altering the public class size. A "virtual hook" technique can be used to add new virtual behavior to a class hierarchy without adding new virtual functions, but requires proactive implementation.
40:23 The removed API only Pattern (Cheating): A technique for safely evolving an API. The new function is implemented, and the old function's declaration is hidden from external callers using a macro (e.g., _removed_API_only). The old function definition (now calling the new function internally) is compiled into a dedicated translation unit, preserving the ABI symbol for existing applications while presenting a modern API to new ones.
47:52 Rolling BC Window: The technique can be extended to implement a configurable rolling BC window, where a user can define a cutoff version to automatically remove old ABI symbols from the library if they wish to.
48:45 The Virtual Function Problem: The primary challenge in C++ ABI evolution is virtual functions. Adding a new virtual function to a base class breaks binary compatibility for derived classes compiled against an older version, as it changes the layout of the virtual function table (vtable). This limitation fundamentally constrains ABI evolution across major library versions.