Here's a summary of the provided transcript in the requested format:

Not Getting Lost in Translation - A Deep Dive into Compile-Time String Translation

• 50:21 Introduction: The presentation focuses on string translation, both in the context of human languages and computer languages (compilation). It emphasizes the importance of clear and comprehensible communication to both users and compilers.

• 51:28 Use Case: Industrial Machine Inspection: The speaker's company builds machines that inspect train wheel axles for defects. This real-world application motivates the need for robust and adaptable string translation.

• 54:27 Evolution of String Handling:

  • Initially (2009), the company used boost::format for string concatenation, supporting only German.
  • Later (2014), they needed to support multiple languages due to international customers, leading them to boost::locale and the gettext system.

• 56:34 gettext and boost::locale:

  • gettext is a widely used, mature translation system with tools for marking translatable strings, extracting them, and managing translations.
  • boost::locale provides a C++ interface to gettext functionality.
  • Key concepts: message catalogs (collections of translations), PO files (portable object, text-based translation files), and MO files (machine object, compiled translation files).

• 1:00:05 Standardization of gettext: The gettext approach has recently been standardized, validating the speaker's company's choice.

• 1:02:34 Portable Output Files. Text-based and hold key-value pairs of original strings with their respective translations.

• 1:04:53 Plural Forms and Unicode: The Unicode Standard (specifically the Common Locale Data Repository, or CLDR) provides rules for handling plural forms in different languages, which can vary significantly.

• 1:07:56 Plural Forms Formulas. Plural formulas are expressed as single-line C expressions.

• 1:08:08 Issues with boost::locale: While functional, boost::locale performs unnecessary encoding conversions (and thus string allocations) even when only working with Unicode, impacting performance.

• 1:09:21 Transition to fmt::format: Around 2018, the company switched to fmt::format (later std::format) for better type safety and performance. This involved a shift from type-erased to type-full interfaces (although fmt cleverly avoids code bloat).

• 1:11:53 Type-Based Metaprogramming: The presentation delves into type-based metaprogramming techniques used in fmt::format, including template specialization, overload resolution, and constexpr. It highlights the importance of SFINAE (Substitution Failure Is Not An Error) and the potential performance costs (Rule of Heule).

• 1:14:41 The Rule of Heule: Ranks template metaprogramming techniques from cheapest to most expensive.

• 1:17:32 Impact of std::format (C++20): C++20 adopted std::format with changes (P2216) that made the format string a compile-time constant (consteval). This prevents runtime translation using the standard library alone.

• 1:23:47 consteval and Constant Evaluation: The presentation explains consteval and constant evaluation, emphasizing that the compiler has access to all information during compilation.

• 1:25:54 The Challenge: The core problem is to achieve runtime string translation (based on user preference) while still leveraging the compile-time checks of std::format.

• 1:27:56 Custom Solution: The speaker developed a custom library that mimics std::format's interface but adds runtime translation capabilities. This involves:

  • A custom format_string type.
  • Compile-time checks for format string validity (delegated to std::format).
  • Compile-time checks for consistency between singular/plural forms and arguments.

• 1:30:29 Compile-Time Hash Calculation: The library goes further by calculating hashes of the untranslated strings at compile time. This means the runtime representation only needs to store a hash value, not the entire string.

• 1:31:52 Runtime Lookup: At runtime, the library uses the pre-calculated hash and the selected language/plural form to look up the correct translated string.

• 1:33:01 Remaining Questions (and Solutions):

  1. Which argument determines the plural form? Solved using a find_if-like operation on the type list (implemented with fold expressions due to C++20 constraints).
  2. How to get the value of that argument? Solved by creating a type-erased array of pointers to the arguments, then using the index from step 1 to retrieve and cast the correct value.
  3. How to handle translation of arguments themselves? Solved using type-based metaprogramming to select a translation function based on the argument type.

• 1:34:55 C++26 Features: The speaker mentions that C++26 features (reflection, pack indexing) would simplify some of the techniques used.

• 1:40:41 Fold Expressions. Used in C++17 to replace recursive template expansions.

• 1:43:15 Compile-Time Integration Test: The most impressive part: the speaker demonstrates a compile-time integration test that:

  • Extracts translatable strings from the source code.
  • Uses gettext tools (in the build system) to create PO and MO files.
  • Embeds the MO file data into the source code (using a custom embed-like mechanism).
  • Performs the entire translation process (including plural form selection) at compile time, verifying the output against expected results. This involves millions of evaluation steps by the compiler.

• 1:47:53 Runtime Equivalence: The same test can also be run at runtime, using the MO file from the file system, demonstrating that the compile-time and runtime logic are equivalent.

• 1:48:40 Resources: The speaker provides links to relevant papers and proposals. A GitHub repository for the library is planned but not yet available.

• Q&A:

  • Translation process sends .po files.
  • Floating point localization handled elsewhere.
  • Custom embed solution
  • Fold expression vs recursive template instantiation

• 1:54:32 Linguistic Clarification Translation should focus on full sentences instead of direct word replacements.