Abstract:

This Hacker News discussion explores the decision to replace Docker with Nix for production workloads, particularly in high-performance Machine Learning (ML) environments. Proponents argue that the inherent rigidity of "golden" Docker containers leads to significant wasted resources by preventing rapid adoption of critical driver, kernel, and firmware updates necessary for the highly immature ML hardware and software stack. This inflexibility exacerbates issues like GPU component failures and data corruption, evidenced by high rates of unexpected job interruptions cited in the Llama 3 whitepaper. Critics, however, caution that Nix introduces a burdensome "complexity fetish" and suffers from a poorer Developer Experience (DX) compared to Docker, suggesting that simple relocatable builds might be a better solution than adding new container layers. The discussion also touches upon alternative container runtimes like systemd-nspawn.

Replacing Docker with Nix: Production Challenges in Machine Learning

• Initial Comment (nickysielicki): Expresses zero surprise that knowledgeable ML teams are migrating from Docker to Nix. The inflexibility of "golden" Docker containers prevents necessary software upgrades (drivers, kernels, toolkits), leading to millions of dollars wasted due to widespread bugs and stability issues in the ML ecosystem.
• Detailed Cost of Outdated Software (nickysielicki reply): Explains that modern ML cards operate in an unstable, immature ecosystem rife with bugs. Using outdated software out of "superstition" exacerbates problems.
• Llama 3 Evidence (nickysielicki reply): Cites the Llama 3 whitepaper, noting that 419 unexpected job interruptions occurred during a 54-day period, with roughly 78% attributed to confirmed or suspected hardware/driver issues (GPU failures, silent data corruption).
• Technical Dependency Issues (nickysielicki reply): Highlights the complex dependencies between drivers, firmware, and low-level software (like NCCL, which contains many workarounds), making validation impossible across all permutations. Specifically mentions the importance of kernel features like HMM (Heterogeneous Memory Manager), introduced in v6.1, for Nvidia UVM.
• Critiques and Developer Experience (DX) (CuriouslyC): While liking Nix, the commenter argues that Docker still offers a vastly superior DX, advising users to "use the right tool."
• Complexity Fetishism (imiric, HumanOstrich): Critics joke that adopting Nix (or rewriting everything in Rust) often stems from a "complexity fetish," creating two problems instead of solving one.
• Alternative: Relocatable Builds (forrestthewoods): Suggests producing relocatable builds that vendor dependencies and don't require external infrastructure like Docker or Nix, arguing that Docker was created because "Linux sucks at running computer programs."
• Docker Overkill and Portability Failure (musicale): Argues that Docker is excessive for simple app packaging, noting that its promised portability fails in ML anyway, as the CUDA version in the container must strictly match the kernel driver and GPU firmware.
• Systemd Alternatives (mianos, miladyincontrol): Mentions that systemd-nspawn is already capable of running OCI containers and offers robust, scalable features like fine-grained permissions and leveraging BTRFS snapshots for fast, ephemeral containers.