This blog post introduces Fork Union, a minimal (~300-line) C++ and Rust library designed for high-performance, fork-join parallelism. The author argues that popular thread-pool libraries like C++'s Taskflow and Rust's Rayon are often an order of magnitude slower than the industry-standard OpenMP for simple parallel workloads. This performance gap is attributed to the overhead of their asynchronous task queue models and four common performance pitfalls, dubbed the "four horsemen": mutexes, heap allocations, compare-and-swap (CAS) stalls, and false sharing. Through benchmarks on a 96-core AWS Graviton 4 machine, Fork Union is shown to significantly outperform these libraries and achieve performance within 20% of OpenMP, demonstrating the efficiency of a lean, purpose-built approach that avoids these pitfalls.
Introducing Fork Union: A Lean Alternative to Modern Thread Pools
The Performance Gap in Modern Libraries: The author finds that many popular C++ (Taskflow) and Rust (Rayon, Tokio) thread-pool libraries are up to 10x slower than OpenMP for classic fork-join workloads. This is because they are built on heavier, asynchronous task queue abstractions not ideal for simple, blocking parallelism.
The "Four Horsemen" of Poor Performance: The performance issues in heavier libraries are traced back to four key problems:
Locks and Mutexes: These can lead to expensive system calls and context switches under contention.
Memory Allocations: Frequent heap allocations for futures, promises, and tasks in memory-constrained environments create unpredictable performance.
Atomics and CAS: Over-reliance on Compare-and-Swap (CAS) operations leads to significant stalls and performance degradation under contention, especially compared to faster atomic instructions like LOCK ADD.
False Sharing: Failure to properly align and pad internal counters can cause different cores to thrash the same cache line, creating a major bottleneck.
Fork Union: A Minimalist Solution: To solve this, the author created Fork Union, a ~300-line library for C++ and Rust. It avoids the "four horsemen" by using a simple, lightweight design with no external dependencies, no exceptions/panics in its core logic, and over-aligned internal counters to prevent false sharing.
Benchmark Results: On a 96-core AWS Graviton 4 CPU, Fork Union demonstrates significant performance gains.
C++: It is over 6x faster than Taskflow and achieves ~80% of OpenMP's throughput.
Rust: It is ~9x faster than Rayon and ~10x faster than Smol's async executor.
A Simple, Focused API: Fork Union provides a minimal API focused on fork-join patterns, similar to OpenMP directives. It includes functions like for_each_thread, for_each_static, and for_each_dynamic, which directly map to common parallel loop structures, unlike the more complex task-graph APIs of libraries like Taskflow.
Key Takeaway: For high-performance, fork-join parallelism, a lean, specialized library that deliberately avoids common concurrency overheads can vastly outperform more feature-rich, general-purpose asynchronous frameworks. The right tool must be chosen for the specific workload.