Okay, here's a summary of the provided transcript in a self-contained bullet list format, including timestamps, details, and key takeaways:
Trading at Light Speed: Designing Low Latency Systems in C++
0:00 Introduction: The talk focuses on designing low-latency trading systems in C++, emphasizing the importance of design over mere code optimization.
0:13 Automated Trading: It's a "high-stakes game" with increasing trading volume, driven by retail investors and the need for both trading and technological expertise.
1:43 The Need for Speed: Market makers, like Optiver, need low latency to manage a large number of orders and react quickly to market events (e.g., news headlines). Canceling obsolete buy orders quickly is critical.
3:29 System Design Goals: The talk outlines building a low-latency system, covering data model access, system tuning, and performance measurement.
4:35 Design for Performance, Not Just Optimization: A key distinction is made between strategy (overall approach to meet latency goals) and tactics (implementation details). Performance should be considered from the start (strategy), but premature optimization (tactics) should be avoided.
7:44 How Fast is Fast?: Five years prior, 2.5 microseconds was considered fast. Now, trigger-to-target latencies (exchange event to order response) are approaching 10 nanoseconds, achieved with FPGAs, not C++.
10:29 C++ is Not Obsolete: Low-latency C++ is still crucial for: (1) Post-trade checks and risk management around FPGAs. (2) Developing trading strategies and determining trigger prices. (3) Strategies that don't require nanosecond latency, where microsecond-level C++ is sufficient.
13:28 Data Modeling is Crucial: A story is shared about a smart but slow algorithm (B) failing to match the speed of an older one (A) due to poor data model design, leading to cache misses and slow performance "everywhere."
15:28 Underwhelming Profiling: Profiling a poorly designed system reveals no clear hotspots, only general slowness due to poor data locality.
16:36 Data Model Strategy: Size and Locality: Data should fit in CPU caches (L1, L2, L3). Know your data size upfront, consider splitting applications if needed, and be aware of other processes sharing the last-level cache (LLC).
18:07 Real-World Example of Data Model Impact: Moving data around to improve cache fit (without fancy code changes) drastically reduced latency in a real-world trading system.
19:11 Instrument Store Example: A common data structure, the instrument store, is used to illustrate data modeling. Requirements: small size (<1KB), 1-10k instances, fast lookup (not critical), and stable references.
20:12std::unordered_map is Often a Poor Choice: While providing stable references, std::unordered_map scatters data across memory (node-based container), leading to poor locality and cache misses.
21:29std::vector (with lookup) is Better, but Loses Stability:std::vector has good locality, but resizing invalidates references.
21:53stable_vector (Custom Container): A std::vector of boost::static_vector (similar to std::array with a size). It's like std::deque, but with a customizable chunk size for better locality. This provides both contiguous memory and stable pointers.
23:56 Strategy: Keep Things Local and Small: Use the Working Set Size (WSS) to measure data locality. A benchmark shows a huge WSS difference between stable_vector (1MB) and std::unordered_map (400MB) due to page fragmentation.
27:37 Concurrent Data Sharing: The talk moves to sharing data between multiple cores/applications.
27:56 Strategy for Market Data Distribution: A market data receiver distributes data to many applications (e.g., 30 per server). Requirements: low latency (<100ns), low jitter, many consumers, one producer, and consumers should not block the producer.
29:23 Events vs. State: Two approaches for sharing data: Events (notifications of changes) and State (the entire data snapshot). Both are needed: events for low-latency reactions, state for periodic updates (e.g., GUIs).
31:37 Sharing State: Seqlock: Seqlock (originally from the Linux kernel) is presented as an underrated, high-performance lock for this scenario. It is ideal where there are few producers, many consumers, and low contention.
33:07 Seqlock Code Example: A simplified C++ implementation is shown. The writer increments a version, copies data, and increments the version again. The reader checks the version, copies data, and rechecks the version to ensure consistency.
35:09 Seqlock Properties: Optimistic locking (assumes no contention), favors readers, writer is wait-free (always gets the lock). Works well with small data and many small elements.
36:55 Latency Seqlock: Latency equals memory copy time
37:20 Sharing Events: For events (with notifications), a single-producer multi-consumer (SPMC) queue on shared memory is proposed.
38:37 SPMC Queue (V1): Initial queue design uses two indices (pending and index), both updated by the writer. Readers check for overflow using both indices.
42:11 SPMC Queue (V1) Performance: Benchmarked against Disruptor (Java), boost::lockfree::spsc_queue, and MoodyCamel's queue. Good performance, but room for improvement.
43:14 SPMC Queue (V2): Improving Contention: Less contention by spreading atomic counters across the queue elements. Each block has a "valid" bit and a version number.
44:56 SPMC Queue (V2) Code: Essentially a ring buffer of Seqlocks. Benefits from Seqlock's properties.
45:44 SPMC Queue (V2) Performance: Improved performance, especially with multiple readers, due to reduced contention. Scales well.
46:57 System Tuning: The talk moves to tuning the entire server for low latency.
47:02 Checking Server Tuning with rdtsc: A test using the rdtsc instruction (reads the time-stamp counter) reveals whether the system is properly tuned. Poorly tuned systems show high variance in rdtsc latency.
48:30 C-states and P-states: CPU power-saving features (C-states - idle, P-states - frequency scaling) can significantly impact latency. For low-latency servers, these should be disabled, especially C-states (CPU sleeping).
50:37 Shared LLC Optimization: Moving low-latency applications to the same cores and ensuring they don't thrash the LLC can significantly improve performance without code changes.
51:33 Other System Tuning Considerations: NUMA awareness, huge pages.
53:16 Continuous Performance Measurement: Continuously measure performance in production to ensure long-term stability and identify issues.
53:57 Metrics: Collect timestamps of critical system parts (e.g., I/O processing time), queue capacities, and use a queue to publish metrics to a database for visualization (e.g., Grafana).
56:45 Automated Alerting: Have code check metrics against expected values and trigger alerts if thresholds are exceeded.
58:02 Conclusion: The talk covers building blocks for low-latency trading systems, emphasizing the importance of data model design, concurrent data sharing, system tuning, and continuous performance monitoring.