Abstract:

In this video, Rust expert and self-described "ML Luddite" Jon Gjengset puts agentic AI coding assistants to the test by attempting to implement complex changes in his type-safe spatial math crate, Sguaba. His goal is to determine if tools like Claude Code can handle non-trivial logic and advanced type system manipulations, tasks he suspects are beyond their pattern-replication capabilities.

The stream documents his "pair programming" session with the AI to implement two features: a macro with compile-time validation and a type-state builder that enforces a specific method call order. The process is a detailed, iterative dialogue, requiring Jon to provide constant, expert-level guidance and corrections to steer the AI toward a correct and idiomatic solution. While the AI successfully generates the code and thorough tests, it completes only two of the four planned tasks before exhausting the user's token limit. Jon concludes that for this kind of complex, non-boilerplate work, the tool did not necessarily save time and required his deep expertise to guide, though he acknowledges its value in generating comprehensive tests.

A Skeptic's Guide to Vibe Coding: Testing AI on Complex Rust

• 0:00:00 - The Skeptic's Hypothesis: Jon outlines his skepticism toward ML coding tools. He believes they excel at pattern replication (e.g., boilerplate, common APIs) but struggle with tasks requiring deep reasoning, such as navigating Rust's advanced type system and borrowing rules.
• 0:07:00 - Introducing the Testbed: The Sguaba Crate: He provides an in-depth explanation of the sguaba crate, a library designed for type-safe 3D spatial mathematics. Its reliance on Rust's type system to prevent errors in coordinate transformations makes it a challenging test for an AI assistant.
• 0:25:18 - Task 1: A Compile-Time Asserting Macro: The first goal is to create a macro for constructing Bearing and WGS84 coordinate types. The macro must validate inputs like elevation and latitude at compile time, eliminating the need for runtime expect() calls on constant values.
• 0:34:42 - The Need for Expert Guidance: The process reveals the necessity of detailed, iterative prompting. Jon has to reject Claude's initial, flawed proposals and guide it step-by-step toward a robust macro, correcting its approach to syntax, handling units (degrees vs. radians), and dealing with negative numbers.
• 1:08:10 - Enforcing Correctness with Compile-Fail Tests: A key part of the process involves Jon teaching the AI to write compile_fail doc tests. This ensures that the macros correctly produce compilation errors for invalid inputs, a more advanced testing strategy.
• 1:30:38 - Task 2: A Type-State Builder for Ordered Operations: The second goal is to create a builder for an Orientation that uses the type-state pattern. This enforces that users set the yaw, pitch, and roll angles in the correct, sequential order, preventing logical errors.
• 1:40:36 - Combining Patterns for an Elegant Solution: The AI successfully implements the feature, but only after Jon guides it to combine multiple patterns. He prompts it to use a single type parameter for the state and a second generic parameter to create a unified builder for two different target types (Orientation and Rotation), avoiding significant code duplication.
• 2:27:02 - Hitting the Token Limit: After approximately 2.5 hours of intensive prompting and coding on two features, Jon exhausts his daily usage limit on the Claude Pro plan, forcing the experiment to end.
• 2:28:17 - The More Complex, Unfinished Tasks: He briefly describes the two more difficult features he was unable to attempt: making a Vector type generic over physical quantities (like velocity or acceleration) and adding support for N-vectors, a different system for representing coordinates on a sphere.
• 2:36:06 - Final Verdict: Skepticism Remains: Jon concludes that the experience did not disprove his initial hypothesis. He notes the process wasn't significantly faster than coding himself and required his constant expert intervention. However, he praises the AI for generating more thorough tests and commit messages than he might have written, but questions if the time, token, and energy cost is justified for such complex, non-standard tasks.