This video investigates the capability of AI coding agents to perform Test-Driven Development (TDD), a process characterized by iterative "red, green, refactor" cycles. The presenter highlights that current AI agents typically bypass this, opting for a "big bang" approach, and that direct prompting for TDD often yields mixed results. To address this, the video introduces and experiments with "TDD Guard," an external library designed to enforce TDD principles.
The experiment involves implementing a "split basket" feature in a .NET e-commerce application. The first attempt, using only a carefully crafted TDD-suggesting prompt with Cloud Code, completed in approximately 4 minutes but adopted a "test-first" strategy rather than strict TDD, generating all tests and then all production code in large batches. The second attempt, utilizing TDD Guard, successfully enforced single-test-at-a-time development and outside-in coding, preventing the AI from deviating from TDD principles. However, this method took 10 minutes, significantly longer and consuming more tokens, due to TDD Guard's internal mechanism of spinning up an "AI judge" (another LLM instance) to validate each code change against TDD rules.
While TDD Guard effectively enforces strict TDD and can lead to slightly higher code quality by preventing AI "freewheeling," the presenter concludes that its trade-offs in terms of speed and token consumption currently make it personally "not worth it," though he recommends it for those specifically seeking strong TDD enforcement.
Summarizing AI Coding Agents and Test-Driven Development
0:00:03 Test-Driven Development (TDD) Overview: TDD is an iterative software development process consisting of three phases: "red" (write a failing test), "green" (write simplest code to make the test pass), and "refactor" (clean up code). It helps engineers avoid goldplating and stay focused.
0:00:59 AI Agents' Challenge with TDD: AI coding agents typically do not naturally follow TDD's iterative cycle, preferring a "big bang" approach where they generate all code and tests at once. Prompting techniques, even from TDD pioneers like Ken Beck, yield only "average at best" results in enforcing TDD.
0:01:55 Introducing TDD Guard: The presenter found "TDD Guard," a library designed to provide stronger guard rails to enforce the TDD process with AI coding agents.
0:02:12 Experiment Setup: The experiment uses the .NET eShopOnWeb reference application to implement a new feature: "splitting a basket" (dividing items into a "cheap" basket for items under $100 and an "expensive" basket).
0:04:55 Experiment 1: Best Effort Prompting (No TDD Guard):
0:06:01 Process: The AI (Cloud Code) planned to write all failing unit tests first, then implement all production code in a "big bang" approach, which is "test-first" but not true TDD.
0:07:43 Outcome: The AI successfully implemented the feature in approximately 4 minutes but did not follow the iterative TDD cycle. It also invented a method signature not initially requested.
0:10:38 Enforcement Mechanism: TDD Guard works by implementing hooks that intercept code edits from the AI. It uses its own internal LLM ("AI judge") to evaluate whether proposed changes adhere to TDD principles (e.g., only one new failing test at a time, outside-in development). If violations are detected, the changes are blocked.
0:11:04 Outcome: TDD Guard successfully enforced the iterative TDD cycle, allowing tests and code changes one at a time and guiding the AI to work top-down (outside-in) from the service layer, rather than bottom-up.
0:14:40 AI Learning: The AI even detected and updated its own "bad test" that wasn't asserting enough.
0:15:48 Performance: This approach took 10 minutes, significantly longer than the "best effort" attempt.
0:15:53 How TDD Guard Works: It's an npm package that integrates with AI assistant hooks. It utilizes a language-specific component (e.g., a .NET test framework replacement) to run tests and output results in a specific format. An internal LLM acts as a "judge," receiving prompts defining TDD principles, current test outcomes, and proposed file changes, then outputs JSON indicating approval or rejection.
0:19:09 Conclusion and Trade-offs: The presenter appreciates TDD Guard's philosophy and its ability to enforce TDD where prompting fails, leading to potentially higher code quality. However, the trade-offs of increased time (slower) and token consumption (due to the internal AI judge) make it personally "not worth it" for him, though he recommends it as the best tool available for strict TDD enforcement with AI.