This video reviews two new models from OpenAI: GPT-5.1 Pro and GPT-5.1 Codex Max.
As far as I can tell, GPT-5.1 Pro is an exceptionally powerful reasoning model, demonstrated by its ability to solve a complex, multi-step Defcon cryptographic puzzle in under an hour—a task that took the human presenter days. However, its utility is severely limited. It is incredibly slow, with response times reaching 30 minutes, and it is exclusively available through the buggy ChatGPT web interface, with no API access. This makes its intelligence largely inaccessible for practical integration.
GPT-5.1 Codex Max is an agentic coding model designed for long-running tasks. The presenter's experience with it is initially negative. When tasked with a repository upgrade, the model produced type-unsafe code riddled with as any, failed to validate its work, and required multiple attempts to fix bugs it introduced. It only succeeded after receiving an extremely detailed, "hand-holding" prompt that specified the exact testing and validation commands to run. The conclusion is that while potentially powerful, these new models require a much more prescriptive and explicit prompting style to function reliably.
OpenAI's New Models: Powerful Reasoning, Flawed Execution
00:00:06 New Models Announced: OpenAI has released two new models: GPT-5.1 Pro, a reasoning-focused model, and GPT-5.1 Codex Max, an agentic coding model.
03:06:01 GPT-5.1 Pro's Reasoning Test: The presenter tests GPT-5.1 Pro with a complex Defcon "Gold Bug" puzzle, a multi-layered cryptographic challenge that took his team days to solve.
06:36:01 Astonishing Performance: In under an hour, the model correctly identifies obscure clues (like it being an ADFGX cipher related to the movie Romancing the Stone), deduces the logical path of a smuggled item, and executes the final decryption to solve the puzzle, outputting the correct answer: "WALLET PICKER".
11:29:01 Severe Practical Limitations: Despite its intelligence, GPT-5.1 Pro is frustrating to use. It's extremely slow (taking 30+ minutes for one step) and is only accessible via the buggy ChatGPT web UI, with no API available. This makes its power feel trapped and impractical.
15:48:01 GPT-5.1 Codex Max for Agentic Coding: The second model, Codex Max, is designed for long-running coding tasks and is tested on a project upgrade to the latest Vercel AI SDK.
22:11:01 Initial Coding Failure: The model fails the task repeatedly. It misuses tools, introduces bugs, and writes poor TypeScript, excessively using as any and breaking type safety. Crucially, it never attempts to run a type-check to validate its own code.
30:12:01 Success Requires "Hand-Holding": The model only succeeds after the presenter provides a much more detailed prompt. This new prompt explicitly instructs the model on exactly how to test its work (e.g., "run bun run dryrun") and validate its code (tsc).
30:50:01 Key Takeaway on Prompting: The presenter concludes that these advanced, long-running models require a fundamentally different prompting strategy. To get reliable results, you must provide extremely strict, specific instructions that include the process for verification and testing, as the model will not infer these steps on its own.