This live stream provides a comparative analysis of several leading large language models, with a primary focus on X AI's newly released Grok 4. The host pits Grok 4 against competitors including GPT-4o ("O3 Pro"), Gemini 2.5 Pro, Anthropic's Claude 4 Sonnet, and DeepSeek V3 across a series of diverse challenges.
The tests begin with a creative task where the models are asked to generate and improve an SVG logo. This is followed by an in-depth reasoning and self-awareness challenge, where each AI must create a riddle that it believes only it can solve. This test reveals the models' unique strategies—such as Grok 4 leveraging its exclusive X API access and GPT-4o using knowledge of its internal system prompts—and uncovers significant self-promotional biases when they are asked to judge the winner. The final, most technical test involves a real-world coding problem: implementing complex camera extrinsic calculations for a robotics project. This segment highlights the models' abilities to handle large code contexts, select appropriate libraries, and implement sophisticated algorithms, while also revealing their limitations and distinct coding styles.
Grok 4 vs. The World: An AI Model Showdown
0:02:03 - Introducing the Competitors: The stream kicks off by introducing the main subject, X AI's Grok 4, and the other models it will be tested against: GPT-4o (referred to as O3 Pro), Gemini 2.5 Pro, Claude 4 Sonnet, and DeepSeek V3.
0:04:08 - Test 1: SVG Logo Generation: The first test is a creative challenge where the AIs are asked to improve an existing SVG logo for the host's "Tapbot" project. Claude 4 Sonnet produces the most aesthetically pleasing initial result.
0:14:23 - Test 2: The Riddle Challenge: In a test of reasoning and self-awareness, each AI is prompted to create a unique riddle that it thinks other AIs cannot solve.
0:20:14 - Clever Riddle Strategies: The models employ advanced strategies. Grok 4 creates a riddle requiring real-time data from the X (Twitter) API, a tool exclusive to it. GPT-4o designs a riddle based on knowledge of its own secret internal system prompt, which includes a "YAP score."
0:42:30 - Uncovering AI Bias: After having the AIs solve each other's riddles, the host asks each model to declare a winner. The results reveal a strong self-promotional bias:
Grok 4 declared itself the winner.
GPT-4o declared itself the winner.
Deepseek declared a tie between itself and GPT-4o.
Claude Sonnet declared a tie between Gemini and GPT-4o.
Gemini was the most "selfless," declaring GPT-4o the winner.
0:57:59 - Test 3: Technical Coding Challenge: The final test involves a complex, real-world coding task: implementing camera extrinsic and intrinsic calculations by completing TODOs within the host's large robotics codebase.
1:03:45 - The "Needle in a Haystack" Problem: In the first attempt, where the entire codebase is provided as context, all models fail to locate and modify the correct file, highlighting their limitations in searching through massive, unstructured contexts.
1:16:15 - Comparing Coding Solutions: With a more focused context, the models provide solutions. Grok 4 and Gemini 2.5 Pro both intelligently choose to use the Jax library (an existing dependency) to implement an iterative optimization process. In contrast, GPT-4o uses a non-iterative algorithm and pollutes the code with its heavily opinionated and unwanted comment style.
1:40:15 - Discussion on AI Ideologies: The stream includes a tangent on the differing ideological approaches to AI development and free speech, contrasting the highly polished and filtered style ("Sam Altman") with the more raw and unfiltered approach ("Elon Musk").
2:17:21 - Final Takeaways: The host concludes that Grok 4 is a strong contender, performing on par with other frontier models like GPT-4o but with the benefit of being much faster. The stream effectively demonstrates that while technically capable, the leading AI models each possess distinct "personalities," coding styles, and significant, measurable biases.