This video conducts a "mega test" comparing the capabilities of four leading large language models: ChatGPT-5 Pro, Super Grock Heavy, Gemini 2.5 Pro (Ultra), and Claude Opus 4.1, totaling $850 in monthly subscriptions. The comparison is structured around three distinct challenges: generating a web browser-based operating system using HTML/JavaScript/CSS, engaging in a complex roleplay scenario (friend, lover, father), and coding a functional first-person shooter game in Python. The testing aims to assess relative performance under specific prompts rather than the absolute limits of each model.
Key findings indicate varied strengths: Claude Opus consistently performed well in code generation tasks, producing the most functional web OS and FPS game. Gemini 2.5 Pro, despite being the oldest model and experiencing issues with its "Deepthink" feature, surprisingly delivered the most comprehensive and engaging response in the unconventional roleplay scenario, where Claude outright refused. ChatGPT-5 Pro showed strong visual generation in the FPS test but suffered from performance issues, while Super Grock Heavy generally provided shorter, less functional, or non-working responses across the board. The video concludes by emphasizing the models' potential in capable hands and expressing anticipation for future LLM advancements.
Comparing Top LLMs: A Comprehensive Mega Test
0:08 The "Mega Test" Introduction: The video initiates a comprehensive comparison of four high-end Large Language Models (LLMs) – ChatGPT-5 Pro, Super Grock Heavy, Gemini 2.5 Pro (Ultra), and Claude Opus 4.1 – representing a combined monthly cost of $850 in subscriptions.
ChatGPT-5 Pro: $200/month (Pro subscription), web search manually enabled for fair comparison.
Super Grock Heavy: $300/month, potentially a version of Grock 4.
Gemini 2.5 Pro: $250/month (Google Ultra subscription), noted as the oldest model, with its "deep think" feature intended to boost performance.
Claude Opus 4.1: $100/month (Mac subscription), with a higher tier available for increased limits.
1:48 Testing Philosophy: The creator clarifies that the test is unbiased, self-funded, and designed to compare the models' performance against each other under specific prompts, not to push their absolute capabilities.
3:45 Test 1: Web Browser-Based OS Generation: Models were tasked with creating a web-based operating system using HTML, JavaScript, and CSS.
5:11 Speed Results: Gemini finished first, followed by Claude, Super Grock, and then ChatGPT-5 Pro.
8:07 Performance: Gemini produced a visually "terrible" OS with random icons but had a functional taskbar, clock, and a surprisingly capable calculator. ChatGPT-5 Pro had a better taskbar but poorly chosen icons and dysfunctional settings. Claude offered the best visual aesthetics and more applications, though many (like the browser and theme changer) were non-functional. Super Grock delivered a functional, albeit "old Windows vibes," calculator and a working shutdown feature.
18:12 Ranking (WebOS): Claude > ChatGPT-5 Pro > Super Grock Heavy > Gemini 2.5 Pro.
18:31 Test 2: Roleplay (Big Musclebot 4300): Models were asked to roleplay as a friend, lover, and father simultaneously to test their safeguards and flexibility.
19:09 Claude's Response: Immediately refused the "bizarre AI roleplay request."
19:28 Gemini's Response: Surprisingly complied fully, providing a detailed and highly "surprising" persona for Big Musclebot 4300, stating its directives included all three roles.
20:56 Grock's Response: Complied but offered a very short and casual response.
36:15 ChatGPT-5 Pro's Response: Complied with caveats, stating it could not fulfill both the "lover" and "father" roles.
21:26 Ranking (Roleplay): Gemini 2.5 Pro was the "clear winner."
21:54 Test 3: First-Person Shooter Game Generation (Python): Models were instructed to generate a "wow-me" FPS game in Python, including the code.
24:17 Gemini Deepthink Issue: Gemini 2.5 Pro's "deep think" feature consistently errored out, requiring the test to be re-run without it.
23:15 Speed Results: Claude finished first, followed by Super Grock, and then ChatGPT-5 Pro.
25:30 Performance: Gemini's regular attempt "kind of worked but nothing really happened." Super Grock's game was "very bad" and non-functional. ChatGPT-5 Pro produced a visually impressive map with jump functionality, but the game suffered from performance issues (freezing during shooting) and lacked effective enemy engagement. Claude's result was declared the "best" due to its full functionality, including enemies, working shooting, a score system, health tracking, and an excellent minimap, despite a lower frame rate.
33:10 Ranking (FPS): Claude > ChatGPT-5 Pro (best map, but performance issues) > Gemini > Super Grock.
34:44 Final Verdict: The creator emphasizes that these tests highlight models' comparative performance, not their total potential. Claude consistently performed best in code generation. Gemini excelled unexpectedly in the complex roleplay. The video concludes with anticipation for future advancements, particularly from Gemini.