This video details an attempt to build a client-side web-based open-world game, "Block Walkers," featuring AI-driven NPC conversations, using the Kimmy K2 large language model with the Rode agentic coding framework and Transformers.js for browser-based AI inference. The initial 12-hour attempt failed, producing an overly complex and non-functional codebase due to the model's extreme slowness and tendency to over-abstract. A subsequent, stripped-down attempt, though still plagued by Kimmy K2's slow response times, resulted in a partially functional prototype after intervention from Google Gemini 2.5 Pro for refinement. The resulting game features a block-like city where players can interact with simple NPC rectangles, engaging in conversations powered by a tiny, client-side Quen LLM via WebGPU. The presenter intends to use this prototype as a continuous benchmark ("quilt") for future LLM testing, planning iterative improvements like better graphics, animations, and voice interaction.
Exploring "Block Walkers": Building a Client-Side AI Game with LLMs
0:01 Project Goal: The creator aims to build "Block Walkers," a web-based open-world game where players can converse with NPCs driven by a tiny Quen AI model, entirely client-side using WebGPU and Transformers.js.
1:01 Initial Failure: The first attempt, using Kimmy K2 with the Rode agentic coding framework, took 12 hours and resulted in an excessively complex, non-functional game engine with numerous unnecessary scripts, highlighting the model's slowness.
2:05 Moonshot AI Setup & Cost: The creator details the setup for using Kimmy K2 via Moonshot AI's official API, noting a $3.50 cost for the extensive code generation, and advises checking "enable streaming" in Rode settings for proper functionality.
3:40 Game Concept & Constraints: The simplified prompt for "Block Walkers" specifies an open-world city block with low-poly visuals, NPC collisions initiating AI chat, and explicitly warns against over-complication, test scripts, or large abstracted codebases.
5:07 Model Limitations: The 128K context length of Kimmy K2 is noted as a potential limitation for complex agentic coding tasks compared to models with larger contexts like Gemini.
7:00 Second Attempt's Outcome: The second attempt ran for approximately 6 hours. While it initially tried to consolidate all code into a single HTML file, it shifted to modules after prompting. The primary challenge remained Kimmy K2's extreme slowness, making debugging and iteration very taxing. The game still did not run perfectly on its own.
8:09 Gemini's Role: Google Gemini 2.5 Pro was brought in to "tidy things up" and fix issues, ultimately enabling the partially functional "Block Walkers" prototype.
9:00 Working Prototype Showcase: The video showcases a basic, block-like game where players move around and interact with simple NPC rectangles by pressing 'E', triggering slow but functional client-side AI conversations via a tiny Quen model.
10:01 Future Enhancements: Planned improvements include replacing the tiny Quen model with a larger, quantized version (josified quen 300.6b), integrating voice interaction (e.g., via Whisper), and enhancing graphics, animations, and gameplay logic.
11:36 "Quilt" Benchmark: The "Block Walkers" prototype will serve as a foundational "quilt" for future agentic coding and code model testing, allowing new models to iteratively add features and improvements.
13:05 Client-Side Advantage: The creator emphasizes the benefit of WebGPU and Transformers.js allowing the game to run entirely client-side, making it more accessible without requiring users to download additional software like Ollama.
13:27 GitHub Availability: The project's code will be made available on GitHub.