Andrej Karpathy's keynote introduces "Software 3.0," a new paradigm where Large Language Models (LLMs) are programmed directly using natural language, primarily English. He likens LLMs to a new kind of computer, possessing characteristics of utilities (centralized, metered access), fabs (high capex, deep tech), and early operating systems (1960s-era time-sharing). Despite their superhuman capabilities, LLMs exhibit cognitive deficits like hallucination and context limitations, necessitating a new approach to software development. Karpathy advocates for "partial autonomy apps" with "autonomy sliders" and emphasizes the importance of fast generation-verification loops, often via custom GUIs. The discussion highlights opportunities like "vibe coding" (making software accessible to non-programmers) and "building for agents" by designing interfaces and documentation (e.g., llms.txt) specifically for LLM consumption. The talk sparked debate on the clarity of "Software 3.0" as a concept, the deterministic nature of LLM outputs, the slow progress of full autonomy in areas like self-driving, and the challenges of integrating LLMs into user interfaces.
Summarizing "Software Is Changing (Again)" by Andrej Karpathy and the Discussion
0:00 Software Evolution (1.0, 2.0, 3.0): Andrej Karpathy introduces "Software 3.0," a fundamental shift where Large Language Models (LLMs) are programmable directly in natural language (English), acting as a new kind of computer. This follows Software 1.0 (human-written code) and Software 2.0 (neural networks programmed by data and optimization).
4:34 LLMs Eating the Stack: Similar to how neural networks (Software 2.0) replaced C++ code in Tesla's autopilot, LLMs (Software 3.0) are now eating through existing software stacks, requiring fluency in all three paradigms.
Utilities: LLMs are like electricity, requiring significant capital expenditure (capex) by labs (e.g., OpenAI, Anthropic) to train, and operational expenditure (opex) to serve via metered APIs. Their outages can cause "intelligence brownouts."
Fabs: LLMs involve substantial investment and proprietary "tech trees," akin to semiconductor fabrication plants.
Operating Systems: LLMs are increasingly complex software ecosystems, comparable to Windows/macOS (closed-source) and Linux (open-source alternatives like Llama). They function like a CPU, orchestrating memory (context windows) and compute for problem-solving.
11:04 1960s Computing Era: LLM compute is currently expensive and centralized in the cloud, leading to a "time-sharing" model reminiscent of 1960s mainframe computing. The "personal computing revolution" for LLMs has yet to occur.
12:48 Flipped Technology Diffusion: Unlike traditional transformative technologies (electricity, GPS) that start with governments/corporations and then diffuse to consumers, LLMs diffused instantly to billions of individuals, starting with consumer uses.
14:39 LLM Psychology ("People Spirits"): LLMs are stochastic simulations of people, trained on human data, exhibiting "emergent psychology." They possess superhuman encyclopedic knowledge (like "Rain Man") but also cognitive deficits: hallucinations, "jagged intelligence" (superhuman in some areas, fallible in others), "entrograde amnesia" (context window limitations, not native learning over time), and gullibility/security risks.
18:16 Opportunities: Partial Autonomy Apps: Software is moving towards partial autonomy. Examples include Cursor (for coding) and Perplexity (for search/research). Key features of these LLM apps are context management, multi-LLM orchestration, application-specific GUIs for human auditing, and an "autonomy slider" to control the level of AI involvement.
22:09 Fast Generation-Verification Loop: To work productively with fallible AIs, developers need to speed up the generation-verification loop. GUIs are crucial for visual auditing, making it easier for humans to review and accept/reject AI output. Keeping the AI "on a leash" by using small, concrete prompts is essential.
29:05 Software Accessibility ("Vibe Coding"): Since LLMs are programmed in English, software becomes highly accessible. "Vibe coding" allows non-programmers to build basic apps (e.g., iOS app, MenuGen) rapidly, without deep technical knowledge.
33:36 Building for Agents: While "vibe coding" makes application creation easy, deployment (DevOps, authentication, payments) remains manual and tedious. Karpathy argues for building software interfaces and documentation specifically for AI agents, as they are a new class of digital information consumers alongside humans and other programs.
34:00 Agent-Friendly Infrastructure: Ideas include llms.txt (similar to robots.txt) to describe website purpose to LLMs, moving documentation to LLM-readable formats like Markdown, and replacing "click" instructions with curl commands. Tools like get_ingest and deep wiki help convert human-oriented content into LLM-friendly formats.
39:06 Iron Man Suit Analogy: The future of software will see an increasing shift along an "autonomy slider," building more "Iron Man suits" (augmentations that can also act as agents) rather than fully autonomous "Iron Man robots," emphasizing the need for human oversight and collaboration.
Discussion Highlights:
Critiques of Versioning: Commenters (e.g., politelemon, gchamonlive) debated if "Software 3.0" implies replacement or improvement, with Karpathy clarifying it denotes different categories of software. The analogies were seen by some as cherry-picked or marketing-driven.
Structured Outputs:miki123211 and abdullin highlighted structured outputs (JSON mode) as a "criminally underused" tool that solves LLM unpredictability, boosts performance, and enables auditable "chains of thought."
Formal Verification and Ambiguity:practal proposed using formal verification to keep AI "on a leash," but noted the challenge of translating informal English into unambiguous formal specifications.
GUI for LLMs: The idea of a customized, non-linear GUI generated by an LLM sparked excitement (hgl) but also concerns about non-determinism, unlearnability, and a "crappier UX" (superfrank, sensanaty, jonny_eh).
Self-Driving Autonomy: The analogy to self-driving technology led to discussions on the slow, incremental nature of achieving full autonomy (AlotOfReading) and debate on whether driving is a "general problem" for AI (tshaddox vs. shakna).
"Vibe Coding" Concerns: Commenters raised concerns about the security implications of LLM-generated code and the potential for building on "potentially stolen code, books, and copyrighted training data" (belter).
Death of the Web: The concept of llms.txt led to a broader discussion on the decline of human-readable websites due to confusing design and SEO, advocating for clean, accessible formats for LLMs (nothrabannosir, alightsoul).
Importance of "Under the Hood":dmitrijbelikov emphasized that while Software 3.0 offers convenience, understanding the underlying layers remains crucial for developers, as it is a tool for acceleration, not a replacement for foundational knowledge.