In this AMS Josiah Willard Gibbs lecture, Yann LeCun, a pioneering figure in AI, argues that current approaches, particularly auto-regressive Large Language Models (LLMs), are fundamentally flawed and will not lead to human-level intelligence. He posits that these models are a "divergent process" incapable of robust reasoning and planning, and that they are trained on text, which contains far less information than the sensory data humans use to build an intuitive understanding of the physical world.
LeCun proposes a new paradigm called "objective-driven AI," centered on building systems that can learn "world models" through observation. The core of his proposal is the Joint Embedding Predictive Architecture (JEPA), a non-generative, energy-based model that learns by predicting abstract representations of the world, effectively filtering out unpredictable details. This approach, he argues, can enable machines to reason, plan complex actions hierarchically, and acquire common sense. He outlines the significant technical challenges, such as preventing "model collapse" during training, and concludes with a call for the research community to shift its focus away from generative AI and towards these new architectures to achieve the next breakthroughs.
Yann LeCun on the Obstacles to and a New Path for Human-Level AI
0:02:46 The Flawed Path to Human-Level AI: Yann LeCun asserts that while we have a chance to build human-level AI, current methods, especially Large Language Models (LLMs), are not the right path. He identifies several fundamental obstacles, including mathematical ones, that need to be overcome.
0:07:05 LLMs Are "Doomed": LeCun explains that LLMs are auto-regressive models that predict the next word in a sequence. He argues this is a "divergent process" where the probability of generating a long, factually correct answer decreases exponentially with length, making them inherently unreliable and prone to "hallucinations."
0:12:25 The Data Disparity: A core problem with LLMs is their reliance on text. LeCun argues that a human child absorbs enormously more information from sensory input (vision, touch, etc.) in a few years than an LLM does from all the text on the internet. This sensory data is crucial for learning common sense and intuitive physics.
0:18:42 A New Architecture: Objective-Driven AI: LeCun proposes a new cognitive architecture for "Advanced Machine Intelligence" (AMI). This paradigm shifts from simple input-output functions to "inference by optimization," where the system finds an optimal output that minimizes a given cost or "energy" function.
0:23:07 Learning with Energy-Based Models (EBMs): This architecture is formalized using EBMs, which learn a scalar energy function to capture the compatibility between inputs and potential outputs. This is more powerful than standard models as it can handle tasks with multiple correct solutions (e.g., translation, prediction).
0:28:53 The Unsolved Problem of Hierarchical Planning: LeCun highlights that a key capability missing in all current AI systems is the ability to learn hierarchical planning. Humans effortlessly break down complex goals (e.g., "travel to Paris") into a hierarchy of sub-goals, a feat no AI can learn to do on its own.
0:33:59 JEPA: A Better Way to Learn World Models: To learn how the world works, LeCun advocates abandoning generative models in favor of the Joint Embedding Predictive Architecture (JEPA). Instead of predicting high-dimensional data like video pixels, JEPA learns to predict abstract representations, allowing it to ignore unpredictable details and focus on the underlying structure of the world.
0:45:35 Practical Solutions for Training: A key challenge is preventing "model collapse" during training. LeCun prefers regularized methods over contrastive ones and introduces VICReg (Variance-Invariance-Covariance Regularization) as a practical technique to ensure the model learns rich, informative representations.
0:50:53 Proof of Concept: Planning with a World Model: The lecture shows a demonstration of a robot learning to manipulate objects. The system uses a learned world model to predict the outcome of its actions in an abstract space, enabling it to plan a sequence of moves to achieve a goal.
0:53:51 A Call to Action for Researchers: LeCun concludes with a strong recommendation for the AI community: abandon generative models, reinforcement learning, and an over-reliance on LLMs. He urges researchers, especially PhD students, to instead focus on the foundational problems of learning world models, hierarchical planning, and the mathematical underpinnings of energy-based learning.