This lecture details practical methods for enabling Large Language Models (LLMs) to interact with the external world, addressing their inherent limitations like knowledge cutoffs and an inability to perform actions. The discussion is structured in three main parts.
First, Retrieval-Augmented Generation (RAG) is presented as a solution for providing LLMs with up-to-date, unstructured information. This involves a multi-stage process of retrieving relevant document chunks from a knowledge base using techniques like semantic search and BM25, reranking them for precision, and then augmenting the user's prompt with this information before generation.
Second, the concept of Tool Calling is introduced, which allows LLMs to use predefined functions or APIs to interact with structured data and external systems. The lecture outlines the mechanism—where the LLM predicts the correct tool and its arguments for external execution—and discusses training methodologies, including Supervised Fine-Tuning (SFT) and advanced prompting. It also addresses scaling challenges through tool selection (routing) and standardization efforts like the Model Context Protocol (MCP).
Finally, the lecture explores Agentic Systems, where an LLM autonomously pursues goals through iterative loops of reasoning and action. The "React" (Reason + Act) framework is explained as a model for these loops, which consist of observing the environment, planning a step, and acting via a tool. The discussion touches upon multi-agent communication protocols and concludes by emphasizing the critical importance of safety measures to mitigate the risks associated with granting LLMs action-taking capabilities.
Lecture Summary: Integrating LLMs with External Systems
0:06:46 The Knowledge Cutoff Problem: LLMs are limited by the static data they were trained on. Simply adding all new information to the context is impractical due to context length limits, performance degradation (the "needle in a haystack" problem), and cost.
0:15:13 Retrieval-Augmented Generation (RAG): RAG is a technique to provide LLMs with current, relevant information. It works in three steps: 1) Retrieve relevant documents, 2) Augment the prompt with them, and 3) Generate a response.
0:24:27 Two-Stage Retrieval: The critical retrieval step is typically done in two stages. First, a fast "Candidate Retrieval" uses methods like semantic similarity search and keyword-based BM25 to find a broad set of potentially relevant documents. Second, a more precise "Reranking" stage uses a computationally heavier cross-encoder model to sort these candidates for maximum relevance.
0:59:54 Tool Calling for Actions and Structured Data: Tool calling allows LLMs to go beyond text generation and interact with external systems via APIs or functions. This is useful for tasks involving structured data, calculations, or performing real-world actions like sending an email.
1:08:16 Tool Calling Mechanism: The LLM is given the definitions of available tools. When a user makes a request, the LLM predicts which tool to use and what arguments to pass to it. This tool is then executed outside the LLM, and its structured output is returned to the model to synthesize a final, human-readable answer.
1:11:15 Training Models for Tool Use: An LLM can be taught to use tools either through Supervised Fine-Tuning (SFT) with specific examples or, for more advanced models, by providing a detailed in-context explanation of how the tools work.
1:26:22 Scaling with Tool Selection: To handle a large number of available tools without exceeding context limits, a preliminary "tool selection" or "routing" model can be used. This router first identifies a small subset of relevant tools for a given query, which are then passed to the main LLM.
1:32:05 Agentic Systems: Agents are an extension of tool use where an LLM autonomously pursues a goal through multiple, iterative loops of reasoning and action.
1:33:48 The React Framework: "React" (Reason + Act) is a common framework for agents. It involves a continuous loop of Observe (assessing the current state), Plan (deciding the next action), and Act (using a tool to execute the plan) until the goal is achieved.
1:42:19 Safety is a Critical Concern: Granting LLMs the ability to take actions introduces significant safety risks, including data exfiltration and executing harmful commands. Mitigating these risks requires robust safety measures during both training (e.g., harmlessness data) and at inference time (e.g., safety classifiers).
1:47:34 Advice for Building with Agents: Start with a simple, well-defined task. Use the most capable model available to test the limits of what's possible before optimizing. The ability to debug the model's chain of thought is crucial for identifying and fixing failures.