Abstract:

This video, "Emacs at Lunch," delves into the creation and strategic use of "tools" for Large Language Models (LLMs) within an Emacs environment. It defines a tool as a pairing of an executable function and an interface declaration, enabling the LLM to intelligently decide when, which, and how to invoke functions, and how to interpret their results, especially in complex, multi-step processes. The discussion covers practical tool development, including defining arguments with JSON type mapping for Elisp, structuring inputs, and managing output (which is predominantly coerced to strings). Key tool configuration options like async for non-blocking operations, confirm for sensitive actions, and crucially, include-results for maintaining conversational context and enabling retrieval-augmented generation are explained. A significant emphasis is placed on the description field, which functions as a miniature system prompt, guiding the LLM's internal logic and defining the interconnections and sequence of tool usage for effective, goal-oriented problem-solving.

Building LLM Tools in Emacs: A Comprehensive Guide

• 0:00:01 - Advanced LLM Capabilities: The video opens by highlighting new LLM features in Emacs, including recursive transitive lookups across documentation and source code, enabling fact-based code queries, synthesis, summarization, and translation.
• 0:00:30 - Defining an LLM Tool: A "tool" is a combination of an Emacs Lisp function and an interface declaration that instructs the LLM on how and when to call the function effectively.
• 0:00:45 - When to Employ Tools: Tools are most useful when an LLM needs to act as a heuristic to decide: (1) when to call a specific function, (2) which function to call (dynamic dispatch), (3) the appropriate input arguments for a function, or (4) which part of a function's output represents the desired result. These applications can be compounded for multi-step processes.
• 0:01:31 - When Tools are Unnecessary: Tools are not required for simple, single-step transformations where the AI consistently performs a predefined action and the output doesn't need structural adaptation.
• 0:01:53 - Core Tool Declaration Components: A basic tool declaration comprises the Emacs Lisp function itself, a JavaScript-style name for system reference, defined arguments (including their type and a description for the LLM), and a comprehensive description that acts as a mini-system prompt for the LLM.
• 0:02:20 - Argument Handling and JSON Mapping: Tool arguments are positional and are deserialized from JSON. Key JSON types are mapped to Elisp equivalents (e.g., null to nil, false to json-false, array to vector). Structured input requires properties and required keys, arrays use an items type, and choices are handled with an enum string type. intern-soft is recommended for symbols to prevent unintended symbol creation from LLM hallucinations.
• 0:04:04 - Managing Tool Output: The GPTA backend typically converts all tool output to strings. The best practice is to return literal JSON answers, ensuring Elisp types are converted to JSON-compatible formats (e.g., vconcat for lists), or to use print-one-to-string for meaningful Elisp structures. Error messages should be clear to facilitate LLM recovery.
• 0:05:32 - Tool Integration: Tools are created using gpel-make-tool and added to the gpel-tools list, making them active and discoverable within the gpel-chat interface and menu.
• 0:05:59 - Asynchronous Tools for UI Responsiveness: For operations that might block the Emacs UI, async tools are used. They accept a callback function as their first argument to return results once the slow operation is complete.
• 0:06:14 - confirm Option for Safety: The confirm option is available for tools that perform potentially dangerous actions (e.g., file deletion, data leakage), prompting the user for confirmation before execution.
• 0:06:31 - include-results (Essential for Context): This critical option writes the tool's output into the message log, allowing the LLM to refer back to previous results. This is vital for building context in retrieval-augmented generation and for enabling step-by-step problem-solving.
• 0:07:26 - The Pivotal Role of Tool Descriptions: The tool's description is paramount for the LLM (not humans), serving as a "miniature system prompt" or "program." It defines how tools interconnect and how their outputs should be used in subsequent steps, often containing more strategic logic than the function being called.
• 0:08:18 - Guiding LLM Progress: The presenter emphasizes that the interplay between tool descriptions, the main system prompt, and the data introduced via included tool calls is crucial for consistently directing the LLM towards the desired goal.
• 0:08:43 - Future Content and Community Support: The video concludes by announcing upcoming content on gpel configuration and encourages viewers to subscribe, like, share, and support the project via GitHub sponsorship to foster continued Emacs AI integration and Prize Forge development.