This presentation delves into integrating Large Language Models (LLMs) with external APIs through the Multi-Agent Communication Protocol (MCP) and gRPC, specifically in the context of Redpanda Data's cloud services. The speaker, Johannes, highlights the inherent limitations of LLMs, which cannot perform actions like API calls or web searches, and explains how tool calling frameworks address this. While general function calling exists, MCP standardizes this process, offering benefits in schema definition, pluggable client-server architecture, and distribution. A core contribution is "protoc-gen-go-mcp," a protoc plugin designed to automatically generate MCP tool specifications from existing gRPC Protobuffer definitions, thereby leveraging the robust Protobuffer ecosystem and minimizing maintenance effort for Go-based applications. The talk details the multi-step MCP tool calling workflow, emphasizing the importance of high-quality APIs for LLM comprehension. It also addresses significant LLM compatibility challenges, such as varying JSON schema support across different models (e.g., OpenAI's limitations with oneOf, dynamic maps, and nesting levels), and outlines future work to enhance the plugin and explore direct gRPC tool calls.
Summarizing LLMs and gRPC Tool Calling with MCP
0:03 Introduction to Redpanda Data: Johannes from Redpanda Data, a cloud team member, introduces the company's Kafka-compatible streaming engine written in C++, offering multi-tenant SaaS and serverless solutions with APIs for engineers. The goal is to make these APIs accessible to Large Language Models (LLMs).
0:49 Agenda Overview: The presentation will deep dive into MCP (Multi-Agent Communication Protocol), explain MCP tool calling, discuss how gRPC can help, introduce protoc-gen-go-mcp, address LLM compatibility pitfalls, showcase a demo, and cover future work.
1:47 LLM Limitations and Tool Calling: LLMs only know what they've been trained on and their context window. They cannot search the web, call APIs, perform file system operations, or execute commands. "Tool calling" is the concept that enables LLMs to perform these actions by interacting with external functions/APIs.
2:31 General Function/Tool Calling: Tool calling is essentially defining a function (e.g., get_current_temperature) with a JSON schema description. An LLM, when prompted, decides whether to call such a function. If it does, the application executes the call and feeds the result back to the LLM.
3:18 Why MCP is Useful: MCP standardizes how applications provide context and allow LLMs to perform actions. Key features include:
Standardization: Open protocol for defining tools.
Schema Definition: Uses JSON schema for tool definitions.
Pluggable Client-Server: Enables packaging and distributing MCP servers, fostering marketplaces.
Standardized Transport: Supports standard IO (like LSP) and HTTP; gRPC transport is anticipated.
5:53 In-Depth MCP Tool Calling Workflow: A multi-step process involves:
A client (e.g., Cloud Code) requests available tools from an MCP server.
The MCP server lists tools with their definitions.
The client sends a user prompt (e.g., "weather in SF?") along with tool definitions to the LLM.
The LLM decides to perform a tool call (e.g., get_weather for "San Francisco").
The client executes this tool call on the MCP server.
The MCP server returns the tool's output (e.g., "temp 22, rainy").
The client sends this tool output back to the LLM.
The LLM generates a human-readable response based on all this information (e.g., "It's 22 degrees and rain... bring an umbrella."). This round-trip process can be slow.
Authoritative IDL: Keeps Protobuffers as the primary IDL for API definitions.
Leverage Ecosystem: Utilizes the existing, rich Protobuffer ecosystem (e.g., proto-validate, in-house tooling, RBAC) to avoid duplicated work.
High-Quality APIs: gRPC best practices (Google AIP, Buf resources) ensure high-quality, reliable APIs, which are crucial as LLMs get confused by poor APIs more easily than humans.
10:32protoc-gen-go-mcp Plugin: A protoc plugin for Go that generates MCP tool specifications directly from gRPC Protobuffer definitions, requiring minimal (or no) maintenance effort. It's open-source and integrates with the existing Protobuffer ecosystem, translating field_behavior into JSON schema.
11:16 Plugin Usage: The plugin integrates via buf generate YAML, creating a Go package with tool definitions for each RPC and register functions to wire up the MCP server with handlers (either in-process or for remote gRPC servers).
13:16 Demo of protoc-gen-go-mcp in Action: A video demonstrates an LLM using generated tools to interact with Redpanda Cloud APIs: listing resource groups/clusters, creating a serverless cluster, checking its readiness, and creating a Kafka topic. This highlights how API quality allows LLMs to "outsource thinking" for complex tasks.
Anthropic's Claude supports almost all JSON schema.
OpenAI lacks oneOf, anyOf, and requires all fields to be marked required (with workarounds for optional fields using type | null).
Dynamic maps and recursive schemas are also often unsupported or require workarounds (e.g., array of key-value pairs, falling back to string).
OpenAI has a maximum nesting level of six.
Key Takeaway: Rich APIs and detailed error messages are extremely important for LLMs, necessitating rigorous validation and best practices.
17:05 Future Work: Includes integrating proto-validate rules into JSON schema, adding custom options (e.g., skipping tools, custom names/descriptions, custom type mappings), enabling direct gRPC tool calls without MCP (for server-side AI agents), and improving interceptor support.
19:01 Q&A: Go Binding Necessity: The protoc-gen-go-mcp is Go-specific for type safety. A more general solution for other languages would involve a server-side application dynamically translating using reflection. Redpanda uses the binding by embedding an MCP server as a subcommand in their CLI, offering a local, low-risk proxy for gRPC calls.
21:03 Q&A: LLM Use of Output Schema: The speaker believes LLMs likely don't use the output JSON schema (a recent MCP addition) because LLMs primarily consume and produce text. The schema is more beneficial for human developers, not the LLM's internal processing.
22:43 Q&A: Tool Cost Optimization: To guide LLMs on tool cost, embed "this RPC is expensive" in the tool's description within the prompt (prompt engineering). The goal is to outsource business logic and cost-aware decision-making to the LLM, aligning with the promise of agentic AI.