This presentation explores the integration of gRPC with the emerging Model Context Protocol (MCP) for connecting Large Language Models (LLMs) to various resources and APIs. While MCP is gaining adoption, the speaker critiques its current design for being ad-hoc, lacking robust versioning, security, and relying on less efficient transports like standard I/O (fast CGI) and basic HTTP streaming.
The core argument advocates for leveraging gRPC's proven scalability, performance, robustness, multi-language support, built-in streaming capabilities, service discovery, and security features to overcome MCP's limitations. The speaker outlines three deployment models: a local binary acting as a gRPC client, a gRPC to MCP gateway, and an Envoy plugin. A key point is gRPC's serialization agnosticism, allowing it to transport JSON payloads. While acknowledging challenges with MCP's stateful nature and evolving specification, the presentation highlights ongoing efforts, including a prototype implementation and Google's commitment to standardizing gRPC as a first-class transport for MCP, aiming for a more stateless and pluggable transport abstraction.
Summarizing the Integration of gRPC with Model Context Protocol (MCP)
0:00 Introduction to MCP and LLMs: The presentation focuses on using gRPC to connect Large Language Models (LLMs) with various resources and APIs via the Model Context Protocol (MCP).
1:01 Model Context Protocol (MCP) Overview: MCP is a protocol enabling chat applications or IDEs (referred to as MCP hosts) to interact with LLMs and an MCP server, which provides resources.
2:23 MCP Transport Protocols and Design Critique: Current MCP implementations primarily use standard I/O (likened to fast CGI) and HTTP with streaming for client-to-server notifications. The speaker criticizes MCP's ad-hoc design, suggesting it repeats past networking mistakes and lacks a principal-based approach, despite its growing adoption.
Resources: Identifiable data (e.g., calendar entries, files, database records) with IDs/URLs, fetched by the MCP server, and fed as context to the LLM.
Prompts: Application commands (like Slack slash commands) provided by the user, translated by the MCP server into strings for the LLM.
Tools: APIs invoked by the LLM for actions, described via JSON responses (similar to gRPC reflection), enabling the host app to perform operations.
6:29 Why gRPC is Superior for MCP: gRPC offers a trusted, proven, and scalable infrastructure with benefits like:
Performance & Robustness: Based on HTTP/2, with health checks, deadline propagation, and circuit breaking.
Streaming RPCs: A core feature that aligns well with MCP's need for server-to-client notifications.
Language Agnostic: SDKs available for many languages, avoiding the need for ad-hoc HTTP servers.
Advanced Features: Service discovery (XDS), load balancing, versioning, robust authentication, security, and comprehensive observability (stats, tracing).
9:00 Deployment Models for gRPC-enabled MCP: Three models are proposed:
Local Binary: An MCP server running locally, communicating via gRPC to a remote server.
gRPC to MCP Gateway: Similar to existing gRPC gateway transcoding solutions.
Envoy Plugin: An Envoy proxy plugin for mapping MCP requests to gRPC services.
9:41 gRPC's Serialization Agnosticism: gRPC can effectively transport JSON objects, retaining most of its benefits even without protobuf.
10:08 Transcoding Considerations: While prompts and resources have fixed APIs suitable for transcoding to protobuf, tool inputs/outputs might best remain JSON for now due to the evolving MCP spec, offering flexibility.
11:13 MCP's Stateful Nature and Bidirectional Streaming: Unlike REST APIs, MCP is fundamentally stateful and relies on bidirectional streaming, making gateway and backend service design more complex (requiring state management).
12:10 Prototype and Future Standardization Efforts: A prototype for the gateway model exists. There are active discussions within the MCP transport workgroup to standardize gRPC as a first-class MCP transport, aiming for a more stateless protocol design, pluggable transports, and improved authentication handling. Google is committed to this effort.
16:43 Envoy Plugin Status: The Envoy plugin model is currently conceptual, with implementation yet to begin, but it is considered a promising deployment model fitting the gRPC ecosystem.
18:39 Service Discovery in MCP: Service discovery (e.g., XDS) primarily operates at the gRPC client level, which could be within the local MCP binary or the edge gateway.
21:43 Project Status: The discussed gRPC-MCP integration work is an "intent to build," not a currently downloadable release. A proof-of-concept Python SDK with gRPC support is anticipated for October/November.