This presentation provides a comprehensive overview of application-level (L7) load balancing within gRPC. Ishwar, a software engineer at Google, explains how gRPC distributes client traffic across server backends to ensure high availability, scalability, and performance. The talk differentiates between server-side and client-side load balancing, highlighting gRPC's native support for a robust client-side approach. It then delves into the gRPC client architecture, detailing the roles of the Load Balancing Policy (LB policy), name resolvers, service configuration, and sub channels. Key aspects covered include sub channel connectivity states, state aggregation algorithms, and a thorough explanation of the gRPC LB policy API (Builder, Balancer, ClientCon, and Picker interfaces), along with their methods and return types.
Load Balancing in gRPC: Architecture and API Deep Dive
0:02 Introduction to gRPC Load Balancing: Ishwar from Google introduces load balancing in gRPC, focusing on distributing client traffic across multiple server backends (L7 load balancing) to ensure high availability, scalability, and performance.
Server-side: Clients connect to a proxy that distributes traffic. Simple for clients but the proxy can become a performance/scalability bottleneck.
Client-side: Clients connect directly to backends, improving performance. gRPC handles the complex load balancing logic in clients across multiple languages, eliminating the need for application-specific implementations.
3:15 Load Balancing Policy (LB Policy) Components: The core gRPC component for load balancing comprises:
Connection Management: Receives backend addresses from the name resolver and manages connections.
Call Management (Picker): Invoked per-request to select an appropriate backend for an RPC.
4:01 Plugin Architecture for LB Policies: gRPC supports a plugin architecture, allowing users to implement and register custom LB policies. These policies are selected at runtime based on the "service config" provided by the name resolver.
Pick First: Connects to the first reachable backend and directs all traffic there. Useful for clients behind a reverse proxy or those requiring "soft affinity" (e.g., for caching).
Round Robin: Connects to all backends in parallel and distributes RPCs across them. Generally recommended for most other use cases.
6:38 gRPC Client Channel Architecture: A gRPC channel is a virtual client-to-server connection that transparently manages name resolution, connection, and load balancing details for the application.
7:23 Name Resolver and Service Configuration: When a channel becomes active, a name resolver provides backend addresses and a "service config." This config specifies channel behaviors like RPC timeouts, retry settings, and crucially, the LB policy name and its configuration.
8:15 Sub Channels and Connectivity States: The LB policy creates "sub channels" (actual connections to backend addresses) and monitors their lifecycle. Sub channels can be in states like Idle, Connecting, Ready, Transient Failure (connection failed), or Shutdown (closed).
11:51 State Aggregation: LB policies aggregate the states of multiple sub channels into a single overall connectivity state (e.g., Ready, Connecting, Idle, Transient Failure) for the gRPC channel.
13:00 LB Policy API (Experimental Status): The API for custom LB policies is currently "experimental" in Java and Go. This designation indicates that while the implementations are mature and widely used, the underlying transport might evolve, potentially affecting API signatures. Users are encouraged to provide feedback for stabilization.
Builder: Used to create new LB policy instances and register the policy's name.
Balancer: Defines the core load balancing logic, processing updates from the name resolver (e.g., updateClientConState for new addresses/configs) without simultaneous method invocations.
ClientCon: An interface implemented by the gRPC client, used by the LB policy to create new sub channels (newSubCon) and report overall channel state and picker to the channel (updateState).
Picker: Invoked for every RPC to quickly select a sub channel. It can return Complete (sub channel found), Queue (RPC queued, waiting for active sub channel), Fail (RPC fails, possibly retried), or Drop (RPC fails, no retries).
18:08 Client Connection Scenarios: The speaker clarifies that using "pick first" with randomized shuffling can help distribute individual clients across different servers, even if each client initially connects to a single backend.