This presentation details Apple's use of gRPC Swift within their open-source containerization framework, which enables applications to run Linux containers as lightweight virtual machines (VMs) for enhanced security and privacy. The core mechanism involves a gRPC server, dubbed the VM init daemon (VMD), running inside each container's VM. This VMD acts as an agent, orchestrating low-level runtime configurations—such as process startup, filesystem mounts (including for Rosetta x86 emulation on Apple Silicon), and network management—by receiving RPC calls from a client on the macOS host.
The Virtualization Framework facilitates this cross-boundary communication by exposing a Virtio socket (VSOC) device. gRPC Swift proved highly effective for this system-level application, offering crucial benefits like high performance for rapid container startup, stable tooling, robust versioning with Protocol Buffers, and seamless integration with Swift's async/await concurrency model. The decision to unify the stack on Swift, enabled by gRPC Swift's capabilities and Swift's cross-platform nature, significantly improved the developer experience.
Summarizing the Containerization Framework and gRPC Swift
0:00:01 Introduction: Augam, a software engineer at Apple on the Swift server team, discusses the use of gRPC Swift in Apple's open-source containerization framework.
0:00:23 gRPC Server for Runtime Orchestration: When a container is started (e.g., container run hello world), a gRPC server is spun up to orchestrate runtime configurations.
0:01:04 Containerization Project Overview: Released at WWDC and open-source on GitHub, this project allows applications to use Linux containers. Uniquely, each container runs as a lightweight virtual machine (VM) for improved security and privacy.
0:02:07 What is a Container? A single unit that packages an application (e.g., a Linux server app) and its dependencies into an image, enabling it to be run and network-addressed.
0:03:35 Leveraging Virtualization: The containerization framework uses Apple's Virtualization Framework to execute each Linux container within its own lightweight VM.
0:04:18 The Container Agent (VM init daemon): A process called VM init (VMD) runs inside the VM, hosting the gRPC server. Its role is to manage and configure the runtime environment for the container.
0:04:47 System Architecture: A client (e.g., the container CLI on macOS) communicates with the VMD (running in the VM) to send startup requirements, crossing the virtualized host boundary.
0:07:58 Cross-Host Communication with VSOC: The Virtualization Framework enables communication between the host and guest VMs using a Virtio socket device (VSOC), which provides port-based connections for gRPC RPC calls.
0:09:14 VMD Functions: The VMD performs low-level operations, orchestrated via gRPC API calls, to configure the VM to behave as a container. This includes:
0:10:03 Mount/Unmount Calls: Used to mount filesystems, notably for enabling Rosetta emulation to run x86 workloads on Apple Silicon within Linux containers.
0:10:37 Networking Layer: Manages IP addresses, brings interfaces up/down, and configures specific network routes.
0:10:53 Process Supervision: Starts user-specified processes (e.g., Postgress) and ensures child processes are properly reaped upon container shutdown.
0:11:29 gRPC Swift for Success: The open-source gRPC Swift library is highlighted as central to the project's success.
Performance: Delivers high performance crucial for rapid container startup.
Stable Tooling: Provides reliable and familiar gRPC tooling within the Swift ecosystem.
Cross-Platform Capability: Works effectively in data centers and on local machines, especially for crossing virtualization boundaries.
Versioning & Iteration: Protocol Buffers allow for flexible type evolution (v1 to v3) by adding new fields and utilizing optional fields.
0:13:39 Code Generation & Concurrency: gRPC Swift automatically generates client and server code. It also seamlessly integrates with Swift's async/await concurrency features, simplifying asynchronous programming.
0:16:30 Swift for Systems Programming: Swift proved to be a great language for container runtime, offering expressiveness, safety, speed, excellent C interop, and cross-platform capabilities (macOS and Linux). Unifying the stack on Swift improved the developer experience.
0:18:45 Key Takeaways: The containerization architecture with a gRPC-enabled agent effectively manages runtime configurations. The gRPC communication channel over VSOC is a successful solution for host-guest interaction across virtualized boundaries. gRPC Swift works excellently with Swift, supporting versioning, code generation, and Swift's async features for robust process management.
0:19:33 Next Steps: Users are encouraged to try out containers, watch related talks, install the CLI, write their own client, contribute to the open-source repo, and explore further resources like George's talk on gRPC Swift.