Okay, here is the abstract and bulleted summary for the provided PyData talk transcript.

Abstract:

This PyData Global tutorial by Nathan Colbert, Senior Manager of ML Architecture at Peacock, guides participants through building the foundational components of a core Machine Learning Platform (MLP) from scratch. Starting from the requirement for personalized user predictions (inference), the session demonstrates a Test-Driven Development (TDD) approach, working backwards to motivate the necessary components. Key concepts from Domain-Driven Design (DDD) are introduced and applied, including Ubiquitous Language, Bounded Contexts, and Aggregates. The tutorial focuses on implementing core elements like an inference engine, model registry, and feature store, emphasizing decoupling through the repository pattern. The session culminates in demonstrating how these services can communicate via an event-driven message bus to handle scenarios like missing models or features dynamically, highlighting the benefits for building scalable, testable, and maintainable ML infrastructure.

Building a Core ML Platform from Scratch: From Inference to Features

• 0:42 Introduction: Nathan Colbert, Senior Manager of ML Architecture at Peacock, introduces the tutorial on building a core ML platform from scratch, noting the title change to "From Inference to Features" to align with the Test-Driven Development (TDD) approach (1:38). His background includes music, analysis, data pipelines, feature stores, model deployment, and current work on a global ML platform (2:03). He highlights learning from past mistakes in architecture (2:52).
• 3:06 What is an ML Platform?: A monolithic but modular core MLP will be built. An ML platform supports the data science workflow (predictions needing features/models, training). Core components typically include an Inference Engine, Feature Store, Model Registry, Orchestrator (Message Bus), and potentially a Model Trainer (3:12). The goal is to decouple these components (4:28).
• 4:52 Focus and Scope: The talk aims to build intuition for creating scalable and testable platforms (4:52). It will not directly cover infrastructure, CI/CD, cost optimization, or operational models, though good architecture enables these (5:06).
• 8:34 Design Principles: The tutorial utilizes Test-Driven Development (TDD) and Domain-Driven Design (DDD) (8:34). Key DDD concepts covered are Ubiquitous Language (code names match business terms), Bounded Contexts (clear system boundaries), and Aggregates (grouping related objects, e.g., use case, user, features, model) (9:03).
• 12:53 Setting Up: The project structure is created with source (src) and test (tests) directories. Dependencies (pytest, pre-commit) are installed using UV (13:19). Pre-commit hooks are set up to automate testing and formatting before commits (14:27).
• 16:19 Building the Inference Engine (TDD): Development starts with a test for the simplest prediction: a constant value for a given user and use case. Shared types (UserID, UseCase) are defined (19:07). The first test fails, motivating the creation of a basic get_prediction function (23:45).
• 25:35 Expanding Inference: A second test is added to require different predictions for different use cases. This fails, showing the need for the get_prediction function to differentiate based on use case.
• 27:10 Introducing Models and Repositories: The InferenceEngine is refactored into a class (27:10). The concept of a Model (with a predict method) and a ConstantModel (implementing the Model interface) is introduced (28:17). To allow the Inference Engine to access models without knowing how they are stored, the ModelRepository pattern (an abstraction layer for data access) is implemented with an abstract base class (ModelRepository) and a concrete InMemoryModelRepository implementation (34:54).
• 41:39 Inference Engine Uses Model Repository: The InferenceEngine is updated to depend on (accept via composition) a ModelRepository instance, retrieving the correct model based on the use case (41:39). Tests are updated to inject an InMemoryModelRepository containing ConstantModel instances, and they pass (45:45).
• 51:00 Incorporating Features: The next step is to make predictions for different users, requiring features. Shared types for features are defined (Feature, FeatureVector, FeatureSet mapping use case -> user -> features) (52:56). A RulesBasedModel (inheriting from Model) is introduced, which requires features for its predict method (55:08).
• 57:25 Feature Repository: Similar to models, the Inference Engine needs to get features without knowing the storage mechanism. A FeatureRepository pattern is introduced (57:25) with an abstract base class and an InMemoryFeatureRepository implementation (58:12), including error handling for missing use cases or users.
• 1:01:11 Inference Engine Uses Feature Repository: The InferenceEngine is updated to accept a FeatureRepository via composition (1:01:11). The get_prediction method now retrieves features using the use case and user ID from the repository and passes them to the model (1:02:08).
• 1:02:19 End-to-End (E2E) Testing: An E2E test is created to simulate the system's behavior. It sets up InMemoryFeatureRepository and InMemoryModelRepository (containing a RulesBasedModel) and creates the InferenceEngine with these dependencies (1:02:19). The test verifies that providing specific features for a user via the feature repository results in the expected prediction from the rules-based model injected via the model repository (1:04:40).
• 1:05:08 Benefits of Decoupling and Testing: The effort invested in defining interfaces (repositories) and decoupling components provides significant control for testing (1:06:09). It makes the system testable in isolation and allows swapping infrastructure dependencies (GCS, Redis, etc.) by simply implementing the defined repository interfaces (1:07:15). Testing repositories separately is easier than coupling application logic to infrastructure (1:08:09).
• 1:08:58 System-Level Orchestration (Message Bus): To handle communication between decoupled services (like the Inference Engine needing features from the Feature Store), Event-Driven Design using a Message Bus is introduced (1:11:35). Components communicate via Commands (requests to do something), Events (what happened), and Errors (what went wrong) (1:15:51).
• 1:13:55 Message Handling: When the Inference Engine needs something (e.g., a model not found), it raises a domain exception (1:13:55). A handler or translator consumes this exception/error and translates it into the next necessary Command (1:17:24). Example: ModelNotFound error -> "Publish Model for Inference" command -> Model Registry handles command -> Model Registry emits "New Model Ready for Inference" event -> Inference Engine handles event -> Inference Engine adds model to its repository.
• 1:19:32 Message Bus Orchestration: The MessageBus manages a queue of commands, dispatching them to appropriate handlers/services (1:19:32). It can prioritize error handling commands to resolve issues before retrying the original command (1:20:33).
• 1:21:19 E2E Test with Message Bus: A complex E2E test demonstrates the message bus flow. It creates services (FeatureStore, ModelRegistry, InferenceEngine initially without features/models). It defines handlers and a translator. It dispatches a GetPrediction command via the message bus (1:21:19). The bus orchestrates the recovery (requesting model/features, adding them to the inference engine's repositories) allowing the original command to eventually succeed and return the prediction (1:22:12, 1:25:08).
• 1:26:10 Challenge Questions: Topics not covered but enabled by this architecture include model training, hyperparameter tuning, model comparison, handling different hardware/runtimes (e.g., TF Serving), scaling to distributed systems (service mesh), and integrating with external message queues (Kafka, Pub/Sub) (1:26:18).
• 1:27:41 Key Takeaway: The hardest part of building an ML platform is testing (1:27:41). Strategies like dependency injection, dependency inversion, and defining reliable interfaces (ports and adapters/repositories) using DDD and Event-Driven Design provide the necessary tools (1:27:53).
• 1:28:16 Conclusion: Nathan concludes, reiterating the importance of testability and providing contact information.