Architecture of a Real-World Machine Learning System

Clive Lancaster

Clive Lancaster

— 1 min read
Architecture of a Real-World Machine Learning System

Machine learning (ML) systems are complex, involving multiple interconnected components that work together to create, deploy, and maintain ML models. This blog post explores the key components of a real-world ML system.

Core Components

  1. Client: The application used by end-users, which requests predictions from the ML system.
  2. Orchestrator: A program responsible for evaluating models, typically run on a schedule or as part of a CI/CD pipeline.
  3. Model Builder: Creates and optimizes ML models using training data.
  4. Model Server: Makes trained models available via an API.
  5. Front-end: Implements domain-specific logic and interfaces with the client.
  6. Performance Monitor: Tracks and visualizes model performance on production data.
  1. Ground-truth Collector: Continuously acquires new data for the machine to learn from, including user feedback.
  2. Data Labeller: Facilitates manual labelling of input data when necessary.
  3. Featurizer: Computes and prepares features for model input, either in batches or real-time.
  4. Evaluator: Assesses model performance using predefined metrics and test datasets.

Workflow and Orchestration

The orchestrator manages the ML pipeline, which typically includes:

  1. Data extraction, transformation, and loading (ETL)
  2. Data splitting into training, validation, and test sets
  3. Feature engineering and preparation
  4. Model training and optimization
  5. Model evaluation and deployment decisions

Implementation Considerations

  • Containerization: Using Docker containers and orchestration tools like Kubernetes for managing ML workflows.
  • Cloud Platforms: Leveraging services like Google AI Platform, Databricks, or Amazon EKS for scalable ML infrastructure.
  • Workflow Management: Utilizing tools such as Apache Airflow for coordinating ML tasks.

Best Practices

  1. Implement the evaluator early, before building complex ML models.
  2. Start with simple baseline models to establish a performance benchmark.
  3. Monitor model performance continuously in production.
  4. Consider active learning techniques for efficient data labelling.

By understanding and implementing these components, organizations can build robust, scalable, and effective ML systems that deliver value in real-world applications.

Read more