Continuous Deployment Flow Structures & Pipeline Invocation

This guide covers how version control strategies like Gitflow and GitHub Flow act as the primary triggers for automated CI/CD pipelines, specifically within the AWS ecosystem for Machine Learning Engineering.

Learning Objectives

• Differentiate between Gitflow and GitHub Flow branching strategies.
• Understand how repository events (commits, merges, tags) invoke AWS CodePipeline.
• Configure pipeline triggers based on specific branch patterns.
• Map MLOps requirements to appropriate deployment flow structures.

Key Terms & Glossary

• Trunk-Based Development: A version control strategy where developers merge small, frequent updates to a core "main" branch.
• Webhook: An HTTP callback that triggers an action (like starting a pipeline) when a specific event occurs in a repository.
• Artifact: A deployable component (e.g., a Docker image or a serialized ML model file) produced by a build process.
• Feature Branch: A temporary branch used to develop a specific piece of functionality, isolated from the main codebase.

The "Big Idea"

In modern MLOps, the Git repository is the single source of truth. By applying structured flow patterns, we move away from manual deployments. Every code change undergoes automated testing and validation via pipelines, ensuring that only "known-good" models and infrastructure code reach production. The branching strategy you choose dictates the speed and safety of your delivery cycle.

Formula / Concept Box

Hierarchical Outline

• I. Branching Strategies
  • GitHub Flow: Simple, agile, focused on continuous delivery to production.
  • Gitflow: Robust, structured, utilizes long-lived branches for different environments (Dev, QA, Prod).
• II. Pipeline Invocation Mechanisms
  • Polling: AWS checks the repo periodically (deprecated/inefficient).
  • Webhooks: Real-time push notifications from GitHub/GitLab to AWS.
  • EventBridge: Centralized event bus for triggering pipelines from AWS native events.
• III. AWS CI/CD Services
  • AWS CodeBuild: Compiles code, runs unit tests, and packages models.
  • AWS CodeDeploy: Handles the logic of Blue/Green or Canary deployments.
  • AWS CodePipeline: The orchestrator that connects the repository to the deployment.

Visual Anchors

The GitHub Flow Lifecycle

CI/CD Pipeline Architecture

\begin{tikzpicture}[node distance=2cm] \draw[thick, rounded corners, fill=blue!10] (0,0) rectangle (2,1) node[pos=0.5] {Source (Git)}; \draw[->, thick] (2,0.5) -- (3,0.5); \draw[thick, rounded corners, fill=green!10] (3,0) rectangle (5,1) node[pos=0.5] {Build (Test)}; \draw[->, thick] (5,0.5) -- (6,0.5); \draw[thick, rounded corners, fill=orange!10] (6,0) rectangle (8,1) node[pos=0.5] {Staging}; \draw[->, thick] (8,0.5) -- (9,0.5); \draw[thick, rounded corners, fill=red!10] (9,0) rectangle (11,1) node[pos=0.5] {Production};

\end{tikzpicture}

Definition-Example Pairs

• Hotfix Branch: A temporary branch created to fix a critical bug in production immediately.
  • Example: An ML model starts returning null values in production due to a schema change; a developer branches off main, fixes the logic, and merges it back via an expedited pipeline.
• Pull Request (PR): A request to merge code changes from one branch to another, usually involving a peer review.
  • Example: A Data Scientist completes a new feature engineering script and opens a PR; CodePipeline automatically runs unit tests on the PR code before a human reviews the logic.

Worked Examples

Example 1: Configuring a GitHub Trigger for CodePipeline

Scenario: You want to invoke your training pipeline only when a change is pushed to the models/ directory in the main branch.

1. Define Source: Select GitHub (Version 2) as the source provider in AWS CodePipeline.
2. Filter Events: Use the "Filter" configuration to specify:
   • Branch: main
   • File Path: models/**
3. Result: Changes to documentation or UI code in other folders will not trigger the expensive ML training job, saving costs.

Example 2: Implementing a Manual Approval Gate

Scenario: A model is built and tested, but needs a Lead Data Scientist's sign-off before being deployed to the production endpoint.

1. Add Stage: In CodePipeline, add a stage between Build and Deploy.
2. Action Type: Select Manual Approval.
3. Notification: Configure an SNS topic to email the lead engineer when a model is ready.
4. Result: The pipeline pauses; once the engineer clicks "Approve" in the AWS Console, the CodeDeploy stage begins.

Checkpoint Questions

1. Which branching strategy is better suited for a team requiring frequent, multiple daily deployments to production?
2. In AWS CodePipeline, what is the difference between a "Source" stage and a "Build" stage?
3. Why is it recommended to use Webhooks instead of periodic Polling for pipeline triggers?
4. What role does Amazon EventBridge play in MLOps pipeline invocation?

Muddy Points & Cross-Refs

• Gitflow Complexity: Students often struggle with the difference between develop and release branches. Tip: Think of 'develop' as the kitchen where everyone is cooking, and 'release' as the staging area where the plate is polished before being served (Master/Production).
• Model Registry vs Git: While code lives in Git, models live in the Amazon SageMaker Model Registry. The pipeline usually triggers when the code changes, which then produces a versioned model in the registry.
• Cross-Ref: For more on deployment patterns, see the Deployment Strategies chapter (Blue/Green vs. Canary).

Comparison Tables