Implementing CI/CD Pipelines

This guide covers the fundamental and advanced concepts of Continuous Integration and Continuous Delivery (CI/CD) specifically within the AWS ecosystem, focusing on AWS CodePipeline and its orchestration capabilities.

Learning Objectives

After studying this chapter, you should be able to:

• Explain the core components of AWS CodePipeline including stages, actions, and transitions.
• Design a pipeline that incorporates manual approvals and parallel execution.
• Manage artifacts and secrets securely across the software development lifecycle (SDLC).
• Integrate automated testing (unit, integration, and security) into the deployment workflow.
• Implement cross-region and multi-account deployment patterns.

Key Terms & Glossary

• Continuous Delivery (CD): An automated process where code changes are automatically built, tested, and prepared for a release to production.
• Pipeline: A declarative JSON/YAML document defining the release process workflow.
• Revision: A specific change made to the source location (e.g., a specific commit in GitHub or a versioned S3 object).
• Artifact: The file or set of files (like compiled code or zip files) produced by an action and consumed by subsequent actions.
• Transition: The link between two stages that can be disabled to "gate" the progression of code.
• runOrder: A parameter used to determine the sequence of actions within a stage (lower numbers run first; same numbers run in parallel).

The "Big Idea"

At its core, CI/CD is about reducing lead time and increasing reliability. Instead of manual, monolithic releases, we treat our release process as code. AWS CodePipeline acts as the "orchestrator" that glues together specialized tools (CodeBuild for compiling, CodeDeploy for releasing, and S3 for storage) into a single, repeatable, and observable engine for software delivery.

Formula / Concept Box

Hierarchical Outline

• I. AWS CodePipeline Core Architecture
  • Declarative Structure: Defined via JSON; enables versioning of the pipeline itself.
  • Stages: Logical divisions (e.g., Build, Staging, Production).
  • Actions: Specific tasks within stages (e.g., Invoke a Lambda function, Source from GitHub).
• II. Advanced Orchestration
  • Parallel Execution: Using the same runOrder value for multiple actions to speed up execution.
  • Manual Approvals: Custom SNS notifications to human operators before sensitive deployments.
  • Triggers: PollForSourceChanges vs. Event-driven (Webhooks/EventBridge) triggers.
• III. Security & Artifact Management
  • Secrets Management: Integrating AWS Secrets Manager or SSM Parameter Store to avoid hardcoding credentials.
  • Artifact Encryption: S3 buckets used for artifact storage should use KMS for encryption at rest.
• IV. Automated Testing Integration
  • Shift-Left Testing: Moving unit and security scans (SAST) to the earliest possible stages (Build).
  • Gating: Failing the pipeline if code coverage or unit tests do not meet thresholds.

Visual Anchors

Pipeline Workflow Flowchart

Artifact Flow Mechanism

Definition-Example Pairs

• Manual Approval: A pause in the pipeline that waits for an IAM user to click "Approve" or "Reject".
  • Example: A pipeline stops after the "Staging" deploy to allow a QA Lead to verify the UI before the code hits the "Production" stage.
• Parallel Action: Two or more actions running simultaneously in the same stage.
  • Example: Running a suite of unit tests and a security vulnerability scan at the same time in the "Test" stage to reduce the total wait time by 50%.
• Invoke Action: An action type that triggers an AWS Lambda function.
  • Example: Triggering a Lambda function to clear a CloudFront cache or update a Jira ticket status after a successful deployment.

Worked Examples

Scenario: Configuring Parallel Execution

The Problem: Your pipeline currently takes 20 minutes. 10 minutes are spent running Unit Tests, and 8 minutes are spent running a Security Scan. They currently run one after the other.

The Solution:

1. Open the Pipeline JSON definition.
2. Locate the Test stage.
3. Set the runOrder for the UnitTests action to 1.
4. Set the runOrder for the SecurityScan action to 1.
5. Result: Both actions start simultaneously. The stage duration drops from 18 minutes to 10 minutes (the duration of the longest task).

Scenario: Managing Secrets in CodeBuild

The Problem: Your build process needs an API key to upload assets to a third-party CDN, but you cannot put the key in the buildspec.yml file.

The Solution:

1. Store the key in AWS Secrets Manager as CDN_API_KEY.
2. In the CodeBuild environment configuration, map an environment variable to the Secret ARN.
3. Reference the variable in buildspec.yml using $CDN_API_KEY.

Checkpoint Questions

1. What is the mandatory first stage of any AWS CodePipeline?
2. How does a pipeline communicate that a manual approval is needed?
3. If two actions in the same stage have a runOrder of 1 and 2 respectively, will they run in parallel or serial?
4. Where are the files (artifacts) passed between stages stored?

▶View Answers

1. The Source stage.
2. Via an Amazon SNS (Simple Notification Service) topic.
3. Serial (1 completes before 2 starts).
4. In an Amazon S3 bucket (the Artifact Store).

Muddy Points & Cross-Refs

• Polling vs. Webhooks: Many learners are confused by PollForSourceChanges. In modern setups, EventBridge events are preferred over polling because they are near-instant and more cost-effective.
• Cross-Account Deployments: This requires complex IAM Role assumption and S3 bucket policy updates. Reference the IAM and Organizations chapter for details on AssumeRole operations.
• Artifact Formats: Artifacts are almost always .zip files. If your Build action produces a folder, CodeBuild will automatically zip it before uploading to the Artifact Store.

Comparison Tables

Deployment Pattern Comparison

Secrets Storage Comparison