Lab: Integrating Automated Testing into AWS CI/CD Pipelines

In this lab, you will learn how to shift-left your testing strategy by integrating automated unit tests into an AWS CI/CD pipeline. You will configure AWS CodeBuild to execute tests during the build phase and use AWS CodePipeline to ensure that code is only eligible for deployment if it passes all quality checks.

[!WARNING] Remember to run the teardown commands at the end of the lab to avoid ongoing charges.

Prerequisites

An AWS Account with Administrator access.
AWS CLI installed and configured on your local machine.
Basic knowledge of Python (we will use a simple Python application for this lab).
Git installed locally for repository interactions.

Learning Objectives

Configure an AWS CodeBuild project to run automated unit tests.
Use a buildspec.yml file to define test commands and interpret application exit codes.
Orchestrate a multi-stage workflow using AWS CodePipeline.
Implement a "fail-fast" mechanism where failed tests stop the pipeline deployment.

Architecture Overview

This architecture demonstrates a standard CI/CD flow where testing is decoupled from deployment but acts as a mandatory gate.

Loading Diagram...

Figure 1 — Mermaid diagram

Step-by-Step Instructions

Step 1: Initialize the Code Repository

First, we need a repository and a simple application with a test suite.

CLI Instructions:

bash

# Create a CodeCommit repository
aws codecommit create-repository --repository-name brainybee-lab-repo --repository-description "Lab for automated testing"

# Clone the empty repo (using HTTPS helper)
git clone https://git-codecommit.<YOUR_REGION>.amazonaws.com/v1/repos/brainybee-lab-repo
cd brainybee-lab-repo

Console Alternative:

▶Click to expand Console steps

Navigate to CodeCommit > Repositories.
Click Create repository.
Name it brainybee-lab-repo and click Create.

Step 2: Create the Application and Tests

Create a simple calculator app and a corresponding test file.

Add app.py:

python

def add(a, b):
    return a + b

Add test_app.py:

python

import unittest
from app import add

class TestApp(unittest.TestCase):
    def test_add(self):
        self.assertEqual(add(2, 3), 5)

if __name__ == '__main__':
    unittest.main()

Step 3: Define the Build Specification

The buildspec.yml tells CodeBuild how to run your tests. If python test_app.py returns a non-zero exit code, CodeBuild marks the build as FAILED.

Add buildspec.yml:

yaml

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.11
  pre_build:
    commands:
      - echo "Installing dependencies..."
  build:
    commands:
      - echo "Running unit tests..."
      - python test_app.py
artifacts:
  files:
    - app.py

Commit and Push:

bash

git add .
git commit -m "Add app, tests, and buildspec"
git push

Step 4: Create the CodeBuild Project

Now, create the build project that will execute the buildspec.yml commands.

CLI Instructions:

bash

# Create a service role for CodeBuild (simplification for lab)
# In production, use least privilege
aws codebuild create-project --name brainybee-test-project \
    --source { "type": "CODECOMMIT", "location": "https://git-codecommit.<YOUR_REGION>.amazonaws.com/v1/repos/brainybee-lab-repo" } \
    --artifacts { "type": "NO_ARTIFACTS" } \
    --environment { "type": "LINUX_CONTAINER", "image": "aws/codebuild/standard:7.0", "computeType": "BUILD_GENERAL1_SMALL" } \
    --service-role <YOUR_CODEBUILD_ROLE_ARN>

[!TIP] Use the Console if you don't have a pre-existing IAM role ARN ready; the Console wizard can create one for you automatically.

Step 5: Orchestrate with CodePipeline

Link CodeCommit to CodeBuild so tests run on every push.

Console Instructions:

Go to CodePipeline > Create pipeline.
Name: brainybee-pipeline.
Source stage: Select AWS CodeCommit, repo brainybee-lab-repo, branch main.
Build stage: Select AWS CodeBuild, project brainybee-test-project.
Deploy stage: Skip for this lab.
Review and Create pipeline.

Checkpoints

Checkpoint	Expected Result
CodePipeline Execution	The pipeline should start automatically after creation and reach the "Build" stage.
CodeBuild Logs	Open the Build details. You should see `python test_app.py` returning `OK` in the logs.
Pipeline Status	The Build stage should show a green "Succeeded" status.

Troubleshooting

Error	Cause	Fix
`AccessDeniedException`	CodeBuild role lacks permission to CodeCommit.	Attach `AWSCodeCommitReadOnly` to the CodeBuild service role.
`YAML_FILE_ERROR`	Syntax error in `buildspec.yml`.	Validate YAML indentation and ensure version is `0.2`.
`Command not found`	Python version mismatch.	Ensure the `runtime-versions` in `buildspec.yml` matches the image version.

Challenge

Intentional Failure: Modify test_app.py so that the test fails (e.g., self.assertEqual(add(2, 3), 6)). Push the change to CodeCommit.

Goal: Observe CodePipeline failing at the Build stage. Verify that the pipeline stops and does not proceed, preventing "broken" code from reaching downstream stages.

Clean-Up / Teardown

To avoid costs, delete the resources in this order:

bash

# 1. Delete the Pipeline
aws codepipeline delete-pipeline --name brainybee-pipeline

# 2. Delete the CodeBuild Project
aws codebuild delete-project --name brainybee-test-project

# 3. Delete the CodeCommit Repository
aws codecommit delete-repository --repository-name brainybee-lab-repo

Cost Estimate

Service	Estimated Cost (30 mins)
AWS CodePipeline	$0.00 (First active pipeline is free per month)
AWS CodeBuild	~$0.01 (Standard1.small is free for first 100 mins/month)
AWS CodeCommit	$0.00 (First 5 active users are free)
Total	$0.00 (within Free Tier)

Concept Review

Key Testing Concepts in CI/CD

Test Type	Phase	Purpose
Unit Test	Build	Verifies individual functions/methods in isolation. Fast execution.
Integration Test	Post-Build	Verifies that the application interacts correctly with other services (e.g., Databases).
Linting	Pre-Build	Checks code style and syntax without executing the code.
Security Scan	Build	Scans for hardcoded secrets or vulnerable dependencies.

Why Exit Codes Matter

AWS CodeBuild determines the success of a command based on the POSIX exit code.

0: Success. The pipeline continues.
Non-zero (e.g., 1): Failure. The pipeline stops immediately, preventing faulty code from being deployed.

Lab: Integrating Automated Testing into AWS CI/CD Pipelines

[!WARNING] Remember to run the teardown commands at the end of the lab to avoid ongoing charges.

Prerequisites

An AWS Account with Administrator access.
AWS CLI installed and configured on your local machine.
Basic knowledge of Python (we will use a simple Python application for this lab).
Git installed locally for repository interactions.

Learning Objectives

Configure an AWS CodeBuild project to run automated unit tests.
Use a buildspec.yml file to define test commands and interpret application exit codes.
Orchestrate a multi-stage workflow using AWS CodePipeline.
Implement a "fail-fast" mechanism where failed tests stop the pipeline deployment.

Architecture Overview

This architecture demonstrates a standard CI/CD flow where testing is decoupled from deployment but acts as a mandatory gate.

Loading Diagram...

Figure 1 — Mermaid diagram

Step-by-Step Instructions

Step 1: Initialize the Code Repository

First, we need a repository and a simple application with a test suite.

CLI Instructions:

bash

# Create a CodeCommit repository
aws codecommit create-repository --repository-name brainybee-lab-repo --repository-description "Lab for automated testing"

# Clone the empty repo (using HTTPS helper)
git clone https://git-codecommit.<YOUR_REGION>.amazonaws.com/v1/repos/brainybee-lab-repo
cd brainybee-lab-repo

Console Alternative:

▶Click to expand Console steps

Navigate to CodeCommit > Repositories.
Click Create repository.
Name it brainybee-lab-repo and click Create.

Step 2: Create the Application and Tests

Create a simple calculator app and a corresponding test file.

Add app.py:

python

def add(a, b):
    return a + b

Add test_app.py:

python

import unittest
from app import add

class TestApp(unittest.TestCase):
    def test_add(self):
        self.assertEqual(add(2, 3), 5)

if __name__ == '__main__':
    unittest.main()

Step 3: Define the Build Specification

The buildspec.yml tells CodeBuild how to run your tests. If python test_app.py returns a non-zero exit code, CodeBuild marks the build as FAILED.

Add buildspec.yml:

yaml

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.11
  pre_build:
    commands:
      - echo "Installing dependencies..."
  build:
    commands:
      - echo "Running unit tests..."
      - python test_app.py
artifacts:
  files:
    - app.py

Commit and Push:

bash

git add .
git commit -m "Add app, tests, and buildspec"
git push

Step 4: Create the CodeBuild Project

Now, create the build project that will execute the buildspec.yml commands.

CLI Instructions:

bash

# Create a service role for CodeBuild (simplification for lab)
# In production, use least privilege
aws codebuild create-project --name brainybee-test-project \
    --source { "type": "CODECOMMIT", "location": "https://git-codecommit.<YOUR_REGION>.amazonaws.com/v1/repos/brainybee-lab-repo" } \
    --artifacts { "type": "NO_ARTIFACTS" } \
    --environment { "type": "LINUX_CONTAINER", "image": "aws/codebuild/standard:7.0", "computeType": "BUILD_GENERAL1_SMALL" } \
    --service-role <YOUR_CODEBUILD_ROLE_ARN>

[!TIP] Use the Console if you don't have a pre-existing IAM role ARN ready; the Console wizard can create one for you automatically.

Step 5: Orchestrate with CodePipeline

Link CodeCommit to CodeBuild so tests run on every push.

Console Instructions:

Go to CodePipeline > Create pipeline.
Name: brainybee-pipeline.
Source stage: Select AWS CodeCommit, repo brainybee-lab-repo, branch main.
Build stage: Select AWS CodeBuild, project brainybee-test-project.
Deploy stage: Skip for this lab.
Review and Create pipeline.

Checkpoints

Checkpoint	Expected Result
CodePipeline Execution	The pipeline should start automatically after creation and reach the "Build" stage.
CodeBuild Logs	Open the Build details. You should see `python test_app.py` returning `OK` in the logs.
Pipeline Status	The Build stage should show a green "Succeeded" status.

Troubleshooting

Error	Cause	Fix
`AccessDeniedException`	CodeBuild role lacks permission to CodeCommit.	Attach `AWSCodeCommitReadOnly` to the CodeBuild service role.
`YAML_FILE_ERROR`	Syntax error in `buildspec.yml`.	Validate YAML indentation and ensure version is `0.2`.
`Command not found`	Python version mismatch.	Ensure the `runtime-versions` in `buildspec.yml` matches the image version.

Challenge

Intentional Failure: Modify test_app.py so that the test fails (e.g., self.assertEqual(add(2, 3), 6)). Push the change to CodeCommit.

Goal: Observe CodePipeline failing at the Build stage. Verify that the pipeline stops and does not proceed, preventing "broken" code from reaching downstream stages.

Clean-Up / Teardown

To avoid costs, delete the resources in this order:

bash

# 1. Delete the Pipeline
aws codepipeline delete-pipeline --name brainybee-pipeline

# 2. Delete the CodeBuild Project
aws codebuild delete-project --name brainybee-test-project

# 3. Delete the CodeCommit Repository
aws codecommit delete-repository --repository-name brainybee-lab-repo

Cost Estimate

Service	Estimated Cost (30 mins)
AWS CodePipeline	$0.00 (First active pipeline is free per month)
AWS CodeBuild	~$0.01 (Standard1.small is free for first 100 mins/month)
AWS CodeCommit	$0.00 (First 5 active users are free)
Total	$0.00 (within Free Tier)

Concept Review

Key Testing Concepts in CI/CD

Test Type	Phase	Purpose
Unit Test	Build	Verifies individual functions/methods in isolation. Fast execution.
Integration Test	Post-Build	Verifies that the application interacts correctly with other services (e.g., Databases).
Linting	Pre-Build	Checks code style and syntax without executing the code.
Security Scan	Build	Scans for hardcoded secrets or vulnerable dependencies.

Why Exit Codes Matter

AWS CodeBuild determines the success of a command based on the POSIX exit code.

0: Success. The pipeline continues.
Non-zero (e.g., 1): Failure. The pipeline stops immediately, preventing faulty code from being deployed.