Artifact Lifecycle Considerations: Generation, Storage, and Management

This guide covers the critical aspects of building, securing, and managing artifacts within an AWS CI/CD ecosystem, focusing on tools like AWS CodeBuild, Amazon S3, AWS CodeArtifact, and Amazon ECR.

Learning Objectives

After studying this guide, you should be able to:

Configure build tools to generate and store output artifacts.
Implement secure access patterns for artifact repositories using IAM.
Design artifact lifecycle strategies to manage storage costs and retention.
Differentiate between artifact storage types (S3 vs. CodeArtifact vs. ECR).

Key Terms & Glossary

Artifact: A deployable component (e.g., .zip, .jar, Docker image) produced during the build phase.
Buildspec: A YAML file used by AWS CodeBuild to define the build commands and output artifacts.
Immutability: The practice of never modifying an artifact once created; instead, a new version is produced.
S3 Lifecycle Policy: Rules that automatically transition or delete objects based on age or version.
Semantic Versioning (SemVer): A versioning scheme (Major.Minor.Patch) used to track artifact iterations.

The "Big Idea"

Artifacts represent the truth of what is being deployed. In a mature DevOps pipeline, artifacts must be generated once, stored securely, and treated as immutable. The "Lifecycle" isn't just about creation; it's about ensuring the right versions are available for deployment while old or failed versions are purged to maintain security and reduce storage overhead.

Formula / Concept Box

Concept	Implementation Tool	Key Configuration Parameter
Build Definition	AWS CodeBuild	`buildspec.yml` (artifacts/files section)
Software Packages	AWS CodeArtifact	Upstream repositories & Domain permissions
Container Images	Amazon ECR	Lifecycle Policies (untagged vs. tagged)
Object Storage	Amazon S3	Versioning & Lifecycle transition rules

Hierarchical Outline

Artifact Generation
- AWS CodeBuild: Uses buildspec.yml to define output files.
- Secondary Artifacts: Ability to output multiple artifact sets to different S3 locations.
Artifact Repositories
- Amazon S3: Best for raw binaries and deployment bundles.
- Amazon ECR: Specialized for Docker/OCI container images.
- AWS CodeArtifact: Managed repository for language-specific dependencies (npm, pip, maven).
Security & Access
- IAM Roles: Granting CodeBuild s3:PutObject and CodeDeploy s3:GetObject permissions.
- Resource Policies: Restricting ECR or S3 access to specific VPCs or Accounts.
Lifecycle Management
- Expiration: Deleting old versions to save costs.
- Transition: Moving older artifacts to S3 Glacier for long-term audit compliance.

Visual Anchors

Artifact Pipeline Flow

Loading Diagram...

Figure 1 — Mermaid diagram

S3 Lifecycle State Machine

Compiling TikZ diagram…

⏳

Running TeX engine…

This may take a few seconds

Figure 2 — TikZ diagram

Definition-Example Pairs

Immutable Artifact: An artifact that is never changed after creation.
- Example: Instead of overwriting app-v1.zip in S3, you upload app-v2.zip. This allows for instant rollbacks if v2 fails.
Upstream Repository: A source repository from which an artifact repository fetches packages.
- Example: Configuring AWS CodeArtifact to use npmjs.com as an upstream so it caches public packages internally.

Worked Examples

1. Defining Artifacts in `buildspec.yml`

To output a Java JAR file as an artifact, your buildspec.yml must include the artifacts section:

yaml

version: 0.2
phases:
  build:
    commands:
      - mvn package
artifacts:
  files:
    - target/my-app.jar
    - appspec.yml
    - scripts/**/*
  discard-paths: yes

Explanation: This tells CodeBuild to grab the JAR, the deployment spec, and all helper scripts, zip them, and send them to the S3 bucket configured in the CodeBuild project.

2. ECR Lifecycle Policy for Untagged Images

To prevent cost bloat from failed builds, we can delete untagged images after 1 day:

json

{
    "rules": [
        {
            "rulePriority": 1,
            "description": "Expire untagged images",
            "selection": {
                "tagStatus": "untagged",
                "countType": "imageCountMoreThan",
                "countNumber": 1
            },
            "action": { "type": "expire" }
        }
    ]
}

Checkpoint Questions

Why is it considered a best practice to use S3 Versioning for artifact buckets?
In CodeBuild, what happens if the artifacts section is missing from the buildspec.yml?
Which AWS service would you use to share private Python packages across multiple development teams?
How does an "Immutable" deployment pattern reduce the risk during a rollback?

Muddy Points & Cross-Refs

S3 vs. CodeArtifact: Use S3 for your own build outputs (deployable zips). Use CodeArtifact for dependencies and libraries (npm/maven) used by other builds.
Artifact vs. Source: A "Source" is the raw code. An "Artifact" is the result of a build process. CodePipeline manages the transition between the two.
Cross-Account Access: If your build is in Account A and your S3 bucket is in Account B, you need both an IAM Role in A and a Bucket Policy in B.

Comparison Tables

Feature	Amazon S3	AWS CodeArtifact	Amazon ECR
Primary Use Case	Deployable Bundles	Software Packages (npm/pip)	Docker/OCI Images
Versioning	Optional (S3 Versioning)	Native (SemVer)	Native (Image Tags)
Cleanup Logic	S3 Lifecycle Rules	Manual / API	ECR Lifecycle Policies
Integration	CodeDeploy, Beanstalk	CodeBuild, Jenkins	ECS, EKS, Lambda

Artifact Lifecycle Considerations: Generation, Storage, and Management

This guide covers the critical aspects of building, securing, and managing artifacts within an AWS CI/CD ecosystem, focusing on tools like AWS CodeBuild, Amazon S3, AWS CodeArtifact, and Amazon ECR.

Learning Objectives

After studying this guide, you should be able to:

Configure build tools to generate and store output artifacts.
Implement secure access patterns for artifact repositories using IAM.
Design artifact lifecycle strategies to manage storage costs and retention.
Differentiate between artifact storage types (S3 vs. CodeArtifact vs. ECR).

Key Terms & Glossary

Artifact: A deployable component (e.g., .zip, .jar, Docker image) produced during the build phase.
Buildspec: A YAML file used by AWS CodeBuild to define the build commands and output artifacts.
Immutability: The practice of never modifying an artifact once created; instead, a new version is produced.
S3 Lifecycle Policy: Rules that automatically transition or delete objects based on age or version.
Semantic Versioning (SemVer): A versioning scheme (Major.Minor.Patch) used to track artifact iterations.

The "Big Idea"

Formula / Concept Box

Concept	Implementation Tool	Key Configuration Parameter
Build Definition	AWS CodeBuild	`buildspec.yml` (artifacts/files section)
Software Packages	AWS CodeArtifact	Upstream repositories & Domain permissions
Container Images	Amazon ECR	Lifecycle Policies (untagged vs. tagged)
Object Storage	Amazon S3	Versioning & Lifecycle transition rules

Hierarchical Outline

Artifact Generation
- AWS CodeBuild: Uses buildspec.yml to define output files.
- Secondary Artifacts: Ability to output multiple artifact sets to different S3 locations.
Artifact Repositories
- Amazon S3: Best for raw binaries and deployment bundles.
- Amazon ECR: Specialized for Docker/OCI container images.
- AWS CodeArtifact: Managed repository for language-specific dependencies (npm, pip, maven).
Security & Access
- IAM Roles: Granting CodeBuild s3:PutObject and CodeDeploy s3:GetObject permissions.
- Resource Policies: Restricting ECR or S3 access to specific VPCs or Accounts.
Lifecycle Management
- Expiration: Deleting old versions to save costs.
- Transition: Moving older artifacts to S3 Glacier for long-term audit compliance.

Visual Anchors

Artifact Pipeline Flow

Loading Diagram...

Figure 1 — Mermaid diagram

S3 Lifecycle State Machine

Compiling TikZ diagram…

⏳

Running TeX engine…

This may take a few seconds

Figure 2 — TikZ diagram

Definition-Example Pairs

Immutable Artifact: An artifact that is never changed after creation.
- Example: Instead of overwriting app-v1.zip in S3, you upload app-v2.zip. This allows for instant rollbacks if v2 fails.
Upstream Repository: A source repository from which an artifact repository fetches packages.
- Example: Configuring AWS CodeArtifact to use npmjs.com as an upstream so it caches public packages internally.

Worked Examples

1. Defining Artifacts in `buildspec.yml`

To output a Java JAR file as an artifact, your buildspec.yml must include the artifacts section:

yaml

version: 0.2
phases:
  build:
    commands:
      - mvn package
artifacts:
  files:
    - target/my-app.jar
    - appspec.yml
    - scripts/**/*
  discard-paths: yes

Explanation: This tells CodeBuild to grab the JAR, the deployment spec, and all helper scripts, zip them, and send them to the S3 bucket configured in the CodeBuild project.

2. ECR Lifecycle Policy for Untagged Images

To prevent cost bloat from failed builds, we can delete untagged images after 1 day:

json

{
    "rules": [
        {
            "rulePriority": 1,
            "description": "Expire untagged images",
            "selection": {
                "tagStatus": "untagged",
                "countType": "imageCountMoreThan",
                "countNumber": 1
            },
            "action": { "type": "expire" }
        }
    ]
}

Checkpoint Questions

Why is it considered a best practice to use S3 Versioning for artifact buckets?
In CodeBuild, what happens if the artifacts section is missing from the buildspec.yml?
Which AWS service would you use to share private Python packages across multiple development teams?
How does an "Immutable" deployment pattern reduce the risk during a rollback?

Muddy Points & Cross-Refs

S3 vs. CodeArtifact: Use S3 for your own build outputs (deployable zips). Use CodeArtifact for dependencies and libraries (npm/maven) used by other builds.
Artifact vs. Source: A "Source" is the raw code. An "Artifact" is the result of a build process. CodePipeline manages the transition between the two.
Cross-Account Access: If your build is in Account A and your S3 bucket is in Account B, you need both an IAM Role in A and a Bucket Policy in B.

Comparison Tables

Feature	Amazon S3	AWS CodeArtifact	Amazon ECR
Primary Use Case	Deployable Bundles	Software Packages (npm/pip)	Docker/OCI Images
Versioning	Optional (S3 Versioning)	Native (SemVer)	Native (Image Tags)
Cleanup Logic	S3 Lifecycle Rules	Manual / API	ECR Lifecycle Policies
Integration	CodeDeploy, Beanstalk	CodeBuild, Jenkins	ECS, EKS, Lambda

Artifact Lifecycle Considerations: Generation, Storage, and Management

Artifact Lifecycle Considerations: Generation, Storage, and Management

Learning Objectives

Key Terms & Glossary

The "Big Idea"

Formula / Concept Box

Hierarchical Outline

Visual Anchors

Artifact Pipeline Flow

S3 Lifecycle State Machine

Definition-Example Pairs

Worked Examples

1. Defining Artifacts in buildspec.yml

2. ECR Lifecycle Policy for Untagged Images

Checkpoint Questions

Muddy Points & Cross-Refs

Comparison Tables

Artifact Lifecycle Considerations: Generation, Storage, and Management

Artifact Lifecycle Considerations: Generation, Storage, and Management

Learning Objectives

Key Terms & Glossary

The "Big Idea"

Formula / Concept Box

Hierarchical Outline

Visual Anchors

Artifact Pipeline Flow

S3 Lifecycle State Machine

Definition-Example Pairs

Worked Examples

1. Defining Artifacts in buildspec.yml

2. ECR Lifecycle Policy for Untagged Images

Checkpoint Questions

Muddy Points & Cross-Refs

Comparison Tables

1. Defining Artifacts in `buildspec.yml`

1. Defining Artifacts in `buildspec.yml`