Change Management Processes for IaC-based Platforms

This study guide focuses on the methodologies and AWS-native tools used to manage infrastructure changes safely, predictably, and at scale. In the context of the AWS DevOps Engineer Professional exam, this covers the lifecycle of an Infrastructure as Code (IaC) change from development to multi-account deployment.

Learning Objectives

After studying this guide, you should be able to:

• Describe the lifecycle of an infrastructure change using AWS CloudFormation.
• Evaluate the impact of proposed changes using Change Sets.
• Implement automated safety mechanisms like Rollback Triggers and Termination Protection.
• Orchestrate changes across multi-account and multi-Region environments using StackSets.
• Detect and remediate Infrastructure Drift using AWS Config and native CloudFormation features.

Key Terms & Glossary

• Idempotency: The property where an operation can be performed multiple times without changing the result beyond the initial application. IaC tools must be idempotent to ensure the environment remains consistent.
• Change Set: A preview of how proposed changes to a CloudFormation stack will affect running resources (e.g., will a database be updated or deleted?).
• Drift: A condition where the actual configuration of a resource deviates from its defined state in the IaC template, often due to manual "out-of-band" changes.
• Immutable Infrastructure: A strategy where infrastructure is never modified after deployment. Instead, new versions are provisioned to replace old ones.
• StackSet: A CloudFormation feature that allows you to create, update, or delete stacks across multiple AWS accounts and Regions with a single operation.

The "Big Idea"

In a DevOps culture, infrastructure is treated as software. This means changes are not manual "clicks" in a console, but version-controlled code updates. The "Big Idea" is to move from a high-risk, manual change process to a low-risk, automated pipeline where every change is peer-reviewed, tested in staging, and deployed using tools that provide safety nets like automatic rollbacks and drift detection.

Formula / Concept Box

Hierarchical Outline

• I. The Change Management Lifecycle
  • A. Authoring: Writing templates in YAML/JSON or using AWS CDK (Constructs).
  • B. Versioning: Storing code in AWS CodeCommit or GitHub for auditability.
  • C. Testing: Using task-specific tools like cfn-lint or taskcat for multi-Region testing.
• II. Advanced CloudFormation Features
  • A. Rollback Triggers: Integrating with CloudWatch to monitor health during updates.
  • B. Termination Protection: Preventing accidental deletion of critical stacks (e.g., Production DBs).
  • C. Nested Stacks: Creating reusable components to manage complex architectures.
• III. Multi-Account Governance
  • A. AWS Organizations: The foundation for multi-account management.
  • B. StackSets: Managing global footprints; using Service-Managed Permissions for auto-deployment to new accounts.
  • C. AWS Service Catalog: Providing pre-approved, compliant IaC templates to end-users.

Visual Anchors

Infrastructure Change Pipeline

This flowchart illustrates the path a code change takes to safely reach production.

The Relationship of Change Sets

The following diagram represents how a Change Set acts as a "middleman" between the new template and the live stack to prevent accidental outages.

Definition-Example Pairs

• Drift Detection: The process of identifying when a resource's property has been changed manually outside of the IaC template.
  • Example: An administrator manually changes the instance type of an EC2 instance from t3.medium to m5.large. Drift detection will flag the stack as "DRIFTED."
• Blue/Green Deployment (Infrastructure): Deploying a second, identical stack (Green) alongside the current one (Blue) and shifting traffic.
  • Example: Creating a new CloudFormation stack with an updated AMI and switching the Route 53 record to point to the new Load Balancer once health checks pass.
• Stack Policy: A JSON document that defines the update actions that can be performed on designated resources.
  • Example: Attaching a policy to a stack that prevents any user from updating or deleting a specific RDS instance, even if they have the permissions to update the stack.

Worked Examples

Scenario: Managing a Production Database Update

Problem: A DevOps engineer needs to update an RDS instance in a CloudFormation stack to increase its storage. However, they must ensure the database is not replaced, which would cause data loss.

Step-by-Step Breakdown:

1. Modify Template: Update the AllocatedStorage property in the YAML template.
2. Create Change Set: Run aws cloudformation create-change-set.
3. Inspect Change Set: Use the AWS Console or describe-change-set to check the Replacement field.
   • If Replacement: True, the DB will be deleted and recreated. Stop here.
   • If Replacement: False (or Conditional), the update is an in-place modification.
4. Execute: Once verified that Replacement is not True, run execute-change-set.
5. Monitor: Use Rollback Triggers to watch for DB connection errors in CloudWatch. If errors spike, CloudFormation will revert the storage setting (if supported) or halt the update.

Checkpoint Questions

1. What is the difference between a Replacement: True and Replacement: False in a CloudFormation Change Set?
2. How does AWS CloudFormation StackSets handle the deployment of resources to a new AWS account added to an Organization Unit (OU)?
3. What mechanism should be used to stop an update if a CloudWatch Alarm goes into the ALARM state during the deployment window?
4. Can you use Drift Detection to automatically revert resources to their template-defined state? (Hint: Think about the difference between detection and remediation).

Muddy Points & Cross-Refs

• Manual Remediations: A common confusion is that CloudFormation automatically fixes drift. It does not. It only detects it. To fix it, you must either update the template to match reality or manually change the resource back.
• DeletionPolicy vs. Termination Protection: Termination Protection prevents the entire stack from being deleted. DeletionPolicy (e.g., Retain, Snapshot) controls what happens to individual resources when the stack is deleted.
• Service Catalog vs. StackSets: Use Service Catalog for end-user self-service (standardizing what they can launch). Use StackSets for administrative overhead (standardizing what exists across all accounts, like IAM roles or VPCs).

Comparison Tables

Change Sets vs. Direct Updates

StackSets: Self-Managed vs. Service-Managed

[!IMPORTANT] For the DOP-C02 exam, always prioritize Change Sets for production environments and StackSets for maintaining security baselines in a multi-account landing zone.