Scalable and Resilient Architectures: Scaling, Balancing, and Caching

This study guide focuses on Domain 3 of the AWS Certified DevOps Engineer - Professional (DOP-C02) exam, specifically regarding the implementation of solutions that scale to meet business requirements while ensuring high availability through load balancing and caching.

Learning Objectives

By the end of this module, you should be able to:

• Translate business resiliency requirements into technical AWS architectures.
• Implement appropriate Auto Scaling policies using both standard and custom metrics.
• Configure Elastic Load Balancing (ELB) to support cross-AZ and cross-Region resiliency.
• Identify single points of failure (SPOFs) and remediate them using distributed architectures.
• Optimize performance and cost using caching solutions like Amazon CloudFront and ElastiCache.

Key Terms & Glossary

• Horizontal Scaling: Adding more instances to a resource pool (scaling out) rather than increasing the power of a single instance (scaling up).
• Target Tracking Policy: A scaling policy that increases or decreases the current capacity of a group based on a target value for a specific metric (e.g., maintain average CPU at 50%).
• Cooldown Period: A configurable setting for Auto Scaling that ensures the group does not launch or terminate additional instances before the previous scaling activity takes effect.
• Sticky Sessions (Session Affinity): A load balancer feature that binds a user's session to a specific target instance for the duration of the session.
• RTO (Recovery Time Objective): The maximum acceptable delay between the interruption of service and restoration of service.
• RPO (Recovery Point Objective): The maximum acceptable amount of data loss measured in time.

The "Big Idea"

Resiliency is the ability of an application to resist or recover from hardware or software failure. In the AWS ecosystem, this is achieved by moving away from monolithic, "fragile" systems toward loosely coupled, distributed architectures. By combining Auto Scaling (to handle volume), Load Balancing (to ensure health and distribution), and Caching (to reduce latency and origin load), you create a system that can self-heal and grow dynamically without manual intervention.

Formula / Concept Box

Hierarchical Outline

1. Auto Scaling Strategies
   • Scaling Policies: Target Tracking (recommended), Step Scaling (for aggressive response), and Scheduled Scaling (for predictable patterns).
   • Custom Metrics: Using the CloudWatch Agent to scale on Memory utilization or disk space (not available by default).
   • Lifecycle Hooks: Pausing instances during launch or termination to perform software installation or data backup.
2. Load Balancing Architecture
   • Application Load Balancer (ALB): Layer 7; supports path-based and host-based routing; ideal for microservices/containers.
   • Network Load Balancer (NLB): Layer 4; ultra-high performance; static IP support; ideal for TCP/UDP traffic.
   • Health Checks: Differentiating between EC2 status checks (hardware) and ELB health checks (application layer response).
3. Caching and Data Distribution
   • Edge Caching: Using CloudFront to cache static and dynamic content globally.
   • Database Caching: Implementing ElastiCache (Redis/Memcached) to offload read pressure from RDS.
   • Global Resiliency: Using Route 53 (Failover/Latency routing) and DynamoDB Global Tables.

Visual Anchors

Auto Scaling Decision Flow

Multi-Region Resilient Architecture

Definition-Example Pairs

• Loose Coupling: An architecture where components have little or no knowledge of the definitions of other separate components.
  • Example: Using an SQS queue between a frontend web server and a backend image processor so the frontend doesn't break if the backend is down.
• Vertical Scaling: Increasing the capacity of an existing resource.
  • Example: Changing an EC2 instance type from t3.medium to m5.large during a maintenance window.
• Pilot Light: A DR strategy where a minimal version of an environment is always running in another region.
  • Example: Keeping an RDS Read Replica in a secondary region, but only launching the EC2 application fleet when a disaster occurs.

Worked Examples

Example 1: Scaling based on SQS Queue Depth

Problem: A video processing application is falling behind on jobs. Standard CPU scaling isn't working because the processing is I/O bound. Solution:

1. Identify the metric ApproximateNumberOfMessagesVisible in SQS.
2. Create a Custom Metric in CloudWatch called BacklogPerInstance (Queue Depth / Number of Instances in ASG).
3. Configure a Target Tracking Policy in the Auto Scaling Group to keep BacklogPerInstance at 10.
4. Result: As the queue grows, the ASG scales out proportionally to the workload, regardless of CPU usage.

Example 2: Eliminating SPOF in a Web Tier

Problem: A legacy application runs on a single large EC2 instance with an Elastic IP. Solution:

1. Create an Amazon Machine Image (AMI) of the instance.
2. Create a Launch Template using that AMI.
3. Deploy an ALB across three Availability Zones.
4. Create an ASG with a minimum capacity of 2 across two AZs.
5. Update DNS to point to the ALB's DNS name instead of the Elastic IP.

Checkpoint Questions

1. What is the main difference between an ASG health check and an ELB health check?
2. Why would a DevOps engineer use a "Step Scaling" policy instead of "Target Tracking"?
3. Which load balancer should be used for a legacy application that requires fixed IP addresses for its clients?
4. How does CloudFront improve resiliency for the origin server during a traffic spike?

▶Click to see answers

1. ASG health checks focus on instance state (running/impaired); ELB health checks test the application responsiveness (e.g., HTTP 200 OK).
2. Step scaling allows for different scaling increments based on the size of the breach (e.g., add 1 instance at 60% CPU, but add 4 instances at 90% CPU).
3. Network Load Balancer (NLB), as it supports Elastic IPs.
4. By serving cached content from the edge, it reduces the number of requests that reach the origin (request collapsing and offloading).

Muddy Points & Cross-Refs

• Cooldown vs. Warm-up: The Cooldown (Default 300s) happens AFTER a scaling activity to let the metric stabilize. Warm-up is part of a Target Tracking policy and dictates how long an instance needs before it contributes to the group's metrics.
• Cross-Zone Load Balancing: In ALB, this is always on. In NLB, it is off by default. If off, each load balancer node only distributes traffic to targets in its own AZ.
• Deep Dive: For more on automating the CloudWatch agent, see the Configuration Management module.

Comparison Tables

Load Balancer Selection

Caching Strategies