Identifying and Remediating Scaling Issues

This guide covers the critical skills required for the AWS Certified DevOps Engineer - Professional (DOP-C02) exam regarding Domain 3: Resilient Cloud Solutions, specifically focusing on Task Statement 3.2: Implement solutions that are scalable to meet business requirements.

Learning Objectives

After studying this chapter, you should be able to:

• Identify appropriate metrics for triggering scaling actions.
• Differentiate between horizontal and vertical scaling and when to use each.
• Implement remediation strategies for scaling bottlenecks in compute, database, and storage layers.
• Configure auto-scaling for various AWS services including EC2, ECS, and DynamoDB.
• Design loosely coupled architectures that prevent cascading failures during scale-out events.

Key Terms & Glossary

• Vertical Scaling (Scaling Up/Down): Increasing or decreasing the capacity of a single resource (e.g., changing to a larger EC2 instance type).
• Horizontal Scaling (Scaling Out/In): Adding or removing resource instances (e.g., adding more EC2 instances to an Auto Scaling Group).
• Cooldown Period: A configurable setting for an Auto Scaling group that prevents the group from launching or terminating additional instances before the previous scaling activity takes effect.
• Throttling: The process of limiting the number of requests a user can make to a service in a given period to prevent resource exhaustion.
• Connection Draining (Deregistration Delay): A process where a load balancer stops sending new requests to a de-registering instance while allowing existing in-flight requests to complete.

The "Big Idea"

Scalability is the ability of a system to handle increased load without compromising performance. In the AWS DevOps Professional context, remediation is as important as identification. True resiliency involves a feedback loop: monitoring for bottlenecks (Identifying), applying automated responses (Auto-scaling), and refining architecture (Remediating Single Points of Failure) to ensure the system is elastic and cost-effective.

Formula / Concept Box

[!IMPORTANT] For SQS-based scaling, use the Backlog Per Instance metric: $BacklogPerInstance = \frac{ApproximateNumberOfMessagesVisible}{TotalCapacityOfASG}$

Hierarchical Outline

• I. Identifying Scaling Issues
  • CloudWatch Metrics: Analyzing CPU, Memory (Custom Metric), Network I/O, and Disk I/O.
  • Application Latency: Identifying bottlenecks in the Application Load Balancer (ALB) TargetResponseTime.
  • Database Bottlenecks: RDS CPU utilization, FreeableMemory, and DiskQueueDepth.
• II. Remediating Compute Scaling
  • EC2 Auto Scaling: Implementing lifecycle hooks for custom initialization.
  • Container Scaling: Using ECS Capacity Providers and EKS Cluster Autoscaler or Karpenter.
  • Serverless Scaling: Managing Lambda concurrency limits and API Gateway throttling.
• III. Remediating Data & Caching Issues
  • Caching: Using Amazon ElastiCache (Redis/Memcached) to offload read-heavy DB workloads.
  • DynamoDB Scaling: Enabling Auto Scaling for RCU/WCU or using Global Tables for multi-region scale.
  • RDS Read Replicas: Scaling read operations horizontally for relational databases.

Visual Anchors

The Scaling Feedback Loop

Multi-AZ Scalable Architecture

Definition-Example Pairs

• Loose Coupling: An architectural principle where components have little to no knowledge of the definitions of other separate components.
  • Example: Using Amazon SQS between a web front-end and a processing back-end. If the back-end cannot scale fast enough, the messages simply wait in the queue rather than causing the front-end to crash.
• Horizontal Pod Autoscaling (HPA): Scaling the number of pods in a Kubernetes deployment based on observed CPU/Memory utilization.
  • Example: An EKS-hosted microservice experiences a surge in requests; HPA increases the pod replicas from 2 to 10 within the existing node capacity.

Worked Examples

Example 1: Remediating a Database Bottleneck

Scenario: A marketing campaign causes a 10x spike in traffic. The web tier scales perfectly, but the RDS MySQL database reaches 100% CPU, causing application timeouts.

Remediation Steps:

1. Identify: Check CloudWatch for CPUUtilization and DatabaseConnections.
2. Short-term Fix: Perform a vertical scale-up of the RDS instance (requires brief downtime unless Multi-AZ).
3. Long-term Remediation:
   • Implement Amazon ElastiCache for frequently accessed, non-changing data.
   • Create RDS Read Replicas and update the application code to split Read/Write traffic.
   • Enable RDS Proxy to manage connection pooling more efficiently.

Example 2: Throttling in Serverless Architectures

Scenario: A Lambda-based API Gateway endpoint returns 429 Too Many Requests errors during peak hours.

Remediation Steps:

1. Identify: Check 4XXError metrics in API Gateway and Throttles in Lambda.
2. Remediate:
   • Increase the Account-level Concurrency Limit via AWS Support if needed.
   • Implement Reserved Concurrency for that specific function to ensure it always has capacity.
   • Use API Gateway Caching to reduce the number of calls hitting the Lambda function.

Checkpoint Questions

1. What is the primary difference between Step Scaling and Target Tracking?
2. Why is SQS Queue Depth often a better metric than CPU for scaling a background worker tier?
3. Which AWS service can automatically remediate scaling issues for DynamoDB by spreading data across multiple regions?
4. What happens to a request if a target instance is marked as 'Unhealthy' by an ALB?

▶Click to see answers

1. Target Tracking keeps the metric at a setpoint; Step Scaling allows for different scaling increments based on the magnitude of the alarm.
2. CPU might remain low if the worker is waiting on I/O, even if there is a massive backlog of work. Queue Depth directly represents the work pending.
3. DynamoDB Global Tables.
4. The ALB stops routing new traffic to that instance and redirects it to remaining healthy targets.

Muddy Points & Cross-Refs

• Step Scaling vs. Simple Scaling: Simple scaling has a mandatory cooldown before another scaling event can happen. Step scaling allows for multiple alarms to fire and "step" the capacity up without waiting for the full cooldown.
• Cooldowns vs. Warmups: Cooldowns happen after scaling in EC2; "Warmup" is used in Target Tracking to specify how long an instance takes to be ready to contribute to the metrics.
• Cross-Ref: See Unit 4: Monitoring and Logging for details on setting up the CloudWatch agent for Memory utilization (which is not a default metric).

Comparison Tables

ECS Scaling Options