AWS CloudWatch Agent: Collecting Custom Metrics

This study guide covers the configuration and deployment of the CloudWatch agent to collect system-level and application-level metrics, a critical skill for the AWS Certified DevOps Engineer – Professional (DOP-C02) exam.

Learning Objectives

After studying this guide, you will be able to:

• Differentiate between standard metrics and custom metrics.
• Identify the specific system-level metrics only available via the CloudWatch Agent.
• Choose the appropriate installation method (CLI, SSM, or CloudFormation).
• Configure the agent to ingest application metrics using StatsD and collectd.
• Compare the CloudWatch Agent with the SSM Agent and custom scripts.

Key Terms & Glossary

• Custom Metric: Any metric not automatically collected by AWS services (e.g., Memory utilization). These are billed per metric per month.
• StatsD: An open-source network daemon for aggregating and pushing application metrics, supported by the CloudWatch Agent on Linux and Windows.
• collectd: A Unix daemon that collects system performance statistics, supported by the CloudWatch Agent on Linux only.
• Namespace: A container for CloudWatch metrics (e.g., CWAgent is the default for the agent).
• Dimension: A name/value pair that is part of a metric's identity (e.g., InstanceId or InstanceType).

The "Big Idea"

By default, CloudWatch only sees what the hypervisor sees (CPU, Network In/Out, Disk Metadata). It cannot see inside the Operating System. The CloudWatch Agent acts as a bridge, granting visibility into the "black box" of the OS (Memory, Disk Space) and the internal performance of your applications. This data is essential for high-fidelity auto-scaling and root cause analysis.

Formula / Concept Box

Hierarchical Outline

1. Metric Sources
   • Hypervisor Metrics: CPU, Network, Status Checks (Automatic/Free).
   • Agent Metrics: Memory utilization, Disk space used, Swap usage (Custom/Paid).
   • Application Metrics: Custom logs and telemetry via StatsD/collectd.
2. The CloudWatch Agent
   • Function: Collects system-level metrics + logs.
   • Installation:
     • CLI: Manual installation for one-off instances.
     • SSM: Recommended for fleet-wide management using the AWS-ConfigureAWSPackage document.
     • CloudFormation: For Infrastructure-as-Code (IaC) consistency.
3. Protocols & Integration
   • StatsD: Cross-platform (Linux/Windows).
   • collectd: Linux-specific.
4. Security & Permissions
   • IAM Role: Instances require CloudWatchAgentServerPolicy to push data.

Visual Anchors

Metric Data Flow

Monitoring Visibility Layers

Definition-Example Pairs

• Standard Resolution: Metrics with 1-minute granularity.
  • Example: Default EC2 CPU Utilization metrics.
• High Resolution: Metrics with 1-second granularity.
  • Example: Monitoring a flash sale application where a 10-second spike in memory could crash the server.
• Dimension-based Filtering: Segregating metrics by specific attributes.
  • Example: Viewing memory usage specifically for InstanceID: i-123456 rather than an aggregate of the whole Auto Scaling Group.

Worked Example: Deploying via SSM

Scenario: You need to install the CloudWatch agent on 50 EC2 instances across two regions.

1. IAM Setup: Attach the CloudWatchAgentServerPolicy to the IAM Role used by the EC2 instances.
2. Parameter Store: Save the agent configuration (JSON) in the SSM Parameter Store (e.g., name it AmazonCloudWatch-linux).
3. Installation: Use SSM Run Command with the AWS-ConfigureAWSPackage document. Set the action to Install and name to AmazonCloudWatchAgent.
4. Configuration: Use SSM Run Command with the AmazonCloudWatch-ManageAgent document. Point to the Parameter Store name created in Step 2.
5. Verification: Check the CloudWatch Console under the CWAgent namespace to see incoming data.

Checkpoint Questions

1. Which protocol is supported for custom application metrics on Windows servers using the CloudWatch Agent?
2. What IAM policy is required for an EC2 instance to send metrics to CloudWatch via the agent?
3. True/False: Standard metrics (like CPUUtilization) provided by the hypervisor are billed as custom metrics.
4. How does the CloudWatch Agent handle log files?

▶Click to reveal answers

1. StatsD (collectd is Linux-only).
2. CloudWatchAgentServerPolicy.
3. False. Only metrics sent by the agent or custom API calls are "Custom Metrics".
4. It can tail log files and send them to CloudWatch Logs groups.

Muddy Points & Cross-Refs

• CW Agent vs. SSM Agent: People often confuse these. The SSM Agent is for management (running commands, patching), while the CW Agent is for telemetry (metrics, logs). You use the SSM Agent to install the CW Agent.
• Billing Trap: Each unique combination of dimensions counts as a separate custom metric. Be careful when adding dimensions like ProcessID, as it can lead to thousands of billed metrics.
• Cross-Ref: See Exam Domain IV: Monitoring and Logging for more on Log Subscriptions and Metric Filters.

Comparison Tables

Metric Collection Methods