Mastering AWS Network Monitoring: CloudWatch, VPC Flow Logs, and Traffic Mirroring

This study guide covers the essential tools used to collect and analyze logs and metrics within AWS, specifically tailored for the Advanced Networking Specialty (ANS-C01) exam. We explore the hierarchy of observability, from high-level metrics to deep-packet inspection.

Learning Objectives

After studying this guide, you should be able to:

• Distinguish between metadata-level logging (VPC Flow Logs) and packet-level capture (VPC Traffic Mirroring).
• Configure VPC Traffic Mirroring components including targets, filters, and sessions.
• Analyze network connectivity and performance using CloudWatch Logs and Metrics.
• Identify the appropriate tool for troubleshooting specific network issues (e.g., packet loss, routing loops, or unauthorized access).
• Leverage AWS Reachability Analyzer and Transit Gateway Network Manager for path validation.

Key Terms & Glossary

• CloudWatch Logs: A service that centralizes logs from various AWS resources (EC2, Lambda, VPC Flow Logs) for searching, filtering, and long-term storage.
• VPC Flow Logs: A feature that enables you to capture information about the IP traffic moving to and from network interfaces in your VPC.
• Traffic Mirroring: An Amazon VPC feature that allows you to copy network traffic from an ENI of an EC2 instance and send it to security and monitoring appliances.
• ENI (Elastic Network Interface): A logical networking component in a VPC that represents a virtual network card.
• CloudTrail: A service that records API calls made within an AWS account for auditing and compliance.
• Reachability Analyzer: A static configuration analysis tool that enables you to perform connectivity testing between a source and destination.

The "Big Idea"

In AWS Networking, observability follows a depth-of-data hierarchy. High-level health is monitored via CloudWatch Metrics (the "what"); traffic patterns and connectivity outcomes are captured via VPC Flow Logs (the "who/where"); and the granular contents of the packets themselves are accessed via Traffic Mirroring (the "how"). Understanding which level of depth is required for a given troubleshooting or security scenario is the key to passing the ANS-C01 exam.

Formula / Concept Box

Hierarchical Outline

1. CloudWatch: The Observability Foundation
   • Metrics: Real-time monitoring of bandwidth, packet loss, and latency.
   • Logs: Aggregates data from VPC Flow Logs, Route 53, and Load Balancers.
   • Alarms: Automated actions based on metric thresholds (e.g., SNS notifications).
2. VPC Flow Logs: Connectivity Metadata
   • Scope: Can be enabled at the VPC, Subnet, or ENI level.
   • Key Fields: Source/Destination IP, Source/Destination Port, Protocol, Action (Accept/Reject).
   • Destinations: S3 (cost-effective storage), CloudWatch Logs (real-time analysis), Kinesis Firehose.
3. VPC Traffic Mirroring: Deep Inspection
   • Components:
     • Mirror Source: The ENI to be monitored.
     • Mirror Target: An ENI, NLB (Network Load Balancer), or GWLB (Gateway Load Balancer).
     • Mirror Filter: Rules defining which traffic to copy (Inbound/Outbound, Subnets, Ports).
     • Mirror Session: Binds the source, target, and filter with a priority level.
4. Specialized Troubleshooting Tools
   • Reachability Analyzer: Tests pathing without sending actual packets.
   • Transit Gateway Network Manager: Global view of hybrid network topology.

Visual Anchors

Tool Selection Logic

Traffic Mirroring Architecture

\begin{tikzpicture}[node distance=2cm, every node/.style={font=\small}] % Define styles \tikzstyle{box} = [rectangle, draw, rounded corners, minimum width=2.5cm, minimum height=1cm, align=center] \tikzstyle{arrow} = [thick,->,>=stealth]

\end{tikzpicture}

Definition-Example Pairs

• VPC Flow Log Action: The result of security evaluation for a packet.
  • Example: A REJECT action in a flow log entry for destination port 22 indicates that either a Security Group or a Network ACL is blocking SSH traffic.
• Mirror Filter: A set of criteria that determines which packets are copied.
  • Example: Creating a filter to only mirror traffic from a web server on port 443 allows a security appliance to inspect HTTPS handshake issues without being overwhelmed by other traffic (like database or SSH flows).
• CloudWatch Alarm: A mechanism to trigger actions based on metric deviations.
  • Example: Setting an alarm on the HealthyHostCount metric for a Load Balancer to trigger an Auto Scaling event when the number of backends drops below three.

Worked Examples

Scenario: Troubleshooting a "Connection Refused" error between two EC2 instances.

1. Step 1: Check VPC Flow Logs.
   • Identify the source and destination IP. Search the logs in CloudWatch Insights.
   • Observation: You see an entry: 2 123456789012 eni-1235b 10.0.1.5 10.0.2.10 443 22 6 20 1000 1601234567 1601234627 REJECT OK.
   • Analysis: The REJECT confirms the network security layer is dropping the traffic.
2. Step 2: Run Reachability Analyzer.
   • Source: Instance A, Destination: Instance B, Port: 443.
   • Result: Analyzer reports "Not Reachable" and points to the specific Security Group missing an inbound rule for 443.
3. Step 3: Remediation.
   • Update the Security Group inbound rules. Re-run Reachability Analyzer to confirm success.

Checkpoint Questions

1. Which service would you use to find out which IAM user deleted a Transit Gateway route table?
2. True or False: VPC Traffic Mirroring can send mirrored traffic to an S3 bucket directly.
3. In a VPC Flow Log, what does a REJECT action typically signify?
4. What is the main advantage of Reachability Analyzer over traditional ping or traceroute?

▶Click to see answers

1. CloudTrail (it tracks API calls).
2. False. Traffic Mirroring must go to a Network Interface (ENI), NLB, or GWLB. You can then use those to process and save to S3 if needed.
3. It signifies traffic was blocked by either a Security Group or a Network ACL.
4. It is a logical test that considers configuration (route tables, SGs, ACLs) without requiring the instances to be running or for actual traffic to be sent, avoiding "silent drops" in the wire.

Muddy Points & Cross-Refs

• Flow Logs vs. Traffic Mirroring: Students often confuse these. Remember: Flow Logs = Record of the Conversation (Metadata), Traffic Mirroring = Recording the actual speech (Packet capture).
• CloudWatch Metrics Latency: Standard metrics have a 1-minute or 5-minute resolution. For sub-minute visibility, you must use High-Resolution Metrics (1-second intervals).
• Cross-Reference: For security-specific analysis of these logs, refer to the AWS GuardDuty and AWS Security Hub modules.

Comparison Tables