Lab: Validating and Auditing Network Security with AWS Logging Services

This hands-on lab guides you through the process of implementing a robust network auditing strategy using VPC Flow Logs, Amazon CloudWatch, and Logs Insights. You will simulate unauthorized traffic and build an automated alerting mechanism to detect security anomalies.

Prerequisites

AWS Account: An active AWS account with permissions to manage VPCs, EC2, IAM, and CloudWatch.
AWS CLI: Configured with credentials for a user with AdministratorAccess (for lab purposes).
Basic Networking Knowledge: Understanding of IP addresses, ports (specifically TCP 22 for SSH), and Security Groups.
Region Selection: Use <YOUR_REGION> (e.g., us-east-1) consistently throughout the lab.

Learning Objectives

By the end of this lab, you will be able to:

Configure VPC Flow Logs to capture all IP traffic (Accept/Reject) for a specific subnet.
Audit Security Groups by analyzing log patterns using CloudWatch Logs Insights.
Automate Threat Detection by creating metric filters and CloudWatch Alarms for rejected connection attempts.
Validate Compliance through real-time monitoring of network interface activity.

Architecture Overview

This lab implements a flow-based monitoring pipeline. Traffic hitting an Elastic Network Interface (ENI) generates log entries that are shipped to CloudWatch for analysis and alerting.

Loading Diagram...

Step-by-Step Instructions

Step 1: Create a CloudWatch Log Group

Before enabling flow logs, we need a destination to store the captured data.

bash

aws logs create-log-group --log-group-name "/aws/vpc/flowlogs-audit"

▶Console alternative

Navigate to CloudWatch > Logs > Log groups.
Click Create log group.
Log group name: /aws/vpc/flowlogs-audit.
Click Create.

Step 2: Set Up IAM Permissions for Flow Logs

VPC Flow Logs require an IAM role to deliver logs to CloudWatch.

Create a file named trust-policy.json:
json
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "vpc-flow-logs.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
Create the role and attach the logging policy:
bash
aws iam create-role --role-name "FlowLogsRole" --assume-role-policy-document file://trust-policy.json aws iam put-role-policy --role-name "FlowLogsRole" --policy-name "FlowLogsPolicy" --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents","logs:DescribeLogGroups","logs:DescribeLogStreams"],"Resource":"*"}]}'

Step 3: Enable VPC Flow Logs

We will enable logging for your default VPC to capture all traffic, focusing on REJECT records.

bash

# Replace <VPC_ID> with your default VPC ID and <ROLE_ARN> with the FlowLogsRole ARN
aws ec2 create-flow-logs \
    --resource-type VPC \
    --resource-ids <VPC_ID> \
    --traffic-type ALL \
    --log-group-name "/aws/vpc/flowlogs-audit" \
    --deliver-logs-permission-arn <ROLE_ARN>

[!TIP] In a production environment, you might only log "REJECT" traffic to save costs, but for auditing, "ALL" is preferred to correlate successful connections with unauthorized ones.

Step 4: Generate "Suspicious" Traffic

To audit the logs, we need to generate data. We will attempt to SSH into an instance that has a Security Group blocking port 22.

Identify an EC2 instance in your VPC or launch a t2.micro.
Ensure the Security Group DOES NOT allow SSH from your IP.
Attempt to connect from your local terminal:
bash
ssh -o ConnectTimeout=5 ec2-user@<INSTANCE_PUBLIC_IP>
(The connection should hang/timeout).

Checkpoints

Log Delivery Check: In the CloudWatch Console, open the /aws/vpc/flowlogs-audit log group. Do you see log streams appearing (usually takes 2-5 minutes)?
Field Validation: Open a log stream. Verify you see fields like srcaddr, dstaddr, dstport, and action (e.g., REJECT).

Step 5: Audit with CloudWatch Logs Insights

Now we will query the logs to find exactly who is trying to access our network on port 22 but is being rejected.

Navigate to CloudWatch > Logs Insights.
Select the log group /aws/vpc/flowlogs-audit.
Run the following query:

sql

filter dstPort = 22 and action = "REJECT"
| stats count(*) as rejectCount by srcAddr
| sort rejectCount desc

[!NOTE] This query identifies the top IP addresses attempting unauthorized SSH access, a critical metric for security auditing.

Step 6: Automate Alerting

Create a Metric Filter to count the number of REJECTs on port 22 and trigger an alarm if it exceeds 5 in a minute.

bash

# Create Metric Filter
aws logs put-metric-filter \
    --log-group-name "/aws/vpc/flowlogs-audit" \
    --filter-name "SSHRejects" \
    --filter-pattern "[version, accountid, interfaceid, srcaddr, dstaddr, srcport, dstport=22, protocol, packets, bytes, start, end, action=REJECT, logstatus]" \
    --metric-transformations metricName="SSHRejectCount",metricNamespace="NetworkAudit",metricValue="1"

Teardown

[!WARNING] Failure to delete these resources may result in ongoing charges for CloudWatch storage and flow log processing.

Delete Flow Logs:
bash
# List flow logs to find ID aws ec2 describe-flow-logs aws ec2 delete-flow-logs --flow-log-ids <FLOW_LOG_ID>
Delete Log Group:
bash
aws logs delete-log-group --log-group-name "/aws/vpc/flowlogs-audit"
Delete IAM Role:
bash
aws iam delete-role-policy --role-name "FlowLogsRole" --policy-name "FlowLogsPolicy" aws iam delete-role --role-name "FlowLogsRole"

Troubleshooting

Problem	Likely Cause	Solution
No logs in CloudWatch	IAM Role lacks permissions	Verify `FlowLogsPolicy` includes `logs:PutLogEvents`.
Logs appear but are empty	No traffic activity	Run a ping or curl command from/to the instance to generate IP packets.
Query returns no results	Log format mismatch	Check if you are using the default flow log format or a custom one (this lab assumes default).

Concept Review

Service	Role in Security Audit	Comparison
VPC Flow Logs	Captures 5-tuple data of network traffic.	Similar to NetFlow/IPFIX.
CloudTrail	Captures API activity (Who changed a Security Group?).	Focuses on Management Plane vs. Data Plane.
Traffic Mirroring	Deep Packet Inspection (DPI).	More resource-intensive than Flow Logs; captures full payload.
CloudWatch Insights	Interactive log analysis.	Faster than Athena for small-to-medium datasets.

Stretch Challenge

Visualizing Anomalies: Use the "Visualize" tab in Logs Insights to create a bar chart of REJECT vs. ACCEPT traffic over time using this query:

sql

stats count(*) by bin(1m), action

Try to match the pattern shown in the conceptual graph below:

Compiling TikZ diagram…

⏳

Running TeX engine…

This may take a few seconds

Cost Estimate

Resource	Estimated Cost (US-East-1)
VPC Flow Logs	$0.25 per GB of data processed (First 5GB/month free).
CloudWatch Logs	$0.50 per GB ingested; $0.03 per GB stored.
EC2 (t2.micro)	Free Tier eligible ($0.0116/hr otherwise).
Total for Lab	~$0.05 (if completed in 30 mins and cleaned up).