AWS Security Log Analysis Curriculum Overview

This curriculum provides a structured path for mastering the detection and analysis of security events within the AWS ecosystem. It focuses on using native tools like CloudWatch Logs Insights, Amazon Athena, and Security Hub to transform raw log data into actionable security intelligence.

Prerequisites

Before beginning this curriculum, learners should possess the following foundational knowledge and access:

AWS Fundamentals: Deep understanding of IAM roles, policies, and cross-account access patterns.
Log Generation: Knowledge of how to enable and configure AWS CloudTrail, VPC Flow Logs, and Route 53 Resolver logs.
Data Storage: Familiarity with Amazon S3 bucket policies and CloudWatch Log Group retention settings.
Querying Basics: Basic proficiency in Structured Query Language (SQL) and regular expressions (Regex).
Tools: Access to an AWS Sandbox environment with existing log data (e.g., CloudTrail events).

Module Breakdown

Module	Topic	Difficulty	Primary Tools
1	CloudWatch Logs Insights	Intermediate	CloudWatch, Filter Patterns
2	Amazon Athena & S3 Data Lakes	Advanced	S3, Athena, SQL, Kinesis Firehose
3	Security Hub & Findings Analysis	Intermediate	Security Hub, AWS Config
4	Log Normalization & Correlation	Advanced	Lambda, OpenSearch, OCSF

Learning Objectives per Module

Module 1: CloudWatch Logs Insights

Interactive Querying: Execute ad-hoc queries against log groups using the purpose-built query syntax.
Field Discovery: Utilize automatic discovery of JSON fields and system fields (e.g., @message, @ingestionTime).
Visualization: Transform query results into time-series graphs and bar charts to identify volume-based anomalies.

Module 2: Amazon Athena & S3 Data Lakes

Serverless SQL: Create external tables in Athena to query logs stored in Amazon S3 (e.g., long-term CloudTrail storage).
Data Pipelining: Configure Kinesis Data Firehose to stream and partition logs for optimized Athena performance.
Cost Management: Apply partitioning and use Parquet/ORC formats to minimize the amount of data scanned per query.

Module 3: Security Hub & Findings Analysis

Standardized Findings: Analyze findings across multiple accounts using the AWS Security Finding Format (ASFF).
Insight Aggregation: Use built-in insights to identify the most critical resources and top-offending accounts.

Module 4: Log Normalization & Correlation

Parsing at Scale: Use AWS Lambda to normalize heterogeneous logs into the Open Cybersecurity Schema Framework (OCSF).
Cross-Source Analysis: Correlate VPC Flow Logs with CloudTrail events to identify the source and impact of a security event.

Visual Architecture

Log Analysis Pipeline

Loading Diagram...

The Detection Pillar

\begin{tikzpicture}[node distance=2cm, every node/.style={rectangle, draw, rounded corners, minimum height=1cm, text width=3cm, align=center}] \node (detect) [fill=blue!10] {\textbf{Detection}}; \node (log) [below left of=detect, xshift=-1cm, fill=green!10] {Logging \ (CloudTrail/VPC)}; \node (analyze) [below right of=detect, xshift=1cm, fill=orange!10] {Analysis \ (Athena/Insights)}; \node (finding) [below of=detect, yshift=-1.5cm, fill=red!10] {Findings \ (Security Hub)};

code

\draw[->, thick] (log) -- (analyze);
\draw[->, thick] (analyze) -- (finding);
\draw[->, thick] (detect) -- (log);
\draw[->, thick] (detect) -- (analyze);

\end{tikzpicture}

Success Metrics

Learners have mastered this curriculum when they can successfully:

Draft a CloudWatch Insights Query that identifies the top 10 users performing ConsoleLogin failures over the last 24 hours.
Configure an Athena Table that correctly parses VPC Flow Logs with custom partitions based on year, month, and day.
Perform Root Cause Analysis (RCA) by correlating an Amazon GuardDuty finding in Security Hub with specific API calls recorded in CloudTrail via Athena.
Implement a Subscription Filter that sends specific security-related log events from CloudWatch to a Lambda function for real-time alerting.

Real-World Application

In a professional setting, these skills are applied to:

Forensic Investigation: Quickly searching months of historical log data to determine the blast radius of a compromised credential.
Compliance Auditing: Generating automated reports for PCI-DSS or SOC2 by querying S3-based log archives.
Automated Remediation: Triggering AWS Step Functions or Lambda remediations based on specific patterns identified through log analysis.

[!IMPORTANT] Amazon Athena separates storage (S3) from compute. To optimize costs, always use the LIMIT clause during query development and ensure logs are partitioned by date to avoid full bucket scans.