Curriculum Overview: Log Search and Correlation for Security Events

This curriculum provides a structured pathway to mastering the identification, aggregation, and analysis of security logs across AWS environments and applications. It is specifically designed to align with the AWS Certified Security - Specialty (SCS-C03) exam objectives, focusing on Content Domain 1 (Detection) and Domain 2 (Incident Response).

Prerequisites

Before starting this curriculum, learners should possess the following foundational knowledge and access:

• AWS Core Services: Familiarity with Amazon S3, EC2, and IAM (Identity and Access Management) permissions.
• Security Fundamentals: Understanding of the "Shared Responsibility Model" and common security threats (e.g., OWASP Top 10).
• Data Formats: Basic ability to read and parse JSON (the primary format for AWS logs).
• Querying Basics: Introductory knowledge of SQL (Structured Query Language) for use with Amazon Athena.
• Tools: Access to an AWS Account and the AWS CLI (Command Line Interface).

Module Breakdown

Learning Objectives per Module

Module 1: Establishing the Log Baseline

• Configure AWS CloudTrail for organizational-wide API auditing.
• Enable and interpret VPC Flow Logs to identify unauthorized network traffic patterns.
• Deploy the CloudWatch Logs Agent to capture application-level logs from EC2 instances.

Module 2: Building the Data Lake

• Design secure S3 buckets for long-term log retention with Object Lock and Versioning.
• Implement Amazon Security Lake to automatically centralize security data from cloud and on-premises sources into the OCSF format.

Module 3: On-Demand Querying

• Execute high-speed queries using CloudWatch Logs Insights to filter security events.
• Use Amazon Athena to run SQL queries against logs stored in S3 without the need for complex ETL (Extract, Transform, Load).

Module 4: Advanced Correlation

• Normalize and parse disparate log formats using AWS Lambda.
• Set up Amazon OpenSearch Service for real-time indexing and visual graphing of security anomalies.
• Use Kinesis Data Firehose to stream logs into third-party SIEM (Security Information and Event Management) tools.

Module 5: Deep-Dive Investigations

• Utilize Amazon Detective to conduct root-cause analysis by visualizing relationships between users, IP addresses, and AWS resources.
• Aggregate and prioritize findings using AWS Security Hub.

Visual Pipeline

Success Metrics

To demonstrate mastery of this curriculum, the learner must be able to:

1. Reduce Noise: Successfully filter out "normal" traffic from security logs to identify a single malicious API call within a 24-hour window.
2. Cross-Service Correlation: Trace a security event from a WAF block through to the specific IAM user or IP address responsible in CloudTrail.
3. Automation: Implement an EventBridge rule that triggers a Lambda function to isolate a resource based on a specific log pattern.
4. Forensic Integrity: Prove that logs are stored in a non-repudiable manner using S3 Glacier Vault Lock.

Real-World Application

In a professional setting, these skills transition directly into the role of a Security Operations Center (SOC) Analyst or Security Engineer.

[!IMPORTANT] Log correlation isn't just about storage; it's about context. For example, a single failed login is a non-event, but 500 failed logins from a single IP followed by a successful login and an S3 bucket policy change is a critical incident.

Common Use Cases:

• Incident Response: Rapidly identifying the "Blast Radius" of a compromised IAM credential.
• Compliance Auditing: Providing auditors with immutable evidence of who accessed sensitive data and when.
• Threat Hunting: Proactively searching OpenSearch dashboards for unusual traffic spikes that bypass automated alerts.

Comparison of Search Tools