Incident Response Automation: Containing a Compromised EC2 Instance
Respond to security events
Lab: Incident Response Automation
Prerequisites
Before starting this lab, ensure you have the following:
- An AWS Account with administrative privileges.
- AWS CLI installed and configured on your local machine.
- Basic knowledge of IAM roles, Amazon EC2, and Amazon EventBridge.
- A region selected (e.g.,
us-east-1).
[!WARNING] This lab involves creating resources that may incur costs if not deleted. Ensure you follow the Teardown section at the end.
Learning Objectives
By the end of this lab, you will be able to:
- Isolate a compromised EC2 instance by programmatically modifying its Security Groups.
- Capture forensic evidence by automating the creation of EBS snapshots.
- Implement automated remediation using Amazon EventBridge and AWS Lambda based on Amazon GuardDuty findings.
Architecture Overview
This lab implements a "Defense-in-Depth" strategy. When a security threat is detected, we automate the containment and evidence collection phases of the Incident Response (IR) lifecycle.
Step-by-Step Instructions
Step 1: Create a Quarantine Security Group
We need a Security Group that has no ingress or egress rules. This will act as a "network jail" for the affected instance.
# Create the Security Group
aws ec2 create-security-group \
--group-name "Quarantine-SG" \
--description "Isolated security group with no traffic allowed" \
--vpc-id <YOUR_VPC_ID>▶Console alternative
- Navigate to VPC Console > Security Groups.
- Click Create security group.
- Name:
Quarantine-SG. - Description:
No traffic allowed. - Remove all default Inbound and Outbound rules.
- Click Create.
Step 2: Create the Remediation Lambda Function
This function will perform two tasks: detach existing security groups (replacing them with the Quarantine SG) and take a snapshot of the root volume.
# Lambda Logic Snippet (Python 3.12)
import boto3
ec2 = boto3.client('ec2')
def lambda_handler(event, context):
# Extract Instance ID from GuardDuty finding
instance_id = event['detail']['resource']['instanceDetails']['instanceId']
# 1. Containment: Swap SG to Quarantine-SG
ec2.modify_instance_attribute(
InstanceId=instance_id,
Groups=['<QUARANTINE_SG_ID>']
)
# 2. Forensics: Snapshot all volumes
volumes = ec2.describe_volumes(Filters=[{'Name': 'attachment.instance-id', 'Values': [instance_id]}])
for vol in volumes['Volumes']:
ec2.create_snapshot(
VolumeId=vol['VolumeId'],
Description=f"Forensic snapshot for {instance_id} - IR Event"
)
return {"status": "contained", "instance": instance_id}[!IMPORTANT] Ensure the Lambda IAM Role has
ec2:ModifyInstanceAttribute,ec2:DescribeVolumes, andec2:CreateSnapshotpermissions.
Step 3: Configure Amazon EventBridge
We must link GuardDuty findings to our Lambda function.
# Create the Rule
aws events put-rule \
--name "GuardDutyToRemediation" \
--event-pattern '{"source": ["aws.guardduty"], "detail-type": ["GuardDuty Finding"]}'
# Add Lambda as Target
aws events put-targets \
--rule "GuardDutyToRemediation" \
--targets "Id"="1","Arn"="<LAMBDA_FUNCTION_ARN>"▶Console alternative
- Navigate to Amazon EventBridge > Rules.
- Click Create rule. Name:
GuardDutyToRemediation. - Event pattern: Data source
AWS services, ServiceGuardDuty, Event typeGuardDuty Finding. - Select Lambda function as the target and choose your function.
Step 4: Simulate a GuardDuty Finding
To test our automation without actually compromising a server, we can use GuardDuty's sample findings.
# Generate sample findings
aws guardduty create-sample-findings \
--detector-id <YOUR_DETECTOR_ID> \
--finding-types "Backdoor:EC2/C&CActivity.B!dns"Checkpoints
| Verification Step | Expected Result |
|---|---|
| Check EC2 Networking | The test instance should now only be associated with Quarantine-SG. |
| Check EBS Snapshots | A new snapshot should exist with the description "Forensic snapshot...". |
| Lambda Logs | CloudWatch Logs for the function should show a success message for the specific Instance ID. |
Troubleshooting
| Issue | Cause | Fix |
|---|---|---|
| Lambda Access Denied | Missing IAM permissions. | Add ec2:ModifyInstanceAttribute to the Lambda execution role. |
| Rule not firing | GuardDuty is disabled. | Ensure GuardDuty is Enabled in the specific region you are testing. |
| SG swap fails | Instance has multiple ENIs. | The script assumes a single ENI; modify the script to loop through NetworkInterfaces. |
Clean-Up / Teardown
[!WARNING] Failure to delete these resources will result in continued charges for EBS snapshots and potential EC2 usage.
- Delete EventBridge Rule:
bash
aws events remove-targets --rule "GuardDutyToRemediation" --ids "1" aws events delete-rule --name "GuardDutyToRemediation" - Delete Snapshots:
bash
aws ec2 delete-snapshot --snapshot-id <SNAPSHOT_ID> - Delete Lambda Function:
bash
aws lambda delete-function --function-name <FUNCTION_NAME> - Remove Security Group:
bash
aws ec2 delete-security-group --group-id <QUARANTINE_SG_ID>
Stretch Challenge
Automated Forensic Forensics: Modify the Lambda function to use AWS Systems Manager (SSM) to run a script on the instance (before isolation) that captures volatile memory (RAM) and uploads it to an S3 bucket for deeper analysis.