Lab: Automating Event-Driven Monitoring and Remediation

[!WARNING] Remember to run the teardown commands at the end of this lab to avoid ongoing charges. Most resources used here are Free Tier eligible, but costs can accumulate if left running.

Prerequisites

Before starting this lab, ensure you have:

An AWS Account with administrative access.
AWS CLI installed and configured locally with <YOUR_PROFILE>.
Basic knowledge of Python (for the Lambda remediation script).
A verified email address to receive SNS notifications.

Learning Objectives

By the end of this lab, you will be able to:

Configure an Amazon SNS topic for real-time alerting.
Develop an AWS Lambda function to programmatically remediate unauthorized security group changes.
Create an Amazon EventBridge rule to detect specific API calls via CloudTrail and trigger automation.
Implement an event-driven, asynchronous design pattern for system governance.

Architecture Overview

This lab implements a self-healing pattern. When a user creates a "wide-open" (0.0.0.0/0) security group rule, EventBridge detects the event and triggers a Lambda function to revoke the rule and notify the security team via SNS.

Loading Diagram...

Figure 1 — Mermaid diagram

Step-by-Step Instructions

We need a communication channel to notify us when a remediation event occurs.

bash

aws sns create-topic --name brainybee-security-alerts

▶Console alternative

Navigate to SNS > Topics.
Click Create topic.
Type: Standard. Name: brainybee-security-alerts.
Click Create topic.

Replace <YOUR_EMAIL> with your actual email address.

bash

aws sns subscribe \
    --topic-arn arn:aws:sns:<YOUR_REGION>:<YOUR_ACCOUNT_ID>:brainybee-security-alerts \
    --protocol email \
    --notification-endpoint <YOUR_EMAIL>

[!IMPORTANT] You must click the "Confirm Subscription" link in the email sent by AWS before you can receive alerts.

Step 3: Create the Remediation Lambda Function

This function will parse the EventBridge event, identify the security group, and revoke any rules that allow 0.0.0.0/0.

First, create a file named lambda_function.py:

python

import boto3

ec2 = boto3.client('ec2')

def lambda_handler(event, context):
    group_id = event['detail']['requestParameters']['groupId']
    items = event['detail']['requestParameters']['ipPermissions']['items']
    
    for item in items:
        for ip_range in item.get('ipRanges', {}).get('items', []):
            if ip_range.get('cidrIp') == '0.0.0.0/0':
                print(f"Remediating group {group_id} - Revoking 0.0.0.0/0")
                ec2.revoke_security_group_ingress(
                    GroupId=group_id,
                    IpPermissions=[item]
                )
    return {'status': 'remediated'}

Zip the file and deploy via CLI (ensure your IAM role has ec2:RevokeSecurityGroupIngress permissions):

bash

zip function.zip lambda_function.py
aws lambda create-function --function-name SecurityGroupRemediator \
    --zip-file fileb://function.zip --handler lambda_function.lambda_handler \
    --runtime python3.9 --role <YOUR_LAMBDA_EXECUTION_ROLE_ARN>

Step 4: Configure the EventBridge Rule

We will trigger our automation whenever a AuthorizeSecurityGroupIngress event is recorded.

bash

aws events put-rule --name DetectWideOpenSG \
    --event-pattern '{"source":["aws.ec2"],"detail-type":["AWS API Call via CloudTrail"],"detail":{"eventSource":["ec2.amazonaws.com"],"eventName":["AuthorizeSecurityGroupIngress"]}}'

Connect the Lambda function as a target:

bash

aws events put-targets --rule DetectWideOpenSG \
    --targets "Id"="1","Arn"="arn:aws:lambda:<YOUR_REGION>:<YOUR_ACCOUNT_ID>:function:SecurityGroupRemediator"

Checkpoints

Verification: Go to the EC2 console and create a security group rule allowing 0.0.0.0/0 on port 22.
Observation: Wait 1-2 minutes for CloudTrail to process the API call.
Result: Refresh the security group page. The rule should be gone.
Logs: Check CloudWatch Logs for /aws/lambda/SecurityGroupRemediator to see the execution details.

Troubleshooting

Issue	Likely Cause	Solution
Lambda doesn't trigger	CloudTrail Delay	CloudTrail can take up to 15 mins to deliver events to EventBridge in some regions.
Permission Denied	Missing IAM Policy	Ensure Lambda role has `ec2:RevokeSecurityGroupIngress` and `logs:CreateLogGroup`.
SNS No Email	Unconfirmed Sub	Check your spam folder for the AWS Subscription Confirmation email.

Clean-Up / Teardown

Run these commands to remove lab resources:

bash

# Delete EventBridge Rule Targets
aws events remove-targets --rule DetectWideOpenSG --ids "1"

# Delete EventBridge Rule
aws events delete-rule --name DetectWideOpenSG

# Delete Lambda Function
aws lambda delete-function --function-name SecurityGroupRemediator

# Delete SNS Topic
aws sns delete-topic --topic-arn arn:aws:sns:<YOUR_REGION>:<YOUR_ACCOUNT_ID>:brainybee-security-alerts

Stretch Challenge

Difficulty: Challenge Instead of a custom Lambda function, use AWS Config with a managed rule vpc-sg-open-only-to-authorized-ports and configure an SSM Automation Document as a remediation action. This is the enterprise-standard way to handle configuration drift at scale.

Cost Estimate

Service	Usage	Estimated Cost (Monthly)
EventBridge	< 1,000 events	$0.00 (Free Tier)
Lambda	< 100 requests	$0.00 (Free Tier)
SNS	< 1,000 notifications	$0.00 (Free Tier)
CloudTrail	1 Management Trail	$0.00 (First copy is free)

Concept Review

Remediation Logic Flow

Compiling TikZ diagram…

⏳

Running TeX engine…

This may take a few seconds

Figure 2 — TikZ diagram

Comparison: EventBridge vs. AWS Config

Feature	Amazon EventBridge	AWS Config
Trigger	Real-time API calls / State changes	Configuration changes / Periodic snapshots
Latency	Near real-time	Minutes to hours
Use Case	Asynchronous workflows, custom apps	Compliance, auditing, and governance
Remediation	Lambda, SSM, Step Functions	SSM Automation, Lambda

[!TIP] In a complex environment, use EventBridge for immediate operational response and AWS Config for long-term compliance reporting.

Lab: Automating Event-Driven Monitoring and Remediation

[!WARNING] Remember to run the teardown commands at the end of this lab to avoid ongoing charges. Most resources used here are Free Tier eligible, but costs can accumulate if left running.

Prerequisites

Before starting this lab, ensure you have:

An AWS Account with administrative access.
AWS CLI installed and configured locally with <YOUR_PROFILE>.
Basic knowledge of Python (for the Lambda remediation script).
A verified email address to receive SNS notifications.

Learning Objectives

By the end of this lab, you will be able to:

Configure an Amazon SNS topic for real-time alerting.
Develop an AWS Lambda function to programmatically remediate unauthorized security group changes.
Create an Amazon EventBridge rule to detect specific API calls via CloudTrail and trigger automation.
Implement an event-driven, asynchronous design pattern for system governance.

Architecture Overview

Loading Diagram...

Figure 1 — Mermaid diagram

Step-by-Step Instructions

We need a communication channel to notify us when a remediation event occurs.

bash

aws sns create-topic --name brainybee-security-alerts

▶Console alternative

Navigate to SNS > Topics.
Click Create topic.
Type: Standard. Name: brainybee-security-alerts.
Click Create topic.

Replace <YOUR_EMAIL> with your actual email address.

bash

aws sns subscribe \
    --topic-arn arn:aws:sns:<YOUR_REGION>:<YOUR_ACCOUNT_ID>:brainybee-security-alerts \
    --protocol email \
    --notification-endpoint <YOUR_EMAIL>

[!IMPORTANT] You must click the "Confirm Subscription" link in the email sent by AWS before you can receive alerts.

Step 3: Create the Remediation Lambda Function

This function will parse the EventBridge event, identify the security group, and revoke any rules that allow 0.0.0.0/0.

First, create a file named lambda_function.py:

python

import boto3

ec2 = boto3.client('ec2')

def lambda_handler(event, context):
    group_id = event['detail']['requestParameters']['groupId']
    items = event['detail']['requestParameters']['ipPermissions']['items']
    
    for item in items:
        for ip_range in item.get('ipRanges', {}).get('items', []):
            if ip_range.get('cidrIp') == '0.0.0.0/0':
                print(f"Remediating group {group_id} - Revoking 0.0.0.0/0")
                ec2.revoke_security_group_ingress(
                    GroupId=group_id,
                    IpPermissions=[item]
                )
    return {'status': 'remediated'}

Zip the file and deploy via CLI (ensure your IAM role has ec2:RevokeSecurityGroupIngress permissions):

bash

zip function.zip lambda_function.py
aws lambda create-function --function-name SecurityGroupRemediator \
    --zip-file fileb://function.zip --handler lambda_function.lambda_handler \
    --runtime python3.9 --role <YOUR_LAMBDA_EXECUTION_ROLE_ARN>

Step 4: Configure the EventBridge Rule

We will trigger our automation whenever a AuthorizeSecurityGroupIngress event is recorded.

bash

aws events put-rule --name DetectWideOpenSG \
    --event-pattern '{"source":["aws.ec2"],"detail-type":["AWS API Call via CloudTrail"],"detail":{"eventSource":["ec2.amazonaws.com"],"eventName":["AuthorizeSecurityGroupIngress"]}}'

Connect the Lambda function as a target:

bash

aws events put-targets --rule DetectWideOpenSG \
    --targets "Id"="1","Arn"="arn:aws:lambda:<YOUR_REGION>:<YOUR_ACCOUNT_ID>:function:SecurityGroupRemediator"

Checkpoints

Verification: Go to the EC2 console and create a security group rule allowing 0.0.0.0/0 on port 22.
Observation: Wait 1-2 minutes for CloudTrail to process the API call.
Result: Refresh the security group page. The rule should be gone.
Logs: Check CloudWatch Logs for /aws/lambda/SecurityGroupRemediator to see the execution details.

Troubleshooting

Issue	Likely Cause	Solution
Lambda doesn't trigger	CloudTrail Delay	CloudTrail can take up to 15 mins to deliver events to EventBridge in some regions.
Permission Denied	Missing IAM Policy	Ensure Lambda role has `ec2:RevokeSecurityGroupIngress` and `logs:CreateLogGroup`.
SNS No Email	Unconfirmed Sub	Check your spam folder for the AWS Subscription Confirmation email.

Clean-Up / Teardown

Run these commands to remove lab resources:

bash

# Delete EventBridge Rule Targets
aws events remove-targets --rule DetectWideOpenSG --ids "1"

# Delete EventBridge Rule
aws events delete-rule --name DetectWideOpenSG

# Delete Lambda Function
aws lambda delete-function --function-name SecurityGroupRemediator

# Delete SNS Topic
aws sns delete-topic --topic-arn arn:aws:sns:<YOUR_REGION>:<YOUR_ACCOUNT_ID>:brainybee-security-alerts

Stretch Challenge

Cost Estimate

Service	Usage	Estimated Cost (Monthly)
EventBridge	< 1,000 events	$0.00 (Free Tier)
Lambda	< 100 requests	$0.00 (Free Tier)
SNS	< 1,000 notifications	$0.00 (Free Tier)
CloudTrail	1 Management Trail	$0.00 (First copy is free)

Concept Review

Remediation Logic Flow

Compiling TikZ diagram…

⏳

Running TeX engine…

This may take a few seconds

Figure 2 — TikZ diagram

Comparison: EventBridge vs. AWS Config

Feature	Amazon EventBridge	AWS Config
Trigger	Real-time API calls / State changes	Configuration changes / Periodic snapshots
Latency	Near real-time	Minutes to hours
Use Case	Asynchronous workflows, custom apps	Compliance, auditing, and governance
Remediation	Lambda, SSM, Step Functions	SSM Automation, Lambda

[!TIP] In a complex environment, use EventBridge for immediate operational response and AWS Config for long-term compliance reporting.

Lab: Automating Event-Driven Monitoring and Remediation

Lab: Automating Event-Driven Monitoring and Remediation

Prerequisites

Learning Objectives

Architecture Overview

Step-by-Step Instructions

Step 1: Create an SNS Topic for Alerts

Step 2: Subscribe to the Topic

Step 3: Create the Remediation Lambda Function

Step 4: Configure the EventBridge Rule

Checkpoints

Troubleshooting

Clean-Up / Teardown

Stretch Challenge

Cost Estimate

Concept Review

Remediation Logic Flow

Comparison: EventBridge vs. AWS Config

Lab: Automating Event-Driven Monitoring and Remediation

Lab: Automating Event-Driven Monitoring and Remediation

Prerequisites

Learning Objectives

Architecture Overview

Step-by-Step Instructions

Step 1: Create an SNS Topic for Alerts

Step 2: Subscribe to the Topic

Step 3: Create the Remediation Lambda Function

Step 4: Configure the EventBridge Rule

Checkpoints

Troubleshooting

Clean-Up / Teardown

Stretch Challenge

Cost Estimate

Concept Review

Remediation Logic Flow

Comparison: EventBridge vs. AWS Config