Securely Managing Application Secrets with AWS Secrets Manager

Hardcoding credentials in application code is a major security risk. If source code is leaked or a server is compromised, your database and API credentials are exposed in plain text. This lab teaches you how to use AWS Secrets Manager to externalize sensitive data, allowing your application to retrieve credentials dynamically at runtime.

Prerequisites

AWS Account: An active account with permissions to manage IAM and Secrets Manager.
AWS CLI: Configured with credentials (aws configure).
Python 3.x: Installed on your local machine to run the SDK retrieval script.
Boto3: The AWS SDK for Python (pip install boto3).
IAM Permissions: You need secretsmanager:CreateSecret, secretsmanager:GetSecretValue, and secretsmanager:DeleteSecret permissions.

Learning Objectives

Create and store a sensitive secret (JSON) in AWS Secrets Manager.
Understand the relationship between Secrets Manager and AWS KMS for encryption.
Implement programmatic secret retrieval using the AWS SDK (Boto3).
Configure IAM policies to allow fine-grained access to specific secrets.

Architecture Overview

In this architecture, the application never stores the database password. Instead, it uses its IAM identity to request the secret value at runtime. Secrets Manager interacts with AWS KMS to decrypt the data before returning it over an encrypted HTTPS connection.

Loading Diagram...

Step-by-Step Instructions

Step 1: Create the Secret

First, we will store a mock database credential in Secrets Manager. We will use a JSON string to store both a username and a password.

bash

aws secretsmanager create-secret \
    --name "brainybee/lab/db-creds" \
    --description "Database credentials for the inventory app" \
    --secret-string '{"username":"admin","password":"P@ssw0rd2024!"}'

▶Console alternative

Log in to the AWS Management Console.
Navigate to Secrets Manager.
Click Store a new secret.
Select Other type of secret.
Enter Key: username, Value: admin. Click Add row.
Enter Key: password, Value: P@ssw0rd2024!.
Click Next, name it brainybee/lab/db-creds, and click Store.

Step 2: Verify Secret Metadata

Ensure the secret was created successfully and note the Amazon Resource Name (ARN).

bash

aws secretsmanager list-secrets --filters Key="name",Values="brainybee/lab/db-creds"

[!TIP] In a production environment, you would use a Customer Managed Key (CMK) in KMS for better control, but by default, Secrets Manager uses the AWS-managed key aws/secretsmanager.

Step 3: Retrieve the Secret Programmatically

Now, we will write a small Python script to simulate how an application would fetch these credentials. Create a file named get_secret.py:

python

import boto3
import json
from botocore.exceptions import ClientError

def get_secret():
    secret_name = "brainybee/lab/db-creds"
    region_name = "us-east-1" # Change to your region

    # Create a Secrets Manager client
    session = boto3.session.Session()
    client = session.client(service_name='secretsmanager', region_name=region_name)

    try:
        get_secret_value_response = client.get_secret_value(SecretId=secret_name)
    except ClientError as e:
        raise e

    # Decrypts secret using the associated KMS key
    secret = get_secret_value_response['SecretString']
    return json.loads(secret)

if __name__ == "__main__":
    creds = get_secret()
    print(f"Successfully retrieved username: {creds['username']}")
    # In a real app, you'd use creds['password'] to connect to the DB

Run the script:

bash

python3 get_secret.py

Step 4: Implement a Resource-Based Policy (Optional)

You can attach a policy directly to the secret to restrict who can read it.

bash

# Example: Viewing the current policy (usually empty by default)
aws secretsmanager get-resource-policy --secret-id brainybee/lab/db-creds

Checkpoints

Checkpoint	Action	Expected Result
Creation	Run `aws secretsmanager list-secrets`	You see `brainybee/lab/db-creds` in the list.
Encryption	Check Console > Secrets Manager	The secret value is hidden/masked until you click "Retrieve secret value".
SDK Access	Run `python3 get_secret.py`	Terminal prints "Successfully retrieved username: admin".

Clean-Up / Teardown

[!WARNING] Secrets Manager has a cost per secret per month. Always delete lab secrets immediately after use.

By default, Secrets Manager enforces a recovery window (7-30 days). To delete it immediately for this lab, use --force-delete-without-recovery.

bash

aws secretsmanager delete-secret \
    --secret-id brainybee/lab/db-creds \
    --force-delete-without-recovery

Troubleshooting

Error	Cause	Fix
`AccessDeniedException`	IAM user/role lacks `GetSecretValue`	Attach an IAM policy with `secretsmanager:GetSecretValue` for this ARN.
`ResourceNotFoundException`	Name mismatch or wrong region	Ensure the CLI region matches the region where the secret was created.
`ModuleNotFoundError`	Boto3 not installed	Run `pip install boto3`.

Stretch Challenge

Environment Variable Encryption: Instead of calling the API every time, some developers store the Secret ARN in an environment variable. Can you modify your Python script to read the secret name from an environment variable using os.environ.get('SECRET_ARN')?

▶Show Hint

python

import os
secret_name = os.environ.get('MY_APP_SECRET_NAME', 'default/path')

Cost Estimate

Secrets Manager: $0.40 per secret per month (pro-rated). For this 30-minute lab, the cost is effectively <$0.01.
KMS: The first 20,000 requests to KMS per month are free under the Free Tier.
API Calls: $0.05 per 10,000 API calls.

Concept Review

Secrets Manager vs. Parameter Store

Feature	Secrets Manager	Systems Manager Parameter Store
Primary Use	Sensitive secrets (DB, API Keys)	Configuration (URLs, feature flags)
Automatic Rotation	Native support for RDS, Redshift	Requires custom Lambda logic
Encryption	Required (KMS)	Optional (SecureString)
Cost	Paid per secret	Standard parameters are free

Encryption Mechanics

Secrets Manager uses Envelope Encryption. The data is encrypted with a unique data key, and that data key is encrypted with the KMS Master Key.

\begin{tikzpicture}[node distance=2cm] \draw[thick, fill=blue!5] (0,0) rectangle (4,1.5) node[midway] {Plaintext Secret}; \draw[->, thick] (4.2, 0.75) -- (5.8, 0.75) node[midway, above] {Encrypt}; \draw[thick, fill=red!5] (6,0) rectangle (10,1.5) node[midway] {Encrypted Data}; \node at (5, -1) {\textbf{KMS Key Protection Layer}}; \draw[dashed] (-0.5, -1.5) rectangle (10.5, 2.5); \end{tikzpicture}