Hands-On Lab: Implementing Transparent and Explainable Models in AWS

[!NOTE] Estimated Time: 30 minutes | Difficulty: Guided | Cloud Provider: AWS

While AI technologies offer immense benefits, they also present significant challenges regarding bias, fairness, and ethical considerations. Transparency (openness about a model's design and data sources) and Explainability (the specific reasoning for individual decisions) are critical for establishing trust in AI, especially in regulated industries like finance and healthcare.

In this guided lab, you will act as an AI Governance Engineer. You will create a structured system of record using Amazon SageMaker Model Cards to document an AI model's transparency, explainability framework (like SHAP or LIME), and risk ratings.

Prerequisites

Before starting this lab, ensure you have the following:

AWS Account: An active AWS account with billing enabled.
IAM Permissions: A user or role with AmazonSageMakerFullAccess and AmazonS3FullAccess.
CLI Tools: AWS CLI installed and configured (aws configure) with your access keys and default region.
Prior Knowledge: Basic understanding of JSON formatting and foundational Machine Learning concepts.

Learning Objectives

By the end of this lab, you will be able to:

Differentiate between Transparency (system-level openness) and Explainability (decision-level reasoning) in AI.
Provision AWS storage securely for AI governance and explainability artifacts.
Construct an Amazon SageMaker Model Card to document responsible AI metrics.
Verify compliance and model transparency via the AWS CLI and AWS Management Console.

Architecture Overview

The following diagram illustrates the workflow you will build to document model transparency:

Loading Diagram...

Concept Review: Transparency vs. Explainability

Before touching the console, let's clarify the core concepts documented in our Model Card.

Compiling TikZ diagram…

⏳

Running TeX engine…

This may take a few seconds

Real-world Example (Transparency): Apple and Goldman Sachs publishing the datasets and parameters used to train their credit card approval algorithm.
Real-world Example (Explainability): A doctor receiving an AI diagnostic report that highlights the specific pixels in an X-ray that led the model to predict pneumonia (using feature attribution).

Step-by-Step Instructions

Step 1: Create an S3 Bucket for Explainability Assets

To be transparent, an organization must store and share its evaluation reports, fairness metrics, and bias detection results (e.g., from SageMaker Clarify). We need an S3 bucket to hold these assets.

bash

aws s3 mb s3://brainybee-xai-assets-<YOUR_ACCOUNT_ID>

[!TIP] Replace <YOUR_ACCOUNT_ID> with your 12-digit AWS account number to ensure the bucket name is globally unique.

▶Console alternative

Navigate to Amazon S3 in the AWS Console.
Click Create bucket.
Enter brainybee-xai-assets-<YOUR_ACCOUNT_ID> in the Bucket name field.
Leave all other settings as default and click Create bucket.

📸 Screenshot: S3 Create Bucket Form

Step 2: Define the Model Transparency Metadata

We will create a JSON file that acts as the blueprint for our SageMaker Model Card. This payload explicitly details the model's intended use, its risk rating, and the explainability tools used.

Create a file named model_card.json on your local machine and paste the following content:

bash

cat << EOF > model_card.json
{
  "ModelId": "credit-scoring-model-v1",
  "ModelDescription": "A random forest model used to predict credit default risk.",
  "ModelCreator": "BrainyBee Financial Risk Team",
  "IntendedUses": {
    "PurposeOfModel": "To assist loan officers in determining creditworthiness.",
    "IntendedUses": "Financial risk assessment for retail banking.",
    "RiskRating": "High"
  },
  "BusinessDetails": {
    "BusinessProblem": "Automate and standardize loan approval processes.",
    "BusinessStakeholders": "Retail Banking Division"
  },
  "EvaluationDetails": {
    "Datasets": [{"Name": "Historical_Loan_Data_2020_2023"}],
    "EvaluationObservation": "Model demonstrates high accuracy but requires continuous monitoring for demographic bias."
  },
  "AdditionalInformation": {
    "CustomDetails": {
      "ExplainabilityFramework": "SHAP (SHapley Additive exPlanations)",
      "TransparencyLevel": "High - Training data sources are documented internally",
      "ArtifactLocation": "s3://brainybee-xai-assets-<YOUR_ACCOUNT_ID>/reports/"
    }
  }
}
EOF

Step 3: Create the SageMaker Model Card

Now, submit the JSON payload to Amazon SageMaker to create the official system of record for this model. This serves as a single source of truth for governance and compliance.

bash

aws sagemaker create-model-card \
  --model-card-name "CreditScoring-Transparent-Model" \
  --model-card-status "Approved" \
  --content file://model_card.json

▶Console alternative

Navigate to Amazon SageMaker > Governance > Model Dashboard > Model Cards.
Click Create model card.
Name it CreditScoring-Transparent-Model.
Manually fill in the sections matching the JSON file (Intended Uses, Business Details, Explainability Frameworks).
Set Status to Approved and Save.

📸 Screenshot: SageMaker Model Card Creation Screen

Step 4: Verify Model Card Creation

Retrieve the details of your newly created Model Card to verify that the transparency and explainability metadata has been correctly stored.

bash

aws sagemaker describe-model-card \
  --model-card-name "CreditScoring-Transparent-Model"

Review the JSON output. You should clearly see the ExplainabilityFramework listed as SHAP.

Checkpoints

Verify your progress by performing these checks:

S3 Verification: Run aws s3 ls and verify brainybee-xai-assets-<YOUR_ACCOUNT_ID> is listed.
Model Card Verification: Check the model card status using:
bash
aws sagemaker list-model-cards --query "ModelCardSummaries[?ModelCardName=='CreditScoring-Transparent-Model'].ModelCardStatus"
Expected Output:

Troubleshooting

Error	Cause	Fix
`BucketAlreadyExists`	S3 bucket names must be globally unique.	Ensure you appended your specific AWS account ID or a random string to the bucket name.
`AccessDeniedException`	Missing IAM permissions for SageMaker or S3.	Attach the `AmazonSageMakerFullAccess` and `AmazonS3FullAccess` policies to your IAM user.
`ValidationError: Invalid JSON`	The `model_card.json` file is malformed.	Ensure all quotes are properly closed and commas are correctly placed in the JSON file.

Teardown / Clean-Up

[!WARNING] Cost Warning: Leaving SageMaker resources and S3 buckets active can result in ongoing charges. Always clean up resources when you are finished.

Execute the following commands to delete the resources provisioned in this lab:

1. Delete the SageMaker Model Card

bash

aws sagemaker delete-model-card \
  --model-card-name "CreditScoring-Transparent-Model"

2. Delete the S3 Bucket

bash

# Empty the bucket first (if you added any files manually)
aws s3 rm s3://brainybee-xai-assets-<YOUR_ACCOUNT_ID> --recursive

# Delete the bucket
aws s3 rb s3://brainybee-xai-assets-<YOUR_ACCOUNT_ID>

▶Console alternative

Navigate to SageMaker > Model Cards, select the card, and choose Delete.
Navigate to S3, empty the bucket, then select the bucket and click Delete.

Stretch Challenge

Challenge: Currently, the Model Card is just a JSON record. Can you create a SageMaker Model Card Export Job to generate a shareable PDF of this model card and save it directly into your S3 bucket?

▶Click to show solution

bash

aws sagemaker create-model-card-export-job \
  --model-card-export-job-name "export-transparency-report" \
  --model-card-name "CreditScoring-Transparent-Model" \
  --output-config S3OutputPath="s3://brainybee-xai-assets-<YOUR_ACCOUNT_ID>/exports/"

After running this, use aws s3 ls s3://brainybee-xai-assets-<YOUR_ACCOUNT_ID>/exports/ to see the generated PDF report!