AWS Application Storage Patterns: EBS, EFS, and S3

This guide covers the core storage services within AWS (Amazon EBS, Amazon EFS, and Amazon S3) and how to select the correct pattern for instance-based, containerized, and serverless applications.

Learning Objectives

After studying this guide, you should be able to:

• Differentiate between Block, File, and Object storage paradigms.
• Identify the correct storage service based on throughput, latency, and access requirements.
• Design hybrid storage solutions using AWS Storage Gateway.
• Implement storage patterns that support high availability and disaster recovery (RPO/RTO).

Key Terms & Glossary

• Block Storage: Data is stored in fixed-size blocks; ideal for low-latency database workloads.
• Object Storage: Data is stored as objects with metadata and a unique key; ideal for unstructured data and web scaling.
• POSIX: A family of standards for maintaining compatibility between operating systems (EFS/FSx for Lustre are POSIX-compliant).
• IOPS (Input/Output Operations Per Second): A performance metric used to measure the speed of storage devices.
• Throughput: The amount of data moved from one place to another in a given time period (typically MB/s).
• WORM (Write Once Read Many): A data storage technology that prevents the erasure or modification of data (S3 Object Lock).

The "Big Idea"

In the AWS Cloud, storage is not just a place to put files—it is a decoupling mechanism. By choosing the right storage pattern, you separate the application's "state" from the compute layer. This allows you to treat EC2 instances and containers as ephemeral (disposable) resources, enabling seamless auto-scaling, blue/green deployments, and high resiliency.

Formula / Concept Box

Hierarchical Outline

• I. Block Storage (Amazon EBS)
  • Characteristics: Low latency, single-AZ by default, attached to one instance at a time (mostly).
  • Use Cases: Primary storage for file systems, Relational Databases, and raw block access.
• II. File Storage (Amazon EFS & FSx)
  • Amazon EFS: Fully managed NFS v4 for Linux; supports thousands of concurrent connections.
  • Amazon FSx for Lustre: High-performance parallel storage for HPC and Machine Learning.
  • Amazon FSx for Windows: Native SMB support for Microsoft environments.
• III. Object Storage (Amazon S3)
  • Architecture: Buckets (Global unique name) and Objects (Key/Value).
  • Capabilities: Static website hosting, Cross-Region Replication, and lifecycle policies.
• IV. Hybrid Patterns (Storage Gateway)
  • File Gateway: S3 access via NFS/SMB.
  • Volume Gateway: Block storage with cloud backup (Cached or Stored).
  • Tape Gateway: Replaces physical tapes with virtual tapes in S3/Glacier.

Visual Anchors

Storage Selection Decision Tree

EBS Architecture Layout

Definition-Example Pairs

• Pattern: Shared Configuration Management

  • Definition: Using a single source of truth for configuration files across a fleet of Linux servers.
  • Example: Mounting an Amazon EFS volume to /etc/config on ten different EC2 instances so they all read the same settings simultaneously.

• Pattern: Static Asset Delivery

  • Definition: Offloading non-computational data to a high-availability storage tier to reduce server load.
  • Example: Storing application images and CSS files in an Amazon S3 bucket and serving them directly to users via CloudFront.

• Pattern: High-Performance Database Volume

  • Definition: Dedicated low-latency throughput for random read/write operations.
  • Example: Attaching a Provisioned IOPS (io2) EBS volume to an EC2 instance running a MySQL database.

Worked Examples

Example 1: Migrating a Legacy Windows App

Problem: A company has a legacy .NET application that requires an SMB file share to store user documents. They want to move to AWS with minimal code changes. Solution:

1. Provision Amazon FSx for Windows File Server.
2. Join the FSx file system to the company's Active Directory.
3. Map the network drive (e.g., Z:) on the EC2 instances to the FSx DNS name. Why: This preserves the SMB protocol and NTFS permissions the app expects.

Example 2: Building a Log Processing Pipeline

Problem: You need to collect logs from 100 containers and analyze them for security threats every 24 hours. Solution:

1. Containers push logs to Amazon S3 using the AWS SDK or a logging driver.
2. Configure an S3 Event Notification to trigger an AWS Lambda function whenever a new log file is uploaded.
3. Use Amazon Athena to query the logs directly in S3 using SQL for the 24-hour report.

Checkpoint Questions

1. Which storage service would you use for a High-Performance Computing (HPC) cluster requiring sub-millisecond latencies and hundreds of GB/s throughput?
2. True or False: Amazon EBS volumes are automatically replicated across multiple Availability Zones.
3. What is the primary protocol used to access Amazon EFS?
4. Which AWS Storage Gateway type should be used to replace physical tape backup systems?

▶Click to see answers

1. Amazon FSx for Lustre.
2. False (EBS is replicated within a single AZ; use snapshots or Multi-Attach for different patterns).
3. NFSv4 (Network File System).
4. Tape Gateway.

Muddy Points & Cross-Refs

• EFS vs. S3 for Shared Files: Use EFS if your application needs to use standard OS commands (like ls, cd, fopen) and needs to modify parts of files. Use S3 if you are accessing whole files via API and need global scale.
• EBS vs. Instance Store: Remember that Instance Store is ephemeral (data is lost on stop/terminate), whereas EBS is persistent. For DevOps exams, always lean toward EBS unless "maximum possible IOPS/temporary data" is specified.
• Cross-Reference: For security, see AWS KMS for encrypting all these storage types at rest.

Comparison Tables

S3 Storage Classes

[!TIP] Use S3 Intelligent-Tiering if you have unknown or changing access patterns; it automatically moves data between tiers to save costs without operational overhead.