Scaling Network Architectures for AWS

Learning Objectives

After studying this guide, you should be able to:

Plan VPC CIDR blocks that accommodate future growth and prevent IP exhaustion.
Design multi-tier subnet architectures that allow for security segmentation and high availability across multiple Availability Zones.
Compare connectivity options (VPN, Direct Connect, PrivateLink) based on scalability and bandwidth requirements.
Evaluate edge networking services like Amazon CloudFront and AWS Global Accelerator to improve global performance.
Optimize network throughput using Enhanced Networking and appropriate NAT Gateway configurations.

Key Terms & Glossary

CIDR (Classless Inter-Domain Routing): A method for allocating IP addresses and IP routing. Example: 10.0.0.0/16 provides 65,536 addresses.
VPC Endpoint: A private connection between your VPC and supported AWS services without requiring an internet gateway or NAT device.
Direct Connect (DX): A dedicated network connection from your premises to AWS, providing consistent 1 Gbps, 10 Gbps, or 100 Gbps bandwidth.
NAT Gateway: A managed service that allows instances in a private subnet to connect to the internet while preventing the internet from initiating a connection with those instances.
Anycast IP: An IP address that is advertised from multiple locations globally, used by AWS Global Accelerator to route users to the nearest edge location.

The "Big Idea"

Network scalability is the foundation of cloud architecture. If a network is designed too small (IP exhaustion) or too rigidly (single Availability Zone), the application cannot grow regardless of how much compute power is added. A truly scalable network must be elastic, redundant, and globally distributed to meet future demands without requiring a complete re-architecture.

Formula / Concept Box

Concept	Metric / Rule	Significance
VPC Max Size	`/16` (65,536 IPs)	Largest CIDR block allowed in a VPC.
VPC Min Size	`/28` (16 IPs)	Smallest CIDR block allowed in a VPC.
Reserved IPs	5 IPs per Subnet	AWS reserves the first four and the last IP in every subnet.
VPN Bandwidth	~1.25 Gbps	Maximum throughput per VPN tunnel.
DX Bandwidth	1, 10, or 100 Gbps	Standard physical port speeds for Direct Connect.

Hierarchical Outline

VPC Foundation & IP Planning
- CIDR Strategy: Always choose a larger block than currently needed (e.g., /16).
- Overlapping IPs: Ensure VPC CIDRs do not overlap with on-premises networks to allow for future VPN or Direct Connect integration.
Subnet Design for Scalability
- Multi-Tier Architecture: Separate web, application, and database layers into distinct subnets.
- Availability Zone (AZ) Spread: Deploy subnets across at least three AZs to leverage AWS infrastructure growth.
- Spare Capacity: Leave unallocated CIDR space within the VPC to create new subnets for new services or AZs.
Scaling Connectivity
- Hybrid Connectivity: Scale from Site-to-Site VPN to Direct Connect as data volume increases.
- PrivateLink: Scale service-to-service communication privately without managing complex VPC peering or NATs.
Performance & Edge Scaling
- Content Delivery: Use Amazon CloudFront to offload traffic from the origin.
- Latency Reduction: Use AWS Global Accelerator for non-HTTP traffic or multi-region failover.

Visual Anchors

Multi-Tier Scalable VPC Architecture

Loading Diagram...

Subnet IP Allocation Logic

\begin{tikzpicture} \draw[thick] (0,0) rectangle (8,4); \node at (4,3.5) {VPC CIDR: 10.0.0.0/16 (65,536 IPs)};

code

\draw[fill=blue!10] (0.5,0.5) rectangle (2.5,2.5);
\node[align=center] at (1.5,1.5) {Subnet A\\/24\$256 IPs)};

\draw[fill=green!10] (3,0.5) rectangle (5,2.5);
\node[align=center] at (4,1.5) {Subnet B\\/24\$256 IPs)};

\draw[dashed, fill=gray!10] (5.5,0.5) rectangle (7.5,2.5);
\node[align=center] at (6.5,1.5) {Future\\Subnets\$Spare Capacity)};

\end{tikzpicture}

Definition-Example Pairs

Horizontal Scaling (Network): Adding more subnets or NAT Gateways to handle increased load.
- Example: Deploying one NAT Gateway per Availability Zone instead of a single shared one to prevent a single point of failure and bottleneck.
Edge Caching: Storing content closer to users to reduce latency and origin load.
- Example: A video streaming site using Amazon CloudFront to serve 4K content from edge locations in London for UK users, rather than fetching from an S3 bucket in Virginia.
Enhanced Networking: Using Single Root I/O Virtualization (SR-IOV) to provide high performance and low CPU utilization.
- Example: Enabling the Elastic Fabric Adapter (EFA) on EC2 instances for high-performance computing (HPC) workloads that require sub-millisecond network latency.

Worked Examples

Example 1: Calculating IP Headroom

Scenario: A company plans to deploy a microservices application. They expect to have 50 services, each requiring 10 instances across 3 AZs for high availability. They also need space for Load Balancers and RDS instances. Calculation:

Total Instances: 50 services × 10 instances = 500 instances.
AZ Distribution: 500 / 3 ≈ 167 instances per AZ.
Subnet Sizing: A /24 subnet provides 251 usable IPs (256 - 5 reserved).
Decision: Using a /24 per subnet (3 subnets) provides 753 usable IPs. This accommodates the 500 instances plus roughly 50% headroom for scaling and secondary resources (ELB nodes, VPC endpoints).

Example 2: Choosing Connectivity for Massive Data Migrations

Scenario: An enterprise needs to move 500 TB of data to AWS over a month and requires a consistent 5 Gbps connection for ongoing synchronization. Solution:

VPN is insufficient (capped at 1.25 Gbps and traverses the public internet).
Direct Connect (DX) is the scalable choice. The company should provision a 10 Gbps Dedicated Connection or multiple 1 Gbps hosted connections in a Link Aggregation Group (LAG) to meet the 5 Gbps requirement with room for growth.

Checkpoint Questions

What is the primary risk of choosing a /24 CIDR block for your entire VPC?
Why should you deploy a NAT Gateway in each Availability Zone rather than one for the whole VPC?
Which service uses Anycast IPs to route traffic to the nearest healthy endpoint over the AWS global network?
If your application requires 100 Gbps network speeds between EC2 instances, which network interface should you use?
How many IP addresses are reserved by AWS in a /28 subnet?

[!TIP] Always leave at least 50% of your VPC CIDR space unassigned when first creating subnets. This "spare room" is vital when AWS launches new services or Availability Zones in a region.

[!WARNING] VPC peering does not support transitive routing. If VPC A is peered with B, and B with C, A cannot talk to C. For large-scale multi-VPC environments, use AWS Transit Gateway to simplify and scale connectivity.