Implementing Hybrid Routing and Connectivity: On-Premises to AWS

This guide covers the critical strategies and technical implementations required to bridge on-premises data centers with the AWS Cloud, focusing on Domain 2.1 of the AWS Certified Advanced Networking Specialty (ANS-C01).

---

Learning Objectives

After studying this guide, you should be able to:

• Distinguish between AWS Direct Connect (DX) and Site-to-Site VPN for hybrid connectivity.
• Configure Border Gateway Protocol (BGP) for dynamic route advertisement between environments.
• Implement Layer 1, 2, and 3 requirements for physical and virtual interconnects.
• Optimize routing paths using BGP attributes and static routing where appropriate.
• Troubleshoot connectivity issues using tools like VPC Reachability Analyzer and CloudWatch.

Key Terms & Glossary

• Autonomous System (AS): A collection of connected IP routing prefixes under the control of one or more network operators that presents a common routing policy to the internet.
• BGP (Border Gateway Protocol): The industry-standard path-vector protocol used to exchange routing and reachability information between autonomous systems.
• eBGP (External BGP): BGP used between different autonomous systems (e.g., between your data center and AWS).
• Virtual Private Gateway (VGW): The edge connector on the AWS side of a VPN or Direct Connect association.
• Transit Gateway (TGW): A network transit hub that can be used to interconnect VPCs and on-premises networks at scale.
• LOA-CFA (Letter of Authorization - Connecting Facility Assignment): A document providing permission to connect at a specific colocation facility.

The "Big Idea"

The core goal of hybrid connectivity is to create a single contiguous network. By extending your on-premises environment into the AWS Cloud, applications can access resources across boundaries with the same security, reliability, and performance expectations as if they were in the same building.

Formula / Concept Box

Hierarchical Outline

1. Connectivity Fundamentals
   • Physical Layer (L1): Colocation, LOA, cross-connects, and optics.
   • Data Link Layer (L2): 802.1Q VLANs, Link Aggregation Groups (LAG).
   • Network Layer (L3): IP addressing (CIDR), routing protocols.
2. Dynamic Routing with BGP
   • Peering: Establishing sessions between Customer Gateways (CGW) and AWS Virtual Gateways (VGW).
   • Prefix Advertisement: Controlling which on-premises subnets are visible to AWS.
   • Path Selection: Using AS-Path prepending and Multi-Exit Discriminators (MED) to influence traffic.
3. Redundancy & Failover
   • Designing for high availability using multiple DX locations.
   • Using VPN as a cost-effective backup for Direct Connect.

Visual Anchors

Hybrid Connectivity Architecture

BGP Peering Logic

\begin{tikzpicture}[node distance=2cm, every node/.style={draw, rectangle, rounded corners, inner sep=5pt, align=center}] \node (onprem) {On-Premises Router \ (ASN 65000)}; \node (aws) [right=3cm of onprem] {AWS Router \ (ASN 64512)};

\draw[<->, thick] (onprem) -- (aws) node[midway, above] {BGP Session (TCP 179)};

\draw[->] (onprem) -- +(0,1) node[above] {Advertise Prefixes: 10.0.0.0/16}; \draw[<-] (aws) -- +(0,1) node[above] {Receive Prefixes: 172.16.0.0/16}; \end{tikzpicture}

Definition-Example Pairs

• Link Aggregation Group (LAG): A method of grouping multiple physical Direct Connect connections into one logical interface.
  • Example: An enterprise combines two 10Gbps DX connections into a single 20Gbps LAG to handle peak data migration traffic.
• Static Routing: Manually configured routes that do not change automatically based on network state.
  • Example: A small office with a single VPN connection uses static routes to point all 10.x.x.x traffic to the VPN tunnel because the network topology never changes.
• Jumbo Frames: Ethernet frames with more than 1500 bytes of payload (up to 9001 bytes in AWS).
  • Example: Enabling jumbo frames on a Direct Connect connection to reduce CPU overhead and improve throughput for large database backups.

Worked Examples

Scenario: Configuring Failover with BGP

Problem: A company has a 1Gbps Direct Connect and a Site-to-Site VPN. They want traffic to prefer Direct Connect and only use VPN if the DX connection fails.

Step-by-Step Implementation:

1. Configure BGP on DX: Advertise the on-premises prefix (e.g., 192.168.0.0/24) via the DX connection.
2. Configure BGP on VPN: Advertise the same prefix via the VPN.
3. Influence Inbound Traffic (AWS to On-Prem): AWS prefers Direct Connect over VPN by default if the prefix lengths are identical. However, to be certain, you can use AS-Path Prepending on the VPN BGP session to make the VPN path look longer/less desirable.
4. Influence Outbound Traffic (On-Prem to AWS): On the customer router, set a higher Local Preference for routes learned via the Direct Connect interface.
5. Verification: Shut down the DX interface and verify that the routing table updates to point to the VPN tunnel using show ip bgp on the on-premises router.

Checkpoint Questions

1. What is the default preference for AWS when it receives the same prefix via Direct Connect and a Site-to-Site VPN?
2. Which BGP attribute is typically used to influence how traffic leaves AWS and enters your on-premises network?
3. What document must you download from the AWS Console to provide to your colocation provider to establish the physical fiber cross-connect?
4. True or False: You can use iBGP to establish a peering session between a Customer Gateway and a Virtual Private Gateway.

[!TIP] Answers: 1. Direct Connect is preferred. 2. AS-Path Prepending (to make a path less preferred) or MED. 3. Letter of Authorization (LOA-CFA). 4. False; AWS requires eBGP for hybrid connectivity.

Muddy Points & Cross-Refs

• Overlapping IP Addresses: A common issue when merging networks. If on-premises and VPC CIDRs overlap, routing will fail. Use PrivateLink or NAT Gateways as a workaround.
• BGP ASN Ranges: Remember that for 32-bit ASNs, the range is 1–4,294,967,295. AWS supports both 2-byte and 4-byte ASNs.
• MTU Mismatch: If you enable Jumbo Frames (9001 MTU) on AWS but your on-premises router or intermediate provider only supports 1500, packets will be dropped or fragmented, leading to performance degradation.

Comparison Tables

BGP Routing vs. Static Routing

Direct Connect Virtual Interface (VIF) Types