Mastering BGP Traffic Engineering for AWS Hybrid Connectivity

This guide explores the strategic use of Border Gateway Protocol (BGP) attributes to design traffic patterns between on-premises environments and the AWS Cloud. Understanding how to manipulate BGP ensures high availability (Active/Passive) and optimal throughput (Load Sharing) for hybrid architectures.

Learning Objectives

By the end of this study guide, you should be able to:

• Identify the core BGP attributes used for path selection (AS_PATH, Local Preference, MED).
• Design an Active/Passive routing strategy for hybrid connectivity using AS_PATH prepending.
• Configure load sharing (Active/Active) across multiple Direct Connect or VPN connections.
• Evaluate the AWS BGP path selection order to predict routing behavior in multi-path scenarios.
• Distinguish between attributes that influence inbound traffic versus those that influence outbound traffic.

Key Terms & Glossary

• Autonomous System (AS): A collection of IP networks under a single administrative domain (e.g., AWS has its own ASNs like 64512, 16509).
• eBGP (External BGP): BGP sessions between different ASNs (e.g., between your router and AWS).
• iBGP (Internal BGP): BGP sessions within the same ASN.
• NLRI (Network Layer Reachability Information): The prefix and length information advertised in BGP updates.
• AS_PATH Prepending: Artificially lengthening the list of ASNs in a route to make it look less desirable to BGP routers.
• Multi-Exit Discriminator (MED): An attribute used to suggest a preferred path into an AS when multiple entry points exist.
• Local Preference: An attribute used to tell internal routers which exit point to prefer for outbound traffic.

The "Big Idea"

BGP is essentially a policy-based routing database. Unlike OSPF or EIGRP, which focus on finding the fastest "shortest" path, BGP allows administrators to enforce business logic on traffic flows. In the AWS context, BGP is the "language" used by Direct Connect (DX) and Site-to-Site VPN to negotiate how data enters and leaves your VPC. Traffic engineering is the art of telling AWS how to reach you (Inbound) and telling your own network how to reach AWS (Outbound).

Formula / Concept Box

AWS BGP Path Selection Priority (High to Low)

Hierarchical Outline

• I. BGP Fundamentals in AWS
  • Autonomous System Numbers (ASN): Use of Public ASNs vs. Private ASNs (64512–65534).
  • Peering Types: Standard eBGP sessions over Direct Connect VIFs or VPN Tunnels.
• II. Influencing Inbound Traffic (To On-Premises)
  • AS_PATH Prepending: Adding your own ASN multiple times to the advertisement sent to AWS.
  • MED (Multi-Exit Discriminator): Passing a lower MED to the preferred link; useful for multiple connections in one region.
  • Community Tags: AWS-specific BGP communities to limit route propagation.
• III. Influencing Outbound Traffic (To AWS)
  • Local Preference: Setting a higher value on your on-prem router for the preferred AWS-bound link.
  • Weight: (Cisco specific) Local to the router; highest weight wins.
• IV. Design Patterns
  • Active/Passive: Using prepending to ensure one link is a standby.
  • Active/Active (ECMP): Ensuring prefixes, AS_PATH, and MED are identical to enable Equal-Cost Multi-Pathing.

Visual Anchors

Traffic Direction Control

AS_PATH Prepending for Active/Passive

Definition-Example Pairs

• Attribute: AS_PATH Prepending

  • Definition: Appending the local ASN multiple times to a BGP route advertisement to make the path appear longer.
  • Example: To make a backup VPN link less attractive than a Direct Connect link, the on-prem router advertises 172.16.0.0/12 with path 65111 65111 65111 to the VPN, but simply 65111 to the Direct Connect.

• Pattern: Load Sharing (ECMP)

  • Definition: Distributing traffic across multiple paths that have equal BGP attributes.
  • Example: Configuring two Direct Connect connections to advertise the exact same CIDR with identical AS_PATH lengths and MED values allows the AWS Transit Gateway to balance traffic across both.

Worked Examples

Scenario: Designing High Availability with Direct Connect and VPN

Problem: You have a 1Gbps Direct Connect (DX) and a 1Gbps Site-to-Site VPN. You want all traffic to use DX normally, and only use VPN if DX fails.

Step-by-Step Breakdown:

1. Prefix Advertisement: Advertise the same on-premises prefixes (e.g., 192.168.0.0/16) over both DX and VPN.
2. Inbound Influence (AWS to On-Prem): On the VPN router, configure AS_PATH Prepending. Add your ASN 3 times to the VPN advertisement. AWS will see the DX path (Length 1) as shorter than the VPN path (Length 4) and prefer DX.
3. Outbound Influence (On-Prem to AWS): On your on-premise core router, receive the AWS routes. Set a higher Local Preference (e.g., 200) for routes learned via DX and a lower value (e.g., 100) for routes learned via VPN.
4. Verification: Check the BGP table on your router. The > (best) symbol should appear next to the DX-learned routes.

Checkpoint Questions

1. If AWS receives a /24 prefix over a VPN and a /16 prefix over Direct Connect, which path will it choose for a destination within that /24? (Answer: The VPN, because the more specific prefix wins before BGP attributes are considered).
2. Which BGP attribute is local to an Autonomous System and never passed to eBGP peers? (Answer: Local Preference).
3. True or False: To enable ECMP (Load Sharing) on AWS, the AS_PATH and MED must be identical for all participating paths. (Answer: True).

Muddy Points & Cross-Refs

• Asymmetric Routing: A common "muddy point" occurs when you influence inbound traffic but forget outbound. Traffic might go to AWS via DX but return via VPN. Always ensure your Local Preference (Outbound) matches your Prepending logic (Inbound).
• AWS Default Preference: By default, AWS prefers Direct Connect over VPN if the prefixes and AS_PATH lengths are equal. However, if the VPN advertises a more specific prefix, the VPN wins. Refer to the Direct Connect Gateway documentation for specific path selection nuances in multi-region setups.

Comparison Tables

Inbound vs. Outbound Control

Active/Passive vs. Active/Active