Study Guide: Encapsulation and Encryption Technologies (GRE & IPsec)

This guide covers the fundamental technologies used to establish secure, private communication paths over public or untrusted networks, focusing on Generic Routing Encapsulation (GRE) and Internet Protocol Security (IPsec).

Learning Objectives

After studying this module, you should be able to:

• Explain the primary purpose and mechanics of Generic Routing Encapsulation (GRE).
• Describe the security limitations of GRE and how to mitigate them.
• Compare and contrast IPsec Transport Mode and IPsec Tunnel Mode.
• Identify specific use cases for multicast support in tunneling (e.g., routing protocols).
• Determine which encryption/encapsulation technology to apply based on compliance and functional requirements.

Key Terms & Glossary

• Encapsulation: The process of wrapping a data packet inside another packet's header to allow it to traverse a network that might not otherwise support the original packet's protocol.
• Tunneling: A method of transporting data across a network by using a wrapper protocol to hide the original payload.
• Payload: The actual data or original packet being carried inside the encapsulation headers.
• Multicast: A communication pattern where a single sender transmits data to multiple recipients simultaneously; often used by routing protocols like OSPF.
• ESP (Encapsulating Security Payload): An IPsec protocol that provides confidentiality, data origin authentication, and connectionless integrity.
• AH (Authentication Header): An IPsec protocol that provides authentication and integrity but no encryption for the payload.

The "Big Idea"

In hybrid networking, we often need to treat the public internet as a private "virtual wire." GRE provides the "wire" (the ability to carry diverse traffic like multicast) but lacks "armor." IPsec provides the "armor" (encryption and authentication) but is often restricted in what it can carry (typically unicast IP). By combining them—often referred to as GRE over IPsec—we create a secure, flexible tunnel that can handle complex routing and enterprise-grade security simultaneously.

Formula / Concept Box

Hierarchical Outline

1. Generic Routing Encapsulation (GRE)
   • Functionality: Creates point-to-point private paths over public IP networks.
   • Encapsulation Process: Original IP packet → GRE Header → New IP Header.
   • Advantages: Supports Multicast (OSPF, BGP, video streams) and non-IP protocols.
   • Security: Minimal; no native encryption or authentication.
2. IPsec (Internet Protocol Security)
   • Overview: A family of protocols for end-to-end encryption and authentication.
   • Key Components:
     • AH: Integrity/Auth only.
     • ESP: Encryption + Auth.
   • Modes of Operation:
     • Transport Mode: Encrypts payload only; IP header remains visible.
     • Tunnel Mode: Encrypts entire packet; adds a completely new IP header.

Visual Anchors

GRE Tunneling Logic

IPsec Tunnel Mode Structure

\begin{tikzpicture}[node distance=0cm, start chain=1 going right] \node [on chain=1, draw, fill=gray!20, minimum height=1cm, minimum width=2.5cm] (newip) {New IP Header}; \node [on chain=1, draw, fill=blue!20, minimum height=1cm, minimum width=2.5cm] (ipsec) {IPsec Header}; \node [on chain=1, draw, fill=red!20, minimum height=1cm, minimum width=2.5cm] (origip) {Orig. IP Header}; \node [on chain=1, draw, fill=green!20, minimum height=1cm, minimum width=3cm] (payload) {Data Payload};

\draw [decorate, decoration={brace, amplitude=10pt, mirror}] (origip.south west) -- (payload.south east) node [midway, yshift=-0.6cm] {Encrypted portion in Tunnel Mode};

\end{tikzpicture}

Definition-Example Pairs

• Tunnel Mode: Encapsulates the entire IP packet (including original headers) inside a new IP packet.
  • Example: An AWS Site-to-Site VPN uses Tunnel Mode to hide the internal private IP addresses of your VPC while traffic crosses the internet.
• Transport Mode: Only the payload of the IP packet is encrypted; the original IP header is preserved.
  • Example: Host-to-host communication where two servers on the same local network want to encrypt their specific SQL traffic without changing routing.

Worked Examples

Scenario 1: Routing Protocol Support

Problem: A network engineer needs to run OSPF between an on-premises data center and an EC2 instance to share routing tables. A standard IPsec VPN is failing to form the adjacency. Solution:

1. Identify that OSPF uses multicast (224.0.0.5/6).
2. Determine that standard IPsec tunnels often drop multicast traffic.
3. Implement a GRE tunnel to encapsulate the OSPF multicast packets.
4. Wrap the GRE tunnel in an IPsec tunnel for security (GRE-over-IPsec).

Scenario 2: Compliance-Driven Encryption

Problem: A healthcare application requires that all data in transit be encrypted using FIPS-validated modules. The connection is over a dedicated Direct Connect line. Solution:

1. Direct Connect provides a private path but no native encryption.
2. Deploy an IPsec VPN on top of the Direct Connect Virtual Interface (VIF).
3. Ensure the IPsec configuration uses ESP for encryption (rather than just AH) to satisfy confidentiality requirements.

Checkpoint Questions

1. Which protocol should be used if you need to transport non-IP protocols across an IP-based network?
2. What is the main security risk of using a "bare" GRE tunnel?
3. In IPsec, what is the primary difference between the Authentication Header (AH) and Encapsulating Security Payload (ESP)?
4. Why is IPsec Tunnel Mode preferred for Site-to-Site VPNs over Transport Mode?

Muddy Points & Cross-Refs

• MTU and Fragmentation: Encapsulation adds headers (GRE = 24 bytes; IPsec = ~50-70 bytes). This reduces the effective Maximum Transmission Unit (MTU). If packets are not resized, fragmentation occurs, killing performance. Cross-Ref: See "Jumbo Frames" and "Path MTU Discovery (PMTUD)".
• IPsec over Direct Connect: A common misconception is that Direct Connect is "encrypted" because it is "private." It is not. You must layer IPsec on top for confidentiality.

Comparison Tables

GRE vs. IPsec

IPsec: Transport vs. Tunnel Mode

[!IMPORTANT] For the AWS Advanced Networking Exam, remember: GRE = Multicast/Flexibility, IPsec = Security/Confidentiality. Use them together for the best of both worlds.