IPSec and IKE

Duncan S. Wong
City University of Hong Kong

v1.1 1
 Outline
• Virtual Private Networks
• Tunneling
• IPSec
• IKE
• Critiques of IPSec (under construction)
• IKEv2 (under construction)

Readings
• Microsoft TechNote Windows 2000 Server Internetwork
Guide, Chap. 9 – Virtual Private Networking
(
http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/windows2000
)

v1.1 2
 Virtual Private Networks (VPNs)

• A virtual private network (VPN) is the

extension of a private network.
• A VPN enables you to send data between
two computers across the Internet in a
manner that emulates the properties of
a point-to-point private link.
• To emulate a point-to-point link, data is
encapsulated (or wrapped) with a header
that provides routing information
allowing it to traverse the Internet to
reach its endpoint.
• To emulate a private link, the data being
sent is encrypted for confidentiality.

v1.1 3
 Components of a VPN Connection

• VPN server: A computer that

accepts VPN connections from
VPN clients.
• VPN client: A computer that
initiates a VPN connection to a
VPN server.

• Tunnel: The portion of the connection in which your data is encapsulated (or
wrapped) according to a tunneling protocol.
• VPN connection: The portion of the connection in which your data is
encrypted.
• Remark: It is possible to create a tunnel and send the data through the
tunnel without encryption. This is not a VPN connection because the private
data is sent across the Internet in an unencrypted and easily readable form.

v1.1 4
 Tunneling
• Tunneling protocols: emulate point-to-point link by encapsulating
data.

• Layer 2 Tunneling Examples:

• Point-to-Point Tunneling Protocol (PPTP)
• Layer Two Forwarding (L2F)
• Layer Two Tunneling Protocol (L2TP)

• Layer 3 Tunneling Example:

• Internet Protocol Security (IPSec)

• Remark: Data that is tunneled must also be encrypted to be a VPN

connection.

v1.1 5
 VPN Connections

• Remote Access VPN connection

• Router-to-Router VPN connection

v1.1 6
 IPSec Overview
• IPSec (Internet Protocol Security) is a suite of standards for
providing a rich set of security services at the network layer.
• Transparent to applications (below transport layer – TCP, UDP)

IPSec Features:
• Data source authentication (MAC / Signature)
• Message authentication and integrity check (MAC)
• Confidentiality (encryption)
• Detection of replayed messages
• Limited protection against denial-of-service attack
• Optional identity anonymity

v1.1 7
 IPSec Architecture
IPSec Suite

Manual
IPSec
Keying
Policy
Manager &
Database X.509
Applications IKE Certificate
Manager

IPSec
TCP/IP Packet Engine Cryptographic
Interceptor Services
stack

Network
adapter

v1.1 8
 Major IPSec Components
• Security Association (SA) Database
• Each entry of a SA database refers to all the security
parameters of one communication direction
• For two-way communications, at least two SAs are needed.
• Two Protocols
• AH – Authentication Header
• ESP – Encapsulating Security Payload
1. Encryption only
2. Encryption with authentication
• Two Encapsulation modes
1. Transport mode
2. Tunnel mode
• Policy Database (SPD)

v1.1 9
 Security Associations (SAs)
• Simplex: a one way relationship between a sender and a receiver.
• Set up manually or through the IKE (Internet Key Exchange)
protocol.
• Each SA is uniquely identified by three parameters:
1. Security Parameter Index (SPI)
2. IP Destination address
3. Security Protocol Identifier (e.g. AH, ESP)
• It contains all the security parameters needed for one way
communication
• Sequence number counter
• Anti-replay window
• Protocol (e.g. AH / ESP)
• Transform mode (e.g. transport / tunnel mode / wildcard)
• Protocol parameters (e.g. AES, 128-bit, CBC mode, SHA-1)
• Lifetime of the SA
• etc.

v1.1 10
 Two IPSec Base Protocols
• Authentication Header (AH)
• Provides message authentication and
integrity check of IP data payload.
• Also Provides authentication for as
much of the IP header as possible.
• Next header: TCP, UDP, etc.
• Sequence Number: Starts at 1, never
recycle (optional)

• Encapsulating Security Payload (ESP)

• Provides confidentiality and
authentication. Either is optional.
• When not used, the NULL algorithm
defined in RFC-2410 is used.
• The authentication trailer must be
omitted if not used.
• Either encryption or authentication
(or both) must be enabled
(NULL-NULL is an invalid option)
v1.1 11
 Two Encapsulation Modes
• IPsec defines two encapsulation modes for an IP packet
– Transport
– Tunnel

Original IP TCP
data
IP packet header header

Transport mode IP IPsec TCP

data
protected packet header header header

Tunnel mode IP IPsec IP TCP

data
protected packet header header header header

v1.1 12
 Transport mode
• Intercept Network layer packets
Encrypt / Authenticate these packets preserving most of
the original IP header

Network

A B

Original IP header TCP header data

IP packet

Transport mode
IP header IPsec header TCP header data
protected packet

v1.1 13
 Tunnel mode
• Intercept Network layer packets
Encrypt / Authenticate these packets, while encapsulating
the original IP packet entirely

Original IP TCP
data
IP packet header header

Tunnel mode IP IPsec IP TCP

data
protected packet header header header header

• Versatile and has many deployment modes

– Host-to-host
– Host-to-router (i.e. remote access)
– Router-to-router (a.k.a. Gateway-to-gateway)

v1.1 14
 Tunnel mode
(Router-to-router / Gateway-to-gateway)

Secure Tunnel
Host Host
A B

Network
RA RB
1.1.1.1 1.1.1.2 2.2.2.1 2.3.2.2 3.3.3.1 3.3.3.2

Packet flow

Nested packet format IP header IPsec header IP header TCP header data
Src = 2.2.2.1 Src = 1.1.1.1
Dst = 2.3.2.2 Dst = 3.3.3.2

v1.1 15
 Tunnel mode
(Host-to-Router / Remote Access)

Secure Tunnel

Host Host
A B

Internet / Intranet

SGW

v1.1 16
 Tunnel mode
(Host-to-host)

• Both clients imitate SGWs

• Not much of a use… (besides for compatibility testing)

v1.1 17
 Tunnel mode (Invalid configuration)

Wrong: Tunnel 2

Tunnel 1

SGW SGW SGW SGW

v1.1 18
 Encapsulation Modes

Transport Mode Tunnel Mode

Authenticates IP payload and Authenticates entire
selected portions of IP header inner IP packet plus
AH
and IPv6 extension headers selected portions of
outer IP header
ESP Encrypts IP payload and any Encrypts inner IP packet
IPv6 extension header

Encrypts IP payload and any Encrypts inner IP

ESP with IPv6 extesion header. packet. Authenticates
authentication Authenticates IP payload but inner IP packet but no
no IP header outer IP header

v1.1 19
 Authentication Header (AH) Protocol
• Original IP packets

• Transport Mode AH
• Host-to-host
authentication

• Tunnel Mode AH
• Host-to-host
• Host-to-router (i.e.
remote access)
• Router-to-router

v1.1 20
 Encapsulating Security Payload (ESP) Protocol
• Transport Mode ESP

• Tunnel Mode ESP

v1.1 21
 IPv4 header
TCP / UDP / ICMP / IPPCP / IPsec (AH/ ESP)

Version IHL Type of Service Total Length

Identifier Flags Fragment Offset
Time To Live Protocol Header Checksum
Source Address
Destination Address
Options + Padding

32 bits

• Note the mutable and immutable fields of an IPv4 header

v1.1 22
 Selection of Protocol Modes
(Host-to-Host)

• Transport Mode
• Tunnel Mode

v1.1 23
 Selection of Protocol Modes
(Router-to-Router)

• Tunnel Mode

v1.1 24
 Selection of Protocol Modes
(Pass-through IPSec)

• Tunnel mode for gateway-to-gateway

• Transport mode / tunnel mode for host-to-host

v1.1 25
Encryption and Authentication Algorithms
• Encryption:
– DES, Triple DES
– RC5
– IDEA, Triple IDEA
– CAST
– Blowfish, Twofish
– AES (Rijndael)
• Authentication:
– HMAC-MD5-96
– HMAC-SHA-1-96

HMAC: Hash-based Message Authentication Code

MD5 / SHA-1: Hash algorithms
96: Length (in bits) of the MAC output

v1.1 26
 Key Management

• Two types:
– Manual
– Automated
• Internet Key Exchange (IKE)

v1.1 27
 IKE – layout

Port 500
UDP

Application Layer IKE IKE Application Layer

TCP/UDP TCP/UDP

IP + IPSec IPSec + IP

Data Link Data Link

AH / ESP

v1.1 28
 IKE Phases
Phase 1: Two peers authenticate each other and set up a secure
channel for subsequent communications.
• Main Mode
• Aggressive Mode
The differences between them are the number of message
flows needed and the services they provide.
Phase 2: The two peers negotiate various parameters for IPSec. They
include the base protocol, encapsulation mode, keying
materials, etc. The end result is going to be one or more
SAs.
• Quick Mode

v1.1 29
 IKE Phase 1: Main Mode
Purposes
• Authenticated key exchange for establishing the IKE SA.
• Protect the identities of the two parties.

Four keys (secret information) are to be created after phase 1:

1. SKEYID : This value will be used to create the other three secret
values.
• For signatures: SKEYID = prf(Ni | Nr, gxy )
• For public key encryption: SKEYID = prf(hash(Ni | Nr), CKY-I | CKY-R)
• For pre-shared keys: SKEYID = prf(pre-shared-key, Ni | Nr)
1. SKEYIDd: Used to derive keying material for IPSec protocols.
SKEYIDd = prf(SKEYID, gxy | CKY-I | CKY-R | 0)
3. SKEYIDa: Used to derive keys for authentication and data integrity.
SKEYIDa = prf(SKEYID, SKEYIDd | gxy | CKY-I | CKY-R | 1)
4. SKEYIDe: Used to derive keys for confidentiality.
SKEYIDe = prf(SKEYID, SKEYIDa | gxy | CKY-I | CKY-R | 2)
v1.1 30
 IKE Phase 1: Main Mode
• The Main Mode consists of six message flows (i.e. three rounds).
• First round: security parameters negotiation
• Second round: key exchange
• Third round: mutual authentication

The following values are used for authentication:

• HASHi = prf(SKEYID, gx | gy | CKY-I | CKY-R | <list of SAs> | IDi )
• This is to be the response from the initiator.
• This value or its signature will be transmitted.
• HASHr = prf(SKEYID, gx | gy | CKY-I | CKY-R | <list of SAs> | IDr )
• This is to be the response from the responder.

v1.1 31
 IKE Phase 1: Authentication Using Signatures
(Main Mode)

Initiator (Alice) Responder (Bob)

CKY-I, <list of SAs>

CKY-I, CKY-R, <chosen SA> } negotiation

CKY-I, CKY-R, gx, Ni

CKY-I, CKY-R, gy, Nr } keyexchange

CKY-I, CKY-R, {IDi, Signi, [Certi]}SKEYID e

CKY-I, CKY-R, {IDr, Signr, [Certr]}SKEYID e } authentication

• Depending on the signature scheme selected, Signi or Signr is the
corresponding signature of HASHi or HASHr respectively.
• Identities are protected using symmetric key encryption.
• Certificates are optional.
v1.1 32
 IKE Phase 1: Authentication Using Signatures
(Aggressive Mode)

Initiator (Alice) Responder (Bob)

CKY-I, <list of SAs>, gx, Ni, IDi

CKY-I, CKY-R, <chosen SA>, gy, Nr, IDr, Signr, [Certr]

CKY-I, CKY-R, Signi, [Certi]

• Only three message flows

• No identity protection

Other authentication methods defined for IKE Phase 1:

• Authentication using public key encryption
• Authentication using pre-shared keys
v1.1 33
 IKE Phase 2
(Quick Mode)

Initiator (Alice) Responder (Bob)

CKY-I, CKY-R, HASH1, <list of SAs>, Ni, [gx]

CKY-I, CKY-R, HAHS2, <chosen SA>, Nr, [gy]

CKY-I, CKY-R, HASH3

• HASH1 = prf(SKEYIDa, <list of SAs> | Ni | [gx])

• HASH2 = prf(SKEYIDa, Nr | <chosen SA> | Nr | [gy]
• HASH3 = prf(SKEYIDa, 0 | Ni | Nr)

• The optional Diffie-Hellman key exchange is for Perfect Forward Secrecy

(PFS).

v1.1 34
 Standards
• RFC2401 IPSec
• RFC2402 AH
• RFC2403 HMAC MD5
• RFC2404 HMAC SHA-1
• RFC2405 DES CBC with IV
• RFC2406 IP ESP
• RFC2407 DOI for ISAKMP
• RFC2408 ISAKMP
• RFC2409 IKE
• RFC2410 Null
• RFC2451 ESP CBC Mode

v1.1 35