IPsec/IKE Protocol Overview

Jump to: navigation, search

Table of Contents >> IPsec/IKE Programmer's Reference

Authentication Header (AH)

AH comes after the basic IP header and contains cryptographic hashes of the data and identification information. It is used to provide data integrity, data origin authentication, and optional anti-replay services to IP.

AH has been assigned the IP protocol number 51. The header format of AH is shown in Figure 1.

Figure 1: AH Header Format

Figure 1: AH Header Format

Figure 2 illustrates AH positioning for a typical IP packet, on a before and after basis, for both transport mode and tunnel mode. (a) is for IPv4 packet, and (b) is for IPv6 packet

Figure 2: AH Position in IPv4 and IPv6

Figure 2: AH Position in IPv4 and IPv6

For either transport mode or tunnel mode, the AH header authenticates the whole IP packet except for the mutable fields in outside IP header. For the definition of mutable and immutable fields, please refer to RFC 2401 and RFC 2402.


Treck AH provides the following features:

  • Supports HMAC-MD5-96, HMAC-SHA1-96, HMAC-SHA-256-128, HMAC-SHA-256-96, HMACSHA-384-192, HMAC-SHA-384-96, HMAC-SHA-512-256, HMAC-SHA-512-96, HMACRIPEMD160-96, and NULL authentication algorithm
  • Tunnel and transport mode
  • Nested tunnel
  • SA bundle (with ESP)
  • Easy to add or remove algorithms
  • Supports both IPv4 and IPv6.

Encapsulating Security Payload (ESP)

Figure 3: ESP Header Format

Figure 3: ESP Header Format

ESP allows rewriting of the payload in encrypted form. It provides confidentiality, data origin authentication, anti-replay, and data integrity services to IP. ESP does not apply to IP header fields preceding it.

ESP has been assigned the IP protocol number 50. If a NULL encryption algorithm is used, there is no 'Initial Vector' field. Additionally, if a NULL authentication algorithm is used, there is no 'Authentication Data' field. However, encryption algorithm and authentication algorithm may not both be NULL. The padding length may be from 0 to 255 bytes. However, Treck IPsec, like most others, uses minimum padding.

Figure 4 illustrates ESP positioning for a typical IP packet on a 'before-and-after basis' for both transport and tunnel mode. Diagram (a) is for IPv4 packet and (b) is for IPv6 packet.


Figure 4: ESP Position in IPv4 and IPv6

Figure 4: ESP Position in IPv4 and IPv6

Treck's ESP provides the following features:

  • Supports NULL, DES, 3DES (Both DES and 3DES has configurable weak key checking and parity bit checking), BLOWFISH (key size 40-448 bits), CAST128 (key size 40-128 bits), RC5 (key size 40-2040 bits), IDEA (key size 128 bits), AES (RIJNDAEL, key size 128, 192, 256 bits), TWOFISH (key size 128,192, 256 bits) algorithm and in CBC mode.
  • Supports HMAC-MD5-96, HMAC-SHA1-96, HMAC-SHA-256-128, HMAC-SHA-256-96, HMACSHA-384-192, HMAC-SHA-384-96, HMAC-SHA-512-256, HMAC-SHA-512-96, HMACRIPEMD160-96, and NULL authentication algorithm
  • Tunnel and transport mode
  • Nested tunnel
  • SA bundle (with AH)
  • Minimum padding
  • Easy to add or remove algorithms
  • Supports IPv4 and IPv6

Internet Key Exchange (IKE)

IKE negotiates the SA properties between peers. IKE includes two phases. There are two modes for the phase one negotiation, main mode and aggressive mode, and only one mode for phase two, quick mode.

The phase one negotiation results in a secure channel, i.e. ISAKMP SA, between peers. All the following negotiations will be protected in this channel. IKE phase two negotiates IPsec SA properties.

Phase One Negotiation Attributes

IKE is a negotiation protocol. Treck IKE supports the following attributes, per RFC 2409.

The phase 1 protocol identifier is TM_DOI_PROTO_ISAKMP. The transform identifier is TM_DOI_KEY_IKE.


Encryption algorithms

TM_IKE_DES_CBC 1
TM_IKE_IDEA_CBC 2
TM_IKE_BLOWFISH_CBC 3
TM_IKE_RC5_R16_B64_CBC 4
TM_IKE_3DES_CBC 5
TM_IKE_CAST_CBC 6
TM_IKE_AES_CBC 7
TM_IKE_TWOFISH_CBC 13


Note Note: The above algorithms are specified in RFC 2409. AES is specified in RFC 3602. For Treck IKE, TWOFISH interoperability with other implementations is not guaranteed.


Hash algorithms

TM_IKE_MD5 1 MD5 hash algorithm
TM_IKE_SHA1 2 SHA1 hash algorithm
TM_IKE_SHA256 4 SHA256 hash algorithm
TM_IKE_SHA384 5 SHA384 hash algorithm
TM_IKE_SHA512 6 SHA512 hash algorithm
TM_IKE_RIPEMD 65001 RIPEMD hash algorithm
Note Note: The Tiger Hash algorithm is not supported by Treck IKE.


Authentication methods

TM_IKE_PRESHARED_KEY 1 pre-shared key
TM_IKE_DSS_SIG 2 DSA signature
TM_IKE_RSA_SIG 3 RSA signature


Diffie-Hellman Groups

TM_DHGROUP_1 1 768 MODP group
TM_DHGROUP_2 2 1024 MODP group
TM_DHGROUP_5 5 1536 MODP group
TM_DHGROUP_14 14 2048 MODP group


Lifetime Type

TM_IKE_LIFETYPE_SECONDS 1 life time in seconds
TM_IKE_LIFETYPE_KBYTES 2 life time in kilo-bytes


Lifetime Duration

Treck IKE requires that the SA life duration must be either two octets (0 to 18.2 hours of lifetime for AF bit set case) or four octets (0 to 136 years, AF bit not set) long. No other data length is supported.


Pseudo-Random Function

According to RFC 2409 Treck IKE uses the HMAC version of the negotiated hash algorithm to be the PRF. No other PRF is supported.


Key Length

Some encryption algorithms may accept different size of key. For Treck IKE, the following key size range is supported. And as RFC recommended, Treck IKE doesn’t support key length that is not of multiple of 8 Bits (1 octet byte). For those algorithms with fixed key length, no key-length attribute is accepted. (RFC 2409 prohibits such behavior. This attribute MUST NOT be used when the specified encryption algorithm uses a fixed length key.)

TM_IKE_DES_CBC 64 Bits
TM_IKE_BLOWFISH_CBC 40 to 448 Bits
TM_IKE_RC5_R16_B64_CBC 40 to 2040 Bits
TM_IKE_3DES_CBC 192 Bits
TM_IKE_CAST_CBC 40 to 128 Bits
TM_IKE_AES_CBC 128, 192, or 256 Bits
TM_IKE_TWOFISH_CBC 128, 192, or 256 Bits

Phase Two Negotiation Attributes

For phase 2, we support the following attributes, per RFC 2407.

Protocol Identifier and Transform Identifier

Treck IKE supports AH and ESP. IP Compression (IPCOMP) is not currently supported.

TM_DOI_PROTO_IPSEC_AH 2
TM_DOI_PROTO_IPSEC_ESP 3


For AH, the following transform identifiers are supported.

TM_DOI_AH_MD5 2 HMAC_MD5_96
TM_DOI_AH_SHA 3 HMAC_SHA1_96
TM_DOI_AH_SHA2_256 5 HMAC_SHA2_256_128/96
TM_DOI_AH_SHA2_384 6 HMAC_SHA2_384_192/96
TM_DOI_AH_SHA2_512 7 HMAC_SHA2_512_256/96
TM_DOI_AH_RIPEMD 8 HMAC_RIPEMD160_96


For ESP, the following transform identifiers are supported.

TM_DOI_ESP_DES_IV64 1
TM_DOI_ESP_DES 2
TM_DOI_ESP_3DES 3
TM_DOI_ESP_RC5 4
TM_DOI_ESP_IDEA 5
TM_DOI_ESP_CAST128 6
TM_DOI_ESP_BLOWFISH 7
TM_DOI_ESP_NULL 11
TM_DOI_ESP_AES 12
TM_DOI_ESP_TWOFISH 13
Note Note: When TWOFISH is used interoperability is not guaranteed with non-Treck implementations.


Lifetime Type

TM_DOI_SATOL_LIFE_SECONDS 1 life time in seconds
TM_DOI_SATOL_LIFE_SECONDS 2 life time in kilo-bytes


Lifetime Duration

Treck IKE requires that the SA life duration must be either 2 octets (0 to 18.2 hours of lifetime for AF bit set case) or 4 octets (0 to 136 years, AF bit not set) long. No other data length is supported.


Diffie-Hellman Group Description

TM_DHGROUP_1 1 768 MODP group
TM_DHGROUP_2 2 1024 MODP group
TM_DHGROUP_5 2 1536 MODP group
TM_DHGROUP_14 2 2048 MODP group


Treck supports the predefined Diffie-Hellman MODP group 1, 2, 5, and 14. Treck IKE only supports predefined MODP groups. Group 3, group 4 and custom groups are not supported.


Encapsulation Mode

TM_DOI_ENCAPMODE_TUNNEL 1
TM_DOI_ENCAPMODE_TRANSPORT 2

Transport mode and tunnel mode are both supported.


Authentication Algorithms

TM_DOI_AUTHALG_HMAC_MD5 1
TM_DOI_AUTHALG_HMAC_SHA_1 2
TM_DOI_AUTHALG_HMAC_SHA2_256 5
TM_DOI_AUTHALG_HMAC_SHA2_384 6
TM_DOI_AUTHALG_HMAC_SHA2_384 7
TM_DOI_AUTHALG_HMAC_RIPEMD 8

Treck IKE supports the above authentication algorithms.


Key Length

Some encryption algorithms may accept a different size key. For Treck IKE, the following key size range is supported. As the RFC's recommended, Treck IKE does not support key lengths that are not a multiple of 8 Bits (1 octet byte). For those algorithms with a fixed key length, a no key-length attribute is accepted. (RFC 2409 prohibits such behavior. This attribute MUST NOT be used when the specified encryption algorithm uses a fixed key length.)

TM_DOI_ESP_DES_IV64 64 Bits
TM_DOI_ESP_DES 64 Bits
TM_DOI_ESP_3DES 192 Bits
TM_DOI_ESP_RC5 40 to 2040 Bits
TM_DOI_ESP_CAST 40 to 128 Bits
TM_DOI_ESP_BLOWFISH 40 to 448 Bits
TM_DOI_ESP_NULL N/A
TM_DOI_ESP_AES 128, 192, or 256 Bits
TM_DOI_ESP_TWOFISH 128, 192, or 256 Bits
Note Note: Treck IKE supports TWOFISH, but interoperation is not guaranteed with non-Treck implementations.


Key Rounds

For some encryption algorithms, a user defined ROUND value may be used. However, it affects the interoperation. Treck always uses the default round value. For example, RC5, Blowfish, Cast algorithm use 16 rounds.

Treck IKE features

Treck IKE supports the following features:

  • Supports main mode, aggressive mode, and quick mode.
  • Supports Pre-shared Key, DSA and RSA signatures authentication methods
  • Supports Diffie-Hellman predefined group 1, 2, 5 and 14.
  • Support ID types - IPv4 and IPv6 Address, Subnet and Range, FQDN, USER_FQDN, and ASN1_DN
  • Supports encryption algorithm DES, 3DES, BLOWFISH, CAST, AES (RIJNDAEL) and TWOFISH.
  • Supports Perfect Forward Secrecy. If Treck IKE is the initiator for phase two, it always uses the phase one Diffie-Hellman group as the PFS proposal. If Treck IKE is the responder, it accepts group 1, 2, 5 or 14.
  • Supports Hash algorithms MD5, SHA1, SHA256, SHA384, SHA512 and RIPEMD. The corresponding HMAC version hash algorithm is used as the Pseudo Random Function.
  • Ignores commit bit and auth-only bit based on IKE bake-offs. Uses pre-setup incoming SA to avoid losing packet. Unique message ID list is maintained to work against replay.
  • Cookie is generated using continuously refreshing secret key. Cookie verification is configurable.
  • Lifetime is based on time and kilobytes.
  • New group mode is not supported.
  • Supports ISAKMP Informational Exchanges, including DELETE, INITIAL-CONTACT message. Other informational messages will be silently discarded. (RFC states we MAY send an error notification.) We do not send RESPONDER_LIFETIME exchange because any participant may choose to renegotiate at its preferred time.
  • One SA negotiation at each quick mode exchange
  • For the pair exchanges, the requester is responsible for retransmit. The responder will retransmit only upon receiving a retransmitted message from the requester.
  • Supports both IPv4 and IPv6.
  • Packets are queued when negotiating SA, while time and bytes limitation apply.
  • Once connection is established, Treck IKE will try to keep continuous channel between peers. Phase 1 ISAKMP SA will be automatically re-keyed once expires.

Table of Contents >> IPsec/IKE Programmer's Reference