IPSec Process Flow

In order to setup an IPSec connection you must firstly establish a secure channel with the help of an IKE (Internet Key Exchange) policy. IKE itself relies upon ISAKMP (Internet Security Association and Key Management Protocol) and defines how it should establish the secure connection e.g. type of encryption, authentication, timeouts and so on.

The main attributes attached to an IKE Policy are:

- Encryption Algorithm (e.g. DES, 3DES, AES)
- Hash Algorithm (MD5, SHA1, SHA256)
- Authentication Method: (Pre-shared key or certificate)
- DF (Diffie-helman group): (1,2,5 and so on)
- Lifetime (in seconds - 86400 by default)

During this phase Diffie-helman (asymetric) encryption is used to secure communication and once complete the symmetric keys are used (to save CPU time.) Once this phase (aka IKE Phase 1) has successfully completed a 'security association' / SA is formed.

The next step is too define establish the IPSec SA (aka IKE Phase 2) - by creating 'transform-set's' we are able to define the encryption and hashing algorithms that we will use for this phase. Although unlike phase 1 the phase 2 SA's are unidirectional - meaning that you must explicitly define the hashing and encryption options within transform sets.

After implementing an IPSec tunnel the ISAKMP SA will be established immediately, although the IPSec SA will only be established when some "interesting traffic" (traffic that is destined to go over the tunnel) traverses the tunnel.

Both Authentication Header (AH) and ESP (Encapsulating Security Protocol) require IKE to function correctly. AH provides data integrity by encapsulating an IP packet (which itself operates on Protocol 51) and performing a hash computation on the whole IPv4 packet (minus and mutable fields in the IP packet header) using the keys only known to the hosts within the IPSec SA - hence providing data integrity and authenticity of the source. ESP also provides it's own authentication and encryption service - as AH alone will not encrypt any data! Typically data will be encrypted using ESP and then wrapped by AH.