IPSec VPN Negotiations - Phase 1 and 2

 A brilliant article from WatchGuard that explains the IPSec VPN Negotiation process:

About IPSec VPN Negotiations

The devices at either end of an IPSec VPN tunnel are IPSec peers. When two IPSec peers want to make a VPN between them, they exchange a series of messages about encryption and authentication, and attempt to agree on many different parameters. This process is known as VPN negotiations. One device in the negotiation sequence is the initiator and the other device is the responder.
VPN negotiations happen in two distinct phases: Phase 1 and Phase 2.

Phase 1

The main purpose of Phase 1 is to set up a secure encrypted channel through which the two peers can negotiate Phase 2. When Phase 1 finishes successfully, the peers quickly move on to Phase 2 negotiations. If Phase 1 fails, the devices cannot begin Phase 2.

Phase 2

The purpose of Phase 2 negotiations is for the two peers to agree on a set of parameters that define what traffic can go through the VPN, and how to encrypt and authenticate the traffic. This agreement is called a Security Association.

The Phase 1 and Phase 2 configurations must match for the devices on either end of the tunnel.

Phase 1 Negotiations

In Phase 1 negotiations, the two peers exchange credentials. The devices identify each other and negotiate to find a common set of Phase 1 settings to use. When Phase 1 negotiations are completed, the two peers have a Phase 1 Security Association (SA). This SA is valid for only a certain amount of time. After the Phase 1 SA expires, if the two peers must complete Phase 2 negotiations again, they must also negotiate Phase 1 again.
Phase 1 negotiations include these steps:

1. The devices exchange credentials.

The credentials can be a certificate or a pre-shared key. Both gateway endpoints must use the same credential method. If one peer uses a pre-shared key, the other peer must also use a pre-shared key, and the keys must match. If one peer uses a certificate, the other peer must also use a certificate.

2. The devices identify each other.

Each device provides a Phase 1 identifier, which can be an IP address, domain name, domain information, or an X500 name. The VPN configuration on each peer contains the Phase 1 identifier of the local and the remote device, and the configurations must match.

3. The peers decide whether to use Main Mode or Aggressive Mode.

Phase 1 negotiations can use one of two different modes: Main Mode or Aggressive Mode. The device that starts the IKE negotiations (the initiator) sends either a Main Mode proposal or an Aggressive Mode proposal. The responder can reject the proposal if it is not configured to use that mode. Aggressive Mode is less secure but faster than Main Mode.

When you use Aggressive mode, the number of exchanges between two endpoints is fewer than it would be if you used Main Mode, and the exchange relies mainly on the ID types used in the exchange by both appliances. Aggressive Mode does not ensure the identity of the peer. Main Mode ensures the identity of both peers, but can only be used if both sides have a static IP address.

4. The peers agree on Phase 1 parameters.
   • Whether to use NAT traversal
   • Whether to send IKE keep-alive messages (supported between Firebox or XTM devices only)
   • Whether to use Dead Peer Detection (RFC 3706)
5. The peers agree on Phase 1 Transform settings.

Transform settings include a set of authentication and encryption parameters, and the maximum amount of time for the Phase 1 SA. The settings in the Phase 1 transform must exactly match a Phase 1 transform on the IKE peer, or IKE negotiations fail.

The items you can set in the transform are:

• Authentication — The type of authentication (SHA1 or MD5).
• Encryption — The type of encryption algorithm (DES, 3DES or AES).
• SA Life — The amount of time until the Phase 1 Security Association expires.
• Key Group — The Diffie-Hellman key group.

Phase 2 Negotiations

After the two IPSec peers complete Phase 1 negotiations, Phase 2 negotiations begin. The purpose of Phase 2 negotiations is to establish the Phase 2 SA (sometimes called the IPSec SA). The IPSec SA is a set of traffic specifications that tell the device what traffic to send over the VPN, and how to encrypt and authenticate that traffic. In Phase 2 negotiations, the two peers agree on a set of communication parameters. When you configure the BOVPN tunnel in Policy Manager or in Fireware XTM Web UI, you specify the Phase 2 parameters.
Because the peers use the Phase 1 SA to secure the Phase 2 negotiations, and you define the Phase 1 SA settings in the BOVPN Gateway settings, you must specify the gateway to use for each tunnel.
Phase 2 negotiations include these steps:

1. The peers use the Phase 1 SA to secure Phase 2 negotiations.

Phase 2 negotiations can only begin after Phase 1 SA has been established.

2. The peers exchange Phase 2 identifiers (IDs).

Phase 2 IDs are always sent as a pair in a Phase 2 proposal: one indicates which IP addresses behind the local device can send traffic over the VPN, and the other indicates which IP addresses behind the remote device can send traffic over the VPN. This is also known as a tunnel route. You can specify the Phase 2 IDs for the local and remote peer as a host IP address, a network IP address, or an IP address range.

3. The peers agree on whether to use Perfect Forward Secrecy (PFS).

PFS specifies how Phase 2 keys are derived. When PFS is selected, both IKE peers must use PFS, or Phase 2 rekeys fail. PFS guarantees that if an encryption key used to protect the data transmission is compromised, an attacker can access only the data protected by that key, not subsequent keys. If the peers agree to use PFS, they must also agree on the Diffie-Hellman key group to use for PFS.

4. The peers agree on a Phase 2 proposal.

The Phase 2 proposal includes the IP addresses that can send traffic over the tunnel, and a group of encryption and authentication parameters. Fireware XTM sends these parameters in a Phase 2 proposal. The proposal includes the algorithm to use to authenticate data, the algorithm to use to encrypt data, and how often to make new Phase 2 encryption keys.

The items you can set in a Phase 2 proposal include:

Type

For a manual BOVPN, you can select the type of protocol to use: Authentication Header (AH) or Encapsulating Security Payload (ESP). ESP provides authentication and encryption of the data. AH provides authentication without encryption. We recommend you select ESP. Managed BOVPN and Mobile VPN with IPSec always use ESP.

Authentication

Authentication makes sure that the information received is exactly the same as the information sent. You can use SHA or MD5 as the algorithm the peers use to authenticate IKE messages from each other. SHA1 is more secure.

Encryption

Encryption keeps the data confidential. You can select DES, 3DES, or AES. AES is the most secure.

Force Key Expiration

To make sure Phase 2 encryption keys change periodically, always enable key expiration. The longer a Phase 2 encryption key is in use, the more data an attacker can collect to use to mount an attack on the key.