Five Steps of IPSec Revisited

Step 1—Determine Interesting Traffic

Data communications covers a wide gamut of topics, sensitivity, and security requirements. Just as all rumors you hear aren’t worth repeating, often much network traffic isn’t worth securing, such as an employee’s personal web browsing. Any security measures can come at the expense of network resources and performance, so a method must exist to determine what traffic is protected and what traffic travels normally through the public network.

The VPN security policy generally determines which type of traffic is considered “interesting” and, therefore, protected. How the VPN security policy gets implemented depends on the device platform. Cisco routers and the PIX Firewall use access lists to define the traffic to secure. The access lists are then incorporated in a crypto policy, which causes traffic associated with permit statements to be encrypted, while traffic associated with deny statements is sent unencrypted.

Cisco VPN Client and some VPN management software tools allow the use of menu windows to define connections to be secured by IPSec.

Both methods of identifying interesting traffic are covered in Chapters 10 through 16.

Step 2—IKE Phase One

IKE is a key management protocol standard used in conjunction with IPSec. While IPSec can be configured without IKE, the use of IKE enhances the IPSec with additional features and makes it scalable. IKE authenticates each peer in an IPSec session, automatically negotiates two levels of SAs, and handles the exchange of session keys. This is accomplished in two phases: Phase One and Phase Two.

IKE Phase One’s main purpose is to authenticate the IPSec peers and to set up a secure channel between the peers. To accomplish this, IKE Phase One performs the following functions:

• Authenticates while protecting the identities of the IPSec peers

• Negotiates a common IKE SA policy between peers to protect the IKE exchange

• Performs a DH exchange to create shared secret keys

• Sets up a secure tunnel to negotiate IKE Phase Two parameters

IKE Phase One occurs in two modes: Main mode and Aggressive mode.

Peer Authentication

Would-be IPSec peers must authenticate themselves to each other before IKE can proceed. IKE Phase One has three methods to authenticate IPSec peers in Cisco products. The two peers must negotiate a common authentication protocol from the following choices:

• Preshared keys—A key value entered into each peer manually (out of band) and used to authenticate the peer.

• RSA signatures—Uses a digital certificate authenticated by an RSA signature.

• RSA encrypted nonces—Uses RSA encryption to encrypt a nonce value (a random number generated by the peer) and other values.

A common value used by all authentication methods to help identify the peer is the peer identity (ID). Some ID values include the peer’s IP address or their FQDN, such as ian.testco.com.

Preshared Key Authentication

With this method, the same preshared key is configured on each IPSec peer. These IKE peers can authenticate each other by generating a hash of their ID, plus the key appended to the ID before transmission. If the receiving peer can create the same hash value using its preshared key, it authenticates the source peer. Figure 9-18 shows the preshared key authentication exchange.

Figure 9-18: Preshared key authentication exchange

While preshared keys are easier to configure than configuring the IPSec policy values on each IPSec peer, they don’t scale well and potential security risks can be associated with getting the key to the peer.

RSA Signature Authentication

RSA Signature Authentication is a public-key cryptosystem supported by IPSec for IKE Phase One authentication. This technology was developed in 1977 by Ron Rivest, Adi Shamir, and Leonard Adleman. RSA is the first letter of each developer’s last name.

RSA signatures use a CA to generate a unique digital-identity certificate for each peer for authentication. The digital-identity certificate is similar in function to the pre-shared key, but provides stronger security.

To authenticate for an IKE session, the initiator and responder send each other their ID values, identity digital certificates, and an RSA signature value consisting of various IKE values, all encrypted using the negotiated IKE encryption method (DES or 3DES). Figure 9-19 shows an RSA signature authentication exchange.

Figure 9-19: RSA signature authentication exchange

RSA Encryption Authentication

The RSA-encrypted nonces authentication method uses the RSA encryption public key cryptography algorithm. This technology requires that each party generate a pseudorandom number (a nonce) and encrypt it (and possibly other publicly and privately available information), using the other party’s RSA public key. Authentication occurs when each party decrypts the other party’s nonce with its local private key, and then uses the decrypted nonce to compute a keyed hash.

A drawback to this method is this: either side of the exchange can plausibly deny that it took part in the exchange. Cisco IOS software is the only Cisco product that supports this authentication method. Figure 9-20 shows an RSA encryption authentication exchange.

Figure 9-20: RSA encryption authentication exchange

CAs and Digital Certificates

CAs and Digital Certificates are covered in greater detail in Chapter 11, but for our purposes here, they represent a digital identification system whereby an independent third party vouches for them. Conceptually, this is similar to most state driver’s licenses when used as identification. The merchant accepting the ID accepts that the state has performed adequate verification to be reasonably sure the identity is valid.

The distribution of security keys through an untrusted network, such as the Internet, involves questionable levels of risk.

CAs are trusted third-party organizations, such as Verisign, Entrust, and Netscape, which provide digital certificates. The peers wanting to use digital certificates register with a CA. Once the CA verifies the client’s credentials, a digital certificate is issued. The digital certificate contains the certificate bearer’s identity (name or IP address), the certificate serial number, the certificate expiration date, and a copy of the certificate bearer’s public key.

The digital certificate standard format is defined in the X.509 specification. Cisco supports X.509 version 3.

Step 3—IKE Phase Two

IKE Phase Two has only one mode, Quick mode, which occurs after IKE has established the secure tunnel in Phase One. In Quick mode, IKE

• Negotiates a shared IPSec policy

• Establishes IPSec SAs

• Derives shared secret keys used for the IPSec security algorithms

To negotiate the IPSec SAs, the sender forwards one or more transformsets, including transform combinations and related settings that represent the sender’s security requirements/preferences for the new IPSec session. The receiving peer compares these requirements to its own transform sets (requirements/preferences). If one matches, then the recipient returns that single transform set, indicating a mutually agreed on transform and algorithms for the IPSec session.

Quick mode is also used to renegotiate a new IPSec SA any time the IPSec SA lifetime expires. If PFS (next section) isn’t specified, Quick mode refreshes the key generation material used to create the shared secret keys derived from the DH exchange in Phase One.

Step 4—IPSec Data Transfer

Information is exchanged via the IPSec session based on the method for defining interesting traffic. Packets are encrypted and decrypted at the IPSec peers using any encryption specified in the IPSec SA.

Step 5—Session Termination

The IPSec session can be terminated because the traffic ended and the IPSec SA was deleted or the SA can time –out based on either SA lifetime setting. The SA timeout can be after a specified number of seconds or a specified number of bytes passed through the connection. The keys are discarded when SAs terminate, requiring IKE to perform a new Phase Two and, possibly, a new Phase One negotiation. New SAs can be established before the current ones expire, maintaining uninterrupted data flows.