Authentication and Identity Protocols

In the wireless world, you need to ascertain the identity of the users (and devices) using authentication mechanisms. This is important because access control is established depending on the user's identity. For example, users belonging to an organization would be allowed access to its internal network, whereas guests would only be allowed Internet access.


In many environments, after you establish a peer-to-peer link-level connection, additional access control mechanisms you can sometimes deploy at higher levels of the protocol stack?for example, permitting access to hosts with certain IP addresses accessing specific applications. Also, you can assign virtual LANs and subnets to users depending on their identity.

The following sections look at, in brief, some protocols that are relevant to the wireless domain.

PPP Authentication Protocols

Passwords are incorporated into many protocols that provide authentication services. For dial-in connections, the Point-to-Point Protocol (PPP) is most often used to establish a dial-in connection over serial lines or ISDN. PPP authentication mechanisms include the Password Authentication Protocol (PAP), the Challenge Handshake Authentication Protocol (CHAP), and the Extensible Authentication Protocol (EAP). In all these cases, the user of the device is being authenticated rather than the peer device.


PPP is relevant to the wireless domain because the evolution of PPP authentication protocols lead to EAP, which is the basis for many WLAN security protocols.

PPP is a standardized Internet encapsulation of IP over point-to-point links. PPP addresses issues such as assignment and management of IP addresses, asynchronous (start/stop) and bit-oriented synchronous encapsulation, network protocol multiplexing, link configuration, link quality testing, error detection, and option negotiation for such capabilities as network-layer address negotiation and data compression negotiation. PPP addresses these issues by providing an extensible Link Control Protocol (LCP) and a family of Network Control Protocols (NCPs) to negotiate optional configuration parameters and facilities. After the link has been established, PPP provides for an optional authentication phase before proceeding to the network-layer protocol phase.

The PPP protocol data unit (PDU) uses the high-level data link control (HDLC) frame as stipulated in ISO 3309-1979 (and amended by ISO 3309-1984/PDAD1).

PPP negotiation consists of LCP and NCP negotiation. LCP is responsible for establishing the connection with certain negotiated options, maintaining the connection, and providing procedures to terminate the connection. To perform these functions, LCP is organized into the following four phases:

  1. Link establishment and configuration negotiation

  2. Link quality determination

  3. Network-layer protocol configuration negotiation

  4. Link termination

To establish communications over a point-to-point link, each end of the PPP link must first send LCP packets to configure the data link during the link establishment phase. After the link has been established, PPP provides for an optional authentication phase before proceeding to the network-layer protocol phase. The NCP phase then establishes and configures different network layer protocols such as IP.

By default, authentication before the NCP phase is not mandatory. If authentication of the link is desired, an implementation will specify the authentication protocol configuration option during the link establishment phase. These authentication protocols are intended for use primarily by hosts and routers that connect to a PPP network server through switched circuits or dial-up lines, but they also can be applied to dedicated links. The server can use the identification of the connecting host or router in the selection of options for network layer negotiations.

PPP Password Authentication Protocol

The Password Authentication Protocol (PAP) provides a simple way for a peer to establish its identity to the authenticator using a two-way handshake. This is done only at initial link establishment. The three PAP frame types are shown in Figure 2-9.

Figure 2-9. Three PPP PAP Frame Types

After the link establishment phase is completed, the authenticate-request packet is used to initiate the PAP authentication. This packet contains the peer name and password, as shown in Figure 2-10.

Figure 2-10. PPP PAP Authentication Request

This request packet is sent repeatedly until a valid reply packet is received or an optional retry counter expires. If the authenticator receives a Peer-ID/Password pair that is both recognizable and acceptable, it should reply with an Authenticate-Ack (where Ack is short for acknowledge). If the Peer-ID/Password pair is not recognizable or acceptable, the authenticator should reply with an Authenticate-Nak (where Nak is short for negative acknowledge).


Most implementations of PPP only allow for a single attempt at a correct Peer-ID/Password.

Figure 2-11 shows the sequence of PPP negotiations for a branch router (the peer) trying to authenticate to the network access server (NAS), which is the authenticator.

Figure 2-11. PPP PAP Authentication

PAP is not a strong authentication method. PAP authenticates only the peer, and passwords are sent over the circuit "in the clear." There is no protection from replay attacks or repeated trial-and-error attacks. The peer is in control of the frequency and timing of the attempts. As PAP assumes, there is no chance of "wiretapping"; this renders it unfit for use in a wireless setting.

PPP Challenge Handshake Authentication Protocol

The Challenge Handshake Authentication Protocol (CHAP) is used to periodically verify the identity of a host or end user using a three-way handshake. CHAP is performed at initial link establishment and can be repeated any time after the link has been established. The four CHAP frame types are shown in Figure 2-12.

Figure 2-12. PPP CHAP Frame Types

Figure 2-13 shows a scenario in which a branch router (the peer) tries to authenticate to the NAS (the authenticator).

Figure 2-13. PPP CHAP Authentication

CHAP imposes network security by requiring that the peers share a plain text secret. This secret is never sent over the link. The secret is exchanged or installed out of band. (In Figure 2-13, the string "trustme" is this shared secret. The following sequence of steps is carried out:

Step 1.
After the link establishment phase is complete, the authenticator sends a challenge message to the peer. The challenge consists of an identifier (ID), a random number, and either the hostname of the local device or the name of the user on the remote device.

Step 2.
The receiving peer calculates a hash value of the random number using a one-way hash function; the secret (in this scenario, the string "trustme") is the input to the one-way hash function.

Step 3.
The peer sends the challenge response, which consists of the following:

- An encrypted version of the ID

- A secret password (the calculated hash value [the string "0A483F6D" in this scenario])

- The random number

- Either the host name of the remote device or the name of the user on the remote device

Step 4.
When the authenticator receives the challenge response, it verifies the secret by looking up the name given in the response and performing the same encryption operation. The authenticator checks the response against its own calculation of the expected hash value.

Step 5.
If the values match, the authenticator acknowledges the authentication and sends a success message, and the LCP establishes the link.

As the figure indicates, the shared secret is never transmitted over the wire?just the response to the challenge as a hash value. The secret passwords must be identical on the remote and local devices. These secrets should be agreed on, generated, and exchanged out of band in a secure manner. Because the secret is never transmitted, other devices are prevented from stealing it and gaining illegal access to the system. Without the proper response, the remote device cannot connect to the local device.


It is important to understand that CHAP is not compatible with some authentication implementations based on having the password in cleartext or having access to a common hash function.

Also, CHAP is vulnerable to offline brute-force/dictionary attacks. That is, the challenge and response can be captured and eventually will give away enough data to recover the secret (for example, via an offline dictionary attack).

CHAP provides protection against playback attack through the use of an incrementally changing identifier and a variable challenge value. The use of repeated challenges is intended to limit the time of exposure to any single attack. The authenticator is in control of the frequency and timing of the challenges.


Typically, MD5 is used as the CHAP one-way hash function; it is required that the shared secrets be stored in plain-text form. Microsoft has a variation of CHAP (MS-CHAP) in which the password is stored encrypted in both the peer and the authenticator. Therefore, MS-CHAP can take advantage of irreversibly encrypted password databases that are commonly available, whereas the standards-based CHAP cannot.

PPP Extensible Authentication Protocol

The PPP Extensible Authentication Protocol (EAP) is a general protocol for PPP authentication that supports multiple authentication mechanisms. EAP does not select a specific authentication mechanism at the link control phase; rather, it postpones this until the authentication phase so that the authenticator can request more information before determining the specific authentication mechanism. This arrangement also permits the use of a "back-end" server, which actually implements the various authentication mechanisms, whereas the PPP authenticator merely passes through the authentication exchange.

Figure 2-14 shows how PPP EAP works.

Figure 2-14. PPP EAP Authentication

In the figure, the branch router (the peer) is trying to authenticate to the NAS (the authenticator). The sequence of steps is as follows:

Step 1.
When the link establishment phase is complete, the authenticator sends one or more requests to authenticate the peer. The request has a type field to indicate what is being requested. Examples of request types include identity, MD5-challenge, S/Key, generic token card, and so on. The MD5-challenge type corresponds closely to the CHAP authentication protocol.


Typically, the authenticator sends an initial identity request followed by one or more requests for authentication information. However, an initial identity request is not required and can be bypassed in cases in which the identity is presumed (for example, with leased lines, dedicated dial-ups, and so on).

Step 2.
The peer sends a response packet in reply to each request. As with the request packet, the response packet contains a type field that corresponds to the type field of the request.

Step 3.
The authenticator ends the authentication phase with a success or failure packet.

EAP adds more flexibility to PPP authentication and provides the capability to use new technologies?such as digital certificates?when they become widely available (hence, the "Extensible" in the protocol name).

The TACACS+ Protocol

The TACACS+ protocol is the latest generation of the Terminal Access Controller Access Control System (TACACS). TACACS is a simple UDP-based access control protocol originally developed by BBN for the MILNET.


Integration with enterprise authentication, authorization, and accounting (AAA) is an important part of WLAN security. The EAP protocols, TACACS, and RADIUS are all part of an integrated WLAN security solution, and that is why these topics are covered in this chapter. Later chapters cover the security protocols in more detail.

Cisco has enhanced (extended) TACACS several times, and the Cisco implementation, based on the original TACACS, is referred to as XTACACS. The fundamental differences between TACACS, XTACACS, and TACACS+ are as follows:

  • TACACS?Combined authentication and authorization process.

  • XTACACS?Separated AAA.

  • TACACS+?XTACACS with extended attribute control and accounting. This is the most common implementation.

TACACS+ uses TCP for its transport. The server daemon usually listens at port 49, the login port assigned for the TACACS protocol. This port is reserved in the assigned number's RFC for both UDP and TCP. Current TACACS and extended TACACS implementations also use port 49.

TACACS+ is a client/server protocol; the TACACS+ client is typically a NAS, and the TACACS+ server is usually a daemon process running on some UNIX or NT machine. A fundamental design component of TACACS+ is the separation of AAA.

TACACS+ Authentication

TACACS+ allows for arbitrary length and content authentication exchanges, which allow any authentication mechanism to be used with TACACS+ clients (including PPP PAP, PPP CHAP, PPP EAP, token cards, and Kerberos). Authentication is not mandatory; it is a site-configured option. Some sites do not require it at all; others require it only for certain services.

TACACS+ authentication has three packet types:

  • START, which is always sent by the client

  • CONTINUE, which is always sent by the client

  • REPLY, which is always sent by the server

Authentication begins with the client sending a START message to the server. The START message describes the type of authentication to be performed (for example, simple cleartext password, PAP, or CHAP) and might contain the username and some authentication data. The START packet is sent only as the first message in a TACACS+ authentication session or as the packet immediately following a restart. (A restart might be requested by the server in a REPLY packet.) A START packet always has a sequence number equal to 1.

In response to a START packet, the server sends a REPLY. The REPLY message indicates whether the authentication is finished or should continue. If the REPLY indicates that authentication should continue, the message also indicates what new information is requested. The client gets that information and returns it in a CONTINUE message. This process repeats until all authentication information is gathered, and then the authentication process concludes.

TACACS+ Authorization

Authorization is the action of determining what a user is allowed to do. Generally, authentication precedes authorization, but this is not required. An authorization request might indicate that the user is not authenticated (that is, it is not known who he or she is). In this case, it is up to the authorization agent to determine whether an unauthenticated user is allowed the services in question.

When authentication is completed (if authentication is used), the client can start the authorization process, if authorization is required. An authorization session is defined as a single pair of messages: a REQUEST followed by a RESPONSE. The authorization REQUEST message contains a fixed set of fields that describes the authenticity of the user or process, and a variable set of arguments that describes the services and options for which authorization is requested.


In TACACS+, authorization does not merely provide yes or no answers; it might also customize the service for the particular user. Here are some examples of when authorization would be performed: when a user first logs in and wants to start a shell or when a user starts PPP and wants to use IP over PPP with a particular IP address. The TACACS+ server daemon might respond to these requests by allowing the service, by placing a time restriction on the login shell, or by requiring IP access lists on the PPP connection.

TACACS+ Accounting

Accounting is typically the third action after authentication and authorization. Accounting is the action of recording what a user is doing or has done. Accounting in TACACS+ can serve two purposes:

  • It can be used to account for services used, such as in a billing environment.

  • It can be used as an auditing tool for security services.

To this end, TACACS+ supports three types of accounting records:

  • Start records indicate that a service is about to begin.

  • Stop records indicate that a service has just terminated.

  • Update records are intermediate notices that indicate that a service is still being performed.

TACACS+ accounting records contain all the information used in the authorization records and also contain accounting-specific information such as start and stop times (when appropriate) and resource usage information.

TACACS+ Transactions

Transactions between the TACACS+ client and TACACS+ server are authenticated through the use of a shared secret, which is never sent over the network. Typically, the secret is manually configured in both entities. TACACS+ encrypts all traffic between the TACACS+ client and the TACACS+ server daemon.

Figure 2-15 shows the interaction between a dial-in user and the TACACS+ client and server.

Figure 2-15. A TACACS+ Exchange

The RADIUS Protocol

The Remote Address Dial-In User Service (RADIUS) protocol was developed by Livingston Enterprises, Inc., as an access server authentication and accounting protocol. In June 1996, the RADIUS protocol specification was submitted to the Internet Engineering Task Force (IETF). The RADIUS specification (RFC 2058) and RADIUS accounting standard (RFC 2059) are now proposed standard protocols.

RADIUS uses UDP as its transport. Generally, the RADIUS protocol is considered to be a connectionless service. Issues related to server availability, retransmission, and timeouts are handled by the RADIUS-enabled devices rather than by the transmission protocol.

RADIUS is a client/server protocol. The RADIUS client is typically a NAS; the RADIUS server is usually a daemon process running on some UNIX or NT machine. The client is responsible for passing user information to designated RADIUS servers and then acting on the response that is returned. RADIUS servers are responsible for receiving user connection requests, authenticating the user, and then returning all configuration information necessary for the client to deliver the service to the user. A RADIUS server can act as a proxy client to other RADIUS servers or to other kinds of authentication servers.

RADIUS Authentication

The RADIUS server can support a variety of methods to authenticate a user. When the server is provided with the username and user's original password, the server can support PPP PAP or CHAP, UNIX login, and other authentication mechanisms. What is supported depends on what a vendor has implemented.

Typically, a user login consists of a query (Access-Request) from the NAS to the RADIUS server and a corresponding response (Access-Accept or Access-Reject) from the server. The Access-Request packet contains the username, password, or shared secret between the NAS and the RADIUS server, NAS IP address, and port. The format of the request also provides information about the type of session that the user wants to initiate.

When the RADIUS server receives the Access-Request packet from the NAS, it searches a database for the username listed. If the username does not exist in the database, either a default profile is loaded or the RADIUS server immediately sends an Access-Reject message. This Access-Reject message can be accompanied by an optional text message, which can indicate the reason for the refusal.

RADIUS Authorization

In RADIUS, the authentication and authorization functionalities are coupled together. If the username is found and the password is correct, the RADIUS server returns an Access-Accept response, including a list of attribute-value pairs that describe the parameters to be used for this session. Typical parameters include service type (shell or framed), protocol type, IP address to assign the user (static or dynamic), access list to apply, and a static route to install in the NAS routing table. The configuration information in the RADIUS server defines what will be installed on the NAS.

RADIUS Accounting

The accounting features of the RADIUS protocol can be used independently of RADIUS authentication or authorization. The RADIUS accounting functions allow data to be sent at the start and end of sessions, indicating the amount of resources (such as time, packets, bytes, and so on) used during the session. An Internet service provider (ISP) might use RADIUS access control and accounting software to meet special security and billing needs.

RADIUS Transactions

Transactions between the client and the RADIUS server are authenticated through the use of a shared secret, which is never sent over the network. In addition, any user passwords are sent encrypted between the client and the RADIUS server to eliminate the possibility that someone snooping on an unsecure network could determine a user's password.

Figure 2-16 shows the RADIUS login and authentication process.

Figure 2-16. RADIUS Login and Authentication


With both TACACS+ and RADIUS, it is important to remember that encryption is performed between the TACACS+/RADIUS client and the TACACS+/RADIUS server. If the TACACS+/ RADIUS client is a NAS and not the client PC, any communication between the PC and the NAS is not encrypted (see Figure 2-17).

Figure 2-17. TACACS+/RADIUS Encryption

Also, not all information between the NAS and the RADIUS server is encrypted.

The exchange can be made more secure in CHAP and other protocols, which might use encryption or a hash between the user and the RADIUS client.

The Kerberos Protocol

Kerberos is a secret-key network authentication protocol, developed at Massachusetts Institute of Technology (MIT), that uses the data encryption standard (DES) cryptographic algorithm for encryption and authentication. The Kerberos Version 5 protocol is an Internet standard specified by RFC 1510.

Kerberos was designed to authenticate user requests for network resources. Kerberos is based on the concept of a trusted third party that performs secure verification of users and services.

In the Kerberos protocol, this trusted third party is called the key distribution center (KDC), sometimes also called the authentication server. The primary use of Kerberos is to verify that users and the network services they use are really who and what they claim to be. To accomplish this, a trusted Kerberos server issues "tickets" to users. These tickets have a limited lifespan and are stored in the user's credential cache. They can later be used in place of the standard username-and-password authentication mechanism.

Kerberos Authentication Request and Reply

Initially, the Kerberos client has knowledge of an encryption key known only to the user and the KDC: Kclient. Similarly, each application server shares an encryption key with the KDC, Kserver (see Figure 2-18).

Figure 2-18. Kerberos Keys

When the client wants to create an association with a particular application server, the client uses the authentication request and response to first obtain a ticket and a session key from the KDC (see Figure 2-19).

Figure 2-19. Kerberos Authentication Request and Reply

The steps are as follows:

Step 1.
The client sends an authentication request to the KDC. This request contains the following information:

- Its claimed identity

- The name of the application server

- A requested expiration time for the ticket

- A random number that will be used to match the authentication response with the request

Step 2.
The KDC verifies the client access rights and creates an authentication response.

Step 3.
The KDC returns the response to the client. The authentication response contains the following information:

- The session key, Ksession

- The assigned expiration time

- The random number from the request

- The name of the application server

- Other information from the ticket

All this information is encrypted with the user's password, which was registered with the authentication server, Kclient. The KDC also returns a Kerberos ticket containing the random session key, Ksession, that will be used for authentication of the client to the application server; the name of the client to whom the session key was issued; and an expiration time after which the session key is no longer valid. The Kerberos ticket is encrypted using Kserver.

Step 4.
When the client receives the authentication reply, it prompts the user for the password. This password, Kclient, is used to decrypt the session key, Ksession.

Now the client is ready to communicate with the application server.


Kclient is used as the bootstrap mechanism; however, in subsequent communication between the KDC and the client, a short-term client key, Kclient-session, is used. Kclient-session is created by having the KDC convert the user's password to the short-term client key. The KDC sends the short-term client key, Kclient-session, encrypted with the user's password, to the client. The user decrypts the short-term client key, and the subsequent KDC-to-client communication uses Kclient-session.

Kerberos Application Request and Response

The application request and response is the exchange in which a client proves to an application server that it knows the session key embedded in a Kerberos ticket. The exchange is shown in Figure 2-20.

Figure 2-20. Kerberos Application Request and Reply

The steps in the application request and response are as follows:

Step 1.
The client sends two things to the application server as part of the application request:

- The Kerberos ticket (described in the preceding section)

- An authenticator, which includes the following (among other fields):

  • The current time

  • A checksum

  • An optional encryption key

All these elements are encrypted with the session key, Ksession, from the accompanying ticket.

Step 2.
After receiving the application request, the application server decrypts the ticket with Kserver; extracts the session key, Ksession; and uses the session key to decrypt the authenticator.

If the same key was used to encrypt the authenticator as was used to decrypt it, the checksum will match, and the verifier can assume that the authenticator was generated by the client named in the ticket and to whom the session key was issued. By itself, this check is not sufficient for authentication because an attacker can intercept an authenticator and replay it later to impersonate the user. For this reason, the verifier also checks the timestamp. If the timestamp is within a specified window (typically five minutes) centered around the current time on the verifier, and if the timestamp has not been seen on other requests within that window, the verifier accepts the request as authentic.

At this point, the server has verified the identity of the client. For some applications, the client also wants to be sure of the server's identity. If such mutual authentication is required, a third step is necessary.

Step 3.
The application server generates an application response by extracting the client's time from the authenticator and then returns it to the client with other information, all encrypted using the session key, Ksession.


IPv6 started out as IPng (IP: The Next Generation). Since the first RFC in early 1993, there have been about 120 or so RFCs on various aspects of this technology. Counting all versions, the list could be around 700 entries long.

Earlier proposals for IPng focused on the address space (which is constrained in IPv4) from a routing perspective and assumed a migratory universal protocol (that is, no bifurcation of "access" protocols and long-haul protocols [RFC 1454]). IPv4 thus needed to be replaced by a better scheme. The fractured network with artificial boundaries to accommodate for IPv4 gets in the way of implementing various features such as integrated security and mobility. In addition to the unambiguous address capability, IPv6 also offers the capability to develop innovative applications with end-to-end QoS, security, flow, and so on. RFC 1726 laid out the requirements as follows:

  • Architectural simplicity

  • Implement what is possible and don't stand in the way of other layers that will extend the functionality

  • One protocol

  • Provide a point of commonality between protocols

  • Live long (address the longevity by thinking about a longer-term solution than a quick and dirty one)

  • Build in long-term vision and migration capability

  • Live long and prosper (build in extensibility and flexibility?assume that, to prosper, others will extend your artifacts)

  • Provide operational benefits such as larger address space

  • Cooperative anarchy

  • Decentralization and loose coupling


It is amusing to observe that many of these goals fit other current software layers such as the web services and services-oriented architecture (SOA) paradigms.

One of the major issues addressed by IPv6 is the scalability of the Internet by tackling the limited address space due to the "class" way of adding partitioning the address space in IPv4. IPv6 uses a 128-bit address as opposed to IPv4, which uses a 32-bit address (RFC 3513). This increases the number of addresses from 4 x 109 in IPv4 to 3 x 1038 in IPv6! This enables features such as wireless mobility, addressing wirelessly networked appliances and devices. In fact, in the recent RFID domain, many of the RFID tags are IPv6 addresses.

IPv6 Address Structure and Representation

The addresses in IPv6 are expressed in eight chunks of 16-bit hexadecimal numbers as x:x:x:x:x:x:x:x. One set of consecutive zeros can be collapsed by the :: notation. For example, x:x:x:0:0:0:0:x can be shortened by x:x:x::x.

The three types of IPv6 addresses are Unicast (to a single address), Anycast (to the nearest interface), and Multicast (to a set of interfaces).

The Multicast is an interesting one; the Broadcast type in IPv4 is a special case of Multicast (all-nodes Multicast). You can also construct the Manycast with this type. The scope field (4 bits wide) turns the Multicast into node interface local, link local, site/admin local, organization local, and global scope.

The requirement that each network interface have a unique IP address (in IPv4) is not carried over to IPv6; in IPv6, multiple interfaces can have the same IP address, the assumption being that there is enough intelligence inside the interfaces to differentiate.

IPv6 Header

IPv6 simplified the header (as compared to IPv4) while adding extensibility. Whereas IPv4 had one IP header (with around 12 fields and variable length), the IPv6 header has eight fields (with a fixed size of 40 bytes) and adds optional IPv6 extension headers. The two extensions relevant to security (RFC 2401) are the authentication header (AH; RFC 2402) and the encapsulating security payload (ESP) header (RFC 2406). RFC 2401 defines IPSec, the goal of which is to provide interoperable cryptographic security for IPV4 and IPV6. You will look more closely at these headers in the "IPSec" section later in this chapter.


On the scalability side, IPv6 adds ICMPv6 (which includes neighbor discovery protocols), RIPng (RFC 2080), flow labels (for QoS), and DHCPv6 (which has a few automatic configuration and mobile IP features).


The main barrier to adoption of IPv6 is the transition from IPv4, which still is a big hurdle. The two main mechanisms (RFC 1933) are dual IP stack architecture (where devices would have IPv4 and IPv6 stacks) and tunneling (IPv6 in IPV4).


IP security (IPSec) adds protocols, message formats, and concepts to achieve confidential pipes across an untrusted network like the Internet. One of the most important features of IPSec is the capability to bootstrap trust anchors. The virtual private network (VPN) implementations leverage the IPSec mechanisms.

IPSec is expressed as a collection of RFCs that provide mechanisms (including cryptographic algorithms), protocols, and message formats to achieve point-to-point confidentiality and integrity for the IP. The RFCs also detail the protocols and message formats for authentication, which is a prerequisite for achieving confidentiality and integrity. Table 2-2 shows a summary view of the IPSec mechanisms.

Table 2-2. IPSec Mechanisms: Summary View







ESP header, ESP packet

ESP protocol and processing rules


Origin authentication




Sequence number in the ESP header


Data integrity

Integrity check value (ICV)




Authentication header (AH)

Security association mechanics

Key management


Internet Key Exchange(IKE)

The following subsections provide an introductory summary of the major IPSec formats and protocols. Refer to IPSec-specific books for more in-depth discussion.

Authentication Header

When added to an IP datagram, this header ensures the data integrity and origin authentication, including the invariant fields in the outer IP header. It does not provide confidentiality protection. Authentication header (AH) commonly uses a keyed hash function rather than digital signatures because digital signature technology is too slow and greatly reduces network throughput. AH is an appropriate protocol to employ when confidentiality is not required (or not permitted, as when government regulations restrict the use of encryption).

Encapsulating Security Payload

When added to an IP datagram, this header protects data confidentiality, integrity, and origin authentication. The scope of the authentication offered by ESP is narrower than it is for AH. (The IP header that is "outside" of the ESP header is not protected.) If only the upper-layer protocols must be authenticated, ESP authentication is an appropriate choice and is more space efficient than using AH to encapsulate ESP.

Security Associations

The concept of a security association (SA) is fundamental to IPSec. An SA is a relationship between two or more entities that describes how the entities use security services to communicate securely. The SA includes the following:

  • An encryption algorithm

  • An authentication algorithm

  • A shared session key

Because an SA is unidirectional, two SAs (one in each direction) are required to secure typical, bidirectional communication between two entities. The security services associated with an SA can be used for AH or ESP but not both. If both AH and ESP protection is applied to a traffic stream, two (or more) SAs are created for each direction to protect the traffic stream.

The SA is uniquely identified by a randomly chosen unique number called the security parameter index (SPI) and the destination IP address of the destination. When a system sends a packet that requires IPSec protection, it looks up the SA in its database and applies the specified processing and security protocol (AH/ESP), inserting the SPI from the SA into the IPSec header. When the IPSec peer receives the packet, it looks up the SA in its database by destination address, protocol, and SPI and then processes the packet as required.

Key Management

IPSec uses cryptographic keys for authentication/integrity and encryption services. Both manual and automatic distribution of keys is supported.

The lowest (but least desirable) level of management is manual management, in which a person manually configures each system by keying material and SA management data relevant to secure communication with other systems. Manual techniques are practical in small, static environments, but they do not scale well. If the number of sites using IPSec security services is small, and if all the sites come under a single administrative domain, manual key management techniques might be appropriate. Manual key management might also be appropriate when only selected communications must be secured within an organization for a small number of hosts or gateways. Manual management techniques often employ statically configured, symmetric keys; however, other options also exist.

The default automated key management protocol selected for use with IPSec is the Internet Key Management Protocol (IKMP), sometimes simply referred to as the Internet Key Exchange (IKE). IKE authenticates each peer involved in IPSec, negotiates the security policy, and handles the exchange of session keys.