Cisco IOS Cryptosystem Components

Cisco IOS Cryptosystem Components

You should be familiar with two terms used in network security. While these terms are quite closely related, they aren’t quite as interchangeable as they might first seem.


The process of translating data into a secret code, which can only be read or processed after being decrypted using a secret key or password. Encryption typically involves using a mathematical algorithm for combining the original data, referred to as plaintext or Cleartext, with one or more keys—character strings of numbers and/or text known only to the sender and the recipient. The resulting output is ciphertext. Encryption is the most effective way to achieve data security.


The science or study of encryption and decryption techniques. A system for encrypting and decrypting data is called a cryptosystem. While it’s technically possible for encrypted data to be deciphered using cryptanalysis, or code-breaking techniques, most modern cryptography techniques are virtually unbreakable.

The Internet, wireless technologies, and other forms of electronic communication are making electronic security technologies increasingly important. Current cryptography technologies being widely used include SSL for web traffic, Secure Shell (SSH) for Telnet sessions, Pretty Good Privacy (PGP) to protect e-mail messages, and IPSec Layer 3 secure communications.

How Encryption Works

Simply encryption uses a mathematical algorithm to scramble a message to create the cipher text. A key is used by the algorithm to determine the scramble pattern. The same key can then be used to unscramble or decipher the message. While this isn’t a perfect analogy, a deadbolt on a door is nothing more than a chunk of metal until someone inserts a key that can manipulate the tumblers and advance the bolt. The lock and the door are then secure until the correct key is used to reverse the process. Like the door analogy, more than one person might have the key but, without the correct key, the lock won’t budge. Also, somewhat like the door analogy, cipher text might require multiple keys both to lock and unlock. Figure 9-14 shows a common, simplified view of encryption.

Click To expand
Figure 9-14: Cleartext being processed using a key to produce cipher text

The key terms in the process include data, key, encryption algorithm, and cipher text.


The data is the information to be encrypted. This data could be text from an e-mail, text and numbers from a credit card transaction, user name and password combinations, music, graphics, videos, or voice conversations. Data arriving to be encrypted might possibly have already been encrypted one or more times in earlier processes. If cipher text is the result of three encryptions, then it must be deciphered three times in the correct order with the appropriate key(s).


A key is a fixed-length digital sequence of characters used to scramble the source data. A longer key typically provides a higher level of security than a shorter key. DES is an industry-standard, symmetric-key encryption method that uses a 56-bit key. Triple-DES (3DES) uses three separate encryptions with unique 56-bit (168-bit) keys for stronger security. Newer AES technology uses 128-, 192-, or 256-bit keys for even greater security.

Government Regulation

Recognizing that encryption keys are tightly regulated in many countries is important. Exporting from the United States or Canada technologies using encryption keys longer than 56 bits—even for use by overseas branches of domestic companies—is currently illegal. Similarly, possession of some key technologies could be a serious crime in certain countries. Export controls on strong encryption (greater than 56-bit DES strength) vary according to type, strength, destination territory, end use, and end user. A person would be foolhardy to assume that violations of import or export controls on this type of technology would be overlooked or resolved with an apology. Terrorism, drug trafficking, and racketeering concerns can make even a sloppy transaction to a third party, who then transships the technology illegally, a serious legal problem.

Encryption Algorithm

The mathematical formula used by the encryption process to scramble the data is an encryption algorithm, and DES and MD5 are examples. Algorithms are often used in combinations to encryption cryptosystems like IPSec.

Cipher Text

The encrypted result is virtually useless to anyone who lacks the correct key. The length of the key, and the number and types of incorporated encryptions determines the cipher text’s capability to withstand unauthorized access. Ultimately, the security of a cryptosystem depends on the secrecy of one or more of the keys, rather than with the secrecy of the encryption algorithm. A strong cryptosystem has a large number of possible keys, making it time- and resource-consuming to try all possible keys (brute force). Brute force attacks, as depicted on television, where a computer generates potential keys, are typically more fantasy than reality.

Cryptography Types

Three basic types of cryptographic functions are used for authentication, integrity, and confidentiality. They include the following:

Symmetric encryption

Often called secret key encryption, uses a shared key and the same encryption algorithm to encrypt and decrypt a message.

Asymmetric encryption

Often called public key encryption, uses two different, but related, keys for the encryption algorithm. To encrypt a message, both a public key and a private key are used. The sender uses the public key to encrypt the message and the recipient uses the private key to decrypt the encrypted message.

Hash function

A hash function is a one-way encryption used to authenticate information. The information is “hashed,” and then transmitted with the hash value. The destination peer can perform the same hash, and then compare the result to the transmitted hash value. If they match, the probability is high that the data is complete and unchanged.

Encryption Alternatives

Encryption can occur at various levels within the network to meet the organization’s specific objectives. Using the Open Systems Interconnection (OSI) or the Transmission Control Protocol/Internet Protocol (TCP/IP) reference model, the encryption can be placed at the data-link layer, the network layer, or the application layer. Figure 9-15 shows a diagram of encryption at each of the layers.

Click To expand
Figure 9-15: Encryption implementation options

Data-Link Layer

Data-link layer (L2) encryption can be implemented in devices other than the router on switched links where security is a concern. Because this method encrypts the network layer header, as well as the data, traffic must be decrypted before entering the next router. The concern would be if traffic had to pass through many routers, the decryption and reencryption would need to be repeated each time, leading to latency issues. Another concern involves trust because the data passes through these routers as Cleartext.

Network Layer

Network layer (L3) encryption is done between two layer 3 devices. While Transport mode involves encrypted connections between an end host and a VPN endpoint device, such as routers on the network, the more common implementation is between two VPN endpoints, such as a VPN hardware client device, a router, a firewall, or a VPN concentrator. This means Cleartext data travels between the actual host and the VPN source endpoint where it’s encrypted and forwarded on to the VPN destination endpoint, where the transmission is decrypted and forwarded to the destination host as Cleartext. One security concern is that the data is unencrypted during part of the trip, although we hope it would be in the internal secure LAN networks on both ends.

Note, the VPN endpoints might be many routers (hops) away and the interim routers needn’t be VPN-aware. Because the packets must be routed through the network, the IP header must remain in Cleartext. While IPSec uses this approach, the layer 3 header information that’s vulnerable refers to the VPN tunnel endpoints and not to the source and destination hosts. The original IP header is safely encrypted in the data section of this transport packet.

Layer 3 implementations should use Quality of Service (QoS) features end-to-end to ensure priority delivery through the network, particularly with time-sensitive data.

Application Layer

Application layer (TCP/IP 4 or OSI 7) encryption is implemented on each host using encryption features built in to the user applications. SSL providing secure web transactions or e-mail applications using PGP are both examples of application-layer encryption. While providing the ultimate in end-to-end security, application layer encryption does require all users to share the same encryption tools, even if they work on different hardware and software platforms. And it also relies on each user be somewhat knowledgeable about the encryption tools.


As explained earlier, a hash is a one-way encryption method used to authenticate that data received is exactly the same as the data transmitted. The following basics about hashing are important to understand.

  • A hash algorithm’s function is to provide data integrity verification, not necessarily character-for-character translations of data.

  • The hash process typically generates a fixed length number value derived from a variable length string of text.

  • The hash output is often substantially smaller than the source text itself.

  • The hash algorithm is sufficiently complex to make producing the same hash value with any other text combination extremely unlikely.

Hashed Message Authentication Codes (HMAC)

IPSec supports improved versions of Message Digest 5 (MD5) and Secure Hash Algorithm 1 (SHA-1) hash functions for cryptographically secure authentication. As with everything else in the security arena, hashing algorithms have evolved into HMACs, which build on the proven security of hashing algorithms by implementing additional cryptographic processes.

The HMAC algorithm involves having the original message text encrypted (hashed) using the sender’s private key, resulting in a keyed, fixed-length checksum as output. The checksum value is appended to the message as a signature. The destination peer runs the same HMAC on the received message data, using the same private key. The resultant hash is compared with the received hash value, which should match exactly. Figure 9-16 shows this combination of source message and encryption key.

Click To expand
Figure 9-16: HMAC hash process showing the private encryption key

Also known as HMAC-MD5 (RFC 1321), this hashing technique is based on MD5 developed by Ronald Rivest of MIT and RSA Data Security Incorporated. HMAC-MD5 uses a 128-bit secret key to produce a 128-bit authenticator value. Once produced, the 128-bit value is truncated to only the first 96 bits and stored in the AH or ESP authenticator field of the new packet. At the destination peer, the 128-bit authenticator value is computed and the first 96 bits are compared to the value stored in the authenticator field.

While the MD5 algorithm used alone was found vulnerable to collision search attacks and other known weaknesses, the vulnerability doesn’t impact HMAC-MD5.


Also known as HMAC-SHA-1 (RFC 2404), this hashing technique is based on SHA-1 specified in FIPS-190-1 combined with HMAC technology. HMAC-SHA-1 uses a 160-bit secret key to produce a 160-bit authenticator value. The 160-bit authenticator value is truncated to the first 96 bits and stored in the AH or ESP authenticator field of the new packet. At the destination peer, the 160-bit authenticator value is computed and the first 96 bits are compared to the value stored in the authenticator field.

Because of the longer key value, HMAC-SHA-1 is cryptographically stronger than HMAC-MD5, but it requires more CPU cycles to compute. Therefore, HMAC-SHA-1 should be the choice if slightly superior security is required. HMAC-MD5 should be the choice if slightly superior performance is required.

Diffie-Hellman Key Agreement (DH)

Diffie-Hellman (DH) is a public key encryption method that provides a mechanism for two IPSec peers to establish a shared secret key that only they know, while communicating over an insecure channel. DH is critical to IPSec operations because the shared secret key is used to encrypt data using the secret key encryption algorithms specified in the IPSec Security Associations, such as DES or MD5. Security Associations are covered later in this section.

DH algorithms and calculations are quite complex, using exponentiation of large numbers, between 768 and 1,536 bytes. Even simple calculations can take up to four seconds on a Cisco 2500. Fortunately, DH boils down to the following relatively simple conceptual steps:

  • Each IPSec peer generates a public and private key pair. The public key is a mathematical derivative of the private key, conceptually like a series of unique office door keys might all be derived from a master key that can open every office.

  • Each peer keeps its own private key secret and never shares it with anyone.

  • Each peer sends its public key over the insecure channel.

  • Each peer combines the other peer’s public key with its own private key and computes the shared secret number that will be the same on each peer.

  • The shared secret number is converted by each peer into a shared secret key. This shared secret key is never exchanged between the peers, which avoids the chance of it being captured in transit on the insecure link.

The end result of the DH process is this: each IPSec peer has the three following keys for each IPSec peer relationship it maintains.

  • A private key, which is kept secret and never shared. The private key is used to “sign” outgoing messages to the IPSec destination peer.

  • A public key, which is shared with the IPSec destination peer. The public key is used by the destination peer to verify a “signature” on a transmission.

  • A shared secret key, which is used to encrypt data using any shared secret encryption algorithm, such as DES, 3DES, MD5, SHA-1, and so forth. This shared secret key is derived from the DH key generation process just described.

Defining a DH Group

The only real configuration in defining the DH process is to assign a DH group level. The groups supported might vary based on the device and the operating system (OS) release. Cisco IOS devices currently support DH Groups 1, 2, and 5. Group 5 provides the greatest security and is the most recently added, in IOS v12.1(3)T. The default group on most devices is Group 1.

DH Group



The 768-bit Diffie-Hellman group


The 1,024-bit Diffie-Hellman group


The 1,536-bit Diffie-Hellman group

Part III: Virtual Private Networks (VPNs)