10/2/2002 -- Encryption is a topic that finds its way into every nook and cranny of the networking world. Everyone has their own ideas of how it should work and their own understanding of how it does work. The problem with encryption is that there are several different types, and these types work in several different ways. And while most companies want encryption, they don't necessarily know what type of encryption they want, or what the differences are between the types.

This article is designed to give you a good overview of current encryption technologies as well as an in-depth look at the most popular method of standards-based encryption today, IPsec.

Encryption Types
While there are many different types of encryption, for simplicity's sake you can break them down into two categories: session layer and network layer.

Session layer encryption is something that we don't often give much thought to -- perhaps because it's not as sexy as network layer encryption. Session layer encryption occurs any time you have encryption done with an application - for example, WinZip. It also occurs with an HTTPS session. With session layer encryption, the encryption occurs at the session layer of the OSI model.

Network layer encryption happens in spite of any applications that are crossing the connection. IPsec is one of the more popular types of network layer encryption (others include PPTP and the lesser-used L2F, L2TP and GRE). Because it occurs at the Network layer, IP will include header information pointing to the encryption protocol in use. For example, looking at an IPsec packet in a sniffer won't show any information about the traffic crossing the link in the header; instead, it simply shows how the packet was protected.

No matter what encryption method you're considering, you'll need to decide whether you want a commercial solution or one that is publicly available. But before we get into these possibilities, let's look at what encryption is all about.

Encryption Basics
Most of today's block ciphers are based on a paper written by Dr. Horst Feistel in 1970. There are more recent pieces of information, though: Diffie and Hellman produced work in 1976, and RSA was completed in 1977. While the methods of implementation get stronger, faster and better, nothing revolutionary has actually happened in this field for more than 30 years.

Data Encryption Standard
The Data Encryption Standard (DES), adopted in 1977 by the National Institute of Standards and Technology (NIST), was the first encryption method to achieve widespread acceptance. DES is based on an IBM developed algorithm called LUCIFER. It refined a 128-bit key process to one using a 56-bit key, making it easier to place in hardware. When the U.S. government issued a request for proposals for a national encryption standard, IBM submitted it.

DES operates by taking 64-bit blocks of clear text information and encrypting them with the 56-bit key in a 16-step process. While the relative weakness of the 56-bit key has always been one of the arguments people use against DES, the weakness is relative -- and growing. A key length of 56 bits will produce up to 2^52 keys. In 1977, Diffie and Hellman showed that a machine with 1 million encryption devices that could each perform one encryption per microsecond and only had to search half of the key length could figure out the key in 10 hours. In 1977, this machine would have cost $20 million. In 1997, Wiener showed that a $10 million machine could break the key in 21 minutes. With the power of distributed computing, this time has grown much shorter.

Triple-DES
While most communications are perfectly safe with DES, there are a number that aren't. When it comes to national security, the U.S. government will do everything it has to be that much more secure. This led to Triple-DES (3DES).

3DES uses the same mechanism that DES does; it just does it multiple times. Using a 16- or 24-byte key string, it divides the key into 8-byte segments. Each segment (56-bits of usable key because 8 bits are for parity) is used against the 64-bit block of clear text with the same 16-cycle process.

DES can run in either 112-bit mode or 168-bit mode. If the given key isn't long enough to produce three distinct 8-byte key values, it will use the same value for the first and the third keys.

Hashing
A hash is a method of encrypting data so that it cannot be decrypted. In essence, the process takes a certain amount of clear text data, encrypts it and then truncates it down to a set value. Because some of the output is now missing, it cannot be reliably reversed into its original form. For example, lets use the number 45678 as the clear text value. The encryption process is to multiply the text by a certain number, 194 in this case. The protocol reverses the number and makes sure only the five most significant digits are displayed.

For example:

45678          The original clear text
x 194           The number to multiply by, the agreed upon key
------------
8861532      The encrypted output

2351688      Flip the number around

23516         Remove all but the five most significant digits.

As you can see, it's impossible to figure out what the original key was, even when you know exactly how the hashing algorithm works. Anyone watching the wire is left wondering if the number 23516 was the result before truncation or if there were additional numbers.

There are two methods of hashing that are supported on most Cisco equipment. MD-5 does 128-bit encryption and SHA-1 uses 168-bit encryption. Both will truncate after 96 bits of information have been gathered. If 96-bits can't be gathered, 0s are used for padding.

Hashing is better than encryption with some types of traffic. Take, for example, a routing update from RIPv2. Chances are, you don't want anyone tampering with your routing updates, but do you care if they see a packet advertising the 192.168.4.0 network? Not usually. Hashing simply makes sure that any tampering with the packet is detected. By not using encryption, the routers save processor cycles.

When packet data is hashed, the hashed value travels with the packet. When it reaches the destination, that device will hash the data it sees and compare it to the hashed value of the packet that traveled with it. If they match, the packet is valid. If they don't, it isn't.

Tunnels
A tunnel is the logical connection from one device to another. Each end of the tunnel has a device that is responsible for IPsec processing (which we'll get into in more detail below). There are actually two types of tunnels, depending on what you're doing. When a device is participating in a type of tunneling, it is in a particular mode.

Transport Mode
Transport mode tunnels are the easiest to define. There is only one question you need to ask yourself, "Will IPsec be used from the source of the traffic all the way to its destination?" If the answer is yes, use transport mode.

IPsec in transport mode makes more modifications to the packet than tunnel mode (below) does. Transport mode leaves the original IP header alone and just inserts IPsec data in where it needs to go.

Examples of a transport mode tunnel are an SSL session in a Web browser. Your browser is the source and the Web server on the other end is the destination. The encryption goes from one machine all the way to the other. SSH for secure telnet is another example.

Tunnel Mode
Tunnel mode should be used whenever the tunnel does not protect the conversation from source to destination. For example, a VPN connection from your home to a VPN Concentrator at work would be a tunnel mode connection because it only protects the connection from your PC to the edge of the corporate network. Once inside the corporate network, there is no VPN.

Tunnel mode is the default setting on Cisco IOS devices because tunnel mode places the original packet into a brand new packet made just for the IPsec tunnel. The original packet is placed entirely within the data part of the IPsec packet. This allows the IPsec packet to encrypt everything about the original packet, including where it came from. If two routers are the ends of the IPsec tunnel, a sniffer would detect a packet going from one router to another, it has no clue about the source or destination of the original packet.

IPsec In-Depth
So just how does IPsec work? You need to realize that IPsec is actually only part of a tunnel. In order for an IPsec session to get going, for the tunnel to be set up, the tunnel parameters need to be exchanged. Since you currently don't have a tunnel to protect those parameters, and since sending the tunnel keys across an unencrypted link would be rather silly, we need a secure way to figure out how to bring up the IPsec tunnel. Enter IKE.

Internet Key Exchange
The Internet Key Exchange (IKE), also known as the Internet Security Association Key Management Protocol (ISAKMP), is responsible for letting two IPsec devices negotiate tunnel properties. It provides a secure tunnel for this purpose, but first there needs to be a way for the initial security to be set up. There are two methods for a device to authenticate a peer: pre-shared keys and certificate authorities.

A pre-shared key is a key that both devices participating in the tunnel agree on. This key will be statically set on a device. Then, every time the tunnel needs to come up, this key will protect it. Most Cisco devices allow you to configure a pre-shared key for each potential tunnel. So, if router A wants to be able to form a tunnel to routers B and C, router A would have two separate entries for pre-shared keys: One for use when connecting to router B, and one for use when connecting to router C. These keys could be the same or different.

Because each device needs a key for every tunnel it wants to participate in, pre-shared keys are fine if you only have a few devices. But they don't scale well. Imagine having 100 routers in your organization that have VPN tunnels to each other. Installing one more router involves setting up 100 tunnels on the new device and a new tunnel on each of the 100 old ones. While the pre-shared key information may only one line per device, you still have record keeping and the potential for error. Fortunately, there is another way available: Certificate Authorities.

A trusted device, called a Certificate Authority (CA), can generate a digital certificate. The certificate generated has identifying information about the IPsec device as well as a key to be used. Each device will create a mathematically linked public and private key. The public key is given to the certificate authority that pastes it into the certificate and sends the certificate to the device that requested it.

When a tunnel needs to be brought up, the peers exchange certificates. Router A will encrypt the packet using router B's public key, which only router B can decrypt. Router B will encrypt packets with router A's public key. This is called asymmetric encryption.

Diffie-Hellman Key Exchange
Authenticating the peer device is the first step of getting an IPsec tunnel active, but it doesn't move user data, it only moves the setup information for the next step. Now, we want to make sure that the IPsec tunnel negotiations are secure. However, we don't have a tunnel yet. Enter Diffie-Hellman (DH), an ultra-secure tunnel used only to provide a secure way to negotiate IPsec parameters.

Cisco IOS devices support two types of Diffie-Hellman tunnels: group 1 is 768-bit encryption, while group 2 is 1024-bit encryption. There are other DH standards supported on other equipment, like the VPN Concentrator. Because of the increased security in IPv6, DH encryption will soon top 8000-bit.

With DH, each device participating in the tunnel will create a public and private key then send a copy of the public key to the peer. Each device will then mathematically combine its private key with the peer's public key, creating something called a Shared Secret key. Because of the mathematics behind the algorithms, the keys are the same. This is an example of symmetric encryption.

The IPsec Tunnel
Once the Diffie-Hellman tunnel has been brought up, the IPsec process can begin. For the most part, there is little that needs to be done. You will provide the basic framework for the tunnel and the routers negotiate the specifics. The reason why the keys are negotiated between the routers is because a secure network requires the keys to expire occasionally. Every so often, the routers will renegotiate the keys and the tunnel continues on.

Protocols
As mentioned above, IPsec operates at the network layer. This means that the important encryption information is included in the IP header. There are two
protocols that can be referenced in the header, the presence of which allows for different features to be included. The primary features supported are encryption
of the payload and verification that the data hasn't changed.

The protocols IPsec used are Encapsulating Security Payload (ESP), which is protocol 50 and Authentication Header (AH), which is protocol 51. ESP is the one you will need to use if you want to encrypt data because AH doesn't provide for that capability. Both ESP and AH can verify that data hasn't been changed by examining an authentication data field in the respective header.

AH is of limited usefulness in many networks; however, where it can be used, it shines. AH protects only the integrity of the data that was send across the wire. The AH header has a field called Authentication Data that keeps some control information regarding the data. A process is run when the packet is created. This process is run again when the packet arrives at the destination. If the answer the process gets doesn't equal the value found in the Authentication Data field, the packet has been tampered with and is discarded.

AH protects everything from Layer 3 on up, with the exception of a few fields in the IP header that change constantly. An example of an excluded field would be TTL. Unfortunately, the Source IP Address field is included in the protected area, meaning any change to the source IP on the packet will result in the values not matching and the packet being discarded. AH doesn't work well with NAT. That said, there are some implementations where AH doesn't include the source IP address and those implementations work well with NAT.

AH adds a header to the packet right behind the IP header and before the Layer 4 header. The information for the authentication component is derived from a hashing process (described above).

In contrast, ESP doesn't protect components of the IP header, just layer 4 and up. This allows ESP to work with NAT natively. Just as AH does, ESP will allow IPsec to make sure that data hasn't been changed but the big reason why it's used is because it can protect data from being seen by encrypting it. If both authentication and encryption are used, the process first encrypts the data then does the process for authentication.

ESP adds a header to the packet right behind the IP header and before the Layer 4 header. ESP also uses a trailer for both the encryption process and the authentication process.

You will need to provide a few parameters in order to build the framework. There are four things to need to decide on:

1. Do you want to use encryption?
2. Do you want hashing for ESP?
3. Do you want hashing for AH?
4. Of the parameters you want to use, what algorithms will be used? (If you're using encryption, do you want to encrypt using DES or 3DES? What algorithms will be used for hashing?)

The configuration you specify is called a transform set. Each component is called a transform. An IPsec tunnel can be made up of up to three transforms, one AH authentication, one ESP encryption and one ESP authentication.

What Next?
Now that we've gotten the basics covered, this brings us to the end of this article. Stay tuned though: Next month, we'll tackle more of the how-to side with a look at implementing IPsec on Cisco devices.

Questions? Comments? Post your thoughts below! To be notified when the follow-up article is posted, sign up for CertCities.com's weekly e-mail newsletter.

More articles by Eric Quinn:

• Data Protection 102: IPsec Implementation
• My Top 5 Cisco Security Tips
• Using Multilink Frame Relay
  
• CCIE Fundamentals: Network Design and Case Studies, Second Edition