Troubleshooting VPN Problems

The following is a list of common problems and resolutions that relate to establishing a VPN. Note that any error messages you see in the SmartView Tracker/Log Viewer are documented in the Check Point manuals. Some of the more common errors follow.

11.13 General Troubleshooting Guidelines for VPN Problems

Ensure that the appropriate kinds of traffic are being permitted between the two endpoints. If there are any filtering routers along the way, make sure they permit the following protocols:

  • IP protocols 50 and 51 (for any IPSec-related scheme)

  • UDP port 500 (for IKE)

Also, you should make sure that NAT is not being performed on any of the packets.

Sometimes you may need to put explicit rules in the firewall permitting this traffic. In most cases, this isn't necessary. The rules are shown in Figure 11.25.

Figure 11.25. Rules to permit IKE and ESP to the firewall


You may also want to use a packet sniffer (e.g., tcpdump, snoop, fw monitor) to verify that packets are reaching the gateway. If the packets are not reaching the gateway, FireWall-1 cannot encrypt or decrypt them.

11.14 No Response from Peer

The "No response from peer" error message usually points to one of the following problems.

  • The encryption domains are not correct. The encryption domain for firewall A should contain all the hosts behind firewall A and any translated IP addresses (including hides). The firewall should be included if it is used as the hide address. The same is true for firewall B?its encryption domain should contain all the hosts behind firewall B, any translated IP addresses, and firewall B itself if it is used as a hide address.

  • The remote end does not currently have a rule that will decrypt the packet.

  • The remote firewall is not set up with encryption.

  • Something is blocking communication between the VPN endpoints. Check to make sure the remote firewall is properly receiving the IP packets by using a packet sniffer. Look for IP protocol 50 or UDP port 500 packets.

11.15 AddNegotiation: Try to Handle Too Many Negotiations

A key negotiation occurs when a connection is first established from one host to another. If you see this "AddNegotiation" message, it means that FireWall-1 is handling more than 200 key negotiations at once. Connections that have this message associated with them in the log will fail. In NG FP3, you can configure a firewall to support more IKE negotiations by editing the gateway object and going to the Capacity Optimization frame. Set the maximum concurrent IKE connections there.

11.16 Debugging Interoperability Issues with IKE

Everyone has a different interpretation about how to follow standards. As a result, when third-party products talk to one another, communication doesn't always work. One way to debug is to turn on IKE debugging.

In FireWall-1 4.1, it was necessary to stop and restart FireWall-1 in order to enable debugging. In NG, you can enable this on the firewall module with a simple command: vpn debug ikeon. You can also disable it with vpn debug ikeoff. When you enable debugging, $FWDIR/log/ike.elg gets created. This file contains the results of all IKE negotiations that occur. This file is a little difficult to read on its own. Fortunately, Check Point has a tool called IKEView that allows you to view this file in a more readable form. Unfortunately, it is available only to Check Point Certified Service Partners.

Most interoperability issues actually come down to one of the following things.

  • A parameter mismatch has occurred, that is, one IKE parameter is configured differently on one end of the VPN.

  • There is a topology or encryption domain mismatch.

11.17 Known Interoperability Issues

The following subsections detail some known interoperability issues, with fixes where appropriate.

FireWall-1 NG FP2/FP3 and Cisco PIX

In NG FP2 and FP3, you may experience a problem when trying to establish a VPN with a Cisco PIX firewall. Check Point released a hotfix to address this problem. For NG FP3, request hotfix SHF_FW1_FP3_0006 from Check Point or your support provider. For NG FP2, request SHF_FW1_FP2_0248.

Nokia Crypto Cluster

In NG FP3 and before, there are several interoperability issues with the Nokia Crypto Cluster (CC) product line, which are likely to show up in other situations as well. In some cases, you will need to take the following steps.

  • Disable ISAKMP Commit Processing in the CC.

  • Enable Defer Main Mode Deletion in the CC.

  • Ensure that you do not restrict access to the VPN based on services on the CC. FireWall-1 is not RFC compliant in how it negotiates an IKE SA[3] because it always assumes all services are permitted, whereas the CC products negotiate the allowed services as part of the SA,[4] just as the RFCs state must be done.

    [3] In my conversations with Check Point on this issue, the representatives with whom I spoke did not believe that negotiating the allowed services in the SA is secure because it essentially advertises what is allowed. A valid point, but the behavior is not interoperable with devices acting in an RFC-compliant manner. What part of "not RFC compliant" do they not understand?

    [4] At least one developer who worked on the CC products actually wrote the Internet RFCs related to IPSec.

  • If you are using a version of FireWall-1 prior to NG FP3 Hotfix-2, define your encryption domain in terms of the largest possible subnets because FireWall-1 tends to simplify the encryption domains down to the largest possible subnets. This is despite having an option in objects_5_0.C that supposedly turns this off (see FAQ 11.18).

IPSec SAs Are Not Reestablished Properly after IKE Rekey with Cisco and/or Sonicwall

The initial VPN tunnel is established and VPN traffic flows. The subsequent IPSec rekeys work fine. However, when one end is VPN-1/FireWall-1 and the other end is either a Cisco or Sonicwall device, VPN traffic fails after an IKE rekey until an IPSec rekey is done.

RFC2408 (Section 5.15), the relevant RFC for IKE, states: "The receiving entity SHOULD clean up its local SA database." Check Point interprets this section to mean that upon IKE rekey, ISAKMP Delete should be sent or acknowledged in order to clean up the IPSec SAs at the same time. Cisco and Sonicwall have not taken this approach and maintain the IPSec SAs across the IKE SA rekeys. This difference in behavior is what causes VPN traffic to fail.

To determine whether this behavior is occurring, display the IPSec SPI numbers before and after an IKE SA rekey operation on the third-party device. If the SPIs are the same, the device is preserving the IPSec SA across IKE rekeys. VPN traffic will fail until the next IPSec rekey.

11.18 Encryption Failure: Packet Is Dropped as There Is No Valid SA

You might see this error message when both ends of the VPN do not have the same definition for the encryption domain. First ensure that both ends of the VPN are defined with the same encryption domain. If they are the same, you should create objects that are exactly the same size as what is created on the remote end.

One annoying behavior FireWall-1 NG exhibits that FireWall-1 4.1 and earlier did not is the automatic simplification of subnets in IPSec SAs. For example, if your encryption domain contains explicit objects for and, Check Point would attempt to negotiate an IPSec SA with instead of generating SAs based on the network objects you created. To eliminate this behavior, use dbedit to make the following changes on your management console (see FAQ 4.2 for details on editing objects_5_0.C):

dbedit> modify properties firewall_properties
               ike_use_largest_possible_subnets false
dbedit> update properties firewall_properties

You must then reload the security policy for this change to take effect.

11.19 Traceroute Does Not Appear to Work through a VPN

Traceroute works by sending out packets with successively larger time to live (TTL) values (see Chapter 1). Each hop along the way generally returns an ICMP Time Exceeded message, an ICMP Destination Unreachable message, or an ICMP Echo Reply.

In an IPSec VPN, all communication between the sites is encapsulated. When FireWall-1 encapsulates a traceroute packet, the new packet inherits the TTL value of the packet being encapsulated. As a result, each hop between the firewalls sends an ICMP Time Exceeded packet back to the firewall. These packets are ignored by the firewall. Users will see these messages in their traceroute as "request timed out."

Interestingly enough, with SecureClient on NG, all hops between the firewall and client are skipped, so traceroute appears to work.

11.20 VPN Fails When Transferring Large Packets

Some applications set the Don't Fragment bit on certain packets. When the IPSec headers are added to the already large packet, the packet requires fragmentation in order to pass through the firewall. When Check Point creates the IPSec packet, the Don't Fragment bit from the original packet is maintained. FireWall-1 creates a fragmented packet that has the Don't Fragment bit set, so it cannot be fragmented and thus gets dropped at the next router.

You can force FireWall-1 to clear the Don't Fragment bit by changing the ipsec_dont_fragment property in objects_5_0.C to false. You can do this with the following commands in dbedit on the management console (craig is the firewall in this example):

dbedit> modify network_objects craig VPN:ipsec_dont_fragment
dbedit> update network_objects craig

Alternatively, you can use the GUIdbedit tool to change the parameter. Either way, you must then reinstall the security policy for this change to take effect.