Section 13.7. Analyzing Protocol Problems

Problems caused by bad TCP/IP configurations are much more common than problems caused by bad TCP/IP protocol implementations. Most of the problems you encounter will succumb to analysis using the simple tools we have already discussed. But on occasion, you may need to analyze the protocol interaction between two systems. In the worst case, you may need to analyze the packets in the data stream bit by bit. Protocol analyzers help you do this.

snoop is the tool we'll use. It is provided with the Solaris operating system.[12] Although we use snoop in all of our examples, the concepts introduced in this section should be applicable to the analyzer that you use, since most protocol analyzers function in basically the same way. Protocol analyzers allow you to select, or filter, the packets you want to examine, and to examine those packets byte by byte. We'll discuss both of these functions.

[12] If you use Linux, try tcpdump. It is similar to snoop.

Protocol analyzers watch all the packets on the network. Therefore, you only need one system that runs analyzer software on the affected part of the network. One Solaris system with snoop can monitor the network traffic and tell you what the other hosts are (or aren't) doing. This, of course, assumes a shared media network. If you use an Ethernet switch, only the traffic on an individual segment can be seen. Some switches provide a monitor port. For others you may need to take your monitor to the location of the problem.

13.7.1 Packet Filters

snoop reads all the packets on an Ethernet. It does this by placing the Ethernet interface into promiscuous mode.[13] Normally, an Ethernet interface only passes packets that are destined for the local host up to the higher layer protocols. In promiscuous mode, all packets are accepted and passed to the higher layer. This allows snoop to view all packets and to select packets for analysis, based on a filter you define. Filters can be defined to capture packets from, or to, specific hosts, protocols, ports, or combinations of all these. As an example, let's look at a very simple snoop filter. The following snoop command displays all packets sent between the hosts crab and rodent:

[13] This works only if the interface supports promiscuous mode; not all interfaces do.

# snoop host crab and host rodent 

Using device /dev/le (promiscuous mode) -> ICMP Echo request -> ICMP Echo reply -> RLOGIN C port=1023 -> RLOGIN R port=1023


The filter "host crab and host rodent" selects only those packets that are from rodent to crab, or from crab to rodent. The filter is constructed from a set of primitives, and associated hostnames, protocol names, and port numbers. The primitives can be modified and combined with the operators and, or, and not. The filter may be omitted; this causes snoop to display all packets from the network.

Table 13-2 shows the primitives used to build snoop filters. There are a few additional primitives and some variations that perform the same functions, but these are the essential primitives. See the snoop manpage for additional details.

Table 13-2. Expression primitives


Matches packets

dst host | net | port destination

To destination host, net, or port

src host | net | port source

From source host, net, or port

host destination

To or from destination host

net destination

To or from destination network

port destination

To or from destination port

ether address

To or from Ethernet address


Of protocol type (icmp, udp, or tcp)

Using these primitives with the operators and and or, complex filters can be constructed. However, filters are usually simple. Capturing the traffic between two hosts is probably the most common filter. You may further limit the data captured to a specific protocol, but often you're not sure which protocol will reveal the problem. Just because the user sees the problem in ftp or telnet does not mean that is where the problem actually occurs. Analysis must often start by capturing all packets, and can only be narrowed after test evidence points to some specific problem. Modifying analyzer output

The example in the previous section shows that snoop displays a single line of summary information for each packet received. All lines show the source and destination addresses, and the protocol being used (ICMP and RLOGIN in the example). The lines that summarize the ICMP packets identify the packet types (Echo request and Echo reply in the example). The lines that summarize the application protocol packets display the source port and the first 20 characters of the packet data.

This summary information is sufficient to gain insight into how packets flow between two hosts and into potential problems. However, troubleshooting protocol problems requires more detailed information about each packet. snoop has options that give you control over what information is displayed. To display the data contained in a packet, use the -x option. It causes the entire contents of the packet to be dumped in hex and ASCII. In most cases, you don't need to see the entire packet; usually, the headers are sufficient to troubleshoot a protocol problem. The -v option displays the headers in a well-formatted and very detailed manner. Because of the large number of lines displayed for each packet, use -v only when you need it.

The following example shows an ICMP Echo Request packet displayed with the -v option. The same type of packet was summarized in the first line of the previous example.

# snoop -v host crab and host minasi 

Using device /dev/le (promiscuous mode) 

ETHER:  ----- Ether Header ----- 


ETHER:  Packet 3 arrived at 16:56:57.90 

ETHER:  Packet size = 98 bytes 

ETHER:  Destination = 8:0:20:22:fd:51, Sun 

ETHER:  Source      = 0:0:c0:9a:d0:db, Western Digital 

ETHER:  Ethertype = 0800 (IP) 


IP:   ----- IP Header ----- 


IP:   Version = 4 

IP:   Header length = 20 bytes 

IP:   Type of service = 0x00 

IP:         xxx. .... = 0 (precedence) 

IP:         ...0 .... = normal delay 

IP:         .... 0... = normal throughput 

IP:         .... .0.. = normal reliability 

IP:   Total length = 84 bytes 

IP:   Identification = 3049 

IP:   Flags = 0x0 

IP:         .0.. .... = may fragment 

IP:         ..0. .... = last fragment 

IP:   Fragment offset = 0 bytes 

IP:   Time to live = 64 seconds/hops 

IP:   Protocol = 1 (ICMP) 

IP:   Header checksum = fde0 

IP:   Source address =, 

IP:   Destination address =, 

IP:   No options 


ICMP:  ----- ICMP Header ----- 


ICMP:  Type = 8 (Echo request) 

ICMP:  Code = 0 

ICMP:  Checksum = ac54 ICMP:

The detailed formatting done by snoop maps the bytes received from the network to the header structure. Look at the description of the various header fields in Chapter 1 and Appendix G for more information.