Traceroute Across an MPLS-enabled Network
The traceroute facility is a useful troubleshooting tool that allows you to trace the path a packet takes from an IP source to an IP destination. This tool is used extensively in the IP community and, therefore, its importance warrants discussion in this book.
Although the MPLS architecture does not change the inherent behavior of the traceroute facility, it does handle the forwarding of traceroute packets slightly differently to a normal IP network. In a normal IP environment, the Cisco implementation of traceroute works as illustrated in Figure 5-9.
Figure 5-9. Operation of Traceroute Across an IP Network
As Figure 5-9 shows, the operation of traceroute across an IP network can be summarized as follows. These steps constitute the use of traceroute both within an IP network and within an MPLS-enabled network:
The source of the traceroute sends an IP packet to a particular destination with a Time-to-Live (TTL) of 1 and a destination UDP port of 33434 (Step 1).
The first router in the path of the packet sends an Internet Control Message Protocol (ICMP) "Time Exceeded" message back to the source of the packet. This is because the TTL of the IP packet reaches zero after the router decrements it by 1 (Step 2).
The source sends a second packet, this time with a TTL of 2. The first hop router routes this packet. When it reaches the second router in the path, an ICMP "Time Exceeded" message is sent again (Step 3 and Step 4).
This process continues (with the source incrementing the TTL by 1 on each iteration), until it reaches the final destination of the packet, or a maximum number of hops is reached (the default value is 30 hops). The final destination router (or host) sends an ICMP "Port Unreachable" message back to the source. Using the ICMP response messages, the source can tell whether the response is from a transit router or from the final destination of the packet (Step 5 and Step 6).
This process certainly is adequate in a normal IP backbone where all transit routers carry external routes. However, as discussed in Chapter 2, it is desirable in an MPLS network to not carry external routing information and just to label switch traffic to the BGP next-hop of these external destinations. This presents a couple of problems with the use of traceroute:
Traceroute relies on the fact that the source address of the traceroute packet is reachable by any router that needs to respond to the packet with an ICMP message.
TTL propagation must be possible across the network for traceroute to function.
Note
This issue is discussed fully in draft-ietf-mpls-label-encaps, section 2.3.2, "Tunneling Private Addresses Through a Public Backbone."
Because the source address might not be reachable (for example, when running VPN or when the core of the network does not carry BGP routes) in an MPLS environment, you can re-use the label stack from the original packet to label switch the ICMP message back to the source. This means that the packet can be sent to the original destination, which then can forward the packet back across the MPLS network to the original source of the packet. In the example shown in Figure 5-9, this behavior causes the Washington router to forward the ICMP "Time Exceeded" message (Step 2 in Figure 5-9) to the San Jose router, which then forwards the packet back to the Washington router with a label stack to reach the Paris router. Figure 5-10 shows this process.
Figure 5-10. Traceroute in an MPLS Environment
Figure 5-10 shows that although the TTL of the incoming packet (Step 1) reaches zero, the Washington router can direct the ICMP "Time Exceeded" message back to the Paris router by using the original label stack of the packet. Example 5-3 provides some debug output to show this process in action. The addresses shown are 10.2.1.21 (the address of the Paris router interface connecting it to Washington), 10.1.1.13 (the address of the San Jose router interface connecting it to Washington), and 194.22.15.2 (the loopback interface address on the San Jose router).
Example 5-3 Traceroute Across an MPLS Network
Paris# debug ip icmp Paris# traceroute 194.22.15.2 Type escape sequence to abort. Tracing the route to 194.22.15.2 1 10.2.1.22 4 msec 0 msec 0 msec 2 10.1.1.13 4 msec * 0 msec ICMP: dst (10.2.1.21) port unreachable rcv from 10.1.1.13 ICMP: dst (10.2.1.21) port unreachable rcv from 10.1.1.13 Washington# debug ip icmp Washington# debug tag packet TAG: Et0/1/0: recvd: CoS=0, TTL=1, Tag(s)=36 TAG: AT0/0/0.1: xmit: (no tag) ICMP: time exceeded (time to live) sent to 10.2.1.21 (dest was 194.22.15.2) TAG: Et0/1/0: recvd: CoS=0, TTL=1, Tag(s)=36 TAG: AT0/0/0.1: xmit: (no tag) ICMP: time exceeded (time to live) sent to 10.2.1.21 (dest was 194.22.15.2) TAG: Et0/1/0: recvd: CoS=0, TTL=1, Tag(s)=36 TAG: AT0/0/0.1: xmit: (no tag) ICMP: time exceeded (time to live) sent to 10.2.1.21 (dest was 194.22.15.2) TAG: Et0/1/0: recvd: CoS=0, TTL=2, Tag(s)=36 TAG: AT0/0/0.1: xmit: (no tag) TAG: Et0/1/0: recvd: CoS=0, TTL=2, Tag(s)=36 TAG: AT0/0/0.1: xmit: (no tag) TAG: Et0/1/0: recvd: CoS=0, TTL=2, Tag(s)=36 TAG: AT0/0/0.1: xmit: (no tag) San Jose# debug ip icmp ICMP: dst (194.22.15.2) port unreachable sent to 10.2.1.21 ICMP: dst (194.22.15.2) port unreachable sent to 10.2.1.21
Note
draft-ietf-mpls-icmp describes an extension to the traceroute facility to include MPLS label information. This is a useful extension and provides information not only on the path a packet takes, but also on the MPLS labels used throughout that path.
Although the previous description provides all the necessary functionality for traceroute to work in a Frame-mode MPLS environment, you also need to consider the effects of traceroute across an MPLS network that is constructed with ATM-LSRs in the topology.
Chapter 1, "Multiprotocol Label Switching (MPLS) Architecture Overview," defines an ATM-LSR as an LSR with a number of LC-ATM interfaces that forward cells between these interfaces using labels carried in the VPI/VCI field. The consequence of this is that TTL is not available in the header of an ATM cell and, therefore, cannot be manipulated at each hop in the network. For this reason, when ATM-LSRs are within the path, the ATM portion of the network is treated as one IP hop.
Note
draft-ietf-mpls-atm discusses the manipulation of the TTL in an ATM environment in section 10, "TTL Manipulation."
Note
You can disable the TTL in a Frame-mode network using the [no] tag-switching ip propagate-ttl command. Chapter 13 discusses this command in detail.
When this command is disabled, the IP TTL field is not copied into the MPLS TTL field during label imposition?a value of 255 is copied instead. This action effectively disables traceroute across the MPLS network and the output of the traceroute command shows only non-MPLS hops (hops where the packet is IP-forwarded) within the output.
Note
Because each ICMP "Time Exceeded" message is propagated using the original label stack of the packet, the delay output shown in the traceroute is no longer meaningful because it does not reflect an accurate delay encountered by packets traversing the backbone.
