Route Cloning
Cloned routes are a concept unique to BSD networks stacks. The concept refers to on-demand generation (cloning) of host routes (/32). In other words (quoted from the FreeBSD arp(4) manual page), "The ARP cache is stored in the system routing table as dynamically created host routes. The route to a directly attached Ethernet network is installed as a 'cloning' route (one with the RTF_CLONING flag set), causing routes to individual hosts on that network to be created on demand."[1] The actual cloning template (or parent) is marked with (C = generate new routes on use), the instantiated cloned host route (child) with (W = was cloned) in the system routing table. The associated ref_counter indicates how many existing connections use that particular entry, which is also correlated with an expire_timer (usually 3600 seconds). Cloned routes time out periodically after initial validation as long as they are not used.
Examples 8-3 through 8-5 show the differences in arp and netstat command output on OpenBSD, Linux, and FreeBSD operating systems to demonstrate the connection between next-hop/interface Media Access Control (MAC) resolution and similarities between route and netstat commands. In addition, interface statistics with netstat are presented, as are usage statistics of routing table entries. All routing tables present prefix entries, flags, a reference counter for the number of uses of a prefix, and a usage counter for the number of packets that were forwarded along that route out of the associated physical interface. Additional parameters of netstat output are system-specific.
Example 8-3. OpenBSD arp and netstat Output
[root@ganymed:~#] arp -an ? (192.168.1.1) at 52:54:05:e3:51:87 ? (192.168.1.2) at 08:00:46:64:74:1b ? (192.168.2.7) at 00:10:5a:c4:2c:04 ? (111.11.117.1) at 00:05:9a:5b:23:fc [root@ganymed:~#] netstat -rna -f inet Routing tables Internet: Destination Gateway Flags Refs Use Mtu Interface default 111.11.117.1 UGS 3 11991 1500 ne5 127/8 127.0.0.1 UGRS 0 0 33224 lo0 127.0.0.1 127.0.0.1 UH 2 0 33224 lo0 192.168.1/24 link#1 UC 0 0 1500 ne3 192.168.1.1 52:54:5:e3:51:87 UHL 0 8801 1500 ne3 192.168.1.2 8:0:46:64:74:1b UHL 1 4451 1500 ne3 192.168.1.254 127.0.0.1 UGHS 0 0 33224 lo0 192.168.2/24 link#2 UC 0 0 1500 ne4 192.168.2.7 0:10:5a:c4:2c:4 UHL 0 2111 1500 ne4 192.168.44.1 192.168.44.1 UH 0 0 33224 lo1 192.168.45/24 link#1 UC 0 0 1500 ne3 111.11.117/24 link#3 UC 0 0 1500 ne5 111.11.117.1 0:5:9a:5b:23:fc UHL 1 0 1500 ne5 [root@ganymed:~#] netstat -in -f inet Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Colls lo0 33224 <Link> 0 0 0 0 0 lo0 33224 fe80::/64 fe80::1 0 0 0 0 0 lo0 33224 ::1/128 ::1 0 0 0 0 0 lo0 33224 127/8 127.0.0.1 0 0 0 0 0 lo1 33224 <Link> 0 0 0 0 0 lo1 33224 192.168.44/ 192.168.44.1 0 0 0 0 0 lo1 33224 fe80::/64 fe80::1 0 0 0 0 0 lo1 33224 ::1/128 ::1 0 0 0 0 0 ne3 1500 <Link> 48:54:e8:8c:0a:3f 17263 0 13427 0 329 ne3 1500 192.168.1/2 192.168.1.254 17263 0 13427 0 329 ne3 1500 fe80::/64 fe80::4a54:e8ff:f 17263 0 13427 0 329 ne3 1500 192.168.45/ 192.168.45.254 17263 0 13427 0 329 ne4 1500 <Link> 52:54:05:e3:e4:2f 2503 234 2247 0 0 ne4 1500 192.168.2/2 192.168.2.254 2503 234 2247 0 0 ne4 1500 fe80::/64 fe80::5054:5ff:fe 2503 234 2247 0 0 ne5 1500 <Link> 52:54:05:e3:51:87 11531 1253 12040 0 0 ne5 1500 111.11.117/ 111.11.117.206 11531 1253 12040 0 0 ne5 1500 fe80::/64 fe80::5054:5ff:fe 11531 1253 12040 0 0 [root@ganymed:~#] netstat -rs routing: 0 bad routing redirects 0 dynamically created routes 0 new gateways due to redirects 10 destinations found unreachable 0 uses of a wildcard route
Example 8-4 also demonstrates an advanced feature of Linux: TCP parameters such as the TCP Maximum Segment Size (MSS) and the TCP Window Size, which can be altered on a per-prefix basis (shaded text). For a better understanding, consider the following technical details quoted from the Linux route(8) manual page:
mss M:
set the TCP Maximum Segment Size (MSS) for connections over this route to M bytes. The default is the device MTU minus headers, or a lower MTU when path mtu discovery occurred [sic]. This setting can be used to force smaller TCP packets on the other end when path mtu discovery does not work (usually because of misconfigured firewalls that block ICMP Fragmentation Needed)
window W:
set the TCP window size for connections over this route to W bytes. This is typically only used on AX.25 networks and with drivers unable to handle back to back frames.[2]
Example 8-4. Linux arp and netstat Output
[root@callisto:~#] arp -an ? (192.168.1.2) at 08:00:46:64:74:1B [ether] on eth1 ? (192.168.1.254) at 48:54:E8:8C:0A:3F [ether] on eth1 [root@callisto:~#] netstat -rnva Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 192.168.1.0 0.0.0.0 255.255.255.0 U 40 0 0 eth1 192.168.1.0 0.0.0.0 255.255.255.0 U 40 0 0 ipsec0 192.168.14.0 0.0.0.0 255.255.255.0 U 40 0 0 eth0 127.0.0.0 0.0.0.0 255.0.0.0 U 40 0 0 lo 0.0.0.0 192.168.1.254 0.0.0.0 UG 40 0 0 eth1 [root@callisto:~#] netstat -i Kernel Interface table Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg eth0 1500 0 276 0 0 0 166 0 0 0 BMRU eth1 1500 0 14889 0 0 0 9260 0 0 0 BMRU ipsec 16260 0 0 0 0 0 0 0 0 0 ORU lo 16436 0 64 0 0 0 64 0 0 0 LRU [root@callisto:~#] route -nee Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface MSS Window irtt 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 40 0 0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ipsec0 40 0 0 192.168.14.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 40 0 0 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo 40 0 0 0.0.0.0 192.168.1.254 0.0.0.0 UG 0 0 0 eth1 40 0 0
The highlighted text in Example 8-5 emphasizes the timer correlation of ARP cache entries and the forwarding table on FreeBSD for cloned routes (ARP neighbors). On BSD systems, you can manually adjust the route_expire sysctl parameter net.inet.ip.rtexpire, which defaults to 3600 seconds. Connected routes are created for each interface attached to the local host. Examples of the ip Linux facility are left to the lab because it is specific only to Linux, whereas netstat and route are generic tools of all Unices.
Example 8-5. FreeBSD arp and netstat Output
[root@castor:~#] arp -an ? (192.168.2.254) at 52:54:05:e3:e4:2f on xl0 [ethernet] ? (192.168.7.254) at 00:00:0c:1a:a9:a8 on ed0 [ethernet] [root@castor:~#] netstat -rnaW -f inet Routing tables Internet: Destination Gateway Flags Refs Use Mtu Netif Expire default 192.168.2.254 UGSc 4 6 1500 xl0 127.0.0.1 127.0.0.1 UH 0 0 16384 lo0 192.53.103.103 192.168.2.254 UGHW3 0 63 1500 xl0 3314 192.53.103.104 192.168.2.254 UGHW 1 64 1500 xl0 192.168.1.2 192.168.2.254 UGHW 1 1207 1500 xl0 192.168.2 link#1 UC 2 0 1500 xl0 192.168.2.254 52:54:05:e3:e4:2f UHLW 3 3 1500 xl0 1028 192.168.7 link#2 UC 1 0 1500 ed0 192.168.7.254 00:00:0c:1a:a9:a8 UHLW 1 5 1500 ed0 1038 195.34.133.10 192.168.2.254 UGHW3 0 14 1500 xl0 3440 [root@castor:~#] netstat -i -f inet Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll xl0 1500 192.168.2 192.168.2.7 2260 - 3303 - - ed0 1500 192.168.7 castor 260 - 1214 - - lo0 16384 your-net localhost 0 - 0 - - [root@castor:~#] netstat -rs routing: 0 bad routing redirects 0 dynamically created routes 0 new gateways due to redirects 3 destinations found unreachable 0 uses of a wildcard route 1 route not in table but not freed
