Securing the Core
In this section, we discuss security considerations for the core network as a whole. That is, once all routers are secured independently, securing the network?which includes network-wide security measures such as infrastructure access lists (iACLs)?must be done by the SP network manager.
Infrastructure Access Lists (iACLs)
In this subsection, we emphasize that the core routers do not need to be accessible from the outside (with few exceptions, such as routing). If you block all packets from reaching the core routers, you are able to avert most attack forms.
In an effort to protect routers from various risks?both accidental and malicious?infrastructure protection ACLs should be deployed at network ingress points. These IPv4 and IPv6 ACLs deny access from external sources to all infrastructure addresses such as router interfaces; at the same time, the ACLs permit routine transit traffic to flow uninterrupted and provide basic RFC 1918, RFC 3330, and antispoof filtering.
Data received by a router can be divided into two broad categories?traffic that passes through the router via the forwarding path and traffic destined for the router via the receive path for route processor handling. In normal operations, the vast majority of traffic simply flows through a router in route to its ultimate destination. However, the route processor (RP) must handle certain types of data directly, most notably routing protocols, remote router access (such as Secure Shell [SSH]), and network management traffic such as Simple Network Management Protocol (SNMP). In addition, protocols such as Internet Control Message Protocol (ICMP) and IP options might require direct processing by the RP. Most often, direct infrastructure router access is required only from internal sources. A few notable exceptions include External Border Gateway Protocol (eBGP) peering, protocols that terminate on the actual router (such as generic routing encapsulation [GRE] or IPv6 over IPv4 tunnels), and potentially limited ICMP packets for connectivity testing such as echo-request or ICMP unreachables and time-to-live (TTL) expired messages for traceroute. Please bear in mind that ICMP is often used for simple denial of service (DoS) attacks and should only be permitted from external sources if necessary.
All RPs have a performance envelope in which they operate. Excessive traffic destined for the RP can overwhelm the router, causing high CPU usage and ultimately resulting in packet and routing protocol drops that cause a denial of service. By filtering access to infrastructure routers from external sources, many of the external risks associated with direct router attacks are mitigated. Externally sourced attacks can no longer access infrastructure equipment; the attack is simply dropped on ingress interfaces into the autonomous system (AS).
The filtering techniques described in this document are intended to filter data destined for network infrastructure equipment. Do not confuse infrastructure filtering with generic filtering; the infrastructure protection ACL has a singular purpose: to control on a granular level what protocols and sources can access critical infrastructure equipment.
Network infrastructure equipment encompasses the following:
All router and switch management addresses, including loopback interfaces
All internal link addresses: router-to-router links (point-to-point and multiple access)
Internal servers or services that should not be accessed from external sources
In this subsection, transit traffic refers to all traffic not destined for the infrastructure. The implementation of iACLs is the foundation block for other security mechanisms such as rACL and CoPP mechanisms.
Techniques
Infrastructure protection can be achieved using a variety of techniques, such as the following:
Individual router ACLs? Routers can also be protected by defining ACLs that permit only authorized traffic to the router's interfaces, denying all others except for transit traffic, which must be explicitly permitted. This ACL is logically similar to an rACL but does affect transit traffic and therefore can have a negative performance impact on a router's forwarding rate.
NOTE
Note, however, that in an MPLS network where Penultimate Hop Popping (PHP) may be functioning but an explicit null could be activated for LDP, a labeled packet is not filtered by IP ACLs, so a hop-by-hop router filter mechanism may not be appropriate here. Instead, the use of individual router ACLs is recommended for the network engineer. Additionally, routers deploying 6PE possess a construct where an incoming IPv6 packet is always labeled only emphasizing the importance of individual router ACL implementation rather than a hop-by-hop construct.
Edge filtering via infrastructure ACLs? ACLs can be applied to the edge of the network; in the case of a service provider (SP), this would be the edge of the AS. This ACL will explicitly filter traffic destined for infrastructure address space. Deployment of edge infrastructure ACLs requires that you clearly define your infrastructure space and the required/authorized protocols that access this space. The ACL is applied at ingress to your network on all externally facing connections, such as peering connections, customer connections, and so on.
NOTE
There is an important difference between the use of rACLs and the edge or individual router ACLs: The first is the explicit permit type that terminates with a deny any any, while the latter is an explicit deny with a permit any any at the end. Theoretically, the first is more secure, and we recommend deploying both types because of the differences in command principles.
ACL Examples
The IPv4 and IPv6 access lists shown in Examples 5-12 and 5-13 provide simple yet realistic examples of typical entries required in a protection ACL. These basic ACLs need to be customized with local, site-specific configuration details. In dual IPv4 and IPv6 environments, both types of access lists should be deployed.
Example 5-12. IPv4 Example
!--- Antispoofing entries are shown here. !--- Deny special-use address sources. !--- Refer to RFC 3330 for additional special-use addresses. access-list 110 deny ip host 0.0.0.0 any access-list 110 deny ip 127.0.0.0 0.255.255.255 any access-list 110 deny ip 192.0.2.0 0.0.0.255 any access-list 110 deny ip 224.0.0.0 31.255.255.255 any !--- Filter RFC 1918 space. access-list 110 deny ip 10.0.0.0 0.255.255.255 any access-list 110 deny ip 172.16.0.0 0.15.255.255 any access-list 110 deny ip 192.168.0.0 0.0.255.255 any !--- Deny your space as source from entering your AS. !--- Deploy only at the AS edge. access-list 110 deny ip YOUR_CIDR_BLOCK any !--- Permit BGP. access-list 110 permit tcp host bgp_peer host router_ip eq bgp access-list 110 permit tcp host bgp_peer eq bgp host router_ip !--- Deny access to internal infrastructure addresses. access-list 110 deny ip any INTERNAL_INFRASTRUCTURE_ADDRESSES !--- Permit transit traffic. access-list 110 permit ip any any
NOTE
We recommend that the network engineer validate all examples in a test lab environment that reflects the appropriate service provider network reference architecture. Example 5-12 could result in performance issues associated with a 12000 Engine 2 line card because filtering on the source port is performed on the Line Card CPU in software. One ISP uses the following command example as a workaround:
access-list 110 permit tcp host bgp peer host router ip gt 10000
Example 5-13. IPv6 Example (IPv6 access-list must be applied as an extended, named access-list)
!--- Configure the access-list. ipv6 access-list iacl !--- Deny your space as source from entering your AS. !--- Deploy only at the AS edge. deny ipv6 YOUR_CIDR_BLOCK_IPV6 any !--- Permit multiprotocol BGP. permit tcp host bgp_peer_ipv6 host router_ipv6 eq bgp permit tcp host bgp_peer_ipv6 eq bgp host router_ipv6 !--- Deny access to internal infrastructure addresses. deny ipv6 any INTERNAL_INFRASTRUCTURE_ADDRESSES_IPV6 !--- Permit transit traffic. permit ipv6 any any
NOTE
The log keyword can be used to provide additional detail about source and destinations for a given protocol. Although this keyword provides valuable insight into the details of ACL hits, excessive hits to an ACL entry that uses the log keyword increase CPU utilization. The performance impact associated with logging varies by platform. Also, using the log keyword disables Cisco Express Forwarding (CEF) switching for packets that match the access-list statement. Those packets are fast switched instead.
Developing a Protection ACL
In general, an infrastructure ACL is composed of four sections:
Special-use address and antispoofing entries that deny illegitimate sources and packets with source addresses that belong within your AS from entering the AS from an external source
NOTE
RFC 3330 defines IPv4 special-use addresses that might require filtering. RFC 1918 defines an IPv4 reserved address space that is not a valid source address on the Internet. RFC 3513 defines the IPv6 addressing architecture. RFC 2827 provides ingress filtering guidelines.
Explicitly permitted externally sourced traffic destined to infrastructure addresses
deny statements for all other externally sourced traffic to infrastructure addresses
permit statements for all other traffic for "normal" backbone traffic in route to noninfrastructure destinations
The final line in the infrastructure ACL will explicitly permit transit traffic: permit ip any any for IPv4 and permit ipv6 any any for IPv6. This entry ensures that all IP protocols are permitted through the core and that customers can continue to run applications without issues.
The first step when developing an infrastructure protection ACL is to understand the required protocols. Although every site has specific requirements, certain protocols are commonly deployed and will be well understood. For instance, external BGP to external peers needs to be explicitly permitted. Note that because both sides can initiate a BGP session (a session to TCP port 179), both directions must be permitted in the iACL. Any other protocols that require direct access to the infrastructure router will need to be explicitly permitted as well. For example, if you terminate a GRE tunnel on a core infrastructure router, then protocol 47 (GRE) also needs to be explicitly permitted. Similarly, if you terminate an IPv6 over IPv4 tunnel on a core infrastructure router, then protocol 41 (IPv6 over IPv4) also needs to be explicitly permitted.
To help identify the required protocols, a classification ACL should be used. The classification ACL is composed of permit statements for the various protocols that could be destined for an infrastructure router. Using the show access-list command to display a count of access control entry (ACE) hits will identify required protocols. Suspicious or surprising results should be investigated and well understood prior to creating permit statements for unexpected protocols.
For example, the IPv4 ACL shown in Example 5-14 would help determine whether GRE, IPsec (ESP), and IPv6 tunneling (IP Protocol 41) need to be permitted.
Example 5-14. IPv4 ACL to Permit GRE, Ipsec, and IPv6 Tunneling
access-list 101 permit GRE any infrastructure_ips access-list 101 permit ESP any infrastructure_ips access-list 101 permit 41 an infrastructure_ips access-list 101 permit ip any infrastructure_ips log !--- The log keyword provides more details !--- about other protocols that are not explicitly permitted. access-list 101 permit ip any any interface int ip access-group 101 in This IPv6 ACL can be used to determine if GRE and IPsec (ESP) need to be permitted. ipv6 access-list determine_protocols permit GRE any infrastructure_ips_ipv6 permit ESP any infrastructure_ips_ipv6 permit ipv6 any infrastructure_ips_ipv6 log !--- The log keyword provides more details !--- about other protocols that are not explicitly permitted. permit ipv6 any any interface int ipv6 traffic-filter determine_protocols in
In addition to required protocols, infrastructure address space needs to be identified because this is the space the ACL will be protecting. Infrastructure address space includes any addresses that are used for the internal network and should rarely, if ever, be accessed by external sources such as router interfaces, point-to-point link addressing, and critical infrastructure services. Since these addresses will be used for the destination portion of the infrastructure ACL, summarization is critical and, wherever possible, these addresses should be grouped into classless interdomain routing (CIDR) blocks.
Using the protocols and addresses identified, the infrastructure ACL can be built to permit the protocols and protect the addresses. In addition to direct protection, the ACL should also provide a first line of defense against certain types of invalid traffic on the Internet:
RFC 1918 space should be denied.
Packets with a source address that falls under special-use address space, as defined in RFC 3330, should be denied.
Antispoof filters should be applied. (Your address space should never be the source of packets from outside your AS.)
This newly constructed ACL should be applied inbound on all ingress interfaces.
iACL Risk Assessment
When deploying infrastructure protection ACLs, remember to consider two key areas of risk:
Ensure that the appropriate permit/deny statements are in place. For the ACL to be effective, all required protocols must be permitted and the correct address space must be protected by the deny statements.
ACL performance varies from platform to platform. Before deploying ACLs, review the performance characteristics of your hardware.
NOTE
Note that the best practice recommendation is to implement rACL, CoPP, and iACLs in order to secure core infrastructure routers.
As always, we recommend that you test this design in the lab prior to deployment. Also, do not forget to update the ACLs when changes to the core are made, specifically when more address space is added.
Deployment Examples
We provide deployment examples for both IPv4 and IPv6 in the following subsections.
IPv4
The IPv4 example shown in Example 5-15 shows an infrastructure ACL protecting a router based on the following addressing:
The service provider's address block is 169.223.0.0/16.
The service provider's infrastructure block is 169.223.252.0/22.
The loopback for the router is 169.223.253.1/32.
The router is a peering router and peers with 169.254.254.1 (to address 169.223.252.1).
The infrastructure protection ACL shown in Example 5-15 was developed based on this information. The ACL permits external BGP peering to the external peer, provides antispoof filters, and protects the infrastructure from all external access.
Example 5-15. IPv4
! no access-list 110 ! ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !--- Phase 1 ? Antispoofing Denies !--- These ACEs deny fragments, RFC 1918 space, !--- invalid source addresses, and spoofs of !--- internal space (space as an external source). ! !--- Deny fragments to the infrastructure block. access-list 110 deny tcp any 169.223.252.0 0.0.3.255 fragments access-list 110 deny udp any 169.223.252.0 0.0.3.255 fragments access-list 110 deny icmp any 169.223.252.0 0.0.3.255 fragments !--- Deny special-use address sources. !--- See RFC 3330 for additional special-use addresses. access-list 110 deny ip host 0.0.0.0 any access-list 110 deny ip 127.0.0.0 0.255.255.255 any access-list 110 deny ip 192.0.2.0 0.0.0.255 any access-list 110 deny ip 224.0.0.0 31.255.255.255 any !--- Filter RFC 1918 space. access-list 110 deny ip 10.0.0.0 0.255.255.255 any access-list 110 deny ip 172.16.0.0 0.15.255.255 any access-list 110 deny ip 192.168.0.0 0.0.255.255 any !--- Deny our internal space as an external source. !--- This is only deployed at the AS edge. access-list 110 deny ip 169.223.0.0 0.0.255.255 any !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !--- Phase 2 ? Explicit Permit !--- Permit only applications/protocols whose destination !--- address is part of the infrastructure IP block. !--- The source of the traffic should be known and authorized. ! !--- Note: This template must be tuned to the network's !--- specific source address environment. Variables in !--- the template need to be changed. !--- Permit external BGP. access-list 110 permit tcp host 169.254.254.1 host 169.223.252.1 eq bgp access-list 110 permit tcp host 169.254.254.1 eq bgp host 169.223.252.1 ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !--- Phase 3 ? Explicit Deny to Protect Infrastructure access-list 110 deny ip any 169.223.252.0 0.0.3.255 ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !--- Phase 4 ? Explicit Permit for Transit Traffic access-list 110 permit ip any any
IPv6
The IPv6 example shown in Example 5-16 shows an infrastructure ACL protecting a router based on the following addressing:
The service provider's address block is 3FFE:B00:C18:::/48.
The router is a peering router and peers with 3FFE:B00:C19:2:1::F (to address 3FFE:B00:C18:2:1::1).
The infrastructure protection ACL shown in Example 5-16 was developed based on this information. The ACL permits external multiprotocol BGP peering to the external peer, provides antispoof filters, and protects the infrastructure from all external access.
Example 5-16. IPv6
no ipv6 access-list iacl ipv6 access-list iacl !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !--- Phase 1 ? Antispoofing and Fragmentation Denies !--- These ACEs deny fragments and spoofs of !--- internal space as an external source. !--- Deny fragments to the infrastructure block. deny tcp any 3FFE:B00:C18:3::/64 fragments deny udp any 3FFE:B00:C18:3::/64 fragments deny icmp any 3FFE:B00:C18:3::/64 fragments !--- Deny our internal space as an external source. !--- This is only deployed at the AS edge. deny ipv6 3FFE:B00:C18:::/48 any !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !--- Phase 2 ? Explicit Permit !--- Permit only applications/protocols whose destination !--- address is part of the infrastructure IP block. !--- The source of the traffic should be known and authorized. !--- Note: This template must be tuned to the network's !--- specific source address environment. Variables in !--- the template need to be changed. !--- Permit multiprotocol BGP. permit tcp host 3FFE:B00:C19:2:1::F host 3FFE:B00:C18:2:1::1 eq bgp permit tcp host 3FFE:B00:C19:2:1::F eq bgp host 3FFE:B00:C18:2:1::1 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !--- Phase 3 ? Explicit Deny to Protect Infrastructure deny ipv6 any 3FFE:B00:C18:3::/64 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !--- Phase 4 ? Explicit Permit for Transit Traffic permit ipv6 any any
