Internet Access

Probably the most common VPN user requirement is that their service provider offer them Internet access in addition to VPN connectivity. However, being accessible from the Internet automatically is assumed to carry a certain risk for the VPN customer as well as for the MPLS VPN provider. This section discusses the various options for how to design an MPLS core for Internet access such that VPNs remain secure.

But is the Internet really so dangerous? Is a service provider not also at risk from connected VPNs? Can one VPN be a threat to another? As in all security questions it is difficult to draw a clear line, and of course a VPN user can also attack an MPLS core network.

The difference in security between a VPN user and the Internet consists mainly in the order of magnitude: A VPN user can attack the core, and a worm can come from a VPN also. But the size of the VPN user's network is usually several orders of magnitude smaller than the global Internet; therefore, the impact of an attack from the Internet is much higher.

Moreover, a service provider holds a contract with the VPN user but not with the Internet at large. This contract usually allows for counter measures to be taken: for example if DoS attacks are detected from a VPN connection, the connection can often be cut until the problem is resolved. Such drastic measures are often not possible with the Internet.

Therefore, the risk from VPNs versus from the Internet is not intrinsically different in type, but it is significantly different in the order of magnitude and in terms of legal possibilities. Internet provisioning must be done with security in mind.

Technically, there are a number of options for how to provide Internet services on an MPLS core. While all those options achieve their goal in connecting a VPN user to the Internet, their security properties are completely different. We therefore strongly recommended considering all of the Internet options in detail before making a design choice.

Technically, there are several ways to provide an Internet service on an MPLS core:

• Internet routes held in a VRF.

• Internet routes held in the global routing table. Here we distinguish two subcases:

  - The entire core holds the Internet routing (PE and P).

  - Only PEs hold the Internet routing, the so-called "Internet-free" MPLS core.

At first sight, all of those options appear to have similar properties; however, their security exposure varies significantly. The following section examines a core without Internet connectivity as a baseline to compare the other design options to. Then those options, as well as generic Internet design guidelines, are explained in detail.

MPLS Core Without Internet Connectivity

In this model, the MPLS core has no connection to the Internet. Only VPNs connect to the core, which means that all external connections end in a VPN routing/forwarding instance (VRF). This is the purest form of an MPLS VPN core, and all the considerations from the previous chapters apply directly:

• VPNs are completely separated, including addressing, data traffic, and routing.

• The core can be well secured against intrusions and DoS attacks from the outside.

• The core is invisible from the outside.

• VPN spoofing is impossible.

Note that all these items depend on correct configuration and operation of the core.

In this model, the entire core is inaccessible to the outside?that is, the connected VPNs, with the exception of the peering PE interfaces, which are part of the VPN. Figure 4-1 shows that those PE interfaces are the only interfaces to the outside, allowing for the core to be easily secured. Chapter 5, "Security Recommendations," describes this in detail.

Figure 4-1. MPLS Core Without Internet (VPN Only)

In this model, the core is by architecture unreachable, the security of internal parts of the network (the internal routing, LDP, SNMP, and so on) is guaranteed against attacks from the outside (see the following recommendation, however). The only way to threaten the security of the core is to attack from the inside: service provider engineers have the power to change security by mistake or deliberately. This threat is covered in Chapter 8, "Secure Operation and Maintenance of an MPLS Core."

The fact that no packets can get into the core may lead to the conclusion that the internal parts of the core do not need to be secured. Why protect Interior Border Gateway Protocol (iBGP) sessions with routing authentication with MD5 if it is impossible to send packets to the routers? This statement is true if and only if the network is 100 percent correctly implemented and operated. An engineer might connect a customer connection by mistake to a wrong interface, for example, terminating this connection in the global routing table.

In this case, the customer does have access to the core, and the usual security mechanisms (routing authentication, for example) are required. Therefore, it is strongly recommended to configure all possible security mechanisms, even if a single one is theoretically enough.

It is in reality impossible to know whether an MPLS core has Internet connectivity because a VPN customer might have a private Internet connection, unknown to the service provider. Figure 4-2 shows how, in this case, traffic from the Internet might travel over the core within the VPN of that customer. In the figure, VPN A has a private Internet connection at its left site, and a user on another site can access it. The VPN includes the peering PE interfaces, and that now makes the PE reachable from the Internet.

Figure 4-2. Hidden Internet in a VPN

As a consequence, the Internet traffic is implicitly part of a VPN, here VPN A. Note that routing does not necessarily change significantly within the VPN because default routing can be used. Has the security exposure on the MPLS core changed because of this hidden Internet service? Are other VPN customers on the same MPLS core now at greater risk than before?

For the MPLS core, the situation does not change significantly: the Internet has effectively become part of a VPN, but the fundamental properties of the MPLS architecture listed at the beginning of this chapter still hold true. There is a possibility of DoS attacks from the Internet across the core, but those would likely be limited by the access capacity of the VPN customer. In this example, if the user gets attacked from the Internet across the MPLS core, the maximum bandwidth on the core would be limited by the access line on the left VPN site. (In the section "Designing DoS-resistant Networks," later in this chapter, we examine the effect of DoS attacks against PE routers because theoretically an attack against a single PE may affect all VPNs connected to that PE.)

For other VPN users, the security also does not change significantly because of this hidden Internet connectivity: the separation between VPNs remains fully intact, and the only risk is flooding attacks. However, those attacks are limited by the access capacity, which in correct network capacity planning cannot do harm to core or other customers. And again, the service provider actually holds a contract with each VPN, which should allow for disconnection of sites if the site "misbehaves."

The important fact for security in this example is that all external routes are kept in a VRF. If there are hidden Internet connections, the security exposure is similar to the case where all the Internet routes are held in a VRF. This is also discussed in the section "Designing DoS-Resistant Networks."

In the following sections, we discuss various ways to provide Internet services on an MPLS VPN core: first some generic recommendations that apply to all models, and then each model in detail.

Generic Internet Design Recommendations

If the service provider chooses to provide an Internet service over its MPLS core, it has to follow general design guidelines. These are not specific to MPLS core networks but are listed here for completeness.

When providing Internet connectivity, the following factors have to be taken into account:

• The maximum ingress capacity from the Internet, limited by the line between Internet PE and CE or further upstream. Note that both the packet-per-second (pps) rate and the bit-per-second (bps) rate are important in this calculation: the capacity of routers and servers is determined by a pps rate, whereas the capacity of a core line is determined by bps.

• The switching capacity of the Internet PE and CE routers and of P routers in the core (measured in pps).

• The capacity of core lines (measured in bps).

• The capacity of PE routers where VPN customers are connected (measured in pps).

The MPLS core must be designed such that all components of the core can process a potential DoS attack from the Internet. Figure 4-3 illustrates the components to be taken into account.

Figure 4-3. Designing the Core to Sustain DoS Attacks

[View full size image]

The network design has to accommodate the worst-case scenario: This is a DoS attack from the Internet where all connections from the Internet are completely loaded and all this traffic goes to a single destination (the customer CE)^[1]. To design a network for this worst-case scenario, you must take into account the following considerations:

• The ingress capacity must be sufficient to serve the customers who have contracted an Internet service.

• Internet PE and CE routers must be able to forward the full theoretical maximum ingress bandwidth (in pps) from the Internet. In addition to pure packet forwarding, they must be able to apply security features on ingress. As a minimum, ACLs at line rate are required.^[2] These routers are paramount security tools and must be fully operational under all circumstances, including under a DoS attack. (Note: While the core routers should be fully operational, the customer under attack will be affected by the attack. The goal is to keep the core operational and to limit the impact on other customers.)

  Note

  The stability of the core routers in a network, including the routers to upstream providers, is the single most important factor for core security. Counter-measures against DoS attacks frequently involve features such as ACLs and committed access rate (CAR) on ingress routers. These features can only be applied if those routers are 100 percent stable, even under the full theoretical load.

  Modern routers, such as the Cisco 12000, can forward packets at full line rate, while applying features. This type of router is usually not affected by data plane DoS attacks.

  
  

• The core capacity must be designed such that even if a worst-case attack (full ingress capacity) is going to a single destination, VPN traffic can still be served according to the service level agreements (SLAs) with the VPN. In other words, even under extreme stress from the Internet, VPN users should not be affected. There are two ways of achieving this:

  - By providing sufficient capacity in the core

  - By using QoS mechanisms to prioritize VPN traffic over Internet traffic

• P routers must be able to switch traffic at line rate.

• Egress PE routers should be designed such that they cannot be affected by DoS attacks from the Internet. This requirement is hard to fulfill in practice: Internet capacity is usually larger than what small PEs can handle. The section "Designing DoS-Resistant Networks" discusses this PE design issue and explains ways to handle it.

The dimensioning of the customer capacity (the PE-CE link into a VPN), which requires Internet over this link, is not of concern to the service provider. It is the choice of the VPN customer, according to the customer's Internet needs. In practice, this link is usually overloaded under a DoS attack. This problem occurs on all types of networks and is not specific to MPLS VPNs.

The Internet CE (as shown in Figure 4-3) is a normal router, such as a router connecting to an Internet Exchange Point (IXP). In theory, the Internet PE could directly peer with ISPs. However, Figure 4-4 shows why it is an advantage if the Internet CE is also controlled by the provider of the MPLS core: this way, the address space between PE and CE does not have to be announced externally. This means that even if this address space is public, it can be blocked on the Internet CE to make the PE completely inaccessible from the outside. Also, infrastructure ACLs can be applied here, which protect all address space that is visible to the outside: the Internet PE's external address and all PEs facing Internet customers (not shown in Figure 4-4). Infrastructure ACLs are explained in Chapter 5. Naturally, an Internet CE is an additional expense that has to be considered when designing the Internet access.

Figure 4-4. Protection of the Internet PE

If the Internet PE does not carry any other VPNs, the recommendation to shield the Internet PE with another CE router can be disregarded. In that case, it does not make a difference if an attack goes against the Internet CE or the Internet PE. But if the Internet PE holds other VPNs as well, a DoS attack against the PE might affect those VPNs, too. In this case, the Internet CE adds significantly to the security of the VPNs connected to the Internet PE because it protects the Internet PE against direct attacks. The subject of PE routers that carry both Internet and VPNs is discussed in detail in the section "Designing DoS-Resistant Networks."

Previously, we mentioned that the dimensioning of routers in the MPLS core (PE and P) should be done such that they can handle all traffic from all Internet connections concurrently, for example under a DoS attack. This is, of course, unrealistic in practice: a small, customer-facing PE router would often not be able to sustain a flood from the Internet. Therefore, if this issue cannot be solved by design, it needs to be addressed on the operational level: The service provider must be able to detect and counteract packet floods from the Internet quickly, so that the SLAs with the customer can be met.

In other words, most of today's Internet networks assume a certain traffic distribution, which can be severely distorted under a DoS attack. There are two ways to make sure that the overall service, specifically the VPN part, remains operational even under a severe DoS attack:

• Design method? Each backbone router (PE and P) and each line must be designed such that a DoS attack does not affect VPN traffic or the backbone itself. This is difficult to achieve because capacities are not normally sufficient to cope with a DoS attack, especially on the customer edge router and line. However, in the section "Designing DoS-Resistant Networks" we explain how in order to achieve this goal a core can be built such that VPN traffic remains separate from Internet traffic. One way to do this is to strictly separate Internet and VPN PEs such that Internet floods never touch a VPN PE.

• Reaction method? If the network design is such that a DoS attack from the Internet might affect a portion of the network carrying VPN traffic, the alternative is to have a fast DoS detection and mitigation scheme to avert the DoS attack. In other words, if you know parts of your network will melt in a DoS attack, you should be very fast in detecting and blocking the DoS attack. If this reaction happens within the timelines guaranteed in the SLAs, this is a commercially viable solution.

To define a DoS strategy for your network, consider several levels of priority when it comes to an attack:

• Core? The core itself must never be affected by a DoS attack, first because a single affected core router is likely to affect a number of customers, and second because the service provider must be able to apply counter-measures on the core devices. This works only if core devices are fully operational. So the core is top priority.

• VPN traffic? This traffic usually has higher SLAs than Internet traffic; therefore, the next priority is to make sure VPN traffic is not affected by a DoS attack.

These first two goals must be achieved by a service provider. With correct design and operations, this is possible.

• Internet traffic to other destinations than the attacked one? If the core and VPNs are still working, the provider should try to make sure that Internet traffic to nonattacked customers is working. However, this goal might not be achievable anymore; for example, if the entire uplink into the service provider network is overloaded, it will affect all Internet customers. This can only be solved by upstream providers and is thus not under full control of the service provider any longer.

• Internet traffic to the site under attack is the lowest priority? In a first approach, the customer is actually paying to receive traffic from the Internet, which ironically includes DoS attacks. However, often the customer is relying on the provider to mitigate DoS attacks. So the provider should offer help, even though this service might incur an extra charge.

This section's recommendations are not only applicable to an MPLS backbone, but to a large extent to any Internet backbone. In the following sections, we go through the various MPLS-specific options for providing Internet services:

• Internet in a VRF

• Internet in the global routing table (with various suboptions)

Internet in a VRF

If the Internet is required on an MPLS VPN core, the most secure way is to provision it in a VRF. In this model, as in the previous example, Internet traffic stays within a VPN on the MPLS core, but here it is designed by the service provider.

Figure 4-5 illustrates the Internet in a VRF model. Upstream connections to other ISPs are connected to a PE router directly into a VRF. The MPLS core carries the Internet routes as if the Internet were just another VPN on the core network, and customers who need Internet connectivity connect into that VPN.

Figure 4-5. Internet Provisioned in a VRF

The Internet router shown in Figure 4-5 is seen by the core as a CE. Since Internet is just another VPN on the core, all the statements at the beginning of this chapter hold true here. The difference to a normal VPN is contractual: this CE is not a customer, but part of the overall service.

All the security features discussed in previous chapters also hold true for the case of Internet in a VRF. Here's a summary of them:

• Separation between VPNs is maintained because Internet is handled just the same as a VPN.

• The core can be secured against attacks from the outside, specifically from the Internet; because just like other VPNs, the Internet has no access to the core.

• The core is invisible to the outside. This also applies if the Internet is provided in a VRF because it constitutes another VPN.

• Spoofing is impossible between VPNs and the Internet in a VPN, following the reasons discussed in Chapter 3, "MPLS Security Analysis."

These architectural features of MPLS VPN networks do not require specific configurations. As discussed in the previous section, packet floods are one remaining potential issue from the Internet: there is a possibility that large DoS attacks may have an impact on the PEs or on core lines. This might indirectly also affect VPN customers. The solution is to either overprovision the core or to use QoS to give VPN traffic a higher precedence. (Please refer to the previous section for details.)

If the Internet is provided in a VRF, the overall network benefits from the intrinsic security properties of the MPLS architecture. The downside of this approach is scalability, as previously discussed. Therefore, many service providers choose to carry Internet in the global routing table.

Internet in the Global Routing Table

Due to memory limitations and the previously mentioned issue of VRF prefixes needing more memory than prefixes in the global table, many providers prefer to keep Internet routing in the global table. There are two ways to implement this:

• Configure the MPLS core like a traditional IP core, with each router holding the full routing table.

• Use the option called Internet-free core.

Hop-by-Hop Internet Routing in the Core

Traditionally, the Internet follows the hop-by-hop routing paradigm, where every router makes a forwarding decision based on its local routing information. This paradigm requires every router to know a route to every destination on the Internet that needs to be reached. Smaller ISPs and their customers usually use default routing to avoid having to keep the full routing table. The large ISPs in the core of the Internet run their core "default-free," which means their core routers hold the full routing table and therefore do not require a default route.

MPLS offers an alternative to the hop-by-hop routing behavior: As described in the previous section, only the PE routers need to know routes external to the MPLS core. P routers only know routes within their own autonomous system (AS). The forwarding on the core is done using label switch paths (LSPs), ATM virtual circuits (VCs), or another encapsulation method such as GRE, L2TPv3, or IPsec.

However, the two paradigms can be combined on a single core network: the PEs carry VPN-related information in VRFs, and the Internet routing table in the global routing table. Figure 4-6 illustrates this setup.

Figure 4-6. MPLS with Hop-by-Hop Internet Routing in the Core

In most cases where such a design is used, all PE and P routers hold the entire Internet routing table. PEs that connect only VPN customers would not require the Internet routing table; however, for consistency reasons, service providers often have the same routing policy for all PE routers. Note that in a default MPLS core, all next hops on a PE router will resolve to an egress label, such that an LSP would be used. Special configurations are required to actually get hop-by-hop routing on an MPLS core.

In this model, one of the important security aspects of MPLS security does not apply: here, all routers are accessible from the Internet by default. From a security point of view, two aspects have to be analyzed for each router in the network:

• Reachability from the Internet to a router? Packets originating from the Internet can be addressed to a router. This exposes the router to directed attacks and should therefore be avoided or at least limited in scale if possible.

• Reachability from a router to the Internet? The router can reach destinations on the Internet. This may seem less relevant for security, but it is important: it allows connections to be established?for example, an SSH control connection. If the router does not know how to reach Internet addresses, return traffic will not reach potential attackers on the Internet, making two-way connections impossible.

To reach a router from the Internet, one of the addresses of this router must be routed on the Internet. To reach the Internet from a router, the router needs to keep the Internet routing table. In MPLS VPN networks without Internet, the core is never accessible from the outside. By importing the Internet routing table into the core, and by announcing at least some of the core address area to the Internet, the separation between the core and the Internet is fundamentally broken.

This in itself is not specific to this architecture: traditional Internet networks without MPLS have likewise always had this property; nevertheless they where operated securely. If the architecture of the core network does not provide separation, additional security measures must be taken.

In traditional IP networks, each network element is secured independently. The assumption here is that the device might be hit by attacks any time; therefore, all possible device security measures have to be taken.

In addition, since about 2003, several providers have started to implement infrastructure ACLs. The idea is to block all traffic into the core on ingress (see Figure 4-7).

Figure 4-7. Infrastructure ACLs

Infrastructure ACLs are applied on all edges into the core for all packets destined to core devices:

deny ip any core-address-space

permit ip any any

Because the core address space usually cannot be expressed in a single prefix, several deny statements are usually needed. There will also be exceptions in practice, such as for routing protocols to the PE router. The last line permits transit traffic across the core. Infrastructure ACLs are discussed in more detail in Chapter 5.

With infrastructure ACLs on the ingress ports into the core, reachability of the core can be controlled. There are other ways for achieving the same goal (for example, by not announcing the internal address space to the Internet or by using private address space on the core). All of those mechanisms have the common goal of securing the core against attacks from the outside by limiting reachability to the core. When Internet on an MPLS core is provisioned in the global routing table, such measures are required.

In summary, Internet in the global routing table makes all core devices accessible and, therefore, attackable from the outside by default. So a strong layer of defense around the entire core, limiting reachability of the core, is required. As in every model, every router must be individually secured.

There is also a compromise between the two models (Internet in a VRF and Internet in the global routing table): the Internet-free core.

Internet-Free MPLS Core

The Internet-free core is a hybrid between Internet in a VRF and Internet in the global routing table. Although the Internet routes are carried in the global table on the PE routers, the P routers do not carry Internet routes. Instead, LSPs (or other tunnels) are used to forward Internet traffic. Figure 4-8 illustrates this setup.

Figure 4-8. Internet-Free Core

The PE routers hold the Internet routing table in their global routing table. Also, in the previous case, P routers held Internet routing information distributed by iBGP; here, the P routers only run an Interior Gateway Protocol (IGP), in which only reachability of PEs is advertised. The PEs maintain iBGP peering with all other PEs directly or through route reflectors. But because here the reachability of the egress PE is achieved through the IGP and because the Label Distribution Protocol (LDP) is used to establish LSPs between the PEs, these LSPs are also used for Internet traffic. Therefore, the P routers do not require Internet routing information in this model.

Schematically, this setup is the same as when forwarding between VRFs. But here, instead of the VRF table, the global routing table is used.

In this model, the PEs maintain Internet routes and are thus by default fully reachable from the Internet. Therefore, the PE routers need to be secured as in a traditional core network: every router independently, plus infrastructure protection, as described in the previous section.

It may appear at first glance that P routers are not reachable from the outside, but this is not the case. P routers do not hold the Internet routing table; therefore, they cannot send packets into the Internet. However, the opposite direction is open: the Internet PE router holds all Internet routes in its global routing table, but it also needs the IGP routes there for the interior routing within the core. Therefore, if a packet reaches the PE with a destination of one of the P routers, the PE router does have a route to the P router and will forward the packet.

The consequence is that, by default, traffic from the Internet can reach any PE or P router in this setup, but only PE routers have the route back to the Internet. Therefore, a PE router can receive and send traffic to the Internet, allowing full connections to be established. A P router may be accessible from the Internet unidirectionally, but it cannot send traffic into the Internet. This allows at least a DoS attack against the P router.

To secure such a network, the same mechanisms are required as in the previous section: device security plus infrastructure ACLs. With those mechanisms, an Internet-free core can be secured like any other IP backbone.

Overview of Internet Provisioning

The Internet is perceived to be the source of attacks against network infrastructure. While attacks might come also from connected customers, those are usually easier to handle than attacks from the Internet. Therefore, you must be careful how you provision Internet services on an MPLS VPN core network.

Table 4-1 summarizes the default reachability for various options for providing Internet on an MPLS VPN backbone and compares it to an MPLS VPN backbone without Internet service.

Table 4-1. Default Reachability for Various Options for Providing Internet Services

	MPLS Without Internet	Internet in VRF	Internet in Global Table	Internet-Free Core
PE reachable	No	Yes	Yes	Yes
P reachable	No	No	Yes, bidirectional	Yes, unidirectional
VPNs reachable	No	No	No	No

Table 4-1 shows the default behavior of various MPLS VPN architectures. This does not necessarily imply a security issue! Reachable devices just require additional security measures. The traditional Internet was built with the paradigm that every router is reachable, yet it can be secured.

Specifically, MPLS VPNs are not reachable from the outside, independently of which of the before-mentioned Internet options is chosen.

The various options discussed here, however, impact two areas:

• Security of the service provider network? Devices that are unreachable cannot be hacked. If, for example, a PE router is reachable, it is exposed to attempts to intrude into the device and potentially alter the configuration. This may have a secondary effect on VPNs if security mechanisms on the PE are disabled, for example. This may also lead to a breach of separation between VPNs, depending on the PE configuration. This has to be secured with traditional security measures.

• Exposure to DoS attacks? A reachable device can also receive a DoS attack, which might affect availability of the backbone and, with that, potentially connected VPNs. If a router is not reachable, it can only be attacked with transit traffic, and that is significantly more difficult.

In addition, in an MPLS VPN environment all external connections, to the Internet as well as to customers, must be secured. Chapter 5 explains the necessary security measures in detail.

This way, either of the Internet provisioning models can be made secure; the difference lies in the number of security mechanisms that protect the core. Architectural security is in every case stronger because it cannot be misconfigured.

As you can see from Table 4-1, Internet in a VRF provides the highest security level of all Internet options. However, there are a number of other design considerations to be taken into account, such as memory consumption. Routes in VRF require significantly more memory than in the global table. Therefore, decisions on design options cannot be taken solely based on security considerations.

For all options, additional security mechanisms are available to further secure the backbone. Looking at the MPLS market, the various types of Internet architectures described here have all been successfully implemented.

In addition to VPN and Internet connections into an MPLS core, it is possible to provide extranets?shared areas that several VPNs can access.

Copyright eTutorials.org 2008-2023. All rights reserved.