URL Filtering

As of Cisco IOS 12.2(11)YU and 12.2(15)T, the Cisco IOS supports URL filtering. One of the problems of using extended ACLs or even CBAC inspection with Java blocking is that your router filters traffic only based on what is in either the ACL or the inspection statement. As an example, imagine that you want to prevent users from accessing pornography, gambling, and peer-to-peer file-sharing sites. Theoretically, you could do this with extended ACLs; however, you would have to know about the IP address of every web page (not just every website). Unfortunately, you cannot screen web pages with normal extended ACLs?only Layer 4 and lower information. Even if you decide to block all pages from these web servers, you easily would have tens of thousands of ACL entries to create and maintain.

Because of these issues, ACLs typically are not used to implement these kinds of policies. Instead, a content-filtering server is used. Cisco currently supports server products from N2H2 (Sentian) and Websense to perform this filtering process. Actually, the content-filtering servers contain the access policies, and your Cisco router uses these external policies to enforce URL access from your users.

Operation of URL Filtering

To help you understand how URL filtering works, take a look at a simple example. The network shown in Figure 10-2 illustrates how URL filtering works.

Figure 10-2. URL Filtering Process

[View full size image]

In this example, an internal user sends an HTTP request to download information from an external web server. Here are the steps that occur through the HTTP request and response process, with reference to the numbering in Figure 10-2:

One of the interesting things about this process is that the router allows the initial HTTP request to exit the network instead of first verifying the user's access. This allows the router, in most cases, to receive a response from the content-filtering server before the HTTP reply comes back from the external web server. This produces no noticeable delay in the user's HTTP connection.

Advantages and Limitations of URL Filtering

The next two sections cover the advantages and limitations of using a content-filtering server. Obviously, you have much more control over your users' URL access; however, there are downsides to URL filtering.

Advantages of URL Filtering

URL filtering provides many advantages and offers a robust security solution for your network, including the following advantages:

• With a content-filtering server, you more easily can implement your policies based on types of access, such as sports, pornography, gaming, politics, gambling, religion, and other groupings of information. However, you still can define your own access rules and restrictions on a per-host (or per-user) basis.

• Automatic updates can be downloaded to the content servers for new sites that belong to specified categories, as well as the removal of dead links that no longer function. This removes a lot of the management overhead of having to do this process manually.

• You can keep a detailed log on your Websense or N2H2 content server of who is accessing what resources. This is important for accountability when it comes time to identify rule breakers and develop statistics on the type of user web access.

• You can define policies on a per-host or a per-user basis, providing more control over access policies.

• You can define multiple content-filtering servers on your router to provide for redundancy. In this situation, one content-filtering server is defined as the primary, and your router sends all traffic to it. If the primary server is not reachable or is down, your router can use a configured secondary server. If all servers go down, you have the option of either allowing or preventing all HTTP traffic to the router.

• To speed up lookup requests, the router supports an IP cache table that contains a list of IP addresses of web servers that are allowed access by all internal users. You also can define exclusive domains: These are domain names that always should be allowed access by your users, such as http://www.cisco.com, http://www.quizware.com, or http://www.boson.com. With the exclusive domain option, you can use wildcard names or enter domain names with partial URL references.

• The Cisco IOS can buffer up to 200 simultaneous HTTP requests, to reduce the likelihood of dropping HTTP connections because of response-time issues between the router and the content-filtering server.

Restrictions of URL Filtering

This section covers the URL filtering restrictions of N2H2 and Websense. You must use only one product: The Cisco IOS does not support both products simultaneously. N2H2 and Websense have the following restrictions:

• You can specify multiple N2H2 or Websense servers for redundancy. However, the Cisco IOS actively uses only one server.

• The setup of URL filtering works with only one content-filtering product: either Websense or N2H2. Cisco does not support the use of both products simultaneously on routers.

• Only TCP is supported for the connection between the router and the N2H2 server (Websense supports both TCP and UDP).

Both N2H2 and Websense support user-based policies based on a user's authentication information, such as a username and password. Normally, this is done with a proxy method to the content-filtering server. However, with the router handling the HTTP access, the Cisco IOS does not retrieve the username from the client. Therefore, by default, the router supports only host (IP address) and global filtering.

Although the Cisco IOS does not support user authentication, Websense and N2H2 do. With both implementations of user authentication, either Microsoft's internal authentication or LDAP is used. Because the router does not support user authentication, these content-filtering products handle this issue in one of two methods:

• When the user initially logs into the Microsoft Windows domain, an installed Websense or N2H2 agent automatically associates users to their IP addresses, which the content-filtering server then can use for filtering requests received from the router.

• The first point has two problems: It requires a user to log out gracefully to keep track of who is using a particular IP address, and it works only with Microsoft Windows. Therefore, a second option is to have the user directly access the N2H2 or Websense server through a web browser connection to authenticate. Alternatively, the first time a user accesses an external web server, the content-filtering server causes the router to redirect the client to the content-filtering server to authenticate first.

URL Filtering Implementation

Setting up URL filtering is a straightforward process. However, you might want to set up or enable configuration options. This section deals with how to set up your router to interact with Websense and N2H2 content-filtering server products. As you will see in this section, CBAC, which performs application layer inspection, is required to perform URL filtering.

Content Server Location

One of the first items that you need to deal with in web content filtering on a router is the location of your new content-filtering server: Where should you place this device in your network? One main concern with the router and URL filtering is the response time between the content-filtering server and the router.

As you recall from the "Operation of URL Filtering" section, the router forwards the user's request to the external web server and then sends an access request to the content server. If the external web server's reply comes back to the router before the policy action from the content-filtering server, the router buffers the external server's reply, which introduces delay into the traffic stream and puts an additional burden on the router. The more late responses that your router receives from the content-filtering server, the more of an impact this will have on your throughput.

Therefore, the recommended network design is to put the content server in the same subnet as one of the router's interfaces. However, the more devices that are in the subnet, the more competition there is between these devices when sending traffic through the router on the same interface. Therefore, you need to ensure that there is enough bandwidth between the router and the content-filtering server to handle the communication traffic. In a worst-case situation, you might need a dedicated connection on the router to the filtering server. Figure 10-2 shows an example of connecting the content-filtering server to the same subnet as the router.

URL Filtering Setup

After you have installed the content-filtering software on its server and connected it to the network, you are ready to configure your router to interact with it. The following paragraphs explain the commands that you use to configure your router to use the content-filtering server.

CBAC Inspection

To set up inspection of URLs, you need to configure CBAC for HTTP inspection. This is required to perform URL filtering. Here is the CBAC command:

Router(config)# ip inspect name inspection_name http urlfilter

  [java-list ACL_#_or_name]

  [alert {on | off}] [audit-trail {on | off}] [timeout seconds]

Note that you must specify the keyword urlfilter to enable content filtering for HTTP connections. If you omit this keyword, HTTP inspection is done with Java filtering. The rest of the parameters were discussed in Chapter 9.

Server Location

After you enable inspection for HTTP traffic, you must specify the type and location of the content-filtering server. This is required for the router to perform URL filtering. Use this command to specify this information:

Router(config)# ip urlfilter server vendor

  {websense | n2h2} IP_address [port port-number] [timeout seconds]

  [retransmit number]

The ip urlfilter server vendor command specifies how to connect to the content-filtering server. The first parameter that you must specify is what product your content-filtering server is running: websense or n2h2. Following this is the server's IP address. The remaining four parameters are optional. For an N2H2 server, the default port number is 4005; Websense uses 15,868. If you change the port number during the server installation, you need to reflect that change with the port parameter. Following this is the timeout value for the server connection. This is the amount of time that the router waits for a response from the content-filtering server; if no response is seen within this time limit, the router uses a second server, if configured. The default timeout is 5 seconds. The router also retransmits requests when a response does not arrive from the server. The default number of retransmissions is 2, but this can be changed with the retransmit parameter.

URL Cache Size

CBAC uses a URL-filtering cache to store the destination IP addresses of web servers that users are and are not allowed to reach. The more entries the cache can store, the fewer lookups CBAC must perform for the content-filtering server. The default number of entries is 5000, but this can be changed with the following command:

Router(config)# ip urlfilter cache #_of_entries

The number of entries can range from 0 to 2,147,483,647. Even though you can create very large cache sizes, you must have the appropriate amount of RAM on your router to store this information. You also must configure your content-filtering server to support this option. As this writing, only Websense supports this option.

URL Cache Clearing

The URL cache is cleared in a few ways:

• Every 12 hours, the cache is cleared.

• If the cache reaches 80 percent capacity, the Cisco IOS begins to remove idle entries in the cache at 1-minute intervals until the URL cache capacity falls below 80 percent. A connection is considered idle if it has not been used in the last 10 minutes.

• When doing a cache lookup, CBAC removes all entries that have been idle for more than an hour.

• All entries that have elapsed (based on the content server's configuration) are removed. This is often greater than 15 hours. Therefore, the 12-hour cache clearing typically removes these entries before they have elapsed.

To clear the URL cache table manually, use the following command:

Router# clear ip urlfilter cache {IP_address | all}

You can clear all entries in the cache with the all parameter, or you can clear only entries with the specified destination IP address.

Exclusive Domains

As mentioned in the "Operation of URL Filtering" section, URL filtering places an additional burden on your router, especially if the router constantly must send lookup requests to the content-filtering server. You can reduce the number of lookups in two ways:

• Create a larger cache size.

• Use exclusive domains.

Exclusive domains enable you to define local filtering rules on the router. In this situation, the router first looks at its cache to see if there is a match, then it looks at the locally listed domains, and then it sends a request to the content-filtering server if nothing is found locally on the router. This configuration is very useful for listing sites that always or never should be allowed (and these sites never change). Commonly used sites should be listed. For example, if your network administrators constantly access the Cisco website, include this as a permit action in your exclusive domain configuration.

This is the command to define an exclusive domain:

Router(config)# ip urlfilter exclusive-domain

  {permit | deny} domain_name

You can specify two actions: permit or deny. When specifying a domain name, you can be specific by specifying the fully qualified domain name, such as www.quizware.com; a partial domain name, such as .quizware.com; or a fully qualified domain name and a partial URL, such as www.quizware.com/dealgroup. You can list as many domain entries as you want. Example 10-2 shows a simple example of defining two domains that always should be allowed and one that always should be denied.

Example 10-2. Setting Up Exclusive Domains for URL Filtering

Router(config)# ip urlfilter exclusive-domain permit .cisco.com

Router(config)# ip urlfilter exclusive-domain permit .quizware.com

Router(config)# ip urlfilter exclusive-domain deny .sex.com

Lost Server Connection

Sometimes the Cisco IOS will not be capable of contacting your content-filtering server to authorize users' HTTP connections. For example, you might have to reboot your server because of system patches that you have applied to it. In this situation, the Cisco IOS can take one of two actions concerning HTTP traffic during this blackout period:

• Drop all traffic until the connection between the router and the URL content filter server is restored.

• Permit all traffic until the connection between the router and the URL content filter server is restored.

The default is to drop all traffic. This can be changed with the following command:

Router(config)# ip urlfilter allowmode [on | off]

on specifies that connection requests should be permitted when the router cannot reach the content-filtering server. off specifies that the connections should be dropped.

Whenever your router loses contact with the content-filtering server, an alert message is displayed, along with the status of the allow mode that the router will use. In this example, the router drops traffic until the connection between the router and server is re-established:

%URLF-3-ALLOW_MODE: Connection to all URL filter servers are down

  and ALLOW MODE if OFF

Maximum Requests

The Cisco IOS temporarily holds up to 1000 pending URL requests. These are requests that the Cisco IOS has sent to the content server and is waiting for a response for. When this limit is reached, new user connection requests are dropped, causing the user's web browser to perform a retransmission. To change the number of pending requests that the Cisco IOS will store, use this command:

Router(config)#  ip urlfilter max-request #_of_requests

The number of requests that you can specify ranges from 1 to 2,147,483,647.

Maximum Responses

As mentioned in the "Operation of URL Filtering" section, a router might receive the returning HTTP traffic from the external server before it receives the policy reply from the content-filtering server. In this situation, the router can buffer the HTTP response while waiting for the reply from the filtering server. However, by default, the router buffers only 200 connection responses. This can be changed with this command:

Router(config)# ip urlfilter max-resp-pak #_of_responses

The number of responses the Cisco IOS can buffer ranges from 0 to 20,000.

Alerts

URL-filtering alerts are enabled by default. These alerts display messages when the content-filtering server is not reachable, when all servers are not reachable, or when a URL lookup exceeded the timeout value. Alerts can be disabled or re-enabled with the following command:

Router(config)# [no] ip urlfilter alert

Example 10-3 shows some common alert messages that the Cisco IOS might display.

Example 10-3. Examining Cisco IOS Alerts for URL Filtering

%URLF-3-SERVER_DOWN:Connection to the URL filter server           (1)

  192.1.2.2 is down

%URLF-3-ALLOW_MODE:Connection to all URL filter servers           (2)

  are down and ALLOW MODE is OFF

%URLF-5-SERVER_UP:Connection to an URL filter server              (3)

  192.1.2.2 is made, the system is returning

  from ALLOW MODE

%URLF-4-URL_TOO_LONG:URL too long (more than 3072 bytes),         (4)

  possibly a fake packet?

%URLF-4-MAX_REQ:The number of pending request exceeds the         (5)

  maximum limit <1000>

The following is an explanation of these messages, with reference to the numbering on the right side of Example 10-3:

Many other alert messages exist, but these are the most common ones.

Audits

As with CBAC audits, audits for URL filtering are disabled by default. Enabling audits allows you to log information such as who is making an HTTP connection request and what web server is trying to be accessed. To enable auditing, use the following command:

Router(config)# [no] ip urlfilter audit-trail

Example 10-4 shows some common auditing messages that you will see with auditing enabled.

Example 10-4. Examining IDS Audits for URL Filtering

%URLF-6-URL_ALLOWED:Access allowed for URL                        (1)

  http://www.quizware.com; client 192.1.1.12:32828

  server 200.1.1.1:80

%URLF-6-SITE_ALLOWED:Client 192.1.1.12:32834 accessed             (2)

  server 200.1.1.1:80

%URLF-6-URL_BLOCKED:Access denied URL http://www.sex.com;         (3)

  client 192.1.1.12:32983 server 209.81.7.93:80

%URLF-4-SITE-BLOCKED: Access denied for the site                  (4)

  `www.gambling.com'; client 12.54.192.6:34557

  server 207.139.179.17:80

The following is an explanation of these messages, with reference to the numbering on the right side of Example 10-4:

Logging

By default, any log messages generated by CBAC URL filtering are displayed on the console or sent to the router's local buffer or a syslog server, if you have configured these last two options. Another option is to forward these messages to your content-filtering server. This feature enables you to keep all your URL filtering log messages in one location for easy access and monitoring. As I mentioned in the last tip, a content-filtering server does not see log messages for matches that the router finds in its local cache. With this option, you can forward your log messages to the content-filtering server when the router finds a response in the cache. This provides a record of all transactions for users' HTTP connections. This option is disabled by default but can be enabled with the following command:

Router(config)# ip urlfilter urlf-server-log

CBAC Activation

After you have set up your inspection rules for CBAC, specified the location and type of the URL content-filtering server, and tuned URL filtering, you are ready to activate your CBAC inspection process on your router's interface. This is done with the following configuration:

Router(config)# interface type [slot_#]port_#

Router(config-if)#  ip inspect inspection_name {in | out}

This configuration was discussed in Chapter 9. When you have activated the inspection group on your router's interface, CBAC begins inspecting URL requests.

URL Filtering Verification

When you have set up and activated URL inspection of HTTP traffic, you can monitor it with various show and debug commands. The following two sections cover the use of these monitoring and troubleshooting commands.

show Commands

You can use three basic show commands to monitor and troubleshoot URL inspection:

• show ip urlfilter cache? Displays the destination IP addresses in the router's URL cache

• show ip urlfilter config? Displays the URL filter configuration on the router, including the size of the cache, the maximum number of pending requests allowed, the maximum number of buffered packets while waiting for a content server's response, the type of server and its address(es), and other configuration information

• show ip urlfilter statistics? Displays URL inspection statistics, including the number of requests sent to and received from the content-filtering server, the number of pending and failed requests, and the number of blocked connections

Take a look at some sample output of these commands. Example 10-5 shows sample output from the show ip urlfilter cache command.

Example 10-5. Using the show ip urlfilter cache Command

Router# show ip urlfilter cache

Maximum number of entries allowed: 5000

Number of entries cached: 4

IP addresses cached ....

 200.1.1.1

 200.1.1.2

 200.2.2.1

 200.3.3.3

The default number of entries allowed in the cache is 5000, which can be seen in Example 10-5. The number of entries in the cache is 4, which is followed by the actual IP addresses of the web servers that your users are trying to access.

Example 10-6 shows sample output from the show ip urlfilter config command.

Example 10-6. Using the show ip urlfilter config Command

Router# show ip urlfilter config

URL filter is ENABLED

Primary Websense server configurations

===========================

Websense server IP address: 192.1.2.1

Websense server port: 15868

Websense retransmit time out: 5 (seconds)

Websense number of retransmit:2

Secondary Websense server configurations:

==============================

None.

Other configurations

===============

Allow mode: OFF

System Alert: ON

Log message on the router: OFF

Log message on URL filter server:ON

Maximum number of cache entries :5000

Cache timeout :12 (hours)

Maximum number of packet buffers:200

Maximum outstanding requests:1000

Here you can see the URL filtering configuration on the router. This router is using a Websense server (192.1.2.1) to perform URL filtering. You also can see that this configuration is using the default values, such as 1000 outstanding requests and 200 buffered HTTP responses.

Example 10-7 shows sample output from the show ip urlfilter statistics command.

Example 10-7. Using the show ip urlfilter statistics Command

Router# show ip urlfilter statistics

URL filtering statistics

================

Current requests count:30

Current packet buffer count(in use): 12

Current cache entry count: 800

Maxever request count: 1000

Maxever packet buffer count:200

Maxever cache entry count:5000

Total requests sent to URL Filter Server: 38565

Total responses received from URL Filter Server: 38350

Total requests allowed: 38120

Total requests blocked: 238

This display has a lot of useful information. The Current Requests Count displays the number of requests (30) that the router has sent to the server and is waiting for a response for. The Current Packet Buffer Count displays the number of HTTP server replies (12) that the router has buffered while waiting for a response from the content-filtering server. The Current Cache Entry Count displays the current number of destination IP addresses (800) that the Cisco IOS has cached in the URL cache table. The Maxever lines display peak values that the system has seen since last powerup. These values typically are used to assist in choosing an appropriate value for the ip urlfilter cache, ip urlfilter packets, and ip urlfilter maxrequest configuration commands.

For instance, the maximum number of simultaneous HTTP authorization requests that the router will handle is 1000. Below these three lines are some totals: the total number of requests that the router has forwarded to the content server for authorization (38,565), the number of responses that the router has received from the content-filtering server (38,350), the number of times the content-filtering server allowed HTTP connections (38,120), and the number of times it denied them (238).

debug Commands

You can use three debug commands for more detailed troubleshooting of URL-filtering functions:

• debug ip urlfilter events? Displays URL-filtering events, such as when a timer event is reached, connection events, and queuing events

• debug ip urlfilter detailed? Displays detailed information about the events that the debug ip urlfilter events command displays

• debug ip urlfilter function-trace? Displays the functions called by the Cisco IOS when performing URL filtering

URL Filtering Example

Now that you have a basic understanding of URL content filtering, take a look at an example of setting this up on your router. Figure 10-3 illustrates this example.

Example 10-8 shows a URL filtering configuration for this network.

Example 10-8. Simple Configuration of URL Filtering on a Router

Router(config)# ip inspect name RULES http urlfilter              (1)

Router(config)# ip inspect name RULES ftp

Router(config)# ip inspect name RULES smtp

Router(config)# ip inspect name RULES tcp

Router(config)# ip inspect name RULES udp

Router(config)# ip urlfilter server vendor websense 192.1.2.2     (2)

Router(config)# ip urlfilter cache 7000                           (3)

Router(config)# ip urlfilter max-request 1500                     (4)

Router(config)# ip urlfilter max-resp-pack 300                    (5)

Router(config)# ip urlfilter exclusive-domain permit .cisco.com   (6)

Router(config)# ip urlfilter exclusive-domain permit www.quizware.com

Router(config)# ip urlfilter exclusive-domain deny .msn.com

Router(config)# ip urlfilter exclusive-domain deny .aol.com

Router(config)# ip urlfilter audit-trail                          (7)

Router(config)# ip urlfilter alert

Router(config)# ip urlfilter urlf-server-log

Router(config)# ip access-list extended external_ACL

Router(config-ext-nacl)# ! <--enter ACL policies here-->

Router(config-ext-nacl)# deny ip any any

Router(config-ext-nacl)# exit

Router(config)# interface Ethernet1

Router(config-if)# ip inspect RULES out                           (8)

Router(config-if)# ip access-group external_ACL in

The following is an explanation of the configuration in Example 10-8, with reference to the numbering on the right side:

I have added some optional parameters to tune this configuration to work better for this specific network. The only required configurations are Steps 1, 2, and 8.