Setting Up News Feeds

Configuring hosts to feed you

To configure the host computers that feed articles to your news server, you need to configure the /etc/news/incoming.conf file. In this file, you can set various key/value parameters that affect how these news feeds behave. Other entries are either peer entries or group entries.

The key/value entries set values that are assigned to every peer and group entry. Those values can be overridden for particular peers or groups by adding new key/value entries within peer and group entries. Peer entries identify the fully-qualified domain name (FQDN) of a computer that can feed news to your server, along with any key/value entries. Group entries are a way of assigning groups of peers to have particular key/value entries.

The whole thing seems a bit complicated when all you are doing is defining which hosts can send news to you and how they are allowed to do that. Here is an example of the contents of an incoming.conf file from its man page:

max-connections:    5      # per feed
   
# A peer definition.
peer uunet {
     hostname: usenet1.uu.net
}
   
peer vixie {
     hostname: gw.home.vix.com
     max-connections: 10 # override global value.
}
   
# A group of two peers who can open more
# connections than normal
group fast-sites {
     max-connections: 15
   
     # Another peer. The ``max-connections'' value from the
     # ``fast-sites'' group scope is used. The hostname value
     # defaults to the peer's name.
     peer data.ramona.vix.com {
     }

     peer bb.home.vix.com {
          hostname: bb.home.vix.com
          max-connections: 20 # he can really cook.
     }
 }

The only key/value pair in this example is max-connections: 5, which defines the maximum number of connections from any one host to be five (unless overridden in a peer or group value). Two individual hosts are defined as news feeders: uunet (usenet1.uu.net) and vixie (gw.home.vix.com). The vixie definition is an example of using a key/value pair to override a default value.

The group example is just a way to set key/value entries for several hosts at the same time. This example sets the maximum number of connections to 15 and assigns that value to all the peers in the group (data.ramona.vix.com and bb.home.vix.com). Then, as an illustration, the value is overridden for the second of those hosts by setting the value to 20.

The host name can be a full host.domain name or an IP address. As you have already seen, max-connections can set the maximum number of connections that are enabled at a time from a host (0 enables unlimited connections). Here are some key/values that you can set globally, for a particular peer, or within a group:

• hostname:???Identifies the host.domain name or IP address of the news server.

• streaming:???Defines whether streaming commands are enabled (true or false).

• password:???Assigns a string to this key that must be used by the host as a password before it can connect. By default, no password is required.

• noresendid:???Causes the innd daemon to send a 431 RESENDID response to an article that has already been received from another peer.

Configuring hosts that you feed

The entries that you place in the /etc/news/newsfeeds file define how the articles that your news server receives are fed to other news servers. This file offers a lot of opportunity for configuration. The main reason this file is so complex is that it enables you to select which newsgroup articles to forward to each news server (based primarily on what they will accept). You can also set up definitions that apply to groups of servers.

Within the entries in the newsfeeds file, certain wildcard characters can be used to match or exclude whole sets of newsgroups. You can probably figure out how they work in the context of the examples. If not, however, you can refer to the "Understanding Wildmat Characters" sidebar for information on using the wildcard characters.

Understanding Wildmat Characters

When you need to identify newsgroups in your newsfeeds configuration file, you can use several different wildcard characters to simplify the process. These characters are defined on the wildmat man page (type man wildmat). Here's what they do:

• !: The exclamation point is used to indicate that the newsgroup name that follows should not be matched.

• *: An asterisk at the end of a newsgroup name indicates that all newsgroups following the one shown (those lower in the hierarchy) should be matched.

• [abc]: Any single character included within the brackets is matched. For example, [abc]* matches any group name that begins with a, b, or c. You can also specify number or letter ranges, such as 3–9 or a–r, with the braces to include all those numbers or letters.

• ?: A question mark matches any single character. So, c?mp matches comp, camp, and a lot of stuff that makes no sense at all.

• [^abc]: When a ^ is placed at the beginning of a set of brackets, the letters in the pattern match any character that is not in the brackets. For example, [^abc]* matches any group that does not begin with a, b, or c.

The default /etc/news/newsfeeds file has only an ME parameter entry in it and a controlchan entry (and a lot of comments). The ME entry is required; you can have only one of them in your newsfeeds file, and it must appear before any other news feed lines. This entry contains a subscription list that is automatically added first to the subscription list of every other entry. Here is the default ME line:

ME:!*/!local,!collabra-internal::

This default ME line specifically indicates some articles that are not forwarded. This line causes all incoming articles with local or collabra-internal in the Path header to be rejected. Articles that come in with either of those headers indicate that they are coming from a misconfigured server.

The following is an example of an ME entry that includes additional restrictions:

ME\
:*,@alt.binaries.warez.*,!junk,!control*,!local*,!foo.*\
 /world,usa,na,gnu,bionet,pubnet,u3b,eunet,vmsnet,inet,ddn,k12\
::

With this entry, all newsgroups are propagated to every server, with the exception of junk, control, local, and foo groups. The exclamation mark says to exclude the name that follows.

With the ME line set, you can go about defining how your specific news feeds are done. Here is an example of an innfeed line you can add to your newsfeeds file. This example funnels all news feeds to the startinfeed command.

innfeed!:\
     !*\
     :Tc,Wnm*,S16384:/usr/bin/startinnfeed -y

This line runs the startinnfeed command to start the innfeed program. The innfeed program, in turn, carries out the actual transfer of news articles between the news servers.

After the ME and innfeed! entries, you need to add entries that define the actual news servers to which you will feed articles. You should have one entry for each news server that you feed. The general format of those entries is as follows:

remote-peer.domain.com/name-in-header.domain.com\
    :newsgroup-list\
    :Tm:innfeed!

You need to replace remote-peer.domain.com with the FQDN of the server that you are feeding the news to. Next, replace the name-in-header with any alias names that the remote news server uses. The aliases are names that the remote news server puts in the Path header of the articles it forwards. (If no aliases exist, leave off the entire /name-in-header part.) You need to enter a newsgroup-list only if you want to feed newsgroups that are different from the newsgroups that are set by default (in your ME entry). The last part of the entry (:Tm:innfeed!) should be left as it is.

If your server has the controlchan feature turned on (usecontrolchan: true in the inn.conf file), you can use the entry for the controlchan program in the newsfeeds file or create your own entry. The default entry is meant to reduce the load if, in a short period of time, many control messages arrive at your news server. This entry runs the /usr/bin/controlchan command.

controlchan!\
     :!*,control,control.*,!control.cancel\
     :Tc,Wnsm:/usr/bin/controlchan

You can use a mind-numbing number of options within the newsfeeds file. If you are interested in delving deeper, read the comments in the newsfeeds file and refer to the newsfeeds manual page (type man newsfeeds). That manual page will also point you to related man pages.

Getting a list of active newsgroups

The Internet Software Consortium (www.isc.org) maintains a listing of all officially active newsgroups. ISC stores these newsgroups in two files: newsgroups and active. The newsgroups file contains each newsgroup name and a short description of each. The active file stores the newsgroups to indicate which newsgroups your news server offers.

There are two ways you can go about creating your newsgroups and active files. For a minimal news server, you can use the default newsgroups and active files in /var/lib/news and add the information you want. Then you can add newsgroups later using the ctlinnd newsgroup command. Alternatively, you can get complete active and newsgroups files from ISC and edit those down.

You can download the latest copies of the active and newsgroups files from the ISC FTP server: ftp://ftp.isc.org/pub/usenet/CONFIG/. From that directory, you can download either uncompressed versions of those files (each is more than 1MB in size) or compressed versions. Choose the active.gz and newsgroups.gz files, which you can uncompress in Red Hat Linux by using the gunzip command (gunzip active.gz newsgroup.gz).

There are minimal, default active and newsgroups files in your /var/lib/news directory. Copy the new versions of those files to /var/lib/news, and then edit the active file manually. The newsgroups file provides the names and descriptions of the newsgroups offered to the users of your news server. The active file is the official list of newsgroups that is read by the innd daemon so that it knows what newsgroups it should accept articles for.

The active and newsgroups files need to be owned by the news user. So you should either copy those files to your /var/lib/news directory while you are logged in as the news user or change their ownership by typing the following with the files in the current directory:

# chown news.news active newsgroups

Creating the history database

Before you can start your INN server, you need to create the history database. You can to this from the /var/lib/news directory by typing the following (while logged in as the news user):

$ touch history
$ /usr/lib/news/bin/makedbz -i
$ mv history.n.dir history.dir
$ mv history.n.hash history.hash
$ mv history.n.index history.index
$ chmod 644 history*