Ftp Web Hosting - J2Ee Hosting Sun Java Developing Blog

The perm field details if the entry applies to moderated, unmoderated, or any groups. It may take the values m, u, or x, which denote moderated, unmoderated, or any type. The third field, times, usually contains only a single number. This is the number of days after which articles expire if they haven’t been assigned an artificial expiration date in an Expires: field in the article header. Note that this is the number of days counting from its arrival at your site, not the date of posting. The times field may, however, be more complex than that. It may be a combination of up to three numbers separated from one another by dashes. The first denotes the number of days that have to pass before the article is considered a candidate for expiration, even if the Expires: field would have it expire already. It is rarely useful to use a value other than zero. The second field is the previously mentioned default number of days after which it will be expired. The third is the number of days after which an article will be expired unconditionally, regardless of whether it has an Expires: field or not. If only the middle number is given, the other two take default values. These may be specified using the special entry /bounds/, which is described a little later. The fourth field, archive, denotes whether the newsgroup is to be archived and where. If no archiving is intended, a dash should be used. Otherwise, you either use a full pathname (pointing to a directory) or an at sign (@). The at sign denotes the default archive directory, which must then be given to doexpire by using the -a flag on the command line. An archive directory should be owned by news. When doexpire archives an article from say, comp.sources.unix, it stores it in the directory comp/sources/unix below the archive directory, creating it if necessary. The archive directory itself, however, will not be created. There are two special entries in your explist file that doexpire relies on. Instead of a list of newsgroups, they have the keywords /bounds/ and /expired/. The /bounds/ entry contains the default values for the three values of the times field described previously. The /expired/ field determines how long C News will hold onto lines in the history file. C News will not remove a line from the history file once the corresponding article(s) have been expired, but will hold onto it in case a duplicate should arrive after this date. If you are fed by only one site, you can keep this value small. Otherwise, a couple of weeks is advisable on UUCP networks, depending on the delays you experience with articles from these sites. Here is a sample explist file with rather tight expiry intervals: # keep history lines for two weeks. No article gets more than three months /expired/ x 14 - /bounds/ x 0-1-90 - # groups we want to keep longer than the rest comp.os.linux.announce m 10 - comp.os.linux x 5 - alt.folklore.computers u 10 - rec.humor.oracle m 10 - soc.feminism m 10 - # Archive *.sources groups comp.sources,alt.sources x 5 @ # defaults for tech groups comp,sci x 7 - # enough for a long weekend misc,talk x 4 - # throw away junk quickly junk x 1 - # control messages are of scant interest, too control x 1 - # catch-all entry for the rest of it all x 2 - Expiring presents several potential problems. One is that your newsreader might rely on the third field of the active file described earlier, which contains the number of the lowest article online. When expiring articles, C News does not update this field. If you need (or want) to have this field represent the real situation, you need to run a program called updatemin after each run of doexpire. (In older versions of C News, a script called upact did this.)

transport The last field, transport, describes the transport to be used. A number of standard commands for different transports are available; their names begin with via. sendbatches passes them the destination sitename on the command line. If the batchparms entry is not /default/, sendbatches derives the sitename from the site field by stripping it of anything after and including the first dot or slash. If the batchparms entry is /default/, the directory names in out.going are used. To perform batching for a specific site, use the following command: # su news -c “/usr/lib/news/batch/sendbatches site” When invoked without arguments, sendbatches handles all batch queues. The interpretation of “all” depends on the presence of a default entry in batchparms. If one is found, all directories in /var/spool/news/out.going are checked; otherwise, sendbatches cycles through all entries in batchparms, processing just the sites found there. Note that sendbatches, when scanning the out.going directory, takes only those directories that contain no dots or at signs (@) as sitenames. There are two commands that use uux to execute rnews on the remote system: viauux and viauuxz. The latter sets the -z flag for uux to keep older versions from returning success messages for each article delivered. Another command, viamail, sends article batches to the user rnews on the remote system via mail. Of course, this requires that the remote system somehow feeds all mail for rnews to its local news system. For a complete list of these transports, refer to the newsbatch manual page. All commands from the last three fields must be located in either out.going/site or /usr/lib/news/batch. Most of them are scripts; you can easily tailor new tools for your personal needs. They are invoked through pipes. The list of articles is fed to the batcher on standard input, which produces the batch on standard output. This is piped into the muncher, and so on. Here is a sample file: # batchparms file for the brewery # site | size |max |batcher |muncher |transport #————-+——–+——-+———+———–+———– /default/ 100000 22 batcher compcun viauux swim 10000 10 batcher nocomp viauux Expiring News In B News, expiration needs to be performed by a program called expire, which took a list of newsgroups as arguments, along with a time specification after which articles had to be expired. To have different hierarchies expire at different times, you had to write a script that invoked expire for each of them separately. C News offers a more convenient solution. In a file called explist, you may specify newsgroups and expiration intervals. A command called doexpire is usually run once a day from cron and processes all groups according to this list. Occasionally, you may want to retain articles from certain groups even after they have been expired; for example, you might want to keep programs posted to comp.sources.unix. This is called archiving. explist permits you to mark groups for archiving. An entry in explist looks like this: grouplist perm times archive grouplist is a comma-separated list of newsgroups to which the entry applies. Hierarchies may be specified by giving the group name prefix, optionally appended with all. For example, for an entry applying to all groups below comp.os, enter either comp.osor comp.os.all. When expiring news from a group, the name is checked against all entries in explist in the order given. The first matching entry applies. For example, to throw away the majority of comp after four days, except for comp.os.linux.announce, which you want to keep for a week, you simply have an entry for the latter, which specifies a seven-day expiration period, followed by an expiration period for comp, which specifies four days.

#! cunbatch Sometimes, when the news server sends batches via mail software that removes the eighth bit from all data, a compressed batch may be protected using what is called c7-encoding; these batches will be marked by c7unbatch. When a batch is fed to rnews on the remote site, it checks for these markers and processes the batch appropriately. Some sites also use other compression tools, like gzip, and precede their gzipped files with the word zunbatch instead. C News does not recognize nonstandard headers like these; you have to modify the source to support them. In C News, article batching is performed by /usr/lib/news/batch/sendbatches, which takes a list of articles from the site/togo file and puts them into several newsbatches. It should be executed once per hour, or even more frequently, depending on the volume of traffic. Its operation is controlled by the batchparms file in /var/lib/news. This file describes the maximum batch size allowed for each site, the batching and optional compression program to be used, and the transport for delivering it to the remote site. You may specify batching parameters on a per-site basis, as well as a set of default parameters for sites not explicitly mentioned. When installing C News, you will most likely find a batchparms file in your distribution that contains a reasonable default entry, so there’s a good chance that you won’t have to touch the file. Just in case, we describe its format. Each line consists of six fields, separated by spaces or tabs: site size max batcher muncher transport site site is the name of the site to which the entry applies. The togo file for this site must reside in out.going/togo below the news spool. A site name of /default/ denotes the default entry and is to match any site not directly specified with an entry unique to it. size size is the maximum size of article batches created (before compression). For single articles larger than this, C News makes an exception and puts each in a single batch by itself. max max is the maximum number of batches created and scheduled for transfer before batching stalls for this particular site. This is useful in case the remote site should be down for a long time, because it prevents C News from cluttering your UUCP spool directories with zillions of newsbatches. C News determines the number of queued batches using the queuelen script in /usr/lib/news/. If you’ve installed C News in a prepackaged format, the script should not need any editing, but if you choose to use a different flavor of spool directories, for example, Taylor UUCP, you might have to write your own. If you don’t care about the number of spool files (because you’re the only person using your computer and you don’t write articles by the megabyte), you may replace the script’s contents by a simple exit 0 statement. batcher The batcher field contains the command used for producing a batch from the list of articles in the togo file. For regular feeds, this is usually batcher. For other purposes, alternative batchers may be provided. For instance, the ihave/sendme protocol requires the article list to be turned into ihave or sendme control messages, which are posted to the newsgroup to.site. This is performed by batchih and batchsm. muncher The muncher field specifies the compression command. Usually, this is compcun, a script that produces a compressed batch.128 Alternatively, suppose you create a muncher that uses gzip, say gzipcun (note that you have to write it yourself). You have to make sure that uncompress on the remote site is patched to recognize files compressed with gzip. If the remote site does not have an uncompress command, you may specify nocomp, which does not do any compression. As shipped with C News, compcun uses compress with the 12-bit option, since this is the lowest common denominator for most sites. You may produce a copy of the script, say compcun16, for which you use 16-bit compression. The improvement is not too impressive, though.

The active File The active file is located in /etc/, and lists all groups known at your site and the articles currently online. You will rarely have to touch it, but we explain it nevertheless for sake of completion. Entries take the following form: newsgroup high low perm newsgroup is the group’s name. low and high are the lowest and highest numbers of articles currently available. If none are available at the moment, low is equal to high+1. At least that’s what the low field is meant to do. However, for efficiency, C News doesn’t update this field. This wouldn’t be such a big loss if there weren’t newsreaders that depend on it. For instance, trn checks this field to see if it can purge any articles from its thread database. To update the low field, you therefore have to run the updatemin command regularly (or, in earlier versions of C News, the upact script). perm is a parameter detailing the access users are granted to the group. It takes one of the following values: y Users are allowed to post to this group. n Users are not allowed to post to this group. However, the group may still be read. This group has been disabled locally. This happens sometimes when news administrators (or their superiors) take offense at articles posted to certain groups. Articles received for this group are not stored locally, although they are forwarded to the sites that request them. m This denotes a moderated group. When a user tries to post to this group, an intelligent newsreader notifies her of this and send the article to the moderator instead. The moderator’s address is taken from the moderators file in /var/lib/news. =real-group This marks newsgroup as being a local alias for another group, namely real-group. All articles posted to newsgroup will be redirected to it. In C News, you will generally not have to access this file directly. Groups can be added or deleted locally using addgroup and delgroup (see the section “Maintenance Tools and Tasks” later in this chapter). A newgroup control message adds a group for the whole of Usenet, while a rmgroup message deletes a group. Never send such a message yourself! For instructions on how to create a newsgroup, read the monthly postings in news.announce.newusers. The active.times file is closely related to the active file. Whenever a group is created, C News logs a message to this file containing the name of the group created, the date of creation, whether it was done by a newgroup control message or locally, and who did it. This is convenient for newsreaders that may notify the user of any recently created groups. It is also used by the NEWGROUPS command of NNTP. Article Batching News batches follow a particular format that is the same for B News, C News, and INN. Each article is preceded by a line like this: #! rnews count count is the number of bytes in the article. When you use batch compression, the resulting file is compressed as a whole and preceded by another line, indicated by the message to be used for unpacking. The standard compression tool is compress, which is marked by:

flags This option describes certain parameters for the feed. It may be empty or a combination of the following: F This flag enables batching. f This is almost identical to the F flag, but allows C News to calculate the size of outgoing batches more precisely, and should probably be used in preference. I This flag makes C News produce an article list suitable for use by ihave/sendme. Additional modifications to the sys and the batchparms file are required to enable ihave/sendme. n This creates batch files for active NNTP transfer clients like nntpxmit (see Chapter 22, NNTP and the nntpd Daemon). The batch files contain the article’s filename along with its message ID. L This tells C News to transmit only articles posted at your site. This flag may be followed by a decimal number n, which makes C News transfer articles posted only within n hops from your site. C News determines the number of hops from the Path: field. u This tells C News to batch only articles from unmoderated groups. m This tells C News to batch only articles from moderated groups. You may use at most one of F, f, I, or n. cmds This field contains a command that will be executed for each article, unless you enable batching. The article will be fed to the command on standard input. This should be used for very small feed only; otherwise, the load on both systems will be too high. The default command is: uux–r-z remote-system!rnews This invokes rnews on the remote system, feeding it the article on standard input. The default search path for commands given in this field is /bin:/usr/bin:/usr/lib/news/batch. The latter directory contains a number of shell scripts whose names start with via; they are briefly described later in this chapter. If batching is enabled using one of the F, f, I, or n flags, C News expects to find a filename in this field rather than a command. If the filename does not begin with a slash (/), it is assumed to be relative to /var/spool/news/out.going. If the field is empty, it defaults to remote-system/togo. The file is expected to be in the same format as the remote-system/togo file and contain a list of articles to transmit. When setting up C News, you will most probably have to write your own sys file. Here is a sample file for vbrew.com, from which you may copy what you need: # We take whatever they give us. ME:all/all:: # We send everything we receive to moria, except for local and # brewery-related articles. We use batching. moria/moria.orcnet.org:all,!to,to.moria/all,!local,!brewery:f: # We mail comp.risks to jack@ponderosa.uucp ponderosa:comp.risks/all::rmail jack@ponderosa.uucp # swim gets a minor feed swim/swim.twobirds.com:comp.os.linux,rec.humor.oracle/all,!local:f: # Log mail map articles for later processing usenet-maps:comp.mail.maps/all:F:/var/spool/uumaps/work/batch

You also have to add a line for each site to which you feed news. Each line looks like this: site[/exclusions]:grouplist[/distlist][:flags[:cmds]] Entries may be continued across newlines using a backslash (\) at the end of the line to be continued. A hash sign (#) denotes a comment. site This is the name of the site the entry applies to. One usually chooses the site’s UUCP name for this. There has to be an entry for your site in the sys file too, or you will not receive any articles yourself. The special site name ME denotes your site. The ME entry defines all groups you are willing to store locally. Articles that aren’t matched by the ME line will go to the junk group. C News rejects any articles that have already passed through this site to prevent loops. C News does this by ensuring that the local site name does not appear in the Path: of the article. Some sites may be known by a number of valid names. For example, some sites use their fully qualified domain name in this field, or an alias like news.site.domain. To ensure the loop prevention mechanism works, it is important to add all aliases to the exclusion list, separating them by commas. For the entry applying to site moria, for instance, the site field would contain moria/moria.orcnet.org. If moria were also by an alias of news.orcnet.org, then our site field would contain moria/moria.orcnet.org,news.orcnet.org. grouplist This is a comma-separated subscription list of groups and hierarchies for this particular site. A hierarchy may be specified by giving the hierarchy’s prefix (such as comp.os for all groups whose names start with this prefix), optionally followed by the keyword all (e.g., comp.os.all). You can exclude a hierarchy or group from forwarding by preceding it with an exclamation mark. If a newsgroup is checked against the list, the longest match applies. For example, if grouplist contains this list: !comp,comp.os.linux,comp.folklore.computers no groups from the comp hierarchy except comp.folklore.computers and all groups below comp.os.linux will be fed to that site. If the site requests to be forwarded all news you receive yourself, enter all as grouplist. distlist This value is offset from the grouplist by a slash and contains a list of distributions to be forwarded. Again, you may exclude certain distributions by preceding them with an exclamation mark. All distributions are denoted by all. Omitting distlist implies a list of all. For example, you may use a distribution list of all,!local to prevent news meant only for local use from being sent to remote sites. There are usually at least two distributions: world, which is often the default distribution used when none is specified by the user, and local. There may be other distributions that apply to a certain region, state, country, etc. Finally, there are two distributions used by C News only; these are sendme and ihave, and are used for the sendme/ihave protocol. The use of distributions is a subject of debate. The distribution field in a news article can be created arbitrarily, but for a distribution to be effective, the news servers in the network must know it. Some misbehaving news- readers create bogus distributions by simply assuming the top-level newsgroup hierarchy of the article destination is a reasonable distribution. For example, one might assume comp to be a reasonable distribution to use when posting to the comp.os.linux.networking newsgroup. Distributions that apply to regions are often questionable, too, because news may travel outside of your region when sent across the Internet.127 Distributions applying to an organization, however, are very meaningful; e.g., to prevent confidential information from leaving the company network. This purpose, however, is generally served better by creating a separate newsgroup or hierarchy. It is not uncommon for an article posted in say, Hamburg, to go to Frankfurt via reston.ans.net in the Netherlands, or even via some site in the U.S.

whoami Your site’s name for news purposes. Quite often, the UUCP site name is used, e.g., vbrew. explist You should probably edit this file to reflect your preferred expiration times for special newsgroups. Disk space may play an important role in your choices. To create an initial hierarchy of newsgroups, obtain active and newsgroups files from the site that feeds you. Install them in /etc/news, making sure they are owned by news and have a mode of 644, using the chmod command. Remove all to.* groups from the active file, and add to.my-site, to.feed-site, junk, and control. The to.* groups are normally used for exchanging ihave/sendme messages, but you should list them regardless of whether you plan to use ihave/sendme or not. Next, replace all article numbers in the second and third field of active using the following commands: # cp active active.old # sed ’s/ [0-9]* [0-9]* / 0000000000 00001 /’ active.old > active # rm active.old The second command invokes the sed stream editor. This invocation replaces two strings of digits with a string of zeroes and the string 000001, respectively. Finally, create the news spool directory and the subdirectories used for incoming and outgoing news: # cd /var/spool # mkdir news news/in.coming news/out.going news/out.master # chown -R news.news news # chmod -R 755 news If you’re using precompiled newsreaders sourced from a different distribution to the C News server you have running, you may find that some expect the news spool in /usr/spool/news rather than /var/spool/news. If your newsreader doesn’t seem to find any articles, create a symbolic link from /usr/spool/news to /var/spool/news like this: # ln -sf /usr/spool/news /var/spool/news Now you are ready to receive news. Note that you don’t have to create the individual newsgroup spool directories. C News automatically creates spool directories for any newsgroup it receives an article for, if one doesn’t already exist. In particular, this happens to all groups to which an article has been cross-posted. So, after a while, you will find your news spool cluttered with directories for newsgroups you have never subscribed to, like alt.lang.teco. You may prevent this by either removing all unwanted groups from active, or by regularly running a shell script that removes all empty directories below /var/spool/news (except out.going and in.coming, of course). C News needs a user to send error messages and status reports to. By default, this is usenet. If you use the default, you have to set up an alias for it that forwards all of its mail to one or more responsible person. You may also override this behavior by setting the environment variable NEWSMASTER to the appropriate name. You have to do so in news’s crontab file, as well as every time you invoke an administrative tool manually, so installing an alias is probably easier. Mail aliases are described in Chapter 18, Sendmail, and Chapter 19, Getting Exim Up and Running. While you’re hacking /etc/passwd, make sure that every user has her real name in the pw_gecos field of the password file (this is the fourth field). It is a question of Usenet netiquette that the sender’s real name appears in the From: field of the article. Of course, you will want to do so anyway when you use mail. The sys File The sys file, located in /etc/news, controls which hierarchies you receive and forward to other sites. Although there are maintenance tools named addfeed and delfeed, we think it’s better to maintain this file by hand. The sys file contains entries for each site to which you forward news, as well as a description of the groups you will accept. The first line is a ME entry that describes your system. It’s a safe bet to use the following: ME:all/all::

Figure 21.1 shows the news flow through relaynews. Articles may be relayed to the local site (denoted by ME), to a site named ponderosa via email, and a site named moria, for which batching is enabled. Figure 21.1: News flow through relaynews Installation C News should be available in a prepackaged format for any modern Linux distribution, so installation will be easy. If not, or if you want to install from the original source distribution, then of course you can.126 No matter how you install it, you will need to edit the C News configuration files. Their formats are described in the following list: sys The sys file controls which newsgroups your site receives and forwards. We discuss it in detail in the following section. active Not usually edited by the administration; contains directions for handling articles in each newsgroup the site handles. organization This file should contain your organization’s name, for example, “Virtual Brewery, Inc.” On your home machine, enter “private site,” or anything else you like. Most people will not consider your site properly configured if you haven’t customized this file. newsgroups This file is a list of all newsgroups, with a one-line description of each one’s purpose. These descriptions are frequently used by your newsreader when displaying the list of all groups to which you are subscribed. mailname Your site’s mail name, e.g., vbrew.com. You can obtain the C News source distribution from its home site at ftp.cs.toronto.edu /pub/c-news/c-news.tar.Z

Chapter 21 - C News One of the most popular software packages for Netnews is C News. It was designed for sites that carry news over UUCP links. This chapter will discuss the central concepts of C News, basic installation, and maintenance tasks. C News stores its configuration files in /etc/news, and most of its binaries are kept below the /usr/lib/news/ directory. Articles are kept below /var/spool/news. You should make sure that virtually all files in these directories are owned by user news or group news. Most problems arise from files being inaccessible to C News. Use suto become the user news before you touch anything in the directory. The only exception is the setnewsids command, which is used to set the real user ID of some news programs. It must be owned by root and have the setuid bit set. In this chapter, we describe all C News configuration files in detail and show you what you have to do to keep your site running. Delivering News Articles can be fed to C News in several ways. When a local user posts an article, the newsreader usually hands it to the inews command, which completes the header information. News from remote sites, be it a single article or a whole batch, is given to the rnews command, which stores it in the /var/spool/news/in.coming directory, from where it will be picked up at a later time by newsrun. With any of these two techniques, however, the article will eventually be handed to the relaynews command. For each article, the relaynews command first checks if the article has already been seen at the local site by looking up the message ID in the history file. Duplicate articles are dropped. Then relaynews looks at the Newsgroups: header line to find out if the local site requests articles from any of these groups. If it does, and the newsgroup is listed in the active file, relaynews tries to store the article in the corresponding directory in the news spool area. If this directory does not exist, it is created. The article’s message ID is then logged to the history file. Otherwise, relaynews drops the article. Sometimes relaynews fails to store an incoming article because a group to which it has been posted is not listed in your active file. In this case, the article is moved to the junk group.124 relaynews also checks for stale or misdated articles and reject them. Incoming batches that fail for any other reason are moved to /var/spool/news/in.coming/bad, and an error message is logged. After this, the article is relayed to all other sites that request news from these groups, using the transport specified for each particular site. To make sure an article isn’t sent to a site that has already seen it, each destination site is checked against the article’s Path: header field, which contains the list of sites the article has traversed so far, written in the UUCP-style bang-path source-routing style described in Chapter 17, Electronic Mail. If the destination site’s name does not appear in this list, the article is sent to it. C News is commonly used to relay news between UUCP sites, although it is also possible to use it in an NNTP environment. To deliver news to a remote UUCP site, either in single articles or whole batches, uux is used to execute the rnews command on the remote site and feed the article or batch to it on standard input. Refer to Chapter 16, Managing Taylor UUCP, for more information on UUCP. Batching is the term used to describe sending large bundles of individual articles all in one transmission. When batching is enabled for a given site, C News does not send any incoming article immediately; instead, it appends its path name to a file, usually called out.going/site/togo. Periodically, a program is executed from a crontab entry by the cron program, which reads this file and bundles all of the listed articles into one or more file, optionally compressing them and sending them to rnews at the remote site.125 124 There may be a difference between the groups that exist at your site and those that your site is willing to receive. For example, the subscription list might specify comp.all, which should send all newsgroups below the comp hierarchy, but at your site you might not list several of the comp newsgroups in the active file. Articles posted to those groups will be moved to junk. 125 Note that this should be the crontabof news; file permissions will not be mangled.

amounts of news over TCP/IP, the server described in that chapter may be enough for you. To install a heavy- duty news server that can handle huge volumes of material, go on to read about InterNet News in Chapter 23.