Ftp Web Hosting - J2Ee Hosting Sun Java Developing Blog

y Posting directly to this news server is allowed. n Posting directly to this news server is not allowed. This prevents newsreaders from posting directly to this news server. New articles may only be received from other news servers. m The group is moderated. Any articles posted to this newsgroup are forwarded to the newsgroup moderator for approval before they enter the newsgroup. Most newsgroups are unmoderated. j Articles in this group are not kept, but only passed on. This causes the news server to accept the article, but all it will do with it is pass it to the “up-stream” news servers. It will not make the articles available to newsreaders reading from this server. Articles cannot be posted to this newsgroup. The only way that news articles are delivered to this server is by feeding them from another news server. Newsreaders may not directly write articles to this server. =foo.bar Articles are locally filed into the “foo.bar'’ group. In our simple server configuration we’ll carry a small number of newsgroups, so our /var/lib/news/active file will look like: control 0000000000 0000000001 y junk 0000000000 0000000001 y rec.crafts.brewing 0000000000 0000000001 y rec.crafts.brewing.ales 0000000000 0000000001 y rec.crafts.brewing.badtaste 0000000000 0000000001 y rec.crafts.brewing.brandy 0000000000 0000000001 y rec.crafts.brewing.champagne 0000000000 0000000001 y rec.crafts.brewing.private 0000000000 0000000001 y The himark and lomark numbers in this example are those you would use when creating new newsgroups. The himark and lomark numbers will look quite different for a newsgroup that has been active for some time. The newsgroups file is even simpler. It provides one-line descriptions of newsgroups. Some newsreaders are able to read and present this information to a user to help them decide whether they want to subscribe. The format of the newsgroups file is simply: name description The name field is the name of a newsgroup, and the

The next line defines what hostname inews is going to use when adding a From: line to articles posted by local users. Most newsreaders use the From: field when composing a reply mail message to the author of an article. If you omit this field, it will default to your news host’s fully qualifed domain name. This is ot always the best choice. You might, for example, have news and mail handled by two different hosts. In this case, you would supply the fully qualified domain name of your mail host after the fromhost statement. The pathhost line defines the hostname INN is to add to the Path: header field whenever it receives an article. In most cases, you will want to use the fully qualified domain name of your news server; you can then omit this field since that is the default. Occasionally you may want to use a generic name, such as news.vbrew.com, when serving a large domain. Doing this allows you to move the news system easily to a different host, should you choose to at some time. The next line contains the organization keyword. This statement allows you to configure what text inews will put into the Organization: line of articles posted by your local users. Formally, you would place a description of your organization or your organization’s name in full here. Should you not wish to be so formal, it is fashionable for organizations with a sense of humor to exhibit it here. The organization keyword is mandatory and specifies the pathname of the mail transport agent that will be used for posting moderator messages. %s is replaced by the moderator email address. The moderatormailer entry defines a default address used when a user tries to post to a moderated newsgroup. The list of moderator addresses for each newsgroup is usually kept in a separate file, but you will have a hard time keeping track of all of them. The moderatormailer entry is therefore consulted as a last resort; if it is defined, inews will replace the %s string with the (slightly transformed) newsgroup name and send the entire article to this address. For instance, when posting to soc.feminism, the article is mailed to socfeminism@ uunet.uu.net, given the above configuration. At UUNET, there should be a mail alias installed for each of these submissions addresses that automatically forwards all messages to the appropriate moderator. Finally, each of the remaining entries specifies the location of some component file or executable belonging to INN. If you’ve installed INN from a package, these paths should have been configured for you. If you’re installing from source, you’ll need to ensure that they reflect where you’ve installed INN. Configuring Newsgroups The news administrator on a system is able to control which newsgroups users have access to. INN provides two configuration files that allow the administrator to decide which newsgroups to support and provide descriptions for them. The active and newsgroups files The active and newsgroups files are used to store and describe the newsgroups hosted by this news server. They list which newsgroups we are interested in receiving and serving articles for, and administrative information about them. These files are found in the /var/lib/news/ directory. The active file determines which newsgroups this server supports. Its syntax is straightforward. Each line in the active file has four fields delimited by whitespace: name himark lomark flags The name field is the name of the newsgroup. The himark field is the highest number that has been used for an article in that newsgroup. The lomark field is the lowest active number in use in the newsgroup. To illustrate how this works, consider the follow scenario. Imagine that we have a newly created newsgroup: himark and lowmark are both 0 because there are no articles. If we post 5 articles, they will be numbered 1 through 5. himark will now equal 5, the highest numbered article, and lowmark will equal 1, the lowest active article. If article 5 is cancelled there will be no change; himark will remain at 5 to ensure that that article number is not reallocated and lowmark will remain at 1, the lowest active article. If we now cancel article 1, himark will remain unchanged, but lowmark will now equal 2, because 1 is no longer active. If we now post a new article, it will be assigned article number 6, so himark will now equal 6. Article 5 has been in use, so we won’t reassign it. low- mark remains at 2. This mechanism allows us to easily allocate unique article numbers for new articles and to calculate approximately how many active articles there are in the group: himark-lowmark. The field may contain one of the following:

INN Configuration Files Having completed these general tasks, you can now turn to the really interesting part of INN: its configuration files. All these files reside in /etc/news. Some changes to configurations files were introduced in Version 2, and it is Version 2 that we describe here. If you’re running an older version, you should find this chapter useful to guide you in upgrading your configuration. During the next few sections, we will discuss them one by one, building the Virtual Brewery’s configuration as an example. If you want to find out more about the features of individual configuration files, you can also consult the manual pages; the INN distribution contains individual manual pages for each of them. Global Parameters There are a number of INN parameters that are global in nature; they are relevant to all newsgroups carried. The inn.conf file INN’s main configuration file is inn.conf. Among other things, it determines the name by which your machine is known on Usenet. Version 2 of INN allows a baffling number of parameters to be configured in this file. Fortunately, most parameters have default values that are reasonable for most sites. The inn.conf(5) file details all of the parameters, and you should read it carefully if you experience any problems. A simple example inn.conf might look like: # Sample inn.conf for the Virtual Brewery server: vlager.vbrew.com domain: vbrew.com fromhost: vbrew.com pathhost: news.vbrew.com organization: The Virtual Brewery mta: /usr/sbin/sendmail -oi %s moderatormailer: %s@uunet.uu.net # # Paths to INN components and files. # pathnews: /usr/lib/news pathbin: /usr/lib/news/bin pathfilter: /usr/lib/news/bin/filter pathcontrol: /usr/lib/news/bin/control pathdb: /var/lib/news pathetc: /etc/news pathrun: /var/run/news pathlog: /var/log/news pathhttp: /var/log/news pathtmp: /var/tmp pathspool: /var/spool/news patharticles: /var/spool/news/articles pathoverview: /var/spool/news/overview pathoutgoing: /var/spool/news/outgoing pathincoming: /var/spool/news/incoming patharchive: /var/spool/news/archive pathuniover: /var/spool/news/uniover overviewname: .overview The first line tells the programs rnews and inews which host to contact when delivering articles. This entry is absolutely crucial; to pass articles to innd, they have to establish an NNTP connection with the server. The domain keyword should specify the domain portion of the host’s fully qualified domain name. A couple of programs must look up your host’s fully qualified domain name; if your resolver library returns the unqualified hostname only, the name given in the domain attribute is tacked onto it. It’s not a problem to configure it either way, so it’s best to define domain.

Newsreaders and INN Newsreaders running on the same machine as the server (or having mounted the server’s news spool via NFS) can read articles from the spool directly. To post an article composed by the user, they invoke the inewsprogram, which adds any header fields that are missing and forwards them to the daemon via NNTP. Alternatively, newsreaders can access the server remotely via NNTP. This type of connection is handled differently from NNTP-based news feeds, to avoid tying up the daemon. Whenever a newsreader connects to the NNTP server, innd forks a separate program called nnrpd, which handles the session while innd returns to the more important things (receiving incoming news, for example).138 You may be wondering how the innd process can distinguish between an incoming news feed and a connecting newsreader. The answer is quite simple: the NNTP protocol requires that an NNTP-based newsreader issue a mode reader command after connecting to the server; when this command is received, the server starts the nnrpd process, hands the connection to it, and returns to listening for connections from another news server. There used to be at least one DOS-based newsreader which was not configured to do this, and hence failed miserably when talking to INN, because innd itself does not recognize any of the commands used to read news if it doesn’t know the connection is from a news reader. We’ll talk a little more about newsreader access to INN under “Controlling Newsreader Access,” later in the chapter. Installing INN Before diving into INN’s configuration, let’s talk about its installation. Read this section, even if you’ve installed INN from one of the various Linux distributions; it contains some hints about security and compatibility. Linux distributions included Verson INN-1.4sec for quite some time. Unfortunately, this version had two subtle security problems. Modern versions don’t have these problems and most distributions include a precompiled Linux binary of INN Version 2 or later. If you choose, you can build INN yourself. You can obtain the source from ftp.isc.org in the /isc/inn/ directory. Building INN requires that you edit a configuration file that tells INN some detail about your operating system, and some features may require minor modifications to the source itself. Compiling the package itself is pretty simple; there’s a script called BUILD that will guide you through the process. The source also contains extensive documentation on how to install and configure INN. After installing all binaries, some manual fixups may be required to reconcile INN with any other applications that may want to access its rnewsor inews programs. UUCP, for instance, expects to find the rnewsprogram in /usr/bin or /bin, while INN installs it in /usr/lib/bin by default. Make sure /usr/lib/bin/ is in the default search path, or that there are symbolic links pointing to the actual location of the rnews and inews commands. Configuring INN: the Basic Setup One of the greatest obstacles beginners may face is that INN requires a working network setup to function properly, even when running on a standalone host. Therefore, it is essential that your kernel supports TCP/IP networking when running INN, and that you have set up the loopback interface as explained in Chapter 5, Configuring TCP/IP Networking. Next, you have to make sure that innd is started at boot time. The default INN installation provides a script file called boot in the /etc/news/ directory. If your distribution uses the SystemV-style init package, all you have to do is create a symbolic link from your /etc/init.d/inn file pointing to /etc/news/boot. For other flavors of init, you have to make sure /etc/news/boot is executed from one of your rc scripts. Since INN requires networking support, the startup script should be run after the network interfaces are configured. The name apparently stands for NetNews Read & Post Daemon.

Figure 23.1: INN architecture (simplified for clarity) When receiving an article, innd first looks up its message ID in the history file. Duplicate articles are dropped and the occurrences are optionally logged. The same goes for articles that are too old or lack some required header field, such as Subject:.136 If innd finds that the article is acceptable, it looks at the Newsgroups: header line to find out what groups it has been posted to. If one or more of these groups are found in the active file, the article is filed to disk. Otherwise, it is filed to the special group junk. Individual articles are kept below /var/spool/news, also called the news spool. Each newsgroup has a separate directory, in which each article is stored in a separate file. The file names are consecutive numbers, so that an article in comp.risks may be filed as comp/risks/217, for instance. When innd finds that the directory it wants to store the article in does not exist, it creates it automatically. Apart from storing articles locally, you may also want to pass them on to outgoing feeds. This is governed by the newsfeeds file that lists all downstream sites along with the newsgroups that should be fed to them. Just like innd’s receiving end, the processing of outgoing news is handled by a single interface, too. Instead of doing all the transport-specific handling itself, innd relies on various backends to manage the transmission of articles to other news servers. Outgoing facilities are collectively dubbed channels. Depending on its purpose, a channel can have different attributes that determine exactly what information innd passes on to it. For an outgoing NNTP feed, for instance, innd might fork the innxmit program at startup, and, for each article that should be sent across that feed, pass its message ID, size, and filename to innxmit’s standard input. For an outgoing UUCP feed, on the other hand, it might write the article’s size and file name to a special logfile, which is head by a different process at regular intervals in order to create batches and queue them to the UUCP subsystem. Besides these two examples, there are other types of channels that are not strictly outgoing feeds. These are used, for instance, when archiving certain newsgroups, or when generating overview information. Overview information is intended to help newsreaders thread articles more efficiently. Old-style newsreaders had to scan all articles separately in order to obtain the header information required for threading. This would put an immense strain on the server machine, especially when using NNTP; furthermore, it was very slow.137 The overview mechanism alleviates this problem by prerecording all relevant headers in a separate file (called .overview) for each newsgroup. This information can then be picked up by newsreaders either by reading it directly from the spool directory, or by using the XOVER command when connected via NNTP. INN has the innd daemon feed all articles to the overchan command, which is attached to the daemon through a channel. We’ll see how this is done when we discuss configuring news feeds later. 136 This is indicated by the Date: header field; the limit is usually two weeks. 137 Threading 1,000 articles when talking to a loaded server could easily take around five minutes, which only the most dedicated Usenet addict would find acceptable.

Chapter 23 - Internet News The Internet News daemon (INN) is arguably the most popular Netnews server in use today. INN is extremely flexible and is suitable for all but the smallest news sites.135 INN scales well and is suited to large news server configurations. The INN server comprises a number of components, each with their own configuration files that we will discuss in turn. Configuration of INN can be a little involved, but we’ll describe each of the stages in this chapter and arm you with enough information to make sense of the INN manual pages and documentation and build configurations for just about any application. Some INN Internals INN’s core program is the innd daemon. innd’s task is to handle all incoming articles, storing them locally, and to pass them on to any outgoing newsfeeds if required. It is started at boot time and runs continually as a background process. Running as a daemon improves performance because it has to read its status files only once when starting. Depending on the volume of your news feed, certain files such as history (which contain a list of all recently processed articles) may range from a few megabytes to tens of megabytes. Another important feature of INN is that there is always only one instance of innd running at any time. This is also very beneficial to performance, because the daemon can process all articles without having to worry about synchronizing its internal states with other copies of innd rummaging around the news spool at the same time. However, this choice affects the overall design of the news system. Because it is so important that incoming news is processed as quickly as possible, it is unacceptable that the server be tied up with such mundane tasks as serving newsreaders accessing the news spool via NNTP, or decompressing newsbatches arriving via UUCP. Therefore, these tasks have been broken out of the main server and implemented in separate support programs. Figure 23.1 attempts to illustrate the relationships between innd, the other local tasks, and remote news servers and newsreaders. Today, NNTP is the most common means of transporting news articles around, and innd doesn’t directly support anything else. This means that innd listens on a TCP socket (port 119) for connections and accepts news articles using the “ihave” protocol. Articles arriving by transports other than NNTP are supported indirectly by having another process accept the articles and forward them to innd via NNTP. Newsbatches coming in over a UUCP link, for instance, are traditionally handled by the rnews program. INN’s rnews decompresses the batch if necessary, and breaks it up into individual articles; it then offers them to innd one by one. Newsreaders can deliver news when a user posts an article. Since the handling of newsreaders deserves special attention, we will come back to this a little later. 135 Very small news sites should consider a caching NNTP server program like leafnode, which is available at http://wpxx02.toxi.uniwuerzburg. de/~krasel/leafnode.html.

This conversation shows nntpd’s proper reaction; the message Got it tells you that it already has this article. If you get a message of 335 Ok instead, the lookup in the history file failed for some reason. Terminate the conversation by typing Ctrl-D. You can check what has gone wrong by checking the system log; nntpd logs all kinds of messages to the daemon facility of syslog. An incompatible dbm library usually manifests itself in a message complaining that dbminit failed.

This is a sample nntp_access file: # # by default, anyone may transfer news, but not read or post default xfer no # # public.vbrew.com offers public access via modem. We allow # them to read and post to any but the local.* groups public.vbrew.com read post !local # # all other hosts at the brewery may read and post *.vbrew.com read post NNTP Authorization The nntpd daemon provides a simple authorization scheme. If you capitalize any of the access tokens in the nntp_access file, nntpd requires authorization from the client for the respective operation. For instance, when specifying a permission of Xfer or XFER, (as opposed to xfer), nntpd will not let the client transfer articles to your site unless it passes authorization. The authorization procedure is implemented by means of a new NNTP command named AUTHINFO. Using this command, the client transmits a username and a password to the NNTP server. nntpd validates them by checking them against the /etc/passwd database and verifies that the user belongs to the nntp group. The current implementation of NNTP authorization is only experimental and has therefore not been implemented very portably. The result of this is that it works only with plain-style password databases; shadow passwords are not recognized. If you are compiling from source and have the PAM package installed, the password check is fairly simple to change. nntpd Interaction with C News When nntpd receives an article, it has to deliver it to the news subsystem. Depending on whether it was received as a result of an IHAVEor POST command, the article is handed to rnewsor inews, respectively. Instead of invoking rnews, you may also configure it (at compile time), to batch the incoming articles and move the resulting batches to /var/spool/news/in.coming, where they are left for relaynews to pick them up at the next queue run. nntpd has to have access to the history file to be able to properly perform the ihave/sendme protocol. At compile time, you have to make sure the path to that file is set correctly. If you use C News, make sure that C News and nntpd agree on the format of your history file. C News uses dbm hashing functions to access it; however, there are quite a number of different and slightly incompatible implementations of the dbm library. If C News has been linked with a different dbm library than you have in your standard libc, you have to link nntpd with this library, too. nntpd and C news disagreement sometimes produces error messages in the system log that nntpd can not open it properly, or you might see duplicate articles being received via NNTP. A good test of a malfunctioning news transfer is to pick an article from your spool area, telnet to the nntp port, and offer it to nntpd as shown in the next example. Of course, you have to replace msg@id with the message ID of the article you want to feed to nntpd: $ telnet localhost nntp Trying 127.0.0.1… Connected to localhost Escape characters is ‘^ ]’. 201 vstout NNTP[auth] server version 1.5.11t (16 November 1991) ready at Sun Feb 6 16:02:32 1194 (no posting) IHAVE msg@id435 Got it. QUIT

Installing the NNTP Server The NNTP server (nntpd) may be compiled in two ways, depending on the expected load on the news system. There are no compiled versions available, because of some site-specific defaults that are hardcoded into the executable. All configuration is done through macros defined in common/conf.h. nntpd may be configured as either a standalone server that is started at system boot time from an rc file, or a daemon managed by inetd. In the latter case, you have to have the following entry in /etc/inetd.conf: nntp stream tcp nowait news /usr/etc/in.nntpd nntpd The inetd.conf syntax is described in detail in Chapter 12, Important Network Features. If you configure nntpd as standalone, make sure that any such line in inetd.conf is commented out. In either case, you have to make sure the following line appears in /etc/services: nntp 119/tcp readnews untp # Network News Transfer Protocol To temporarily store any incoming articles, nntpd also needs a .tmp directory in your news spool. You should create it using the following commands: # mkdir /var/spool/news/.tmp # chown news.news /var/spool/news/.tmp Restricting NNTP Access Access to NNTP resources is governed by the file nntp_access in /etc/news. Lines in this file describe the access rights granted to foreign hosts. Each line has the following format: site read|xfer|both|no post|no [!exceptgroups] If a client connects to the NNTP port, nntpd attempts to obtain the host’s fully qualified domain name from its IP address using reverse lookup. The client’s hostname and IP address are checked against the site field of each entry in the order in which they appear in the file. Matches may be either partial or exact. If an entry matches exactly, it applies; if the match is partial, it applies only if there is no other match following it that is at least as good. site may be specified in one of the following ways: Hostname This is a fully qualified domain name of a host. If this matches the client’s canonical hostname literally, the entry applies, and all following entries are ignored. IP address This is an IP address in dotted quad notation. If the client’s IP address matches this, the entry applies, and all following entries are ignored. Domain name This is a domain name, specified as *.domain. If the client’s hostname matches the domain name, the entry matches. Network name This is the name of a network as specified in /etc/networks. If the network number of the client’s IP address matches the network number associated with the network name, the entry matches. Default The string default matches any client. Entries with a more general site specification should be specified earlier, because any matches will be overridden by later, more exact matches. The second and third fields describe the access rights granted to the client. The second field details the permissions to retrieve news by pulling (read), and transmit news by pushing (xfer). A value of both enables both; no denies access altogether. The third field grants the client the right to post articles, i.e., deliver articles with incomplete header information, which is completed by the news software. If the second field contains no, the third field is ignored. The fourth field is optional and contains a comma-separated list of groups to which the client is denied access.

Organization: The Virtual brewery Lines: 2 Message-ID: <7g5bhm$8f$2@news.vbrew.com> NNTP-Posting-Host: localhost X-Server-Date: 27 Apr 1999 21:51:50 GMT Body: Xref: news.vbrew.com junk:2 . Retrieving an Article Body Only If, on the other hand, the user decides she does want to read the article, her newsreader needs a way of requesting that the message body be transmitted. The body command is used for this purpose. It operates in much the same way as the head command, except that only the message body is returned: body 2 222 2 <7g5bhm$8f$2@news.vbrew.com> body This is another test message, please feel free to ignore it too. . Reading an Article from a Group While it is normally most efficient to separately transfer the headers and bodies of selected articles, there are occasions when we are better off transferring the complete article. A good example of this is in applications through which we want to transfer all of the artices in a group without any sort of preselection, such as when we are using an NNTP cache program like leafnode.134 Naturally, NNTP provides a means of doing this, and not surprisingly, it operates almost identically to the head command as well. The article command also accepts an article number or message ID as an argument, but returns the whole article including its header: article 1 220 1 <7g2o5r$aa$6@news.vbrew.com> article Path: news.vbrew.com!not-for-mail From: terry@richard.geek.org.au Newsgroups: junk Subject: test message number 1 Date: 26 Apr 1999 22:08:59 GMT Organization: The Virtual brewery Lines: 2 Message-ID: <7g2o5r$aa$6@news.vbrew.com> NNTP-Posting-Host: localhost X-Server-Date: 26 Apr 1999 22:08:59 GMT Body: Xref: news.vbrew.com junk:1 This is a test message, please feel free to ignore it. . If you attempt to retrieve an unknown article, the server will return a message with an appropriately coded response code and perhaps a readable text message: article 4 423 Bad article number We’ve described how the most important NNTP commands are used in this section. If you’re interested in developing software that implements the NNTP protocol, you should refer to the relevant RFC documents; they provide a great deal of detail that we couldn’t include here. Let’s now look at NNTP in action through the nntpd server. leafnode is available by anonymous FTP from wpxx02.toxi.uni-wuerzburg.de in the /pub/ directory.