[Previous][Up to Table of Contents] [Next]

InterNetNews...[Salz92]: InterNetNews architecture


There are four key programs in the InterNetNews package (see Figure 2):

Of these programs, the most important is innd. We first mention enough of its architecture to give a context for the other programs, and then discuss its design and features in more detail at the end of this section.


Figure 2 : Innd architecture


Innd is a single daemon that receives all incoming articles, files them, and queues them up for propagation. It waits for connections on the NNTP port. When a connection is received, a getpeername (2) is done. If the host is not in an access file, then an nnrpd process is spawned with the connection tied to its standard input and output. (Unlike other implementations , no single INN program implements the entire NNTP protocol.) It is worth noting that nnrpd is only about 3,500 lines of code, and 20% of them are for the ``POST'' command, used to verify the headers in a user's article. Nnrpd provides all NNTP commands except for ``IHAVE'' and an incomplete version of ``NEWNEWS''. On the other hand, it does provide extensions for pattern -matching on an article header and listing exactly what articles are in a group. The NNTP protocol seems to be a good example of the UNIX philosophy: it is small, general, and powerful and can be implemented in a very small program.

Articles are usually forwarded by having innd record the article in a ``batchfile'' which is processed by another program. For Internet sites, the innxmit program is used to offer articles to the host specified on its command line.(Other programs, like nntplink , are supported but not part of the INN distribution.) The input to innxmit is a set of lines containing a pathname to the article and its Message -ID. Since the Message -ID is in the batchfile, innxmit does not have to open the article and scan it before offering the article to the remote site. This can give significant savings if the remote site already has a percentage of the articles.

Until recently, innxmit used writev to send its data to the remote host. At start -up it filled a three - element struct iovec array with the following elements:
 [0] { ".", 1 }; 
 [1] { placeholder }; 
 [2] { "\r\n" }
To write a line, the placeholder was filled in with a pointer to the buffer, and its length, and a single writev was done, starting from either element zero or one. While this implementation was clever, and simpler then what was done elsewhere, it was not very fast. Innxmit now uses a 16k buffer and only does a write when the next line would not fit. This is also consistent with ideas used throughout the rest of INN: use the read and write system calls, referencing the data out of large buffers while avoiding the copying commonly done by the standard I/O library.

The ctlinnd program is used to tell the innd server to perform special tasks. It does this by communicating over a UNIX -domain datagram socket. The socket is behind a firewall directory that is mode 770, so that only members of the news administration group can send messages to it. It is a very small program that parses the first parameter in its command line and converts it to an internal command identifier. This identifier and the remaining parameters are sent to innd which processes the command, and sends back a formatted reply. For example, the following commands stops the server from accepting any new connections, adds a newsgroup, and then tells it to recompute the list of hosts that are authorized newsfeeds: 

 ctlinnd pause "Clearing out log files" 
 ctlinnd newgroup comp.sources.unix m vixie@pa.dec.com 
 ctlinnd reload newsfeeds "Added OSF feed" 
 ctlinnd go ""
The text arguments are sent to syslog (8) for audit purposes.
The most commonly -used ctlinnd command is ``flush.'' This directs the server to close the batchfile that is open for a site, and is typically used as follows: 
 mv batchfile batchfile.work 
 ctlinnd flush sitename 
 innxmit sitename batchfile.work

The flush command points out another difference between INN and other Usenet software. The B News inews program needed no external locking. Files were opened and closed for a very short window, the time needed to process one article. The C News relaynews could be running for a longer period of time. The only way to get access to a batchfile is to either lock the entire news system, which is overkill for the desired task, or to rename the file and wait until the original name shows up again. The INN approach is more efficient and conceptually cleaner.
InterNetNews(Salz) [Source:"InterNetNews: Usenet transport for Internet sites"] [Copyright: 1992 Rich Salz]
InterNetNews(Salz)[Previous][Up to Table of Contents] [Next]
This is from the Mib Software Usenet RKT

RKT Rapid-Links:[Search] [RKT Tips] Path:Usenet RKT / InterNetNews...[Salz92] / 0019.htm