INN 2.7.2

2024-06-22
Full changes and diff from previous release

New Features

• Added two new options in storage.conf, contributed by Christoph Biedl:

  • a filtered option to store articles which have been rejected by a Perl or Python filter for innd into a separate storage class, when dontrejectfiltered is set to true in inn.conf;

  • a path option to store articles by the contents of the Path header field, notably to isolate the storage of spammy sites from other sites.

• The delayer program, which previously was in the contrib directory, is now installed by default. It notably permits generating a delayed feed, for instance to give cancel control articles and NoCeM messages time to arrive, and therefore actually cancel articles before they are locally stored. Thanks to Christoph Biedl for the documentation, testing, and addition of several useful flags to delayer.

• Added the INN::ovsqlite_client Perl module to access an ovsqlite overview database through ovsqlite-server from a Perl script. This module provides search, add, remove and expire functions for information stored in an ovsqlite database (newsgroups and overview data associated to articles in these newsgroups). Many thanks to Bo Lindbergh for it, as well as for two samples in the contrib directory (ovsqlite-dump and ovsqlite-undump) showing how to use the module.

• Added several new options to better parameterize perl-nocem. The location of its configuration file, the keyring to use, and the full gpg and gpgv commands to run can now be modified by flags passed to the program. Unprocessed NoCeM notices will also show up in daily Usenet reports generated by innreport. Log verbosity can be controlled, cancelled articles can be backed up into files, and perl-nocem can write logs to a separate perl-nocem.log file. Local rules to fine-tune within a NoCeM notice which articles get cancelled can also be applied. See its manual page for more information about the use of these options.

• innreport now supports high-precision timestamps like 2023-07-29T04:15:01.889064+02:00 that syslog can be parameterized to use; daily Usenet reports otherwise indicated all these logs as unknown entries because a format like Jul 29 04:15:01 was expected.

• innreport now alphabetically sorts lines which have the same rank instead of displaying them in random order.

• scanspool now detects empty files in a tradspool news spool and directories with an all-digit component (which may conflict with a possible file with the same name), correctly parses continuation lines in header fields, and can automatically remove articles reported to have a problem (when run with the new -r flag).

• Added a new ovflushcount parameter in inn.conf, defaulting to 50, to parameterize the number of articles received between flushing their overview data to disk, when using the buffindexed overview storage method. (In previous releases, that number was ten times the value of icdsynccount, but a dedicated parameter is better.)

Bug Fixes

• perl-nocem now correctly parses the identifiers of NoCeM issuers with special characters like + in the configuration file.

• Fixed a hang when posting articles if COMPRESS DEFLATE is active but TLS is not. Thanks to Enrik Berkhan for the patch for nnrpd.

• innd no longer dies when ctlinnd trace is run on an invalid channel.

• INN now properly handles header field names starting with a leading dot: innd accurately computes the :bytes metadata item of articles with such header field names, and nnrpd correctly preserves the leading initial dot at injection time and rightly parses such header field names in HDR, XHDR and XPAT commands.

• nnrpd now rejects articles with invalid dot-stuffing (that is to say when a dot at the beginning of a line is not doubled) as it is a violation of the NNTP protocol. (innd still goes on accepting and propagating such articles as they have already been injected and propagated.)

• Fixed the computation of the Bytes header field by pullnews (in header-only mode with the -B flag).

• Fixed the generation of rnews batches by pullnews (when using the -r flag) which had not the expected native LF line termination.

• Fixed inndf on 32-bit architectures with large file support (previously, inndf could not count more than 2^32 bytes or inodes on these architectures).

• Fixed an issue preventing articles from expiring when using the ovsqlite method, in a very rare case. When an article had more than 100,000 bytes of overview data (for instance with a Subject header field of that length), overview expiration was no longer done for newsgroups carrying this article. Such articles, which most certainly are spams anyway, are no longer added to the ovsqlite database. Thanks for Jesse Rehmer for the bug report.

• Fixed a database lock issue when running ovsqlite-util on a running server with the transtimelimit parameter in ovsqlite.conf set to a higher value than the default busy timeout of 30 seconds of ovsqlite-util. Thanks to Jesse Rehmer for the bug report and Bo Lindbergh for the fix.

• Fixed a foreign key issue preventing ovsqlite-util from fixing problems found when running with the -F flag.

• innd no longer malfunctions nor throttles when the maximum number of file descriptors supported by the system is reached. If needing to use more file descriptors than the default system limit, a new LARGE_FD_SETSIZE option can be set at build time. See the documentation for rlimitnofile in inn.conf for more information. Thanks to Jesse Rehmer for the bug report.

Other Change

• Up-to-date control.ctl and nocem.ctl files are provided with this release. You may want to manually update your configuration with the new information recorded about hierarchies and NoCeM issuers, and make sure the relevant PGP keys are installed on your system.