v2.0.0
20th April 2016
Log Courier 2.x is compatible with the 1.x Logstash plugins.
_Breaking Changes_
- CurveZMQ transport has been removed
- The
lc-curvekey
utility has been removed - Multiline codecs can now be configured with multiple patterns. As such, the
pattern
configuration has been replaced withpatterns
and it is now an array
of patterns - Multiline codecs have a new
match
configuration that can be set toall
or
any
, to control how multiple patterns are used. The default isany
. - Multiline patterns can be individually negated with a "!" prefix. The
negate
configuration directive has been removed. A "=" prefix is also possible to allow
patterns that need to start with a literal "!" - Filter codec patterns now also accept negation in the same method. The
negate
configuration directive has also been removed for filters and a new
match
configuration added that also defaults toany
- Multiple codecs can now be specified. As such, the
codec
configuration has
been renamed tocodecs
and must now be an array. - The
persist directory
configuration is now required, unless it was built in
at build time (which it will be for RPM and DEB packages - see Build Changes). - All
admin
prefixed configurations have been moved from thegeneral
section
into their ownadmin
section, and the prefix removed - The
reconnect
transport configuration has been removed, and replaced with
reconnect backoff
andreconnect backoff max
_Changes_
- A new
global fields
configuration is available in thegeneral
section
where fields that are to be added to all events from all paths can be specified.
This complements the currentfields
configuration that is per-path. - A new
method
configuration directive has been added to thenetwork
section, which allowsrandom
,failover
orloadbalance
network modes.
Information on these new values can be found in the configuration documentation.
The default israndom
for backwards compatibility. - The
dead time
stream configuration directive now defaults to 1 hour - New backoff configurations,
failure backoff
andfailure backoff max
, have
been added, to allow backoff from problematic remote endpoints where, for
example, connection always succeeds but transmission attempts always timeout. - Configuration files can now be in YAML format by giving the file a
.yaml
extension, and this will be the preferred format. JSON format will continue to
be used for.conf
and.json
configuration files - The
config
parameter can now be omitted fromlog-courier
if a default
configuration file was specified during build (see Build Changes) lc-admin
can now be given a configuration file to load the
admin
.listen address
from using theconfig
parameter. In the absense of
bothconnect
andconfig
parameters it will load the default configuration file if
one was specified during build (see Build Changes)- Log Courier remote administration via
lc-admin
is now a REST interface,
allowing third-party integrations and monitoring - Harvester snapshot information reported by
lc-admin
now contains additional
information such as the last known file size and percentage completion (#239) - Harvester snapshot information is now updated even when the remote server is
down - Publisher snapshot information reported by
lc-admin
now contains a list of
enabled endpoints and their statuses (#199) - If the harvester detects a log file has had data written without a new line
ending, it will report a warning to the log so it can be investigated
_Fixes_
- Do not open dead files on startup (which causes excessive memory usage if
there are many of them) if their size has not changed since they were last
opened (#242) - When a long line was split, the "splitline" tag was not being appended if the
fields configuration already contained tags
_RPM Packaging_
- Fix broken systemd configuration (#237)
- Fix non-fatal %postun script failure, and fix starting of Log Courier during
package upgrade even if it was not originally started (#228)
_DEB Packaging_
- Fix broken systemd configuration (#237)
_Build Changes_
- Some configuration parameters, such as default configuration file and default
persist directory can now be specified at build time by setting environment
variables for them. For example:LC_DEFAULT_CONFIGURATION_FILE
and
LC_DEFAULT_GENERAL_PERSIST_DIR
. These will be used in the RPM and DEB packages
to set platform specific defaults - Simplified the entire build process to use only Go tooling to make
cross-platform building significantly easier, especially on Windows - Discarded Ruby test framework in favour of developing a Go test framework over
time