A java library for parsing valid Syslog IETF RFC 5424 logs. The library provides it's own parser implementation, but also exposes the Antlr generated base classes and interfaces should you want your own implemenation.
A simple, default usage to parser a Syslog RFC 5424 log line is to build a SyslogParser with the defaults, and pass it the line.
SyslogParser parser = new SyslogParserBuilder().build();
Map<String,Object> syslogMap = parser.parseLine(syslogLine);
To parse a number of Syslog lines together, say from a file you would create
a Reader
and all parseLines
List<Map<String,Object>> syslogMapList = null;
SyslogParser parser = new SyslogParserBuilder().build();
try (Reader reader = new BufferedReader(new FileReader(new File(fileName)))) {
syslogMapList = parser.parseLines(reader);
}
Both parseLine
and parseLines
also provide a functional interface if you prefer that style.
Just pass a Consumer
to the function.
SyslogParser parser = new SyslogParserBuilder().build();
syslogMap = parser.parseLine(syslogLine, (syslogMap) -> {
// do something with map
});
SyslogParser parser = new SyslogParserBuilder().build();
try (Reader reader = new BufferedReader(new FileReader(new File(fileName)))) {
parser.parseLines(reader, (map) -> {
// do something with each map
});
}
SyslogParser parser = new SyslogParserBuilder().build();
try (Reader reader = new BufferedReader(new FileReader(new File(fileName)))) {
parser.parseLines(reader, (map) -> {
// do something with each map
}, (line, throwable) -> {
// do something for a failed line
});
}
The SyslogParserBuilder
supports options for changing the AllowableVariations
, the SyslogSpecifictation
and the KeyProvider
.
Allowable deviations from the RFC 5424 specification. This allows for fields required by the specification, but perhaps omitted by convention to be missing, and a line that is by specificiation technically incorrect to still parse.
This is specificed by an {@code EnumSet}
/**
* Properly formed RFC 5424 Syslog.
*/
NONE,
/**
* RFC 5424 Syslog that does not have PRIORITY.
*/
PRIORITY,
/**
* RFC 5424 Syslog that does not have VERSION.
*/
VERSION
A KeyProvider
is used to provide the map keys for the Syslog data.
The default KeyProvider
: DefaultKeyProvider
provides keys using the SyslogKeys
:
MESSAGE("syslog.message"),
HEADER_APPNAME("syslog.header.appName"),
HEADER_HOSTNAME("syslog.header.hostName"),
HEADER_PRI("syslog.header.pri"),
HEADER_PRI_SEVERITY("syslog.header.severity"),
HEADER_PRI_FACILITY("syslog.header.facility"),
HEADER_PROCID("syslog.header.procId"),
HEADER_TIMESTAMP("syslog.header.timestamp"),
HEADER_MSGID("syslog.header.msgId"),
HEADER_VERSION("syslog.header.version"),
STRUCTURED_BASE("syslog.structuredData."),
STRUCTURED_ELEMENT_ID_FMT("syslog.structuredData.%s"),
STRUCTURED_ELEMENT_ID_PNAME_FMT("syslog.structuredData.%s.%s"),
STRUCTURED_ELEMENT_ID_PNAME_PATTERN("syslog.structuredData\\.(.*)\\.(.*)$");
A custom KeyProvider
can be supplied to the SyslogParserBuilder
if there is a different key strategy required.
The NilPolicy
governs how the parser handles nil message parts. That is message parts that can be nil as part of a valid message;
- HOSTNAME
- APPNAME
- PROCID
- MSGID
- TIMESTAMP
- OMIT : The map will not contain an entry for that field
- NULL : The map will contain a null entry for that field
- DASH : The map will contain a '-' for that field
The default policy is OMIT
.
Simple Syslog 5424 uses Antlr 4 to generate the Listener
that the parser is based on.
The generated Rfc5424Listener
and Rfc5424Visitor
interfaces, or Rfc5424BaseListener
and Rfc5424BaseVisitor
classes,
may be used to implement new parsers as well in the event that you prefer different handling.
Implementors would then build their own parsers or builders etc. In other words the use of this library would minimally be the Antlr classes alone.
For example you would build a 'parser' that used your implementations, most likely implemented like this:
Rfc5424Lexer lexer = new Rfc5424Lexer(new ANTLRInputStream(syslogLine));
Rfc5424Parser parser = new Rfc5424Parser(new CommonTokenStream(lexer));
Rfc5424Listener listener = new MyCustomListener(keyProvider);
parser.addParseListener(listener);
Rfc5424Parser.Syslog_msgContext ctx = parser.syslog_msg();
return listener.getMyCustomResult();
-
Why not just use java-grok?
Though I love java-grok (it is used in Apache Metron and Apache Nifi which I contribute to), and have even submitted PRs to it, it and other Regex based parsers do not handle Syslog 5424 Structured Data. I wanted something that did. I have not found any regex based approach which handles structured data in a single pass. If you find one, let me know!
-
Why not fix the groks to handle it? Or create regexes outside of grok?
I'm not good enough at regex, and couldn't write something that worked single pass.
-
Why not write a custom state machine type parser?
I like Antlr and wanted to try it ;)
<dependency>
<groupId>com.github.palindromicity</groupId>
<artifactId>simple-syslog-5424</artifactId>
<version>0.0.16</version>
<type>pom</type>
</dependency>