Journald read in 1001 seconds bursts

[terba]

I'm testing the journald log collector of v3.7.0. I have one configured on the ossec server and an another on a remote machine via ossec agent. Both have the following config entry:

  <localfile>
    <log_format>journald</log_format>
    <location>all</location>
  </localfile>

The problem is that ossec server creates the alerts based on these journals around every 16 minutes in bursts. Like the following which is the first of such a burst (see the timestamps):

** Alert 1704962147.501876: - pam,syslog,authentication_success,
2024 Jan 11 09:35:47 (myremoteagenthost) 123.123.123.123->journald
Rule: 5501 (level 3) -> 'Login session opened.'
2024-01-11T09:19:17+01:00 myremoteagenthost sudo: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=997)

The clocking of the bursts is separated, so the two collectors do the bursts at different times, but both repeat every ~1000 seconds like this:

host1 (ossec server):
10:06:09
10:22:50
10:39:31

host2 (ossec agent):
10:09:07
10:25:48
10:42:30

I started the collector in the foreground with debug on, but there is nothing interesting there:

2024/01/11 08:09:20 ossec-logcollector: DEBUG: Starting ...
2024/01/11 08:09:20 ossec-logcollector: DEBUG: Waiting main daemons to settle.
2024/01/11 08:09:26 ossec-logcollector: INFO: (unix_domain) Maximum send buffer set to: '212992'.
2024/01/11 08:09:26 ossec-logcollector: DEBUG: Entering LogCollectorStart().
2024/01/11 08:09:26 ossec-logcollector(1951): INFO: Analyzing journald log: 'all'.
2024/01/11 08:09:26 ossec-logcollector: INFO: Started (pid: 2534).
2024/01/11 08:11:36 ossec-logcollector(1904): INFO: File not available, ignoring it: 'all'.

What can I do to overcome this? Thanks in advance.

[terba]

Found the cause: logff[i].ign for the journald reader is uninitialized, so it may have a random value. And the journald read function is not called until ((curr_time - logff[i].size) >= logff[i].ign). I zeroed it at the LogCollectorStart initialization part and it works.

As I see in the code this uninitialized ign could be a problem with command log types too.

[terba]

Here it is:

diff --git a/src/logcollector/logcollector.c b/src/logcollector/logcollector.c
index 92e9c0b4..d83d833b 100644
--- a/src/logcollector/logcollector.c
+++ b/src/logcollector/logcollector.c
@@ -167,6 +167,8 @@ void LogCollectorStart()
             logff[i].command = NULL;
             logff[i].fp = NULL;
             logff[i].ptr = NULL;
+            logff[i].size = 0;
+            logff[i].ign = 0;
         }
 
         else {

[terba]

Other problem, which causes the 1000s bursts is that the logff[i].ign gets incremented after some time. I don't know the cause yet, but this is the log when it gets incremented by one:

2024/01/13 09:07:08 ossec-logcollector: DEBUG: logff.ign = 0
2024/01/13 09:07:08 ossec-logcollector(1103): ERROR: Could not open file 'all' due to [(2)-(No such file or directory)].
2024/01/13 09:07:10 ossec-logcollector: DEBUG: logff.ign = 1

Until it reaches this code, and ign stays at 999 forever:

            /* Too many errors for the file */                                                                                    
            if (logff[i].ign > open_file_attempts) {                                                                              
                /* 999 Maximum ignore */                                                                                          
                if (logff[i].ign == 999) {                                                                                        
                    continue;                                                                                                     
                }
                /* ... */
                logff[i].ign = 999;                                                                                               
                continue;                                                                                                                                                                                            

[bntuser]

Hi @terba
I would like to know how did u manage to start monitoring the logs from journald.
thanks in advance

[terba]

Hi, I switched to CrowdSec.

[bntuser]

So should i do the same?

[terba]

It's up to you.

[bntuser]

Thanks @terba. One last thing, could you help me with #2122?