Redis Log without log-rotate

[idev]

Hello,

we identified a log which grow (and still is growing) to 22 GB.

Log File seems to be from the redis container:
a2e6544627ee17f8ebe3aa85432296e84308fa362e2613ddf0b6514a95f59bf7-json.log

last few entries:
`{"log":"1:M 23 Jul 2019 10:19:35.569 # Background AOF rewrite terminated with error\n","stream":"stdout","time":"2019-07-23T10:19:35.569321238Z"}
{"log":"1:M 23 Jul 2019 10:19:35.669 * Starting automatic rewriting of AOF on 262479% growth\n","stream":"stdout","time":"2019-07-23T10:19:35.669917108Z"}
{"log":"1:M 23 Jul 2019 10:19:35.674 * Background append only file rewriting started by pid 16547\n","stream":"stdout","time":"2019-07-23T10:19:35.674740779Z"}
{"log":"16547:C 23 Jul 2019 10:19:35.674 # Opening the temp file for AOF rewrite in rewriteAppendOnlyFile(): Permission denied\n","stream":"stdout","time":"2019-07-23T10:19:35.674770152Z"}
{"log":"1:M 23 Jul 2019 10:19:35.775 # Background AOF rewrite terminated with error\n","stream":"stdout","time":"2019-07-23T10:19:35.775292049Z"}
{"log":"1:M 23 Jul 2019 10:19:35.875 * Starting automatic rewriting of AOF on 262479% growth\n","stream":"stdout","time":"2019-07-23T10:19:35.875845499Z"}
{"log":"1:M 23 Jul 2019 10:19:35.881 * Background append only file rewriting started by pid 16548\n","stream":"stdout","time":"2019-07-23T10:19:35.881380514Z"}
{"log":"16548:C 23 Jul 2019 10:19:35.881 # Opening the temp file for AOF rewrite in rewriteAppendOnlyFile(): Permission denied\n","stream":"stdout","time":"2019-07-23T10:19:35.881541614Z"}
{"log":"1:M 23 Jul 2019 10:19:35.981 # Background AOF rewrite terminated with error\n","stream":"stdout","time":"2019-07-23T10:19:35.981993017Z"}
{"log":"1:M 23 Jul 2019 10:19:36.083 * Starting automatic rewriting of AOF on 262479% growth\n","stream":"stdout","time":"2019-07-23T10:19:36.083339357Z"}
{"log":"1:M 23 Jul 2019 10:19:36.087 * Background append only file rewriting started by pid 16549\n","stream":"stdout","time":"2019-07-23T10:19:36.087995192Z"}
{"log":"16549:C 23 Jul 2019 10:19:36.087 # Opening the temp file for AOF rewrite in rewriteAppendOnlyFile(): Permission denied\n","stream":"stdout","time":"2019-07-23T10:19:36.089309445Z"}
{"log":"1:M 23 Jul 2019 10:19:36.188 # Background AOF rewrite terminated with error\n","stream":"stdout","time":"2019-07-23T10:19:36.18858401Z"}
{"log":"1:M 23 Jul 2019 10:19:36.288 * Starting automatic rewriting of AOF on 262479% growth\n","stream":"stdout","time":"2019-07-23T10:19:36.294436515Z"}
{"log":"1:M 23 Jul 2019 10:19:36.293 * Background append only file rewriting started by pid 16550\n","stream":"stdout","time":"2019-07-23T10:19:36.294532578Z"}
{"log":"16550:C 23 Jul 2019 10:19:36.294 # Opening the temp file for AOF rewrite in rewriteAppendOnlyFile(): Permission denied\n","stream":"stdout","time":"2019-07-23T10:19:36.294549359Z"}
{"log":"1:signal-handler (1563877176) Received SIGTERM scheduling shutdown...\n","stream":"stdout","time":"2019-07-23T10:19:36.373753739Z"}
{"log":"1:M 23 Jul 2019 10:19:36.394 # User requested shutdown...\n","stream":"stdout","time":"2019-07-23T10:19:36.402955923Z"}
{"log":"1:M 23 Jul 2019 10:19:36.394 # There is a child rewriting the AOF. Killing it!\n","stream":"stdout","time":"2019-07-23T10:19:36.402998044Z"}
{"log":"1:M 23 Jul 2019 10:19:36.394 * Calling fsync() on the AOF file.\n","stream":"stdout","time":"2019-07-23T10:19:36.403004901Z"}
{"log":"1:M 23 Jul 2019 10:19:36.394 # Redis is now ready to exit, bye bye...\n","stream":"stdout","time":"2019-07-23T10:19:36.403009592Z"}
{"log":"1:C 23 Jul 2019 10:20:21.028 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo\n","stream":"stdout","time":"2019-07-23T10:20:21.028509079Z"}
{"log":"1:C 23 Jul 2019 10:20:21.028 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=1, just started\n","stream":"stdout","time":"2019-07-23T10:20:21.036454065Z"}
{"log":"1:C 23 Jul 2019 10:20:21.028 # Configuration loaded\n","stream":"stdout","time":"2019-07-23T10:20:21.03647338Z"}
{"log":"1:M 23 Jul 2019 10:20:21.029 * Running mode=standalone, port=6379.\n","stream":"stdout","time":"2019-07-23T10:20:21.036482606Z"}
{"log":"1:M 23 Jul 2019 10:20:21.029 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.\n","stream":"stdout","time":"2019-07-23T10:20:21.036487174Z"}
{"log":"1:M 23 Jul 2019 10:20:21.029 # Server initialized\n","stream":"stdout","time":"2019-07-23T10:20:21.036494051Z"}
{"log":"1:M 23 Jul 2019 10:20:21.029 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.\n","stream":"stdout","time":"2019-07-23T10:20:21.036498002Z"}
{"log":"1:M 23 Jul 2019 10:47:19.678 * DB loaded from append only file: 1618.647 seconds\n","stream":"stdout","time":"2019-07-23T10:47:19.78794494Z"}
{"log":"1:M 23 Jul 2019 10:47:19.679 * Ready to accept connections\n","stream":"stdout","time":"2019-07-23T10:47:19.808241437Z"}

`

Maybe there is some problem with the redis instance, or log-rotate is missing.

Kind regards

[8ear]

Thanks four your issue. I will look at this.
In which timespan does the log grow to 22GB?

[idev]

Thanks four your issue. I will look at this.
In which timespan does the log grow to 22GB?

Don't really know, we just noticed the unusual growing disk space.

[idev]

Forgot to ask, if it is a problem to delete the file?

[8ear]

Hi Sorry for my late reply.
Normally it is no problem. But the best way is to replace the redis docker container.
docker exec -ti misp-robot bash -c "docker-compose -f /srv/MISP-dockerized/current/docker-compose.yml -f /srv/MISP-dockerized/current/docker-compose.override.yml up -d --force-recreate misp-redis"

This should use the misp-robot container to recreate the misp-redis container with the help of docker-compose.

[8ear]

Hi @idev,
I'm sorry to inform you, but I can't fix this issue in this repository.
It is unclear if this development can be integrated into MISP/MISP repository or any other.