We are currently facing some problems with Suricata and mainly around the sending of logs. At any moment, Suricata stops to send logs to our log collector. The service status is still ‘Running’ and there is no error messages.
We saw that all of these logs are also displayed in the systemD journal. When we are typing “systemctl status suricata” or “journalctl -fu suricata” all logs are displayed.
We have already faced this kind of situation with another program which spammed this entry and it revealed it was the root cause of a problem we get.
So, is there a possibility to continue to send these logs by Syslog and avoid to have these logs in SystemD Journal ?
Unfortunately I don’t have a good answer for you, but did take a look how systemd/syslog work together on a modern Linux system.
When logging to syslog, Suricata use thes openlog/syslog library calls provided by the operating system. On a systemd enabled Linux system, it is systemd listening for these messages, not syslog like it used to be in the past. And at least with CentOS 8, it is systemd that passes these messages off to the syslog daemon. I don’t think Ubuntu would be much different here.
Unfortunately I don’t see any way in systemd to filter messages in such a way that systemd/journal won’t hold on to the them yet still send them to syslog.
This makes me wonder if syslog, at least via the library calls makes sense with Suricata when logging more than just alerts due to the high load.