I noticed that there are several ways to store Suricata’s alert output: regular/redis, so I thought of several ways to store alert data in Mysql. Which one is more recommended?
Regular. In regular mode, Suricata will store the alert result in the eve.json file, so I can write a script in the background of the system to automatically read the contents of eve.json into mysql, but this will encounter some performance problems such as eve. The json file is too large, the disk IO problem caused by the large amount of data, the synchronization problem of the read data, and so on.
Redis. In redis mode, Suricata will store the alert results in the redis server, so that I can write scripts to automatically persist Redis data to Mysql. This method also encounters some problems, such as memory usage, data consistency, network bandwidth etc.
Unix-command. I see that the official Suricata documentation also recommends interacting through Unix sockets, setting the unix-command output mode in the Suricata configuration file, and specifying the output to a temporary file, and creating a Python script /path/to/ suricata-unix-command.py, which uses the PyMySQL module to connect to the MySQL database and insert Suricata alarm data into the database, but this will also encounter performance problems: for example, the amount of alert data is too large, causing the python script to Connect to the database many times and so on.
I would like to ask, which method do you recommend? Or is there any other recommended method? Can guarantee low latency and high efficiency of data synchronization?
Looking forward to your reply, sincerely thank you
Although I haven’t measured performance impact and related concerns, it seems that sending the data to a socket (using filetype of unix_dgram (UDP) or unix_stream (TCP)) could yield the best results.
You’d write the other side of the connection, of course
There are also ways to write output plugins – here’s a few examples:
Jason’s is written in Rust and gives an example of how to receive alerts from Suricata and persist them. His example’s using Redis but the same concepts will apply to other storage methods.
If I choose sending the data to a socket (using filetype of unix_dgram (UDP) or unix_stream (TCP)), what contents of Suricata.yaml need to be changed to configure correctly?
In fact, I modified the following two parts of the configuration before, and configured to automatically execute toMysql.py in the directory I specified to write to my mysql, but it didn’t seem to work. I would like to ask if there is a problem with this configuration ? If there is a problem, how should it be configured?
The unix-command section is for suricatasc (suricata socket control) – not for alert output. You can read about suricatasc here
In the outputs section by eve-log set filetype to unix_dgram (or unix_stream). This will instruct Suricata to open a UNIX domain socket with UDP (TCP) if you use unix_dgram (unix_stream). Your program will setup a listening socket to receive requests.
Hi Jeff ，First of all thank you for your kind help
"In the section by set to (or ). This will instruct Suricata to open a UNIX domain socket with UDP (TCP) if you use (). "
What do you mean by the passage you mentioned? I don’t quite understand it, could you elaborate more? Or Could you give an example of how to configure it?What does “set to (or)” mean, and what does “use ()” mean? i’m a little confused
Ok, i got it.It was the question displayed on my side before. I saw that some highlighted parts of your reply were not displayed, so I didn’t understand it. Thank you very much.
