It took me a little bit to ‘mod’ in suricata-update to replace OPNSense’s update/policy rule management - which is nice, but, I want to modify my rules now (change any → $EXTERNAL_NET and such).
Success! Disabled their cron entry, added my own Web GUI cron entries to run a few Shell scripts to run ‘suricata-update’ or copy over some files that OPNSense tends to replace (a very important custom.yaml to get the rest of the functionality and a rule file list)
It seems that “suricata-update” runs Python in a single process/single thread and it takes a very long time (sometimes 30+ minutes) on a decent multiprocessor host running OPNsense 24.7.8-amd64 / FreeBSD 14.1-RELEASE-p6 (python 3.11).
Are there plans to speed up/multiprocess/thread the “suricata-update” script/project? As a python script writer I would love to help improve but do not want to interrupt flow.
My suricata-update start string:
suricata-update --config /root/suricata/update.yaml --suricata-conf /usr/local/etc/suricata/suricata.yaml --suricata /usr/local/bin/suricata --data-dir /usr/local/etc/suricata --threshold-in=/root/suricata/threshold.in --threshold-out=/usr/local/etc/suricata/threshold.config --output /usr/local/etc/suricata/opnsense.rules -v --no-test --no-reload 2>&1 | tee /root/suricata-update.log
Thank you for all suggestions/replies, I want to help and am not trying replicate Aristole2 or any other feature, just interested in how we can leverage modern Python3 and MP/Threads to speed up what amounts to a lot of data analysis.