I am trying to achiev the blocking and also alerting using the Crowdsec - Suricata log parser:
I am having a hard time doing this and I am uncertain of some thing. From what I see the fast.log is not enabled by default.
If I enable it, should I modify the suricata.yaml and enable it from there or also add it cu custom.yaml
Also can someone please point out how to add log rotation to:
/var/log/suricata/fast.log ?
Guessing here, but, you might have an OPNSense?
The important part is to make a new file to get Crowdsec to parse the log file you want it to parse. You can make a new one but will have to manage the ācustom.yamlā file in a way that might not make sense, but since it gets copied over, is necessary.
Thanks for the reply. I did a similar approach but with fast.logs enabled. The things is that I populated the costom.yaml only with the fast.logs config and the eve.json
I am not shure if I need to be 1 on 1 copy like the suricata.yaml file.
I tried first to copy and have the exact same content in both but after doing this, the machine started to run out of memory super fast and eventually when swap was also full it restarted. I donāt know why or which approach is better.
Currently the setup is working and the fast.log get populated with events and the flow is working ok but since I only have this part of the config in custom.yaml I donāt know if everything is working ok or not.
%YAML 1.1
---
default-log-dir: /var/log/suricata/
outputs:
- fast:
enabled: yes
filename: fast.log
append: yes
- eve-log:
enabled: yes
filetype: regular
filename: eve.json
types:
- alert:
tagged-packets: yes
- anomaly:
enabled: yes
- drop:
alerts: yes
flows: start
- dns:
enabled: yes
- tls:
enabled: yes
- http:
enabled: yes
- stats:
enabled: yes
filename: stats.log
append: yes
totals: yes
- syslog:
enabled: no
facility: local5
level: Notice
I also have besides other the following warning:
@jonny5 Maybe if you know and have the time to explain a bit the parsing and duplicate errors from here?
Having those other types enabled will lead to a very noisy āeve.jsonā file
I would recommend you not set those to enabled, and instead enable in the app-layer - protocols area
See example:
outputs:
- fast:
enabled: yes
filename: fast.log
append: yes
- eve-log:
enabled: yes
filetype: regular
filename: eve.json
metadata: yes
pcap-file: false
community-id: true
community-id-seed: 0
types:
- alert:
payload: no
payload-printable: no
packet: no
http-body: no
http-body-printable: no
tagged-packets: yes
metadata:
app-layer: true
flow: true
rule:
metadata: true
raw: true
xff:
enabled: yes
mode: overwrite
deployment: reverse
header: X-Forwarded-For
- frame:
enabled: no
- anomaly:
enabled: no
types:
applayer: no
- stats:
enabled: yes
filename: stats.log
append: yes
totals: yes
app-layer:
protocols:
telnet:
enabled: yes
rfb:
enabled: yes
detection-ports:
dp: 5900, 5901, 5902, 5903, 5904, 5905, 5906, 5907, 5908, 5909
mqtt:
enabled: yes
krb5:
enabled: yes
bittorrent-dht:
enabled: yes
snmp:
enabled: yes
ike:
enabled: yes
tls:
enabled: yes
detection-ports:
dp: 443
ja3-fingerprints: auto
pgsql:
enabled: yes
stream-depth: 0
max-tx: 1024
dcerpc:
enabled: yes
ftp:
enabled: yes
rdp:
ssh:
enabled: yes
http2:
enabled: yes
smtp:
enabled: yes
raw-extraction: no
mime:
decode-mime: yes
decode-base64: yes
decode-quoted-printable: yes
header-value-depth: 2000
extract-urls: yes
body-md5: no
inspected-tracker:
content-limit: 100000
content-inspect-min-size: 32768
content-inspect-window: 4096
imap:
enabled: detection-only
smb:
enabled: yes
detection-ports:
dp: 139, 445
nfs:
enabled: yes
tftp:
enabled: yes
dns:
tcp:
enabled: yes
detection-ports:
dp: 53
udp:
enabled: yes
detection-ports:
dp: 53
http:
enabled: yes
libhtp:
default-config:
personality: IDS
request-body-limit: 100kb
response-body-limit: 100kb
request-body-minimal-inspect-size: 32kb
request-body-inspect-window: 4kb
response-body-minimal-inspect-size: 40kb
response-body-inspect-window: 16kb
response-body-decompress-layer-limit: 2
http-body-inline: auto
swf-decompression:
enabled: no
type: both
compress-depth: 100kb
decompress-depth: 100kb
double-decode-path: no
double-decode-query: no
server-config:
modbus:
enabled: yes
detection-ports:
dp: 502
stream-depth: 0
dnp3:
enabled: yes
detection-ports:
dp: 20000
enip:
enabled: yes
detection-ports:
dp: 44818
sp: 44818
ntp:
enabled: yes
quic:
enabled: yes
dhcp:
enabled: yes
sip:
enabled: yes
Some of the duplicated/parse error comes from Suricata not being Snort, as some rules are written for Snort and cannot be parsed by Suricata. There are other issues possibly, but those are much more in the space of what is actually performing your rule update cycle.
Are you using āsuricata-updateā or are you using Suricata as installed in an OPNSense and their Policy method of rule updates and modifications, or another rule management environment all together?
Thank you for the config, much appreciated. I pasted your config and seems to be working. Regarding the duplicate and parsing errors, I solved that. There are still the flow bit errors that I see in the logs.
I am using Suricata directly as it comes with the OPNSense install.
Happy to help!!
Possible issue about the custom.yaml file on the OPNSense!! By default on an update of Suricata or a full suricata.service restart, the OS/Distro will copy (but not really copy, there is some parsing/modification involved) the custom.yaml from the template area of the OPNSense and replace what is located at /usr/local/etc/suricata/custom.yaml and this is more or less undesirable as your modifications will disappear. If you try to edit the file at the template folder - it will break the service all together.
In the following how-to, thereās a lot covered but in one of the steps, but I explain how to make a shell script (which if you copied the suricataupdate.sh script and near removed all the extra would do what you seek) and in another step make new file that stays around and adds selectable Cron entries to the OPNSense Web Management GUI.
Configuring your script with a schedule can let you ākeepā your custom.yaml as desired - by if necessary, copying the correct one over and appropriately restarting Suricata:
^ The full guide does allow you to use āsuricata-updateā instead of OPNSenseās Policy mod/update method and I can say from experience, suricata-update is faster and more thorough.
To keep the ācustom.yamlā file that you want to update in the expected state, a better fix would be to figure out how/why it is getting reset and update the various Git Repos at Github for OPNSense, but I have not had time to get familiar enough with the full underbelly of OPNSense to do that.