System
Red Hat Enterprise Linux release 8.7 (Ootpa) 4.18.0-425.13.1.el8_7.x86_64
SElinux enforcing but permissive setting does not help suricatsc
Suricata version 7.0.0-rc2-dev (6487c689f 2023-05-07)
suricata --build-info
This is Suricata version 7.0.0-rc2-dev (6487c689f 2023-05-07)
Features: PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HTTP2_DECOMPRESSION HAVE_LIBJANSSON TLS TLS_C11 MAGIC RUST
SIMD support: SSE_4_2 SSE_4_1 SSE_3
Atomic intrinsics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 8.5.0 20210514 (Red Hat 8.5.0-16), C version 201112
compiled with -fstack-protector
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.43, linked against LibHTP v0.5.42
Suricata Configuration:
AF_PACKET support: yes
AF_XDP support: no
DPDK support: yes
eBPF support: no
XDP support: no
PF_RING support: no
NFQueue support: no
NFLOG support: no
IPFW support: no
Netmap support: no
DAG enabled: no
Napatech enabled: no
WinDivert enabled: no
Unix socket enabled: yes
Detection enabled: yes
Libmagic support: yes
libjansson support: yes
hiredis support: no
hiredis async with libevent: no
PCRE jit: yes
LUA support: no
libluajit: no
GeoIP2 support: yes
Non-bundled htp: no
Hyperscan support: yes
Libnet support: no
liblz4 support: yes
Landlock support: no
Based on log output, can you check if perhaps a previous reload is in fact not complete? I suppose this could happen if some of the dpdk interface have no traffic and our dpdk is missing some logic
Yep, every night suricata-update runs and will reload in case of changes. And indeed, 1 out of 4 dpdk interfaces receives traffic, testing in progress. So better only use dpdk interfaces with traffic? If so, I also need to check cpu affinity after modifying the dpdk config in suricata.
Ok, just started suricata service with extra options -vvv , expect suricata-update rond 2am to succeed but a manual reload tomorrow during working hours to fail with an error 1. I’ll be back.
Ok think I’ve found a possible lead in logrotate. suricata-update runs at 2am, all good but around 03:12am suricata still runs but eve.json files are not updating anymore.
And indeed suricatsc just now:
reload-rules
Error:
“Reload already in progress”
This logrote.conf worked for suricata v6, but shreds suricata 7 ?
/var/log/suricata/.log /data/sensor_data/suricata/eve..json /data/sensor_data/suricata/stats.log
{
daily
missingok
rotate 2
compress
minsize 500k
sharedscripts
postrotate
/bin/kill -HUP cat /var/run/suricata.pid 2> /dev/null 2> /dev/null || true
endscript
}
Maybe a clue overhere when restarting suricata via systemd?
May 11 13:16:23 systemd[1]: Stopping Suricata Intrusion Detection Service… – Subject: Unit suricata.service has begun shutting down – Defined-By: systemd – Support: https://access.redhat.com/support
**-- ** – Unit suricata.service has begun shutting down. May 11 13:17:53 systemd[1]: suricata.service: State ‘stop-sigterm’ timed out. Killing. May 11 13:17:53 systemd[1]: suricata.service: Killing process 1755717 (Suricata-Main) with signal SIGKILL. May 11 13:17:57 systemd[1]: suricata.service: Main process exited, code=killed, status=9/KILL May 11 13:17:57 systemd[1]: suricata.service: Failed with result ‘timeout’. – Subject: Unit failed – Defined-By: systemd – Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal
**-- ** – The unit suricata.service has entered the ‘failed’ state with result ‘timeout’. May 11 13:17:57 systemd[1]: Stopped Suricata Intrusion Detection Service. – Subject: Unit suricata.service has finished shutting down – Defined-By: systemd – Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal
**-- ** – Unit suricata.service has finished shutting down. May 11 13:46:57 systemd[1]: Starting Suricata Intrusion Detection Service… – Subject: Unit suricata.service has begun start-up – Defined-By: systemd – Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal
**-- ** – Unit suricata.service has begun starting up. May 11 13:46:57 systemd[1]: Started Suricata Intrusion Detection Service. – Subject: Unit suricata.service has finished start-up – Defined-By: systemd – Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal
**-- ** – Unit suricata.service has finished starting up.
Yep i know, thanks, but in my opinion it does not show a clue but but more retrospective about that point I’m not a developper I’ve attached a tar archive with logging (tar cjf) surlogs.tgz (36.3 KB)