Unable to generate alerts from et/pro signatures in Suricata 8.0.0

Please include the following information with your help request:

  • Suricata version: 7.0.10 and 8.0.0
  • Operating system and/or Linux distribution: Ubuntu 22
  • How you installed Suricata (from source, packages, something else): ppa:oisf/suricata-stable and ppa:oisf/suricata-beta

Hi! Recently I upgraded to the beta Suricata 8.0.0 version since I am interested in the output buffering feature introduced in the beta - but, i’m encountering some basic issues with getting alerts from the ET Pro ruleset to be triggered while in IDS mode. Through some trial and error, it seems like my 8.0.0 deployment will trigger alerts from the default Suricata rules (i.e. usr/share/suricata/rules/*), but not from any other rulesets, even through everything indicates they are loaded.

To test this I went back to the basics - i set up two identical VMs running Ubuntu 22. on VM 1, i ran:

sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt install suricata=1:7.0.10-0ubuntu1

on VM 2, i ran:

sudo add-apt-repository ppa:oisf/suricata-beta
sudo apt install suricata=1:8.0.0~beta1-0ubuntu29

then for each one, the only thing i touched in /etc/suricata/suricata.yaml was the interface for af-packet to match my interface (which has the same name on both VMs, enp6s18):

# Linux high speed capture support
af-packet:
  - interface: enp6s18

i left everything else as is. (i can provide my suricata.yaml files for both VMs)

then on both VMs, i used suricata-update to add the et/pro ruleset:

sudo suricata-update update-sources
sudo suricata-update enable-source et/pro secret-code=<code>

and finally ran suricata-update which produced the following successful output on both VMs:

VM 1 (running Suricata 7.0.10)
$ suricata-update
20/5/2025 -- 20:59:53 - <Info> -- Using data-directory /var/lib/suricata.
20/5/2025 -- 20:59:53 - <Info> -- Using Suricata configuration /etc/suricata/suricata.yaml
20/5/2025 -- 20:59:53 - <Info> -- Using /usr/share/suricata/rules for Suricata provided rules.
20/5/2025 -- 20:59:53 - <Info> -- Found Suricata version 7.0.10 at /usr/bin/suricata.
20/5/2025 -- 20:59:53 - <Info> -- Loading /etc/suricata/suricata.yaml
20/5/2025 -- 20:59:53 - <Info> -- Disabling rules for protocol pgsql
20/5/2025 -- 20:59:53 - <Info> -- Disabling rules for protocol modbus
20/5/2025 -- 20:59:53 - <Info> -- Disabling rules for protocol dnp3
20/5/2025 -- 20:59:53 - <Info> -- Disabling rules for protocol enip
20/5/2025 -- 20:59:53 - <Info> -- Fetching https://rules.emergingthreatspro.com/<secret-code>/suricata-7.0.10/etpro.rules.tar.gz.
100% - 10946876/10946876
20/5/2025 -- 20:59:55 - <Info> -- Done.
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/app-layer-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/decoder-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/dhcp-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/dnp3-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/dns-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/files.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/http2-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/http-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/ipsec-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/kerberos-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/modbus-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/mqtt-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/nfs-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/ntp-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/quic-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/rfb-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/smb-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/smtp-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/ssh-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/stream-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/tls-events.rules
20/5/2025 -- 20:59:55 - <Info> -- Ignoring file f994c46ff0cd7e588d6f6ed7616a2a09/rules/deleted.rules
20/5/2025 -- 20:59:59 - <Info> -- Loaded 106927 rules.
20/5/2025 -- 21:00:00 - <Info> -- Disabled 13 rules.
20/5/2025 -- 21:00:00 - <Info> -- Enabled 0 rules.
20/5/2025 -- 21:00:00 - <Info> -- Modified 0 rules.
20/5/2025 -- 21:00:00 - <Info> -- Dropped 0 rules.
20/5/2025 -- 21:00:00 - <Info> -- Enabled 193 rules for flowbit dependencies.
20/5/2025 -- 21:00:00 - <Info> -- Backing up current rules.
20/5/2025 -- 21:00:03 - <Info> -- Writing rules to /var/lib/suricata/rules/suricata.rules: total: 106927; enabled: 80436; added: 47757; removed 0; modified: 14
20/5/2025 -- 21:00:03 - <Info> -- Writing /var/lib/suricata/rules/classification.config
20/5/2025 -- 21:00:03 - <Info> -- Testing with suricata -T.
20/5/2025 -- 21:00:53 - <Info> -- Done.
VM 2 (running Suricata 8.0.0)
$ suricata-update
20/5/2025 -- 22:30:57 - <Info> -- Using data-directory /var/lib/suricata.
20/5/2025 -- 22:30:57 - <Info> -- Using Suricata configuration /etc/suricata/suricata.yaml
20/5/2025 -- 22:30:57 - <Info> -- Using /usr/share/suricata/rules for Suricata provided rules.
20/5/2025 -- 22:30:57 - <Info> -- Found Suricata version 8.0.0-beta1 at /usr/bin/suricata.
20/5/2025 -- 22:30:57 - <Info> -- Loading /etc/suricata/suricata.yaml
20/5/2025 -- 22:30:57 - <Info> -- Disabling rules for protocol pgsql
20/5/2025 -- 22:30:57 - <Info> -- Disabling rules for protocol modbus
20/5/2025 -- 22:30:57 - <Info> -- Disabling rules for protocol dnp3
20/5/2025 -- 22:30:57 - <Info> -- Disabling rules for protocol enip
20/5/2025 -- 22:30:57 - <Info> -- Fetching https://rules.emergingthreatspro.com/<secret-code>/suricata-8.0.0/etpro.rules.tar.gz.
 100% - 10946876/10946876             
20/5/2025 -- 22:31:02 - <Info> -- Done.
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/app-layer-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/decoder-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/dhcp-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/dnp3-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/dns-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/files.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/http2-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/http-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/ipsec-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/kerberos-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/modbus-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/mqtt-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/nfs-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/ntp-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/quic-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/rfb-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/smb-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/smtp-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/ssh-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/stream-events.rules
20/5/2025 -- 22:31:02 - <Info> -- Loading distribution rule file /usr/share/suricata/rules/tls-events.rules
20/5/2025 -- 22:31:03 - <Info> -- Ignoring file d23584343e24cba2b0f7929ce2d08557/rules/deleted.rules
20/5/2025 -- 22:31:06 - <Info> -- Loaded 106934 rules.
20/5/2025 -- 22:31:07 - <Info> -- Disabled 13 rules.
20/5/2025 -- 22:31:07 - <Info> -- Enabled 0 rules.
20/5/2025 -- 22:31:07 - <Info> -- Modified 0 rules.
20/5/2025 -- 22:31:07 - <Info> -- Dropped 0 rules.
20/5/2025 -- 22:31:07 - <Info> -- Enabled 193 rules for flowbit dependencies.
20/5/2025 -- 22:31:07 - <Info> -- Backing up current rules.
20/5/2025 -- 22:31:10 - <Info> -- Writing rules to /var/lib/suricata/rules/suricata.rules: total: 106934; enabled: 80443; added: 47757; removed 0; modified: 14
20/5/2025 -- 22:31:10 - <Info> -- Writing /var/lib/suricata/rules/classification.config
20/5/2025 -- 22:31:10 - <Info> -- Testing with suricata -T.
20/5/2025 -- 22:31:28 - <Info> -- Done.

then i started up Suricata on both VMs:

sudo systemctl enable suricata.service
sudo service suricata start
sudo service suricata status

and verified they both started up without errors (i can provide my suricata.log and suricata-start.log files for both VMs). in particular, i checked the rules were all loaded in correctly:

VM 1 (Suricata 7.0.10)
219360 - Suricata-Main] 2025-05-20 21:34:02 Config: detect: Loading rule file: /var/lib/suricata/rules/suricata.rules
[219360 - Suricata-Main] 2025-05-20 21:34:17 Info: detect: 1 rule files processed. 80436 rules successfully loaded, 0 rules failed, 0
[219360 - Suricata-Main] 2025-05-20 21:34:17 Info: threshold-config: Threshold config parsed: 0 rule(s) found
[219360 - Suricata-Main] 2025-05-20 21:34:17 Info: detect: 80439 signatures processed. 1222 are IP-only rules, 11465 are inspecting packet payload, 67527 inspect application layer, 109 are decoder event only

VM 2 (Suricata 8.0.0)
[5089 - Suricata-Main] 2025-05-20 21:33:34 Config: detect: Loading rule file: /var/lib/suricata/rules/suricata.rules
[5089 - Suricata-Main] 2025-05-20 21:33:50 Info: detect: 1 rule files processed. 80443 rules successfully loaded, 0 rules failed, 0 rules skipped
[5089 - Suricata-Main] 2025-05-20 21:33:50 Info: threshold-config: Threshold config parsed: 0 rule(s) found
[5089 - Suricata-Main] 2025-05-20 21:33:50 Info: detect: 80446 signatures processed. 1222 are IP-only rules, 11465 are inspecting packet payload, 67533 inspect application layer, 110 are decoder event only

then, following the quickstart guide, i tested each VM’s Suricata deployment by running curl http://testmynids.org/uid/index.html and opening fast.log.

In VM 1 (running 7.0.10) fast.log, i see the expected alert:

05/20/2025-21:45:20.552304  [**] [1:2100498:7] GPL ATTACK_RESPONSE id check returned root [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 18.155.192.31:80 -> <IP>:53192

but in VM 2 (running 8.0.0), the alert doesn’t show up at all.

I double checked that the signature is present in the rules file for both VMs:

VM 1 (Suricata 7.0.10)
$ cat /var/lib/suricata/rules/suricata.rules | grep "GPL ATTACK_RESPONSE id check returned root"
alert ip any any -> any any (msg:"GPL ATTACK_RESPONSE id check returned root"; content:"uid=0|28|root|29|"; classtype:bad-unknown; sid:2100498; rev:7; metadata:created_at 2010_09_23, confidence Medium, signature_severity Informational, updated_at 2019_07_26;)

VM 2 (Suricata 8.0.0)
$ cat /var/lib/suricata/rules/suricata.rules | grep "GPL ATTACK_RESPONSE id check returned root"
alert ip any any -> any any (msg:"GPL ATTACK_RESPONSE id check returned root"; content:"uid=0|28|root|29|"; classtype:bad-unknown; sid:2100498; rev:7; metadata:created_at 2010_09_23, confidence Medium, signature_severity Informational, updated_at 2019_07_26;)

I also double checked the output of suricatasc -c ruleset-stats:

VM 1 (Suricata 7.0.10)
$ suricatasc -c ruleset-stats
{"message": [{"id": 0, "rules_loaded": 80436, "rules_failed": 0, "rules_skipped": 0}], "return": "OK"}

VM 2 (Suricata 8.0.0)
$ suricatasc -c ruleset-stats
{"message":[{"id":0,"rules_loaded":80443,"rules_failed":0,"rules_skipped":0}],"return":"OK"}

I tried triggering a couple more types of signatures from the et/pro ruleset using a tREX server and observed the same behavior - it always produced an alert in VM 1 running 7.0.10, but never produced an alert in VM 2 running 8.0.0. Also, the packet counts in /var/log/suricata/stats.log did similarly increase in both VMs, i so i’m certain they were both successfully receiving the traffic in the same way (i confirmed this as well with tcpdump).

However, i was able to get VM 2 with Suricata 8.0.0 to trigger an alert when I sent a packet matching a signature from Suricata’s default rules - specifically, this signature from /usr/share/suricata/rules/decoder-events.rules:

alert pkthdr any any -> any any (msg:"SURICATA NULL pkt too small"; decode-event:ltnull.pkt_too_small; classtype:protocol-command-decode; sid:2200103; rev:2;)

from another machine, i ran hping3 -d 0 -c 1 -0 <VM IP> twice, once for each VM, and both showed the corresponding alert in their fast.log:

VM 1 (Suricata 7.0.10) fast.log:
05/20/2025-22:04:24.719954  [**] [1:2200033:2] SURICATA TCP packet too small [**] [Classification: Generic Protocol Command Decode] [Priority: 3] {TCP} <IP>:0 -> <VM IP>:0

VM 2 (Suricata 8.0.0) fast.log:
05/20/2025-22:04:24.719954  [**] [1:2200033:2] SURICATA TCP packet too small [**] [Classification: Generic Protocol Command Decode] [Priority: 3] {TCP} <IP>:0 -> <VM IP>:0

so, this pretty much points to it being an issue with trying to trigger signatures from non-default rulesets when using Suricata 8.0.0 (i ultimately tried using both et/pro and tgreen/hunting and got the same result). the behavior seems to pretty much be indicating that at some point i incorrectly configured and loaded these rulesets - but the logs and suricatasc output show everything as expected. I did also try running suricatasc -c ruleset-reload-rules and suricatasc -c reload-rules for good measure, but didn’t see any difference in the behavior.

I feel like this must be some silly mistake I’ve done or something I’ve overlooked, but after spending a few days debugging I’m totally stumped. I did take a look at the 8.0.0 changes described on the ‘Upgrading’ page in the docs, but didn’t catch anything that seemed related (i am pretty new to Suricata, so I definitely might’ve missed something). I checked whether there was anything related to suricata-update as well, but I confirmed in both VMs I am using the latest version of it (1.3.4).

thank you in advance for anyone willing to read through this and make any suggestions! I have spent the last few months learning about Suricata and tuning it for my use case, and have really enjoyed the learning process. i’m excited to continue the process and hopefully take advantage of the new features in v8!
(i will also try to upload the files i mentioned above as a comment - it seems I’m still too new of a user to do it in this initial post)

looks like for now I still can’t upload files as a new user. but I did want to also add the output from suricata --build-info:

VM 1 (7.0.10)
$ suricata --build-info
This is Suricata version 7.0.10 RELEASE
Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HTTP2_DECOMPRESSION HAVE_LUA HAVE_JA3 HAVE_JA4 HAVE_LUAJIT HAVE_LIBJANSSON TLS TLS_C11 MAGIC RUST POPCNT64 
SIMD support: SSE_2 
Atomic intrinsics: 1 2 4 8 byte(s)
64-bits, Little-endian architecture
GCC version 11.4.0, C version 201112
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.50, linked against LibHTP v0.5.50

Suricata Configuration:
  AF_PACKET support:                       yes
  AF_XDP support:                          no
  DPDK support:                            no
  eBPF support:                            no
  XDP support:                             no
  PF_RING support:                         no
  NFQueue support:                         yes
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no 
  DAG enabled:                             no
  Napatech enabled:                        no
  WinDivert enabled:                       no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libjansson support:                      yes
  hiredis support:                         yes
  hiredis async with libevent:             yes
  PCRE jit:                                yes
  LUA support:                             yes, through luajit
  libluajit:                               yes
  GeoIP2 support:                          yes
  JA3 support:                             yes
  JA4 support:                             yes
  Non-bundled htp:                         yes
  Hyperscan support:                       yes
  Libnet support:                          yes
  liblz4 support:                          yes
  Landlock support:                        yes

  Rust support:                            yes
  Rust strict mode:                        no
  Rust compiler path:                      /usr/bin/rustc
  Rust compiler version:                   rustc 1.75.0 (82e1608df 2023-12-21) (built from a source tarball)
  Cargo path:                              /usr/bin/cargo
  Cargo version:                           cargo 1.75.0

  Python support:                          yes
  Python path:                             /usr/bin/python3
  Install suricatactl:                     yes
  Install suricatasc:                      yes
  Install suricata-update:                 yes

  Profiling enabled:                       no
  Profiling locks enabled:                 no
  Profiling rules enabled:                 no

  Plugin support (experimental):           yes
  DPDK Bond PMD:                           no

Development settings:
  Coccinelle / spatch:                     no
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no
  Fuzz targets enabled:                    no

Generic build parameters:
  Installation prefix:                     /usr
  Configuration directory:                 /etc/suricata/
  Log directory:                           /var/log/suricata/

  --prefix                                 /usr
  --sysconfdir                             /etc
  --localstatedir                          /var
  --datarootdir                            /usr/share

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / g++ (real)
  GCC Protect enabled:                     yes
  GCC march native enabled:                no
  GCC Profile enabled:                     no
  Position Independent Executable enabled: yes
  CFLAGS                                   -g -O2 -ffile-prefix-map=/build/suricata-y2IwT9/suricata-7.0.10=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC -std=c11 -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist
  PCAP_CFLAGS                               -I/usr/include
  SECCFLAGS                                -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security
VM 2 (8.0.0)
$ suricata --build-info
This is Suricata version 8.0.0-beta1 RELEASE
Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HTTP2_DECOMPRESSION HAVE_LUA HAVE_JA3 HAVE_JA4 HAVE_LIBJANSSON TLS TLS_C11 MAGIC RUST POPCNT64 
SIMD support: SSE_2 
Atomic intrinsics: 1 2 4 8 byte(s)
64-bits, Little-endian architecture
GCC version 11.4.0, C version 201112
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v2.0.0

Suricata Configuration:
  AF_PACKET support:                       yes
  AF_XDP support:                          no
  DPDK support:                            no
  eBPF support:                            no
  XDP support:                             no
  PF_RING support:                         no
  NFQueue support:                         yes
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no 
  DAG enabled:                             no
  Napatech enabled:                        no
  WinDivert enabled:                       no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libjansson support:                      yes
  hiredis support:                         yes
  hiredis async with libevent:             yes
  PCRE jit:                                yes
  GeoIP2 support:                          yes
  JA3 support:                             yes
  JA4 support:                             yes
  Hyperscan support:                       yes
  Libnet support:                          yes
  liblz4 support:                          yes
  Landlock support:                        yes
  Systemd support:                         yes

  Rust strict mode:                        no
  Rust compiler path:                      /usr/bin/rustc
  Rust compiler version:                   rustc 1.75.0 (82e1608df 2023-12-21) (built from a source tarball)
  Cargo path:                              /usr/bin/cargo
  Cargo version:                           cargo 1.75.0

  Python support:                          yes
  Python path:                             /usr/bin/python3
  Install suricatactl:                     yes
  Install suricatasc:                      yes
  Install suricata-update:                 yes

  Profiling enabled:                       no
  Profiling locks enabled:                 no
  Profiling rules enabled:                 no

  Plugin support (experimental):           yes
  DPDK Bond PMD:                           no

Plugins:
  nDPI:                                    no

Development settings:
  Coccinelle / spatch:                     no
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no
  Fuzz targets enabled:                    no

Generic build parameters:
  Installation prefix:                     /usr
  Configuration directory:                 /etc/suricata/
  Log directory:                           /var/log/suricata/

  --prefix                                 /usr
  --sysconfdir                             /etc
  --localstatedir                          /var
  --datarootdir                            /usr/share

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / g++ (real)
  GCC Protect enabled:                     yes
  GCC march native enabled:                no
  GCC Profile enabled:                     no
  Position Independent Executable enabled: yes
  CFLAGS                                   -g -O2 -ffile-prefix-map=/build/suricata-DPDT3R/suricata-8.0.0~beta1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DOS_LINUX -std=c11 -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist -I../rust/gen
  PCAP_CFLAGS                              
  SECCFLAGS                                -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security

Hi,
Thanks for the detailed writeup.

In general, there should be no differences at this level between 7.0.x releases and 8.0 pre-releases .

Does the 8.0 VM show more drops than the other (see stats.log or the records with event_type stats in eve.json)?

Are you getting the same results when scanning a PCAP instead of real traffic?
(I tried reproducing it myself but got a segfault. :grin:)

et_nids_test.pcapng (2.0 KB)

hi Jeff! thanks for the reply, this morning i started up suricata in both VMs and let them run for a little over an hour without trying to trigger any alerts. all the packets it got were just whatever background stuff is happening, and from a tcpdump looks identical on both VMs (looks like mostly pings between the IP gateway and the VMs). here are the final reports on the stats.log file for each VM:

VM 1 (7.0.10)
Date: 5/21/2025 -- 20:15:07 (uptime: 0d, 01h 12m 36s)
------------------------------------------------------------------------------------
Counter                                       | TM Name                   | Value
------------------------------------------------------------------------------------
capture.kernel_packets                        | Total                     | 7058
capture.afpacket.polls                        | Total                     | 1379647
capture.afpacket.poll_timeout                 | Total                     | 1372715
capture.afpacket.poll_data                    | Total                     | 6900
decoder.pkts                                  | Total                     | 7058
decoder.bytes                                 | Total                     | 667101
decoder.ipv4                                  | Total                     | 537
decoder.ipv6                                  | Total                     | 52
decoder.ethernet                              | Total                     | 7058
decoder.arp                                   | Total                     | 4208
decoder.unknown_ethertype                     | Total                     | 2261
decoder.tcp                                   | Total                     | 512
tcp.syn                                       | Total                     | 2
tcp.synack                                    | Total                     | 3
tcp.rst                                       | Total                     | 3
decoder.udp                                   | Total                     | 25
decoder.icmpv6                                | Total                     | 52
decoder.avg_pkt_size                          | Total                     | 94
decoder.max_pkt_size                          | Total                     | 2962
flow.total                                    | Total                     | 57
flow.tcp                                      | Total                     | 4
flow.udp                                      | Total                     | 21
flow.icmpv6                                   | Total                     | 32
flow.wrk.spare_sync_avg                       | Total                     | 100
flow.wrk.spare_sync                           | Total                     | 26
flow.wrk.flows_evicted_needs_work             | Total                     | 2
flow.wrk.flows_evicted_pkt_inject             | Total                     | 3
flow.wrk.flows_injected                       | Total                     | 2
tcp.sessions                                  | Total                     | 2
tcp.ssn_from_pool                             | Total                     | 2
tcp.pseudo                                    | Total                     | 2
tcp.segment_from_pool                         | Total                     | 18
app_layer.flow.tls                            | Total                     | 1
app_layer.flow.ssh                            | Total                     | 1
app_layer.flow.ntp                            | Total                     | 2
app_layer.tx.ntp                              | Total                     | 2
app_layer.flow.dns_udp                        | Total                     | 2
app_layer.tx.dns_udp                          | Total                     | 4
app_layer.flow.failed_udp                     | Total                     | 17
flow.end.state.new                            | Total                     | 51
flow.end.state.established                    | Total                     | 5
flow.end.state.closed                         | Total                     | 1
flow.end.tcp_state.established                | Total                     | 1
flow.end.tcp_state.closed                     | Total                     | 1
flow.mgr.full_hash_pass                       | Total                     | 1271
flow.mgr.rows_per_sec                         | Total                     | 19005
flow.spare                                    | Total                     | 9652
flow.mgr.rows_maxlen                          | Total                     | 1
flow.mgr.flows_checked                        | Total                     | 134
flow.mgr.flows_notimeout                      | Total                     | 82
flow.mgr.flows_timeout                        | Total                     | 52
flow.mgr.flows_evicted                        | Total                     | 52
memcap_pressure                               | Total                     | 29
memcap_pressure_max                           | Total                     | 29
flow.recycler.recycled                        | Total                     | 55
flow.recycler.queue_max                       | Total                     | 3
tcp.memuse                                    | Total                     | 19922944
tcp.reassembly_memuse                         | Total                     | 3670016
flow.memuse                                   | Total                     | 7805504
VM2 (8.0.0)
Date: 5/21/2025 -- 20:15:17 (uptime: 0d, 01h 12m 43s)
---------------------------------------------------------------------------------------------------
Counter                                                      | TM Name                   | Value
---------------------------------------------------------------------------------------------------
capture.kernel_packets                                       | Total                     | 7320
capture.kernel_drops                                         | Total                     | 5
capture.afpacket.polls                                       | Total                     | 1391143
capture.afpacket.poll_timeout                                | Total                     | 1384373
capture.afpacket.poll_data                                   | Total                     | 6738
decoder.pkts                                                 | Total                     | 7315
decoder.bytes                                                | Total                     | 751030
decoder.ipv4                                                 | Total                     | 734
decoder.ipv6                                                 | Total                     | 52
decoder.ethernet                                             | Total                     | 7315
decoder.arp                                                  | Total                     | 4247
decoder.unknown_ethertype                                    | Total                     | 2282
decoder.tcp                                                  | Total                     | 707
tcp.syn                                                      | Total                     | 2
tcp.synack                                                   | Total                     | 3
tcp.rst                                                      | Total                     | 2
decoder.udp                                                  | Total                     | 27
decoder.icmpv6                                               | Total                     | 52
decoder.avg_pkt_size                                         | Total                     | 102
decoder.max_pkt_size                                         | Total                     | 2962
flow.total                                                   | Total                     | 59
flow.tcp                                                     | Total                     | 4
flow.udp                                                     | Total                     | 23
flow.icmpv6                                                  | Total                     | 32
flow.wrk.spare_sync_avg                                      | Total                     | 100
flow.wrk.spare_sync                                          | Total                     | 23
decoder.event.ethernet.unknown_ethertype                     | Total                     | 2282
flow.wrk.flows_evicted_needs_work                            | Total                     | 2
flow.wrk.flows_evicted_pkt_inject                            | Total                     | 3
flow.wrk.flows_injected                                      | Total                     | 2
tcp.sessions                                                 | Total                     | 2
tcp.ssn_from_pool                                            | Total                     | 2
tcp.pseudo                                                   | Total                     | 2
tcp.ack_unseen_data                                          | Total                     | 2
tcp.segment_from_pool                                        | Total                     | 127
detect.alert                                                 | Total                     | 2282
app_layer.flow.tls                                           | Total                     | 1
app_layer.flow.ssh                                           | Total                     | 1
app_layer.flow.ntp                                           | Total                     | 2
app_layer.tx.ntp                                             | Total                     | 2
app_layer.flow.failed_udp                                    | Total                     | 19
app_layer.flow.dns_udp                                       | Total                     | 2
app_layer.tx.dns_udp                                         | Total                     | 4
flow.end.state.new                                           | Total                     | 53
flow.end.state.established                                   | Total                     | 5
flow.end.state.closed                                        | Total                     | 1
flow.end.tcp_state.established                               | Total                     | 1
flow.end.tcp_state.closed                                    | Total                     | 1
flow.mgr.full_hash_pass                                      | Total                     | 1282
flow.mgr.rows_per_sec                                        | Total                     | 19005
flow.spare                                                   | Total                     | 9956
flow.mgr.rows_maxlen                                         | Total                     | 1
flow.mgr.flows_checked                                       | Total                     | 137
flow.mgr.flows_notimeout                                     | Total                     | 80
flow.mgr.flows_timeout                                       | Total                     | 57
flow.mgr.flows_evicted                                       | Total                     | 57
flow.mgr.flows_evicted_needs_work                            | Total                     | 1
memcap.pressure                                              | Total                     | 29
memcap.pressure_max                                          | Total                     | 29
defrag.memuse                                                | Total                     | 33554432
flow.recycler.recycled                                       | Total                     | 57
flow.recycler.queue_max                                      | Total                     | 2
tcp.memuse                                                   | Total                     | 19922944
tcp.reassembly_memuse                                        | Total                     | 3670016
http.byterange.memuse                                        | Total                     | 168384
http.byterange.memcap                                        | Total                     | 104857600
ippair.memuse                                                | Total                     | 398144
ippair.memcap                                                | Total                     | 398144
host.memuse                                                  | Total                     | 382144
host.memcap                                                  | Total                     | 33554432
flow.memuse                                                  | Total                     | 7805504

seems like VM 1 (7.0.10) had 0 drops, while VM 2 (8.0.0) had 5, not sure what those could be due to.

after shutting down Suricata on both VMs and rerunning suricata-update on both, i repeated this process. except this time, on both VMs a couple minutes after starting Suricata, i ran curl http://testmynids.org/uid/index.html 15 times. i tailed the stats.log files for a few minutes to see if the drop count on VM 2 would jump up, but didn’t see any immediate change. i let it continue to sit for ~1 hour 12 min, here are those final stats report for that:

VM1 (7.0.10)
Date: 5/21/2025 -- 21:36:03 (uptime: 0d, 01h 12m 58s)
------------------------------------------------------------------------------------
Counter                                       | TM Name                   | Value
------------------------------------------------------------------------------------
capture.kernel_packets                        | Total                     | 10272
capture.afpacket.polls                        | Total                     | 1387406
capture.afpacket.poll_timeout                 | Total                     | 1378666
capture.afpacket.poll_data                    | Total                     | 8740
decoder.pkts                                  | Total                     | 10273
decoder.bytes                                 | Total                     | 3196914
decoder.ipv4                                  | Total                     | 3717
decoder.ipv6                                  | Total                     | 50
decoder.ethernet                              | Total                     | 10273
decoder.arp                                   | Total                     | 4232
decoder.unknown_ethertype                     | Total                     | 2274
decoder.tcp                                   | Total                     | 3691
tcp.syn                                       | Total                     | 18
tcp.synack                                    | Total                     | 19
tcp.rst                                       | Total                     | 1
decoder.udp                                   | Total                     | 26
decoder.icmpv6                                | Total                     | 50
decoder.avg_pkt_size                          | Total                     | 311
decoder.max_pkt_size                          | Total                     | 1514
flow.total                                    | Total                     | 80
flow.active                                   | Total                     | 1
flow.tcp                                      | Total                     | 30
flow.udp                                      | Total                     | 20
flow.icmpv6                                   | Total                     | 30
flow.wrk.spare_sync_avg                       | Total                     | 100
flow.wrk.spare_sync                           | Total                     | 28
tcp.sessions                                  | Total                     | 18
tcp.ssn_from_pool                             | Total                     | 18
tcp.segment_from_pool                         | Total                     | 53
detect.alert                                  | Total                     | 18
app_layer.flow.http                           | Total                     | 18
app_layer.tx.http                             | Total                     | 18
app_layer.flow.ntp                            | Total                     | 2
app_layer.tx.ntp                              | Total                     | 2
app_layer.flow.dns_udp                        | Total                     | 4
app_layer.tx.dns_udp                          | Total                     | 8
app_layer.flow.failed_udp                     | Total                     | 14
flow.end.state.new                            | Total                     | 55
flow.end.state.established                    | Total                     | 6
flow.end.state.closed                         | Total                     | 18
flow.end.tcp_state.closed                     | Total                     | 18
flow.mgr.full_hash_pass                       | Total                     | 1276
flow.mgr.rows_per_sec                         | Total                     | 19005
flow.spare                                    | Total                     | 9479
flow.mgr.rows_maxlen                          | Total                     | 1
flow.mgr.flows_checked                        | Total                     | 267
flow.mgr.flows_notimeout                      | Total                     | 188
flow.mgr.flows_timeout                        | Total                     | 79
flow.mgr.flows_evicted                        | Total                     | 79
memcap_pressure                               | Total                     | 29
memcap_pressure_max                           | Total                     | 29
flow.recycler.recycled                        | Total                     | 79
flow.recycler.queue_max                       | Total                     | 3
tcp.memuse                                    | Total                     | 19922944
tcp.reassembly_memuse                         | Total                     | 3670016
flow.memuse                                   | Total                     | 7805504
VM2 (8.0.0)
Date: 5/21/2025 -- 21:35:25 (uptime: 0d, 01h 12m 18s)
---------------------------------------------------------------------------------------------------
Counter                                                      | TM Name                   | Value
---------------------------------------------------------------------------------------------------
capture.kernel_packets                                       | Total                     | 10895
capture.kernel_drops                                         | Total                     | 27
capture.afpacket.polls                                       | Total                     | 1383702
capture.afpacket.poll_timeout                                | Total                     | 1376345
capture.afpacket.poll_data                                   | Total                     | 7357
decoder.pkts                                                 | Total                     | 10868
decoder.bytes                                                | Total                     | 4173949
decoder.ipv4                                                 | Total                     | 4317
decoder.ipv6                                                 | Total                     | 50
decoder.ethernet                                             | Total                     | 10868
decoder.arp                                                  | Total                     | 4231
decoder.unknown_ethertype                                    | Total                     | 2270
decoder.tcp                                                  | Total                     | 4285
tcp.syn                                                      | Total                     | 21
tcp.synack                                                   | Total                     | 22
tcp.rst                                                      | Total                     | 1
decoder.udp                                                  | Total                     | 32
decoder.icmpv6                                               | Total                     | 50
decoder.avg_pkt_size                                         | Total                     | 384
decoder.max_pkt_size                                         | Total                     | 1514
flow.total                                                   | Total                     | 86
flow.active                                                  | Total                     | 2
flow.tcp                                                     | Total                     | 33
flow.udp                                                     | Total                     | 23
flow.icmpv6                                                  | Total                     | 30
flow.wrk.spare_sync_avg                                      | Total                     | 100
flow.wrk.spare_sync                                          | Total                     | 31
decoder.event.ethernet.unknown_ethertype                     | Total                     | 2270
tcp.sessions                                                 | Total                     | 21
tcp.ssn_from_pool                                            | Total                     | 21
tcp.segment_from_pool                                        | Total                     | 58
detect.alert                                                 | Total                     | 2270
app_layer.flow.http                                          | Total                     | 21
app_layer.tx.http                                            | Total                     | 21
app_layer.flow.ntp                                           | Total                     | 3
app_layer.tx.ntp                                             | Total                     | 3
app_layer.flow.failed_udp                                    | Total                     | 14
app_layer.flow.dns_udp                                       | Total                     | 6
app_layer.tx.dns_udp                                         | Total                     | 12
flow.end.state.new                                           | Total                     | 55
flow.end.state.established                                   | Total                     | 8
flow.end.state.closed                                        | Total                     | 21
flow.end.tcp_state.closed                                    | Total                     | 21
flow.mgr.full_hash_pass                                      | Total                     | 1273
flow.mgr.rows_per_sec                                        | Total                     | 19005
flow.spare                                                   | Total                     | 9184
flow.mgr.rows_maxlen                                         | Total                     | 1
flow.mgr.flows_checked                                       | Total                     | 263
flow.mgr.flows_notimeout                                     | Total                     | 179
flow.mgr.flows_timeout                                       | Total                     | 84
flow.mgr.flows_evicted                                       | Total                     | 84
memcap.pressure                                              | Total                     | 29
memcap.pressure_max                                          | Total                     | 29
defrag.memuse                                                | Total                     | 33554432
flow.recycler.recycled                                       | Total                     | 84
flow.recycler.queue_max                                      | Total                     | 4
tcp.memuse                                                   | Total                     | 19922944
tcp.reassembly_memuse                                        | Total                     | 3670016
http.byterange.memuse                                        | Total                     | 168384
http.byterange.memcap                                        | Total                     | 104857600
ippair.memuse                                                | Total                     | 398144
ippair.memcap                                                | Total                     | 398144
host.memuse                                                  | Total                     | 382144
host.memcap                                                  | Total                     | 33554432
flow.memuse                                                  | Total                     | 7805504

so it does seem like the 8.0.0 deployment does consistently have a few drops, whereas the 7.0.10 has none at all. both VMs are well equipped with 32 CPUs and 64 GiB of memory, so i’m not sure what the small number of drops could be - maybe just better filtering of invalid packets?

hi! i have not tried that yet but that would be interesting to test - i’ll give it a shot and see what happens!

also it seems like now I have the ability to upload files here! in case its useful at all, here they are - the ones prepended with 7 are from VM 1 running 7.0.10, and the ones prepended with 8 are the ones from VM 2 running 8.0.0
7_suricata.log (38.6 KB)
7_suricata.yaml (84.9 KB)
8_suricata.log (45.7 KB)
8_suricata.yaml (89.5 KB)

Please show me the command line used to invoke Suricata on each VM.

i’ve been running the same set of commands on both VMs to start them up:

VM 1 (7.0.10)

sudo suricata-update
sudo systemctl enable suricata.service
sudo service suricata start
VM 2 (8.0.0)

sudo suricata-update
sudo systemctl enable suricata.service
sudo service suricata start

Thanks … I am after the specific command line instructions contained in the systemd configuration file for Suricata.

Your issue doesn’t seem to perfectly match the problem I had, but I think this deserves to be mentioned: Fast.log isn't updated when Suricata is running

It’s just about the feature you want. :slightly_smiling_face: And it exactly what that can produce a strange behavior if you don’t put buffer-size: 0 in the fast: section; at least for testing purpose.

OH sorry my bad! I think I understand now. here are the outputs of sudo ps -ef | grep suricata:

VM 1 (7.0.10)
root      235844       1 16 02:46 ?        00:00:21 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -vvv
VM 2 (8.0.0)
root       13558       1  2 02:30 ?        00:00:28 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -vvv

and here are the outputs of sudo service suricata status:

VM 1 (7.0.10)
● suricata.service - LSB: Next Generation IDS/IPS
     Loaded: loaded (/etc/init.d/suricata; generated)
     Active: active (running) since Sat 2025-05-24 02:46:48 UTC; 1min 23s ago
       Docs: man:systemd-sysv-generator(8)
    Process: 235835 ExecStart=/etc/init.d/suricata start (code=exited, status=0/SUCCESS)
      Tasks: 38 (limit: 77020)
     Memory: 560.8M
        CPU: 20.898s
     CGroup: /system.slice/suricata.service
             └─235844 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -vvv

May 24 02:46:48 vcs-ams-suricata0101 systemd[1]: Starting LSB: Next Generation IDS/IPS...
May 24 02:46:48 vcs-ams-suricata0101 suricata[235835]: Likely stale PID 235610 with /var/run/suricata.pid exists, but process is not running!
May 24 02:46:48 vcs-ams-suricata0101 suricata[235835]: Removing stale PID file /var/run/suricata.pid
May 24 02:46:48 vcs-ams-suricata0101 suricata[235835]: Starting suricata in IDS (af-packet) mode... done.
May 24 02:46:48 vcs-ams-suricata0101 systemd[1]: Started LSB: Next Generation IDS/IPS.
VM 2 (8.0.0)
● suricata.service - LSB: Next Generation IDS/IPS
     Loaded: loaded (/etc/init.d/suricata; generated)
     Active: active (running) since Sat 2025-05-24 02:30:19 UTC; 3min 22s ago
       Docs: man:systemd-sysv-generator(8)
    Process: 13552 ExecStart=/etc/init.d/suricata start (code=exited, status=0/SUCCESS)
      Tasks: 38 (limit: 150805)
     Memory: 448.0M
        CPU: 12.106s
     CGroup: /system.slice/suricata.service
             └─13558 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -vvv

May 24 02:30:19 vcs-ams-suricata101 systemd[1]: Starting LSB: Next Generation IDS/IPS...
May 24 02:30:19 vcs-ams-suricata101 suricata[13552]: Starting suricata in IDS (af-packet) mode... done.
May 24 02:30:19 vcs-ams-suricata101 systemd[1]: Started LSB: Next Generation IDS/IPS.

so they seem to look the same. out of curiosity, i did try a diff between the /etc/init.d/suricata files for each VM but that didn’t show any differences

ah interesting! yeah I think what i’m experiencing is a little different. first, my et/pro alerts write to neither fast.log nor eve.json - i see in your thread you mention the alerts do show up in eve.json . second, i am able to get alerts to show up in fast.log and eve.json as long as they are from Suricata’s default rulesets, and not an externally loaded ruleset like et/pro or tgreen/hunting. since those alerts do show up, i wouldn’t expect a buffer to be causing the problem. but i will give it a try anyways, definitely doesn’t hurt to test things out and see what happens!