According to the suricata stats log, the kernel_drops count is larger than the kernel_packets count, which is very confusing:
# tail -n 1 /opt/isd/data/suricata/stats.json | python3 -c 'import sys,json; print(json.load(sys.stdin)["stats"]["capture"])'
{'kernel_packets': 121843217, 'kernel_drops': 207464408, 'bypassed': 0}
Admittedly the machine is likely underpowered for the amount of traffic it receives (we’re in the process of acquiring a more powerful device) but these numbers don’t make sense. Does anyone have ideas of what could be causing this?
We are using pf_ring and listening to a teamed interface. We normally see about 2.5 - 4gbps traffic through this interface.
OS: Rocky Linux
Suricata build info:
# suricata --build-info
This is Suricata version 6.0.5 RELEASE
Features: PCAP_SET_BUFF PF_RING AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LIBJANSSON PROFILING TLS TLS_C11 MAGIC RUST
SIMD support: SSE_4_2 SSE_4_1 SSE_3
Atomic intrinsics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 8.5.0 20210514 (Red Hat 8.5.0-4), C version 201112
compiled with _FORTIFY_SOURCE=0
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.40, linked against LibHTP v0.5.40
Suricata Configuration:
AF_PACKET support: yes
eBPF support: no
XDP support: no
PF_RING support: yes
NFQueue support: no
NFLOG support: no
IPFW support: no
Netmap support: no
DAG enabled: no
Napatech enabled: no
WinDivert enabled: no
Unix socket enabled: yes
Detection enabled: yes
Libmagic support: yes
libnss support: yes
libnspr support: yes
libjansson support: yes
hiredis support: no
hiredis async with libevent: no
Prelude support: no
PCRE jit: yes
LUA support: no
libluajit: no
GeoIP2 support: no
Non-bundled htp: no
Hyperscan support: no
Libnet support: no
liblz4 support: yes
HTTP2 decompression: no
Rust support: yes
Rust strict mode: no
Rust compiler path: /usr/bin/rustc
Rust compiler version: rustc 1.54.0 (Red Hat 1.54.0-3.module+el8.5.0+13074+d655d86c)
Cargo path: /usr/bin/cargo
Cargo version: cargo 1.54.0
Cargo vendor: yes
Python support: yes
Python path: /usr/bin/python3
Python distutils yes
Python yaml yes
Install suricatactl: yes
Install suricatasc: yes
Install suricata-update: not bundled
Profiling enabled: yes
Profiling locks enabled: no
Plugin support (experimental): yes
Development settings:
Coccinelle / spatch: no
Unit tests enabled: no
Debug output enabled: no
Debug validation enabled: no
Generic build parameters:
Installation prefix: /usr
Configuration directory: /etc/suricata/
Log directory: /var/log/suricata/
--prefix /usr
--sysconfdir /etc
--localstatedir /var
--datarootdir /usr/share
Host: x86_64-pc-linux-gnu
Compiler: gcc (exec name) / gcc (real)
GCC Protect enabled: no
GCC march native enabled: yes
GCC Profile enabled: no
Position Independent Executable enabled: no
CFLAGS -D_GNU_SOURCE -std=c11 -march=native -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist
PCAP_CFLAGS
SECCFLAGS
lscpu:
# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 12
On-line CPU(s) list: 0-11
Thread(s) per core: 1
Core(s) per socket: 6
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
BIOS Vendor ID: Intel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Bronze 3204 CPU @ 1.90GHz
BIOS Model name: Intel(R) Xeon(R) Bronze 3204 CPU @ 1.90GHz
Stepping: 7
CPU MHz: 1900.000
BogoMIPS: 3800.00
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 8448K
NUMA node0 CPU(s): 0,2,4,6,8,10
NUMA node1 CPU(s): 1,3,5,7,9,11
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm arat pln pts pku ospke avx512_vnni md_clear flush_l1d arch_capabilities
And I’ve attached the suricata.yaml file (I’ve ommitted the address and port groups - rest assured in the actual configuration those are present)
suricata.yaml (10.4 KB)
Let me know if I can provide any other information that could help. Thanks so much!