I am getting some kernel_drops after some time.
I already optimized suricata.yaml on af-packet section, stream memcap and cpu-affinity. Can´t identify the problem…
There can be many reasons for that, so please provide us with more information about your setup:
- Suricata Version
- OS
- Suricata Configuration
- Hardware
etc.
-
Suricata 6.0.1
-
Ubuntu server 18.04
System: Host: suricata Kernel: 4.15.0-76-generic x86_64 bits: 64
Console: tty 0 Distro: Ubuntu 18.04.4 LTS
Machine: Device: server System: HP product: ProLiant DL380 G6 serial: CZC9410PQM
Mobo: N/A model: N/A serial: N/A
BIOS: HP v: P62 date: 07/24/2009
CPU(s): 2 Quad core Intel Xeon E5520s (-MT-MCP-SMP-)
cache: 16384 KB
clock speeds: max: 2266 MHz 1: 1726 MHz 2: 1865 MHz
3: 1766 MHz 4: 1962 MHz 5: 1788 MHz 6: 1958 MHz 7: 1782 MHz
8: 1950 MHz 9: 1790 MHz 10: 1933 MHz 11: 1784 MHz
12: 1975 MHz 13: 1789 MHz 14: 1944 MHz 15: 1797 MHz
16: 1955 MHz
Graphics: Card: Advanced Micro Devices [AMD/ATI] ES1000
Display Server: N/A driver: radeon
tty size: 73x54 Advanced Data: N/A for root out of X
Network: Card-1: Broadcom and subsidiaries NetXtreme II BCM5709 Gigabit Ethernet
driver: bnx2
IF: enp2s0f0 state: up speed: 1000 Mbps duplex: full
mac: 00:26:55:4a:76:52
Card-2: Broadcom and subsidiaries NetXtreme II BCM5709 Gigabit Ethernet
driver: bnx2
IF: enp2s0f1 state: up speed: 1000 Mbps duplex: full
mac: 00:26:55:4a:76:54
Card-3: Broadcom and subsidiaries NetXtreme II BCM5709 Gigabit Ethernet
driver: bnx2
IF: enp3s0f0 state: down mac: 00:26:55:4a:76:56
Card-4: Broadcom and subsidiaries NetXtreme II BCM5709 Gigabit Ethernet
driver: bnx2
IF: enp3s0f1 state: down mac: 00:26:55:4a:76:58
Drives: HDD Total Size: 737.9GB (2.0% used)
ID-1: /dev/sdb model: LOGICAL_VOLUME size: 587.1GB
ID-2: /dev/sda model: LOGICAL_VOLUME size: 146.8GB
ID-3: USB /dev/sdc model: Flash_Disk size: 4.0GB
Partition: ID-1: / size: 134G used: 1.7G (2%) fs: ext4 dev: /dev/sda2
ID-2: /var size: 526G used: 532M (1%)
fs: ext4 dev: /dev/sdb2
ID-3: swap-1 size: 12.88GB used: 0.00GB (0%)
fs: swap dev: /dev/sdb1
RAID: No RAID devices: /proc/mdstat, md_mod kernel module present
Sensors: System Temperatures: cpu: 51.0C mobo: N/A
Fan Speeds (in rpm): cpu: N/A
Info: Processes: 261 Uptime: 1:51 Memory: 8871.4/24092.6MB
Init: systemd runlevel: 5 Client: Shell (bash) inxi: 2.3.56
Suricata config is attached
suricata.yaml (70.6 KB)
How much traffic do you see?
Can you also post stats.log and suricata.log?
Sure.
stats.log (1.2 MB) suricata.log (53.0 KB)
I see no kernel_drops in the stats log
Oh yeah. I restarted suricata recently. The kernel drops show up at about 1h of uptime
In that case provide the stats log when it happens again.
It would also help to know more about the type of traffic and the traffic rate.
Sure.
I am mirroring an outside port (to the internet). Rate is about 20Mbit/s
Date: 2/27/2021 – 21:22:50 (uptime: 0d, 01h 22m 01s)
Counter | TM Name | Value
capture.kernel_packets | Total | 34865578
capture.kernel_drops | Total | 3679564
decoder.pkts | Total | 31184540
decoder.bytes | Total | 15408956223
decoder.invalid | Total | 3
decoder.ipv4 | Total | 31179326
decoder.ipv6 | Total | 2126
decoder.ethernet | Total | 31184540
decoder.tcp | Total | 18308510
decoder.udp | Total | 12016130
decoder.icmpv4 | Total | 854983
decoder.icmpv6 | Total | 1522
decoder.vlan | Total | 11283861
decoder.avg_pkt_size | Total | 494
decoder.max_pkt_size | Total | 4128
flow.tcp | Total | 2004659
flow.udp | Total | 343429
flow.icmpv4 | Total | 22510
flow.icmpv6 | Total | 174
flow.tcp_reuse | Total | 380
flow.wrk.spare_sync_avg | Total | 100
flow.wrk.spare_sync | Total | 19787
decoder.event.icmpv4.unknown_code | Total | 1
decoder.event.ipv6.zero_len_padn | Total | 285
decoder.event.ipv6.unknown_next_header | Total | 306
decoder.event.tcp.hlen_too_small | Total | 3
flow.wrk.flows_evicted_needs_work | Total | 54500
flow.wrk.flows_evicted_pkt_inject | Total | 74974
flow.wrk.flows_evicted | Total | 341630
flow.wrk.flows_injected | Total | 51319
tcp.sessions | Total | 1964856
tcp.pseudo | Total | 1616
tcp.invalid_checksum | Total | 1077
tcp.syn | Total | 2041399
tcp.synack | Total | 131315
tcp.rst | Total | 58887
tcp.stream_depth_reached | Total | 27
tcp.reassembly_gap | Total | 26091
tcp.overlap | Total | 14631
detect.alert | Total | 25143
app_layer.flow.http | Total | 9853
app_layer.tx.http | Total | 14310
app_layer.flow.ftp | Total | 5
app_layer.tx.ftp | Total | 20
app_layer.flow.smtp | Total | 3160
app_layer.tx.smtp | Total | 3270
app_layer.flow.tls | Total | 44395
app_layer.flow.ssh | Total | 3981
app_layer.flow.dns_tcp | Total | 811
app_layer.tx.dns_tcp | Total | 1668
app_layer.flow.ntp | Total | 3025
app_layer.tx.ntp | Total | 3535
app_layer.flow.tftp | Total | 18
app_layer.tx.tftp | Total | 3
app_layer.flow.dhcp | Total | 3
app_layer.tx.dhcp | Total | 4
app_layer.flow.snmp | Total | 985
app_layer.tx.snmp | Total | 10131
app_layer.flow.sip | Total | 23403
app_layer.tx.sip | Total | 23818
app_layer.flow.failed_tcp | Total | 8050
app_layer.flow.dns_udp | Total | 118736
app_layer.tx.dns_udp | Total | 258027
app_layer.flow.failed_udp | Total | 197259
flow.mgr.full_hash_pass | Total | 21
flow.spare | Total | 10347
flow.mgr.rows_maxlen | Total | 10
flow.mgr.flows_checked | Total | 792259
flow.mgr.flows_notimeout | Total | 249299
flow.mgr.flows_timeout | Total | 542960
flow.mgr.flows_evicted | Total | 1899479
flow.mgr.flows_evicted_needs_work | Total | 51319
tcp.memuse | Total | 30638928
tcp.reassembly_memuse | Total | 51646636
http.memuse | Total | 4510790
flow.memuse | Total | 49278144
(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)
If you can confirm that, I would recommend to revert some of the changes, so you can narrow it down to what option had the most impact in your scenario.
After running suricata for some time on the LAN interface, I am getting zero kernel_drops.
So I am only getting drops from the WAN interface. Maybe broken traffic