Hello,
I’m a Suricata newbie and I need help with higher number of kernel_drops. I went through the docs, but couldn’t follow 9.5. High Performance Configuration — Suricata 5.0.3 documentation, because Suricata is running on Hyper-V and thus doesn’t support almost any changes made by ethtool needed for perfomance tuning.
On the image below you can see rising numbers of kernel_packets and kernel_drops for the last 7 days until suricata got restarted. In the peak point the count is 107 947 240 for kernel_drops and 673 030 883 for kernel_packets, which comes into 16% of dropped packets, which is pretty much high I think.
The virtual machine is running with 8 CPUs/cores/threads, Ubuntu 18.04.3 LTS, kernel version 4.15.0-112-generic. Configuration of af-packet is default (the rest is commented):
# Linux high speed capture support
af-packet:
- interface: eth1
# Number of receive threads. "auto" uses the number of cores
threads: auto
# Default clusterid. AF_PACKET will load balance packets based on flow.
cluster-id: 99
# Default AF_PACKET cluster type. AF_PACKET can load balance per flow or per hash.
# This is only supported for Linux kernel > 3.1
# possible value are:
# * cluster_flow: all packets of a given flow are send to the same socket
# * cluster_cpu: all packets treated in kernel by a CPU are send to the same socket
# * cluster_qm: all packets linked by network card to a RSS queue are sent to the same
# socket. Requires at least Linux 3.14.
# * cluster_ebpf: eBPF file load balancing. See doc/userguide/capture-hardware/ebpf-xdp.rst for
# more info.
# Recommended modes are cluster_flow on most boxes and cluster_cpu or cluster_qm on system
# with capture card using RSS (require cpu affinity tuning and system irq tuning)
cluster-type: cluster_flow
#cluster-type: cluster_qm
# In some fragmentation case, the hash can not be computed. If "defrag" is set
# to yes, the kernel will do the needed defragmentation before sending the packets.
defrag: yes
At the moment of writing this post, after restarting suricata, there are some drops on eth1, where suricata should be sniffing, but not that much:
eth1: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 1500
inet6 fe80::215:5dff:fe19:8f04 prefixlen 64 scopeid 0x20
ether 00:15:5d:19:8f:04 txqueuelen 1000 (Ethernet)
RX packets 1067046 bytes 793598619 (793.5 MB)
RX errors 0 dropped 873 overruns 0 frame 0
TX packets 17 bytes 1362 (1.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
And this is also latest output of stats.log
------------------------------------------------------------------------------------
Date: 8/19/2020 -- 14:40:28 (uptime: 0d, 00h 16m 51s)
------------------------------------------------------------------------------------
Counter | TM Name | Value
------------------------------------------------------------------------------------
capture.kernel_packets | Total | 1148861
capture.kernel_drops | Total | 131999
decoder.pkts | Total | 1017118
decoder.bytes | Total | 733718905
decoder.invalid | Total | 88
decoder.ipv4 | Total | 1008659
decoder.ipv6 | Total | 1896
decoder.ethernet | Total | 1017118
decoder.tcp | Total | 852114
decoder.udp | Total | 155427
decoder.icmpv4 | Total | 1524
decoder.icmpv6 | Total | 294
decoder.vlan | Total | 1017096
decoder.avg_pkt_size | Total | 721
decoder.max_pkt_size | Total | 1534
flow.tcp | Total | 11278
flow.udp | Total | 3939
flow.icmpv4 | Total | 37
flow.icmpv6 | Total | 48
decoder.event.ipv4.trunc_pkt | Total | 88
decoder.event.ipv4.opt_pad_required | Total | 482
decoder.event.ipv6.zero_len_padn | Total | 178
tcp.sessions | Total | 10410
tcp.syn | Total | 12063
tcp.synack | Total | 9391
tcp.rst | Total | 19809
tcp.pkt_on_wrong_thread | Total | 17521
tcp.stream_depth_reached | Total | 27
tcp.reassembly_gap | Total | 432
tcp.overlap | Total | 78461
detect.alert | Total | 5
app_layer.flow.http | Total | 111
app_layer.tx.http | Total | 403
app_layer.flow.tls | Total | 8528
app_layer.flow.smb | Total | 2
app_layer.tx.smb | Total | 11
app_layer.flow.dcerpc_tcp | Total | 17
app_layer.flow.ntp | Total | 23
app_layer.tx.ntp | Total | 24
app_layer.flow.krb5_tcp | Total | 1
app_layer.tx.krb5_tcp | Total | 1
app_layer.flow.dhcp | Total | 45
app_layer.tx.dhcp | Total | 118
app_layer.flow.snmp | Total | 171
app_layer.tx.snmp | Total | 342
app_layer.flow.failed_tcp | Total | 147
app_layer.flow.dns_udp | Total | 2090
app_layer.tx.dns_udp | Total | 4552
app_layer.flow.krb5_udp | Total | 28
app_layer.tx.krb5_udp | Total | 24
app_layer.flow.failed_udp | Total | 1582
flow_mgr.closed_pruned | Total | 8802
flow_mgr.new_pruned | Total | 2544
flow_mgr.est_pruned | Total | 1301
flow.spare | Total | 10000
flow.tcp_reuse | Total | 406
flow_mgr.flows_checked | Total | 29
flow_mgr.flows_notimeout | Total | 25
flow_mgr.flows_timeout | Total | 4
flow_mgr.flows_timeout_inuse | Total | 3
flow_mgr.flows_removed | Total | 1
flow_mgr.rows_checked | Total | 65536
flow_mgr.rows_skipped | Total | 65493
flow_mgr.rows_empty | Total | 15
flow_mgr.rows_maxlen | Total | 2
tcp.memuse | Total | 4587760
tcp.reassembly_memuse | Total | 7337608
http.memuse | Total | 1248
flow.memuse | Total | 8347112
If there is any information I could you give to help resolve this issue, please let me know.
Thanks
