Suricata IPsec performance issues

Help
1

Hello team,

since our update to 6.0.6 we’re experiencing massive performance issues with our IPsec (tested both IKEv1 and IKEv2) S2S tunnels.

Beside our internal north/south and east/west bridges we’re tapping our WAN<->WAN gateway traffic so suricata is able to what the gateway is doing so like the S2S IPsec traffic.
Suricata is running as a bridge in IPS mode via AF-Packet on a dedicated server.

Without suricata the S2S traffic is spinning around 15-20MB/s but with suricata tapping it drops down to 300KB-1MB/s.

I disabled both the IKEv2 parser and the decoder in the config and started suricata without any rules but this didn’t really improve the performance resulting in 500KB-1,3MB/s.

Other protocols like HTTP/s, SSH, OpenVPN and VoIP don’t seem to be affected by this.

Setup:
Custom debian distro
Kernel 5.18
Intel X710 10GB SFP+ nics running the i40e driver
Nic Offloading is disabled

AF-Packet Config (WAN-WAN bridge):

  • interface: eth8
    threads: 8
    defrag: yes
    cluster-type: cluster_ebpf
    ebpf-lb-file: /usr/libexec/suricata/ebpf/lb.bpf
    cluster-id: 98
    copy-mode: ips
    copy-iface: eth9
    use-emergency-flush: no
    #buffer-size: 32768
    ring-size: 150000
    use-mmap: yes
    tpacket-v3: no
  • interface: eth9
    threads: 8
    cluster-id: 97
    defrag: yes
    cluster-type: cluster_ebpf
    ebpf-lb-file: /usr/libexec/suricata/ebpf/lb.bpf
    copy-mode: ips
    copy-iface: eth8
    use-emergency-flush: no
    #buffer-size: 32768
    ring-size: 150000
    use-mmap: yes
    tpacket-v3: no

Any ideas?

Thanks in advance,
jiivas

2

I found out that changing suricata’s runmode from

autofp → workers

improves the IPsec Performance from 300KB/s - 1MB/s to 5-7MB/s but as soon as suricata is running in workers mode both tcp.pkt_on_wrong_thread and stream.wrong_thread are rising.

Workers mode:
stream.wrong_thread | Total | 249
tcp.pkt_on_wrong_thread | Total | 2296

Autofp mode:
stream.wrong_thread | Total | 0
tcp.pkt_on_wrong_thread | Total | 0

Any ideas?

3

In that case you should try to align it to match queues and cores with that NIC, see 9.5. High Performance Configuration — Suricata 6.0.8 documentation