About CPU/Core requirements by Suricata when DPDK is enabled

Developers
1

Hello,

I have recently enabled DPDK in Suricata, and using vhost-virtio interfaces to exchange packets between suricata and my datapath.

When I pass less number of core in worker-cpu-set as compared to the total number of interfaces, I see that Suricata does not start but throws the error, Error: dpdk: Interfaces requested more cores than configured in the threading section (requested 2 configured 1

In the code also it is evident that there is an explicit check to see if the cores allocated are minimum of total thread count from interface configuration. if (total_cpus > UtilAffinityGetAffinedCPUNum(wtaf)).

While this was not the case prior to DPDK. Earlier I was using tap interfaces(capture method AF_PACKET), where I used have 1 core for 2 tap interfaces.

Now I have following questions

  1. what is the reason behind this design? Is it because of PMD? If yes should it allow lesser cores when interrupt-mode is enabled for interface config?
    The reason why I am asking this is, with this condition Suricata would need minimum of 2cores when the mode is IPS. Will it not be a problem when Suricata is tried to run in machines with limited cores? (which is the case with me )
  2. Is there any workaround? I am okay for lesser performance but consistent/robust behaviour.
  3. If I could allocate more cores to Suricata, which is option is better?
    1. Creating multiple vhost-interface pairs
    2. Creating just one pair of Vhost interface with more number of threads.

DPDK version: 23.11

Suricata version: 7.0.9

My config:

threading:
set-cpu-affinity: yes
cpu-affinity:

  • management-cpu-set:
    cpu: [1]
  • worker-cpu-set:
    cpu: [4]
    mode: “balanced”
    prio:
    default: “high”

dpdk:
eal-params:
proc-type: primary
file-prefix: idps
single-file-segments:
iova-mode: va
vdev:
-virtio_user_in0,path=/tmp/vhost_in0,in_order=0,queue_size=1024,mrg_rxbuf=1’

  • ‘virtio_user_in1,path=/tmp/vhost_in1,in_order=0,queue_size=1024,mrg_rxbuf=1’

interfaces:

  • interface: virtio_user_in0
    threads: 1
    interrupt-mode: true
    promisc: false
    multicast: false
    interrupt-mode: true
    checksum-checks: false
    checksum-checks-offload: false
    mtu: 9000
    rss-hash-functions: 0x0
    mempool-size: 3072
    mempool-cache-size: 257
    rx-descriptors: 1024
    tx-descriptors: 1024
    cluster-type: cluster_flow
    defrag: yes
    use-mmap: yes
    checksum-checks: false
    copy-mode: ips
    copy-iface: virtio_user_in0
  • interface: virtio_user_in1
    threads: 1
    interrupt-mode: true
    promisc: false
    multicast: false
    interrupt-mode: true
    checksum-checks: false
    checksum-checks-offload: false
    mtu: 9000
    rss-hash-functions: 0x0
    mempool-size: 3072
    mempool-cache-size: 257
    rx-descriptors: 1024
    tx-descriptors: 1024
    cluster-type: cluster_flow
    defrag: yes
    use-mmap: yes
    checksum-checks: false
    copy-mode: ips
    copy-iface: virtio_user_in1

Thanks

Muttanna

2

This validation step has been in the code base for over 2 years. The point of the check is that DPDK is supposed to be a high-performance capture interface; if you assign 2 workers to one core, you will get unpredictable performance.

  1. Mainly described above. Could be potentially amended but I think for DPDK you still want to use higher-end machines.
  2. You can try to remove the check and see if DPDK brings any benefits.
  3. I cannot really comment on this but here is my guess – to get the absolute best performance it is probably better to use multiple vhost-interface pairs (if they are maxing out the capacities). For real life deployments, I would probably go with one vhost interface and multiple threads. But be sure the vhost interface supports RSS/queue distribution - I don’t think, by default, it is supported and was only supported by perhaps only VMWare (fuzzy memories)

Lukas

3

Thank you so much for your response. I will do some testing with removing the check and update back here.