Cannot start Suricata DPDK using real device or vdev

Help
, ,
1

I’m using

# suricata -V
This is Suricata version 7.0.7 RELEASE

I have a compiled suricata with dpdk:

# suricata --build-info
This is Suricata version 7.0.7 RELEASE
Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HTTP2_DECOMPRESSION HAVE_LUA HAVE_JA3 HAVE_JA4 HAVE_LIBJANSSON TLS TLS_C11 MAGIC RUST POPCNT64 
SIMD support: SSE_4_2 SSE_4_1 SSE_3 SSE_2 
Atomic intrinsics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 11.4.1 20231218 (Red Hat 11.4.1-3), C version 201112
compiled with _FORTIFY_SOURCE=0
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.49, linked against LibHTP v0.5.49

Suricata Configuration:
  AF_PACKET support:                       yes
  AF_XDP support:                          no
  DPDK support:                            yes
  eBPF support:                            yes
  XDP support:                             yes
  PF_RING support:                         no
  NFQueue support:                         yes
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no 
  DAG enabled:                             no
  Napatech enabled:                        no
  WinDivert enabled:                       no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libjansson support:                      yes
  hiredis support:                         yes
  hiredis async with libevent:             yes
  PCRE jit:                                yes
  LUA support:                             yes
  libluajit:                               no
  GeoIP2 support:                          yes
  JA3 support:                             yes
  JA4 support:                             yes
  Non-bundled htp:                         no
  Hyperscan support:                       yes
  Libnet support:                          yes
  liblz4 support:                          yes
  Landlock support:                        yes

  Rust support:                            yes
  Rust strict mode:                        no
  Rust compiler path:                      /usr/bin/rustc
  Rust compiler version:                   rustc 1.75.0 (82e1608df 2023-12-21) (Red Hat 1.75.0-1.el9)
  Cargo path:                              /usr/bin/cargo
  Cargo version:                           cargo 1.75.0

  Python support:                          yes
  Python path:                             /usr/bin/python3
  Install suricatactl:                     yes
  Install suricatasc:                      yes
  Install suricata-update:                 yes

  Profiling enabled:                       no
  Profiling locks enabled:                 no
  Profiling rules enabled:                 no

  Plugin support (experimental):           yes
  DPDK Bond PMD:                           no

Development settings:
  Coccinelle / spatch:                     no
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no
  Fuzz targets enabled:                    no

Generic build parameters:
  Installation prefix:                     /usr
  Configuration directory:                 /etc/suricata/
  Log directory:                           /var/log/suricata/

  --prefix                                 /usr
  --sysconfdir                             /etc
  --localstatedir                          /var
  --datarootdir                            /usr/share

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / g++ (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                no
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -g -O2 -fPIC -std=c11 -I/usr/include/dpdk -include rte_config.h -march=corei7 -mrtm  -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist
  PCAP_CFLAGS                               
  SECCFLAGS

This is my partially suricata.yaml dpdk configuration:

dpdk:
  eal-params:
    proc-type: primary
    allow: [ 0000:0b:00.0 ]
    n: 1

  interfaces:
    - interface: 0000:0b:00.0
      threads: 2
      promisc: true
      multicast: true
      checksum-checks: true
      checksum-checks-offload: true
      mtu: 1500
      mempool-size: 65535
      mempool-cache-size: 257
      rx-descriptors: 1024
      tx-descriptors: 1024
      copy-mode: none
      copy-iface: none

dpdk-testpmd works:

# dpdk-testpmd -v -c 3 --main-lcore 0 -n 1 --socket-mem 4000 -a 0000:0b:00.0
EAL: Detected CPU lcores: 4
EAL: Detected NUMA nodes: 1
EAL: RTE Version: 'DPDK 23.11.0'
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No free 2048 kB hugepages reported on node 0
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
testpmd: No probed ethernet devices
testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Done
No commandline core given, start packet forwarding
io packet forwarding - ports=0 - cores=0 - streams=0 - NUMA support enabled, MP allocation mode: native

  io packet forwarding packets/burst=32
  nb forwarding cores=1 - nb forwarding ports=0
Press enter to exit

Telling cores to stop...
Waiting for lcores to finish...

  +++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
  RX-packets: 0              RX-dropped: 0             RX-total: 0
  TX-packets: 0              TX-dropped: 0             TX-total: 0
  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done.

Bye...

suricata fails:

# suricata --dpdk 
i: suricata: This is Suricata version 7.0.7 RELEASE running in SYSTEM mode
EAL: No free 2048 kB hugepages reported on node 0
TELEMETRY: No legacy callbacks, legacy socket not created
E: dpdk: Interface "0000:0b:00.0": No such device

Also, I cannot use a regular Linux interface. DPDK’s way is having it as parameter. For example:
vdev device: --vdev=net_pcap_iface_1,rx_iface_in=dummy_if1,tx_iface=dummy_if1.

Changing suricata.yaml for using a vdev:

dpdk:
  eal-params:
    proc-type: primary
    allow: [ net_pcap_iface_1 ]
    vdev: [ net_pcap_iface_1,rx_iface=dummy_if1,tx_iface=dummy_if1 ]
    no-huge: true
    m: 1000
    n: 1

  interfaces:
    - interface: net_pcap_iface_1
      threads: 2
      promisc: true
      multicast: true
      checksum-checks: true
      checksum-checks-offload: true
      mtu: 1500
      mempool-size: 65535
      mempool-cache-size: 257
      rx-descriptors: 1024
      tx-descriptors: 1024
      copy-mode: none
      copy-iface: none

Results with (running as root):

i: suricata: This is Suricata version 7.0.7 RELEASE running in SYSTEM mode
EAL: failed to parse device "rx_iface=pb_int_1"
EAL: Unable to parse device 'rx_iface=pb_int_1'
E: dpdk: DPDK EAL initialization error: Operation not permitted

Please help

2

Hello,

can you either try to define EAL params as:

eal-params:
    proc-type: primary
    allow: 0000:0b:00.0
    n: 1

or

eal-params:
    proc-type: primary
    allow: [ "0000:0b:00.0" ]
    n: 1

or just omit the additional allow and n EAL params for the start.

With vdev device I believe it should be more described as:

allow: [ "net_pcap_iface_1,rx_iface=dummy_if1,tx_iface=dummy_if1" ]

or try to define it without the array so:

allow: net_pcap_iface_1
vdev: net_pcap_iface_1,rx_iface=dummy_if1,tx_iface=dummy_if1
3

ok - actually all these did not work :confused:
But I did try to translate my known dpdk commands to the suricata.yaml more carefully.
Resulted with a working example for the vdev - which was for me a higher priority than pci-address.
I did not continue with the pci-address example.

This is a working configuration for a dpdk vdev:

dpdk:
  eal-params:
    proc-type: primary
    vdev: net_pcap_iface_1,rx_iface_in=dummy_if1,tx_iface=dummy_if1
    no-huge: true
    m: 1000
    n: 1
    v: true
    main-lcore: 0

  interfaces:
    - interface: net_pcap_iface_1
      threads: 2
      promisc: true
      multicast: true
      checksum-checks: true
      checksum-checks-offload: true
      mtu: 1500
      mempool-size: 65535
      mempool-cache-size: 257
      rx-descriptors: 1024
      tx-descriptors: 1024
      copy-mode: none
      copy-iface: none
...
threading:
  set-cpu-affinity: yes
  cpu-affinity:
    - management-cpu-set:
        cpu: [ 0 ]  # include only these CPUs in affinity settings
    - receive-cpu-set:
        cpu: [ 1 ]  # include only these CPUs in affinity settings
    - worker-cpu-set:
        cpu: [ 1 ]

Now it works:

$ sudo ./bin/suricata -c ./etc/suricata/suricata.yaml --dpdk
i: suricata: This is Suricata version 7.0.7 RELEASE running in SYSTEM mode
EAL: RTE Version: 'DPDK 21.11.6'
TELEMETRY: No legacy callbacks, legacy socket not created
W: dpdk: net_pcap_iface_1: changing MTU on port 0 is not supported, ignoring the setting
i: threads: Threads created -> W: 1 FM: 1 FR: 1   Engine started.

Question: I’m not sure if receive-cpu-set should be the same as management-cpu-set or worker-cpu-set.

Thank you very much!

1 Like
4

The receive-cpu-set can be removed along with verdict-cpu-set as those are not used in workers runmode.

Cheers,
Lukas

1 Like
5

btw I just inspected your dpdk-testpmd output more closely and it doesn’t work actually there as well. Testpmd reports:

testpmd: No probed ethernet devices

you can also see it in the forwarding info:

  nb forwarding cores=1 - nb forwarding ports=0
1 Like
6

Right, I haven’t noticed.
But I’m not sure I will continue to dig in right now because my main issue was the vdev.

Thank you very much!