# Suricata 7 , dpdk , Redhat 8 initial setup advice and tips appreciated
**URL:** https://forum.suricata.io/t/suricata-7-dpdk-redhat-8-initial-setup-advice-and-tips-appreciated/3394
**Category:** Help
**Tags:** suricata, dpdk
**Created:** [April 6, 2023, 11:08am UTC](https://forum.suricata.io/t/suricata-7-dpdk-redhat-8-initial-setup-advice-and-tips-appreciated/3394 "2023-04-06T11:08:02Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@atbohmer](https://forum.suricata.io/u/atbohmer)
#### Post date: [April 6, 2023, 11:08am UTC](https://forum.suricata.io/t/suricata-7-dpdk-redhat-8-initial-setup-advice-and-tips-appreciated/3394/1 "2023-04-06T11:08:02Z")
Hi,
Finnaly managd to get it working all together, but please need to reflect if I made the right choices and configuration assumptions. For now running with root priviliges but hope to run suricata with dpdk under a non root account.
HPE ProLiant DL380 Gen10 Plus
2 cpu nodes, 128 cores
128Gb memory
2 x Intel 810 100Gb nic’s for spanport data
DPDK source from git version 23.07.0-rc0
Suricata source from git version 7.0.0-rc2-dev (caf9940fd 2023-04-04)
- info on the Intel 810 card(s) before dpdk setup:
```auto
$ ethtool -i eth7
driver: ice
version: 4.18.0-425.13.1.el8_7.x86_64
firmware-version: 3.20 0x80012763 1.3189.0
expansion-rom-version:
bus-info: 0000:84:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
$ ethtool -l eth7
Channel parameters for eth7:
Pre-set maximums:
RX: 128
TX: 128
Other: 1
Combined: 128
Current hardware settings:
RX: 0
TX: 0
Other: 1
Combined: 128
```
Boot options: grub option intel\_iommu=on added
```auto
$ dpdk-devbind.py -s
Network devices using DPDK-compatible driver
============================================
0000:10:00.0 'Ethernet Controller E810-C for QSFP 1592' drv=vfio-pci unused=ice
0000:10:00.1 'Ethernet Controller E810-C for QSFP 1592' drv=vfio-pci unused=ice
0000:84:00.0 'Ethernet Controller E810-C for QSFP 1592' drv=vfio-pci unused=ice
0000:84:00.1 'Ethernet Controller E810-C for QSFP 1592' drv=vfio-pci unused=ice
```
- Huge pages:
```auto
$ dpdk-hugepages.py -s
Node Pages Size Total
0 16384 2Mb 32Gb
1 16384 2Mb 32Gb
```
Hugepages mounted on /dev/hugepages
- Suricata config regarding dpdk:
```yaml
dpdk:
eal-params:
proc-type: primary
interfaces:
threads: auto
mempool-cache-size: 257
copy-mode: none
- interface: default
threads: auto
promisc: true
multicast: true
checksum-checks: true
checksum-checks-offload: true
mtu: 1518
rss-hash-functions: auto
mempool-size: 65535
mempool-cache-size: 257
copy-mode: none
copy-iface: none
- interface: 0000:10:00.1
threads: auto
promisc: true
multicast: true
checksum-checks: true
checksum-checks-offload: true
mtu: 1518
rss-hash-functions: auto
mempool-size: 65535
mempool-cache-size: 257
copy-mode: none
copy-iface: none
- interface: 0000:84:00.0
threads: auto
promisc: true
multicast: true
checksum-checks: true
checksum-checks-offload: true
mtu: 1518
rss-hash-functions: auto
mempool-size: 65535
mempool-cache-size: 257
copy-mode: none
copy-iface: none
- interface: 0000:84:00.1
threads: auto
promisc: true
multicast: true
checksum-checks: true
checksum-checks-offload: true
mtu: 1518
rss-hash-functions: auto
mempool-size: 65535
mempool-cache-size: 257
copy-mode: none
copy-iface: none
```
- suricata systemd service file:
```auto
# Sample Suricata systemd unit file.
[Unit]
Description=Suricata Intrusion Detection Service
After=syslog.target network-online.target
[Service]
# Environment file to pick up $OPTIONS. On Fedora/EL this would be
# /etc/sysconfig/suricata, or on Debian/Ubuntu, /etc/default/suricata.
EnvironmentFile=-/etc/sysconfig/suricata
#EnvironmentFile=-/etc/default/suricata
ExecStartPre=/bin/rm -f /var/run/suricata.pid
ExecStart=/usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid $OPTIONS
ExecReload=/bin/kill -USR2 $MAINPID
LimitNOFILE=1024000
LimitMEMLOCK=infinity
LimitMEMLOCKSoft=infinity
[Install]
WantedBy=multi-user.target
```
- Starting suricata takes way to long , maybe has todo with number of threads?
```auto
tail -f -s0 /var/log/suricata/suricata.log
[5956 - Suricata-Main] 2023-04-06 11:46:40 Info: runmodes: 0000:10:00.0: creating 128 threads
[5956 - Suricata-Main] 2023-04-06 11:46:43 Info: dpdk: 0000:10:00.1: creating packet mbuf pool mempool_0000:10:00.1 of size 65535, cache size 257, mbuf size 2176
[5956 - Suricata-Main] 2023-04-06 11:46:43 Info: runmodes: 0000:10:00.1: creating 128 threads
[15216 - Suricata-Main] 2023-04-06 12:32:32 Notice: suricata: This is Suricata version 7.0.0-rc2-dev (caf9940fd 2023-04-04) running in SYSTEM mode
[15216 - Suricata-Main] 2023-04-06 12:32:32 Info: cpu: CPUs/cores online: 128
[15216 - Suricata-Main] 2023-04-06 12:32:32 Info: suricata: Preparing unexpected signal handling
[15216 - Suricata-Main] 2023-04-06 12:32:32 Info: privs: dropped the caps for main thread
[15216 - Suricata-Main] 2023-04-06 12:32:32 Info: conf: Running in live mode, activating unix socket
[15216 - Suricata-Main] 2023-04-06 12:32:32 Info: logopenfile: eve-log output device (regular) initialized: /data/sensor_data/suricata/eve.json
[15216 - Suricata-Main] 2023-04-06 12:32:32 Info: logopenfile: stats output device (regular) initialized: stats.log
[15216 - Suricata-Main] 2023-04-06 12:32:45 Info: detect: 1 rule files processed. 70441 rules successfully loaded, 0 rules failed
[15216 - Suricata-Main] 2023-04-06 12:32:45 Info: threshold-config: Threshold config parsed: 0 rule(s) found
[15216 - Suricata-Main] 2023-04-06 12:32:45 Info: detect: 70444 signatures processed. 2352 are IP-only rules, 14806 are inspecting packet payload, 53255 inspect application layer, 0 are decoder event only
[15216 - Suricata-Main] 2023-04-06 12:33:18 Info: dpdk: 0000:10:00.0: creating packet mbuf pool mempool_0000:10:00.0 of size 65535, cache size 257, mbuf size 2176
[15216 - Suricata-Main] 2023-04-06 12:33:18 Info: runmodes: 0000:10:00.0: creating 128 threads
[15216 - Suricata-Main] 2023-04-06 12:33:34 Info: dpdk: 0000:10:00.1: creating packet mbuf pool mempool_0000:10:00.1 of size 65535, cache size 257, mbuf size 2176
[15216 - Suricata-Main] 2023-04-06 12:33:38 Info: runmodes: 0000:10:00.1: creating 128 threads
[15216 - Suricata-Main] 2023-04-06 12:40:48 Info: dpdk: 0000:84:00.0: creating packet mbuf pool mempool_0000:84:00.0 of size 65535, cache size 257, mbuf size 2176
[15216 - Suricata-Main] 2023-04-06 12:41:10 Info: runmodes: 0000:84:00.0: creating 128 threads
[20857 - W#128-84:00.0] 2023-04-06 12:56:42 Warning: dpdk: 0000:84:00.0: NIC is on NUMA 1, 128 threads on different NUMA node(s)
[15216 - Suricata-Main] 2023-04-06 12:56:47 Info: dpdk: 0000:84:00.1: creating packet mbuf pool mempool_0000:84:00.1 of size 65535, cache size 257, mbuf size 2176
[15216 - Suricata-Main] 2023-04-06 12:57:20 Info: runmodes: 0000:84:00.1: creating 128 threads
```
---
### Author:  [@vjulien](https://forum.suricata.io/u/vjulien)
#### Post date: [April 7, 2023, 5:15am UTC](https://forum.suricata.io/t/suricata-7-dpdk-redhat-8-initial-setup-advice-and-tips-appreciated/3394/2 "2023-04-07T05:15:38Z")
By listing 4 interfaces each with `threads: auto` with 128 cores, I think it is going to create 512 threads (4x128). That is probably excessive even for this machine. What happens if you limit it to 16 or 32 per interface?
---
### Author:  [@atbohmer](https://forum.suricata.io/u/atbohmer)
#### Post date: [April 7, 2023, 7:22am UTC](https://forum.suricata.io/t/suricata-7-dpdk-redhat-8-initial-setup-advice-and-tips-appreciated/3394/3 "2023-04-07T07:22:03Z")
Indeed, it took a hour to start, but it started 😉 Is there some rule of thumbd regarding theads, interfaces and cores of just try and find out?
32 Instead of auto for threads fired up suricate within minutes:
# tail -f -s0 /var/log/suricata/suricata.log
[275632 - Suricata-Main] 2023-04-07 09:17:31 Info: runmodes: 0000:10:00.0: creating 32 threads
[275632 - Suricata-Main] 2023-04-07 09:17:32 Info: dpdk: 0000:10:00.1: creating packet mbuf pool mempool\_0000:10:00.1 of size 65535, cache size 257, mbuf size 2176
[275632 - Suricata-Main] 2023-04-07 09:17:32 Info: runmodes: 0000:10:00.1: creating 128 threads
[276456 - Suricata-Main] 2023-04-07 09:19:18 Notice: suricata: This is Suricata version 7.0.0-rc2-dev (caf9940fd 2023-04-04) running in SYSTEM mode
[276456 - Suricata-Main] 2023-04-07 09:19:18 Info: cpu: CPUs/cores online: 128
[276456 - Suricata-Main] 2023-04-07 09:19:18 Info: suricata: Preparing unexpected signal handling
[276456 - Suricata-Main] 2023-04-07 09:19:18 Info: privs: dropped the caps for main thread
[276456 - Suricata-Main] 2023-04-07 09:19:18 Info: conf: Running in live mode, activating unix socket
[276456 - Suricata-Main] 2023-04-07 09:19:18 Info: logopenfile: eve-log output device (regular) initialized: /data/sensor\_data/suricata/eve.json
[276456 - Suricata-Main] 2023-04-07 09:19:18 Info: logopenfile: stats output device (regular) initialized: stats.log
[276456 - Suricata-Main] 2023-04-07 09:19:31 Info: detect: 1 rule files processed. 70150 rules successfully loaded, 0 rules failed
[276456 - Suricata-Main] 2023-04-07 09:19:31 Info: threshold-config: Threshold config parsed: 0 rule(s) found
[276456 - Suricata-Main] 2023-04-07 09:19:31 Info: detect: 70153 signatures processed. 2344 are IP-only rules, 14514 are inspecting packet payload, 53264 inspect application layer, 0 are decoder event only
[276456 - Suricata-Main] 2023-04-07 09:20:05 Info: dpdk: 0000:10:00.0: creating packet mbuf pool mempool\_0000:10:00.0 of size 65535, cache size 257, mbuf size 2176
[276456 - Suricata-Main] 2023-04-07 09:20:05 Info: runmodes: 0000:10:00.0: creating 32 threads
[276456 - Suricata-Main] 2023-04-07 09:20:07 Info: dpdk: 0000:10:00.1: creating packet mbuf pool mempool\_0000:10:00.1 of size 65535, cache size 257, mbuf size 2176
[276456 - Suricata-Main] 2023-04-07 09:20:07 Info: runmodes: 0000:10:00.1: creating 32 threads
[276456 - Suricata-Main] 2023-04-07 09:20:08 Info: dpdk: 0000:84:00.0: creating packet mbuf pool mempool\_0000:84:00.0 of size 65535, cache size 257, mbuf size 2176
[276456 - Suricata-Main] 2023-04-07 09:20:08 Info: runmodes: 0000:84:00.0: creating 32 threads
[276869 - W#32-84:00.0] 2023-04-07 09:20:09 Warning: dpdk: 0000:84:00.0: NIC is on NUMA 1, 32 threads on different NUMA node(s)
[276456 - Suricata-Main] 2023-04-07 09:20:09 Info: dpdk: 0000:84:00.1: creating packet mbuf pool mempool\_0000:84:00.1 of size 65535, cache size 257, mbuf size 2176
[276456 - Suricata-Main] 2023-04-07 09:20:09 Info: runmodes: 0000:84:00.1: creating 32 threads
[276905 - W#32-84:00.1] 2023-04-07 09:20:17 Warning: dpdk: 0000:84:00.1: NIC is on NUMA 1, 32 threads on different NUMA node(s)
[276456 - Suricata-Main] 2023-04-07 09:20:21 Info: unix-manager: unix socket ‘/var/run/suricata/suricata-command.socket’
[276456 - Suricata-Main] 2023-04-07 09:20:25 Notice: threads: Threads created → W: 128 FM: 1 FR: 1 Engine started.
---
### Author:  [@atbohmer](https://forum.suricata.io/u/atbohmer)
#### Post date: [April 7, 2023, 8:19am UTC](https://forum.suricata.io/t/suricata-7-dpdk-redhat-8-initial-setup-advice-and-tips-appreciated/3394/4 "2023-04-07T08:19:49Z")
Regarding CPU affinity and NUMA, how do I pin those dpdk interfaces? I know how to find out which interface relates to which node, but how to make configure this right in suricata?
---
### Author:  [@lukashino](https://forum.suricata.io/u/lukashino)
#### Post date: [April 20, 2023, 7:08pm UTC](https://forum.suricata.io/t/suricata-7-dpdk-redhat-8-initial-setup-advice-and-tips-appreciated/3394/5 "2023-04-20T19:08:30Z")
I think you could order your interface in within the `interfaces` list so that you first start with the first NUMA node and then define the NICs located on the other NUMA node.
something like
```auto
dpdk:
interfaces:
- interface: 7b:00.0 (node 0)
threads: 4
- interface: 7b:00.1 (node 0)
threads: 4
- interface: 8c:00.0 (node 1)
threads: 4
- interface: 8c:00.0 (node 1)
threads: 4
threading:
set-cpu-affinity: yes
workers: [0-7 (NUMA 0), 64-71 (NUMA 1)]
```
Hopefully my “illustration” is correct and helpful.
---
### Author:  [@atbohmer](https://forum.suricata.io/u/atbohmer)
#### Post date: [April 21, 2023, 9:08am UTC](https://forum.suricata.io/t/suricata-7-dpdk-redhat-8-initial-setup-advice-and-tips-appreciated/3394/6 "2023-04-21T09:08:00Z")
Thanks, but my NUMA config is more complicated then 1-10 for 0 and 11-20 for 1, it is mixed sigh:
NUMA node0 CPU(s): 0-31,64-95
NUMA node1 CPU(s): 32-63,96-127
So this works for me the way suricata handles affinity with threads: 24 per interface:
cpu: [“6-29”,“38-61”,“64-87”,“96-119”]
And this interface order:
- interface: 0000:10:00.0
- interface: 0000:84:00.0
- interface: 0000:10:00.1
- interface: 0000:84:00.1
---
### Author:  [@lukashino](https://forum.suricata.io/u/lukashino)
#### Post date: [April 21, 2023, 10:10am UTC](https://forum.suricata.io/t/suricata-7-dpdk-redhat-8-initial-setup-advice-and-tips-appreciated/3394/7 "2023-04-21T10:10:35Z")
Perfect, great to hear that.