Suricata v7.0.2
Running on Bluefield-2 SmartNIC from Nvidia. It runs a Ubuntu 22.04.
Suricata was compiled from source.
Hi All, I am running Suricata with the above setting.
Suricata is working on the Bluefield-2 SmartNIC on other modes, e.g., AF_PACKET (+ebpf loadbalancing).
I wanted to try out with DPDK. I compiled DPDK on my own and during the compilation of Suricata, all related libs were found.
My setup looks like this:
Looks a bit complicated, but it is not. Just that everything is in one server, which allows me to do accurate latency measurements.
Eventually, my aim is to run Suricata on the SmartNIC, which can inspect the traffic before it would reach the host itself. Therefore, there is a realky physical interface (termed as bf1_p0 in the figure), and a virtual one facing the host (termed as bf1_pf0hpf). While I have issues in setting up Suricata to use DPDK for both of my interfaces (I donât know how to configure bf1_pf0hpf for now), I was wondering whether I can just make an IPS mode on the same interface, i.e., copy from bf1_p0 back to pf1_p0 itself.
I replay a specific PCAP file and matching rules are installed in Suricata. This use case is working with other modes, so logically they are confirmed.
However, when I run with DPDK, I donât see any alert logs (which should be there) and also no packets are sent back to the same interface.
I also tried running Suricata with none or tap mode, with or without any copy-interface set in order to just see the alerts (without actually sending the packets back). However, I have no success yet I neither have any error. Does anybody has some experience similar to my setup?
My suricata.yamlâs relevant part:
dpdk:
eal-params:
proc-type: primary
# allow: ["0000:03:00.0"] #this is p0 interface
# DPDK capture support
# RX queues (and TX queues in IPS mode) are assigned to cores in 1:1 ratio
interfaces:
- interface: 0000:03:00.0 # PCIe address of the NIC port - this is p0 interface
threads: 4
promisc: true
multicast: true
checksum-checks: true
checksum-checks-offload: true
mtu: 1518
mempool-size: 262143 # The number of elements in the mbuf pool
mempool-cache-size: 511
rx-descriptors: 4096
tx-descriptors: 4096
copy-mode: ips
copy-iface: 0000:03:00.0
...
threading:
set-cpu-affinity: yes
cpu-affinity:
- management-cpu-set:
cpu: [ 0 ]
- receive-cpu-set:
cpu: [ 0 ]
- worker-cpu-set:
cpu: [ "1-4" ]
mode: "exclusive"
prio:
low: [ ]
medium: [ 0 ]
high: [ "1-4" ]
default: "high"
The STDOUT of Suricata after running
suricata -c /etc/suricata/suricata.yaml --dpdk -vv
[1057774] Notice: suricata: This is Suricata version 7.0.2 RELEASE running in SYSTEM mode
[1057774] Info: cpu: CPUs/cores online: 8
[1057774] Info: dpdk: Setting IPS mode
[1057774] Info: exception-policy: master exception-policy set to: auto
[1057774] Info: suricata: Preparing unexpected signal handling
[1057774] Info: conf: Running in live mode, activating unix socket
[1057774] Info: logopenfile: fast output device (regular) initialized: fast.log
[1057774] Info: logopenfile: eve-log output device (regular) initialized: eve.json
[1057774] Info: logopenfile: stats output device (regular) initialized: stats.log
[1057774] Info: detect: 1 rule files processed. 48 rules successfully loaded, 0 rules failed
[1057774] Info: threshold-config: Threshold config parsed: 0 rule(s) found
[1057774] Info: detect: 48 signatures processed. 0 are IP-only rules, 0 are inspecting packet payload, 0 inspect application layer, 0 are decoder event only
[1057774] Perf: detect: TCP toserver: 41 port groups, 41 unique SGH's, 0 copies
[1057774] Perf: detect: TCP toclient: 21 port groups, 21 unique SGH's, 0 copies
[1057774] Perf: detect: UDP toserver: 41 port groups, 41 unique SGH's, 0 copies
[1057774] Perf: detect: UDP toclient: 21 port groups, 21 unique SGH's, 0 copies
[1057774] Perf: detect: OTHER toserver: 254 proto groups, 1 unique SGH's, 253 copies
[1057774] Perf: detect: OTHER toclient: 254 proto groups, 0 unique SGH's, 254 copies
[1057774] Perf: detect: Unique rule groups: 125
[1057774] Perf: detect: Builtin MPM "toserver TCP packet": 0
[1057774] Perf: detect: Builtin MPM "toclient TCP packet": 0
[1057774] Perf: detect: Builtin MPM "toserver TCP stream": 0
[1057774] Perf: detect: Builtin MPM "toclient TCP stream": 0
[1057774] Perf: detect: Builtin MPM "toserver UDP packet": 0
[1057774] Perf: detect: Builtin MPM "toclient UDP packet": 0
[1057774] Perf: detect: Builtin MPM "other IP packet": 0
[1057774] Perf: profiling-rulegroups: Registered 125 rulegroup profiling counters.
[1057774] Perf: profiling-keywords: Registered 276 keyword profiling counters.
[1057774] Perf: profiling-rules: Registered 48 rule profiling counters.
EAL: No available hugepages reported in hugepages-32768kB
EAL: No available hugepages reported in hugepages-64kB
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Invalid NUMA socket, default to 0
EAL: Invalid NUMA socket, default to 0
[1057774] Info: dpdk: 0000:03:00.0: DPDK IPS mode activated: 0000:03:00.0->0000:03:00.0
[1057774] Info: runmodes: 0000:03:00.0: creating 4 threads
[1057785] Perf: threads: Setting prio -2 for thread "W#01-03:00.0" to cpu/core 1, thread id 1057785
[1057786] Perf: threads: Setting prio -2 for thread "W#02-03:00.0" to cpu/core 2, thread id 1057786
[1057787] Perf: threads: Setting prio -2 for thread "W#03-03:00.0" to cpu/core 3, thread id 1057787
[1057788] Perf: threads: Setting prio -2 for thread "W#04-03:00.0" to cpu/core 4, thread id 1057788
[1057789] Perf: threads: Setting prio 0 for thread "FM#01", thread id 1057789
[1057790] Perf: threads: Setting prio 0 for thread "FR#01", thread id 1057790
[1057791] Perf: threads: Setting prio 0 for thread "CW", thread id 1057791
[1057792] Perf: threads: Setting prio 0 for thread "CS", thread id 1057792
[1057774] Info: unix-manager: unix socket '/var/run/suricata/suricata-command.socket'
[1057793] Perf: threads: Setting prio 0 for thread "US", thread id 1057793
[1057774] Notice: threads: Threads created -> W: 4 FM: 1 FR: 1 Engine started.
[1057774] Info: dpdk: 6193 of 6622 of hugepages are free - number of hugepages can be lowered to e.g. 494
^C[1057774] Notice: suricata: Signal Received. Stopping engine.
[1057774] Info: suricata: time elapsed 1214.633s
[1057790] Perf: flow-manager: 0 flows processed
[1057785] Perf: dpdk: Port 0 (0000:03:00.0) - tx_packets: 1
[1057785] Perf: dpdk: Port 0 (0000:03:00.0) - tx_bytes: 74
[1057785] Perf: dpdk: 0000:03:00.0: total RX stats: packets 0 bytes: 0 missed: 0 errors: 0 nombufs: 0
[1057785] Perf: dpdk: 0000:03:00.0: total TX stats: packets 0 bytes: 0 errors: 0
[1057785] Perf: dpdk: (W#01-03:00.0) received packets 0
[1057786] Perf: dpdk: (W#02-03:00.0) received packets 0
[1057787] Perf: dpdk: (W#03-03:00.0) received packets 0
[1057788] Perf: dpdk: (W#04-03:00.0) received packets 0
[1057774] Info: counters: Alerts: 0
[1057774] Perf: ippair: ippair memory usage: 422144 bytes, maximum: 16777216
[1057774] Perf: profiling: Done dumping profiling data.
[1057774] Perf: host: host memory usage: 406144 bytes, maximum: 33554432
[1057774] Perf: profiling-rules: Dumping profiling data for 48 rules.
[1057774] Perf: profiling-rules: Done dumping profiling data.
[1057774] Perf: profiling-keywords: Done dumping keyword profiling data.
[1057774] Perf: profiling-rulegroups: Done dumping rulegroup profiling data.
[1057774] Perf: dpdk: 0000:03:00.0: closing device
[1057774] Notice: device: 0000:03:00.0: packets: 0, drops: 0 (0.00%), invalid chksum: 0
As you can see, there are no errors but also no packets received/processed at all.
Thanks
