Hello,
I have been trying to use AF_XDP mode in suricata for a bit now and I am wondering if there is a way to verify if the installation is correct. I have observed some behavior that leads me to believe my installation is incorrect.
First, from my understanding of AF_XDP sockets, to send data to them an XDP program must be loaded. This program should make use of the BPF_MAP_TYPE_XSKMAP to send data to the AF_XDP socket in user space with a call to bpf_redirect_map(). When running suricata in AF_XDP mode with the command:
suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-xdp -vvvv
I am able to get suricata to run with the following output in the log (shortened to show af-xdp messages):
Info: ioctl: enp65s0f0np0: RX RSS queues: 32 [GetIfaceRSSQueuesNum:util-ioctl.c:743]
Perf: af-xdp: 32 cores, so using 32 threads [ConfigSetThreads:runmode-af-xdp.c:125]
Info: runmodes: enp65s0f0np0: creating 32 threads [RunModeSetLiveCaptureWorkersForDevice:util-runmodes.c:254]
Config: flow-manager: using 1 flow manager threads [FlowManagerThreadSpawn:flow-manager.c:956]
Config: flow-manager: using 1 flow recycler threads [FlowRecyclerThreadSpawn:flow-manager.c:1147]
Info: unix-manager: unix socket '/var/run/suricata/suricata-command.socket' [UnixNew:unix-manager.c:135]
Notice: threads: Threads created -> W: 32 FM: 1 FR: 1 Engine started. [TmThreadWaitOnThreadRunning:tm-threads.c:1884]
In another terminal, when running the xdp-loader tool available as part of xdp-tools, I get an output that shows a program was loaded with no name:
~$ sudo xdp-loader status
CURRENT XDP PROGRAM STATUS:
Interface Prio Program name Mode ID Tag Chain actions
--------------------------------------------------------------------------------------
lo <No XDP program loaded!>
eno8303 <No XDP program loaded!>
eno8403 <No XDP program loaded!>
enp65s0f0np0 native 122 03b13f331978c78c
enp65s0f1np1 <No XDP program loaded!>
enp196s0np0 <No XDP program loaded!>
enp196s0np1 <No XDP program loaded!>
Also the output of the ip link command shows similar information:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno8303: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether cc:48:3a:2b:22:f8 brd ff:ff:ff:ff:ff:ff
altname enp193s0f0
3: eno8403: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether cc:48:3a:2b:22:f9 brd ff:ff:ff:ff:ff:ff
altname enp193s0f1
4: enp65s0f0np0: <BROADCAST,PROMISC,UP,LOWER_UP> mtu 1500 xdp qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether b8:3f:d2:37:ae:f6 brd ff:ff:ff:ff:ff:ff
prog/xdp id 122
5: enp65s0f1np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether b8:3f:d2:37:ae:f7 brd ff:ff:ff:ff:ff:ff
6: enp196s0np0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether 00:15:4d:13:39:22 brd ff:ff:ff:ff:ff:ff
7: enp196s0np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:15:4d:13:39:26 brd ff:ff:ff:ff:ff:f
Second, if in the suricata.yaml file I specify the “force-bind-mode: zero” for zero copy as shown:
af-xdp:
- interface: enp65s0f0np0
force-xdp-mode: drv
force-bind-mode: zero
mem-unaligned: yes
I get the following error:
Perf: af-xdp: 32 cores, so using 32 threads [ConfigSetThreads:runmode-af-xdp.c:125]
Info: runmodes: enp65s0f0np0: creating 32 threads [RunModeSetLiveCaptureWorkersForDevice:util-runmodes.c:254]
Error: af-xdp: Failed to create socket: Invalid argument [OpenXSKSocket:source-af-xdp.c:437]
I believe my installation may be wrong and was wondering if either anyone else has experienced this behavior or if anyone can help point me in the direction to finding my error.
Thanks in advance!