I’ve created a PR that introduces a source that receives packets from a IPC packet sharing library.
Looking for some feedback, and have a couple questions.
Currently it only has one receive thread. Is this useful as is, or should the PR use multiple receive threads? The packet sharing library can share packets up to about 800-900mbps.
How should this be tested? I have a library which can test that this works, but not sure how to incorporate that testing in suricata?
Should this be added? Is it useful? It’s somewhat like dpdk, but can be used from any packet capturing application.
Is it too complicated to add? I would say it should be worth it.
It’s not too hard to add, but does make the PR a little bigger.
In general what specific use cases do you have in mind?
Systems that tie together multiple packet processors. Give packets to bro (can use a similar approach in a plugin), pass packets through ndpi/qosmos, etc. And in environments where you can’t setup DPDK or don’t want to give up an interface.
I think those use cases are interesting, although I’m a bit worried with regards to performance when I see how even systems with just Suricata can struggle sometimes
This PR is similar to testimony, however it uses a library to do the packet sharing, so that packets can be brokered by logic more complicated than what bpf and af_packet fanout provides, such as don’t process modbus, or put all ftp on the same receive thread. It also does the packet copy in the brokering application (could be improved later to not copy) to handle differences in consumer consumption rate, with no copy in suricata.
In short, this enables us to do a complex pipelined and/or processing fanout approach without adding additional overhead to suricata (copy/cleanup of packet data).