IPC Source Feature

I’ve created a PR that introduces a source that receives packets from a IPC packet sharing library.

Looking for some feedback, and have a couple questions.

  • Currently it only has one receive thread. Is this useful as is, or should the PR use multiple receive threads? The packet sharing library can share packets up to about 800-900mbps.
  • How should this be tested? I have a library which can test that this works, but not sure how to incorporate that testing in suricata?
  • Should this be added? Is it useful? It’s somewhat like dpdk, but can be used from any packet capturing application.

Is it too complicated to add? I would say it should be worth it.

In general what specific use cases do you have in mind?

Is it too complicated to add? I would say it should be worth it.

It’s not too hard to add, but does make the PR a little bigger.

In general what specific use cases do you have in mind?

Systems that tie together multiple packet processors. Give packets to bro (can use a similar approach in a plugin), pass packets through ndpi/qosmos, etc. And in environments where you can’t setup DPDK or don’t want to give up an interface.

So is it similar to testimony from Google as done in Testimony feature 3675 v6 by MalakhatkoVadym · Pull Request #4987 · OISF/suricata · GitHub and if yes, what are the differences?

I think those use cases are interesting, although I’m a bit worried with regards to performance when I see how even systems with just Suricata can struggle sometimes :slight_smile:

This PR is similar to testimony, however it uses a library to do the packet sharing, so that packets can be brokered by logic more complicated than what bpf and af_packet fanout provides, such as don’t process modbus, or put all ftp on the same receive thread. It also does the packet copy in the brokering application (could be improved later to not copy) to handle differences in consumer consumption rate, with no copy in suricata.

In short, this enables us to do a complex pipelined and/or processing fanout approach without adding additional overhead to suricata (copy/cleanup of packet data).

That sounds exciting. Did you already test it with regards to performance?

That sounds exciting. Did you already test it with regards to performance?

Quite a bit. Which brings up how I bring up some of that testing as part of the PR. Been using it with GitHub - protectwise/suricata-ipc: IPC Packet Sharing with Suricata (successor to Bellini that I presented on at Suricon).