Hey Andreas,
Thanks for getting back to me.
The box has 96 cores and prior to this full deployment I had a lab setup where I was able to push up to 14 gbps. However, I did have just 1 interface belonging to each port channel, so that is different than the actuall environment I am trying to run in. Also the latency is only seen on TCP traffic, UDP traffic flows through fine. Maybe a checksum issue? I will try testing with all rules disabled and see if that helps and test the TAP mode.
I had the NICs set to 1 RSS queue but still no luck. I will also try increasing the RSS queue but enabling a symmetric hash. I enabled promiscuous mode but that served me no luck. I think that Suricata is having a difficult time calculating a flow’s hash and tracking packets belonging to that flow. Do you know how Suricata sets this flow? Is it a combo of:
layer 2 + layer 3: ((( source_IP XOR dest_IP ) AND 0xffff) XOR ( source_MAC XOR destination_MAC ))
only layer 2: (source_MAC_address XOR destination_MAC)
layer 3 +4: (( source_port XOR dest_port ) XOR (( source_IP XOR dest_IP ) AND 0xffff)
Or is this set by another algorithm? Im thinking that maybe this flow algorithm could contribute to my problem?
Do you know of anyone else running af_packet mode on bonded interfaces? I wouldn’t think I’m the first to try this type of deployment but I dont see anything similar online.
Thanks for the help!
Taylor