Weird result when comparing Suricata detections with snort (using same traffic) - what am I missing?

Hello folks,

First I would like to thank you for being such great community.

I’m running some tests and I noticed some inconsistencies in the number of alerts that am I having trouble to explain. Worth to mention that all the SIDs raised by snort are detected, only the amount differs (plus a bunch of other SIDs that only suricata detects).

To give you some more info about the testing method:

  • Two hosts directly connected (each host have 2x10Gb cards bonded in round-robin)

  • (Almost) Same rulesets used for suri and snort - the difference being snort using also the Talos ones.

  • No perf tuning applied yet (currently configured just basic settings like disable NIC offloads and similar)

  • Tests executed with tcpreplay:

    1. topspeed injection - noticed drops in stats.log
    2. realtime injection - noticed drops in stats.log
    3. injection limited at a lower speed (5 Mbps) - no drops reported in stats.log

This is the result I got:

SIDs snort alerts suricata alerts
2033078 131260 4242
2023472 1527 73
2028651 76 2
2033967 23 1
2009099 4 1
2029340 2 464

Interesting facts:

  • I got very same results with no drops and with 11k+ drops (compared 2 different tests)
  • In only one case, suricata reports more alerts (2029340)
  • snort stats were taken from our SIEM looking at the historical data, not while re-injecting the same pcap

May I have your valuable opinion on that?
I initially thought the lesser amount of alerts was due to drops but latest test discarded that theory.

So am I wondering:

  • is my snort stat (taken with different method) unreliable?
  • can there be different rules with same SID?

Thanks in advance!
Davide

Hi,

there are many possible reasons for that.

  1. Did you ensure to use the suricata specific signatures which differ a bit compared to the ones for snort, rule language is not 100% compatible.

  2. 2028651 for example uses threshold. And even besides that flow timeout values and stuff like that could explain that. So maybe suricata still sees the traffic in one flow and thus might just hit once but snort thinks it’s a new flow.

  3. Since you don’t have the case where one IDS never triggers I would say in general the signatures work as expected.

  4. There could be some misconfiguration or missing optimizations between snort and Suricata.

What would help is to run it against Suricata with -r to reproduce it. I don’t remember if snort has the same feature.

Thanks for your quick input. Very useful and please pardon my newbie-ness (especially when it comes to rules design).

  1. Did you ensure to use the suricata specific signatures which differ a bit compared to the ones for snort, rule language is not 100% compatible.

Makes sense, I wrongly assumed that if sources are the same also rules are the same. Which indeed is not true, since same signature rule differs if, for example, taken from ET-Pro source for suricata.

  1. 2028651 for example uses threshold. And even besides that flow timeout values and stuff like that could explain that. So maybe suricata still sees the traffic in one flow and thus might just hit once but snort thinks it’s a new flow.

That’s interesting, I overlooked that.

  1. Since you don’t have the case where one IDS never triggers I would say in general the signatures work as expected.

This is what I was considering as well. Given also your previous info about flows and thresholds, I’ll know focus mainly in testing if there are no SIDs missing, not being strict about amount of related alerts.

  1. There could be some misconfiguration or missing optimizations between snort and Suricata.

Yep, for sure there are.

I’ll test same pcap with -r option just out of curiosity and move on with further testing.
Again, thanks a lot for your valuable input.

Davide

Is this using a publicly available pcap, or are you able to share the pcap? I find some of the differences rather large.

I suppose one thing to consider as well is that HOME_NET and other vars should be set up the same on both.

Hello Victor,

Thanks for weighing in.

That was internal prod traffic that I cannot share. Those vars are all set the same, the main difference in config is on the networking side (pfring vs af_packet and related tuning).