The environment I use suricata has a lot of http traffic.
So, it is judged that there is a problem with performance.
If I turn off http analysis, I want to know what side effects there will be.
(app-layer> http> enabled: no)
For example, can’t I use http-related options in snort rules?
(eg: http_method, http_uri…)
I wonder what other influences will be.
It will mean that all the rules that use alert http ... or any http_* rule keyword will be ineffective. It will also disable file extraction for http. I would recommend looking at improving performance in other ways, like more tuning, using better hardware, etc.
In my case, Suricata operates in about 20Gbps environment. Kernel drop occurs when the http analysis option is enabled. It seems to be affecting the performance a lot. Are there any options that may create a load during the settings below?
I am running suricata 5.0.1 version. It is used in IDS mode (sniffing), and only RX traffic is used. And I am using emerging-threat rule 4.0.
I’m using a napatech 10G x 4 card, and I’m using a 2.5GHz x 20 x 2 cpu (using hyperthreading totally 80 cores) and 512GB of memory.
Is there any additional information I need to provide?
In order to use a program (similar to tcpdump) that stores packets intermittently not continuously for collecting evidence, 64 streams are available for suricata because the program also uses napatech api and the maximum number of hostbuffer is 128
The cpu-affinity is also used the same as the number of streams.
You can use the Napatech capture tool to see what traffic is on the streams where the drops are occurring. You might find high speed elephant flows (as one example) on the stream(s) where the drops occur.
I am now using Suricata 6.0.1 and a napatech network accelerator card. However, there is a constant drop (stats.napa_total.overflow_drop) occurring.
I create 64 napatech streams, and use the suricata to map the cpu with the cpu-affinity setting. Looking at the cpu status, even if a drop occurs, not all cpu loads are 100%, and only one or two specific cpu continuously occupies 100%.
Likewise, when I look at napatech’s profile, drop occurs only in the streams mapped with the aforementioned 1~2 cpu. I think in this case, it is not a drop caused by a lot of traffic overall.
What are some possible causes?
I tried to check the Top IP by packet dumping the stream, but it is not easy. hardware-bypass is not enabled. Maybe invalid packets are dropped?
Also, I would like to know the source of the drop information that suricata shows. If the information is provided by napatech API, does it mean that dropped packets are only discarded from the stream buffer due to the slow processing speed of suricata? Isn’t it possible to see information that is dropped because suricata thinks it is an invalid packet?
2 CPUs, each with 20 physical cores. 80 when hyperthreading is enabled
Napatech 4x10G card
64 Napatech streams
1GB hostbuffer for each stream
Cores assigned to Suricata: 8-71
Napatech streams used: 8-71
Does the Napatech profiling utility show streams 0-7?
Can you post your Napatech default.ntpl file?
What firmware is loaded on the card?
Are you using the Napatech “flow matcher” (bypass or shunting) feature?
If only a few of the 64 streams are busy, there may be some high speed elephant flows … you can use the capture (napatech utility) to sample the packets from the associated streams to determine traffic/flow mix.