Suspected memleak in Suricata 6.0.3

Help
1

Hi all, recently our development team has been monitoring an installation with Suricata 6.0.3 and noticed a difference in behavior between that and Suricata 5.0.4 (previous installation).

memuse
memuse2551×1193 147 KB

We found that with 6.0.3 that tcp reassembly memuse is reaching the tcp reassembly memcap of 12GB With 5.0.4 it would not hit this cap. Unfortunately, after hitting this cap Suricata is unable to detect and alert on traffic that should be alert (tested with traffic that triggers ~2 per second resulted in ~0).

We think that this may be related to the following tickets:

After trying to understand these issues, we tried a build of 6.0.3 + the backports contained in Next/20211030/v3 by victorjulien · Pull Request #6539 · OISF/suricata · GitHub. Unfortunately there was not a notable improvement for our scenario with this patchset.

Is anyone aware of a potential resolution or mitigation to this, or if it may be resolved in an upcoming release?

2

This might be able to help (been fighting this for a bit) - but do you mind trying - Afpacket improvements/v29 by victorjulien · Pull Request #6578 · OISF/suricata · GitHub , it seems it can help improve things a lot in that direction - at least in my runs.

3

Thank you @pevma. We didn’t explore this avenue as we found flow: be sure to check hash till the end · OISF/suricata@085fe99 · GitHub over the weekend. This is in the 6.0.x branch (as well as master) and either solves or has a very positive impact on this memleak issue.

I guess my next question is whether a new Suricata 6.0.x patch release is imminent?

4

@jonathan I can highly recommend to try 6.0.4 since it has a lot of fixes related to that. For my setups it solved a lot of memory issues I had with 6.0.3

5

Yep, the fix for reassembly memleak was included in 6.0.4 and resolved our issues. :+1:

6

Hello, you can share Suricata Ymal configuration? Because we still have the same problem when we use Suricata 6.0.4 here - after reaching the upper limit of reassembly memcap, the packets reorganized by Suricata drop sharply