Suricata dropping return traffic due to Stream errors

Rhino · March 31, 2023, 4:11pm

I have an OPNsense firewall running on 23.1.1_2 firmware. Suricata package is 6.0.9_1. I have the OPNsense OS installed on physical hardware. The LAN port is apart of a LAN bridge in OPNsense. In the eve.json logs all of the log lines are showing traffic is getting dropped with the reason being stream error. We have to turn IDS off on OPNsense as accessing web pages is so slow we’re unable to get any work done. Below is an output of the stats.log

Date: 3/31/2023 – 23:37:20 (uptime: 0d, 00h 00m 59s)

Counter | TM Name | Value

Any help would be greatly appreciated. Let me know if any further information is required.

Jeff_Lucovsky · April 2, 2023, 12:39pm

Can you share the specific error messages?

bmeeks · April 3, 2023, 1:10pm

Here is some additional information that may be applicable here. First, since you use the word “drop” in your problem description, I assume you actually have IPS (intrusion prevention mode) enabled and not IDS (intrusion detection mode). IPS blocks traffic by dropping it. IDS only generates alerts but does not block traffic.

With the above caveat out of the way, you need to know that Suricata’s IPS mode on OPNsense uses the netmap kernel device. The netmap device is not fully compatible with bridge interfaces in FreeBSD (which is the underlying operating system for OPNsense). The OPNsense team is currently working with another group to improve the netmap device support of bridge and LAGG interfaces in FreeBSD. If you visit the OPNsense forum website and search, you can some information on this. They have a test kernel you can install for OPNsense that includes the proposed netmap fixes for bridge and LAGG interfaces. You can visit their forum here: https://forum.opnsense.org/.

Rhino · April 3, 2023, 2:40pm

{"timestamp":"2023-04-03T22:47:44.634128+0000","flow_id":1873045296818744,"in_iface":"igb1^","event_type":"drop","src_ip":"FirewallExternalIP","src_port":3236,"dest_ip":"RemoteHost","dest_port":443,"proto":"TCP","drop":{"len":40,"tos":0,"ttl":63,"ipid":0,"tcpseq":1454717774,"tcpack":181649289,"tcpwin":4096,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}
{"timestamp":"2023-04-03T22:47:44.903048+0000","flow_id":542198140555400,"in_iface":"igb1^","event_type":"drop","src_ip":"FirewallExternalIP","src_port":23073,"dest_ip":"RemoteHost","dest_port":443,"proto":"TCP","drop":{"len":52,"tos":0,"ttl":63,"ipid":0,"tcpseq":1751236717,"tcpack":441543970,"tcpwin":3247,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}
{"timestamp":"2023-04-03T22:47:45.147771+0000","flow_id":36077046682600,"in_iface":"igb1","event_type":"drop","src_ip":"RemoteHost","src_port":443,"dest_ip":"FirewallExternalIP","dest_port":34591,"proto":"TCP","drop":{"len":52,"tos":0,"ttl":244,"ipid":30711,"tcpseq":1621020740,"tcpack":1713982378,"tcpwin":193,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}
{"timestamp":"2023-04-03T22:47:45.236164+0000","flow_id":1873045296818744,"in_iface":"igb1^","event_type":"drop","src_ip":"FirewallExternalIP","src_port":3236,"dest_ip":"RemoteHost","dest_port":443,"proto":"TCP","drop":{"len":40,"tos":0,"ttl":63,"ipid":0,"tcpseq":1454717774,"tcpack":181649289,"tcpwin":4096,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}
{"timestamp":"2023-04-03T22:47:45.378384+0000","flow_id":542198140555400,"in_iface":"igb1^","event_type":"drop","src_ip":"FirewallExternalIP","src_port":23073,"dest_ip":"RemoteHost","dest_port":443,"proto":"TCP","drop":{"len":52,"tos":0,"ttl":63,"ipid":0,"tcpseq":1751236717,"tcpack":441543970,"tcpwin":3247,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}

Rhino · April 3, 2023, 2:43pm

Hey Bill,

That’s what I was thinking as the documentation in OPNsense says you must use physical adapters. However, I just tried turning on the IPS for the WAN interface, which is a physical adapter and received quite a few drop packets relating to stream errors.

bmeeks · April 3, 2023, 5:12pm

The stream rules are generally meant to be “informational” in nature and do not necessarily indicate malicous activity when triggering. In most setups those rules are not set to DROP traffic. Instead, their action is left at ALERT which will produce a log entry but not drop the traffic.

I am not an OPNsense user, so I don’t know the specifics of how the rule actions are set, but you would generally not want to set the action for stream events rule to DROP. By default they are set to ALERT. If you are unable to set those rules for ALERT only, then you might consider disabling them (or at least disabling those which are giving you problems).

And you may have already checked this, but be sure any hardware offloading is disabled on the interface in question. That means hardware checkum offload and any other offloading options available for the NIC hardware.

Rhino · April 4, 2023, 4:13pm

Wanted to follow up on this issue. I updated my firewall last night to OPNsense version 23.1.5_4. I then turned the Intrusion detection service back on and have not seen any stream error drops so far. The service has also been running for an hour. Just wanted to post this incase anyone else has similar issues.

Topic		Replies	Views
Blocking traffic after upgrade Help	3	484	July 24, 2023
Suricata 7 drops my flows - reason: applayer error Help	4	438	October 20, 2023
Dropping Traffic Using Suricata with XDP Help ips , suricata , xdp	1	84	April 25, 2024
Http traffic partially dropped Help	1	1670	January 26, 2022
Suricata http traffic parse exception in mirror traffic Help	5	762	January 25, 2022

Suricata dropping return traffic due to Stream errors

Date: 3/31/2023 – 23:37:20 (uptime: 0d, 00h 00m 59s)

Counter | TM Name | Value

Related Topics