Suricata dropping return traffic due to Stream errors

Help
1

I have an OPNsense firewall running on 23.1.1_2 firmware. Suricata package is 6.0.9_1. I have the OPNsense OS installed on physical hardware. The LAN port is apart of a LAN bridge in OPNsense. In the eve.json logs all of the log lines are showing traffic is getting dropped with the reason being stream error. We have to turn IDS off on OPNsense as accessing web pages is so slow we’re unable to get any work done. Below is an output of the stats.log

Date: 3/31/2023 – 23:37:20 (uptime: 0d, 00h 00m 59s)

Counter | TM Name | Value

capture.kernel_packets | Total | 21966
decoder.pkts | Total | 21966
decoder.bytes | Total | 17920318
decoder.ipv4 | Total | 21879
decoder.ipv6 | Total | 3
decoder.ethernet | Total | 21966
decoder.tcp | Total | 17547
decoder.udp | Total | 4305
decoder.icmpv4 | Total | 30
decoder.avg_pkt_size | Total | 815
decoder.max_pkt_size | Total | 1514
flow.tcp | Total | 202
flow.udp | Total | 149
flow.wrk.spare_sync_avg | Total | 100
flow.wrk.spare_sync | Total | 5
flow.wrk.flows_evicted_needs_work | Total | 85
flow.wrk.flows_evicted_pkt_inject | Total | 145
flow.wrk.flows_injected | Total | 85
tcp.sessions | Total | 95
tcp.syn | Total | 95
tcp.synack | Total | 95
tcp.rst | Total | 118
tcp.overlap | Total | 6
app_layer.flow.http | Total | 3
app_layer.tx.http | Total | 4
app_layer.flow.tls | Total | 84
app_layer.flow.dcerpc_tcp | Total | 3
app_layer.tx.dcerpc_tcp | Total | 14
app_layer.flow.ntp | Total | 3
app_layer.tx.ntp | Total | 3
app_layer.flow.krb5_tcp | Total | 5
app_layer.tx.krb5_tcp | Total | 5
app_layer.flow.snmp | Total | 1
app_layer.tx.snmp | Total | 42
app_layer.flow.dns_udp | Total | 129
app_layer.tx.dns_udp | Total | 257
app_layer.flow.failed_udp | Total | 16
flow.mgr.full_hash_pass | Total | 1
flow.spare | Total | 9500
flow.mgr.rows_maxlen | Total | 1
flow.mgr.flows_checked | Total | 39
flow.mgr.flows_notimeout | Total | 39
tcp.memuse | Total | 1212416
tcp.reassembly_memuse | Total | 196608
flow.memuse | Total | 7074304

Any help would be greatly appreciated. Let me know if any further information is required.

2

Can you share the specific error messages?

3

Here is some additional information that may be applicable here. First, since you use the word “drop” in your problem description, I assume you actually have IPS (intrusion prevention mode) enabled and not IDS (intrusion detection mode). IPS blocks traffic by dropping it. IDS only generates alerts but does not block traffic.

With the above caveat out of the way, you need to know that Suricata’s IPS mode on OPNsense uses the netmap kernel device. The netmap device is not fully compatible with bridge interfaces in FreeBSD (which is the underlying operating system for OPNsense). The OPNsense team is currently working with another group to improve the netmap device support of bridge and LAGG interfaces in FreeBSD. If you visit the OPNsense forum website and search, you can some information on this. They have a test kernel you can install for OPNsense that includes the proposed netmap fixes for bridge and LAGG interfaces. You can visit their forum here: https://forum.opnsense.org/.

4 
{"timestamp":"2023-04-03T22:47:44.634128+0000","flow_id":1873045296818744,"in_iface":"igb1^","event_type":"drop","src_ip":"FirewallExternalIP","src_port":3236,"dest_ip":"RemoteHost","dest_port":443,"proto":"TCP","drop":{"len":40,"tos":0,"ttl":63,"ipid":0,"tcpseq":1454717774,"tcpack":181649289,"tcpwin":4096,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}
{"timestamp":"2023-04-03T22:47:44.903048+0000","flow_id":542198140555400,"in_iface":"igb1^","event_type":"drop","src_ip":"FirewallExternalIP","src_port":23073,"dest_ip":"RemoteHost","dest_port":443,"proto":"TCP","drop":{"len":52,"tos":0,"ttl":63,"ipid":0,"tcpseq":1751236717,"tcpack":441543970,"tcpwin":3247,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}
{"timestamp":"2023-04-03T22:47:45.147771+0000","flow_id":36077046682600,"in_iface":"igb1","event_type":"drop","src_ip":"RemoteHost","src_port":443,"dest_ip":"FirewallExternalIP","dest_port":34591,"proto":"TCP","drop":{"len":52,"tos":0,"ttl":244,"ipid":30711,"tcpseq":1621020740,"tcpack":1713982378,"tcpwin":193,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}
{"timestamp":"2023-04-03T22:47:45.236164+0000","flow_id":1873045296818744,"in_iface":"igb1^","event_type":"drop","src_ip":"FirewallExternalIP","src_port":3236,"dest_ip":"RemoteHost","dest_port":443,"proto":"TCP","drop":{"len":40,"tos":0,"ttl":63,"ipid":0,"tcpseq":1454717774,"tcpack":181649289,"tcpwin":4096,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}
{"timestamp":"2023-04-03T22:47:45.378384+0000","flow_id":542198140555400,"in_iface":"igb1^","event_type":"drop","src_ip":"FirewallExternalIP","src_port":23073,"dest_ip":"RemoteHost","dest_port":443,"proto":"TCP","drop":{"len":52,"tos":0,"ttl":63,"ipid":0,"tcpseq":1751236717,"tcpack":441543970,"tcpwin":3247,"syn":false,"ack":true,"psh":false,"rst":false,"urg":false,"fin":true,"tcpres":0,"tcpurgp":0,"reason":"stream error"}}
5

Hey Bill,

That’s what I was thinking as the documentation in OPNsense says you must use physical adapters. However, I just tried turning on the IPS for the WAN interface, which is a physical adapter and received quite a few drop packets relating to stream errors.

6

The stream rules are generally meant to be “informational” in nature and do not necessarily indicate malicous activity when triggering. In most setups those rules are not set to DROP traffic. Instead, their action is left at ALERT which will produce a log entry but not drop the traffic.

I am not an OPNsense user, so I don’t know the specifics of how the rule actions are set, but you would generally not want to set the action for stream events rule to DROP. By default they are set to ALERT. If you are unable to set those rules for ALERT only, then you might consider disabling them (or at least disabling those which are giving you problems).

And you may have already checked this, but be sure any hardware offloading is disabled on the interface in question. That means hardware checkum offload and any other offloading options available for the NIC hardware.

7

Wanted to follow up on this issue. I updated my firewall last night to OPNsense version 23.1.5_4. I then turned the Intrusion detection service back on and have not seen any stream error drops so far. The service has also been running for an hour. Just wanted to post this incase anyone else has similar issues.