Accumulation of flow-timeout inuse

Help
1

I am running suricata with napatech card.
And when checking the stat log while checking the cause due to kernel drop, the value of “stats.flow.mgr.flows_timeout_inuse” gradually increased.

If the value increases over a certain level, kernel drop continues to occur, and SSN drop seems to occur after that.
So I would like to know how to reduce “flow-timout inuse”.

flow-timeouts:
default:
new: 30
established: 60
closed: 0
bypassed: 100
emergency-new: 10
emergency-established: 100
emergency-closed: 0
emergency-bypassed: 50
tcp:
new: 30
established: 60
closed: 0
bypassed: 100
emergency-new: 1
emergency-established: 100
emergency-closed: 10
emergency-bypassed: 50
udp:
new: 30
established: 60
bypassed: 100
emergency-new: 10
emergency-established: 100
emergency-bypassed: 50
icmp:
new: 30
established: 60
bypassed: 100
emergency-new: 10
emergency-established: 100
emergency-bypassed: 50

2

Could you add some more details about your setup, especially if there is something that stands out with regards to the traffic.

3

Among the current stat items, there are three stats with accomodation.
(flow memuse, flow timeout inuse, http memuse)

The “flow memuse” increases steadily over several days, and when it reaches memcap, a drop occurs, or if the memcap is not set, all hardware memory is exhausted and the process dies.

I guess the cause of this problem is an increase in “flow-timeout-inuse”. So, I wonder what kind of settings are there. The flow timeout-related settings currently in use are attached below.

Another is the increase in “http memuse”. It increases steadily over several days. This also attaches app-layer related settings below.

Please tell me if there are any additional settings you need to show.

app-layer:    
	http:
      enabled: yes
      libhtp:
         default-config:
           personality: IDS
		   
           request-body-limit: 100kb
           response-body-limit: 100kb

           # inspection limits
           request-body-minimal-inspect-size: 32kb
           request-body-inspect-window: 4kb
           response-body-minimal-inspect-size: 40kb
           response-body-inspect-window: 16kb

           # response body decompression (0 disables)
           response-body-decompress-layer-limit: 2

           # auto will use http-body-inline mode in IPS mode, yes or no set it statically
           http-body-inline: no
		   
           # Decompress SWF files
           swf-decompression:
             enabled: no
             type: both
             compress-depth: 0
             decompress-depth: 0

           # decoding
           double-decode-path: no
           double-decode-query: no

flow-timeouts:
  default:
    new: 30
    established: 60
    closed: 60
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-closed: 10
    emergency-bypassed: 50
  tcp:
    new: 30
    established: 60
    closed: 60
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-closed: 10
    emergency-bypassed: 50
  udp:
    new: 30
    established: 60
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-bypassed: 50
  icmp:
    new: 30
    established: 60
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-bypassed: 50
4

What is the flow section settings in the yaml you use (that section - https://github.com/OISF/suricata/blob/master/suricata.yaml.in#L1179)
And how much traffic/what mix are you throwing at it ?

5

It is in/out traffic of up to 20GB/s, mostly http and https traffic, and many other types of protocol communication occur. And I’m using suricata in IDS mode
The flow configuration is as follows:

flow:
  memcap: 150gb
  hash-size: 566231040
  prealloc: 288000000
  emergency-recovery: 30
  managers: 2
  recyclers: 2
6

Those are excessive settings i think.
I suggest you lower those by at least 20-fold.
Maybe use more aggressive timeouts if needed.
Can you please share a stats.log update?

7

I’ll try to reduce those value and monitor the change.
And I’m only using json log, can I share it as below?

{
  "_index": "suri-stats-2021.01.05",
  "_type": "stats",
  "_id": "XM_-z3YBPb4PvSR7xoXo",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2021-01-05T00:42:29.364Z",
    "event_type": "stats",
    "stats": {
      "detect": {
        "mpm_list": 1,
        "nonmpm_list_delta": 0,
        "match_list": 27,
        "alert": 184292460,
        "nonmpm_list": 47,
        "fnonmpm_list_delta": 0,
        "engines": [
          {
            "rules_loaded": 17692,
            "rules_failed": 72,
            "id": 0,
            "last_reload": "2021-01-05T04:03:50.240094+0900"
          }
        ],
        "match_list_delta": 0,
        "alert_delta": 487,
        "mpm_list_delta": 0,
        "fnonmpm_list": 26
      },
      "uptime": 1189140,
      "tcp": {
        "insert_data_normal_fail": 1458583895163,
        "no_flow": 0,
        "rst": 37360926795,
        "overlap": 365292728,
        "insert_data_normal_fail_delta": 4179786,
        "synack": 54090989407,
        "invalid_checksum_delta": 33,
        "sessions_delta": 137647,
        "pseudo_delta": 0,
        "stream_depth_reached": 12072050,
        "memuse": 23631712704,
        "insert_data_overlap_fail": 765949,
        "midstream_pickups_delta": 0,
        "overlap_diff_data": 16977,
        "ssn_memcap_drop_delta": 0,
        "insert_list_fail": 467,
        "syn": 55103529135,
        "pseudo": 535336,
        "memuse_delta": 343360,
        "midstream_pickups": 0,
        "reassembly_gap_delta": 107020,
        "insert_data_overlap_fail_delta": 1,
        "segment_memcap_drop_delta": 0,
        "ssn_memcap_drop": 0,
        "overlap_delta": 44,
        "reassembly_memuse_delta": -8252,
        "no_flow_delta": 0,
        "pkt_on_wrong_thread": 0,
        "segment_memcap_drop": 0,
        "syn_delta": 152928,
        "pkt_on_wrong_thread_delta": 0,
        "reassembly_gap": 200765354292,
        "sessions": 50088862412,
        "rst_delta": 103850,
        "pseudo_failed": 0,
        "synack_delta": 148046,
        "reassembly_memuse": 42949674944,
        "invalid_checksum": 13093456,
        "insert_list_fail_delta": 0,
        "stream_depth_reached_delta": 44,
        "pseudo_failed_delta": 0,
        "overlap_diff_data_delta": 0
      },
      "app_layer": {
        "tx": {
          "rfb": 3053,
          "sip": 19013057,
          "nfs_tcp": 84,
          "ftp-data": 0,
          "tftp_delta": 11,
          "nfs_udp": 0,
          "sip_delta": 6,
          "http2": 50296,
          "rdp": 1038086,
          "rfb_delta": 0,
          "tls_delta": 0,
          "ftp_delta": 0,
          "http_delta": 173,
          "imap_delta": 0,
          "dns_tcp": 7416771,
          "tftp": 1359545,
          "mqtt_delta": 0,
          "nfs_tcp_delta": 0,
          "dns_udp_delta": 27609,
          "dns_udp": 12355341290,
          "ssh": 0,
          "rdp_delta": 1,
          "nfs_udp_delta": 0,
          "ftp": 164408,
          "tls": 0,
          "http": 303707041,
          "smb_delta": 0,
          "dns_tcp_delta": 2,
          "ftp-data_delta": 0,
          "mqtt": 8339,
          "http2_delta": 0,
          "imap": 0,
          "ssh_delta": 0,
          "smb": 3366053
        },
        "flow": {
          "rfb": 29,
          "sip": 17769326,
          "nfs_tcp": 59,
          "ftp-data": 464,
          "tftp_delta": 11,
          "nfs_udp": 0,
          "sip_delta": 0,
          "http2": 287,
          "failed_tcp": 85168540,
          "rdp": 46224,
          "failed_udp": 350405832,
          "rfb_delta": 0,
          "tls_delta": 5,
          "ftp_delta": 0,
          "http_delta": 1,
          "imap_delta": 0,
          "dns_tcp": 114358,
          "tftp": 1427380,
          "mqtt_delta": 0,
          "nfs_tcp_delta": 0,
          "dns_udp_delta": 11627,
          "dns_udp": 4714813112,
          "ssh": 28241,
          "rdp_delta": 0,
          "nfs_udp_delta": 0,
          "ftp": 2866,
          "tls": 33494974,
          "http": 10604346,
          "smb_delta": 0,
          "dns_tcp_delta": 0,
          "failed_udp_delta": 658,
          "ftp-data_delta": 0,
          "mqtt": 87,
          "http2_delta": 0,
          "failed_tcp_delta": 15,
          "imap": 0,
          "ssh_delta": 0,
          "smb": 157091
        },
        "expectations": 0,
        "expectations_delta": 0
      },
      "napa_dispatch_drop": {
        "byte_delta": 0,
        "pkts_delta": 0,
        "pkts": 0,
        "byte": 0
      },
      "ftp": {
        "memuse": 28033,
        "memuse_delta": 0,
        "memcap": 0,
        "memcap_delta": 0
      },
      "http": {
        "memuse": 2959411320,
        "memuse_delta": -313610,
        "memcap": 0,
        "memcap_delta": 0
      },
      "decoder": {
        "ppp": 0,
        "gre": 19569,
        "ipv4_in_ipv6": 0,
        "chdlc_delta": 0,
        "sll_delta": 0,
        "ipv4": 4174402635611,
        "vlan_qinq": 0,
        "ieee8021ah_delta": 0,
        "max_mac_addrs_dst": 0,
        "vxlan_delta": 0,
        "vlan_delta": 0,
        "ethernet_delta": 12221187,
        "invalid": 4938219,
        "ipv4_delta": 12221061,
        "tcp_delta": 10762512,
        "invalid_delta": 1,
        "pppoe": 0,
        "ipv4_in_ipv6_delta": 0,
        "pkts": 4174186637417,
        "ethernet": 4174186637417,
        "erspan_delta": 0,
        "gre_delta": 0,
        "raw_delta": 0,
        "udp_delta": 1065503,
        "max_pkt_size": 1518,
        "sll": 0,
        "icmpv6": 29786574,
        "pppoe_delta": 0,
        "bytes_delta": 5430844000,
        "teredo": 423405,
        "ppp_delta": 0,
        "vxlan": 20391,
        "bytes": 1655163802565761,
        "null_delta": 0,
        "raw": 0,
        "geneve_delta": 0,
        "ieee8021ah": 0,
        "ipv6_in_ipv6": 0,
        "icmpv4_delta": 7926,
        "tcp": 4032788413050,
        "teredo_delta": 0,
        "max_mac_addrs_dst_delta": 0,
        "event": {
          "ppp": {
            "unsup_proto": 0,
            "pkt_too_small": 0,
            "ip6_pkt_too_small": 0,
            "wrong_type_delta": 0,
            "vju_pkt_too_small_delta": 0,
            "ip4_pkt_too_small_delta": 0,
            "unsup_proto_delta": 0,
            "vju_pkt_too_small": 0,
            "ip6_pkt_too_small_delta": 0,
            "ip4_pkt_too_small": 0,
            "pkt_too_small_delta": 0,
            "wrong_type": 0
          },
          "gre": {
            "version0_malformed_sre_hdr": 0,
            "version1_route": 0,
            "version1_recur_delta": 0,
            "version0_flags_delta": 0,
            "version1_flags": 0,
            "version1_route_delta": 0,
            "version1_malformed_sre_hdr": 0,
            "wrong_version": 112,
            "version0_recur_delta": 0,
            "version0_hdr_too_big_delta": 0,
            "version0_malformed_sre_hdr_delta": 0,
            "version1_recur": 0,
            "version0_recur": 0,
            "version0_flags": 0,
            "version1_flags_delta": 0,
            "version1_no_key": 0,
            "version1_chksum_delta": 0,
            "wrong_version_delta": 0,
            "version1_chksum": 0,
            "pkt_too_small": 0,
            "version1_ssr_delta": 0,
            "version0_hdr_too_big": 0,
            "version1_wrong_protocol_delta": 0,
            "version1_hdr_too_big_delta": 0,
            "version1_malformed_sre_hdr_delta": 0,
            "version1_hdr_too_big": 0,
            "version1_no_key_delta": 0,
            "version1_ssr": 0,
            "version1_wrong_protocol": 0,
            "pkt_too_small_delta": 0
          },
          "tcp": {
            "opt_duplicate": 25,
            "opt_duplicate_delta": 0,
            "pkt_too_small": 0,
            "hlen_too_small_delta": 1,
            "opt_invalid_len": 5571982,
            "hlen_too_small": 17617,
            "opt_invalid_len_delta": 0,
            "invalid_optlen_delta": 0,
            "pkt_too_small_delta": 0,
            "invalid_optlen": 874
          },
          "ipv6": {
            "exthdr_useless_fh_delta": 0,
            "exthdr_dupl_ah": 0,
            "exthdr_useless_fh": 0,
            "hopopts_unknown_opt": 0,
            "zero_len_padn": 16,
            "frag_ignored": 0,
            "trunc_pkt": 0,
            "exthdr_dupl_fh_delta": 0,
            "exthdr_dupl_rh": 0,
            "exthdr_ah_res_not_null": 62,
            "exthdr_invalid_optlen_delta": 0,
            "ipv6_in_ipv6_wrong_version_delta": 0,
            "exthdr_dupl_hh": 0,
            "ipv4_in_ipv6_wrong_version": 1766,
            "fh_non_zero_reserved_field_delta": 0,
            "data_after_none_header_delta": 0,
            "unknown_next_header": 408301,
            "pkt_too_small": 0,
            "frag_overlap": 0,
            "dstopts_unknown_opt": 0,
            "frag_ignored_delta": 0,
            "dstopts_unknown_opt_delta": 0,
            "ipv6_in_ipv6_too_small": 0,
            "unknown_next_header_delta": 0,
            "ipv6_in_ipv6_too_small_delta": 0,
            "dstopts_only_padding_delta": 0,
            "rh_type_0_delta": 0,
            "exthdr_dupl_fh": 0,
            "exthdr_invalid_optlen": 0,
            "trunc_exthdr_delta": 0,
            "data_after_none_header": 3674,
            "hopopts_only_padding": 0,
            "pkt_too_small_delta": 0,
            "icmpv4_delta": 0,
            "rh_type_0": 1,
            "ipv4_in_ipv6_too_small": 0,
            "dstopts_only_padding": 0,
            "trunc_exthdr": 0,
            "zero_len_padn_delta": 0,
            "wrong_ip_version": 0,
            "hopopts_only_padding_delta": 0,
            "frag_pkt_too_large_delta": 0,
            "ipv4_in_ipv6_wrong_version_delta": 0,
            "icmpv4": 2026,
            "frag_overlap_delta": 0,
            "exthdr_dupl_rh_delta": 0,
            "exthdr_dupl_ah_delta": 0,
            "fh_non_zero_reserved_field": 1949,
            "frag_pkt_too_large": 1,
            "wrong_ip_version_delta": 0,
            "exthdr_dupl_dh_delta": 0,
            "exthdr_dupl_eh": 0,
            "hopopts_unknown_opt_delta": 0,
            "exthdr_dupl_dh": 0,
            "ipv6_in_ipv6_wrong_version": 1647,
            "exthdr_dupl_hh_delta": 0,
            "exthdr_ah_res_not_null_delta": 0,
            "trunc_pkt_delta": 0,
            "exthdr_dupl_eh_delta": 0,
            "ipv4_in_ipv6_too_small_delta": 0
          },
          "ipraw": {
            "invalid_ip_version": 0,
            "invalid_ip_version_delta": 0
          },
          "ipv4": {
            "opt_duplicate_delta": 0,
            "hlen_too_small_delta": 0,
            "opt_invalid_len": 0,
            "frag_pkt_too_large_delta": 0,
            "frag_ignored": 0,
            "wrong_ip_version": 0,
            "trunc_pkt": 0,
            "opt_pad_required_delta": 0,
            "frag_overlap_delta": 0,
            "hlen_too_small": 0,
            "opt_eol_required_delta": 0,
            "frag_pkt_too_large": 0,
            "iplen_smaller_than_hlen_delta": 0,
            "wrong_ip_version_delta": 0,
            "opt_malformed": 0,
            "pkt_too_small": 0,
            "opt_invalid_len_delta": 0,
            "frag_overlap": 7325,
            "frag_ignored_delta": 0,
            "opt_pad_required": 19018,
            "icmpv6_delta": 0,
            "icmpv6": 0,
            "opt_unknown": 0,
            "opt_malformed_delta": 0,
            "opt_duplicate": 0,
            "opt_eol_required": 0,
            "trunc_pkt_delta": 0,
            "opt_unknown_delta": 0,
            "iplen_smaller_than_hlen": 0,
            "opt_invalid_delta": 0,
            "pkt_too_small_delta": 0,
            "opt_invalid": 0
          },
          "ltnull": {
            "unsupported_type_delta": 0,
            "pkt_too_small": 0,
            "unsupported_type": 0,
            "pkt_too_small_delta": 0
          },
          "sctp": {
            "pkt_too_small_delta": 0,
            "pkt_too_small": 0
          },
          "udp": {
            "pkt_too_small": 1258,
            "hlen_too_small_delta": 0,
            "hlen_too_small": 0,
            "hlen_invalid_delta": 0,
            "hlen_invalid": 89927,
            "pkt_too_small_delta": 0
          },
          "sll": {
            "pkt_too_small_delta": 0,
            "pkt_too_small": 0
          },
          "icmpv6": {
            "pkt_too_small": 0,
            "experimentation_type": 33,
            "mld_message_with_invalid_hl_delta": 0,
            "unknown_type": 19,
            "unknown_code": 235,
            "ipv6_trunc_pkt": 0,
            "ipv6_unknown_version": 0,
            "ipv6_trunc_pkt_delta": 0,
            "experimentation_type_delta": 0,
            "unknown_type_delta": 0,
            "mld_message_with_invalid_hl": 22,
            "unknown_code_delta": 0,
            "ipv6_unknown_version_delta": 0,
            "unassigned_type_delta": 0,
            "pkt_too_small_delta": 0,
            "unassigned_type": 1615
          },
          "pppoe": {
            "wrong_code": 0,
            "pkt_too_small": 0,
            "wrong_code_delta": 0,
            "malformed_tags": 0,
            "malformed_tags_delta": 0,
            "pkt_too_small_delta": 0
          },
          "icmpv4": {
            "ipv4_unknown_ver": 545,
            "pkt_too_small": 0,
            "unknown_type_delta": 0,
            "unknown_type": 1046,
            "ipv4_trunc_pkt": 93,
            "unknown_code_delta": 0,
            "ipv4_trunc_pkt_delta": 0,
            "ipv4_unknown_ver_delta": 0,
            "pkt_too_small_delta": 0,
            "unknown_code": 67067
          },
          "ethernet": {
            "pkt_too_small_delta": 0,
            "pkt_too_small": 0
          },
          "mpls": {
            "bad_label_router_alert_delta": 0,
            "header_too_small_delta": 0,
            "pkt_too_small": 0,
            "unknown_payload_type": 0,
            "bad_label_implicit_null": 0,
            "bad_label_reserved": 792572,
            "bad_label_implicit_null_delta": 0,
            "bad_label_reserved_delta": 2,
            "unknown_payload_type_delta": 0,
            "bad_label_router_alert": 0,
            "header_too_small": 0,
            "pkt_too_small_delta": 0
          },
          "vxlan": {
            "unknown_payload_type_delta": 0,
            "unknown_payload_type": 20391
          },
          "dce": {
            "pkt_too_small_delta": 0,
            "pkt_too_small": 0
          },
          "vlan": {
            "header_too_small_delta": 0,
            "header_too_small": 0,
            "unknown_type_delta": 0,
            "too_many_layers_delta": 0,
            "unknown_type": 0,
            "too_many_layers": 0
          },
          "ieee8021ah": {
            "header_too_small_delta": 0,
            "header_too_small": 0
          },
          "geneve": {
            "unknown_payload_type_delta": 0,
            "unknown_payload_type": 7
          },
          "erspan": {
            "unsupported_version_delta": 0,
            "header_too_small_delta": 0,
            "header_too_small": 0,
            "too_many_vlan_layers": 0,
            "unsupported_version": 0,
            "too_many_vlan_layers_delta": 0
          }
        },
        "avg_pkt_size_delta": 0,
        "avg_pkt_size": 396,
        "vlan_qinq_delta": 0,
        "icmpv4": 1799473696,
        "chdlc": 0,
        "sctp_delta": 0,
        "vlan": 0,
        "erspan": 0,
        "null": 0,
        "ipv6": 34248934,
        "icmpv6_delta": 98,
        "max_pkt_size_delta": 0,
        "sctp": 1669,
        "ipv6_delta": 101,
        "udp": 107448744567,
        "pkts_delta": 12221187,
        "mpls": 792572,
        "ipv6_in_ipv6_delta": 0,
        "mpls_delta": 2,
        "max_mac_addrs_src": 0,
        "geneve": 7,
        "max_mac_addrs_src_delta": 0
      },
      "flow": {
        "tcp_reuse": 3121200115,
        "icmpv4_delta": 387,
        "tcp": 53844794460,
        "wrk": {
          "flows_injected": 3094144179,
          "spare_sync_avg_delta": 0,
          "flows_evicted_pkt_inject": 7934399408,
          "spare_sync_delta": 1441,
          "spare_sync": 507668504,
          "flows_evicted_needs_work": 4600631779,
          "flows_evicted_pkt_inject_delta": 17096,
          "spare_sync_incomplete": 0,
          "spare_sync_empty": 0,
          "spare_sync_incomplete_delta": 0,
          "flows_injected_delta": 4400,
          "flows_evicted_delta": 10951,
          "spare_sync_avg": 100,
          "flows_evicted_needs_work_delta": 8942,
          "flows_evicted": 5162436670,
          "spare_sync_empty_delta": 0
        },
        "get_used_eval_reject_delta": 0,
        "memuse": 157094507200,
        "get_used_eval_busy_delta": 0,
        "get_used_failed_delta": 0,
        "get_used_failed": 0,
        "tcp_delta": 147216,
        "icmpv4": 93070745,
        "memcap_delta": 0,
        "spare_delta": -48853,
        "memuse_delta": 608000,
        "mgr": {
          "bypassed_pruned": 0,
          "flows_evicted_needs_work_delta": 4400,
          "flows_timeout_delta": 119269,
          "est_pruned_delta": 0,
          "flows_checked": 68763980595,
          "closed_pruned": 0,
          "closed_pruned_delta": 0,
          "full_hash_pass_delta": 0,
          "rows_maxlen": 1758,
          "flows_evicted_delta": 120494,
          "flows_notimeout": 15553368415,
          "full_hash_pass": 9910,
          "est_pruned": 0,
          "bypassed_pruned_delta": 0,
          "flows_timeout_inuse_delta": 2,
          "rows_maxlen_delta": 0,
          "flows_timeout": 53210612180,
          "flows_timeout_inuse": 419085,
          "flows_evicted_needs_work": 3094144179,
          "flows_notimeout_delta": 34691,
          "flows_checked_delta": 153960,
          "new_pruned_delta": 0,
          "new_pruned": 0,
          "flows_evicted": 53771228003
        },
        "udp_delta": 12296,
        "emerg_mode_entered": 0,
        "icmpv6_delta": 2,
        "memcap": 0,
        "udp": 5084415650,
        "get_used_delta": 0,
        "icmpv6": 1017572,
        "get_used_eval_busy": 0,
        "emerg_mode_over": 0,
        "emerg_mode_over_delta": 0,
        "tcp_reuse_delta": 6870,
        "get_used": 0,
        "get_used_eval_delta": 0,
        "spare": 576016986,
        "get_used_eval_reject": 0,
        "get_used_eval": 0,
        "emerg_mode_entered_delta": 0
      },
      "defrag": {
        "max_frag_hits": 0,
        "ipv6": {
          "reassembled": 0,
          "reassembled_delta": 0,
          "fragments": 1963,
          "timeouts_delta": 0,
          "timeouts": 0,
          "fragments_delta": 0
        },
        "max_frag_hits_delta": 0,
        "ipv4": {
          "reassembled": 389355412,
          "reassembled_delta": 294,
          "fragments": 1015130940,
          "timeouts_delta": 0,
          "timeouts": 0,
          "fragments_delta": 702
        }
      },
      "napa_dispatch_host": {
        "byte_delta": 5412768168,
        "pkts_delta": 12206587,
        "pkts": 4175379867139,
        "byte": 1655725854236219
      },
      "stream": {
        "est_invalid_ack_delta": 249,
        "3whs_syn_toclient_on_syn_recv_delta": 0,
        "shutdown_syn_resend_delta": 65,
        "est_packet_out_of_window_delta": 0,
        "3whs_synack_with_wrong_ack": 690031,
        "3whs_syn_resend_diff_seq_on_syn_recv_delta": 24,
        "3whs_wrong_seq_wrong_ack_delta": 7,
        "est_synack_resend_with_diff_ack_delta": 57,
        "pkt_invalid_ack_delta": 483,
        "reassembly_seq_gap": 200765354289,
        "rst_invalid_ack_delta": 15,
        "reassembly_overlap_different_data_delta": 0,
        "closewait_fin_out_of_window_delta": 4,
        "rst_but_no_session_delta": 3648,
        "est_synack_toserver": 824182,
        "est_invalid_ack": 237463862,
        "lastack_invalid_ack": 486371,
        "3whs_async_wrong_seq_delta": 0,
        "4whs_synack_with_wrong_ack": 0,
        "fin_but_no_session": 3795679661,
        "pkt_retransmission_delta": 2959,
        "fin2_invalid_ack": 33317480,
        "3whs_synack_resend_with_diff_ack_delta": 10,
        "closewait_ack_out_of_window": 6830989,
        "lastack_ack_wrong_seq": 101453,
        "pkt_invalid_timestamp": 328272478,
        "pkt_broken_ack_delta": 1444,
        "fin2_fin_wrong_seq_delta": 28,
        "3whs_right_seq_wrong_ack_evasion_delta": 0,
        "3whs_synack_in_wrong_direction": 1838838,
        "timewait_invalid_ack_delta": 3,
        "est_synack_resend": 938561,
        "est_syn_resend_diff_seq_delta": 54,
        "shutdown_syn_resend": 21109380,
        "fin2_fin_wrong_seq": 6534842,
        "3whs_synack_resend_with_diff_seq_delta": 0,
        "rst_invalid_ack": 6336881,
        "est_synack_resend_with_diff_ack": 46092675,
        "closing_invalid_ack": 0,
        "3whs_synack_toserver_on_syn_recv": 854909,
        "3whs_synack_toserver_on_syn_recv_delta": 0,
        "pkt_invalid_ack": 326622791,
        "reassembly_no_segment": 0,
        "est_syn_resend_delta": 29,
        "3whs_synack_resend_with_diff_ack": 7065447,
        "reassembly_overlap_different_data": 16977,
        "3whs_ack_data_inject_delta": 0,
        "fin_out_of_window": 475356,
        "timewait_invalid_ack": 1124981,
        "fin1_invalid_ack": 31264177,
        "3whs_synack_resend_with_diff_seq": 0,
        "3whs_syn_toclient_on_syn_recv": 0,
        "3whs_synack_flood": 267959,
        "est_syn_resend": 2937069,
        "fin2_invalid_ack_delta": 86,
        "fin2_ack_wrong_seq_delta": 0,
        "lastack_ack_wrong_seq_delta": 0,
        "pkt_broken_ack": 396549041,
        "4whs_synack_with_wrong_ack_delta": 0,
        "pkt_bad_window_update_delta": 11,
        "fin1_fin_wrong_seq": 5276409,
        "est_pkt_before_last_ack_delta": 1769,
        "3whs_ack_in_wrong_dir_delta": 0,
        "fin1_ack_wrong_seq": 8421,
        "lastack_invalid_ack_delta": 0,
        "3whs_synack_with_wrong_ack_delta": 1,
        "3whs_ack_data_inject": 0,
        "est_pkt_before_last_ack": 713664761,
        "est_synack_resend_with_diff_seq": 291994,
        "est_syn_toclient_delta": 0,
        "est_synack_resend_with_diff_seq_delta": 0,
        "pkt_retransmission": 1266466402,
        "timewait_ack_wrong_seq": 15820376,
        "reassembly_segment_before_base_seq_delta": 0,
        "closewait_fin_out_of_window": 5999973,
        "suspected_rst_inject_delta": 0,
        "reassembly_segment_before_base_seq": 467,
        "3whs_synack_in_wrong_direction_delta": 0,
        "fin1_fin_wrong_seq_delta": 9,
        "fin_but_no_session_delta": 11092,
        "suspected_rst_inject": 25663,
        "closing_ack_wrong_seq": 0,
        "pkt_bad_window_update": 9721879,
        "reassembly_seq_gap_delta": 107020,
        "est_synack_resend_delta": 0,
        "est_synack_toserver_delta": 0,
        "3whs_async_wrong_seq": 0,
        "rst_but_no_session": 1422397284,
        "wrong_thread_delta": 0,
        "3whs_synack_flood_delta": 1,
        "est_syn_toclient": 0,
        "closewait_invalid_ack": 2122431,
        "3whs_syn_resend_diff_seq_on_syn_recv": 9101640,
        "closewait_ack_out_of_window_delta": 67,
        "4whs_synack_with_wrong_syn_delta": 0,
        "timewait_ack_wrong_seq_delta": 53,
        "4whs_invalid_ack_delta": 0,
        "3whs_right_seq_wrong_ack_evasion": 723726,
        "4whs_synack_with_wrong_syn": 150,
        "4whs_wrong_seq": 0,
        "est_syn_resend_diff_seq": 80890989,
        "fin_out_of_window_delta": 0,
        "reassembly_no_segment_delta": 0,
        "est_packet_out_of_window": 22897789,
        "fin_invalid_ack": 14441732,
        "3whs_wrong_seq_wrong_ack": 4238358,
        "4whs_invalid_ack": 0,
        "fin1_invalid_ack_delta": 59,
        "pkt_invalid_timestamp_delta": 773,
        "closewait_pkt_before_last_ack": 15487466,
        "4whs_wrong_seq_delta": 0,
        "closing_ack_wrong_seq_delta": 0,
        "3whs_ack_in_wrong_dir": 0,
        "fin1_ack_wrong_seq_delta": 0,
        "fin2_ack_wrong_seq": 30760,
        "closewait_pkt_before_last_ack_delta": 30,
        "closewait_invalid_ack_delta": 30,
        "closing_invalid_ack_delta": 0,
        "fin_invalid_ack_delta": 41,
        "wrong_thread": 0
      },
      "flow_bypassed": {
        "local_bytes_delta": 949926543,
        "local_capture_bytes": 0,
        "local_pkts_delta": 3037796,
        "local_capture_pkts": 0,
        "local_capture_pkts_delta": 0,
        "bytes_delta": 0,
        "pkts_delta": 0,
        "local_capture_bytes_delta": 0,
        "closed_delta": 0,
        "pkts": 0,
        "bytes": 0,
        "local_bytes": 338709324812843,
        "local_pkts": 1209211881987,
        "closed": 0
      },
      "napa_total": {
        "byte_delta": 5412768168,
        "pkts_delta": 12206587,
        "overflow_drop_pkts": 907587889,
        "pkts": 4175379867139,
        "overflow_drop_pkts_delta": 0,
        "overflow_drop_byte": 431415121343,
        "overflow_drop_byte_delta": 0,
        "byte": 1655725854236219
      }
    },
    "timestamp": "2021-01-05T09:42:29.170611+0900",
    "host": "001",
    "@version": "1"
  },
  "fields": {
    "stats.detect.engines.last_reload": [
      "2021-01-04T19:03:50.240Z"
    ],
    "@timestamp": [
      "2021-01-05T00:42:29.364Z"
    ],
    "timestamp": [
      "2021-01-05T00:42:29.170Z"
    ]
  },
  "sort": [
    1609807349364
  ]
}
8

Yep, that format is ok.
Please update to see if the changes have any positive effect.

9

First of all, when I set flow settings to about 20 times less as you suggest, the memory wasted less. However, flow-timeout inuse seems to increase equally as below. Should I review the timeout setting?

sum of stats.flow.mgr.flows_timeout_inuse_delta

image

stats.log update

{
  "_index": "suri-stats-2021.01.12",
  "_type": "stats",
  "_id": "FMh29HYBPb4PvSR7qphl",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2021-01-12T02:39:46.167Z",
    "event_type": "stats",
    "stats": {
      "detect": {
        "mpm_list": 1,
        "nonmpm_list_delta": 0,
        "match_list": 26,
        "alert": 20342634,
        "nonmpm_list": 47,
        "fnonmpm_list_delta": 0,
        "engines": [
          {
            "rules_loaded": 17713,
            "rules_failed": 17857,
            "id": 0,
            "last_reload": "2021-01-11T20:24:46.924520+0900"
          }
        ],
        "match_list_delta": 0,
        "alert_delta": 695,
        "mpm_list_delta": 0,
        "fnonmpm_list": 25
      },
      "uptime": 61392,
      "tcp": {
        "insert_data_normal_fail": 29981439338,
        "no_flow": 0,
        "rst": 1610369074,
        "overlap": 119459222,
        "insert_data_normal_fail_delta": 2794408,
        "synack": 2755494892,
        "invalid_checksum_delta": 9,
        "sessions_delta": 161509,
        "pseudo_delta": 4,
        "stream_depth_reached": 4014636,
        "memuse": 11020711432,
        "insert_data_overlap_fail": 432286,
        "midstream_pickups_delta": 0,
        "overlap_diff_data": 0,
        "ssn_memcap_drop_delta": 0,
        "insert_list_fail": 858,
        "syn": 2595300894,
        "pseudo": 117413034,
        "memuse_delta": -2288,
        "midstream_pickups": 0,
        "reassembly_gap_delta": 1113511,
        "insert_data_overlap_fail_delta": 14,
        "segment_memcap_drop_delta": 0,
        "ssn_memcap_drop": 0,
        "overlap_delta": 4262,
        "reassembly_memuse_delta": 289848,
        "no_flow_delta": 0,
        "pkt_on_wrong_thread": 0,
        "segment_memcap_drop": 0,
        "syn_delta": 164816,
        "pkt_on_wrong_thread_delta": 0,
        "reassembly_gap": 17681986800,
        "sessions": 2496500652,
        "rst_delta": 127834,
        "pseudo_failed": 0,
        "synack_delta": 186005,
        "reassembly_memuse": 161061583884,
        "invalid_checksum": 397719,
        "insert_list_fail_delta": 0,
        "stream_depth_reached_delta": 325,
        "pseudo_failed_delta": 0,
        "overlap_diff_data_delta": 0
      },
      "app_layer": {
        "tx": {
          "rfb": 9367,
          "sip": 934313,
          "nfs_tcp": 2336,
          "ftp-data": 0,
          "tftp_delta": 0,
          "nfs_udp": 0,
          "sip_delta": 4,
          "http2": 74400,
          "rdp": 5554845,
          "rfb_delta": 0,
          "tls_delta": 0,
          "ftp_delta": 0,
          "http_delta": 3177,
          "imap_delta": 0,
          "dns_tcp": 6268137,
          "tftp": 1323,
          "mqtt_delta": 0,
          "nfs_tcp_delta": 0,
          "dns_udp_delta": 25024,
          "dns_udp": 592972149,
          "ssh": 0,
          "rdp_delta": 13,
          "nfs_udp_delta": 0,
          "ftp": 337682,
          "tls": 0,
          "http": 612009211,
          "smb_delta": 22,
          "dns_tcp_delta": 31,
          "ftp-data_delta": 0,
          "mqtt": 13581,
          "http2_delta": 0,
          "imap": 0,
          "ssh_delta": 0,
          "smb": 7598581
        },
        "flow": {
          "rfb": 8553,
          "sip": 880610,
          "nfs_tcp": 1171,
          "ftp-data": 14814,
          "tftp_delta": 0,
          "nfs_udp": 0,
          "sip_delta": 1,
          "http2": 9865,
          "failed_tcp": 48026265,
          "rdp": 1831273,
          "failed_udp": 14525667,
          "rfb_delta": 0,
          "tls_delta": 846,
          "ftp_delta": 0,
          "http_delta": 339,
          "imap_delta": 0,
          "dns_tcp": 2621289,
          "tftp": 4544,
          "mqtt_delta": 0,
          "nfs_tcp_delta": 0,
          "dns_udp_delta": 10334,
          "dns_udp": 235240977,
          "ssh": 1668921,
          "rdp_delta": 1,
          "nfs_udp_delta": 0,
          "ftp": 38769,
          "tls": 363726807,
          "http": 202122187,
          "smb_delta": 5,
          "dns_tcp_delta": 0,
          "failed_udp_delta": 918,
          "ftp-data_delta": 0,
          "mqtt": 1047,
          "http2_delta": 0,
          "failed_tcp_delta": 1515,
          "imap": 53,
          "ssh_delta": 1,
          "smb": 2438279
        },
        "expectations": 0,
        "expectations_delta": 0
      },
      "napa_dispatch_drop": {
        "byte_delta": 0,
        "pkts_delta": 0,
        "pkts": 0,
        "byte": 0
      },
      "ftp": {
        "memuse": 2276936,
        "memuse_delta": 0,
        "memcap": 0,
        "memcap_delta": 0
      },
      "http": {
        "memuse": 3057616905,
        "memuse_delta": -780745,
        "memcap": 0,
        "memcap_delta": 0
      },
      "decoder": {
        "ppp": 0,
        "gre": 767,
        "ipv4_in_ipv6": 0,
        "chdlc_delta": 0,
        "sll_delta": 0,
        "ipv4": 206338245844,
        "vlan_qinq": 0,
        "ieee8021ah_delta": 0,
        "max_mac_addrs_dst": 0,
        "vxlan_delta": 0,
        "vlan_delta": 0,
        "ethernet_delta": 12968772,
        "invalid": 218649,
        "ipv4_delta": 12968786,
        "tcp_delta": 12174038,
        "invalid_delta": 42,
        "pppoe": 0,
        "ipv4_in_ipv6_delta": 0,
        "pkts": 206338103367,
        "ethernet": 206338103367,
        "erspan_delta": 0,
        "gre_delta": 0,
        "raw_delta": 0,
        "udp_delta": 680103,
        "max_pkt_size": 1518,
        "sll": 0,
        "icmpv6": 1552082,
        "pppoe_delta": 0,
        "bytes_delta": 5470352930,
        "teredo": 170753,
        "ppp_delta": 0,
        "vxlan": 995,
        "bytes": 79627394501690,
        "null_delta": 0,
        "raw": 0,
        "geneve_delta": 0,
        "ieee8021ah": 0,
        "ipv6_in_ipv6": 0,
        "icmpv4_delta": 5230,
        "tcp": 199475564446,
        "teredo_delta": 0,
        "max_mac_addrs_dst_delta": 0,
        "event": {
          "ppp": {
            "unsup_proto": 0,
            "pkt_too_small": 0,
            "ip6_pkt_too_small": 0,
            "wrong_type_delta": 0,
            "vju_pkt_too_small_delta": 0,
            "ip4_pkt_too_small_delta": 0,
            "unsup_proto_delta": 0,
            "vju_pkt_too_small": 0,
            "ip6_pkt_too_small_delta": 0,
            "ip4_pkt_too_small": 0,
            "pkt_too_small_delta": 0,
            "wrong_type": 0
          },
          "gre": {
            "version0_malformed_sre_hdr": 0,
            "version1_route": 0,
            "version1_recur_delta": 0,
            "version0_flags_delta": 0,
            "version1_flags": 0,
            "version1_route_delta": 0,
            "version1_malformed_sre_hdr": 0,
            "wrong_version": 0,
            "version0_recur_delta": 0,
            "version0_hdr_too_big_delta": 0,
            "version0_malformed_sre_hdr_delta": 0,
            "version1_recur": 0,
            "version0_recur": 0,
            "version0_flags": 0,
            "version1_flags_delta": 0,
            "version1_no_key": 0,
            "version1_chksum_delta": 0,
            "wrong_version_delta": 0,
            "version1_chksum": 0,
            "pkt_too_small": 0,
            "version1_ssr_delta": 0,
            "version0_hdr_too_big": 0,
            "version1_wrong_protocol_delta": 0,
            "version1_hdr_too_big_delta": 0,
            "version1_malformed_sre_hdr_delta": 0,
            "version1_hdr_too_big": 0,
            "version1_no_key_delta": 0,
            "version1_ssr": 0,
            "version1_wrong_protocol": 0,
            "pkt_too_small_delta": 0
          },
          "tcp": {
            "opt_duplicate": 0,
            "opt_duplicate_delta": 0,
            "pkt_too_small": 0,
            "hlen_too_small_delta": 0,
            "opt_invalid_len": 217043,
            "hlen_too_small": 892,
            "opt_invalid_len_delta": 43,
            "invalid_optlen_delta": 0,
            "pkt_too_small_delta": 0,
            "invalid_optlen": 33
          },
          "ipv6": {
            "exthdr_useless_fh_delta": 0,
            "exthdr_dupl_ah": 0,
            "exthdr_useless_fh": 0,
            "hopopts_unknown_opt": 0,
            "zero_len_padn": 0,
            "frag_ignored": 0,
            "trunc_pkt": 0,
            "exthdr_dupl_fh_delta": 0,
            "exthdr_dupl_rh": 0,
            "exthdr_ah_res_not_null": 29,
            "exthdr_invalid_optlen_delta": 0,
            "ipv6_in_ipv6_wrong_version_delta": 0,
            "exthdr_dupl_hh": 0,
            "ipv4_in_ipv6_wrong_version": 691,
            "fh_non_zero_reserved_field_delta": 0,
            "data_after_none_header_delta": 0,
            "unknown_next_header": 165066,
            "pkt_too_small": 0,
            "frag_overlap": 0,
            "dstopts_unknown_opt": 0,
            "frag_ignored_delta": 0,
            "dstopts_unknown_opt_delta": 0,
            "ipv6_in_ipv6_too_small": 0,
            "unknown_next_header_delta": 0,
            "ipv6_in_ipv6_too_small_delta": 0,
            "dstopts_only_padding_delta": 0,
            "rh_type_0_delta": 0,
            "exthdr_dupl_fh": 0,
            "exthdr_invalid_optlen": 0,
            "trunc_exthdr_delta": 0,
            "data_after_none_header": 1399,
            "hopopts_only_padding": 0,
            "pkt_too_small_delta": 0,
            "icmpv4_delta": 0,
            "rh_type_0": 0,
            "ipv4_in_ipv6_too_small": 0,
            "dstopts_only_padding": 0,
            "trunc_exthdr": 0,
            "zero_len_padn_delta": 0,
            "wrong_ip_version": 0,
            "hopopts_only_padding_delta": 0,
            "frag_pkt_too_large_delta": 0,
            "ipv4_in_ipv6_wrong_version_delta": 0,
            "icmpv4": 701,
            "frag_overlap_delta": 0,
            "exthdr_dupl_rh_delta": 0,
            "exthdr_dupl_ah_delta": 0,
            "fh_non_zero_reserved_field": 725,
            "frag_pkt_too_large": 0,
            "wrong_ip_version_delta": 0,
            "exthdr_dupl_dh_delta": 0,
            "exthdr_dupl_eh": 0,
            "hopopts_unknown_opt_delta": 0,
            "exthdr_dupl_dh": 0,
            "ipv6_in_ipv6_wrong_version": 629,
            "exthdr_dupl_hh_delta": 0,
            "exthdr_ah_res_not_null_delta": 0,
            "trunc_pkt_delta": 0,
            "exthdr_dupl_eh_delta": 0,
            "ipv4_in_ipv6_too_small_delta": 0
          },
          "ipraw": {
            "invalid_ip_version": 0,
            "invalid_ip_version_delta": 0
          },
          "ipv4": {
            "opt_duplicate_delta": 0,
            "hlen_too_small_delta": 0,
            "opt_invalid_len": 0,
            "frag_pkt_too_large_delta": 0,
            "frag_ignored": 0,
            "wrong_ip_version": 0,
            "trunc_pkt": 0,
            "opt_pad_required_delta": 0,
            "frag_overlap_delta": 0,
            "hlen_too_small": 0,
            "opt_eol_required_delta": 0,
            "frag_pkt_too_large": 0,
            "iplen_smaller_than_hlen_delta": 0,
            "wrong_ip_version_delta": 0,
            "opt_malformed": 0,
            "pkt_too_small": 0,
            "opt_invalid_len_delta": 0,
            "frag_overlap": 49,
            "frag_ignored_delta": 0,
            "opt_pad_required": 978,
            "icmpv6_delta": 0,
            "icmpv6": 0,
            "opt_unknown": 0,
            "opt_malformed_delta": 0,
            "opt_duplicate": 0,
            "opt_eol_required": 0,
            "trunc_pkt_delta": 0,
            "opt_unknown_delta": 0,
            "iplen_smaller_than_hlen": 0,
            "opt_invalid_delta": 0,
            "pkt_too_small_delta": 0,
            "opt_invalid": 0
          },
          "ltnull": {
            "unsupported_type_delta": 0,
            "pkt_too_small": 0,
            "unsupported_type": 0,
            "pkt_too_small_delta": 0
          },
          "sctp": {
            "pkt_too_small_delta": 0,
            "pkt_too_small": 0
          },
          "udp": {
            "pkt_too_small": 8,
            "hlen_too_small_delta": 0,
            "hlen_too_small": 0,
            "hlen_invalid_delta": 0,
            "hlen_invalid": 1207,
            "pkt_too_small_delta": 0
          },
          "sll": {
            "pkt_too_small_delta": 0,
            "pkt_too_small": 0
          },
          "icmpv6": {
            "pkt_too_small": 0,
            "experimentation_type": 13,
            "mld_message_with_invalid_hl_delta": 0,
            "unknown_type": 8,
            "unknown_code": 83,
            "ipv6_trunc_pkt": 0,
            "ipv6_unknown_version": 0,
            "ipv6_trunc_pkt_delta": 0,
            "experimentation_type_delta": 0,
            "unknown_type_delta": 0,
            "mld_message_with_invalid_hl": 11,
            "unknown_code_delta": 0,
            "ipv6_unknown_version_delta": 0,
            "unassigned_type_delta": 0,
            "pkt_too_small_delta": 0,
            "unassigned_type": 555
          },
          "pppoe": {
            "wrong_code": 0,
            "pkt_too_small": 0,
            "wrong_code_delta": 0,
            "malformed_tags": 0,
            "malformed_tags_delta": 0,
            "pkt_too_small_delta": 0
          },
          "icmpv4": {
            "ipv4_unknown_ver": 16,
            "pkt_too_small": 0,
            "unknown_type_delta": 0,
            "unknown_type": 33,
            "ipv4_trunc_pkt": 0,
            "unknown_code_delta": 0,
            "ipv4_trunc_pkt_delta": 0,
            "ipv4_unknown_ver_delta": 0,
            "pkt_too_small_delta": 0,
            "unknown_code": 4577
          },
          "ethernet": {
            "pkt_too_small_delta": 0,
            "pkt_too_small": 0
          },
          "mpls": {
            "bad_label_router_alert_delta": 0,
            "header_too_small_delta": 0,
            "pkt_too_small": 0,
            "unknown_payload_type": 0,
            "bad_label_implicit_null": 0,
            "bad_label_reserved": 40745,
            "bad_label_implicit_null_delta": 0,
            "bad_label_reserved_delta": 2,
            "unknown_payload_type_delta": 0,
            "bad_label_router_alert": 0,
            "header_too_small": 0,
            "pkt_too_small_delta": 0
          },
          "vxlan": {
            "unknown_payload_type_delta": 0,
            "unknown_payload_type": 995
          },
          "dce": {
            "pkt_too_small_delta": 0,
            "pkt_too_small": 0
          },
          "vlan": {
            "header_too_small_delta": 0,
            "header_too_small": 0,
            "unknown_type_delta": 0,
            "too_many_layers_delta": 0,
            "unknown_type": 0,
            "too_many_layers": 0
          },
          "ieee8021ah": {
            "header_too_small_delta": 0,
            "header_too_small": 0
          },
          "geneve": {
            "unknown_payload_type_delta": 0,
            "unknown_payload_type": 1
          },
          "erspan": {
            "unsupported_version_delta": 0,
            "header_too_small_delta": 0,
            "header_too_small": 0,
            "too_many_vlan_layers": 0,
            "unsupported_version": 0,
            "too_many_vlan_layers_delta": 0
          }
        },
        "avg_pkt_size_delta": 0,
        "avg_pkt_size": 385,
        "vlan_qinq_delta": 0,
        "icmpv4": 83107432,
        "chdlc": 0,
        "sctp_delta": 0,
        "vlan": 0,
        "erspan": 0,
        "null": 0,
        "ipv6": 1922914,
        "icmpv6_delta": 96,
        "max_pkt_size_delta": 0,
        "sctp": 741,
        "ipv6_delta": 101,
        "udp": 5332704158,
        "pkts_delta": 12968772,
        "mpls": 40745,
        "ipv6_in_ipv6_delta": 0,
        "mpls_delta": 2,
        "max_mac_addrs_src": 0,
        "geneve": 1,
        "max_mac_addrs_src_delta": 0
      },
      "flow": {
        "tcp_reuse": 181855460,
        "icmpv4_delta": 274,
        "tcp": 2670204813,
        "wrk": {
          "flows_injected": 386366531,
          "spare_sync_avg_delta": 0,
          "flows_evicted_pkt_inject": 697428781,
          "spare_sync_delta": 1494,
          "spare_sync": 22300100,
          "flows_evicted_needs_work": 473635342,
          "flows_evicted_pkt_inject_delta": 40253,
          "spare_sync_incomplete": 0,
          "spare_sync_empty": 0,
          "spare_sync_incomplete_delta": 0,
          "flows_injected_delta": 21268,
          "flows_evicted_delta": 15815,
          "spare_sync_avg": 100,
          "flows_evicted_needs_work_delta": 26276,
          "flows_evicted": 309867109,
          "spare_sync_empty_delta": 0
        },
        "get_used_eval_reject_delta": 0,
        "memuse": 20458835840,
        "get_used_eval_busy_delta": 0,
        "get_used_failed_delta": 0,
        "get_used_failed": 0,
        "tcp_delta": 174538,
        "icmpv4": 4752843,
        "memcap_delta": 0,
        "spare_delta": -1249,
        "memuse_delta": -15040,
        "mgr": {
          "bypassed_pruned": 0,
          "flows_evicted_needs_work_delta": 21268,
          "flows_timeout_delta": 111058,
          "est_pruned_delta": 0,
          "flows_checked": 2384508357,
          "closed_pruned": 0,
          "closed_pruned_delta": 0,
          "full_hash_pass_delta": 0,
          "rows_maxlen": 1725,
          "flows_evicted_delta": 168496,
          "flows_notimeout": 523936429,
          "full_hash_pass": 510,
          "est_pruned": 0,
          "bypassed_pruned_delta": 0,
          "flows_timeout_inuse_delta": 35,
          "rows_maxlen_delta": 0,
          "flows_timeout": 1860571928,
          "flows_timeout_inuse": 339803,
          "flows_evicted_needs_work": 386366531,
          "flows_notimeout_delta": 26670,
          "flows_checked_delta": 137728,
          "new_pruned_delta": 0,
          "new_pruned": 0,
          "flows_evicted": 2597773528
        },
        "udp_delta": 11253,
        "emerg_mode_entered": 0,
        "icmpv6_delta": 4,
        "memcap": 0,
        "udp": 250651798,
        "get_used_delta": 0,
        "icmpv6": 53897,
        "get_used_eval_busy": 0,
        "emerg_mode_over": 0,
        "emerg_mode_over_delta": 0,
        "tcp_reuse_delta": 17091,
        "get_used": 0,
        "get_used_eval_delta": 0,
        "spare": 83430047,
        "get_used_eval_reject": 0,
        "get_used_eval": 0,
        "emerg_mode_entered_delta": 0
      },
      "defrag": {
        "max_frag_hits": 0,
        "ipv6": {
          "reassembled": 0,
          "reassembled_delta": 0,
          "fragments": 726,
          "timeouts_delta": 0,
          "timeouts": 0,
          "fragments_delta": 0
        },
        "max_frag_hits_delta": 0,
        "ipv4": {
          "reassembled": 9036785,
          "reassembled_delta": 574,
          "fragments": 20864562,
          "timeouts_delta": 0,
          "timeouts": 0,
          "fragments_delta": 1607
        }
      },
      "napa_dispatch_host": {
        "byte_delta": 0,
        "pkts_delta": 0,
        "pkts": 207380326255,
        "byte": 80066607543398
      },
      "stream": {
        "est_invalid_ack_delta": 436,
        "3whs_syn_toclient_on_syn_recv_delta": 0,
        "shutdown_syn_resend_delta": 164,
        "est_packet_out_of_window_delta": 5,
        "3whs_synack_with_wrong_ack": 29345,
        "3whs_syn_resend_diff_seq_on_syn_recv_delta": 18,
        "3whs_wrong_seq_wrong_ack_delta": 6,
        "est_synack_resend_with_diff_ack_delta": 24,
        "pkt_invalid_ack_delta": 886,
        "reassembly_seq_gap": 17681986488,
        "rst_invalid_ack_delta": 37,
        "reassembly_overlap_different_data_delta": 0,
        "closewait_fin_out_of_window_delta": 27,
        "rst_but_no_session_delta": 4874,
        "est_synack_toserver": 10462,
        "est_invalid_ack": 5634503,
        "lastack_invalid_ack": 19101,
        "3whs_async_wrong_seq_delta": 0,
        "4whs_synack_with_wrong_ack": 0,
        "fin_but_no_session": 166367640,
        "pkt_retransmission_delta": 3831,
        "fin2_invalid_ack": 7308337,
        "3whs_synack_resend_with_diff_ack_delta": 25,
        "closewait_ack_out_of_window": 94368,
        "lastack_ack_wrong_seq": 4394,
        "pkt_invalid_timestamp": 29744565,
        "pkt_broken_ack_delta": 1839,
        "fin2_fin_wrong_seq_delta": 37,
        "3whs_right_seq_wrong_ack_evasion_delta": 0,
        "3whs_synack_in_wrong_direction": 32586,
        "timewait_invalid_ack_delta": 21,
        "est_synack_resend": 43836,
        "est_syn_resend_diff_seq_delta": 31,
        "shutdown_syn_resend": 4631860,
        "fin2_fin_wrong_seq": 2045511,
        "3whs_synack_resend_with_diff_seq_delta": 0,
        "rst_invalid_ack": 366659,
        "est_synack_resend_with_diff_ack": 1143899,
        "closing_invalid_ack": 0,
        "3whs_synack_toserver_on_syn_recv": 13235,
        "3whs_synack_toserver_on_syn_recv_delta": 10,
        "pkt_invalid_ack": 15560366,
        "reassembly_no_segment": 0,
        "est_syn_resend_delta": 9,
        "3whs_synack_resend_with_diff_ack": 276585,
        "reassembly_overlap_different_data": 0,
        "3whs_ack_data_inject_delta": 0,
        "fin_out_of_window": 19059,
        "timewait_invalid_ack": 270789,
        "fin1_invalid_ack": 1749291,
        "3whs_synack_resend_with_diff_seq": 0,
        "3whs_syn_toclient_on_syn_recv": 0,
        "3whs_synack_flood": 25937,
        "est_syn_resend": 134031,
        "fin2_invalid_ack_delta": 190,
        "fin2_ack_wrong_seq_delta": 0,
        "lastack_ack_wrong_seq_delta": 1,
        "pkt_broken_ack": 17512958,
        "4whs_synack_with_wrong_ack_delta": 0,
        "pkt_bad_window_update_delta": 17,
        "fin1_fin_wrong_seq": 419252,
        "est_pkt_before_last_ack_delta": 1788,
        "3whs_ack_in_wrong_dir_delta": 0,
        "fin1_ack_wrong_seq": 1174,
        "lastack_invalid_ack_delta": 0,
        "3whs_synack_with_wrong_ack_delta": 2,
        "3whs_ack_data_inject": 0,
        "est_pkt_before_last_ack": 27929069,
        "est_synack_resend_with_diff_seq": 13250,
        "est_syn_toclient_delta": 0,
        "est_synack_resend_with_diff_seq_delta": 0,
        "pkt_retransmission": 55254902,
        "timewait_ack_wrong_seq": 1247377,
        "reassembly_segment_before_base_seq_delta": 0,
        "closewait_fin_out_of_window": 262454,
        "suspected_rst_inject_delta": 0,
        "reassembly_segment_before_base_seq": 858,
        "3whs_synack_in_wrong_direction_delta": 0,
        "fin1_fin_wrong_seq_delta": 43,
        "fin_but_no_session_delta": 20145,
        "suspected_rst_inject": 250,
        "closing_ack_wrong_seq": 0,
        "pkt_bad_window_update": 242496,
        "reassembly_seq_gap_delta": 1113511,
        "est_synack_resend_delta": 3,
        "est_synack_toserver_delta": 6,
        "3whs_async_wrong_seq": 0,
        "rst_but_no_session": 63567479,
        "wrong_thread_delta": 0,
        "3whs_synack_flood_delta": 1,
        "est_syn_toclient": 0,
        "closewait_invalid_ack": 45151,
        "3whs_syn_resend_diff_seq_on_syn_recv": 403155,
        "closewait_ack_out_of_window_delta": 0,
        "4whs_synack_with_wrong_syn_delta": 0,
        "timewait_ack_wrong_seq_delta": 61,
        "4whs_invalid_ack_delta": 0,
        "3whs_right_seq_wrong_ack_evasion": 33739,
        "4whs_synack_with_wrong_syn": 1,
        "4whs_wrong_seq": 0,
        "est_syn_resend_diff_seq": 3893776,
        "fin_out_of_window_delta": 6,
        "reassembly_no_segment_delta": 0,
        "est_packet_out_of_window": 167128,
        "fin_invalid_ack": 164324,
        "3whs_wrong_seq_wrong_ack": 111331,
        "4whs_invalid_ack": 0,
        "fin1_invalid_ack_delta": 182,
        "pkt_invalid_timestamp_delta": 3814,
        "closewait_pkt_before_last_ack": 614707,
        "4whs_wrong_seq_delta": 0,
        "closing_ack_wrong_seq_delta": 0,
        "3whs_ack_in_wrong_dir": 0,
        "fin1_ack_wrong_seq_delta": 0,
        "fin2_ack_wrong_seq": 5169,
        "closewait_pkt_before_last_ack_delta": 30,
        "closewait_invalid_ack_delta": 0,
        "closing_invalid_ack_delta": 0,
        "fin_invalid_ack_delta": 20,
        "wrong_thread": 0
      },
      "flow_bypassed": {
        "local_bytes_delta": 1708923599,
        "local_capture_bytes": 0,
        "local_pkts_delta": 5385435,
        "local_capture_pkts": 0,
        "local_capture_pkts_delta": 0,
        "bytes_delta": 0,
        "pkts_delta": 0,
        "local_capture_bytes_delta": 0,
        "closed_delta": 0,
        "pkts": 0,
        "bytes": 0,
        "local_bytes": 29152490274244,
        "local_pkts": 88630551616,
        "closed": 0
      },
      "napa_total": {
        "byte_delta": 0,
        "pkts_delta": 0,
        "overflow_drop_pkts": 7884555,
        "pkts": 207380326255,
        "overflow_drop_pkts_delta": 0,
        "overflow_drop_byte": 5993172139,
        "overflow_drop_byte_delta": 0,
        "byte": 80066607543398
      }
    },
    "timestamp": "2021-01-12T11:39:46.093233+0900",
    "host": "001",
    "@version": "1"
  },
  "fields": {
    "stats.detect.engines.last_reload": [
      "2021-01-11T11:24:46.924Z"
    ],
    "@timestamp": [
      "2021-01-12T02:39:46.167Z"
    ],
    "timestamp": [
      "2021-01-12T02:39:46.093Z"
    ]
  },
  "sort": [
    1610419186167
  ]
}
10

What version are you using? I might have run into a similar issue and would like to validate.
If it’s 6.0.x try 5.0.6 and check if you can reproduce it.