Strange behavior with pass and drop rule

Hi everyone,

I hope that I can provide sufficient information here, apologies in advance as I’m very new to Suricata. We are running 5.0.2 and have two rules.

  1. pass tls any any <> any any
  2. drop tcp any any → any any (flow:established,to_server; sid: 100; rev:1;)

The strange behavior we are seeing is that after running a curl command, e.g. curl https://www.google.com, in the alert logs we are dropping TCP packets, even though the curl command completes and the flow logs show a complete flow.

Sorry the flow IDs are not the same, but there are cases where they are from the same flow, so please do not focus on that aspect.
Allowed:

{
    "firewall_name": "FW",
    "availability_zone": "us-east-1a",
    "event_timestamp": "1623779566",
    "event": {
        "timestamp": "2021-06-15T17:52:46.000070+0000",
        "flow_id": 1387274044176656,
        "event_type": "netflow",
        "src_ip": "10.0.0.78",
        "src_port": 41874,
        "dest_ip": "52.46.138.63",
        "dest_port": 443,
        "proto": "TCP",
        "app_proto": "tls",
        "netflow": {
            "pkts": 22,
            "bytes": 5207,
            "start": "2021-06-15T17:51:23.588048+0000",
            "end": "2021-06-15T17:51:43.624026+0000",
            "age": 20,
            "min_ttl": 254,
            "max_ttl": 254
        },
        "tcp": {
            "tcp_flags": "1f",
            "syn": true,
            "fin": true,
            "rst": true,
            "psh": true,
            "ack": true
        }
    }
}

{
    "firewall_name": "FW",
    "availability_zone": "us-east-1a",
    "event_timestamp": "1623779566",
    "event": {
        "timestamp": "2021-06-15T17:52:46.000155+0000",
        "flow_id": 1387274044176656,
        "event_type": "netflow",
        "src_ip": "52.46.138.63",
        "src_port": 443,
        "dest_ip": "10.0.0.78",
        "dest_port": 41874,
        "proto": "TCP",
        "app_proto": "tls",
        "netflow": {
            "pkts": 24,
            "bytes": 7151,
            "start": "2021-06-15T17:51:23.588048+0000",
            "end": "2021-06-15T17:51:43.624026+0000",
            "age": 20,
            "min_ttl": 232,
            "max_ttl": 248
        },
        "tcp": {
            "tcp_flags": "1b",
            "syn": true,
            "fin": true,
            "psh": true,
            "ack": true
        }
    }
}

but then also says dropped:

{
    "firewall_name": "FW",
    "availability_zone": "us-east-1a",
    "event_timestamp": "1623779845",
    "event": {
        "timestamp": "2021-06-15T17:57:25.816443+0000",
        "flow_id": 641178119007310,
        "event_type": "alert",
        "src_ip": "10.0.0.78",
        "src_port": 41912,
        "dest_ip": "52.46.138.63",
        "dest_port": 443,
        "proto": "TCP",
        "alert": {
            "action": "blocked",
            "signature_id": 1999999,
            "rev": 1,
            "signature": "Deny all other TCP traffic",
            "category": "",
            "severity": 3
        }
    }
}

Similar behavior was seen when we replaced the drop TLS with drop HTTP. Is there some nuanced behavior with layer mixing rules cases?

Thanks in advance

Hi Kelly,

thanks for reaching out, and welcome to our forum!

In order to make it easier for us to help you, could you please:

  • use the “preformatted text” option to improve readability on the json excerpt? You can do that by adding a new line with ```json before the log record and one with the three quotes right after the excerpt (or you can use the buttons with formatting options, too). You may have to copy and paste the records again, to recover proper indentation.
  • share what is the command you’re using to run Suricata, and how are you passing those rules to Suricata?

I’ve also noticed that:

  • the alert shown is listing a different signature_id (the same as “sid” in the rule) and msg. Did you change that, or is it possible that a different ruleset is being used, somehow?
  • the first rule shown doesn’t include sid and rev. All rules must have at least a sid, because Suricata will default signature_id to 0 – which isn’t accepted. So as good practice, always add one, even if it’s just a local rule! :slight_smile:

Hi Jufajardini,

Thanks so much for the response! Sorry about the formatting, I’ve fixed it per your recommendation.

The command used to run Suricata is

suricata -c /etc/suricata/suricata.yaml -D --af-packet

and we pass the rules in the yaml file using the default-rules-path and the specified rules file.

  • Yes, you are right about me changing the sid :man_facepalming: I was trying to change up the rulestring for anonymity, sorry for the confusion.
  • Thanks for the tip about the sid! I’ve omitted it unintentionally

I hope this answers your questions sufficiently

Hi Kelly!

You’re welcome! And thanks, logs are much better to read, now.

One thing that I didn’t mention in my previous answer (sorry about that!): Suricata’s latest version is 6.0.2, we always encourage users to have the most updated version, if possible, as they’ll have the latest fixes and support will be more accurate. Can you try updating Suricata?

Regarding the logs and rules, it’s hard to analyze the traffic when we are not following the same flow_id, could you upload a file with the whole output to help us understand what is amiss here, and also provide what are the updated rules that you are using, since the alert we see is triggered with a somewhat different rule?

Last but not least, we OISF recently co-authored an article on using Suricata rules with AWS, which seems to be what you are using - please forgive me if that’s not the case. Do you think that following this article could help you, in case you haven’t read it yet? → Scaling threat prevention on AWS with Suricata | AWS Open Source Blog

Can you provide us with the suricata config as well?

In general pass has prio over drop

Hi Ju,

Unfortunately we aren’t able to use 6.0.2 yet, but that’s a work in progress.

The updated rules:
drop tcp any any → any any (flow:established,to_server; msg:“Deny all other TCP traffic”; sid: 19; rev:1;)
pass tls any any <> any any (sid:1; rev:1;)

The logs (alert/flow) after executing curl https://www.google.com:
logs.zip (7.5 KB)

We are indeed using Suricata with AWS, thanks for the article! Will be taking a look

Hi Andreas,

I don’t believe I can share the entire config, but we do follow the general action order:

  • pass
  • drop
  • reject
  • alert

If there is something specific you’d like to have some clarification on I would love to answer that

You can remove sensitiv parts of the config but would be interesting to see how you configured the app-layer, logging, af-packet and vxlan settings.

Can you create a pcap to reproduce that?

Also those log files are they directly from suricata eve json output, cause they have additional output not part of suricata.

Yes, the log files aren’t directly from the suricata eve json output.

The app-layer looks like this:

app-layer:
  protocols:
    krb5:
      enabled: yes
    snmp:
      enabled: yes
    ikev2:
      enabled: yes
    tls:
      enabled: yes
      detection-ports:
        dp: 443   
      ja3-fingerprints: auto
      encryption-handling: default
    dcerpc:
      enabled: yes
    ftp:
      enabled: yes
      # memcap: 64mb
    # RDP, disabled by default.
    rdp:
    #enabled: no
    ssh:
      enabled: yes
    smtp:
      enabled: yes
      raw-extraction: no
      mime:
        decode-mime: yes
        decode-base64: yes
        decode-quoted-printable: yes
        header-value-depth: 2000
        extract-urls: yes
        body-md5: no
      inspected-tracker:
        content-limit: 100000
        content-inspect-min-size: 32768
        content-inspect-window: 4096
    imap:
      enabled: detection-only
    smb:
      enabled: yes
      detection-ports:
        dp: 139, 445
    nfs:
      enabled: yes
    tftp:
      enabled: yes
    dns:
      request-flood: 500
      tcp:
        enabled: yes
        detection-ports:
          dp: 53
      udp:
        enabled: yes
        detection-ports:
          dp: 53
    http:
      enabled: yes
      libhtp:
        default-config:
          personality: IDS
          request-body-limit: 100kb
          response-body-limit: 100kb
          request-body-minimal-inspect-size: 32kb
          request-body-inspect-window: 4kb
          response-body-minimal-inspect-size: 40kb
          response-body-inspect-window: 16kb
          response-body-decompress-layer-limit: 2
          http-body-inline: auto
          swf-decompression:
            enabled: yes
            type: both
            compress-depth: 0
            decompress-depth: 0
          double-decode-path: no
          double-decode-query: no

        server-config:

    modbus:
      enabled: no
      detection-ports:
        dp: 502
    dnp3:
      enabled: no
      detection-ports:
        dp: 20000
    enip:
      enabled: no
      detection-ports:
        dp: 44818
        sp: 44818
    ntp:
      enabled: yes
    dhcp:
      enabled: yes
    sip:
      enabled: no

logging:

- eve-log:
      enabled: yes
      filetype: regular
      filename: place-holder.json
      metadata: no
      pcap-file: false
      community-id: false
      xff:
        enabled: no
      types:
        - alert:
            payload: no
            packet: no
            metadata:
              app-layer: true
              flow: false
              rule:
                metadata: true
                raw: false
            http-body: no
            tagged-packets: yes

  - eve-log:
      enabled: yes
      filetype: regular
      filename: eve-stats.%Y-%m-%d-%H-%M.json
      metadata: no
      pcap-file: false
      community-id: false
      xff:
        enabled: no
      types:
        - stats:
            totals: yes
            threads: no
            deltas: yes
  - eve-log:
      enabled: yes
      filetype: regular
      filename: place-holder.json
      metadata: no
      pcap-file: false
      community-id: false
      xff:
        enabled: no
      types:
        - netflow

  - eve-log:
      enabled: yes
      filetype: regular
      filename: place-holder.json
      metadata: no
      pcap-file: false
      community-id: false
      xff:
        enabled: no
      types:
        - drop:
            alerts: yes
            flows: all

af-packet:

af-packet:
  - interface: SFE_0_TX
    threads: 2
    cluster-id: 99
    cluster-type: cluster_flow
    defrag: no
    use-mmap: yes
    tpacket-v3: no
    ring-size: 100000
    use-emergency-flush: no
    buffer-size: 64535
    disable-promisc: no
    checksum-checks: no
    rollover: no 
    copy-mode: ips
    copy-iface: SFE_0_RX

  - interface: SFE_0_RX
    threads: 2
    cluster-id: 98
    cluster-type: cluster_flow
    defrag: no
    use-mmap: yes
    tpacket-v3: no
    ring-size: 100000
    use-emergency-flush: no
    buffer-size: 64535
    disable-promisc: no
    checksum-checks: no
    rollover: no 
    copy-mode: ips
    copy-iface: SFE_0_TX

  - interface: default
    threads: 1
    defrag: no
    use-mmap: yes
    tpacket-v3: no
    disable-promisc: no
    checksum-checks: no
    use-emergency-flush: no
    rollover: no 

vxlan:

vxlan:
    enabled: true
    ports: 4789

I hope this helps!

Just for testing, could you try a pass rule for tcp?
What might happen, I might be proven wrong, that from the flow the pure tls part is passed but that later packets are not detected as tls but rather just tcp or http and thus would be dropped.
What would help would be a test.pcap with the related full logfiles to validate that besides the pass rule for tcp.

Ahh, okay, that was our initial thoughts as well.

I’ve added a pass tcp rule + the previous rules. With our setup it’s not possible to run suricata with a pcap file, but I’ve attached logs using the same method of curl-ing google.com

log-events-viewer-result (2).csv.zip (5.7 KB)

And what happened with the tcp pass rule enabled?

the event view just shows netflow event types.

The curl worked. Apologies not sure what other information you’d like me to report? The events that were attached are all that I saw.

So that would confirm the assumption we have. Could you run it again and provide the log again but with the tls pass rule enabled to compare both scenarios?
So those events were from the scenario where you had the tcp pass enabled?

Ah I may have misread your previous message. I read “try a pass rule for tcp” as add it on top of the original rules so I ran it with pass tcp, pass tls, and the drop tcp rule. Did you mean just test pass tcp with the drop tcp rule without pass tls?

I think it may be easier for me to understand if the rules for each test case is listed. :sweat_smile:

run this and also

    pass tcp any any <> any any
    drop tcp any any → any any (flow:established,to_server; sid: 100; rev:1;)

so we can compare the output for the whole flow