Intermittent SSL connection drop for whitelisted Pass TLS endpoint in AWS NFW with suricatarules

Rules
1

Hi All
We need a help in a strange issue we are observing in our AWS NetworkFirewall setup with all stateful rule groups.
Our goal is to drop unwanted traffic and allow only passed validated traffic to go out from AWS to External_Net.

We have below 4 Drop blanket rules

drop http $HOME_NET any → [$EXTERNAL_NET] any (msg:“Drop all HTTP traffic”; priority:1; sid:1; rev:1;)

pass http $HOME_NET any → $EXTERNAL_NET any (http.host; content:“amazonaws.com”; endswith; msg:“Allow amazonaws.com HTTP domain”; priority:1; sid:2; rev:1;)

drop tls $HOME_NET any → [$EXTERNAL_NET] any (msg:“Drop all TLS traffic”; sid:3; rev:1;)

pass tls $HOME_NET any → $EXTERNAL_NET any (msg:“amazonaws.com SNI Allow”; tls_sni; content:“amazonaws.com”; endswith; sid:4; rev:1;)

pass tls $HOME_NET any → $EXTERNAL_NET any (msg:“console.cloudendure.com SNI Allow”; tls_sni; content:“console.cloudendure.com”; sid:5; rev:1;)

pass tcp $HOME_NET any <> $EXTERNAL_NET 80 (msg:“Pass tcp handshake to destination port 80”; flow: not_established; sid:10; priority:1; rev:1;)

pass tcp $HOME_NET any <> $EXTERNAL_NET 443 (msg:“Pass tcp handshake to destination port 443”; flow: not_established; sid:11; priority:1; rev:1;)

drop tcp $HOME_NET any <> [$EXTERNAL_NET] any (msg:“Drop any other tcp handshake”; flow: not_established; sid:12; priority:1; rev:1;)

drop ip $HOME_NET any → [$EXTERNAL_NET] any (msg:“Drop any other established flow”; flow: from_client, established; sid:13; priority:2; rev:1;)

After these we have pass TLS for entities URLS for which we want explicit HTTPS connections to be allowed
for example
pass tls $HOME_NET any → $EXTERNAL_NET any (msg:“yxz.abc.com SNI Allow”; tls_sni; content:“xyz.abc.com”; priority:1; sid:130; rev:1;)

Our NFW alert logs show that intermittently there is match for SID rule 13 with drop ip rule for outbound traffic for xyz.abc.com which is explicitly whitelisted in a rule

Surprisingly when we run AB bench test for https://xyz.abc.com/ we see that out of 100 30-50% (random number everytime) failed with SSL Handshake error.
Even curl to xyz.abc.com intermittenly failed after SSL hello and connection just terminated.
This does not happen for all the traffic packets and requests,hence we are not sure why SID rule 13 given below would blocked something already whitelisted by pass rule which has precedence (pritority 1)
drop ip $HOME_NET any → [$EXTERNAL_NET] any (msg:“Drop any other established flow”; flow: from_client, established; sid:13; priority:2; rev:1;)

Note this is not just happening for one endpoint destination like xyz.abc.com but many others. Anything outbound whitelisted via pass tls seems to be affected by rule SID 13 in above set

Kindly help if you see any discrepancy in rules SID 1, 3, 12,13.
Is rule SID 13 redundant? Can we get rid of it or we are having issues due to some mis configurations? AWS support is also checking but its taking long to find the root cause of issue

2

Can you also post stats.log and the suricata.yaml? Especially if you have random behavior, this can be related to different issues (drops, misconfig etc.)

3

Thanks for the response.
Is there a way we can share such files privately ?
Could be information/data sharing obligations our organization (Travelex) might have for this.
Also can you please suggest where can I see stats.log in AWS ? We have logs of AWS Network Firewall Service Though

4

So you use the AWS included feature, I don’t know the internal details of that.
In general if the pass rules are correct they should have prio over the drop rules, but if some packets are missing the pass might not trigger and you end up with the drop rule sid 13 hitting which is close to “drop everything”.

One idea is that maybe parts of the connection are not matched by the pass rules.

Another recommendation would be to create a test setup without the AWS NFW but with a standalone self-configured suricata and try to reproduce it. This would also make it easier for us to debug.