Community flow id inconsistency

Help
1

Hi all,

I am doing some tests filtering alerts using community_id field between Suricata and Zeek NIDS (Zeek release is 3.0.5). But I am seeing inconsistencies.

An example. This alert: ET POLICY curl User-Agent Outbound.

Zeek’s conn.log:

{“_node_name”:“worker-2”,“ts”:“2020-03-06T11:59:50.180917Z”,“uid”:“C8T8ce1oTnt3ln5723”,“id.orig_h”:“172.22.59.4”,“id.orig_p”:54692,“id.resp_h”:“149.28.239.174”,“id.resp_p”:80,“proto”:“tcp”,“service”:“http”,“duration”:0.44036006927490234,“orig_bytes”:121,“resp_bytes”:21647,“conn_state”:“SF”,“local_orig”:true,“local_resp”:false,“missed_bytes”:0,“history”:“ShADadFf”,“orig_pkts”:21,“orig_ip_bytes”:973,“resp_pkts”:19,“resp_ip_bytes”:22419,“community_id”:“1:FEAOBSU+mBA7TJlKI8FANpOagdY=”}

Suricata’s alert output:

{“timestamp”:“2020-03-06T11:59:50.386789+0000”,“flow_id”:327732767925959,“in_iface”:“vtnet3”,“event_type”:“alert”,“src_ip”:“172.22.59.4”,“src_port”:54692,“dest_ip”:“149.28.239.174”,“dest_port”:80,“proto”:“TCP”,“community_id”:“1:2sl7O0BzoS6G46kkIoRtxQMKWi4=”,“tx_id”:0,“alert”:{“action”:“allowed”,“gid”:1,“signature_id”:2013028,“rev”:4,“signature”:“ET POLICY curl User-Agent Outbound”,“category”:“Attempted Information Leak”,“severity”:2,“metadata”:{“updated_at”:[“2011_06_14”],“created_at”:[“2011_06_14”]}},“http”:{“hostname”:“www.ipdeny.com”,“url”:“/ipblocks/data/aggregated/ir-aggregated.zone”,“http_user_agent”:“curl/7.61.1”,“http_content_type”:“text/plain”,“http_method”:“GET”,“protocol”:“HTTP/1.1”,“status”:200,“length”:1134},“app_proto”:“http”,“flow”:{“pkts_toserver”:4,“pkts_toclient”:4,“bytes_toserver”:349,“bytes_toclient”:3052,“start”:“2020-03-06T11:59:50.173767+0000”},“payload”:“R0VUIC9pcGJsb2Nrcy9kYXRhL2FnZ3JlZ2F0ZWQvaXItYWdncmVnYXRlZC56b25lIEhUVFAvMS4xDQpIb3N0OiB3d3cuaXBkZW55LmNvbQ0KVXNlci1BZ2VudDogY3VybC83LjYxLjENCkFjY2VwdDogKi8qDQoNCg==”,“stream”:1}

As you can see here, community_id is different for the same session while timestamp is the same (well, now I see there’s a little millisecond variation).

But can that be the problem? Or am I making a configuration error? Or is it a bug?

Suricata is release 5.0.2

2

How does the suricata yaml set up look like for the community flow id? Is the seed the same in both systems ?

3

Thanks Peter. How can I check the seed is the same in both systems?

4

For Suricata,
the configuration values for the community id are in the outputs section.

outputs:
.
.
.
      # enable/disable the community id feature.
      community-id: false
      # Seed value for the ID output. Valid values are 0-65535.
      community-id-seed: 0

The Zeek community ID is discussed here: https://github.com/corelight/zeek-community-id

5

In suricata.yaml it would be here - https://github.com/OISF/suricata/blob/master/suricata.yaml.in#L126

6

Ok, then all seeds are the same in Zeek and Suricata … Both are using seed 1.

7

I have a rust implementation that I checked against. It looks like Zeek is using seed 0 and suricata is using seed 1.

let id = CommunityId::new(&conn);
assert_eq!(id.inner.as_str(), "1:FEAOBSU+mBA7TJlKI8FANpOagdY=");

let id = CommunityId::new_with_seed(&conn, 1);
assert_eq!(id.inner.as_str(), "1:2sl7O0BzoS6G46kkIoRtxQMKWi4=");
8

I can confirm this with my Go implementation:

$ ./gommunityid tuple -seed 0 6 172.22.59.4 149.28.239.174 54692 80
1:FEAOBSU+mBA7TJlKI8FANpOagdY=
$ ./gommunityid tuple -seed 1 6 172.22.59.4 149.28.239.174 54692 80
1:2sl7O0BzoS6G46kkIoRtxQMKWi4=
9

Thank you both very much for your answers. But I am confused. Is not “1:…” the community_id seed showed in alerts logs?.

I have changed community_id to 0 in suricata.yaml config, but inconsistency persist. Another example.

Suricata alert:

{“timestamp”:“2020-04-30T07:51:40.743983+0000”,“flow_id”:1131726089489524,“in_iface”:“vtnet2”,“event_type”:“alert”,“src_ip”:“172.22.55.20”,“src_port”:64302,“dest_ip”:“52.114.158.91”,“dest_port”:443,“proto”:“TCP”,“metadata”:{“flowbits”:[“FB180732_0”,“FB346039_0”,“FB332502_”]},“community_id”:“1:gb5Buy0mfEM/k4BJ0SqFgSq8mjI=”,“tx_id”:0,“alert”:{“action”:“allowed”,“gid”:1,“signature_id”:2028371,“rev”:2,“signature”:“ET
JA3 Hash - Possible Malware - Fake Firefox Font Update”,“category”:“Unknown Traffic”,“severity”:3,“metadata”:{“updated_at”:[“2019_10_29”],“created_at”:[“2019_09_10”],“former_category”:[“JA3”]}},“tls”:{“subject”:“CN=*.events.data.microsoft.com”,“issuerdn”:“C=US,
ST=Washington, L=Redmond, O=Microsoft Corporation, OU=Microsoft IT, CN=Microsoft IT TLS CA 4”,“serial”:“16:00:0A:BD:A3:28:8A:26:AC:EB:F1:78:5E:00:00:00:0A:BD:A3”,“fingerprint”:“33:b3:b7:e9:da:25:f5:a0:04:e9:63:87:b6:fb:54:77:db:ed:27:eb”,“sni”:“self.events.data.microsoft.com”,“version”:“TLS
1.2”,“notbefore”:“2019-10-10T21:55:38”,“notafter”:“2021-10-10T21:55:38”,“ja3”:{“hash”:“a0e9f5d64349fb13191bc781f81f42e1”,“string”:“771,49196-49195-49200-49199-49188-49187-49192-49191-49162-49161-49172-49171-157-156-61-60-53-47-10,0-5-10-11-13-35-23-65281,29-23-24,0”},“ja3s”:{“hash”:“986571066668055ae9481cb84fda634a”,“string”:“771,49200,5-23-65281”}},“app_proto”:“tls”,“flow”:{“pkts_toserver”:4,“pkts_toclient”:6,“bytes_toserver”:463,“bytes_toclient”:6382,“start”:“2020-04-30T07:51:40.396404+0000”},“payload”:“FgMDANoBAADWAwNeqoONjUziLmLtY5OS7iF53HtDLLjvhjNpwd/Ydm8exyAtRwAADycmqVRadO6pSqRFDiTGYvO0caxLRIZh/ukmVAAmwCzAK8AwwC/AJMAjwCjAJ8AKwAnAFMATAJ0AnAA9ADwANQAvAAoBAABnAAAAIwAhAAAec2VsZi5ldmVudHMuZGF0YS5taWNyb3NvZnQuY29tAAUABQEAAAAAAAoACAAGAB0AFwAYAAsAAgEAAA0AFAASBAEFAQIBBAMFAwIDAgIGAQYDACMAAAAXAAD/AQABAA==”,“payload_printable”:"…^…L…b.c…!y.{C,…3i…vo…
-G…’&.TZt…J.E..b..q.KD.a..&T.&.,.+.0.\/..#.(.’.\n…=.<.5./.\n…g…#.!..self.events.data.microsoft.com…\n…\r…#…",“stream”:1}

Zeek conn.log:

{"_node_name":“worker-prod-mgmt”,“ts”:“2020-04-30T07:51:40.396461Z”,“uid”:“C8IlYb3afW8a57UYwl”,“id.orig_h”:“172.22.55.20”,“id.orig_p”:64302,“id.resp_h”:“52.114.158.91”,“id.resp_p”:443,“proto”:“tcp”,“service”:“ssl”,“duration”:110.15475296974182,“orig_bytes”:2136,“resp_bytes”:6509,“conn_state”:“SF”,“local_orig”:true,“local_resp”:false,“missed_bytes”:0,“history”:“ShADdaFf”,“orig_pkts”:13,“orig_ip_bytes”:2668,“resp_pkts”:11,“resp_ip_bytes”:6961,“community_id”:“1:2/7M/RYFCWgAA6N6QTwBYKkomW8=”}

What am I doing wrong?

10

The 1: is a pseudo version number – unrelated to the seed used to construct the community id.

Perhaps you could post a snippet showing the Suricata config, e.g., for Suricata:

$ src/suricata --dump-config|grep community
outputs.1.eve-log.community-id = false
outputs.1.eve-log.community-id-seed = 0

Offhand, I’m not sure what the corresponding steps are to view the Zeek values.

11

Ouch … Found the problem … there are other eve outputs defined previously, and community_id seed is wrong for the alerts. Fixed and working.

Many thanks Jeff.

1 Like