Alerting on IOCs

Help
1

Hi,

Is there a generic way in suricata to match on stuff from a file? It’s possible for IPs and FQDNs as far as I know.

I have multiple files containing indicators, as follows:

  1. Domains:
    How do I get alerted on domains from a file, matching on a field (eg: dns_query) with any subdomain? For example if I have google.com listed in the file, can I get alerts on images.google.com in dns_query field?

Example file domains.txt would contain:
google.com
facebook.com
apple.com

  1. URLs:
    Same for domains above, but for URLs or URIs. How do I get alerted on them in HTTP traffic for example?

Example file urls.txt would contain:
/example/url.php
/example2/script.js

Any guidance is appreciated. Thanks a lot for your help.

2

Datasets could be helpful.

Using the dataset and datarep keyword it is possible to match on large amounts of data against any sticky buffer.

3

Thank you for your reply.

That looks like an exact match, but how do I use a data entry (eg: google.com.uk) from a dataset in my rule and manipulate it to match random subdomains (eg: random.subdomain.google.com.uk)

Thanks

4

Ah, I see. Should have read your question more carefully :wink:
Indeed, generic subdomain matching won’t work with datasets, only if you can enumerate all potential ‘interesting’ subdomains and explicitly list them in the dataset as well.

Otherwise, if you only have a few thousand domains, you could autogenerate rules that use each domain to match against the dns.query buffer with an endswith keyword.

5

Hi,

A very basic example using http.uri for ICOs:

dataset.rules:
alert http any any -> any any (msg:"HTTP URI en lista http-uri IOCs ";http.uri;dataset:isset,http-uri;classtype:external-ip-check;sid:696974;)

suricata.yaml:

   http-uri:
     type: string
     load: /etc/suricata/rules/http-uri.lst

http-uri:

/index.php
/admin.php
/login.php
/index.html

fast.log:

set
set1634×34 4.86 KB

6

Unfortunately I’ve more than 50k indicators.

7

you can do this by applying a pcrexform transformation on the sticky buffer you want to alert on (eg: dns.query, tls.sni…etc) and then matching against the IOCs datasets

The pcrexform transformation is a regex match with a regex capture expression in it, the first capture group match will be the output that will be compared against the values in your IOCs DataSet.

so the easies thing is to do is to extract the last portion of the domain domain using regex, example:

random.subdomain.google.com.uk —> pcrexform:".+.([-a-zA-Z9-9]+.[-a-zA-Z9-9]+.[-a-zA-Z9-9]+)"; —> google.com.uk

so the final rule should be something similar to:

alert dns any any -> any any (msg:”dns IOCs match”; dns.query;  pcrexform:".+\.([-a-zA-Z9-9]+\.[-a-zA-Z9-9]+\.[-a-zA-Z9-9]+)"; dataset:isset,dns-iocs; sid:123; rev:1;)

the above regex was a quick one, you can find more robust implementations on the internet, but it should work.

3 Likes
8

Interesting. I was not aware of the pcrexform keyword.
Do you know how bad the performance hit is? I would imagine it hurts quite a bit since you would skip MPM and the PCRE would run on each dns query? Would be interesting to compare to say 10k dns.query rules with non-matching text in the content keyword.

9

50k indicators can be manageable depending the amount of traffic and the HW available.

1 Like
10

Hí,

Excuse me. I don’t understand that regex applied to this example

11

I suggest you take a look at https://regexone.com/

12

Adding another resource that I find easy and useful to understand regexes: https://regex101.com