Suricata and IP blacklist

ManuelFFF · January 13, 2021, 8:25pm

Hi,

I would like to know if there’s a way to configure Suricata as IPS to block all traffic to/from bad IP addresses given a blacklist file? I have been researching for a while and it looks like “IP Reputation” mechanism could do the work, but I have not been able to do it on my own.

Can you please help me? If IPREP it is not the way, please point me in the right direction.

Thank you

syoc · January 14, 2021, 9:43am

IP Reputation should be the way to go.
Can you elaborate on what you have tried and why it did not work out for you?

ManuelFFF · January 14, 2021, 2:12pm

Hi @syoc ,

As a side note, I am using Suricata 6.0 that comes as part of SELKS 6 suite.

This is what I have tried so far:

Added new reputation file to config file /etc/suricata/selks6-addin.yaml

# IP Reputation
reputation-categories-file: /etc/suricata/rules/scirius-categories.txt
default-reputation-path: /etc/suricata/rules
reputation-files:
 - scirius-iprep.list
 - test-iprep.list

Added new category to config file /etc/suricata/rules/scirius-categories.txt. Category code (10) and sid (2600000) where chosen by me, trying to follow the same pattern found in file. I may be wrong.

1,2400000,ET DROP Spamhaus DROP Listed Traffic Inbound
2,2402000,ET DROP Dshield Block Listed Source
3,2403300,ET CINS Active Threat Intelligence Poor Reputation IP
4,2404000,ET CNC Shadowserver Reported CnC Server IP
5,2404300,ET CNC Feodo Tracker Reported CnC Server
6,2500000,ET COMPROMISED Known Compromised or Hostile Host Traffic
7,2520000,ET TOR Known Tor Exit Node Traffic
8,2522000,ET TOR Known Tor Relay/Router (Not Exit) Node Traffic
9,2525000,ET 3CORESec Poor Reputation IP
10,2600000,TEST Known bad hosts

Added new iprep list file to /etc/suricata/rules/. This is the same file referenced in /etc/suricata/selks6-addin.yaml
Added at least one line with a bad IP to test 185.144.180.3,10,100. I tried to follow the same pattern found in /etc/suricata/rules/scirius-iprep.list
Then I tried to add a rule to use rule directive iprep
alert ip any any -> any any (msg:"TEST IP Bad Reputation Blacklist"; iprep:any,2600000,<,99; sid:1; rev:1;)
I got errors when trying to add the new rule via Scirius GUI

Source test failure:

    SC_ERR_UNKNOWN_VALUE: unknown iprep category "2600000"
    SC_ERR_INVALID_SIGNATURE: error parsing signature "alert ip any any -> any any (msg:"TEST IP Bad Reputation Blacklist"; iprep:any,2600000,<,99; sid:1; rev:1;)"

Thank you in advance

syoc · January 14, 2021, 6:40pm

Thank you for a detailed description.
I have to admit that I’m not that familiar with IPRep myself, but the docs indicate to me that your approach should work.
Seems like the IPRep categories file might not be properly updated.

syoc · January 14, 2021, 7:07pm

Tested in a Docker container just now. Works fine on my machine. I would check the files for typos on disk.

root@f8600f681624:/etc/suricata# vim suricata.yaml 
root@f8600f681624:/etc/suricata# suricata -T
14/1/2021 -- 20:02:04 - <Info> - Running suricata under test mode
14/1/2021 -- 20:02:04 - <Notice> - This is Suricata version 6.0.1 RELEASE running in SYSTEM mode
14/1/2021 -- 20:02:04 - <Error> - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening ip rep file /etc/suricata/iprep/categories.txt: No such file or directory
14/1/2021 -- 20:02:04 - <Error> - [ERRCODE: SC_ERR_NO_REPUTATION(224)] - failed to load reputation categories file /etc/suricata/iprep/categories.txt
14/1/2021 -- 20:02:04 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /etc/suricata/rules/suricata.rules
root@f8600f681624:/etc/suricata# mkdir iprep
root@f8600f681624:/etc/suricata# cd rules/
root@f8600f681624:/etc/suricata/rules# echo 'alert ip any any -> any any (msg:"TEST IP Bad Reputation Blacklist"; iprep:any,2600000,<,99; sid:1; rev:1;)' > suricata.rules
root@f8600f681624:/etc/suricata/rules# cd ..
root@f8600f681624:/etc/suricata# cd iprep/
root@f8600f681624:/etc/suricata/iprep# ls
root@f8600f681624:/etc/suricata/iprep# vim categories.txt 
root@f8600f681624:/etc/suricata/iprep# echo 185.144.180.3,10,100 > reputation.list
root@f8600f681624:/etc/suricata/iprep# suricata -T
14/1/2021 -- 20:03:24 - <Info> - Running suricata under test mode
14/1/2021 -- 20:03:24 - <Notice> - This is Suricata version 6.0.1 RELEASE running in SYSTEM mode
14/1/2021 -- 20:03:24 - <Notice> - Configuration provided was successfully loaded. Exiting.
root@f8600f681624:/etc/suricata/iprep# cat categories.txt 
10,2600000,TEST Known bad hosts

The only thing I did in suricata.yaml was to uncomment the iprep lines and change the rule path from /var/lib/suricata to /etc/suricata

ManuelFFF · January 14, 2021, 8:35pm

Hi @syoc,

I was able to reproduce your steps, not in Docker, but in SELKS 6. This time I updated Suricata up to v7.0.0. After setup config files and ran Suricata in Test mode, I got the same positive results. I am thinking that perhaps is Scirius (the web GUI that comes with SELKS to manage Suricata), the piece that is failing to update Suricata settings and not Suricata itself.

I just tried one more time setting up Suricata via console and no errors appeared regarding my new settings. However, I tried via Scirius and got the same error when attempting to add new custom source, containing the custom signature.

pevma · January 14, 2021, 9:14pm

You need to add the customized category file as well in that case.

Datasets might be better way to go to do that too.

ManuelFFF · January 14, 2021, 9:40pm

I think I celebrated to soon. I disabled all rules and forced Suricata to use only my custom rule file for testing, but I got the parsing error:

user1@server1:~$ sudo suricata -c /etc/suricata/suricata.yaml -s /etc/suricata/rules/test-blacklist.rules -i enp2s0
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:265) <Info> (ConfYamlParse) -- Including configuration file /etc/suricata/selks6-addin.yaml.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'default-rule-path' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'rule-files' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'classification-file' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'reference-config-file' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'detect' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'default-log-dir' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'stats' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'outputs' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'logging' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'app-layer' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'asn1-max-frames' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:265) <Info> (ConfYamlParse) -- Including configuration file /etc/suricata/selks6-interfaces-config.yaml.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'af-packet' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'pcap' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'pfring' redefined.
[14184] 14/1/2021 -- 16:31:39 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'netmap' redefined.
[14184] 14/1/2021 -- 16:31:39 - (suricata.c:1066) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (f8fef0dd0 2021-01-05) running in SYSTEM mode
[14184] 14/1/2021 -- 16:31:39 - (detect-parse.c:2379) <Error> (DetectEngineAppendSig) -- [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert ip any any -> any any (msg:"TEST IP Bad Reputation Blacklist"; iprep:any,2600000,<,99; sid:1; rev:1;)"
[14184] 14/1/2021 -- 16:31:39 - (detect-engine-loader.c:185) <Error> (DetectLoadSigFile) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert ip any any -> any any (msg:"TEST IP Bad Reputation Blacklist"; iprep:any,2600000,<,99; sid:1; rev:1;)" from file /etc/suricata/rules/test-blacklist.rules at line 1
[14185] 14/1/2021 -- 16:31:39 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 19 files.
[14186] 14/1/2021 -- 16:31:39 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 19 files.
[14187] 14/1/2021 -- 16:31:39 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 19 files.
[14188] 14/1/2021 -- 16:31:39 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 19 files.
[14184] 14/1/2021 -- 16:31:39 - (tm-threads.c:2004) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> W: 4 FM: 1 FR: 1   Engine started.

syoc · January 15, 2021, 8:22am

This is a different error about duplicate signatures. You might have multiple rules in that rule file with sid:1;

Can you use datasets for IP matching?

ManuelFFF · January 15, 2021, 3:27pm

Hi,

I figured out how to fix the duplicate error and also SELKS team told me how to fix the error that I was getting from Scirius. This time I was able to add the custom categories, signatures/rules and reputation file via Scirius without errors.

What would be the best way to test my new signatures/rules?

Thank you

ManuelFFF · January 15, 2021, 3:28pm

Hi,

Could you please explain me how to use datasets for IP matching as recommended by you and Pevma?

Thanks

syoc · January 16, 2021, 9:38am

Might be nice to post or link the solution SELKS here if others have the same problem.

Not so sure about rule testing. If this is something you run on your own home network you can just add the IP for some random website and try visiting it.

pevma · January 16, 2021, 4:50pm

Just FYI for Scirius side - Basically upload all files in the tar - Suricata IPREP rules · Issue #289 · StamusNetworks/SELKS · GitHub

ManuelFFF · January 18, 2021, 1:49pm

Well, Pevma got ahead of me and already published the post with SELKS’s answer . Thank you

ManuelFFF · January 26, 2021, 4:47pm

Hi there,

I had to pause my researching due to hardware issues. Issues were resolved and now I can move forward.

Summarizing what I have so far:

Running Suricata v6 and v7 as part of SELKS 6 suite
Setup Suricata in IPS mode
I am implementing the L2 config approaching (af-packet). Machine 150 is hosting Suricata, so I want Suricata to ignore that traffic for detection.

selks6-interfaces-config.yaml

af-packet:
  - interface: enxa0cec8d92d70
    threads: 1
    defrag: yes
    cluster-type: cluster_flow
    cluster-id: 98
    copy-mode: ips
    copy-iface: enxa0cec8d92e2e
    tpacket-v3: no
    ring-size: 2048
    buffer-size: 64535
    use-mmap: yes
    bpf-filter: not host 192.168.1.150
 
 - interface: enxa0cec8d92e2e
    threads: 1
    cluster-id: 97
    defrag: yes
    cluster-type: cluster_flow
    copy-mode: ips
    copy-iface: enxa0cec8d92d70
    tpacket-v3: no
    ring-size: 2048
    buffer-size: 64535
    use-mmap: yes
    bpf-filter: not host 192.168.1.150

pcap:
  - interface: enxa0cec8d92d70
    cluster-id: 98
    bpf-filter: "not host 192.168.1.150"
  - interface: enxa0cec8d92e2e
    cluster-id: 97
    bpf-filter: "not host 192.168.1.150"

pfring:
  - interface: enxa0cec8d92d70
    cluster-id: 98
    bpf-filter: not host 192.168.1.150
  - interface: enxa0cec8d92e2e
    cluster-id: 97
    bpf-filter: not host 192.168.1.150

netmap:
  - interface: enxa0cec8d92d70
    cluster-id: 98
    bpf-filter: not host 192.168.1.150
  - interface: enxa0cec8d92e2e
    cluster-id: 97
    bpf-filter: not host 192.168.1.150

Added custom Suricata rule via Scirius (webGUI for SELKS)
Added to Suricata config

# IP Reputation
reputation-categories-file: /etc/suricata/rules/scirius-categories.txt
default-reputation-path: /etc/suricata/rules
reputation-files:
- scirius-iprep.list
- test-iprep.list

default-rule-path: /etc/suricata/rules
rule-files:
- scirius.rules

scirius-categories.txt
1,2400000,ET DROP Spamhaus DROP Listed Traffic Inbound
2,2402000,ET DROP Dshield Block Listed Source
3,2403300,ET CINS Active Threat Intelligence Poor Reputation IP
4,2404000,ET CNC Shadowserver Reported CnC Server IP
5,2404029,ET CNC Shadowserver Reported CnC Server
6,2404300,ET CNC Feodo Tracker Reported CnC Server
7,2500000,ET COMPROMISED Known Compromised or Hostile Host Traffic
8,2520000,ET TOR Known Tor Exit Node Traffic
9,2522000,ET TOR Known Tor Relay/Router (Not Exit) Node Traffic
10,2525000,ET 3CORESec Poor Reputation IP
30,TESTBadIP,TEST Known Bad IP Reputation

test-iprep.list
149.20.4.15,30,100

scirius.rules
# Rules file for Default SELKS ruleset generated by Scirius at 2021-01-26 15:49:40.671556+00:00
drop ip any any -> any any (msg:"TEST IP Bad Reputation Blacklist"; iprep:any,TESTBadIP,>,99; sid:1; rev:1;)

The internal af-packet bridge in Suricata seems to be working fine, as I can reach a test machine behind Suricata and also that machine can reach Internet

So far so good.
Now I am testing the only rule enabled in Suricata, which should block IP 149.20.4.15 (www.debian.org). Well, traffic is not being blocked. What could I be doing wrong or missing?

Thank you

ManuelFFF · January 28, 2021, 7:19pm

Hi,

I have Suricata running in IPS mode (L2 configuration) already. I was missing a couple of settings in to convert rules from alert to drop. After that I have been running tests. All rules are IP-Only using the iprep control. During these tests all other Suricata rules are disabled.

First I just added an IP reputation list with a couple of IPs. All were successfully blocked. after that I decided to try next level and automate the sources. Downloaded a few lists from Internet and parsed all list to match the categories within Suricata. The last results showed me that not all the traffic from the new reputation list it’s being blocked. Does anybody have any idea about what could be failing or what should I check? If you need more details, let me know.

Please help!

Thanks

ManuelFFF · January 29, 2021, 4:04pm

Hi,

I continued trying different things. This morning I have been trying this new config, combining several feeds into one iprep file:

Removed all iprep files
Left only two iprep files in
Added all feeds into test-iprep.list
Restarted Suricata
Tried access from test machine behind Suricata

Results:

Custom iprep file has now 1117 lines
First 5 sites (IPs added manually) remain being blocked
At least 8 IPs (this is just a small sample) from different feeds where NOT blocked while were in separate iprep files, and same IPs are not being blocked now that are in the same iprep file.
I did an additional test leaving in the file test-iprep.list only the 5 IPs added manually plus the 8 IPs in conflict. This time the 8 IPs in conflict were successfully blocked. I kept their categories within the iprep file:

149.20.4.15,30,100
128.31.0.62,30,100
176.221.42.32,31,100
113.212.69.128,31,100
108.62.59.27,31,100
95.141.17.244,31,100
216.151.137.155,36,100
173.234.225.161,36,100
108.62.56.222,36,100
95.141.17.10,36,100

This leads me to believe that Suricata is not fully reading the iprep files, or that there is a limit to the number of lines it can read / load.

Is there anything we can do to fix this issue? Is there a variable or config to modify/remove this limit?

Please help!

Thank you

pevma · January 29, 2021, 4:16pm

I think it may not be reading in all lists as you suggest , might worth to open an issue on the suricata redmine tracker.

ManuelFFF · January 29, 2021, 5:12pm

Oh, that’s a pity. I was hopping that it was something I missed or configured incorrectly.
Could you confirm/reproduce the same issue on your end?

Is this the site to report a bug?
https://redmine.openinfosecfoundation.org/projects/suricata/issues?set_filter=1&tracker_id=1

ManuelFFF · January 29, 2021, 5:50pm

Report submitted
https://redmine.openinfosecfoundation.org/issues/4280

If somebody could, please tell me if datasets will serve the same purpose, if this could avoid the problem I have right now, and if somebody could give me more details about its implementation.

Thank you

Topic		Replies	Views
Suricata IP Reputation / Blacklisting Help	2	823	February 10, 2022
Trusted IP's Where a put Help	1	686	October 24, 2020
Config Question: Suricata as an IDS + source IP Blocking on VPS w/1 network interface Help	9	2116	February 14, 2022
I cannot confirm whether Suricata is intercepting malicious traffic as an IPS Help ips	2	52	August 24, 2024
Suricata AF-PACKET IPS not dropping Help ips	7	1186	February 16, 2022

Suricata and IP blacklist

Related topics