Another update
Currently using only one categories file and two reputation files
# IP Reputation
reputation-categories-file: /etc/suricata/rules/scirius-categories.txt
default-reputation-path: /etc/suricata/rules/iprep/
reputation-files:
- scirius-iprep.list
- test-iprep.list
I mixed all blacklists into test-iprep.list , having around 50500 IPs in the list. List includes the IPs I used as small sample in previous post. I am including their line numbers
10067 176.221.42.32,31,100
6326 113.212.69.128,31,100
4177 108.62.59.27,31,100
3370 95.141.17.244,31,100
33835 216.151.137.155,36,100
32561 173.234.225.161,36,100
30574 108.62.56.222,36,100
30106 95.141.17.10,36,100
None of these IPs are being blocked by Suricata. Same result if I leave all blacklists separated in individual iprep lists.
However if I move these IPs next to the top in the same list, then restart Suricata, the same IPs are being blocked as expected. Again, I am not an expert, but it looks to me that Suricata is not reading/loading/processing the entire list.
After few tests I have narrowed the working range up to 1170 lines of IPs within the list. If IP falls beyond that point, it won’t be blocked. That’s not a large number of IPs to check.
Unfortunately solution suggested by Victor Julien did not work.
In this article, somebody was testing iprep with data sets up to million entries with positive results! Having a small list with only 1170 entries vs a million entries working fine, makes me think that perhaps Suricata is not reading values properly from the Host table or Host table mechanism is not working properly. Is there a way to check the host table or a command to force loading data (other that USR2 or restart Suricata)?
Any other ideas or anything else I could check or try?
Thanks