No stored file in filestore

Help
1

Hello
We have some incomprehensible case with files which should be stored when policy is violated.
our custom rules are:

alert http any any → any any (http_content_type; content:“application/json”; sid:7;)
alert http any any → any any (http_content_type; content:“application/ld+json”; sid:8;)
alert http any any → any any (http_content_type; content:“application/x-httpd-php”; sid:9;)
alert http any any → any any (http_content_type; content:“application/xml”; sid:10;)
alert http any any → any any (http_content_type; content:“application/xhtml+xml”; sid:11;)
alert http any any → any any (http_content_type; content:“text/plain”; sid:12;)
alert http any any → any any (http_content_type; content:“text/html”; sid:13;)
alert http any any → any any (http_content_type; content:“text/xml”; sid:14;)
alert http any any → any any (http_content_type; content:“text/csv”; sid:15;)
alert http any any → any any (content:“POST”; http_method; sid:16;)
pass http any any → any any (msg:“IMAGE”; http_content_type; content:“image”; filestore; sid:1;)
pass http any any → any any (msg:“VIDEO”; http_content_type; content:“video”; filestore; sid:2;)

we use the redis.sock to connect and send all info:
and now. in json which is in redis we have:

1645519536.384411 [0 unix:/tmp/redis.sock] “LPUSH” “suricata” “{"timestamp":"2022-02-22T09:45:36.383052+0100","flow_id":1470354718486685,"in_iface":"ens36","event_type":"fileinfo","src_ip":"2a03:2880:f016:0014:face:b00c:0000:0003","src_port":443,"dest_ip":"2001:0a17:6060:3e00:2025:22c5:c1ab:f35c","dest_port":62338,"proto":"TCP","http":{"http_port":0,"url":"/libhtp::request_uri_not_seen","http_content_type":"image/jpeg","status":200,"length":132323},"app_proto":"http","fileinfo":{"filename":"/libhtp::request_uri_not_seen","sid":[1],"magic":"JPEG image data, JFIF standard 1.02, aspect ratio, density 1x1, segment length 16, progressive, precision 8, 2048x1344, frames 3","gaps":false,"state":"CLOSED","sha256":"3c7b8a4225ffe2e363a16fbc53184546a5d4aabe6cadad1b65a062c570957aff","stored":true,"file_id":12045,"size":132323,"tx_id":0},"host":"katwork01-suricata"}”
1645519538.358354 [0 unix:/tmp/redis.sock] “LPUSH” “suricata” "{"timestamp":"2022-02-22T09:45:38.356055+0100","flow_id":751347128480806,"in_iface":"ens36","event_type":"fileinfo","src_ip":"52.84.194.22","src_port":443,"dest_ip":"100.69.244.1","dest_port":55398,"proto":"TCP","http":{"http_port":0,"url":"/libhtp::request_uri_not_seen","http_content_type":"image/webp","status":200,"length":6478},"app_proto":"http","fileinfo":{"filename":"/libhtp::request_uri_not_seen","sid":[1],"magic":"RIFF (little-endian) data, Web/P image, VP8 encoding, 300x168, Scaling: [none]x[none], YUV color, decoders should clamp","gaps":false,"state":"CLOSED","sha256":"8fd48d39d719cdf386d58498bfd494c2b9431cae35a105a64302e9b3586fac20","stored":true,"file_id":12047,"size":6478,"tx_id":0},"host":"katwork01-suricata"}

there is an information that file

8fd48d39d719cdf386d58498bfd494c2b9431cae35a105a64302e9b3586fac20","stored":true,"file_id":12047,"size":6478,

but on host we havent his file on filestore:

root@host:/mnt/ceph_storage/suricata/filestore/katwork01-suricata# ls 37/3c7b8a4225ffe2e363a16fbc53184546a5d4aabe6cadad1b65a062c570957aff
ls: cannot access ‘37/3c7b8a4225ffe2e363a16fbc53184546a5d4aabe6cadad1b65a062c570957aff’: No such file or directory
Blockquote

we run suricata as an container wit parameters below:

    environment:
      - SURICATA_OPTIONS=-i ens36 -vvv -c /opt/suricata.yaml --set outputs.11.file-store.dir=/mnt/ceph_storage/suricata/filestore/{{.Node.Hostname}}-suricata
      - TZ=Europe/Warsaw`Preformatted text`

I have a suspicion that information to redis (in json) are send independently than fs operation. It is possible to send this kind of information(stored:true) after the file is moved from tmp directory to final destination ?

2

What version are you running and how does your config look like?
Can you post stats.log and also the suricata.log?
Do you have examples where it’s working?

3

Hi Andreas
we use version Suricata 6.0.4
this situation not happened each time. very often the files are moved immidietly but also very often are not. i also found that files which is marked as stored:true in redis sometimes are moved after few minutes (even 10 minutes) . it is problem for us because we read information from redis immidietly and the files is absent. it mean that files are in tmp directory but not in destinetion directories (file name with hash)

stats.zip (238.1 KB)
suricata.yml (70.5 KB)

4

The only thing that I catch is segment_memcap_drop which could mean in theory that you miss something.
Do you see a pattern when it doesn’t work?
Could it be related to the storage? or any other process running on the system?

Would it be possible to run tests outside of a docker container? just to rule out that this is the issue (see your other thread with the lib url issue)