Windows 10 -2016 File Hash calculation

Help
,
1

Hi… I have a problem with suricata on windows 10 and windows 2016. I try to calculate file hash but with no luck.
i forced hash-calculation…to both eve-json and in file-store v2 config ( also force all files file store) . I also enable writing every files metadata to different json file and it stores all the files metadata with name 0000000…00.epochtime.num.json . Did i missed to install any windows dependencies?

P.S i want to mention that i have many suricata sensors on linux OSes, that all working great…

On windows 10 , I installed Suricata 6.0.1 with npcap 1.20

PS C:\Program Files\Suricata> .\suricata.exe --build-info
11/4/2021 – 13:17:39 - - Running as service: no
This is Suricata version 6.0.1 RELEASE
Features: PCAP_SET_BUFF HAVE_PACKET_FANOUT HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LIBJANSSON TLS TLS_C11 RUST
SIMD support: none
Atomic intrinsics: 1 2 4 8 byte(s)
64-bits, Little-endian architecture
GCC version 10.2.0, C version 201112
compiled with -fstack-protector
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.36, linked against LibHTP v0.5.36

Suricata Configuration:
AF_PACKET support: no
eBPF support: no
XDP support: no
PF_RING support: no
NFQueue support: no
NFLOG support: no
IPFW support: no
Netmap support: no
DAG enabled: no
Napatech enabled: no
WinDivert enabled: no

Unix socket enabled: no
Detection enabled: yes

Libmagic support: no
libnss support: yes
libnspr support: yes
libjansson support: yes
hiredis support: no
hiredis async with libevent: no
Prelude support: no
PCRE jit: yes
LUA support: yes
libluajit: no
GeoIP2 support: yes
Non-bundled htp: no
Hyperscan support: no
Libnet support: no
liblz4 support: yes

Rust support: yes
Rust strict mode: no
Rust compiler path: /mingw64/bin/rustc
Rust compiler version: rustc 1.48.0
Cargo path: /mingw64/bin/cargo
Cargo version: cargo 1.48.0
Cargo vendor: yes

Python support: yes
Python path: /mingw64/bin/python3
Python distutils yes
Python yaml yes
Install suricatactl: yes
Install suricatasc: yes
Install suricata-update: yes

Profiling enabled: no
Profiling locks enabled: no

Plugin support (experimental): no

Development settings:
Coccinelle / spatch: no
Unit tests enabled: no
Debug output enabled: no
Debug validation enabled: no

Generic build parameters:
Installation prefix: /mingw64
Configuration directory: C:\Program Files\Suricata
Log directory: C:\Program Files\Suricata\log

–prefix /mingw64
–sysconfdir /mingw64/etc
–localstatedir /mingw64/var
–datarootdir /mingw64/share

Host: x86_64-w64-mingw32
Compiler: gcc (exec name) / g++ (real)
GCC Protect enabled: yes
GCC march native enabled: no
GCC Profile enabled: no
Position Independent Executable enabled: no
CFLAGS -g -O2 -DOS_WIN32 -std=c11 -I${srcdir}/…/rust/gen -I${srcdir}/…/rust/dist
PCAP_CFLAGS
SECCFLAGS -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security

Thanks in advance,
Christos

2

Is it possible to paste a fileinfo log/record? (enabled in the eve section in suricata.yaml)
They should be "event_type":"fileinfo" in the eve.json log.

3

I will paste the suricata.log asap. But it is exists the fileinfo with all the json keys like txid filename gaps size etc but without the enabled hash keys like md5 sha1 or sha256. I have seen in my linux setups the json keys md5 sha1 and sha256.

4 
  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-alert.json
        types:
            - alert:
                tagged-packets: yes
            
  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-http.json
        types:
            - http:    
                extended: yes
  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-dns.json
        types:
            - dns
  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-tls.json
        types:
            - tls:
                extended: yes
 
  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-files.json
        append: no
        types:
            - files:
                force-magic: yes
                force-hash: [sha256]
  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-smtp.json
        types:
            - smtp:

  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-ftp.json
        types:
            - ftp:

  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-rdp.json
        types:
            - rdp:

  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-nfs.json
        types:
            - nfs:

  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-smb.json
        types:
            - smb:

  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-tftp.json
        types:
            - tftp:

  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-ssh.json
        types:
            - ssh:

  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-flow.json
        types:
            - flow:   
            
  - eve-log:
        enabled: yes
        filetype: regular
        filename: eve-dnp3.json
        types:
            - dnp3:   
    ###########################################################
  - file-store:
      version: 2
      enabled: yes

      # Set the directory for the filestore. Relative pathnames
      # are contained within the "default-log-dir".
      dir: filestore

      # Write out a fileinfo record for each occurrence of a file.
      # Disabled by default as each occurrence is already logged
      # as a fileinfo record to the main eve-log.
      write-fileinfo: yes

      # Force storing of all files. Default: no.
      force-filestore: yes

      # Override the global stream-depth for sessions in which we want
      # to perform file extraction. Set to 0 for unlimited; otherwise,
      # must be greater than the global stream-depth value to be used.
      stream-depth: 0

      # Uncomment the following variable to define how many files can
      # remain open for filestore by Suricata. Default value is 0 which
      # means files get closed after each write to the file.
      #max-open-files: 1000

      # Force logging of checksums: available hash functions are md5,
      # sha1 and sha256. Note that SHA256 is automatically forced by
      # the use of this output module as it uses the SHA256 as the
      # file naming scheme.
      force-hash: [sha1, md5]
      # NOTE: X-Forwarded configuration is ignored if write-fileinfo is disabled
      # HTTP X-Forwarded-For support by adding an extra field or overwriting
      # the source or destination IP address (depending on flow direction)
      # with the one reported in the X-Forwarded-For HTTP header. This is
      # helpful when reviewing alerts for traffic that is being reverse
      # or forward proxied.

###############################################################

from the eve-files.json

{"timestamp":"2021-04-11T20:06:15.796222+0300","flow_id":1508429768358390,"in_iface":"\\DEVICE\\NPF_{4AA86136-917B-45D2-BE98-087B589B8CA0}","event_type":"fileinfo","src_ip":"98.129.229.158","src_port":80,"dest_ip":"10.0.2.15","dest_port":49877,"proto":"TCP","http":{"hostname":"www.zoomify.com","url":"/images/folders/MichaelvonAichbergerBudapest/TileGroup187/10-337-47.jpg","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0","http_content_type":"image/jpeg","http_refer":"http://www.zoomify.com/","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":14103},"app_proto":"http","fileinfo":{"filename":"/images/folders/MichaelvonAichbergerBudapest/TileGroup187/10-337-47.jpg","sid":[],"gaps":false,"state":"CLOSED","stored":true,"file_id":108,"size":14103,"tx_id":16}}
{"timestamp":"2021-04-11T20:06:15.826608+0300","flow_id":1243022264301536,"in_iface":"\\DEVICE\\NPF_{4AA86136-917B-45D2-BE98-087B589B8CA0}","event_type":"fileinfo","src_ip":"98.129.229.158","src_port":80,"dest_ip":"10.0.2.15","dest_port":49874,"proto":"TCP","http":{"hostname":"www.zoomify.com","url":"/images/folders/MichaelvonAichbergerBudapest/TileGroup184/10-344-46.jpg","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0","http_content_type":"image/jpeg","http_refer":"http://www.zoomify.com/","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":22527},"app_proto":"http","fileinfo":{"filename":"/images/folders/MichaelvonAichbergerBudapest/TileGroup184/10-344-46.jpg","sid":[],"gaps":false,"state":"CLOSED","stored":true,"file_id":109,"size":22527,"tx_id":18}}
{"timestamp":"2021-04-11T20:06:15.902003+0300","flow_id":1511590864287892,"in_iface":"\\DEVICE\\NPF_{4AA86136-917B-45D2-BE98-087B589B8CA0}","event_type":"fileinfo","src_ip":"98.129.229.158","src_port":80,"dest_ip":"10.0.2.15","dest_port":49875,"proto":"TCP","http":{"hostname":"www.zoomify.com","url":"/images/folders/MichaelvonAichbergerBudapest/TileGroup187/10-344-47.jpg","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0","http_content_type":"image/jpeg","http_refer":"http://www.zoomify.com/","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":23302},"app_proto":"http","fileinfo":{"filename":"/images/folders/MichaelvonAichbergerBudapest/TileGroup187/10-344-47.jpg","sid":[],"gaps":false,"state":"CLOSED","stored":true,"file_id":110,"size":23302,"tx_id":17}}
{"timestamp":"2021-04-11T20:06:15.926436+0300","flow_id":1923834710260041,"in_iface":"\\DEVICE\\NPF_{4AA86136-917B-45D2-BE98-087B589B8CA0}","event_type":"fileinfo","src_ip":"98.129.229.158","src_port":80,"dest_ip":"10.0.2.15","dest_port":49876,"proto":"TCP","http":{"hostname":"www.zoomify.com","url":"/images/folders/MichaelvonAichbergerBudapest/TileGroup182/10-337-45.jpg","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0","http_content_type":"image/jpeg","http_refer":"http://www.zoomify.com/","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":10811},"app_proto":"http","fileinfo":{"filename":"/images/folders/MichaelvonAichbergerBudapest/TileGroup182/10-337-45.jpg","sid":[],"gaps":false,"state":"CLOSED","stored":true,"file_id":111,"size":10811,"tx_id":17}}
{"timestamp":"2021-04-11T20:06:15.943770+0300","flow_id":2140629626912887,"in_iface":"\\DEVICE\\NPF_{4AA86136-917B-45D2-BE98-087B589B8CA0}","event_type":"fileinfo","src_ip":"98.129.229.158","src_port":80,"dest_ip":"10.0.2.15","dest_port":49873,"proto":"TCP","http":{"hostname":"www.zoomify.com","url":"/images/folders/MichaelvonAichbergerBudapest/TileGroup190/10-337-48.jpg","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0","http_content_type":"image/jpeg","http_refer":"http://www.zoomify.com/","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":13695},"app_proto":"http","fileinfo":{"filename":"/images/folders/MichaelvonAichbergerBudapest/TileGroup190/10-337-48.jpg","sid":[],"gaps":false,"state":"CLOSED","stored":true,"file_id":112,"size":13695,"tx_id":17}}
5

Sorry for the late follow up.
What is the output of suricata --build-info ?

6

No problem at all,
plz check the first message of this topic… it is posted there

Thanks

7

I apologize for missing the obvious!
I think this could be logged as a bug on our redmine Issues - Suricata - Open Information Security Foundation

Another check maybe that could be useful - do you get the same result when reading a pcap ?
I need to double check this with 6.0.2

8

Goodmorning, Yes i ran 3 - 4 Pcaps (-r mode) which contained files for extraction with the same results . Also as i have read the suricata is working only in pcap mode even with -i mode on Windows so i think that these results even in -r mode are something that should be expected. ( But you never know :smile: )

Thanks in advance