How to block file transfer with any magic bytes?

I’m looking to be able to block file transfers to the internet from my internal network. I see there’s the magic bytes option in the file extraction module which seems to be on the right lines, but I don’t see an option to match any known file signature.

Is it possible to match on any known magic bytes?

Not really sure if that’s what you want. I seem to recall that Suricata will for instance think of HTTP request and response bodies as files.

There is not really a good definition of what a “file” is when data is moving across the wire.
If you really do want to drop a lot of packets then something like drop $HOME_NET any -> $EXTERNAL_NET any (msg:"somename"; filemagic:!"something not in the filemagic file"; sid:1;) might work.

That seems to have got it, thanks!

Is the file extraction module something that I can now use to save or get an idea of the files that are detected with these rules? I’m thinking not since the file transfer would be blocked so it wouldn’t have the data required to reassemble the file?

I would recommend just trying it out. Enable filestore v2 in the suricata.yaml file and see if you get files.

Looks like it’s only getting the first 1460 bytes of the file. Not ideal but it is manageable I suppose; would just need to search storage devices for those particular bytes and not for a hash.

Unless there’s a way to somehow re-route the transfer of the file to the file-store instead of plain dropping it?

Actually on further inspection I don’t think my rules are picking up all file transfer: I’ve just sent a UDP packet containing a few bytes from the start of an executable and the rules haven’t picked it up. I’m trying to detect any sort of file transfer whatsoever as to prevent potential malware file exfil (at least if it’s not encrypted) which could of course take any form.

It’s not an easy task I suppose as there’d be many ways to work around the detection rules.

I would like to repeat this.

If I send an UDP packet containing this is some string from my program to some server, would that be a file? If my program program reads some other string from a file a.txt and sends that string to some server would that be a file? There are no “files” in network traffic. Some protocols might have a concept about files (ftp for example), most don’t.

Very true. I had thought that filemagic would essentially mean doing a content match on known file signatures (inefficient I know) on all packets traversing the IPS. So by doing this I was hoping to catch multiple different ways files could be exfiltrated over the network.

the filemagic rule keyword matches on what Suricata considers files. Only select protocols support this in suricata: http, ftp, smb, nfs and a few others. The protocol parsers inform the engine where files start and end, then filemagic inspects those.

If you drop based on filemagic you will not get the entire file: the stream will be dropped including the rest of the file.