we are using Suricata to analyze hundreds of thousands of PCAP files. We are using the following options to avoid having to restart Suricata for every PCAP file that we submit to the system:
-r <path>
Run in pcap offline mode (replay mode) reading files from pcap file. If specifies a directory, all files in that directory will be processed in order of modified time maintaining flow state between files.
--pcap-file-continuous ``
Used with the -r option to indicate that the mode should stay alive until interrupted. This is useful with directories to add new files and not reset flow state between files.
My question is as follow. Does Suricata reset the knowledge/communication states it gathers between each PCAP file? If not, would it be possible to request a reset to tell the system that every PCAP is independent from one another such as a --pcap-reset option that could be used.
Hi.
My experience is that some state is reset, some is not.
What state do you need to be reset between each pcap?
Not aware of any state resetting options, though you can do live rule reloading.
Thanks for the quick reply. Here is my use cases. We execute PE file on virtual machines where we gather the resulting PCAP file. Of course after each PE file execution we reset the virtual machine to a specific snapshots. We then give these files to Suricata for analysis. As you probably understand all these overlaps (session timestamp, similar connextions). For instance, it some cases the same normal Windows traffic and requests are par of every PCAP because of going back to the snapshot.
So what I would like to confirm if every file given to Suricata using the above mention options will be analyzed independently of each other (not remembering that a TCP session has started in one PCAP and to expect to be closed into another) and that rules that have been triggered were triggered only using the data in the current PCAP files and not using data in the previous one.
If this is not the case, how could we analyzed PCAP file very rapidly and independently. Restarting Suricata every single time is very long.
Does the live rule reloading would accomplish this? It should trigger the TCP session reset since we change the rules live?
Do you have pcaps where you could reproduce a different behavior if you run each pcap with a freshly restarted Suricata compared to the one with all the pcaps in one run?
This problem is easy to reproduce with any large group of pcap files that are very similar and overlap in network traffic (see description above). If we restart Suricata after each PCAP file our program controlling the Suricata process can run for months. However, if we do not restart it after a few thousand pcaps (using the ingest folder option where we constantly add pcap files), it hangs and does not complete. This usually happens within a day. We are currently using Suricata version 5.0.0. We will try upgrade to the latest version and see if this will happen again and report back here.