Conditional pcap-log fails to log packets for some alerts when using "pcap-file-continuous" flag

Help
1

Hello,
I am using the new feature “Conditional PCAP” on Suricata v.7.rc1 to log alert packets.
I run Suricata with pcap-log enabled and conditional set to alerts. My Suricata instance is used only to process PCAPs with pcap-file-continuous enabled.

After a few months of testing this feature, I encountered a disturbing problem. It seems that when I play PCAPs to Suricata with the pcap-file-continuous configuration enabled, Suricata fails to log the packets for some alerts.

Scenarios I tested:

  • Run Suricata with the flag -r <path> → Play PCAPS → extract the packets from pcap-log based on eve.json (using GopherCap) → Works fine!
  • Run Suricata with the flag -r <path> and --pcap-file-continuous → Play the same PCAPS → extract the packets from pcap-log based on eve.json (using GopherCap) → Failed to extract the packets for some of the alerts.
  • Run Suricata with the flag -r <path> and --pcap-file-continuous → Play the same PCAPS → stopped Suricata → extract the packets from pcap-log based on eve.json (using GopherCap) → Works fine!

Here are my pcap-log settings:

enabled: yes
filename: pcap.%n.%t
limit: 20mb
max-files: 5
compression: none
mode: multi
dir: /var/log/suricata/pcap
use-stream-depth: no
honor-pass-rules: no
conditional: alerts

and my stream configuration:

memcap: 64mb
checksum-validation: yes
inline: auto
reassembly:
memcap: 256mb
depth: 1mb # reassemble 1mb into a stream
toserver-chunk-size: 2560
toclient-chunk-size: 2560
randomize-chunk-size: yes

Here is an example of signatures that Suricata usually fails to log packets for (given the PCAPs I played):

alert dns any any → $HOME_NET any (msg:“ET DNS Reply Sinkhole - sinkhole.cert.pl 148.81.111.111”; content:“|00 01 00 01|”; content:“|00 04 94 51 6f 6f|”; distance:4; within:6; classtype:trojan-activity; sid:2016413; rev:5; metadata:created_at 2013_02_15, former_category DNS, updated_at 2022_07_13;)

alert tls $EXTERNAL_NET any → $HOME_NET any (msg:“ET WEB_CLIENT Possible eDellRoot Rogue Root CA”; flow:established,to_client; tls.cert_issuer; content:“CN=eDellRoot”; fast_pattern; reference:url,Dell does a Superfish, ships PCs with easily cloneable root certificates | Ars Technica; classtype:trojan-activity; sid:2022134; rev:4; metadata:affected_product Web_Browsers, affected_product Web_Browser_Plugins, attack_target Client_Endpoint, created_at 2015_11_24, deployment Perimeter, signature_severity Major, tag Web_Client_Attacks, updated_at 2022_03_24;)

Please, Can anyone help me figure out whether this is a bug or this is just a configuration issue?

Thank you in advance!

2

@Regit would you know ?

1 Like
3

Disclaimer, I do not know this pcap-file-continuous option and what it should do

1 Like
4

This is really really strange.

How many alerts are you missing ?

Could you share an missed alert json (don’t need the IP information there) ?

1 Like
5

How many alerts? It depends. but a lot. for example, for a file which raises 72 alerts, 12 are missing PCAP log.

Examples of missed alerts:

{
“timestamp”: “2023-03-21T14:35:36.996508+0000”,
“flow_id”: 22837624374537,
“pcap_cnt”: 12459,
“event_type”: “alert”,
“src_ip”: SRC_IP,
“src_port”: 33072,
“dest_ip”: DST_IP,
“dest_port”: 80,
“proto”: “TCP”,
“pkt_src”: “wire/pcap”,
“tx_id”: 0,
“alert”: {
“action”: “allowed”,
“gid”: 1,
“signature_id”: 900000,
“rev”: 1,
“signature”: “Detected Nmap’s User-Agent within an HTTP request”,
“category”: “29”,
“severity”: 2,
“rule”: “alert http any any → $HOME_NET any (msg:"Detected Nmap’s User-Agent within an HTTP request"; flow:to_server,established; http.user_agent; content:"Nmap"; classtype:exploit-kit; sid:900000; rev:1;)”
},
“app_proto”: “http”,
“direction”: “to_server”,
“flow”: {
“pkts_toserver”: 4,
“pkts_toclient”: 3,
“bytes_toserver”: 483,
“bytes_toclient”: 598,
“start”: “2023-03-21T14:35:36.922821+0000”,
“src_ip”: SRC_IP,
“dest_ip”: DST_IP,
“src_port”: 33072,
“dest_port”: 80
},
“capture_file”: “/suricata_log_output//pcap.2.1679408622”,
“pcap_filename”: “pcaps/attacks_1.pcap”
},

another example:

{
“timestamp”: “2023-03-21T14:50:57.311032+0000”,
“flow_id”: 491447565889281,
“pcap_cnt”: 54618,
“event_type”: “alert”,
“src_ip”: SRC_IP,
“src_port”: 47426,
“dest_ip”: DST_IP,
“dest_port”: 53,
“proto”: “UDP”,
“pkt_src”: “wire/pcap”,
“tx_id”: 0,
“alert”: {
“action”: “allowed”,
“gid”: 1,
“signature_id”: 2022048,
“rev”: 3,
“signature”: “ET MALWARE Cryptowall .onion Proxy Domain”,
“category”: “36”,
“severity”: 1,
“metadata”: {
“created_at”: [
“2015_11_09”
],
“updated_at”: [
“2020_08_18”
]
},
“rule”: “alert dns $HOME_NET any → any any (msg:"ET MALWARE Cryptowall .onion Proxy Domain"; dns.query; content:"3wzn5p2yiumh7akj"; depth:16; nocase; reference:url,CryptoWall 4.0 released with new Features such as Encrypted File Names; classtype:trojan-activity; sid:2022048; rev:3; metadata:created_at 2015_11_09, updated_at 2020_08_18;)”
},
“app_proto”: “dns”,
“direction”: “to_server”,
“flow”: {
“pkts_toserver”: 1,
“pkts_toclient”: 0,
“bytes_toserver”: 105,
“bytes_toclient”: 0,
“start”: “2023-03-21T14:50:57.311032+0000”,
“src_ip”: SRC_IP,
“dest_ip”: DST_IP,
“src_port”: 47426,
“dest_port”: 53
},
“capture_file”: “/suricata_log_output//pcap.2.1679408622”,
“pcap_filename”: “pcaps/attacks_1.pcap”
},

6

@Eric_Leblond you need more examples of the missed alerts?

7

@Eric_Leblond
Actually I can reproduce the bug by loading the following simple rule to Suricata:
alert tcp any any -> any any (msg:"tcp"; classtype:policy-violation; sid:1; rev:1;)
When I feed Suricata with a PCAP file containing a single TCP packet, I get an alert, but the packet is not written to the pcap log.

8

Can you provide that pcap as well?