Http.log failed to log all http\s events

1

Hello,

I’m using Suricata 6.0.5
my http.log failed to log all http requests
But it’s look like alerts log doing well when I triggered alerts.

2

Thanks for including the Suricata version you’re using.

Other things that will help tracking down the problem are

  • A pcap file that demonstrates the issue when replayed
  • Suricata configuration (usually suricata.yaml)
  • The stats log file (usually stats.log)
3

Hi Jeff,
I have sent the files to you in person.
Important- i have checked the YAML in my lab env and it works perfect, so maybe it’s a performance issue that could be handle with different config or so

Thanks.

4

Hi,
I will appreciate any reply :slight_smile:

Thanks

5

Thanks to Jeff who resolve my problem!

The fix is to add this to suricata.yaml:

stream:
   midstream: true

The default value is false; when set to true, Suricata will track sessions even if the session initiation isn’t seen by Suricata.

Also I increased the memcap size and depth

6

Is there packet loss during the rush hours?

Packet loss is caused by back pressure on resources used to handle packets:

  • NIC resources (memory used to receive packets, capacity to receive packets)
  • Memory resources (host memory used to hold packets that are to be analyzed by Suricata)
  • Suricata resources (memory and processing capacity).

Without a full picture of the software and hardware components, increasing the memory limits for Suricata may be one of the recommended actions for reducing packet drops.

7

I believe packet loss is not the problem, because-

  1. at tcpdump I can see all my queries
  2. I made a custom snort signature that triggered every time I add a particular string to my HTTP query, that signature alerts perfectly on any query, even during rush hours.
  3. this statistic:
    image
  4. at rush hours CPU at 60% and memory at 70%

Can you tell the difference between snort sigs and HTTP logs? why do the sigs work fine and the HTTP logs doesn’t?

8

Alerts (generated when traffic matches one or more rules) and logs are closely related but different. When a log entry is written, information that has been collected will be added to the entry.

HTTP logs are influenced by configuration settings. There are 2 types of HTTP logging

You’re using a customized Suricata configuration file with Suricata 6.0.5

I’d suggest using the latest stable Suricata release – 6.0.10 and using the configuration file from that release and then assess things, including the amount of HTTP logs.

Suricata has several memory related configuration settings; some of these are dependent on the amount of compute and memory resources available and the expected traffic load. Generally, values are chosen to avoid drops, resource exhaustions and provide normal operation for the deployment model (ids vs ips).