Help with HTTP events

Help
1

Hi,

I noticed that my suricata setup might not be recording http events correctly. When i make a tcpdump against the mirrored traffic i can see some HTTP requests that are not being replicated as “HTTP events” in Suricata logs.

14:28:18.362522 IP 216.244.66.236.44892 > x.x.x.x.80: Flags [P.], seq 3841640318:3841640538, ack 4181010606, win 229, options [nop,nop,TS val 2076978036 ecr 1151726767], length 220: HTTP: GET /robots.txt HTTP/1.1
E....@@.(.i;..B..u...\.P...~.5(......0.....
{.+tD...GET /robots.txt HTTP/1.1
Host: xxxx
Accept: */*
User-agent: Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)
Accept-Charset: utf-8;q=0.7,iso-8859-1;q=0.2,*;q=0.1


14:28:18.362533 IP 216.244.66.236.44892 > x.x.x.x.80: Flags [P.], seq 0:220, ack 1, win 229, options [nop,nop,TS val 2076978036 ecr 1151726767], length 220: HTTP: GET /robots.txt HTTP/1.1
E....@@.(.i;..B..u...\.P...~.5(......0.....
{.+tD...GET /robots.txt HTTP/1.1
Host: xxxx
Accept: */*
User-agent: Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)
Accept-Charset: utf-8;q=0.7,iso-8859-1;q=0.2,*;q=0.1

For this example, i can see the traffic in Suricata but it is classified as “netflow”. I have http events enabled in the suricata logs

2

Hi,

please provide the Suricata version you are using, how you run it and the config file.
Did you run the capture as a pcap against Suricata as well and still not seeing the http events?

3

I am using Suricata version 6.0.10, running against some mirrored traffic with af-packet. I can´t send the config file because it has sensible information.

I didi not run the pcap against Suricata

4

You can remove the sensible information before uploading it, without more details it’s hard to debug.

5

Here it is

suricata-config.zip (22.3 KB)

6

Can you post the stats.log when this traffic is seen before and after?

Also are there event_type with flow or others seen that you configured for eve-events.json?

You can also capture the traffic as a pcap and run it with -r against a local Suricata to verify the outputs just for this traffic.

7

It is hard to specifically debug each portion of the traffic due to the size of the network and its intensity. But here is another example:

When analysing traffic with tcpdump:

11:09:32.757622 IP x.x.x.x.51483 > 67.26.137.254.80: Flags [P.], seq 1065:1418, ack 1574263, win 5894, length 353: HTTP: GET /d/msdownload/update/software/secu/2022/12/windows10.0-kb5022502-x64-ndp48_9a8ef9caf0e99916c130fab6e4c9947f6dd1c294.cab HTTP/1.1
E.....@...      .
...C......Pv.J.....P.......GET /d/msdownload/update/software/secu/2022/12/windows10.0-kb5022502-x64-ndp48_9a8ef9caf0e99916c130fab6e4c9947f6dd1c294.cab HTTP/1.1
Connection: Keep-Alive
Accept: */*
Range: bytes=2097152-2621439
User-Agent: Microsoft-Delivery-Optimization/10.0
MS-CV: VAsedNv08Uy04ZRp.1.1.8.2.7.1.1.22
Content-Length: 0
Host: 3.au.download.windowsupdate.com

But in Kibana it only show me the “netflow” event:

Screenshot 2023-02-28 111604
Screenshot 2023-02-28 1116041316×59 18.9 KB

8

As I said please also post the stats.log to get an idea if there are actual other events besides netflow. Ideally two stats.log to see how the stats progress between two time slots.

9

stats.zip (5.6 KB)

10

You can see that there should be events in your logfile:

app_layer.flow.http                           | Total                     | 2322

and

app_layer.flow.http                           | Total                     | 2527

So what does the result of cat eve-events.json | jq -c 'select(.event_type=="http")' look like?

Also add flow to the types of that file to look into those as well.

11

I can see a lot of HTTP events with that command.

What it seems is that some events that should be logged as HTTP events are not being classified as that but instead only “netflow events”. It is hard to analyse with greater detail, but the example above should have appeared as an HTTP event but that did not happened

12

There is no “instead” a traffic can produce multiple events with just one flow. So you could see an alert, flow and http event from the same traffic.

To further analyse, record this traffic with tcpdump or other tools and produce a pcap file and run this against suricata to see if the event shows up in that case.