Suricata parse http error, lots of /libhtp::request_uri_not_seen and http request info lose

Help
1

ENV:
CPU: E5-2630
Network pci: 82599ES 10-Gigabit
Memory: 128g
Suricata: pf_ring(or af-xdp) + 32 workers + log to kafka + listen to interface 1 ,another machine replays pcap to interface 1(Connect using an Ethernet cable）
version: 7.0.0.rc2 or 6.0.11

Problem:
There’s a strange bug : once i replay some http pcap using tcpdump, suricata would parse and generate some /libhtp::request_uri_not_seen log random. I patched some log and found that response is coming earlier than request.

Then i tried single worker mode and enable stream inline, doesn’t help at all. And i print something result that many request data are empty or garbled code，suricata failed to do TCPProtoDetect(), but response works fine.

It occur randomly and stats shows that there is no packet lose.

Reproduce：
1.compile args:
./configure --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/ --enable-lua --enable-pfring --enable-old-barnyard2 --enable-unix-socket --enable-geoip --enable-http2decompression --build=x86_64-linux-gnu CFLAGS=“-D_GUN_SOURCE”

2.suricata args:
–pfring-cluster-id=99 --pfring-cluster-type=cluster_flow --pfring-int=xxx -c
/etc/suricata/suricata.yaml

im not sure if suricata.yaml is necessary.

3.replay any http pcap, or a dir contains serval http pcaps by tcpreplay:
tcpreplay -K -i xxx -M 10000 -l 100 --stats=2 /pcapdir

addition:
I also dump all packets by tcpdump while suricata is running. The wrong traffic(url_not_seen or request info lose) in suricata is OK in pcap that dumped by tcpdump.

Here is a example:
URL_NOT_SEEN and no http request data:

image
image977×417 24.5 KB

Pcap captured by tcpdump shows that request exists:

image
image1432×211 64.8 KB

2

This is known and discussed here: Bug #5165: http: request not logged when response comes before request - Suricata - Open Information Security Foundation

1 Like
3

Thanks for your reply @Andreas_Herz ,
Sorry for that i’ve read about the issue #5165 before i post this one. but this one is really different: I print the data which goes into AppLayerHandleTCPData(),
shows that not only request comes before response but also no data or wrong data coming with request(leads to TCPProtoDetect failed) . While response data was fine all.

request : >>>>>>>>>>>>>>
bad (ALPROTO_UNKNOWN) bad (ALPROTO_UNKNOWN)

response:<<<<<<<<<<<<<<
good(ALPROTO_HTTP1) good(ALPROTO_HTTP1)

4

Can you add those details to a new redmine ticket or to this one?