TCP/IP Header Length Calculation Rules using byte_math

Rules
,
1

Please include the following information with your help request:

  • Suricata version:8.0.1
  • Operating system and/or Linux distribution:ubuntu 25.04
  • How you installed Suricata:from source

Hi, I‘m using suricata to identify os fingerprints. Here are some rules which intended to calculate the total length of tcp header and ip header with byte_extract, byte_math and byte_test:

alert tcp any any -> any any (msg:"Windows 8";tcp.flags:S,CE;window:65535;tcp.hdr;content:"|02 04 05 b4 01 03 03 08 01 01 04 02|";endswith;byte_extract:1,12,tcphdrlen;byte_math:bytes tcphdrlen,offset 0,oper >>,rvalue 4,result tcphdrlen;ipv4.hdr;byte_math:bytes 1,offset 0,oper +,rvalue tcphdrlen,result totallen,bitmask 15;byte_test:totallen,<,13,0;ttl:128;sid:1104778; rev:1;)
#this rule is meant to be triggered, which indicates ip header length + tcp header length = 13*4 = 52
alert tcp any any -> any any (msg:"Windows 8";tcp.flags:S,CE;window:65535;tcp.hdr;content:"|02 04 05 b4 01 03 03 08 01 01 04 02|";endswith;byte_extract:1,12,tcphdrlen;byte_math:bytes tcphdrlen,offset 0,oper >>,rvalue 4,result tcphdrlen;ipv4.hdr;byte_math:bytes 1,offset 0,oper +,rvalue tcphdrlen,result totallen,bitmask 15;byte_test:totallen,=,13,0;ttl:128;sid:1104779; rev:1;)
alert tcp any any -> any any (msg:"Windows 8";tcp.flags:S,CE;window:65535;tcp.hdr;content:"|02 04 05 b4 01 03 03 08 01 01 04 02|";endswith;byte_extract:1,12,tcphdrlen;byte_math:bytes tcphdrlen,offset 0,oper >>,rvalue 4,result tcphdrlen;ipv4.hdr;byte_math:bytes 1,offset 0,oper +,rvalue tcphdrlen,result totallen,bitmask 15;byte_test:totallen,>,13,0;ttl:128;sid:1104780; rev:1;)
#alert tcp any any -> any any (msg:"Windows 8";tcp.flags:S,CE;window:65535;tcp.hdr;content:"|02 04 05 b4 01 03 03 08 01 01 04 02|";endswith;ttl:128;sid:1104781; rev:1;)

To my surprise none of the rules above are fired, since theoretically they cover all possible scenarios. Only sid 1104781 will be fired if I uncomment it, which confuses me.:face_with_raised_eyebrow:
Your advice would be of great help to me. Thanks in advance! :grinning_face:

2

update:The rule will be triggered if I use byte_test directly, while extracting bytes to a variable and then applying byte_test to it does not work. :cry:

#works
alert tcp any any -> any any (msg:"Windows 8";tcp.flags:S,CE;window:65535;tcp.hdr;content:"|02 04 05 b4 01 03 03 08 01 01 04 02|";offset:20;byte_test:1,=,0x80,0;ttl:128,12;sid:1104780; rev:1;)
#does not work
alert tcp any any -> any any (msg:"Windows 8";tcp.flags:S,CE;window:65535;tcp.hdr;content:"|02 04 05 b4 01 03 03 08 01 01 04 02|";offset:20;byte_extract:1,12,tcphdrlen;byte_test:tcphdrlen,=,0x80,0;ttl:128;sid:1104781; rev:1;)