How does TLS recognition work?

Description:
I have a TLS server that responds with “\x15\x03\x03\x00\x02\x02P” byte sequence to non-TLS requests. Googling these bytes yielded:
15 == TLS Alert Message Type
03 03 == TLS Protocol version 1.2
00 02 == TLS Frame length
02 == Alert level Fatal

I have a detection that uses Lua script that uses alproto_tc taken from SCFlowAppLayerProto() call to check if server responded with TLS. Lua script is invoked by rule that has flow:stateless. Mentioned byte sequence gets recognized as TLS coming from server.

Created my own server that responds in a similar fashion.

import socket
import sys

# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Bind the socket to the address given on the command line
server_name = 'localhost'
server_address = (server_name, 8443)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
sock.listen(1)


while True:
    print >>sys.stderr, 'waiting for a connection'
    connection, client_address = sock.accept()
    try:
        print >>sys.stderr, 'client connected:', client_address
        data = connection.recv(300)
        print >>sys.stderr, 'received "%s"' % data
        connection.sendall('\x15\x03\x03\x00\x02\x02P')
    finally:
        connection.close()

If we replace

        connection.sendall('\x15\x03\x03\x00\x02\x02P')

with

        connection.sendall('\x15\x03')
        connection.sendall('\x03\x00\x02\x02P')

I get an alert because alproto_tc isn’t set to tls. How does Suricata recognize TLS? Why if we send TLS bytes in one go we get alproto_tc set and in the second case we don’t? What we could change to avoid alert in the second case?

Details:
Suricata 5.0.2

Not really an answer but having a look at suricata/app-layer-ssl.c at master · OISF/suricata · GitHub should tell you something about how Suricata parses TLS.

I think capturing a pcap of the traffic between your client and python TLS server and looking at it in wireshark might give some insight into why splitting the sendall affects protocol detection.

1 Like