Rethinking the SSLproxy/Suricata integration: Divert Mode is a dead end for H2/H3

Hi All,

I’ve just published a new piece that I think is relevant to the current work on the SSLproxy support PR (#13960): I Tried to Add HTTP/2 to SSLproxy. Here is Why I Stopped. (We Need ICAP.).

To be blunt: The current Divert Mode implementation in SSLproxy is a dead end for H2 and H3. While it works for H1, it will never enable Suricata to inspect H2/H3 traffic from SSLproxy without a massive, fragile protocol-translation layer that doesn’t scale.

If you think Divert mode was a mistake, think again—even Blue Coat (ProxySG) uses a similar architecture. But they hit the same wall with H2/H3, which is exactly why their ecosystem eventually offloaded the ‘heavy lifting’ to ICAP-based Content Analysis engines.

I’m elevating the urgency of my ICAP proposal from the PreSuriCon webinar. And I suggest another solution to pass decrypted packets to Suricata, instead of my previous suggestion (a new DAQ module with ICAP server support + packet emulation), which had an unnecessary emulation overhead just to satisfy Suricata’s reassembly needs, as I hinted at the webinar.

The new proposal: Once I implement ICAP client support in SSLproxy, we should develop a standalone ICAP server that integrates Suricata as a library.

This avoids the “packet emulation” mess entirely. Instead of pretending to be a network interface to feed Suricata, we use Suricata’s inspection power directly on the stream buffers provided by the ICAP lifecycle.

I realize that Library Mode isn’t fully mature yet (per recent forum discussions like this), but I believe it is the best path forward if we want to support H2/H3 and finally unblock UDP 443.

In summary, I think ICAP and Library Mode seem to be the missing links for modern protocol inspection. What do you think?

Best,
Soner

For those interested, I have just pushed the h2-divert branch to the GitHub repo of SSLproxy, which contains the PoC code I mention in my LinkedIn article above.

2 Likes

I think the translation to HTTP/1 is an unfortunate design goal, that is not needed for a tool like Suricata to get a lot of value out of H2 support. Suricata supports parsing and tracking HTTP/2 fully (and at high speeds too), so as long as we get the decrypted stream we’re pretty happy. Guess we’d only need some way to get the magic header for the original tuple.

I understand sslproxy has some other goals, esp around the support of the filtering “listening programs”. I think this is why prefixing the http streams with the magic header (as opposed to an inserted http header) was rejected. So I guess there is just a mismatch in use cases.

Wrt the ICAP idea: I would certainly be interested in this. Would this be limited to the HTTP family or also support other TLS based protocols like POP3s and IMAP. I’m currently seeing a lot of value in having decrypt support for generic TLS for the Suricata use case.

Hi Victor,

Thanks for the feedback. I’d like to clarify a few points from the perspective of an inline IPS deployment:

  1. The h2c vs. h2 Reality: While Suricata can handle h2c, in the wild, virtually all H2 is encrypted (h2). Since IDPS engines cannot inspect the encrypted cipher, we must use a proxy.

  2. Loss of Network Context: Once SSLproxy (or for that matter Blue Coat) decrypts H2, we lose the ability to imitate low-level TCP/TLS anomalies in the diverted path. We are left only with the payload. As I argued in my webinar, content inspection becomes the primary goal once the encryption is stripped.

  3. Inline IPS vs. IDS: My focus is on active inline IPS mode. For simple IDS/logging in split mode, mirroring decrypted H2 from SSLproxy is fine (and it seems like a basic ALPN support to allow for H2 upgrade may be sufficient in Split mode). But for real-time blocking, we need a synchronous path.

  4. The Divert Mode “Dead End” for IPS: Currently, the only way for Suricata to be an active inline IPS for SSLproxy’s diverted H2 traffic is a chain like: Client → SSLproxy → [Divert Socket] → Suricata → Listening Program (lp) → SSLproxy → Server. This is highly inefficient. Using a listening program just to “catch” packets so Suricata can inspect them creates the same overhead/packet-emulation issues I raised in my previous DAQ proposal.

  5. ICAP + Library Mode: This is why I believe the ICAP path is superior for the IPS use case. It allows SSLproxy to hand off the decrypted stream directly to a Suricata-powered service without the “kernel-to-user-to-kernel” bounce of divert sockets.

Regarding other protocols: While ICAP was born for HTTP, it is essentially a “Content Adaptation” framework. I absolutely intend to add ICAP support to POP3s and IMAPs, and in fact to all protocols for content inspection. If we build a Suricata ICAP server (using Library-mode), we could encapsulate any decrypted stream into an ICAP-like PDU (extended ICAP) for inspection.

I’m glad to hear you’re interested in the ICAP/Library mode path—I think it’s the most sustainable way to handle H2 and H3 (QUIC) (H3 in bold, because this solution targets H3 and beyond as well).

[Updated with further improvements and explanations]

The ssl-alpn-h2 branch at the GitHub repo of SSLproxy implements basic ALPN support for HTTP/2 upgrade.

  • The changes support both Divert and Split modes, so Suricata can run as active inline IPS and just IDS, respectively.
  • Both https and ssl proxyspecs can be used. The https proxyspec can filter and verify decrypted H1 traffic, but directly passes decrypted H2 traffic.
  • The verify_h2_gateway.sh script uses divert mode to demonstrate how Suricata IPS can inspect h2 traffic inline with lp. It can be edited to use split mode just by removing the up:9080 specification.
  • The script records decrypted h2 traffic into h2.pcap, which can be fed into Suricata. Wireshark does not recognize the TCP flow as h2, unless forced to with the Decode As option.
  • The mirroring option should work as well, but not tested.

The following is copy-pasted directly from the Follow HTTP2 Stream window of Wireshark, showing decrypted h2 contents recorded into h2.pcap in split mode:

:method: GET
:scheme: https
:authority: 127.0.0.1:8080
:path: /
user-agent: curl/8.5.0
accept: */*

:status: 404
server: nghttpd nghttp2/1.59.0
date: Sat, 14 Feb 2026 15:18:16 GMT
content-type: text/html; charset=UTF-8
content-length: 148

<html><head><title>404 Not Found</title></head><body><h1>404 Not Found</h1><hr><address>nghttpd nghttp2/1.59.0 at port 10443</address></body></html>

This is great! It works great with my sslproxy branch in Suricata (needed one additional fix in my branch)

{                                
  "timestamp": "2026-02-16T11:53:14.709789+0100",
  "flow_id": 894124366595096,               
  "event_type": "fileinfo",
  "src_ip": "216.66.8.75",                       
  "src_port": 443,                             
  "dest_ip": "192.168.0.5",                                                                                                                                                                                        
  "dest_port": 38070,
  "proto": "TCP",
  "ip_v": 4,
  "pkt_src": "wire/pcap",
  "http": {
    "version": "2",
    "request_headers": [
      ...
      {
        "name": "x-requested-with",            
        "value": "XMLHttpRequest"                                                      
      },             
      {          
        "name": "user-agent",
        "value": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36"
      },
      {
        "name": "accept",
        "value": "text/plain, */*; q=0.01"
      },
      {                                                                                                                                                                                                            
        "name": "referer",
        "value": "https://forum.suricata.io/t/rethinking-the-sslproxy-suricata-integration-divert-mode-is-a-dead-end-for-h2-h3/6195/5"
      },
      ...
    ]
    "http_method": "POST",
    "http_user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36",
    "url": "/message-bus/5bd5f0ed01734fc1a292bc3172930e21/poll",
    "status": 200,
    "length": 909,
    "http2": {
      "stream_id": 25,
      "request": {
        "priority": 219
      }
    }
  },
  "app_proto": "http2",
  "fileinfo": {
    "gaps": false,
    "state": "CLOSED",
    "stored": false,
    "size": 709,
    "tx_id": 21
  },
  "host": "c2758-ips"
}

Thanks Victor for testing. I’m glad it works with Suricata too. It would be great if you could try active inline IPS mode too, if you can.

I am working on how to match protocols selected by ALPN negotiation on both sides of SSLproxy (client ↔ SSLproxy and SSLproxy ↔ server), and what to do if they don’t match. It’s still PoC level code, hence currently does not support auto protocol selection between h2 and http1.1, even if so advertised during ALPN negotiation.

My tests are with inline mode. Sslproxy uses the lp test program and Suricata sits between them using NFQ. This is my concept of active inline IPS mode. If you mean something different by it, please let me know.

I’m seeing some connections not working, but I can’t say yet if that is a SSLproxy issue or a Suricata issue. Will try to dig into that.

That’s great. I just couldn’t tell from the json output if you were using divert mode, that’s all.

Not sure how to debug this, but I’m seeing an issue with the branch. This is without Suricata.

With the branch I see apt update fail for a specific source:

Err:7 https://esm.ubuntu.com/apps/ubuntu noble-apps-security InRelease
  Connection failed [IP: 185.125.190.23 443]
Err:8 https://esm.ubuntu.com/apps/ubuntu noble-apps-updates InRelease
  Connection failed [IP: 91.189.91.46 443]
Err:9 https://esm.ubuntu.com/infra/ubuntu noble-infra-security InRelease
  Connection failed [IP: 185.125.190.75 443]
Err:10 https://esm.ubuntu.com/infra/ubuntu noble-infra-updates InRelease
  Connection failed [IP: 91.189.91.47 443]

If I recompile SSLproxy to use the master branch, it works. Any guidance on how to debug this?

I’d suggest enabling the DEBUG_PROXY switch in Mk/main.mk, which should be already enabled in the ssl-alpn-h2 branch, and then starting sslproxy with the -D4 option, as in the verify_h2_gateway.sh script. sslproxy produces very verbose debug logs with -D4.

I’m just guessing that the ubuntu update servers do not support h2, but sslproxy cannot enable http1.1. I think that the ssl-alpn-h2 branch works properly with h2 only (well, properly for a PoC code).

I am working exactly on how to manage this alpn negotiation on both sides (my current target is to update the ClientHello parser to return the ALPN protocols too).

Is this helpful?

One thing that I noticed is: protossl_check_h2_enabled: H2 negotiated via ALPN but the traffic show is HTTP/1


[FINEST] [1.9 fd=47 cfd=0] pxy_conn_connect: ENTER
Connecting to [91.189.91.47]:443
[FINEST] [1.9 fd=47 cfd=0] protossl_conn_connect: ENTER
Attempt reuse dst SSL session
[FINEST] [1.9 fd=47 cfd=0] protossl_bufferevent_setup: ENTER, fd=-1
[FINEST] [1.9 fd=47 cfd=0] protossl_bufferevent_setup: bufferevent_openssl_set_allow_dirty_shutdown, fd=-1
[FINEST] [1.9 fd=47 cfd=0] protossl_bev_eventcb_connected_srvdst: ENTER
[FINEST] [1.9 fd=47 cfd=0] protossl_check_h2_enabled: ENTER
[FINE] [1.9 fd=47 cfd=0] protossl_check_h2_enabled: H2 negotiated via ALPN
===> Original server certificate:
Subject DN: /CN=esm.ubuntu.com
Common Names: esm.ubuntu.com/esm.ubuntu.com
Fingerprint: 62:D7:CC:6B:14:D9:AE:20:24:7C0C:97:EB:1F:48:7B:66:C4:3D:90
Certificate cache: HIT
===> Forged server certificate:
Subject DN: /CN=esm.ubuntu.com
Common Names: esm.ubuntu.com/esm.ubuntu.com
Fingerprint: 3C:C5:8B:18:8F:E9:B8:F0:22:9A3D:15:4F:9E:64:B5:D2:8A:BD:BB
[FINEST] [1.9 fd=47 cfd=0] prototcp_bufferevent_setup: ENTER, fd=-1
[FINEST] [1.9 fd=47 cfd=0] protossl_bev_eventcb_connected_dst: ENTER
[FINEST] [1.9 fd=47 cfd=0] protossl_bufferevent_setup: ENTER, fd=47
[FINEST] [1.9 fd=47 cfd=0] protossl_bufferevent_setup: bufferevent_openssl_set_allow_dirty_shutdown, fd=47
[FINER] [1.9 fd=47 cfd=51] pxy_setup_child_listener: Finished setting up child listener, child_fd=51
[FINER] [1.9 fd=47 cfd=51] pxy_set_sslproxy_header: sslproxy_header= SSLproxy: [127.0.0.1]:37381,[192.168.0.5]:41240,[91.189.91.47]:443,s
[FINER] [1.9 fd=47 cfd=51] protossl_enable_src: Enabling src
SSL connected to [91.189.91.47]:443 TLSv1.3 TLS_AES_256_GCM_SHA384
CLIENT_RANDOM XXX 000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Certificate cache: KEEP (SNI match or target mode)
[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_writecb_dst: ENTER
[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_eventcb_connected_src: ENTER
CONN: ssl 192.168.0.5 41240 91.189.91.47 443 sni:esm.ubuntu.com names:esm.ubuntu.com/esm.ubuntu.com sproto:TLSv1.3:TLS_AES_256_GCM_SHA384 dproto:TLSv1.3:TLS_AES_256_GCM_SHA384 origcrt:AAA usedcrt:BBB user:-
SSL connected to [91.189.91.47]:443 TLSv1.3 TLS_AES_256_GCM_SHA384
CLIENT_RANDOM XXX YYY
[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_readcb_src: ENTER, size=611
[FINER] [1.9 fd=47 cfd=51] pxy_try_prepend_sslproxy_header: ENTER
[FINEST] [1.9 fd=47 cfd=51] pxy_try_prepend_sslproxy_header: ORIG packet, size=611:
GET /apps/ubuntu/dists/noble-apps-security/InRelease HTTP/1.1^M
Host: esm.ubuntu.com^M
Cache-Control: max-age=0^M
Accept: text/*^M
If-Modified-Since: Mon, 16 Feb 2026 16:14:10 GMT^M
Authorization: Basic PRIVATE^M
User-Agent: Debian APT-HTTP/1.3 (2.8.3)^M
^M

[FINER] [1.9 fd=47 cfd=51] pxy_insert_sslproxy_header: ENTER
[FINEST] [1.9 fd=47 cfd=51] pxy_try_prepend_sslproxy_header: NEW packet, size=681:
SSLproxy: [127.0.0.1]:37381,[192.168.0.5]:41240,[91.189.91.47]:443,s^M
GET /apps/ubuntu/dists/noble-apps-security/InRelease HTTP/1.1^M
Host: esm.ubuntu.com^M
Cache-Control: max-age=0^M
Accept: text/*^M
If-Modified-Since: Mon, 16 Feb 2026 16:14:10 GMT^M
Authorization: Basic PRIVATE^M
User-Agent: Debian APT-HTTP/1.3 (2.8.3)^M
^M

[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_writecb_dst: ENTER
[FINEST] [1.9 fd=47 cfd=51] pxy_listener_acceptcb_child: ENTER, fd=52, ctx->child_fd=51
[FINEST] [1.9 fd=47 cfd=51] pxy_listener_acceptcb_child: peer addr=[127.0.0.1]:52136, fd=52
[FINER] [1.9 fd=47 cfd=51] check_fd_usage: descriptor_table_size=1024, dtablecount=0, reserve=10
[FINEST] [1.9 fd=47 cfd=51] pxy_conn_ctx_new_child: ENTER, fd=52
[FINEST] [1.9 fd=47 cfd=51] pxy_conn_attach_child: Adding child conn
[FINEST] [1.9 fd=47 cfd=51] prototcp_bufferevent_setup_child: ENTER, fd=52
[FINEST] [1.9 fd=47 cfd=51] protossl_connect_child: ENTER
[FINEST] [1.9 fd=47 cfd=51] prototcp_disable_srvdst: ENTER
[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_eventcb_connected_dst_child: ENTER
Child connecting to [91.189.91.47]:443
[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_readcb_src_child: ENTER, size=681
[FINER] [1.9 fd=47 cfd=51] pxy_try_remove_sslproxy_header: REMOVE
[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_readcb_src_child: NEW packet, size=611:
GET /apps/ubuntu/dists/noble-apps-security/InRelease HTTP/1.1^M
Host: esm.ubuntu.com^M
Cache-Control: max-age=0^M
Accept: text/*^M
If-Modified-Since: Mon, 16 Feb 2026 16:14:10 GMT^M
Authorization: Basic PRIVATE^M
User-Agent: Debian APT-HTTP/1.3 (2.8.3)^M
^M

[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_writecb_src_child: ENTER
[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_writecb_dst_child: ENTER
[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_eventcb_eof_dst_child: ENTER
evbuffer size at EOF: i:0 o:0 i:0 o:0
[FINEST] [1.9 fd=47 cfd=51] prototcp_bev_eventcb_eof_dst_child: !src.closed, terminate conn
[FINEST] [1.9 fd=47 cfd=51] pxy_try_close_conn_end: outbuflen == 0, terminate conn
[FINER] [1.9 fd=47 cfd=51] prototcp_bufferevent_free_and_close_fd: in=0, out=0, fd=52
[FINER] [1.9 fd=47 cfd=51] protossl_bufferevent_free_and_close_fd: in=0, out=0, fd=49
SSL_free() in state 00000001 = 0001 = SSLOK (SSL negotiation finished successfully) [connect socket]
[FINER] [1.9 fd=47 cfd=51] protossl_bufferevent_free_and_close_fd: fd=49, SSL_free() in state 00000001 = 0001 = SSLOK (SSL negotiation finished successfully) [connect socket]
[FINEST] [1.9 fd=47 cfd=51] pxy_try_disconnect_child: other->closed, terminate conn
Child SSL disconnected to [91.189.91.47]:443, child fd=52, fd=47
Child SSL disconnected from [192.168.0.5]:41240, child fd=52, fd=47

Can you test the latest commit on the ssl-alpn-h2 branch please? I think these changes fix the issues you are seeing.

I have implemented proper ALPN negotiation on both sides, which tries to match client’s protocols with server’s, if there is no overlap, the connection is closed by server. See the commit message for details.

It seems to work better, but I’m still seeing quite a few timeouts/stalls in various places. But should run master for a bit as well to see if it is related to the h2 branch or something else.

Seeing quite a few of these

Client-side BEV_EVENT_ERROR
Error from bufferevent: 0:- 167773206:1046:sslv3 alert certificate unknown:20:SSL routines:0:-
Additional SSL error: 1:1:-:0:-:0:-
Client-side BEV_EVENT_ERROR
Error from bufferevent: 0:- 167773206:1046:sslv3 alert certificate unknown:20:SSL routines:0:-
Additional SSL error: 1:1:-:0:-:0:-
Client-side BEV_EVENT_ERROR
Error from bufferevent: 0:- 167773206:1046:sslv3 alert certificate unknown:20:SSL routines:0:-
Additional SSL error: 1:1:-:0:-:0:-
Client-side BEV_EVENT_ERROR
Error from bufferevent: 0:- 167773206:1046:sslv3 alert certificate unknown:20:SSL routines:0:-
Additional SSL error: 1:1:-:0:-:0:-

Ah, that’s good news, because the issues in those logs are related with certificate verification, not with the ssl-alpn-h2 branch (which I have updated recently again btw).

Those SSL errors mean that you have not installed the CA certificate you use with SSLproxy into all of the trusted cert stores on the system or the web browsers or perhaps other software. Note that they may all be using different cert stores, in fact Firefox and Chrome/Brave do use different cert stores. If those errors are due to apt, that’s probably the system cert store.

In short, you need to install the CA cert of SSLproxy into all such cert stores.