Real-World Data: AF_PACKET tpacket-v3 vs v2 (Memory Impact)

7

Final clarification with full lab comparison (root cause identified)

I extended the testing to a full multi-distro lab comparison. With this data, the root cause of the memory differences is now clear.

Complete lab server comparison

● Proof - Before vs After                                                                                                                                                                                     

● ┌────────┬───────────────┬──────────┬─────────┬──────┬────────────────────┬───────────────────┬───────────────┐                                                                                                  
  │ Server │    Distro     │ Suricata │   CPU   │ RAM  │ ring-size (before) │ ring-size (after) │ Memory Change │
  ├────────┼───────────────┼──────────┼─────────┼──────┼────────────────────┼───────────────────┼───────────────┤                                                                                                  
  │ lab    │ Debian 12     │ 6.0.10   │ 2 cores │ 3.7G │ 300,000            │ 20,000            │ 1,600→652 MB  │                                                                                                  
  ├────────┼───────────────┼──────────┼─────────┼──────┼────────────────────┼───────────────────┼───────────────┤                                                                                                  
  │ lab1   │ AlmaLinux 9.7 │ 7.0.13   │ 2 cores │ 3.5G │ 100,000            │ 20,000            │ 804→533 MB    │                                                                                                  
  ├────────┼───────────────┼──────────┼─────────┼──────┼────────────────────┼───────────────────┼───────────────┤                                                                                                  
  │ lab2   │ Ubuntu 24.04  │ 7.0.3    │ 2 cores │ 3.7G │ 20,000             │ 20,000            │ 45→102 MB     │                                                                                                  
  ├────────┼───────────────┼──────────┼─────────┼──────┼────────────────────┼───────────────────┼───────────────┤
  │ lab3   │ AlmaLinux 9.7 │ 7.0.13   │ 2 cores │ 3.5G │ 100,000            │ 20,000            │ 482→531 MB    │
  ├────────┼───────────────┼──────────┼─────────┼──────┼────────────────────┼───────────────────┼───────────────┤
  │ lab4   │ AlmaLinux 9.7 │ 7.0.13   │ 2 cores │ 3.5G │ 100,000            │ 20,000            │ 806→531 MB    │
  └────────┴───────────────┴──────────┴─────────┴──────┴────────────────────┴───────────────────┴───────────────┘
 
  Current labs are 2-core/~3.5GB systems
                                                                                        
  *lab2 increased because mmap was enabled (better performance)

Key findings

Memory correlates directly with ring-size, not with TPACKET version

┌─────────────────────┬─────────┐
│      ring-size      │ Memory  │
├─────────────────────┼─────────┤
│ 300000              │ ~1.6 GB │
│ 100000              │ ~800 MB │
│ default (commented) │ ~44 MB  │
└─────────────────────┴─────────┘

What this proves

  1. Ubuntu 24.04 (lab2) uses very little memory because:

    • use-mmap is commented

    • tpacket-v3 is explicitly disabled

    • default ring-size is small

  2. AlmaLinux systems (lab1/lab3/lab4) consume ~800 MB because:

    • use-mmap: yes

    • tpacket-v3: yes

    • ring-size: 100000

  3. Debian system (lab) consumes ~1.6 GB because:

    • ring-size: 300000

Why my initial conclusion was wrong

In my original testing, I incorrectly attributed the lower memory usage on Ubuntu to TPACKET_V3.

After rebuilding systems from scratch and validating with strace, it is clear that:

  • Ubuntu 24.04 Suricata 7.0.3 defaults to TPACKET_V2

  • The memory difference was not caused by V2 vs V3

  • The real drivers were ring-size and mmap configuration

Final conclusions

  • Ubuntu 24.04 Suricata 7.0.3 defaults to TPACKET_V2 (verified via setsockopt(PACKET_VERSION, [1]))

  • V2 vs V3 has minimal impact on memory usage

  • Ring-size dominates AF_PACKET memory usage

  • To enable V3 explicitly and portably:

    tpacket-v3: yes

Thanks for the discussion — the extended lab testing clarified both my earlier mistake and the real source of the memory differences.