Real-World Data: AF_PACKET tpacket-v3 vs v2 (Memory Impact)

I’d like to share production data showing the memory impact of enabling tpacket-v3 when using AF_PACKET capture mode in Suricata.

Test Environment

  • Suricata 7.x

  • ~48,000 ET Open rules

  • AF_PACKET + use-mmap

  • Multiple distros, identical configs where possible

Measured Results (Production)

Server OS Before (v2) After (v3) Reduction
Alma #1 AlmaLinux 9.7 1,828 MB 454 MB 75%
Alma #2 AlmaLinux 9.7 ~900 MB 168 MB 81%
Alma #3 AlmaLinux 9.7 ~900 MB 167 MB 81%
Ubuntu 24.04 n/a 358 MB (already v3)

Ubuntu 24.04 already defaults to tpacket-v3; Alma/RHEL do not.


Root Cause

  • TPACKET_V2 uses fixed-size ring buffers sized for worst-case MTU

  • Memory scales as:
    ring-size Γ— block-size Γ— threads

  • With large rings (e.g. 100k), this alone can exceed 700–900 MB

TPACKET_V3 uses variable-sized blocks, allocating based on actual traffic, not worst-case assumptions.


The Clue in Logs

Before the fix:

[Warning] - AF_PACKET tpacket-v3 is recommended for non-inline operation

This warning disappears after enabling v3.


Fix (30 seconds)

af-packet:
  - interface: eth0
    use-mmap: yes
    tpacket-v3: yes

Or:

sed -i '/use-mmap: yes/a\    tpacket-v3: yes' /etc/suricata/suricata.yaml
systemctl restart suricata


Impact in Practice

Before:

  • Suricata consumed 45% of RAM on 4 GB VPS nodes

  • Swap activity during suricata-update

  • OOM risk under load

After:

  • Memory ~10% of system RAM

  • No swap pressure

  • Faster reloads

  • More predictable performance


Recommendation

Always explicitly set:

tpacket-v3: yes

…regardless of distro.
It’s a zero-cost optimization with massive memory savings on systems that still default to v2.

1 Like

Update / correction:
After further testing on multiple Ubuntu 24.04 systems, my earlier assumption that Ubuntu defaults to TPACKET_V3 was incomplete.
.
Details below.

Distro-Specific Defaults

Distribution Package Default
Ubuntu 24.04 apt v2
Debian 12 apt v2
AlmaLinux 9 EPEL v2
Rocky Linux 9 EPEL v2
Centos 10 EPEL v2 or v3 (varies)

Ring Buffer Math Example

With tpacket-v2:

100000 Γ— 2048 bytes Γ— 4 threads β‰ˆ 780 MB

With tpacket-v3:

Memory allocated dynamically based on packet sizes
Typical: ~100–200 MB

Performance Summary

Metric v2 v3 Result
Memory usage 900–1800 MB 160–450 MB 75–81% less
Ring efficiency Fixed Variable Better
Packet loss Higher Lower Improved
CPU overhead Slightly higher Slightly lower Marginal

Thanks for the info, this is great.

Can you tell me where you see v3 enabled by default for Suricata 7.0? On Ubuntu, I don’t see it enabled by default in Ubuntu’s repos, or our own PPA. Likewise for CentOS/EPEL 10. I’m just trying to figure out where/how it’s enabled by default.

Suricata 8.0 uses v3 by default though.

Thanks for the notice. After rechecking all my Ubuntu 24.04 lab systems, I can confirm why this was confusing.

The stock Ubuntu suricata.yaml has both use-mmap and tpacket-v3 commented out, so TPACKET_V3 is not explicitly enabled by the package (Jason’s statement is correct).

However, once use-mmap: yes is enabled without setting tpacket-v3, Suricata appears to auto-select TPACKET_V3 at runtime on Ubuntu 24.04 (based on stats and memory behavior). This explains why Ubuntu systems seemed to be β€œalready on V3”.

This behavior was not observed on RPM-based distributions, which is why it wasn’t noticeable before.

To avoid ambiguity across distros, explicitly setting:

use-mmap: yes
tpacket-v3: yes

is the only portable way to guarantee V3.

Can someone confirm whether Suricata 7.x intentionally auto-selects PACKET_VERSION=3 when tpacket-v3 is unset?

I checked, and use-mmap does not enable tpacket-v3 as its always enabled, for example, if you try to set it to false:

W: af-packet: br0: "use-mmap" option is obsolete: mmap is always enabled

This option has been removed in 8 completely, and just exists in 7 to provide a warning as needed that it can be removed.

Note: I’m not questioning your results. Just trying to track down if any packages do happen to enable it by default.

Jason, thanks for the patience and for pointing me in the right direction.

After rebuilding multiple Ubuntu 24.04 systems from scratch and validating with strace,

I can confirm my earlier conclusion was incorrect.

What went wrong in my initial test

I initially assumed:

presence of capture.afpacket.polls stats β‡’ TPACKET_V3

absence of poll stats β‡’ TPACKET_V2

This assumption was wrong.

Both TPACKET_V2 and TPACKET_V3 use poll(). The afpacket.polls counter only shows that Suricata is polling the AF_PACKET socket, not which PACKET_VERSION is in use. So stats alone are not sufficient to distinguish V2 vs V3.

What actually proves the PACKET version

The only definitive proof is the kernel call:

setsockopt(fd, SOL_PACKET, PACKET_VERSION, &version, sizeof(version))

Where:

1 = TPACKET_V2

2 = TPACKET_V3

Using strace -e setsockopt, I verified the following on a clean Ubuntu 24.04.3 LTS install:

Verified results (Suricata 7.0.3, Ubuntu universe package)

Default config (both commented):

#use-mmap: yes
#tpacket-v3: yes
β†’ setsockopt(PACKET_VERSION, [1]) // V2

Explicit tpacket-v3: false:

β†’ setsockopt(PACKET_VERSION, [1]) // V2

Explicit tpacket-v3: yes:

β†’ setsockopt(PACKET_VERSION, [2]) // V3

Conclusion (correction)

Ubuntu does not enable TPACKET_V3 by default

use-mmap being present or obsolete is not relevant

Poll-based stats are not an indicator of V3

Jasons statement was correct

The only portable and unambiguous way to enable V3 is:

tpacket-v3: yes

Jason thanks again for the guidance β€” this helped clarify both my misunderstanding and the correct way to validate PACKET_VERSION reliably.

Final clarification with full lab comparison (root cause identified)

I extended the testing to a full multi-distro lab comparison. With this data, the root cause of the memory differences is now clear.

Complete lab server comparison

● Proof - Before vs After                                                                                                                                                                                     

● β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                                                                                                  
  β”‚ Server β”‚    Distro     β”‚ Suricata β”‚   CPU   β”‚ RAM  β”‚ ring-size (before) β”‚ ring-size (after) β”‚ Memory Change β”‚
  β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€                                                                                                  
  β”‚ lab    β”‚ Debian 12     β”‚ 6.0.10   β”‚ 2 cores β”‚ 3.7G β”‚ 300,000            β”‚ 20,000            β”‚ 1,600β†’652 MB  β”‚                                                                                                  
  β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€                                                                                                  
  β”‚ lab1   β”‚ AlmaLinux 9.7 β”‚ 7.0.13   β”‚ 2 cores β”‚ 3.5G β”‚ 100,000            β”‚ 20,000            β”‚ 804β†’533 MB    β”‚                                                                                                  
  β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€                                                                                                  
  β”‚ lab2   β”‚ Ubuntu 24.04  β”‚ 7.0.3    β”‚ 2 cores β”‚ 3.7G β”‚ 20,000             β”‚ 20,000            β”‚ 45β†’102 MB     β”‚                                                                                                  
  β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
  β”‚ lab3   β”‚ AlmaLinux 9.7 β”‚ 7.0.13   β”‚ 2 cores β”‚ 3.5G β”‚ 100,000            β”‚ 20,000            β”‚ 482β†’531 MB    β”‚
  β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
  β”‚ lab4   β”‚ AlmaLinux 9.7 β”‚ 7.0.13   β”‚ 2 cores β”‚ 3.5G β”‚ 100,000            β”‚ 20,000            β”‚ 806β†’531 MB    β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
 
  Current labs are 2-core/~3.5GB systems
                                                                                        
  *lab2 increased because mmap was enabled (better performance)                                                

Key findings

Memory correlates directly with ring-size, not with TPACKET version

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚      ring-size      β”‚ Memory  β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ 300000              β”‚ ~1.6 GB β”‚
β”‚ 100000              β”‚ ~800 MB β”‚
β”‚ default (commented) β”‚ ~44 MB  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

What this proves

  1. Ubuntu 24.04 (lab2) uses very little memory because:

    • use-mmap is commented

    • tpacket-v3 is explicitly disabled

    • default ring-size is small

  2. AlmaLinux systems (lab1/lab3/lab4) consume ~800 MB because:

    • use-mmap: yes

    • tpacket-v3: yes

    • ring-size: 100000

  3. Debian system (lab) consumes ~1.6 GB because:

    • ring-size: 300000

Why my initial conclusion was wrong

In my original testing, I incorrectly attributed the lower memory usage on Ubuntu to TPACKET_V3.

After rebuilding systems from scratch and validating with strace, it is clear that:

  • Ubuntu 24.04 Suricata 7.0.3 defaults to TPACKET_V2

  • The memory difference was not caused by V2 vs V3

  • The real drivers were ring-size and mmap configuration

Final conclusions

  • Ubuntu 24.04 Suricata 7.0.3 defaults to TPACKET_V2 (verified via setsockopt(PACKET_VERSION, [1]))

  • V2 vs V3 has minimal impact on memory usage

  • Ring-size dominates AF_PACKET memory usage

  • To enable V3 explicitly and portably:

    tpacket-v3: yes
    
    

Thanks for the discussion β€” the extended lab testing clarified both my earlier mistake and the real source of the memory differences.