Building a dedicated Suricata box for inline/transparent bridge IPS function on Linux Mint

Help
,
1

Hi all,

I’m both a linux and suricata newbie, so bear with me - my entire computer career has been dos, windows and some mac - I’m not a coder or anything but have a pretty well-built out homelab (so I’m trying to learn) but the vast majority of my stuff lives on windows in various forms, minus my HAOS server and Linux Mint laptop.

I’m trying to install and configure a functioning version of Suricata 8 on Linux Mint with a dedicated mini pc (minisforum nab9s/16gb/1tb + 2 2.5g intel 226-v) and a usb dongle for a 1gb connection for management.

I’ve got Mint installed and operating and can see all 3 interfaces, but have no idea how to build this from here, and have found so many conflicting or outdated directions online that I decided to come here.

As a non-coder / non-linux native, I have to say the online manual is well done, but there’s no clear “do this, then do this, then do this”. There’s just “install - here’s a mess of possible configs, good luck!”.

From what I’ve read, I’m supposed to bridge the two 2.5g nics, remove or kill some offloading stuff in the OS, assign two of the P-cores to the OS, the rest of the P-cores to suricata, and block the e-cores from suricata completely so the OS can use them for other tasks, then use AF_Packet to define the two nics (but there’s only one entry and one default entry in the yaml and I have 3 nics) but there’s conflicting information about how to do that, where it goes in the af_packet section, and what else to change farther down the yaml.

I’m not afraid to learn this, but I’d love a “here’s how to set up the hardware in the OS before moving to the yaml file changes, and here’s whats ok in the yaml and what’s not” kind of guidance. There’s some basic functionality core to the OS that doesn’t seem to be addressed in the Suricata guide - which is fine - but I’m striking out on directions on how to do it in a Mint-compatible way.

My desired setup is this - Modem → firewall/router → Suricata Device → Core switch. The goal is 2.5g port 1 to firewall, port 2 to switch with suricata doing what it’s supposed to in the middle and securing the network. I have a 2gb symmetrical internet service, so I’m trying to get suricata IPS at line speed, hence the 13900hk, 16gb ram and dual 2.5gb ports on the pc. I added the 1gbps usb dongle after reading you really want a third dedicated connection to make the bridge easier to set up.

I’m installing on Linux Mint because I’m somewhat familiar with it, but I’m far from a power user. In windows as a gamer, I spent my time learning how to edit the registry, debloat and kill telemetry and speed up the systems I built - the more I’ve messed with trying to get this interface built, the more I realize I’m a clueless nub and the two OS’s couldn’t be more different.

I chose not to do this on OPNSense or PFSense because I don’t need the functions of either, I’m just looking to add suricata in IPS with some decent rulesets without losing too much speed on my connection.

Any help/advice or guides would be appreciated - I’ve read the guide for 8.0.6 and am more confused with steps than when I started.

System - Intel 13900hk / 16gb ram / 1tb nvme / dual 2.5g intel i-226v nics / one 1g usb ethernet adapter

OS - Linux Mint Cinnamon, latest release

Suricata 8.0.6 with NFQueue enabled

2

It sounds like you are on the right track. I don’t have a full how-to guide; rather, I have some tips to help you.

Forget about p-cores, e-cores, and all that CPU affinity. That’s optimization for down the road. Get things working first.

You’ve already determined you need 3 NICs, which is good. Get that USB management NIC set up and connected to your switch so the machine can access the internet.

Another important step is to configure your 2 IPS NICs not to acquire an IP address at startup. You don’t want them to have an IP. These are the 2 interfaces you are bridging together.

There are 2 ways to bridge these interfaces; one is using the Linux network bridging tools. Do not do this. If you want Suricata in IPS mode, you let Suricata be the bridge, not Linux. Important step; otherwise, you’ll wonder why you have duplicate packets spraying around and nothing being blocked.

Once you have that, your Suricata configuration file really only needs an af-packet configuration section that looks like:

af-packet:
  - interface: wan0
    cluster-id: 1
    copy-mode: ips
    copy-iface: lan0
  - interface: lan0
    cluster-id: 2
    copy-mode: ips
    copy-iface: wan0
  - interface: default
    threads: auto
    defrag: false

There are many other options, but for the most part, I think the defaults are good enough to get started.

Then you start Suricata with --af-packet , and it will bridge the 2 interfaces you have listed in the interface and copy-iface.

One other point: when your IPS machine needs to hit the internet, it should use the management interface, hit your switch, then be bridged back through the IPS bridge and out to the internet. This is partly why you don’t want these interfaces to have IP addresses. Think of them more like using a coupler between ethernet cables.

Alternatively: Forget about Suricata now. Still do all of the above, but instead of using Suricata as a bridge, use the Linux bridging tools. This can often help ensure that the network layout/topology/configuration is all good without introducing Suricata as a variable. When all that is working, disable the Linux bridge, then start Suricata on the same interfaces you were bridging.

I don’t have specifics for Linux Mint, as I’m not that familiar, but LLMs are really good at getting this right if you have specific directions. There are probably Linux Mint particulars to pay attention to if you want it to survive a reboot.

Hope that helps a little.

And with bridge mode, if you break it, just swap a few cables at your switch, and you’re back online while you try again. But eventually, it’s mostly set-and-forget.

I noticed you mentioned you had built Suricata with NFQueue. It’s not going to hurt anything, but it’s also not required for the AF_PACKET bridging IPS you are setting up here.

Also, if building from source, see if a hyperscan or vectorscan package is available on Mint. Building against hyperscan/vectorscan is often our first recommended performance improvement.

3

Appreciate the help - you already cleared up a mess of what I had issues with on the bridging, I’ll knock out the Bridge I created in Mint and just configure the yaml settings correctly.

I’ll need to google around and see about how to prevent the nics from grabbing IP’s, the only connection I’ve made so far is to the dongle. It’s just a fancy reskin of ubuntu, so it shouldn’t be too hard.

My plan was to get it all installed and set up, THEN insert it into the network and test / optimize everything.

I had not yet run into anything about either hyperscan or vectorscan, but from a quick googling, it seems that I should figure this vectorscan thing out as that’s the most recommended (and open source).

I’ve already figured out how to uninstall and purge suricata once, I can do that again and build the package around vectorscan without including NFQueue.

So, I just build a vectorscan install in-place on the box and then install suricata? or is there some sort of flag in the yaml I need to turn on for it to activate and run?

UPDATE

I just did a vectorscan build in place on the box after purging suricata completely. Had to turn fat library off to get it to take, but it worked and shows as 100% installed.

@ish Is there anything else that I should build on the OS or activate in the suricata install build-in-place to optimize this so I don’t have to go back and re-install during optimization?

4

Ubuntu has a package for it, libhyperscan-dev or something, should be all you need. Otherwise its automatically picked up at build time.

I don’t think so. Personally I run the RPMs we build, we also do PPAs for Ubuntu. I also never optimize until needed.

5

Got it. Thanks sir.
Vectorscan is open source now and intel went closed source on hyperscan a couple years ago, apparently.updates are few and far between and development is stagnant, so I went the VS direction since it’s relatively plug & play.
I’ll get Suricata built and see what I broke, :sweat_smile:

6

@ish is there any type of information on how jumbo frames are handled or what we need to change any pertinent settings regarding?

my router creates the vlans and the entire network is set to handle 9000mtu (9014 up to 9216 with headers), and the i226v’s will handle 9000mtu (plus header) as well.

I know how to modify the nics so they’ll always reboot at 9000, but have no idea if we need to adjust anything in the YAML, since the switches pass data to the router through the suricata box.

There’s nothing I can find in the documentation regarding jumbo frames or mtu sizes for af-packet ips inline

7

Start Suricata with -vvv and look for a line like this:

Info: ioctl: br0: MTU 1500 [GetIfaceMTU:util-ioctl.c:101]

If it matches your MTU you should be good to go.

8

Thanks sir.

I have had it up and running for a few days now, and have noticed some weird behavior I’m trying to track down.

For reference, I have Suricata set to IPS with af-packet and removed nfq, but have it set to alert only for now. I’ve also installed crowdsec side by side and have the suricata plugin thing loaded to monitor the eve-json log, but the bouncer is currently disabled.

Within 30 seconds of booting up, I lost connection to an old OBI302 ATA VOIP device connected to a broadvoice line. I bypassed the Suricata/Crowdsec box completely and plugged it directly into my Firewalla Gold Plus and it came right back up with zero issue.

Then, about 48 hours later, my Polycom Poly Edge E450 VOIP phone (Nextiva) lost connection, and each time I would reboot it or the Suricata box, it would try to connect for 30 seconds or so then fail. Again, I bypassed the suricata box and it connects fine when directly connected to the FWG+.

The weird part is that I have a Vonage ATA device by Grandstream - an HT802 that has never lost connection or had any weird behavior.

All 3 devices have fixed IP’s on the LAN, QOS, traffic shaping, smart queue etc is off, there is no Voice VLAN, and I disabled all security and eee features on my switch they were connected to (D-Link DMS-1250-28).

Topology is Spectrum Modem → FWG+ → NAB9S (suricata/crowdsec on Mint) → DMS-1250 then the three different VOIPs were each on their own port on the switch. I moved the two with issues over to ports 2 & 3 on the FWG+ to eliminate the failure while I troubleshoot.

Other odd symptoms - my primary PC will disconnect from the internet after a bit of inactivity (30 minutes or so?) and when I try to get it to reconnect, it will show a network connection but won’t actually let me out to search without a reboot. This is new with the suricata box.

In the zeek function on the FWG+, it also shows some of the other switches dropping connection (light goes from green for connected to gray for offline) except the connected devices all work and I can access the webgui on those switches - it just reports them as offline now. Oddly, if enough bandwidth goes through them, they come back online.

I’ve tried to get some guidance from various AI agents and read some threads on reddit and other forums but am coming up empty with actual solutions.

The AI was saying it was likely related to either SIP being identified as a threat and blocked (but there’s no bouncer and no ips drops showing in evebox) or some sort of UDP connection timeout. It also suggested issues with offloading on the NICs that are inline and copying in af-packet or possibly a pcie aspm error so I shut those all off in grub and via a script that runs a mess of ethtool commands on reboot.

Not entirely sure what could be causing this. Not catastrophic, but I haven’t started blocking or dropping yet with either crowdsec or suricata - just monitoring and troubleshooting before I make things more complicated.