Suricata Failed to Startup and Failed to Drop Privileges in Docker Container

Help
1

Issue

Suricata cannot startup success with root privilege with the error <Error> - [ERRCODE: SC_ERR_FATAL(171)] - capng_change_id for main thread failed inside docker container.

Expect

I hope to run suricata with root and drop privileges after startup success inside a docker container.
Then I think suricata process is owned by non-root user “suricata” and this practise is much more secure.

Environment

Host Info

host machine: ubuntu 20.04
host machine Kernal: Linux xo 5.15.0-88-generic #98~20.04.1-Ubuntu SMP Mon Oct 9 16:43:45 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Docker info: Docker Server 23.0.1

Container Info

OS: Debian 12(Bookworm)

Suricata Info

suricata.yaml

%YAML 1.1
---



vars:
    HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]" 

    EXTERNAL_NET: "!$HOME_NET"

    HTTP_SERVERS: "$HOME_NET"
    SMTP_SERVERS: "$HOME_NET"
    SQL_SERVERS: "$HOME_NET"
    DNS_SERVERS: "$HOME_NET"
    TELNET_SERVERS: "$HOME_NET"
    AIM_SERVERS: "$EXTERNAL_NET"
    DC_SERVERS: "$HOME_NET"
    DNP3_SERVER: "$HOME_NET"
    DNP3_CLIENT: "$HOME_NET"
    MODBUS_CLIENT: "$HOME_NET"
    MODBUS_SERVER: "$HOME_NET"
    ENIP_CLIENT: "$HOME_NET"
    ENIP_SERVER: "$HOME_NET"

  port-groups:
    HTTP_PORTS: "80"
    SHELLCODE_PORTS: "!80"
    ORACLE_PORTS: 1521
    SSH_PORTS: 22
    DNP3_PORTS: 20000
    MODBUS_PORTS: 502
    FILE_DATA_PORTS: "[$HTTP_PORTS,110,143]"
    FTP_PORTS: 21
    GENEVE_PORTS: 6081
    VXLAN_PORTS: 4789
    TEREDO_PORTS: 3544


default-log-dir: /var/log/suricata/

stats:
  enabled: yes
  interval: 8

outputs:
  - fast:
      enabled: no
      filename: fast.log
      append: yes

  - eve-log:
      enabled: yes
      filename: suricata_event.json

      metadata: yes

      pcap-file: false


      community-id: false
      community-id-seed: 0

      xff:
        enabled: no
        mode: extra-data
        deployment: reverse
        header: X-Forwarded-For

      types:
        - alert:

        - anomaly:
            enabled: no
            types:
        - http:
        - dns:


            enabled: yes



        - tls:
        - files:
        - smtp:

        - dnp3
        - ftp
        - rdp
        - nfs
        - smb
        - tftp
        - ikev2
        - dcerpc
        - krb5
        - snmp
        - rfb
        - sip
        - dhcp:
            enabled: yes
            extended: no
        - ssh
        - mqtt
        - http2
        - stats:
        - flow

        - metadata

  - http-log:
      enabled: yes
      filename: http.log
      append: yes

  - tls-log:
      append: yes

  - tls-store:
      enabled: yes

  - pcap-log:
      enabled: no
      filename: log.pcap

      limit: 1000mb

      max-files: 2000

      compression: none





  - alert-debug:
      enabled: no
      filename: alert-debug.log
      append: yes

  - alert-prelude:
      enabled: no
      profile: suricata
      log-packet-content: yes
      log-packet-header: yes

  - stats:
      enabled: yes
      filename: stats.log

  - syslog:
      enabled: no
      facility: local5

  - file-store:
      version: 2
      enabled: yes

      dir: /var/log/suricata-filestore/filestore

      write-fileinfo: yes

      force-filestore: yes

      stream-depth: 0


      force-hash: [sha1, md5]
      xff:
        enabled: no
        mode: extra-data
        deployment: reverse
        header: X-Forwarded-For

  - tcp-data:
      enabled: no
      type: file
      filename: tcp-data.log

  - http-body-data:
      enabled: no
      type: file
      filename: http-data.log

  - lua:
      enabled: no
      scripts:

logging:
  default-log-level: notice


  default-output-filter:


  outputs:
  - console:
      enabled: yes
  - file:
      enabled: yes
      level: info
      filename: suricata.log
  - syslog:
      enabled: no
      facility: local5
      format: "[%i] <%d> -- "



af-packet:
    cluster-id: 99
    cluster-type: cluster_flow
    defrag: yes

  - interface: default 
    threads: auto
    use-mmap: yes
    tpacket-v3: yes

pcap:
  - interface: eth0
  - interface: default

pcap-file:
  checksum-checks: auto




app-layer:
  protocols:
    rfb:
      enabled: yes
      detection-ports:
        dp: 5900, 5901, 5902, 5903, 5904, 5905, 5906, 5907, 5908, 5909
    mqtt:
      enabled: yes
    krb5:
      enabled: yes
    snmp:
      enabled: yes
    ikev2:
      enabled: yes
    tls:
      enabled: yes
      detection-ports:
        dp: 443



    dcerpc:
      enabled: yes
    ftp:
      enabled: yes
    rdp:
      enabled: yes
    ssh:
      enabled: yes
    http2:
      enabled: yes
      http1-rules: no
    smtp:
      enabled: yes
      raw-extraction: no
      mime:
        decode-mime: yes

        decode-base64: yes
        decode-quoted-printable: yes

        header-value-depth: 2000

        extract-urls: yes
        body-md5: no
      inspected-tracker:
        content-limit: 100000
        content-inspect-min-size: 32768
        content-inspect-window: 4096
    imap:
      enabled: detection-only
    smb:
      enabled: yes
      detection-ports:
        dp: 139, 445


    nfs:
      enabled: yes
    tftp:
      enabled: yes
    dns:
      tcp:
        enabled: yes
        detection-ports:
          dp: 53
      udp:
        enabled: yes
        detection-ports:
          dp: 53
    http:
      enabled: yes



      libhtp:
         default-config:
           personality: IDS

           request-body-limit: 100kb
           response-body-limit: 100kb

           request-body-minimal-inspect-size: 32kb
           request-body-inspect-window: 4kb
           response-body-minimal-inspect-size: 40kb
           response-body-inspect-window: 16kb

           response-body-decompress-layer-limit: 2

           http-body-inline: auto

           swf-decompression:
             enabled: yes
             type: both
             compress-depth: 100kb
             decompress-depth: 100kb


           double-decode-path: no
           double-decode-query: no


         server-config:



    modbus:

      enabled: yes
      detection-ports:
        dp: 502

      stream-depth: 0

    dnp3:
      enabled: yes
      detection-ports:
        dp: 20000

    enip:
      enabled: yes
      detection-ports:
        dp: 44818
        sp: 44818

    ntp:
      enabled: yes

    dhcp:
      enabled: yes

    sip:
      enabled: yes

asn1-max-frames: 256




run-as:
  user: suricata
  group: suricata


pid-file: /var/run/suricata/suricata.pid




coredump:
  max-dump: unlimited

host-mode: auto





unix-command:



legacy:
  uricontent: enabled





engine-analysis:
  rules-fast-pattern: yes
  rules: yes

pcre:
  match-limit: 3500
  match-limit-recursion: 1500


host-os-policy:
  windows: [0.0.0.0/0]
  bsd: []
  bsd-right: []
  old-linux: []
  linux: []
  old-solaris: []
  solaris: []
  hpux10: []
  hpux11: []
  irix: []
  macos: []
  vista: []
  windows2k3: []


defrag:
  hash-size: 65536
  prealloc: yes
  timeout: 60



flow:
  hash-size: 65536
  prealloc: 10000
  emergency-recovery: 30

vlan:
  use-for-tracking: true


flow-timeouts:

  default:
    new: 30
    established: 300
    closed: 0
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-closed: 0
    emergency-bypassed: 50
  tcp:
    new: 60
    established: 600
    closed: 60
    bypassed: 100
    emergency-new: 5
    emergency-established: 100
    emergency-closed: 10
    emergency-bypassed: 50
  udp:
    new: 30
    established: 300
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-bypassed: 50
  icmp:
    new: 30
    established: 300
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-bypassed: 50

stream:
  reassembly:
    memcap: 256mb
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes

host:
  hash-size: 4096
  prealloc: 1000
  memcap: 32mb



decoder:
  teredo:
    enabled: true

  vxlan:
    enabled: true

  vntag:
    enabled: false

  geneve:
    enabled: true



detect:
  profile: medium
  custom-values:
    toclient-groups: 3
    toserver-groups: 25
  sgh-mpm-context: auto
  inspection-recursion-limit: 3000

  prefilter:
    default: mpm

  grouping:

  profiling:
    grouping:
      dump-to-disk: false
      include-mpm-stats: false


mpm-algo: auto


spm-algo: auto

threading:
  set-cpu-affinity: no
  cpu-affinity:
    - management-cpu-set:
    - receive-cpu-set:
    - worker-cpu-set:
        cpu: [ "all" ]
        mode: "exclusive"
        prio:
          low: [ 0 ]
          medium: [ "1-2" ]
          high: [ 3 ]
          default: "medium"
  detect-thread-ratio: 1.0

luajit:
  states: 128

profiling:

  rules:

    enabled: no 
    filename: rule_perf.log
    append: yes


    limit: 10

    json: yes

  keywords:
    enabled: no
    filename: keyword_perf.log
    append: yes

  prefilter:
    enabled: no
    filename: prefilter_perf.log
    append: yes

  rulegroups:
    enabled: no
    filename: rule_group_perf.log
    append: yes

  packets:

    enabled: no
    filename: packet_stats.log
    append: yes

    csv:

      enabled: no
      filename: packet_stats.csv

  locks:
    enabled: no
    filename: lock_stats.log
    append: yes

  pcap-log:
    enabled: no
    filename: pcaplog_stats.log
    append: yes


nfq:

nflog:
  - group: 2
    buffer-size: 18432
  - group: default
    qthreshold: 1
    qtimeout: 100
    max-size: 20000


capture:
  disable-offloading: true

netmap:
 - interface: eth2
 - interface: default

pfring:
  - interface: eth0
    threads: auto

    cluster-id: 99

    cluster-type: cluster_flow



  - interface: default

ipfw:



napatech:

    streams: ["0-3"]

    enable-stream-stats: no

    auto-config: yes

    hardware-bypass: yes

    inline: no

    ports: [0-1,2-3]

    hashmode: hash5tuplesorted


default-rule-path: /opt/suricata/etc/suricata/config/rules

rule-files:
  - suricata.rules


classification-file: /opt/suricata/etc/suricata/config/rules/classification.config
reference-config-file: /opt/suricata/etc/suricata/reference.config
threshold-file: /opt/suricata/etc/suricata/threshold.config

Build Suricata

I have added params as suricata manual said 10.5. Dropping Privileges After Startup — Suricata 6.0.2 documentation

#!/bin/bash

set -eu

CURR_DIR="./suricata"
WORK_DIR="/opt/suricata"
CONF_DIR="${WORK_DIR}/etc"
LOCAL_STATE_DIR="${WORK_DIR}/var"
EXEC_FILE="${WORK_DIR}/bin/suricata"
LIB_DIR="${WORK_DIR}/lib"

echo "=========== Start build suricata ==========="


# Check the build dir whether exist.
if [[ ! -d ${CURR_DIR} ]]; then
    echo "The dir ${CURR_DIR} does not exist."
    exit 1
fi
cd ${CURR_DIR}

./configure --prefix=${WORK_DIR} \
            --sysconfdir=${CONF_DIR} \
             --localstatedir=${LOCAL_STATE_DIR}  \
             --disable-gccmarch-native --enable-geoip \
             --with-libcap_ng-libraries=/usr/local/lib \
             --with-libcap_ng-includes=/usr/local/include

make && make install-full

# Check the binary file whether exist.
if [[ ! -f ${EXEC_FILE} ]]; then
    echo "The binary file ${EXEC_FILE} does not exist."
    exit 1
fi
SO_LIST=$(ldd ${EXEC_FILE} | awk '{if (match($3, "/")){print $3}}')


# Check the lib dir whether exist.
if [[ ! -d ${LIB_DIR} ]]; then
    echo "The dir ${LIB_DIR} does not exist."
    exit 1
fi
cp -L -n ${SO_LIST} ${LIB_DIR}

echo "=========== Build suricata success ==========="

Run suricata

I use the following command to start the container and this is just an example.

docker run -it --name suricata --rm debian:12

The user suricata is created and path access is granted as manual said 5. Security Considerations — Suricata 8.0.0-dev documentation

useradd --no-create-home --system --shell /sbin/nologin suricata

mkdir -p /var/log/suricata    /var/run/suricata

chgrp -R suricata /opt/suricata/etc \
    && chmod -R g+r /opt/suricata/etc \
    && chgrp -R suricata /var/log/suricata  \
    && chmod -R g+rw /var/log/suricata   \
    && chgrp -R suricata /opt/suricata/lib \
    && chmod -R g+srw    /opt/suricata/lib \
    && chgrp -R suricata /var/run/suricata \
    && chmod -R g+srw /var/run/suricata

Current user uid root and The result is below.

root@093ccb9acde2:/# /opt/suricata/bin/suricata  --af-packet  -c /opt/suricata/etc/suricata/suricata.yaml   -D   --user suricata --group suricata
21/11/2023 -- 05:38:07 - <Notice> - This is Suricata version 6.0.9 RELEASE running in SYSTEM mode
root@093ccb9acde2:/#

What Suricata log said

Solution 1

I just run the command above and got the errors.

1/11/2023 -- 05:38:07 - <Notice> - This is Suricata version 6.0.9 RELEASE running in SYSTEM mode
21/11/2023 -- 05:38:07 - <Info> - CPUs/cores online: 6
21/11/2023 -- 05:38:07 - <Info> - Found an MTU of 1500 for 'eth0'
21/11/2023 -- 05:38:07 - <Info> - Found an MTU of 1500 for 'eth0'
21/11/2023 -- 05:38:07 - <Info> - Use pid file /var/run/suricata/suricata.pid from config file.
21/11/2023 -- 05:38:07 - <Error> - [ERRCODE: SC_ERR_FATAL(171)] - capng_change_id for main thread failed

Solution 2

I think the issue above comes from that the normal user “suricata” has no capacity to own the suricata process. So I setcap for suricata.

root@093ccb9acde2:/# setcap 'cap_net_admin,cap_net_raw,cap_sys_nice+eip' /opt/suricata/bin/suricata
root@093ccb9acde2:/# ls -lh /opt/suricata/bin/suricata
-rwxr-xr-x 1 suricata suricata 57M Nov 21 02:21 /opt/suricata/bin/suricata
root@093ccb9acde2:/#

But I got permision denied.

root@093ccb9acde2:/#  /opt/suricata/bin/suricata  --af-packet  -c /opt/suricata/etc/suricata/suricata.yaml   -D   --user suricata --group suricata   
bash: /opt/suricata/bin/suricata: Operation not permitted
root@093ccb9acde2:/#

Questions

  1. For security concern, I hope to startup suricata with root and then suricata process could drop root privileges and it is just owned by non-root user finally. Am I right?
  2. The non-root user should be granted some capacities with linux command setcap as the manual said. Right?
    image
    image739×319 15.7 KB
  3. What can I do to fix the issues above?
  4. How could I vrify that suricata process really drop root privileges?

Thank you for your great support on me.

2

Could you please include the error message? As well as any other relevant information like the version, Linux distribution, etc?

3

Thanks for your patience, Jason
I just finish this post.
Could you please give me some ideas?
Thanks~

4

Did you try

docker run --cap-add=net_admin --cap-add=net_raw --cap-add=sys_nice ...
5

Hello Jason,
Thanks for your help. I got success really.
But I still have a few questions here.

What I Do

Build suricata source

apt-get install libcap-ng-dev, without libcap-ng in my docker container.


...

./configure --prefix=${WORK_DIR} \
            --sysconfdir=${CONF_DIR} \
             --localstatedir=${LOCAL_STATE_DIR}  \
             --disable-gccmarch-native --enable-geoip \
	     --with-libcap_ng-libraries=/usr/local/lib \
             --with-libcap_ng-includes=/usr/local/include

...

Run container with cap-add

d run --cap-add=net_admin --cap-add=net_raw --cap-add=sys_nice  -d --name suricataenvNOng --rm  suricataImgae:6.0.9-1   sleep 99999999998

Run suricata process with root user and give non-root user param.

root@vnbjs:~# /opt/suricata/bin/suricata  --af-packet  -c /opt/suricata/etc/suricata/suricata.yaml   -D   --user secur1ty --group secur1ty

image
image1558×179 28.3 KB

Questions

  1. We still need set capacity for suricata/bin/suricata binary file with setcap binary. I mean we use setcap here, Is it more secure way comparing with starting it with root?
  2. What is the process below in my container?
    root         573       1  0 02:04 pts/0    00:00:00 [sh] <defunct>
root        2880       1  0 02:06 pts/0    00:00:00 [conftest] <defunct>
  3. Is libcap-ng-dev alternative for libcap-ng? Is it OK to use here?
  4. What is the detailed rules for suricata drop root privileges?

Thank you all.

6
  1. You don’t need to set the capabilities on the binary. Even if they are set, the container still needs to be started with them. Suricata’s privilege dropping is is based on starting as root, then dropping privs. I’m not sure if it can make use of the capabilities being set on the binary as I’ve not tested that myself.
  2. I’m not sure, could be related to starting Suricata with -D? That shouldn’t be done in a container, instead of Docker detatch.
  3. libcap-ng-dev is probably requiring libcap-ng. The -dev is required for building Suricata. If you have Suricata already built, you can use the non-dev version.
  4. I’m not sure what you mean here. Have you checked 5. Security Considerations — Suricata 8.0.0-dev documentation?

You might also want to look at my Dockerfiles’s as a reference: GitHub - jasonish/docker-suricata: A Suricata Docker image.. If the container is started with the correct capabilities it will drop privileges, otherwise it won’t.

7

Hi Jason,
Thanks. I checked the suricata doc and your repo.
1. Yes, I have not setcap for the suricata binary file manually. I just setcap for container when I start container with command “docker run --cap-add …”

Another concern is that we setcap for the container and I think if attacker attacks my container, he could do some harmful operations. You can see that all process have the cap

cap_net_admin,cap_net_raw,cap_sys_nice

image
image1902×316 50.5 KB

So is there any security concerns here when we use “docker run --cap-add …” ?

Thank you~