Suricata stats and Telegraf

Help
1

Hello,

Trying to have Telegraf pick op Suricata stats as described via a socket: telegraf/plugins/inputs/suricata/README.md at release-1.27 · influxdata/telegraf · GitHub

If I configure stats under outputs: it is communicating via the socket, but is clearly not in json format because Telegraf goes bezirk.

  • stats:
    enabled: yes
    #filename: stats.log
    #append: yes # append to file (yes) or overwrite it (no)
    #totals: yes # stats for all threads merged together
    #threads: no # per thread stats
    #Telegraf
    filetype: unix_stream
    filename: /var/run/telegraf/suricata-stats.sock
    threads: yes # per thread stats

But I’ve no idea how to configure this under outputs: - eve-log: without messing up al other json data output to regular json files:

  • eve-log:
    enabled: yes
    filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
    filename: /data/sensor_data/suricata/eve.json
    #Enable for multi-threaded eve.json output; output files are amended with
    #an identifier, e.g., eve.9.json
    #threaded: false
    threaded: true
    …

So where should I put

  • eve-log:
    enabled: yes
    filetype: unix_stream
    filename: /tmp/suricata-stats.sock
    types:
    - stats:
    threads: yes

Thanks!

2

Oke seems this a config suricata accepts and connects to the socket but also creates all json files, great only Telegraf does not log any suricata stats yet.

#Stats via Telegraf
- eve-log:
enabled: yes
filetype: unix_stream
filename: /var/run/telegraf/suricata-stats.sock
types:
- stats:
threads: yes

#Extensible Event Format (nicknamed EVE) event log in JSON format
- eve-log:
enabled: yes
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
filename: /data/sensor_data/suricata/eve.json

  #Enable for multi-threaded eve.json output; output files are amended with
  #an identifier, e.g., eve.9.json
  #threaded: false
  threaded: true
3

And a next step: it works on RHEL 7 Suricata 6.13
Now found an old post from Victor and ran this on RH7 Suri6 and the RH 8 Suri7 box:
]# nc -U -l /var/run/telegraf/suricata-stats.sock | tee /tmp/tel.log

On both servers data pops up in json format!
Now stop netcat and let’s see:
Sec6 :
]# cat /tmp/tel.log | jq
nice json formatted data shows

Sec7
]# cat /tmp/tel.log | jq
parse error: Invalid literal at line 1, column 219276

So guess this explains why Telegraf on the Suri7 box does not log data from Suri, it is unable to parse it because Suri sends it garbled or there is some buffer issue.

4

Changed suricata.yaml threads: yes to threads: no and now Telegraf is able to pick up the socket data correctly. Seems fair seen the number of suricata threads (232) on this server:
Main PID: 664696 (Suricata-Main)
Tasks: 232 (limit: 820979)

@inliniac is this a socket limit or a ‘problem’ on the side of suricata stats logging via a socket?

5

The use of threaded in the eve-log section will parallelize handling of file (or regular) output types.

Using threaded: yes (or enabled, 1, or on) would result in a fatal startup error with the message Socket file types do not support threaded output

6

@Jeff_Lucovsky well Suricata 7rc2 does start and functions, even sending data to the socket but not all data. Did not see the error message you mention.
But that’s a detail, I will not use the threaded option for sure.

7

Could you post the outputs section of your suricata configuration file (usually, suricata.yaml)?

8

Here it is, thanks!
suricata.yaml (20.5 KB)

9

The threads configuration setting in the stats section controls whether per-thread statistics are also reported. I’m not sure why this is affecting your situation in a positive way.

10

To be sure:

  • eve-log:
    enabled: yes
    filetype: unix_stream
    filename: /var/run/telegraf/suricata-stats.sock
    types:
    - stats:
    threads: no
    #threads: yes

Enabling threads in this section generates more data which reaches a tipping point somehow because the json data recevied at the socket is not complete/capped

11

I’m glad things are working for you.

1 Like