# JQ cheat sheet for parsing Suricata EVE outputs

**URL:** <https://forum.suricata.io/t/jq-cheat-sheet-for-parsing-suricata-eve-outputs/3763>\
**Category:** Guides\
**Created:** [July 31, 2023, 7:57pm UTC](https://forum.suricata.io/t/jq-cheat-sheet-for-parsing-suricata-eve-outputs/3763 "2023-07-31T19:57:13Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jufajardini](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.suricata.io/jufajardini/32/896_2.png) [@jufajardini](https://forum.suricata.io/u/jufajardini)\
**Post date:** [July 31, 2023, 7:57pm UTC](https://forum.suricata.io/t/jq-cheat-sheet-for-parsing-suricata-eve-outputs/3763/1 "2023-07-31T19:57:13Z")

</div>

# JQ quick commands for some common usage situations for Suricata EVE logs

As shared by @cthomas in July’s 2023 webinar: [Using JQ to parse Suricata logs](https://youtu.be/qBDD6Xq3OTE).

### Pick out single event type

```auto
jq -c 'select(.alert)' eve.json

```

### Pick out event type with single condition

```auto
jq 'select(.event_type=="tls" and .tls.subject==.tls.issuerdn)' eve.json

```

```auto
jq 'select(.event_type=="tls" and .tls.version=="TLSv1")' eve.json

```

### Pick out event type with multiple conditions

```auto
jq 'select(.event_type=="flow" and .flow.state=="established" and .flow.age >1800 and .app_proto=="smb")' eve.json

```

```auto
jq 'select(.event_type=="flow" and .flow.state=="established" and .flow.age>1800 and .app_proto=="smb" and .timestamp>"2019-04-29T14:38")' eve.json 

```

### String comparisons

```auto
jq -c 'select(.event_type=="alert")' eve.json | jq -c 'select(.alert.signature | contains("HTTP") )'

```

```auto
jq -c 'select((.event_type=="alert") and (.alert.signature | contains("HTTP") ) )' eve.json

```

#### alternatives:

```auto
   startswith("str")

```

```auto
   endswith("str")

```

### Conditions, but only a single field

```auto
jq -c 'select((.event_type=="alert") and (.alert.signature | contains("HTTP") ) ) | .alert.signature' eve.json

```

### Counting events

```auto
jq 'select(.event_type=="smb" and .smb.status == "STATUS_ACCESS_DENIED")' eve.json | jq .flow_id | sort | uniq -c

```

```auto
grep '"event_type":' eve.json | jq .event_type | sort -rn | uniq -c | sort -rn

```

### Conditions and multiple fields

```auto
jq -c 'select((.event_type=="alert") and (.alert.signature | contains("HTTP") ) ) | .alert.signature, .flow.bytes_toclient' eve.json

```

### Joining multiple fields

```auto
jq -r 'select((.event_type=="alert") and (.alert.signature | contains("HTTP") ) ) | [.alert.signature, .flow.bytes_toclient] | join(", ")' eve.json

```

### Importing (into csv)

```auto
jq -r 'select((.event_type=="alert") and (.alert.signature | contains("HTTP") ) ) | [.alert.signature, .flow.bytes_toclient] | join(", ")' eve.json | sort -d > /tmp/test.csv

```

### Mixing data types

(pre-command for combining filestore)

```auto
find filestore/ | grep "json$" | xargs cat >> full.json

```

```auto
jq -r '.src_ip +" "+ .dest_ip +" "+ (.src_port | tostring) +" "+
(.dest_port | tostring)' full.json |
while read line ; do read sip dip sp dp <<<$(echo $line) ; echo "srcip = $sip --- dstip = $dip --- srcport = $sp --- dstport = $dp" ; done

```

### Avoiding ‘null’ empty fields

(has null in IPS ‘drop’ eve type)

```auto
jq '.alert.signature' drops.json

```

(removes any null lines)

```auto
jq '.alert.signature | select ( . != null)' drops.json

```

(gets fields that exist on your own priority)

```auto
jq 'if .alert.signature then .alert.signature elif .metadata.flowbits > 0 then .metadata.flowbits[] else .proto end' drops.json

```

### DNS rrname sort

```auto
jq 'select(.event_type=="dns")' eve.json | jq .dns.rrname | sort -rn | uniq -c | sort -rn

```

### PCAP splitting script

```bash
#!/bin/bash
njobs="\j" # special bash job info

jq -c -r '.src_ip +" "+ .dest_ip +" "+ (.src_port|tostring) +" "+ (.dest_port|tostring)' filestore_combined.json |

while read line ; do
   while (( ${njobs@P} >= 29)) ; do wait -n ; done # wait for this process job count to be below 29
   read sip dip sp dp <<<$(echo $line)
   echo "srcip = $sip --- $dstip = $dip --- srcport = $sp --- dstport = $dp"

tcpdump -r /path/to/200someGB.pcap "host $sip and host $dip and port $sp and port $dp" -w /tmp/split-pcaps/${sip}_${sp}-${dip}_${dp}.pcap &

done

```

## Extras Base vs Split EVE json

### Default EVE

```yaml
outputs:
  # a line based alerts log similar to Snort's fast.log
  - fast:
      enabled: yes
      filename: fast.log
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

  # Extensible Event Format (nicknamed EVE) event log in JSON format
  - eve-log:
      enabled: yes
      filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
      filename: eve.json
    ...
    ...
    ...
      types:
        - alert:
            # payload: yes # enable dumping payload in Base64
            # payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
            # payload-printable: yes # enable dumping payload in printable (lossy) format
            # packet: yes # enable dumping of packet (without stream segments)
            # metadata: no # enable inclusion of app layer metadata with alert. Default yes
            # http-body: yes # Requires metadata; enable dumping of HTTP body in Base64
            # http-body-printable: yes # Requires metadata; enable dumping of HTTP body in printable format

            # Enable the logging of tagged packets for rules using the
            # "tag" keyword.
            tagged-packets: yes

```

### Split EVE

```yaml
outputs:
  # a line based alerts log similar to Snort's fast.log
  - fast:
      enabled: yes
      filename: /dev/null
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

  # Stats.log contains data from various counters of the Suricata engine.
  - eve-log:
      enabled: yes
      append: no # append to file (yes) or overwrite it (no)
      filename: stats.json
      types:
      - stats:
          totals: yes # stats for all threads merged together
          threads: no # per thread stats
          deltas: no # include delta values
    ...
    ...
    ...
  # Extensible Event Format (nicknamed EVE) event log in JSON format
  - eve-log:
      enabled: yes
      filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
      filename: eve.json
    ...
    ...
    ...
        #- stats:
        # totals: yes # stats for all threads merged together
        # threads: no # per thread stats
        # deltas: no # include delta values

```