Hello,
I have the problem that apparently the exact protocol of the connection is not detected. I want to detect ssh traffic on all ports other than 22 and have written the following rule:
alert tcp any any -> any !22 (msg:"SSH connect on non default port"; app-layer-protocol:ssh; sid:1000008; rev:1;)
If I remove “app-layer-protocol:ssh” and change the port to, for example, 5555 it is working if I connect to this port. But of course the rule applies to all traffic that is also non-ssh.
When I listen on the interface with tcpdump I see the incoming requests. However suricata does not report them.
Another protocol related problem is saving files. I want to save all pdf files to the filestore and have created the following rule:
alert tcp any any -> any any (msg:"FILE store pdf"; fileext:"pdf"; sid:1; rev:1;)
Although a lot of traffic goes through the interface, only very small xml files are saved to the filestore from time to time. The above rule is not recognized and other files are not saved even with the setting “force-filestore: yes”. It looks to me like tcp streams are only evaluated up to a certain size, which is why only the small files are found.
It is a quite performant server (32cores/128GB RAM) running in IDS mode. On eth5 it gets the traffic of a datacenter mirrored via mirror port.
suricata --build-info
This is Suricata version 6.0.5 RELEASE
Features: PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LIBJANSSON TLS TLS_C11 MAGIC RUST
SIMD support: none
Atomic intrinsics: 1 2 4 8 byte(s)
64-bits, Little-endian architecture
GCC version 7.5.0, C version 201112
compiled with _FORTIFY_SOURCE=0
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.40, linked against LibHTP v0.5.40
Suricata Configuration:
AF_PACKET support: yes
eBPF support: no
XDP support: no
PF_RING support: no
NFQueue support: no
NFLOG support: no
IPFW support: no
Netmap support: no
DAG enabled: no
Napatech enabled: no
WinDivert enabled: no
Unix socket enabled: yes
Detection enabled: yes
Libmagic support: yes
libnss support: yes
libnspr support: yes
libjansson support: yes
hiredis support: no
hiredis async with libevent: no
Prelude support: no
PCRE jit: yes
LUA support: yes
libluajit: no
GeoIP2 support: yes
Non-bundled htp: no
Hyperscan support: yes
Libnet support: no
liblz4 support: yes
HTTP2 decompression: no
Rust support: yes
Rust strict mode: no
Rust compiler path: /usr/bin/rustc
Rust compiler version: rustc 1.60.0
Cargo path: /usr/bin/cargo
Cargo version: cargo 1.60.0
Cargo vendor: yes
Python support: yes
Python path: /usr/bin/python3
Python distutils yes
Python yaml yes
Install suricatactl: yes
Install suricatasc: yes
Install suricata-update: yes
Profiling enabled: no
Profiling locks enabled: no
Plugin support (experimental): yes
Development settings:
Coccinelle / spatch: no
Unit tests enabled: no
Debug output enabled: no
Debug validation enabled: no
Generic build parameters:
Installation prefix: /usr
Configuration directory: /etc/suricata/
Log directory: /var/log/suricata/
--prefix /usr
--sysconfdir /etc
--localstatedir /var
--datarootdir /usr/share
Host: x86_64-pc-linux-gnu
Compiler: gcc (exec name) / gcc (real)
GCC Protect enabled: no
GCC march native enabled: no
GCC Profile enabled: no
Position Independent Executable enabled: no
CFLAGS -g -O2 -std=c11 -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist
PCAP_CFLAGS -I/usr/include
SECCFLAGS
Here is the complete config:
%YAML 1.1
---
##
## network
##
vars:
address-groups:
HOME_NET: "[10.2.0.0/16,83.246.65.0/24,94.100.136.0/24]"
EXTERNAL_NET: "!$HOME_NET"
HTTP_SERVERS: "$HOME_NET"
SMTP_SERVERS: "$HOME_NET"
SQL_SERVERS: "$HOME_NET"
DNS_SERVERS: "$HOME_NET"
TELNET_SERVERS: "$HOME_NET"
AIM_SERVERS: "$EXTERNAL_NET"
DC_SERVERS: "$HOME_NET"
DNP3_SERVER: "$HOME_NET"
DNP3_CLIENT: "$HOME_NET"
MODBUS_CLIENT: "$HOME_NET"
MODBUS_SERVER: "$HOME_NET"
ENIP_CLIENT: "$HOME_NET"
ENIP_SERVER: "$HOME_NET"
port-groups:
HTTP_PORTS: "80"
SHELLCODE_PORTS: "!80"
ORACLE_PORTS: 1521
SSH_PORTS: 22
DNP3_PORTS: 20000
MODBUS_PORTS: 502
FILE_DATA_PORTS: "[$HTTP_PORTS,110,143]"
FTP_PORTS: 21
GENEVE_PORTS: 6081
VXLAN_PORTS: 4789
TEREDO_PORTS: 3544
##
## outputs
##
default-log-dir: /var/log/suricata/
stats:
enabled: yes
interval: 8
#decoder-events: true
#decoder-events-prefix: "decoder.event"
#stream-events: false
outputs:
- fast:
enabled: yes
filename: fast.log
append: yes
filetype: regular
- eve-log:
enabled: yes
filetype: regular
filename: eve.json
ethernet: yes
metadata: yes
community-id: false
community-id-seed: 0
types:
- alert:
payload: yes
payload-buffer-size: 4kb
payload-printable: yes
packet: no
metadata: yes
http-body: yes
http-body-printable: yes
tagged-packets: yes
- anomaly:
enabled: yes
types:
decode: no
stream: no
applayer: yes
packethdr: no
- http:
extended: yes
#custom: [Accept-Encoding, Accept-Language, Authorization]
dump-all-headers: both
- tls:
extended: yes
#session-resumption: no
#custom: [subject, issuer, session_resumed, serial, fingerprint, sni, version, not_before, not_after, certificate, chain, ja3, ja3s]
- files:
force-magic: yes
force-hash: [md5,sha256]
- smtp:
extended: yes
#custom: [received, x-mailer, x-originating-ip, relays, reply-to, bcc]
#md5: [body, subject]
- dnp3
- ftp
- rdp
- nfs
- smb
- tftp
- ikev2
- dcerpc
- krb5
- snmp
- rfb
- sip
- dhcp:
enabled: yes
extended: yes
- ssh
- mqtt:
passwords: yes
- http2
- stats:
totals: yes
threads: no
deltas: no
- flow
- netflow
- metadata
- eve-log:
enabled: yes
filetype: regular
filename: dns.json
types:
- dns:
enabled: no
requests: yes
responses: yes
formats: [detailed, grouped]
types: [a, aaaa, cname, mx, ns, ptr, txt]
- http-log:
enabled: yes
filename: http.log
append: yes
#extended: yes
#custom: yes
#customformat: "%{%D-%H:%M:%S}t.%z %{X-Forwarded-For}i %H %m %h %u %s %B %a:%p -> %A:%P"
filetype: regular
- tls-log:
enabled: yes
filename: tls.log
append: yes
#extended: yes
#custom: yes
#customformat: "%{%D-%H:%M:%S}t.%z %a:%p -> %A:%P %v %n %d %D"
filetype: regular
#session-resumption: no
- tls-store:
enabled: yes
certs-log-dir: certs
- stats:
enabled: yes
filename: stats.log
append: yes
totals: yes
threads: no
#null-values: yes
# To prune the filestore directory see the "suricatactl filestore
# prune" command which can delete files over a certain age.
- file-store:
version: 2
enabled: yes
dir: filestore
write-fileinfo: yes
#force-filestore: yes
stream-depth: 0
#max-open-files: 1000
#force-hash: [sha1, md5]
- lua:
enabled: no
#scripts-dir: /etc/suricata/lua-output/
scripts:
# - script1.lua
logging:
default-log-level: notice
outputs:
- console:
enabled: yes
- file:
enabled: yes
level: info
filename: suricata.log
- syslog:
enabled: no
facility: local5
format: "[%i] <%d> -- "
##
## capture settings
##
af-packet:
- interface: eth5
threads: auto
cluster-id: 99
cluster-type: cluster_flow
defrag: yes
use-mmap: yes
#mmap-locked: yes
tpacket-v3: yes
ring-size: 100000
- interface: default
#threads: auto
#use-mmap: no
#tpacket-v3: yes
##
## App Layer Protocols
##
# "yes", "no", "detection-only"
app-layer:
protocols:
rfb:
enabled: yes
detection-ports:
dp: 5900, 5901, 5902, 5903, 5904, 5905, 5906, 5907, 5908, 5909
mqtt:
enabled: yes
# max-msg-length: 1mb
# subscribe-topic-match-limit: 100
# unsubscribe-topic-match-limit: 100
krb5:
enabled: yes
snmp:
enabled: yes
ikev2:
enabled: yes
tls:
enabled: yes
detection-ports:
dp: 443
ja3-fingerprints: auto
encryption-handling: default
dcerpc:
enabled: yes
ftp:
enabled: yes
# memcap: 64mb
rdp:
enabled: yes
ssh:
enabled: yes
#hassh: yes
# HTTP2: Experimental
http2:
enabled: yes
# use http keywords on HTTP2 traffic
http1-rules: no
smtp:
enabled: yes
raw-extraction: no
mime:
decode-mime: yes
decode-base64: yes
decode-quoted-printable: yes
header-value-depth: 2000
extract-urls: yes
body-md5: no
inspected-tracker:
content-limit: 100000
content-inspect-min-size: 32768
content-inspect-window: 4096
imap:
enabled: detection-only
smb:
enabled: yes
detection-ports:
dp: 139, 445
#stream-depth: 0
nfs:
enabled: yes
tftp:
enabled: yes
dns:
tcp:
enabled: yes
detection-ports:
dp: 53
udp:
enabled: yes
detection-ports:
dp: 53
http:
enabled: yes
libhtp:
default-config:
personality: IDS
request-body-limit: 0
response-body-limit: 0
#request-body-minimal-inspect-size: 32kb
#request-body-inspect-window: 4kb
#response-body-minimal-inspect-size: 40kb
#response-body-inspect-window: 16kb
#response-body-decompress-layer-limit: 2
#http-body-inline: auto
#swf-decompression:
# enabled: yes
# type: both
# compress-depth: 100kb
# decompress-depth: 100kb
#randomize-inspection-sizes: yes
#double-decode-path: no
#double-decode-query: no
modbus:
enabled: no
detection-ports:
dp: 502
stream-depth: 0
dnp3:
enabled: yes
detection-ports:
dp: 20000
enip:
enabled: no
detection-ports:
dp: 44818
sp: 44818
ntp:
enabled: yes
dhcp:
enabled: yes
sip:
enabled: yes
asn1-max-frames: 256
##
## Advanced settings
##
#run-as:
# user: suri
# group: suri
#sensor-name: suricata
coredump:
max-dump: 0
host-mode: sniffer-only
max-pending-packets: 40000
runmode: workers
unix-command:
enabled: auto
#filename: custom.socket
magic-file: /usr/share/misc/magic
geoip-database: /etc/suricata/GeoLite2-Country.mmdb
legacy:
uricontent: enabled
engine-analysis:
rules-fast-pattern: yes
rules: yes
pcre:
match-limit: 3500
match-limit-recursion: 1500
host-os-policy:
windows: []
bsd: []
bsd-right: []
old-linux: []
linux: [0.0.0.0/0]
old-solaris: []
solaris: []
hpux10: []
hpux11: []
irix: []
macos: []
vista: []
windows2k3: []
defrag:
memcap: 32mb
hash-size: 65536
trackers: 65535
max-frags: 65535
prealloc: yes
timeout: 60
flow:
memcap: 12gb
hash-size: 262144
prealloc: 40000
emergency-recovery: 30
vlan:
use-for-tracking: true
flow-timeouts:
default:
new: 30
established: 300
closed: 0
bypassed: 100
emergency-new: 10
emergency-established: 100
emergency-closed: 0
emergency-bypassed: 50
tcp:
new: 60
established: 600
closed: 60
bypassed: 100
emergency-new: 5
emergency-established: 100
emergency-closed: 10
emergency-bypassed: 50
udp:
new: 30
established: 300
bypassed: 100
emergency-new: 10
emergency-established: 100
emergency-bypassed: 50
icmp:
new: 30
established: 300
bypassed: 100
emergency-new: 10
emergency-established: 100
emergency-bypassed: 50
stream:
memcap: 64mb
checksum-validation: yes
inline: auto
reassembly:
memcap: 256mb
depth: 10mb
toserver-chunk-size: 2560
toclient-chunk-size: 2560
randomize-chunk-size: yes
#randomize-chunk-range: 10
#raw: yes
#segment-prealloc: 2048
#check-overlap-different-data: true
host:
hash-size: 4096
prealloc: 1000
memcap: 32mb
decoder:
teredo:
enabled: true
ports: $TEREDO_PORTS
vxlan:
enabled: true
ports: $VXLAN_PORTS
vntag:
enabled: false
geneve:
enabled: true
ports: $GENEVE_PORTS
##
## Performance tuning and profiling
##
detect:
profile: high
sgh-mpm-context: full
inspection-recursion-limit: 3000
prefilter:
default: mpm
# "ac" - Aho-Corasick, default implementation
# "ac-bs" - Aho-Corasick, reduced memory implementation
# "ac-ks" - Aho-Corasick, "Ken Steele" variant
# "hs" - Hyperscan, available when built with Hyperscan support
mpm-algo: hs
# "bm" (Boyer-Moore)
# "hs" (Hyperscan, only available if Suricata has been built with Hyperscan support)
spm-algo: auto
threading:
set-cpu-affinity: yes
cpu-affinity:
- management-cpu-set:
cpu: [ 2, 3, 4, 5 ]
- receive-cpu-set:
cpu: [ 0, 1 ]
- worker-cpu-set:
cpu: [ "6-29" ]
mode: "exclusive"
prio:
low: [ 0, 1 ]
medium: [ 2, 3, 4, 5 ]
high: [ "6-29" ]
default: "high"
detect-thread-ratio: 1.0
stack-size: 8mb
luajit:
states: 128
##
## Advanced Capture Options
##
capture:
disable-offloading: true
##
## Config files
##
default-rule-path: /var/lib/suricata/rules
rule-files:
- suricata.rules
- /etc/suricata/local.rules
classification-file: /etc/suricata/classification.config
reference-config-file: /etc/suricata/reference.config
threshold-file: /etc/suricata/threshold.config
Does anyone have an idea what is going wrong here? Also happy to share general ideas and optimizations regarding the config, many thanks!
