Testing IPS fails

Help
1

Hi, I’m running suricata 6.0.4 on Ubuntu 22 in IPS mode (via iptables, NFQEUE). It reports that 34087 rules successfully loaded, 0 rules failed. One rule is goip-Blocking which I can verify that it is correctly applied and working (blocks also appear in the fast.log). Unfortunately for the other rules I have no clue whether they are working in reality. Using an eicar test (I manually included a rule for that) did not block any download. The Blacksun-Test is not recognized/reported (aka curl -A "BlackSun" www.google.com). The textmynids test is not blocked or reported either curl --max-time 5 http://testmynids.org/uid/index.html

So, I have a system that works somehow, but I have no proof besides geoip-blocking that it really works. What can I do? Thanks for your suppport :slight_smile:

2

Hi,

please share your config and how you start Suricata, ideally also a bit more details about the setup. Did you also ensure the other rules have changed from action alert to drop?

3

Drop works well for connections coming in from WAN (I have many drops reported in fast.log with priority 2 and 3). Also GeoIP blocking works very well. So the general setup seems to work. I use suricata via the inline-mode (Chain INPUT, OUTPUT, FORWARD each with target NFQUEUE as indicated in the official manual).

4

Can you post the output of grep 2100498 on your signature rule file?
And also your suricata.yaml to check for potential issues?
In additon to that, do you see the event for the curl on testmynids in the eve json output (for example in event type http)?

5

Hello,
grep →

drop ip any any → any any (msg:“GPL ATTACK_RESPONSE id check returned root”; content:“uid=0|28|root|29|”; classtype:bad-unknown; sid:2100498; rev:7; metadata:created_at 2010_09_23, updated_at 2010_09_23;)

All logs besides fast.log are empty.

iptables -L:

Chain INPUT (policy ACCEPT)
target prot opt source destination
NFQUEUE tcp – anywhere anywhere multiport dports ssh,http,https,openvpn,1195,2222 NFQUEUE num 0

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
NFQUEUE tcp – anywhere anywhere multiport dports ssh,http,https,openvpn,1195,2222 NFQUEUE num 0

Suricata.yaml:

vars:
address-groups:
HOME_NET: “[88.99.xx.xx/24]”
EXTERNAL_NET: “!$HOME_NET”
HTTP_SERVERS: “$HOME_NET”
SMTP_SERVERS: “$HOME_NET”
SQL_SERVERS: “$HOME_NET”
DNS_SERVERS: “$HOME_NET”
TELNET_SERVERS: “$HOME_NET”
AIM_SERVERS: “$EXTERNAL_NET”
DC_SERVERS: “$HOME_NET”
DNP3_SERVER: “$HOME_NET”
DNP3_CLIENT: “$HOME_NET”
MODBUS_CLIENT: “$HOME_NET”
MODBUS_SERVER: “$HOME_NET”
ENIP_CLIENT: “$HOME_NET”
ENIP_SERVER: “$HOME_NET”
port-groups:
HTTP_PORTS: “80”
SHELLCODE_PORTS: “!80”
ORACLE_PORTS: 1521
SSH_PORTS: 22
DNP3_PORTS: 20000
MODBUS_PORTS: 502
FILE_DATA_PORTS: “[$HTTP_PORTS,110,143]”
FTP_PORTS: 21
GENEVE_PORTS: 6081
VXLAN_PORTS: 4789
TEREDO_PORTS: 3544
default-log-dir: /var/log/suricata/
stats:
enabled: yes
interval: 8
outputs:

  • fast:
    enabled: yes
    filename: fast.log
    append: yes
  • eve-log:
    enabled: no
    filename: eve.json
    pcap-file: false
    community-id: false
    community-id-seed: 0
    xff:
    enabled: no
    mode: extra-data
    deployment: reverse
    header: X-Forwarded-For
    types:
    - alert:
    tagged-packets: yes
    - anomaly:
    enabled: yes
    types:
    - http:
    - dns:
    - tls:
    - files:
    - smtp:
    - ftp
    - rdp
    - nfs
    - smb
    - tftp
    - ikev2
    - dcerpc
    - krb5
    - snmp
    - rfb
    - sip
    - dhcp:
    enabled: yes
    extended: no
    - ssh
    - mqtt:
    - stats:
    - flow
  • http-log:
    enabled: no
    filename: http.log
    append: yes
  • tls-log:
    append: yes
  • tls-store:
    enabled: no
  • pcap-log:
    enabled: no
    filename: log.pcap
    limit: 1000mb
    max-files: 2000
    compression: none
  • alert-debug:
    enabled: no
    filename: alert-debug.log
    append: yes
  • alert-prelude:
    enabled: no
    profile: suricata
    log-packet-content: no
    log-packet-header: yes
  • stats:
    enabled: no
    filename: stats.log
  • syslog:
    enabled: no
    facility: local5
  • file-store:
    version: 2
    enabled: no
    xff:
    enabled: no
    mode: extra-data
    deployment: reverse
    header: X-Forwarded-For
  • tcp-data:
    enabled: no
    type: file
    filename: tcp-data.log
  • http-body-data:
    enabled: no
    type: file
    filename: http-data.log
  • lua:
    enabled: no
    scripts:
    logging:
    default-log-level: notice
    default-output-filter:
    outputs:
  • console:
    enabled: yes
  • file:
    enabled: yes
    level: info
    filename: suricata.log
  • syslog:
    enabled: no
    facility: local5
    format: "[%i] <%d> – "
    af-packet:
  • interface: eth0
    cluster-id: 99
    cluster-type: cluster_flow
    defrag: yes
  • interface: default
    pcap:
  • interface: eth0
  • interface: default
    pcap-file:
    checksum-checks: auto
    app-layer:
    protocols:
    rfb:
    enabled: yes
    detection-ports:
    dp: 5900, 5901, 5902, 5903, 5904, 5905, 5906, 5907, 5908, 5909
    mqtt:
    krb5:
    enabled: yes
    snmp:
    enabled: yes
    ikev2:
    enabled: yes
    tls:
    enabled: yes
    detection-ports:
    dp: 443
    dcerpc:
    enabled: yes
    ftp:
    enabled: yes
    rdp:
    ssh:
    enabled: yes
    http2:
    enabled: no
    http1-rules: no
    smtp:
    enabled: yes
    raw-extraction: no
    mime:
    decode-mime: yes
    decode-base64: yes
    decode-quoted-printable: yes
    header-value-depth: 2000
    extract-urls: yes
    body-md5: no
    inspected-tracker:
    content-limit: 100000
    content-inspect-min-size: 32768
    content-inspect-window: 4096
    imap:
    enabled: detection-only
    smb:
    enabled: yes
    detection-ports:
    dp: 139, 445
    nfs:
    enabled: yes
    tftp:
    enabled: yes
    dns:
    tcp:
    enabled: yes
    detection-ports:
    dp: 53
    udp:
    enabled: yes
    detection-ports:
    dp: 53
    http:
    enabled: yes
    libhtp:
    default-config:
    personality: IDS
    request-body-limit: 100kb
    response-body-limit: 100kb
    request-body-minimal-inspect-size: 32kb
    request-body-inspect-window: 4kb
    response-body-minimal-inspect-size: 40kb
    response-body-inspect-window: 16kb
    response-body-decompress-layer-limit: 2
    http-body-inline: auto
    swf-decompression:
    enabled: yes
    type: both
    compress-depth: 100kb
    decompress-depth: 100kb
    double-decode-path: no
    double-decode-query: no
    server-config:
    modbus:
    enabled: no
    detection-ports:
    dp: 502
    stream-depth: 0
    dnp3:
    enabled: no
    detection-ports:
    dp: 20000
    enip:
    enabled: no
    detection-ports:
    dp: 44818
    sp: 44818
    ntp:
    enabled: yes
    dhcp:
    enabled: yes
    sip:
    asn1-max-frames: 256
    coredump:
    max-dump: unlimited
    host-mode: auto
    unix-command:
    enabled: yes
    filename: /var/run/suricata-command.socket
    geoip-database: /var/lib/GeoIP/GeoLite2-Country.mmdb
    legacy:
    uricontent: enabled
    engine-analysis:
    rules-fast-pattern: yes
    rules: yes
    pcre:
    match-limit: 3500
    match-limit-recursion: 1500
    host-os-policy:
    windows: [0.0.0.0/0]
    bsd:
    bsd-right:
    old-linux:
    linux:
    old-solaris:
    solaris:
    hpux10:
    hpux11:
    irix:
    macos:
    vista:
    windows2k3:
    defrag:
    memcap: 32mb
    hash-size: 65536
    prealloc: yes
    timeout: 60
    flow:
    memcap: 128mb
    hash-size: 65536
    prealloc: 10000
    emergency-recovery: 30
    vlan:
    use-for-tracking: true
    flow-timeouts:
    default:
    new: 30
    established: 300
    closed: 0
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-closed: 0
    emergency-bypassed: 50
    tcp:
    new: 60
    established: 600
    closed: 60
    bypassed: 100
    emergency-new: 5
    emergency-established: 100
    emergency-closed: 10
    emergency-bypassed: 50
    udp:
    new: 30
    established: 300
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-bypassed: 50
    icmp:
    new: 30
    established: 300
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-bypassed: 50
    memcap: 64mb
    reassembly:
    memcap: 256mb
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes
    host:
    hash-size: 4096
    prealloc: 1000
    memcap: 32mb
    decoder:
    teredo:
    enabled: true
    vxlan:
    enabled: true
    vntag:
    enabled: false
    geneve:
    enabled: true
    detect:
    profile: medium
    custom-values:
    toclient-groups: 3
    toserver-groups: 25
    sgh-mpm-context: auto
    inspection-recursion-limit: 3000
    prefilter:
    default: mpm
    grouping:
    profiling:
    grouping:
    dump-to-disk: false
    include-mpm-stats: false
    mpm-algo: auto
    spm-algo: auto
    threading:
    set-cpu-affinity: no
    cpu-affinity:
    • management-cpu-set:
    • receive-cpu-set:
    • worker-cpu-set:
      cpu: [ “all” ]
      mode: “exclusive”
      prio:
      low: [ 0 ]
      medium: [ “1-2” ]
      high: [ 3 ]
      default: “medium”
      detect-thread-ratio: 1.0
      luajit:
      states: 128
      profiling:
      rules:
      enabled: yes
      filename: rule_perf.log
      append: yes
      limit: 10
      json: yes
      keywords:
      enabled: yes
      filename: keyword_perf.log
      append: yes
      prefilter:
      enabled: yes
      filename: prefilter_perf.log
      append: yes
      rulegroups:
      enabled: yes
      filename: rule_group_perf.log
      append: yes
      packets:
      enabled: yes
      filename: packet_stats.log
      append: yes
      csv:
      enabled: no
      filename: packet_stats.csv
      locks:
      enabled: no
      filename: lock_stats.log
      append: yes
      pcap-log:
      enabled: no
      filename: pcaplog_stats.log
      append: yes
      nfq:
      mode: accept
      nflog:
  • group: 2
    buffer-size: 18432
  • group: default
    qthreshold: 1
    qtimeout: 100
    max-size: 20000
    capture:
    netmap:
  • interface: eth2
  • interface: default
    pfring:
  • interface: eth0
    threads: auto
    cluster-id: 99
    cluster-type: cluster_flow
  • interface: default
    ipfw:
    napatech:
    streams: [“0-3”]
    enable-stream-stats: no
    auto-config: yes
    hardware-bypass: yes
    inline: no
    ports: [0-1,2-3]
    hashmode: hash5tuplesorted
    default-rule-path: /
    rule-files:
  • /etc/suricata/rules/geoip.rules
  • /var/lib/suricata/rules/suricata.rules
    classification-file: /etc/suricata/classification.config
    reference-config-file: /etc/suricata/reference.config
6

I would recommend to enable stats.log to check if there are any other issues. In addition to that, also enable eve log with the flow type and ideally httpas well and with that you can check if the http traffic for the tests is actually seen by Suricata.

7

Changes:

# Global stats configuration
stats:
  enabled: yes
  # The interval field (in seconds) controls the interval at
  # which stats are updated in the log.
  interval: 8
  # Add decode events to stats.

  - eve-log:
      enabled: yes
      filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
      filename: eve.json
      # Enable for multi-threaded eve.json output; output files are amended with

 - http-log:
      enabled: yes
      filename: http.log
      append: yes

When I do the curl http://testmynids.org/uid/index.html I see no notification, nowhere. fast.log is populated with a lot of other events, like Drops for certain GeoIPs, ETN AGGRESSIVE IPs Group etc. So it somehow works, but not for all tests e.g. available via GitHub - 3CORESec/testmynids.org: A website and framework for testing NIDS detection. From this only ET SCAN Potential SSH Scan OUTBOUND is detected.

8

Can you post your stats.log and some examples of fast.log?
Since other events are seen, the basic setup seems to be okay.
Try to capture the traffic if it actually arrives as expected.

9

The first problem we have to solve seems to be that stats.log is not populated at all. I have no idea why.

suricata --dump-config|grep -e log-dir -e stats.log:

default-log-dir = /var/log/suricata/
outputs.8.stats.filename = stats.log
profiling.packets.filename = packet_stats.log
profiling.locks.filename = lock_stats.log
profiling.pcap-log.filename = pcaplog_stats.log

Any idea? Reboot doesn’t help.

Here is an excerp of fast.log:

01/18/2023-10:51:29.622745  [Drop] [**] [1:5000165:1673919002] ETN AGGRESSIVE IPs Group 165 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 185.122.204.26:54428 -> 46.38...:22
01/18/2023-10:51:29.622745  [Drop] [**] [1:15555555:1] GeoIP [**] [Classification: (null)] [Priority: 3] {TCP} 185.122.204.26:54428 -> 46.38....:22
01/18/2023-11:02:11.756879  [Drop] [**] [1:2525019:650] ET 3CORESec Poor Reputation IP group 20 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 179.43.154.248:56646 -> 46.38....:443
01/18/2023-11:02:11.756879  [Drop] [**] [1:5000152:1673919002] ETN AGGRESSIVE IPs Group 152 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 179.43.154.248:56646 -> 46.38....:443
01/18/2023-11:02:11.756879  [Drop] [**] [1:5000314:1673919002] ETN AGGRESSIVE IPs Group 314 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 179.43.154.248:56646 -> 46.38....:443
01/18/2023-11:02:11.756879  [Drop] [**] [1:5000475:1673919002] ETN AGGRESSIVE IPs Group 475 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 179.43.154.248:56646 -> 46.38....:443
01/18/2023-11:02:11.756879  [Drop] [**] [1:5000637:1673919002] ETN AGGRESSIVE IPs Group 637 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 179.43.154.248:56646 -> 46.38....:443
01/18/2023-11:02:11.756879  [Drop] [**] [1:15555555:1] GeoIP [**] [Classification: (null)] [Priority: 3] {TCP} 179.43.154.248:56646 -> 46.38....:443
10

In your initial post I see it not being enabled:

stats:
enabled: no
filename: stats.log

So you should enable this to get a stats.log

11

stats.log

------------------------------------------------------------------------------------
Date: 1/18/2023 -- 16:15:04 (uptime: 0d, 00h 05m 48s)
------------------------------------------------------------------------------------
Counter                                       | TM Name                   | Value
------------------------------------------------------------------------------------
decoder.pkts                                  | Total                     | 1444
decoder.bytes                                 | Total                     | 246672
decoder.ipv4                                  | Total                     | 1444
decoder.tcp                                   | Total                     | 1444
decoder.avg_pkt_size                          | Total                     | 170
decoder.max_pkt_size                          | Total                     | 1296
flow.tcp                                      | Total                     | 25
flow.wrk.spare_sync_avg                       | Total                     | 100
flow.wrk.spare_sync                           | Total                     | 1
flow.wrk.flows_evicted                        | Total                     | 1
tcp.sessions                                  | Total                     | 21
tcp.syn                                       | Total                     | 22
tcp.rst                                       | Total                     | 6
detect.alert                                  | Total                     | 8
ips.accepted                                  | Total                     | 1434
ips.blocked                                   | Total                     | 10
flow.mgr.full_hash_pass                       | Total                     | 2
flow.spare                                    | Total                     | 9912
flow.mgr.rows_maxlen                          | Total                     | 1
flow.mgr.flows_checked                        | Total                     | 18
flow.mgr.flows_notimeout                      | Total                     | 6
flow.mgr.flows_timeout                        | Total                     | 12
flow.mgr.flows_evicted                        | Total                     | 12
tcp.memuse                                    | Total                     | 606208
tcp.reassembly_memuse                         | Total                     | 98304
flow.memuse                                   | Total                     | 7394304
12

Looks okay, also enable eve-log and check if the flow appears that is doing the actual curl command. We need to make sure first, that Suricata is actually seeing this traffic that you want the rule to trigger.

13

eve.log enabled as far as I can see. At least eve.json is constantly populated.

14

In that case look for event_type flow and see if you can spot the src ip with the curl request

15

I can spot a bunch of such event types, my own IP address is the destination, source varying. But nothing that I would relate to the curl request. Especially nothing on port 80 where the curl request ist operating.

{"timestamp":"2023-01-18T17:41:46.897848+0100","flow_id":1412213710802263,"event_type":"flow","src_ip":"80.187.102.243","src_port":28424,"dest_ip":"46.38....","dest_port":22,"proto":"TCP","flow":{"pkts_toserver":2,"pkts_toclient":0,"bytes_toserver":256,"bytes_toclient":0,"start":"2023-01-18T17:38:44.937303+0100","end":"2023-01-18T17:38:45.041722+0100","age":1,"state":"new","reason":"timeout","alerted":false},"tcp":{"tcp_flags":"00","tcp_flags_ts":"00","tcp_flags_tc":"00"}}
{"timestamp":"2023-01-18T17:41:46.951724+0100","flow_id":949203351421447,"event_type":"flow","src_ip":"80.187.102.243","src_port":28423,"dest_ip":"46.38....","dest_port":22,"proto":"TCP","flow":{"pkts_toserver":2,"pkts_toclient":0,"bytes_toserver":256,"bytes_toclient":0,"start":"2023-01-18T17:38:44.967175+0100","end":"2023-01-18T17:38:45.074949+0100","age":1,"state":"new","reason":"timeout","alerted":false},"tcp":{"tcp_flags":"00","tcp_flags_ts":"00","tcp_flags_tc":"00"}}
16

If you run tcpdump on the interface an do the curl on the client, do you see the curl?

17

Yes I see it, both in IPv4 and IPv6 (standard is v6):

root@v2202108154998161237:~# tcpdump -i eth0 port 80
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:45:18.330533 IP6 v2202108154998161237.40040 > 2600:9000:2250:c400:18:30b3:e400:93a1.http: Flags [S], seq 960792398, win 43200, options [mss 1440,sackOK,TS val 2578107577 ecr 0,nop,wscale 11], length 0
15:45:18.334497 IP6 2600:9000:2250:c400:18:30b3:e400:93a1.http > v2202108154998161237.40040: Flags [S.], seq 4048576697, ack 960792399, win 65535, options [mss 1220,sackOK,TS val 148081590 ecr 2578107577,nop,wscale 9], length 0
15:45:18.334602 IP6 v2202108154998161237.40040 > 2600:9000:2250:c400:18:30b3:e400:93a1.http: Flags [.], ack 1, win 22, options [nop,nop,TS val 2578107582 ecr 148081590], length 0
15:45:18.334891 IP6 v2202108154998161237.40040 > 2600:9000:2250:c400:18:30b3:e400:93a1.http: Flags [P.], seq 1:93, ack 1, win 22, options [nop,nop,TS val 2578107582 ecr 148081590], length 92: HTTP: GET /uid/index.html HTTP/1.1
15:45:18.338897 IP6 2600:9000:2250:c400:18:30b3:e400:93a1.http > v2202108154998161237.40040: Flags [.], ack 93, win 128, options [nop,nop,TS val 148081594 ecr 2578107582], length 0
15:45:18.339685 IP6 2600:9000:2250:c400:18:30b3:e400:93a1.http > v2202108154998161237.40040: Flags [P.], seq 1:500, ack 93, win 128, options [nop,nop,TS val 148081595 ecr 2578107582], length 499: HTTP: HTTP/1.1 200 OK
15:45:18.339686 IP6 2600:9000:2250:c400:18:30b3:e400:93a1.http > v2202108154998161237.40040: Flags [P.], seq 500:539, ack 93, win 128, options [nop,nop,TS val 148081595 ecr 2578107582], length 39: HTTP
15:45:18.339742 IP6 v2202108154998161237.40040 > 2600:9000:2250:c400:18:30b3:e400:93a1.http: Flags [.], ack 500, win 22, options [nop,nop,TS val 2578107587 ecr 148081595], length 0
15:45:18.339770 IP6 v2202108154998161237.40040 > 2600:9000:2250:c400:18:30b3:e400:93a1.http: Flags [.], ack 539, win 22, options [nop,nop,TS val 2578107587 ecr 148081595], length 0
15:45:18.340492 IP6 v2202108154998161237.40040 > 2600:9000:2250:c400:18:30b3:e400:93a1.http: Flags [F.], seq 93, ack 539, win 22, options [nop,nop,TS val 2578107587 ecr 148081595], length 0
15:45:18.344401 IP6 2600:9000:2250:c400:18:30b3:e400:93a1.http > v2202108154998161237.40040: Flags [F.], seq 539, ack 94, win 128, options [nop,nop,TS val 148081600 ecr 2578107587], length 0
15:45:18.344466 IP6 v2202108154998161237.40040 > 2600:9000:2250:c400:18:30b3:e400:93a1.http: Flags [.], ack 540, win 22, options [nop,nop,TS val 2578107591 ecr 148081600], length 0
18

No idea how to approach this?

19

You could add -j LOG targets before the jump into the NFQUEUE to see, if netfilter actually passes the packets to Suricata

20

I added -j LOG to iptables at the very first position. Syslog then shows the curl:

Jan 24 15:40:02 v2202108154998161237 kernel: [516702.979077] IN= OUT=eth0 SRC=46.38… DST=18.66.122.20 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=42622 DF PROTO=TCP SPT=53980 DPT=80 WINDOW=42340 RES=0x00 SYN URGP=0

What appears strange to me is that http-log is enabled, but the http.log is empty:

 - http-log:
      enabled: yes
      filename: http.log
      append: yes