When adding the protocol file in version 7.0.6, an error occurred and there was no output

1

version:7.0.6
system： ubuntu22.04

describe:
I try to add s7comm

 ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var

 python3 scripts/setup-app-layer.py --detect --logger --parser S7comm S7commbuf

After running the above command, a file reported an error

root@lxx:~/7.6/suricata-7.0.6# python3 scripts/setup-app-layer.py --detect --logger --parser S7comm S7commbuf
Creating directory rust/src/applayers7comm.
Generating rust/src/applayers7comm/mod.rs.
Formatting rust/src/applayers7comm/mod.rs
Error writing files: failed to resolve mod `parser`: /root/7.6/suricata-7.0.6/rust/src/applayers7comm/parser.rs does not exist
Generating rust/src/applayers7comm/s7comm.rs.

after, modify suricata.yaml  

app-layer:
  # error-policy: ignore
  protocols:
    s7comm:
      enabled: yes
      detection-ports:
        dp: 102

after,/root/7.6/suricata7.0.6/rust/src/applayers7comm document modify

#[no_mangle]
pub unsafe extern "C" fn rs_s7comm_register_parser() {
    let default_port = CString::new("[102]").unwrap();
    let parser = RustParser {
        name: PARSER_NAME.as_ptr() as *const c_char,
        default_port: default_port.as_ptr(),
        ipproto: IPPROTO_TCP,

make
make install
make install-conf
suricata -c /etc/suricata/suricata.yaml -r dump.pcapng -vv

But I don’t see any output about s7comm, println!(“HELLO”) also has no output in rs_s7comm_parse_request function

I hope everyone can give me some advice,thanks

2

Hello there, welcome to the Suricata forum, and thanks for the details you’ve provided.

Starting with the first issue you’ve presented:

Error writing files: failed to resolve mod `parser`: /root/7.6/suricata-7.0.6/rust/src/applayers7comm/parser.rs does not exist

The setup-app-layer script can be used for generating boilerplate code to kickstart adding a new application layer protocol to Suricata. By default, it will create the parser and the logger.

Once those are done, then you can run it once again with the --detect and buffer argument, to generate code for having a keyword for the new parser.

So, that error happens because the script was trying to add a new keyword buffer, but the parser, indeed, doesn’t exist yet.

Now, for the second part of your post.

As per the first error, I assume that this can be caused because there’s no underlying parser code created.

Even if there is, generating the boilerplate code is but the first step. As each application layer protocol will have its own particular definitions, once the template code was created, it is necessary to tailor all created functions to understand and actually parse the protocol you want to add to Suricata.

So, my suggestion to confirm these answers for your case would be:

  • delete the code that was generated by the script
  • run setup-app-layer.py s7comm first - that will generate code for the parser and the logger
  • run setup-app-layer.py python3 scripts/setup-app-layer.py --detect S7comm S7commbuf-
  • edit the registration function
  • run ./configure
  • run make
  • edit the suricata.yaml file to enable your proto
  • run Suricata again
  • check output

As I’ve explained, I expect that more tailoring the generated code to understand S7comm traffic. But these steps would be a good start, to see where you stand.

There’s a presentation from a past SuriCon where I go briefly over those steps, for adding the STUN protocol, in case you’re interested: https://youtu.be/PKE9WqvRkHE?si=pL__FJmtm6GeMIWF

Hope that helps!

1 Like
3

version:7.0.6
system： ubuntu22.04

The problem still exists.

root@lxx:~/7.6/suricata-7.0.6# python3 scripts/setup-app-layer.py  S7comm
Creating directory rust/src/applayers7comm.
Generating rust/src/applayers7comm/mod.rs.
Formatting rust/src/applayers7comm/mod.rs
Error writing files: failed to resolve mod `parser`: /root/7.6/suricata-7.0.6/rust/src/applayers7comm/parser.rs does not exist
Generating rust/src/applayers7comm/s7comm.rs.
Formatting rust/src/applayers7comm/s7comm.rs
Generating rust/src/applayers7comm/parser.rs.
Formatting rust/src/applayers7comm/parser.rs
Patching rust/src/lib.rs.
Patching src/app-layer-protos.h.
Patching src/app-layer-protos.c.
Patching src/app-layer-parser.c.
Patching suricata.yaml.in.
Generating src/output-json-s7comm.h.
Generating src/output-json-s7comm.c.
Generating rust/src/applayers7comm/logger.rs.
Formatting rust/src/applayers7comm/logger.rs
Patching rust/src/applayers7comm/mod.rs.
Formatting rust/src/applayers7comm/mod.rs
Patching src/Makefile.am.
Patching src/output.c.
Patching suricata.yaml.in.

An application detector and parser for the protocol S7comm have
now been setup in the files:

    rust/src/applayers7comm/mod.rs
    rust/src/applayers7comm/parser.rs

A JSON application layer transaction logger for the protocol
S7comm has now been set in the file:

    rust/src/applayers7comm/logger.rs

Suricata should now build cleanly. Try running "./configure" and "make".

root@lxx:~/7.6/suricata-7.0.6# python3 scripts/setup-app-layer.py  --detect S7comm S7commbuf
Generating src/detect-s7comm-s7commbuf.h.
Generating src/detect-s7comm-s7commbuf.c.
Patching src/Makefile.am.
Patching src/detect-engine-register.c.
Patching src/detect-engine-register.h.

The following files have been created and linked into the build:

    detect-s7comm-s7commbuf.h
    detect-s7comm-s7commbuf.c

./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var
#s7comm.rs modify port 102
#modify suricata.yaml 
make
make install

The problem still exists.

I am unable to get any output in s7comm.rs, but the pcap file is being read.

Even a simple ‘Hello World’ cannot be output.

root@lxx:~/7.6/suricata-7.0.6# suricata -c /etc/suricata/suricata.yaml -r dump.pcapng -vvv
[65198] Notice: suricata: This is Suricata version 7.0.5 RELEASE running in USER mode
[65198] Info: cpu: CPUs/cores online: 4
[65198] Info: suricata: Setting engine mode to IDS mode by default
[65198] Info: exception-policy: master exception-policy set to: auto
[65198] Config: exception-policy: app-layer.error-policy: ignore
[65198] Config: app-layer-htp: 'default' server has 'request-body-minimal-inspect-size' set to 31186 and 'request-body-inspect-window' set to 4158 after randomization.
[65198] Config: app-layer-htp: 'default' server has 'response-body-minimal-inspect-size' set to 40047 and 'response-body-inspect-window' set to 16192 after randomization.
[65198] Config: smb: read: max record size: 16777216, max queued chunks 64, max queued size 67108864
[65198] Config: smb: write: max record size: 16777216, max queued chunks 64, max queued size 67108864
[65198] Config: app-layer-enip: Protocol detection and parser disabled for enip protocol.
[65198] Config: app-layer-dnp3: Protocol detection and parser disabled for DNP3.
[65198] Notice: s7comm: Rust s7comm parser registered.
[65198] Notice: output-json-s7comm: S7comm JSON logger registered.
[65198] Config: host: allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
[65198] Config: host: preallocated 1000 hosts of size 136
[65198] Config: host: host memory usage: 398144 bytes, maximum: 33554432
[65198] Config: coredump-config: Core dump size set to unlimited.
[65198] Config: exception-policy: defrag.memcap-policy: ignore (defined via 'exception-policy' master switch)
[65198] Config: defrag-hash: allocated 3670016 bytes of memory for the defrag hash... 65536 buckets of size 56
[65198] Config: defrag-hash: preallocated 65535 defrag trackers of size 160
[65198] Config: defrag-hash: defrag memory usage: 14155616 bytes, maximum: 33554432
[65198] Config: exception-policy: flow.memcap-policy: ignore (defined via 'exception-policy' master switch)
[65198] Config: flow: flow size 296, memcap allows for 453438 flows. Per hash row in perfect conditions 6
[65198] Config: stream-tcp: stream "prealloc-sessions": 2048 (per thread)
[65198] Config: stream-tcp: stream "memcap": 67108864
[65198] Config: stream-tcp: stream "midstream" session pickups: disabled
[65198] Config: stream-tcp: stream "async-oneside": disabled
[65198] Config: stream-tcp: stream "checksum-validation": enabled
[65198] Config: exception-policy: stream.memcap-policy: ignore (defined via 'exception-policy' master switch)
[65198] Config: exception-policy: stream.reassembly.memcap-policy: ignore (defined via 'exception-policy' master switch)
[65198] Config: exception-policy: stream.midstream-policy: ignore (defined via 'exception-policy' master switch)
[65198] Config: stream-tcp: stream."inline": disabled
[65198] Config: stream-tcp: stream "bypass": disabled
[65198] Config: stream-tcp: stream "max-syn-queued": 10
[65198] Config: stream-tcp: stream "max-synack-queued": 5
[65198] Config: stream-tcp: stream.reassembly "memcap": 268435456
[65198] Config: stream-tcp: stream.reassembly "depth": 1048576
[65198] Config: stream-tcp: stream.reassembly "toserver-chunk-size": 2639
[65198] Config: stream-tcp: stream.reassembly "toclient-chunk-size": 2582
[65198] Config: stream-tcp: stream.reassembly.raw: enabled
[65198] Config: stream-tcp: stream.liberal-timestamps: disabled
[65198] Config: stream-tcp-reassemble: stream.reassembly "segment-prealloc": 2048
[65198] Config: stream-tcp-reassemble: stream.reassembly "max-regions": 8
[65198] Info: logopenfile: fast output device (regular) initialized: fast.log
[65198] Info: logopenfile: eve-log output device (regular) initialized: eve.json
[65198] Config: runmodes: enabling 'eve-log' module 's7comm'
[65198] Notice: output-json-s7comm: S7comm log sub-module initialized.
[65198] Config: runmodes: enabling 'eve-log' module 'alert'
[65198] Config: runmodes: enabling 'eve-log' module 'frame'
[65198] Config: runmodes: enabling 'eve-log' module 'anomaly'
[65198] Config: runmodes: enabling 'eve-log' module 'http'
[65198] Config: runmodes: enabling 'eve-log' module 'dns'
[65198] Config: runmodes: enabling 'eve-log' module 'tls'
[65198] Config: runmodes: enabling 'eve-log' module 'files'
[65198] Config: runmodes: enabling 'eve-log' module 'smtp'
[65198] Config: runmodes: enabling 'eve-log' module 'ftp'
[65198] Config: runmodes: enabling 'eve-log' module 'rdp'
[65198] Config: runmodes: enabling 'eve-log' module 'nfs'
[65198] Config: runmodes: enabling 'eve-log' module 'smb'
[65198] Config: runmodes: enabling 'eve-log' module 'tftp'
[65198] Config: runmodes: enabling 'eve-log' module 'ike'
[65198] Config: runmodes: enabling 'eve-log' module 'dcerpc'
[65198] Config: runmodes: enabling 'eve-log' module 'krb5'
[65198] Config: runmodes: enabling 'eve-log' module 'bittorrent-dht'
[65198] Config: runmodes: enabling 'eve-log' module 'snmp'
[65198] Config: runmodes: enabling 'eve-log' module 's7comm'
[65198] Notice: output-json-s7comm: S7comm log sub-module initialized.
[65198] Config: runmodes: enabling 'eve-log' module 'rfb'
[65198] Config: runmodes: enabling 'eve-log' module 'sip'
[65198] Config: runmodes: enabling 'eve-log' module 'quic'
[65198] Config: runmodes: enabling 'eve-log' module 'dhcp'
[65198] Config: runmodes: enabling 'eve-log' module 'ssh'
[65198] Config: runmodes: enabling 'eve-log' module 'mqtt'
[65198] Config: runmodes: enabling 'eve-log' module 'http2'
[65198] Config: runmodes: enabling 'eve-log' module 'pgsql'
[65198] Config: runmodes: enabling 'eve-log' module 'stats'
[65198] Config: runmodes: enabling 'eve-log' module 'flow'
[65198] Info: logopenfile: stats output device (regular) initialized: stats.log
[65198] Config: landlock: Landlock is not enabled in configuration
[65198] Config: suricata: Delayed detect disabled
[65198] Config: detect: pattern matchers: MPM: ac, SPM: bm
[65198] Config: detect: grouping: tcp-whitelist (default) 53, 80, 139, 443, 445, 1433, 3306, 3389, 6666, 6667, 8080
[65198] Config: detect: grouping: udp-whitelist (default) 53, 135, 5060
[65198] Config: detect: prefilter engines: MPM
[65198] Config: reputation: IP reputation disabled
[65198] Config: detect: Loading rule file: /etc/suricata/rules/my.rules
[65198] Info: detect: 1 rule files processed. 1 rules successfully loaded, 0 rules failed, 0
[65198] Info: threshold-config: Threshold config parsed: 0 rule(s) found
[65198] Info: detect: 1 signatures processed. 0 are IP-only rules, 1 are inspecting packet payload, 0 inspect application layer, 0 are decoder event only
[65198] Config: detect: building signature grouping structure, stage 1: preprocessing rules... complete
[65198] Perf: detect: TCP toserver: 1 port groups, 1 unique SGH's, 0 copies
[65198] Perf: detect: TCP toclient: 0 port groups, 0 unique SGH's, 0 copies
[65198] Perf: detect: UDP toserver: 0 port groups, 0 unique SGH's, 0 copies
[65198] Perf: detect: UDP toclient: 0 port groups, 0 unique SGH's, 0 copies
[65198] Perf: detect: OTHER toserver: 0 proto groups, 0 unique SGH's, 0 copies
[65198] Perf: detect: OTHER toclient: 0 proto groups, 0 unique SGH's, 0 copies
[65198] Perf: detect: Unique rule groups: 1
[65198] Perf: detect: Builtin MPM "toserver TCP packet": 1
[65198] Perf: detect: Builtin MPM "toclient TCP packet": 0
[65198] Perf: detect: Builtin MPM "toserver TCP stream": 1
[65198] Perf: detect: Builtin MPM "toclient TCP stream": 0
[65198] Perf: detect: Builtin MPM "toserver UDP packet": 0
[65198] Perf: detect: Builtin MPM "toclient UDP packet": 0
[65198] Perf: detect: Builtin MPM "other IP packet": 0
[65198] Config: tmqh-flow: AutoFP mode using "Hash" flow load balancer
[65198] Config: flow-manager: using 1 flow manager threads
[65198] Config: flow-manager: using 1 flow recycler threads
[65204] Info: pcap: Starting file run for dump.pcapng
[65204] Info: pcap: pcap file dump.pcapng end of file reached (pcap err code 0)
[65198] Notice: threads: Threads created -> RX: 1 W: 4 FM: 1 FR: 1   Engine started.
[65198] Notice: suricata: Signal Received.  Stopping engine.
[65198] Info: suricata: time elapsed 0.118s
[65210] Perf: flow-manager: 11 flows processed
[65204] Notice: pcap: read 1 file, 200 packets, 14388 bytes
[65198] Perf: tmqh-flow: AutoFP - Total flow handler queues - 4
[65198] Info: counters: Alerts: 0
[65198] Perf: ippair: ippair memory usage: 414144 bytes, maximum: 16777216
[65198] Perf: host: host memory usage: 398144 bytes, maximum: 33554432
root@lxx:~/7.6/suricata-7.0.6# suricata -c /etc/suricata/suricata.yaml -r dump.pcapng -vvvvv
Warning: Invalid/No global_log_level assigned by user.  Falling back on the default_log_level "Info"
Notice: suricata: This is Suricata version 7.0.5 RELEASE running in USER mode [LogVersion:suricata.c:1146]
Info: cpu: CPUs/cores online: 4 [UtilCpuPrintSummary:util-cpu.c:182]
Info: suricata: Setting engine mode to IDS mode by default [PostConfLoadedSetup:suricata.c:2682]
Info: exception-policy: master exception-policy set to: auto [ExceptionPolicyMasterParse:util-exception-policy.c:200]
Notice: s7comm: Rust s7comm parser registered. [suricata::applayers7comm::s7comm::rs_s7comm_register_parser:s7comm.rs:442]
Notice: output-json-s7comm: S7comm JSON logger registered. [JsonS7commLogRegister:output-json-s7comm.c:172]
Info: logopenfile: fast output device (regular) initialized: fast.log [SCConfLogOpenGeneric:util-logopenfile.c:617]
Info: logopenfile: eve-log output device (regular) initialized: eve.json [SCConfLogOpenGeneric:util-logopenfile.c:617]
Notice: output-json-s7comm: S7comm log sub-module initialized. [OutputS7commLogInitSub:output-json-s7comm.c:119]
Notice: output-json-s7comm: S7comm log sub-module initialized. [OutputS7commLogInitSub:output-json-s7comm.c:119]
Info: logopenfile: stats output device (regular) initialized: stats.log [SCConfLogOpenGeneric:util-logopenfile.c:617]
Info: detect: 1 rule files processed. 1 rules successfully loaded, 0 rules failed, 0 [SigLoadSignatures:detect-engine-loader.c:363]
Info: threshold-config: Threshold config parsed: 0 rule(s) found [SCThresholdConfParseFile:util-threshold-config.c:1045]
Info: detect: 1 signatures processed. 0 are IP-only rules, 1 are inspecting packet payload, 0 inspect application layer, 0 are decoder event only [SigAddressPrepareStage1:detect-engine-build.c:1499]
Info: pcap: Starting file run for dump.pcapng [ReceivePcapFileLoop:source-pcap-file.c:179]
Info: pcap: pcap file dump.pcapng end of file reached (pcap err code 0) [PcapFileDispatch:source-pcap-file-helper.c:163]
Notice: threads: Threads created -> RX: 1 W: 4 FM: 1 FR: 1   Engine started. [TmThreadWaitOnThreadRunning:tm-threads.c:1901]
Notice: suricata: Signal Received.  Stopping engine. [SuricataMainLoop:suricata.c:2806]
Info: suricata: time elapsed 0.190s [SCPrintElapsedTime:suricata.c:1166]
Notice: pcap: read 1 file, 200 packets, 14388 bytes [ReceivePcapFileThreadExitStats:source-pcap-file.c:388]
Info: counters: Alerts: 0 [StatsLogSummary:counters.c:878]

I have already followed the recommendations in the video, but the problem still persists. In the video, when running python3 scripts/setup-app-layer.py --detect --logger --parser --rust S7comm S7commbuf, there is an error with the --rust option.

4

From what you have shared, it seems that the files were generated properly.

Could you share an eve.log of when you ran Suricata after adding the new parser?

Thanks :slight_smile:

5

BTW the --rust option is obsolete, since now parsers are expected to be written in Rust. Hence recent versions of that script do not even have this option:

❯ ./scripts/setup-app-layer.py -h
usage: setup-app-layer.py [-h] [--logger] [--parser] [--detect] proto [buffer]

and I guess it is assumed to be always “enabled”.

6

@ish 's steps worked for me:

$ # Start with an unmodified, current Suricata repo
$ python3 scripts/setup-app-layer.py S7comm
$ python3 scripts/setup-app-layer.py  --detect S7comm S7commbuf
$ ./configure
$ make
7

eve.json (37.2 KB)

8

After multiple rounds of testing and suggestions from everyone, I finally found the root of the problem.

···
#ok
./configure
···

#error
./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var

In version 6.0.15, it works, but in version 7.0.6, it doesn’t.

9

Thanks for the eve.json file.

Do you mean that these configuration options are not working for 7.0.6, that is, Suricata overwrites these? Does this mean that with 6.0.15 (do note that we just released 6.0.20) you are able to see s7comm events in the eve log?

On a side note, I’ve noticed that your eve.log only has flow events, and all app-proto fields are “failed”. In cases like these, enabling anomaly events might be a good idea, to further investigate. Does your pcap only contain S7comm traffic?

10 
./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var

These configuration options are not suitable for version 7.0.6.

This is the eve.json generated by the introduction of the protocol in version 6.0.15.

eve.json (10.1 KB)

The majority of traffic packets in the pcap are S7comm.

The pcap is from GitHub.

dump.pcap

I only introduced the template code and did not write any other code related to eve.json.

11

What leads you to this claim?

How do you expect S7COMM detection if you didn’t write any code? The template will just generate the basics to start working on the protocol parser.

12

I will answer the first question first.

./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var

These options, when modify code, will not change the output of s7comm.rs.
but,

./configure

This way, it is possible to print out the hexadecimal content of the data packet. Changes to the code will also change the output.

The second question.
I want to introduce a protocol template and print out the contents of the traffic packet.This is what I currently want to do. The next step is to parse the traffic and output it to eve.json.

13

They’re meant to indicate where to install Suricata, where should the configuration files be stored, and where should the Suricata logs go. So this shouldn’t impact the code output itself - but they many times lead to confusion if we think we’re running our version of Suricata - with local changes -, or a version that was installed system-wide (in my personal experience).