I have two questions:
packet field in eve.json contains the content of payload field and sometimes it doesn’t. I want to know when each of this cases happen.packet field?Packet data is conditionally included in alerts (event_type == alert) according to configuration settings in Suricata’s configuration file – suricata.yaml.
Here’s a snippet from master
types:
- alert:
# payload: yes # enable dumping payload in Base64
# payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
# payload-printable: yes # enable dumping payload in printable (lossy) format
# packet: yes # enable dumping of packet (without stream segments)
# metadata: no # enable inclusion of app layer metadata with alert. Default yes
# http-body: yes # Requires metadata; enable dumping of HTTP body in Base64
# http-body-printable: yes # Requires metadata; enable dumping of HTTP body in printable format
# Enable the logging of tagged packets for rules using the
# "tag" keyword.
tagged-packets: yes
See 15.1.1. Eve JSON Output — Suricata 7.0.0-dev documentation
I have enabled both packet and payload:
types:
- alert:
payload: yes # enable dumping payload in Base64
payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
# payload-printable: yes # enable dumping payload in printable (lossy) format
packet: yes # enable dumping of packet (without stream segments)
# metadata: no # enable inclusion of app layer metadata with alert. Default yes
# http-body: yes # Requires metadata; enable dumping of HTTP body in Base64
# http-body-printable: yes # Requires metadata; enable dumping of HTTP body in printable format
# Enable the logging of tagged packets for rules using the
# "tag" keyword.
tagged-packets: yes
Let me rephrase me first question, why sometimes payload is a substring (suffix) of packet and sometimes it is not?
For example in the following log all payload data is included in packet:
{
"timestamp": "2021-10-13T16:56:27.865716+0330",
"flow_id": 428744548496820,
"in_iface": "ens33",
"event_type": "alert",
"src_ip": "192.168.41.146",
"src_port": 39744,
"dest_ip": "192.168.41.2",
"dest_port": 53,
"proto": "UDP",
"tx_id": 0,
"alert": {
"action": "allowed",
"gid": 1,
"signature_id": 2024293,
"rev": 6,
"signature": "ET MALWARE Possible WannaCry DNS Lookup 2",
"category": "A Network Trojan was detected",
"severity": 1,
"metadata": {
"affected_product": [
"Windows_XP_Vista_7_8_10_Server_32_64_Bit"
],
"attack_target": [
"Client_Endpoint"
],
"created_at": [
"2017_05_14"
],
"deployment": [
"Perimeter"
],
"former_category": [
"TROJAN"
],
"malware_family": [
"wannacry"
],
"performance_impact": [
"Moderate"
],
"signature_severity": [
"Critical"
],
"tag": [
"Ransomware"
],
"updated_at": [
"2020_09_01"
]
}
},
"dns": {
"query": [
{
"type": "query",
"id": 10158,
"rrname": "www.ifferfsodp9ifjaposdfjhgosurijfaewrwergwea.com",
"rrtype": "AAAA",
"tx_id": 0
}
]
},
"app_proto": "dns",
"flow": {
"pkts_toserver": 1,
"pkts_toclient": 0,
"bytes_toserver": 120,
"bytes_toclient": 0,
"start": "2021-10-13T16:56:27.865716+0330"
},
"payload": "J64BAAABAAAAAAABA3d3dylpZmZlcmZzb2RwOWlmamFwb3NkZmpoZ29zdXJpamZhZXdyd2VyZ3dlYQNjb20AABwAAQAAKQIAAAAAAAAA",
"stream": 0,
"packet": "AFBW7unAAAwpr9jLCABFAABqRCFAAEARIn3AqCmSwKgpAptAADUAVtRMJ64BAAABAAAAAAABA3d3dylpZmZlcmZzb2RwOWlmamFwb3NkZmpoZ29zdXJpamZhZXdyd2VyZ3dlYQNjb20AABwAAQAAKQIAAAAAAAAA",
"packet_info": {
"linktype": 1
}
}
but in this case payload data is not in the packet field:
{
"timestamp": "2021-10-13T16:43:20.386331+0330",
"flow_id": 1592530310207584,
"in_iface": "ens33",
"event_type": "alert",
"src_ip": "192.168.41.146",
"src_port": 47092,
"dest_ip": "217.170.252.28",
"dest_port": 80,
"proto": "TCP",
"tx_id": 25,
"alert": {
"action": "allowed",
"gid": 1,
"signature_id": 2013504,
"rev": 6,
"signature": "ET POLICY GNU/Linux APT User-Agent Outbound likely related to package management",
"category": "Not Suspicious Traffic",
"severity": 3,
"metadata": {
"created_at": [
"2011_08_31"
],
"former_category": [
"POLICY"
],
"updated_at": [
"2020_04_22"
]
}
},
"http": {
"hostname": "ir.archive.ubuntu.com",
"url": "/ubuntu/pool/main/s/squashfs-tools/squashfs-tools_4.4-1ubuntu0.3_amd64.deb",
"http_user_agent": "Debian APT-HTTP/1.3 (2.0.6) non-interactive",
"http_method": "GET",
"protocol": "HTTP/1.1",
"length": 0
},
"app_proto": "http",
"flow": {
"pkts_toserver": 466,
"pkts_toclient": 680,
"bytes_toserver": 29769,
"bytes_toclient": 941672,
"start": "2021-10-13T16:43:19.661600+0330"
},
"payload": "R0VUIC91YnVudHUvcG9vbC9tYWluL3AvcGFtL2xpYnBhbTBnXzEuMy4xLTV1YnVudHU0LjNfYW1kNjQuZGViIEhUVFAvMS4xDQpIb3N0OiBpci5hcmNoaXZlLnVidW50dS5jb20NClVzZXItQWdlbnQ6IERlYmlhbiBBUFQtSFRUUC8xLjMgKDIuMC42KSBub24taW50ZXJhY3RpdmUNCg0KR0VUIC91YnVudHUvcG9vbC9tYWluL3AvcGFtL2xpYnBhbS1tb2R1bGVzLWJpbl8xLjMuMS01dWJ1bnR1NC4zX2FtZDY0LmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9wL3BhbS9saWJwYW0tbW9kdWxlc18xLjMuMS01dWJ1bnR1NC4zX2FtZDY0LmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9wL3BhbS9saWJwYW0tcnVudGltZV8xLjMuMS01dWJ1bnR1NC4zX2FsbC5kZWIgSFRUUC8xLjENCkhvc3Q6IGlyLmFyY2hpdmUudWJ1bnR1LmNvbQ0KVXNlci1BZ2VudDogRGViaWFuIEFQVC1IVFRQLzEuMyAoMi4wLjYpIG5vbi1pbnRlcmFjdGl2ZQ0KDQpHRVQgL3VidW50dS9wb29sL21haW4vbi9uZXRwbGFuLmlvL2xpYm5ldHBsYW4wXzAuMTAzLTB1YnVudHU1JTdlMjAuMDQuMV9hbWQ2NC5kZWIgSFRUUC8xLjENCkhvc3Q6IGlyLmFyY2hpdmUudWJ1bnR1LmNvbQ0KVXNlci1BZ2VudDogRGViaWFuIEFQVC1IVFRQLzEuMyAoMi4wLjYpIG5vbi1pbnRlcmFjdGl2ZQ0KDQpHRVQgL3VidW50dS9wb29sL21haW4vbi9uZXRwbGFuLmlvL25ldHBsYW4uaW9fMC4xMDMtMHVidW50dTUlN2UyMC4wNC4xX2FtZDY0LmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9saWJkL2xpYmRybS9saWJkcm0tY29tbW9uXzIuNC4xMDUtMyU3ZTIwLjA0LjJfYWxsLmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9saWJkL2xpYmRybS9saWJkcm0yXzIuNC4xMDUtMyU3ZTIwLjA0LjJfYW1kNjQuZGViIEhUVFAvMS4xDQpIb3N0OiBpci5hcmNoaXZlLnVidW50dS5jb20NClVzZXItQWdlbnQ6IERlYmlhbiBBUFQtSFRUUC8xLjMgKDIuMC42KSBub24taW50ZXJhY3RpdmUNCg0KR0VUIC91YnVudHUvcG9vbC9tYWluL3UvdXBkYXRlLW1hbmFnZXIvcHl0aG9uMy11cGRhdGUtbWFuYWdlcl8yMC4wNC4xMC45X2FsbC5kZWIgSFRUUC8xLjENCkhvc3Q6IGlyLmFyY2hpdmUudWJ1bnR1LmNvbQ0KVXNlci1BZ2VudDogRGViaWFuIEFQVC1IVFRQLzEuMyAoMi4wLjYpIG5vbi1pbnRlcmFjdGl2ZQ0KDQpHRVQgL3VidW50dS9wb29sL21haW4vdS91cGRhdGUtbWFuYWdlci91cGRhdGUtbWFuYWdlci1jb3JlXzIwLjA0LjEwLjlfYWxsLmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9saWJkL2xpYmRybS9saWJkcm0tYW1kZ3B1MV8yLjQuMTA1LTMlN2UyMC4wNC4yX2FtZDY0LmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9saWJkL2xpYmRybS9saWJkcm0taW50ZWwxXzIuNC4xMDUtMyU3ZTIwLjA0LjJfYW1kNjQuZGViIEhUVFAvMS4xDQpIb3N0OiBpci5hcmNoaXZlLnVidW50dS5jb20NClVzZXItQWdlbnQ6IERlYmlhbiBBUFQtSFRUUC8xLjMgKDIuMC42KSBub24taW50ZXJhY3RpdmUNCg0KR0VUIC91YnVudHUvcG9vbC9tYWluL2xpYmQvbGliZHJtL2xpYmRybS1ub3V2ZWF1Ml8yLjQuMTA1LTMlN2UyMC4wNC4yX2FtZDY0LmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9saWJkL2xpYmRybS9saWJkcm0tcmFkZW9uMV8yLjQuMTA1LTMlN2UyMC4wNC4yX2FtZDY0LmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9tL21lc2EvbGliZWdsLW1lc2EwXzIxLjAuMy0wdWJ1bnR1MC4zJTdlMjAuMDQuM19hbWQ2NC5kZWIgSFRUUC8xLjENCkhvc3Q6IGlyLmFyY2hpdmUudWJ1bnR1LmNvbQ0KVXNlci1BZ2VudDogRGViaWFuIEFQVC1IVFRQLzEuMyAoMi4wLjYpIG5vbi1pbnRlcmFjdGl2ZQ0KDQpHRVQgL3VidW50dS9wb29sL21haW4vbS9tZXNhL2xpYmdibTFfMjEuMC4zLTB1YnVudHUwLjMlN2UyMC4wNC4zX2FtZDY0LmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9tL21lc2EvbGliZ2wxLW1lc2EtZHJpXzIxLjAuMy0wdWJ1bnR1MC4zJTdlMjAuMDQuM19hbWQ2NC5kZWIgSFRUUC8xLjENCkhvc3Q6IGlyLmFyY2hpdmUudWJ1bnR1LmNvbQ0KVXNlci1BZ2VudDogRGViaWFuIEFQVC1IVFRQLzEuMyAoMi4wLjYpIG5vbi1pbnRlcmFjdGl2ZQ0KDQpHRVQgL3VidW50dS9wb29sL21haW4vbS9tZXNhL2xpYmdseC1tZXNhMF8yMS4wLjMtMHVidW50dTAuMyU3ZTIwLjA0LjNfYW1kNjQuZGViIEhUVFAvMS4xDQpIb3N0OiBpci5hcmNoaXZlLnVidW50dS5jb20NClVzZXItQWdlbnQ6IERlYmlhbiBBUFQtSFRUUC8xLjMgKDIuMC42KSBub24taW50ZXJhY3RpdmUNCg0KR0VUIC91YnVudHUvcG9vbC9tYWluL20vbWVzYS9saWJnbGFwaS1tZXNhXzIxLjAuMy0wdWJ1bnR1MC4zJTdlMjAuMDQuM19hbWQ2NC5kZWIgSFRUUC8xLjENCkhvc3Q6IGlyLmFyY2hpdmUudWJ1bnR1LmNvbQ0KVXNlci1BZ2VudDogRGViaWFuIEFQVC1IVFRQLzEuMyAoMi4wLjYpIG5vbi1pbnRlcmFjdGl2ZQ0KDQpHRVQgL3VidW50dS9wb29sL21haW4vbS9teXNxbC04LjAvbGlibXlzcWxjbGllbnQyMV84LjAuMjYtMHVidW50dTAuMjAuMDQuM19hbWQ2NC5kZWIgSFRUUC8xLjENCkhvc3Q6IGlyLmFyY2hpdmUudWJ1bnR1LmNvbQ0KVXNlci1BZ2VudDogRGViaWFuIEFQVC1IVFRQLzEuMyAoMi4wLjYpIG5vbi1pbnRlcmFjdGl2ZQ0KDQpHRVQgL3VidW50dS9wb29sL21haW4vbS9tZXNhL2xpYnhhdHJhY2tlcjJfMjEuMC4zLTB1YnVudHUwLjMlN2UyMC4wNC4zX2FtZDY0LmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9sL2xpbnV4LWZpcm13YXJlL2xpbnV4LWZpcm13YXJlXzEuMTg3LjE5X2FsbC5kZWIgSFRUUC8xLjENCkhvc3Q6IGlyLmFyY2hpdmUudWJ1bnR1LmNvbQ0KVXNlci1BZ2VudDogRGViaWFuIEFQVC1IVFRQLzEuMyAoMi4wLjYpIG5vbi1pbnRlcmFjdGl2ZQ0KDQpHRVQgL3VidW50dS9wb29sL3VuaXZlcnNlL20vbWVzYS9tZXNhLXZhLWRyaXZlcnNfMjEuMC4zLTB1YnVudHUwLjMlN2UyMC4wNC4zX2FtZDY0LmRlYiBIVFRQLzEuMQ0KSG9zdDogaXIuYXJjaGl2ZS51YnVudHUuY29tDQpVc2VyLUFnZW50OiBEZWJpYW4gQVBULUhUVFAvMS4zICgyLjAuNikgbm9uLWludGVyYWN0aXZlDQoNCkdFVCAvdWJ1bnR1L3Bvb2wvbWFpbi9tL21lc2EvbWVzYS12ZHBhdS1kcml2ZXJzXzIxLjAuMy0w",
"stream": 1,
"packet": "AFBW7unAAAwpr9jLCABFAAAoXpxAAEAGHDLAqCmS2ar8HLf0AFApbASfYQLa5lAQ///AHAAA",
"packet_info": {
"linktype": 1
}
}
Now my second question, as you can see in the second log there is hundreds of packets to server and to client. But the packet field definitely does not contain all of them. So the question is which of those packets’ data are in the packet field?
payload represents the stream (for streaming protocols). Configure the size of the payload buffer using payload-buffer-size.
packet is the packet on which the alert triggered. @catenacyber provided some insight into why this may not be the packet you expect here: Bug #3480: EVE JSON - Incorrect Packet Logged - Suricata - Open Information Security Foundation
Yes, you can use stream.inline=true in your yaml configuration to get packet to be more what is intuitive