Suricata 6.0.0 rc1 JSON structure

[suricatalfon]

Hi,

i think only with fileinfo.

Suricata 5.x:

cat eve.json | jq -c 'select(.fileinfo)|.fileinfo.filename’

“/project.aspx”
“/project.aspx”
“/project.aspx”
“/a.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/db.aspx”
“/bounce.aspx”
“/bounce.aspx”
“zest2.cab”
“/project.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/db.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”
“/bounce.aspx”

Suricata 6.0.0 rc1

cat eve.json | jq -c 'select(.fileinfo)|.fileinfo.filename’

jq: error (at :10): Cannot index array with string “filename”
jq: error (at :11): Cannot index array with string “filename”
jq: error (at :12): Cannot index array with string “filename”
“16OMZLLAZ.exe”
“/raw”
“/ono72/DESKTOP-GNJ751U_W10019041.319D0542A2FF74DAB56247F5C17738F9/83/”
“/ono72/DESKTOP-GNJ751U_W10019041.319D0542A2FF74DAB56247F5C17738F9/81/”
“/ono72/DESKTOP-GNJ751U_W10019041.319D0542A2FF74DAB56247F5C17738F9/90”
jq: error (at :62): Cannot index array with string “filename”
jq: error (at :63): Cannot index array with string “filename”
jq: error (at :64): Cannot index array with string “filename”
“/images/shortwave.png”
jq: error (at :68): Cannot index array with string “filename”
jq: error (at :69): Cannot index array with string “filename”
“/images/longrip.png”
jq: error (at :73): Cannot index array with string “filename”
“/images/shortwave.png”
“/ono72/DESKTOP-GNJ751U_W10019041.319D0542A2FF74DAB56247F5C17738F9/81/”

cat eve.json | jq .fileinfo.stored

null
true
null
null
null
jq: error (at :62): Cannot index array with string “stored”
jq: error (at :63): Cannot index array with string “stored”
jq: error (at :64): Cannot index array with string “stored”
true
null
null
jq: error (at :68): Cannot index array with string “stored”
jq: error (at :69): Cannot index array with string “stored”
true
null
null
jq: error (at :73): Cannot index array with string “stored”
true
null
null
null
null

cat eve.json | jq -c ‘select(.fileinfo)|.fileinfo.stored’

jq: error (at :10): Cannot index array with string “stored”
jq: error (at :11): Cannot index array with string “stored”
jq: error (at :12): Cannot index array with string “stored”
true
true
true
true
true
jq: error (at :62): Cannot index array with string “stored”
jq: error (at :63): Cannot index array with string “stored”
jq: error (at :64): Cannot index array with string “stored”
true
jq: error (at :68): Cannot index array with string “stored”

cat eve.json | ag fileinfo | jq ‘select(.app_proto==“http”) | .fileinfo.sha256’ | tr -d ‘"’ | awk ‘{print “./files/“substr($0,0,2)”/”$0}’ | xargs -I{} file {} | colout ‘.:’ green | colout 'PE32.’ red | colout ‘.Cabinet.’ red
jq: error (at :1): Cannot index array with string “sha256”
jq: error (at :2): Cannot index array with string “sha256”
jq: error (at :3): Cannot index array with string “sha256”
jq: error (at :9): Cannot index array with string “sha256”
jq: error (at :10): Cannot index array with string “sha256”
jq: error (at :11): Cannot index array with string “sha256”
jq: error (at :13): Cannot index array with string “sha256”
jq: error (at :14): Cannot index array with string “sha256”
jq: error (at :16): Cannot index array with string “sha256”
./files/71/71d3d6fd849274d43a7d9fd4e0011ad24d15a9c5cb57bbb0dc0eda933c397593: PE32 executable (GUI) Intel 80386, for MS Windows
./files/40/40a58618cb697f749cd36c822c3dff37099016d24102ad127d52c08f06fe3249: ASCII text, with no line terminators
./files/c7/c76ea8b0db60dcccfdc0217059e37eb5b10d10940e649f9bc1d07d4ddcc797d1: ASCII text, with no line terminators
./files/c7/c76ea8b0db60dcccfdc0217059e37eb5b10d10940e649f9bc1d07d4ddcc797d1: ASCII text, with no line terminators
./files/c7/c76ea8b0db60dcccfdc0217059e37eb5b10d10940e649f9bc1d07d4ddcc797d1: ASCII text, with no line terminators
./files/5e/5e4991ae0dadaf490182b54b98a2e5f4957b7d5e6f0f7d42d4448a795625f9c1: PE32 executable (GUI) Intel 80386, for MS Windows
./files/34/343124b9ebf869bd9f0b1d7a85b59256a227d27b89b221f22479ba678098a4b8: PE32 executable (GUI) Intel 80386, for MS Windows
./files/38/3839be21f85ae3918745ff7fe4bef17125d5ae650eaf7fa5bfc98ef4021e1b30: PE32 executable (GUI) Intel 80386, for MS Windows
./files/c7/c76ea8b0db60dcccfdc0217059e37eb5b10d10940e649f9bc1d07d4ddcc797d1: ASCII text, with no line terminators

Thanks

[vjulien (Victor Julien)]

Are you able to (privately) share the pcap and/or the eve.json file?

[suricatalfon]

you already have eve.json

Tanks

[vjulien (Victor Julien)]

It looks like this is caused by the .fileinfo addition into alerts, where its an array of fileinfo records instead of a simple map with a single record.

@ish @Jeff_Lucovsky any ideas on this?

[suricatalfon]

In the versions of Suricata 5.x there was no problem with this as you will have seen in the last captures that I have sent you.

[Jeff_Lucovsky]

@suricatalfon

cat eve.json | jq -c 'select(.fileinfo)|.fileinfo.filename’

Try cat eve.json|jq -c 'select(.event_type=="fileinfo").fileinfo.filename'

In 6.0.0, alerts now contain fileinfo information. Since you’re after the fileinfo records, first, filter on that (event_type=="fileinfo") and then access the object.

[vjulien (Victor Julien)]

That is a workaround for the report, but I think the bigger question if we should have a .fileinfo that can be both a map and an array.

[Jeff_Lucovsky]

An array seems most appropriate, at least for the alert since there can be multiple “file” objects to report.

[suricatalfon]

Thanks

[suricatalfon]

Hi,

I am seeing another major change in the json structure. In anomaly we have:

community_id
metadata.flowbits
anomaly events.

cat eve.json | jq 'select(.anomaly)|.flow_id,.anomaly’

On other occasions, it appears differently:

Best Regads,

[vjulien (Victor Julien)]

Thanks, I’ve opened https://redmine.openinfosecfoundation.org/issues/3928

[suricatalfon]

hi,
Okay. Thank you very much. We await the solution.