After running 7.0.6 for 4 weeks or so without any problems on multiple gigabits and >28 cores in af-packet mode, I experienced some more coredumps. Some were due to SIGABRT and others due to SIGSEGV.
Unfortunately, I cannot run this with additional debug measures, since it takes so long to core dump. I do not expect that someone is able to fix this by looking at the BT, but I cannot deliver more atm.
This is a gdb BT for a coredump due to a SIGABRT.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid'.
Program terminated with signal SIGABRT, Aborted.
#0 __pthread_kill_implementation (threadid=<optimized out>,
signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
44 ./nptl/pthread_kill.c: Datei oder Verzeichnis nicht gefunden.
[Current thread is 1 (Thread 0x7f8cfa8e06c0 (LWP 4148687))]
(gdb) bt
#0 __pthread_kill_implementation (threadid=<optimized out>,
signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1 0x00007f8d07af2e8f in __pthread_kill_internal (signo=6,
threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2 0x00007f8d07aa3fb2 in __GI_raise (sig=sig@entry=6)
at ../sysdeps/posix/raise.c:26
#3 0x00007f8d07a8e472 in __GI_abort () at ./stdlib/abort.c:79
#4 0x00007f8d07ae7430 in __libc_message (action=action@entry=do_abort,
fmt=fmt@entry=0x7f8d07c01459 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#5 0x00007f8d07afc7aa in malloc_printerr (
str=str@entry=0x7f8d07c04590 "malloc(): invalid next size (unsorted)")
at ./malloc/malloc.c:5660
#6 0x00007f8d07aff8e4 in _int_malloc (av=av@entry=0x7f8cf4000030,
bytes=bytes@entry=4097) at ./malloc/malloc.c:4001
#7 0x00007f8d07b00362 in _int_realloc (av=av@entry=0x7f8cf4000030,
oldp=oldp@entry=0x7f7c8e0efeb0, oldsize=oldsize@entry=2064,
nb=nb@entry=4112) at ./malloc/malloc.c:4874
#8 0x00007f8d07b0120f in __GI___libc_realloc (oldmem=0x7f7c8e0efec0,
bytes=4096) at ./malloc/malloc.c:3489
#9 0x0000563407b4162e in SCReallocFunc ()
#10 0x0000563407c1e915 in StreamTcpReassembleRealloc ()
#11 0x0000563407c4ddf8 in StreamingBufferInsertAt ()
#12 0x0000563407c1da69 in StreamTcpReassembleInsertSegment ()
#13 0x0000563407c20571 in StreamTcpReassembleHandleSegmentHandleData ()
#14 0x0000563407c2078a in StreamTcpReassembleHandleSegment ()
#15 0x0000563407c16e4e in ?? ()
#16 0x0000563407c19268 in ?? ()
#17 0x0000563407c1ab86 in StreamTcpPacket ()
#18 0x0000563407c1bdc9 in StreamTcp ()
#19 0x0000563407bd91a5 in ?? ()
#20 0x0000563407bd9717 in ?? ()
#21 0x0000563407b2e90f in TmThreadsSlotVarRun ()
#22 0x0000563407c06bf4 in ?? ()
#23 0x0000563407c06f32 in ?? ()
#24 0x0000563407c07ab4 in ?? ()
#25 0x0000563407b30064 in ?? ()
#26 0x00007f8d07af1134 in start_thread (arg=<optimized out>)
at ./nptl/pthread_create.c:442
#27 0x00007f8d07b717dc in clone3 ()
at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
This is one was due to a SIGSEGV:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 _int_malloc (av=av@entry=0x7f1fb8000030, bytes=bytes@entry=4096)
at ./malloc/malloc.c:4129
4129 ./malloc/malloc.c: Datei oder Verzeichnis nicht gefunden.
[Current thread is 1 (Thread 0x7f1f2d7ff6c0 (LWP 3696699))]
(gdb) bt
#0 _int_malloc (av=av@entry=0x7f1fb8000030, bytes=bytes@entry=4096)
at ./malloc/malloc.c:4129
#1 0x00007f1ffc61f8f9 in __GI___libc_malloc (bytes=4096)
at ./malloc/malloc.c:3323
#2 0x000055f7f40f93b7 in ?? ()
#3 0x000055f7f40f9cd1 in alloc::string::String::try_reserve::h70d7f11456287616
()
#4 0x000055f7f3e01a23 in suricata::jsonbuilder::JsonBuilder::try_new_object_with_capacity::hf6e50a27e0b3d6d4 ()
#5 0x000055f7f3e08816 in jb_new_object ()
#6 0x000055f7f3c80d6e in CreateEveHeader ()
#7 0x000055f7f3c81121 in CreateEveHeaderWithTxId ()
#8 0x000055f7f3c8d9dd in ?? ()
#9 0x000055f7f3c95c98 in ?? ()
#10 0x000055f7f3c78904 in OutputLoggerLog ()
#11 0x000055f7f3c73217 in ?? ()
#12 0x000055f7f3c73717 in ?? ()
#13 0x000055f7f3bc890f in TmThreadsSlotVarRun ()
#14 0x000055f7f3ca0bf4 in ?? ()
#15 0x000055f7f3ca0f32 in ?? ()
#16 0x000055f7f3ca1ab4 in ?? ()
#17 0x000055f7f3bca064 in ?? ()
#18 0x00007f1ffc610134 in start_thread (arg=<optimized out>)
at ./nptl/pthread_create.c:442
#19 0x00007f1ffc6907dc in clone3 ()
at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81