Building 6.0.3 from source in OpenWrt, I’m receiving an SIGILL Illegal Instruction
Compiled with rust 1.57 tuple mips64-openwrt-linux-musl (cloned from mips64-unknown-linux-muslabi64).
10/12/2021 -- 07:17:46 - <Debug> - Entering ... >>
10/12/2021 -- 07:17:46 - <Debug> - LIBHTP default config: 0xfff1d06030
10/12/2021 -- 07:17:46 - <Debug> - LIBHTP default: personality = IDS
10/12/2021 -- 07:17:46 - <Debug> - LIBHTP default: personality = IDS
10/12/2021 -- 07:17:46 - <Debug> - LIBHTP default: personality=IDS (2)
10/12/2021 -- 07:17:46 - <Debug> - LIBHTP personality set to IDS
Illegal instruction
grommish@DESKTOP-AW:~/openwrt$ ./scripts/remote-gdb 192.168.200.102:9000 build_dir/target-mips64_octeonplus_64_musl/suricata-6.0.3/ipkg-mips64_octeonplus/suricata6/usr/bin/suricata
Using target mips64_octeonplus_64 (musl, )
GNU gdb (GDB) 10.1
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=mips64-openwrt-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from build_dir/target-mips64_octeonplus_64_musl/suricata-6.0.3/ipkg-mips64_octeonplus/suricata6/usr/bin/suricata...
warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts
of file /home/grommish/openwrt/build_dir/target-mips64_octeonplus_64_musl/suricata-6.0.3/ipkg-mips64_octeonplus/suricata6/usr/bin/suricata.
Use `info auto-load python-scripts [REGEXP]' to list them.
0x000000fff7f57100 in _dlstart ()
from /home/grommish/openwrt/scripts/../staging_dir/target-mips64_octeonplus_64_musl/root-octeon/lib/ld-musl-mips64-sf.so.1
(gdb) bt
#0 0x000000fff7f57100 in _dlstart ()
from /home/grommish/openwrt/scripts/../staging_dir/target-mips64_octeonplus_64_musl/root-octeon/lib/ld-musl-mips64-sf.so.1
#1 0x0000000000000000 in ?? ()
Backtrace stopped: frame did not save the PC
(gdb) run -vvvv -c /etc/suricata/suricata.yaml -i eth0
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/grommish/openwrt/build_dir/target-mips64_octeonplus_64_musl/suricata-6.0.3/ipkg-mips64_octeonplus/suricata6/usr/bin/suricata -vvvv -c /etc/suricata/suricata.yaml -i eth0
Program received signal SIGILL, Illegal instruction.
0x000000aaab5e0fb8 in compiler_builtins::float::mul::__muldf3 ()
(gdb) bt
#0 0x000000aaab5e0fb8 in compiler_builtins::float::mul::__muldf3 ()
#1 0x000000aaab5e0914 in __muldf3 ()
#2 0x000000aaaaf35f44 in ParseSizeString (size=<optimized out>, res=0xffffffd8c8) at util-misc.c:138
#3 0x000000aaaaf3629c in ParseSizeStringU32 (size=<optimized out>, res=0xaaaba481d0 <cfglist+48>) at util-misc.c:190
#4 0x000000aaaadbb6bc in HTPConfigParseParameters (tree=0xfff7902db0, s=0xfff79613b0, cfg_prec=0xaaaba481a0 <cfglist>)
at app-layer-htp.c:2561
#5 HTPConfigParseParameters (cfg_prec=0xaaaba481a0 <cfglist>, s=0xfff79613b0, tree=0xfff7902db0)
at app-layer-htp.c:2495
#6 0x000000aaaadbf520 in HTPConfigure () at app-layer-htp.c:2873
#7 0x000000aaaadbfb54 in RegisterHTPParsers () at app-layer-htp.c:3172
#8 0x000000aaaadca090 in AppLayerParserRegisterProtocolParsers () at app-layer-parser.c:1601
#9 0x000000aaaad99f28 in AppLayerSetup () at app-layer.c:846
#10 0x000000aaaaefa5e0 in PostConfLoadedSetup (suri=0xaaaba5dbc0 <suricata>) at suricata.c:2509
#11 PostConfLoadedSetup (suri=0xaaaba5dbc0 <suricata>) at suricata.c:2453
#12 0x000000aaaaefbab0 in SuricataMain (argc=<optimized out>, argv=<optimized out>) at suricata.c:2794
#13 0x000000aaaad94f44 in main (argc=<optimized out>, argv=<optimized out>) at main.c:22
(gdb) x/i $pc
=> 0xaaab5e0fb8 <_ZN17compiler_builtins5float3mul8__muldf317hc5658e15f13ba524E>: dmfc1 a4,$f12
(gdb)
dmfc1 is a valid MIPS command for DMFC1 Doubleword Move from Floating Point (pg 196, https://s3-eu-west-1.amazonaws.com/downloads-mips/documents/MIPS_Architecture_MIPS64_InstructionSet_%20AFP_P_MD00087_06.05.pdf)
Everything is being build -msoft-float, but I’m seeing Suricata trying to link -mhard-float
/home/grommish/openwrt/staging_dir/toolchain-mips64_octeonplus_64_gcc-11.2.0_musl/lib/gcc/mips64-openwrt-linux-musl/11.2.0/../../../../mips64-openwrt-linux-musl/bin/ld: warning: suricata uses -msoft-float (set by /home/grommish/openwrt/staging_dir/toolchain-mips64_octeonplus_64_gcc-11.2.0_musl/lib/gcc/mips64-openwrt-linux-musl/11.2.0/../../../../mips64-openwrt-linux-musl/lib/Scrt1.o), ../rust/target/mips64-openwrt-linux-musl/debug/libsuricata.a(object-94eba9f9c40e2aad.object.2f68f6ad-cgu.3.rcgu.o) uses -mhard-float
Suggestions? Is Suricata setting hard-float somewhere in-code?
Appreciate any suggestions!
Build log below
suricata_build.log (1.4 MB)