I’ve already asked this question on discord, but thought this might be a better place.
I was wondering if there’s any future plans to be able to register a custom C (or Rust) content matcher.
The current plugin support is quite amazing, StreamingLogger, TXLoggers, Eveloggers, but its missing a custom matcher for (sticky) buffers and payloads like currently possible in Lua. Unfortunately it’s missing some useful keywords, for example websocket.payload (?) The current matcher
Additionally, Lua is not the most performant and given that the only (afaik) workaround to load some external matcher (from C) will be disabled in 8.0.0, it would be beneficial to have a more efficient and flexible solution built directly into the system.
We are currently running a small (and dirty) patch in the end of the DetectEngineContentInspectionInternal function that bypasses the BUG_ON(1); and instead calls sigmatch_table[smd->type].Transform(&inspectBuffer, smd->ctx);. (In case .Transform is not NULL).
We also required access to the packet (& flow) inside our custom plugin Transformer so we also added a Packet* to the InspectionBuffer.
It was convenient to use the Transform for both the Packet contents and StickyBuffers.
Quite some patches required to get the same functionality as the slow lua sadly (for now at least).
During this, we also noticed that Flowvars allocated by ScFlowvarSet / SCFlowvarSet in Lua do not get their name value free’d, while they are malloced in that function, resulting in a potential memory leak, we thought this was worth mentioning. The same happens for the values, but they are free’d on flowvar free.
As another thing we had to debug through, the sigmatch_table is only allocated after plugins are loaded.
We were dynamically adding sigmatch keywords. However, the sigmatch_table would be NULL, so when we called DetectHelperKeywordRegister it would allocate a size 256 table, while our new keyword idx would be DETECT_TBLSIZE_IDX >= 256 therefore doing a heap OOB, then on the second keyword registration it would reallocate to 2 * 256, enough to fit everything, but miss the first dynamically registered keyword. We currently fixed it with the following in our plugin registration:
static void InitPlugin(void)
{
/* Copied from SigTableSetup
* It would do a heap overflow if not for this..
*/
if (sigmatch_table == NULL) {
DETECT_TBLSIZE = DETECT_TBLSIZE_STATIC + DETECT_TBLSIZE_STEP;
sigmatch_table = SCCalloc(DETECT_TBLSIZE, sizeof(SigTableElmt));
if (sigmatch_table == NULL) {
DETECT_TBLSIZE = 0;
FatalError("Could not allocate sigmatch_table");
return;
}
}
// .. register plugins here
Thanks for the bug report on Redmine, and for sharing a patch, as Victor mentioned on the Redmine ticket, would you be interested in submitting a cleaned up version of the patch to the project on Github?
