Lua rules + Lua output?

We’re experimenting with custom lua scripts for which the answer is a bit more nuanced than a 0/1 match. We anticipate calculating a confidence value.

We’re considering using Lua output scripts to try to return that data, but believe the alert rule logic lifecycle is separate from the logger lifecycle. Am I incorrect to believe that?

Are there other ways to return additional metadata that I should consider?

The last option I have in mind is to write to some sort of external key value store from our Lua script, such that we could marry up the information for subsequent post-processing. Hoping for a cleaner solution so that things more natively come together, though.

Hi @colemanserious, you are correct about the separation between detection and output wrt Lua. In detection each detection thread has an instance of a Lua_State per lua keyword in a rule. In output its a shared (between threads) Lua_State per output script. So there is no shared state between detection and output.

I think other than an external store, you can have a look at Flow ints and vars:

I’d be interested in learning more about your use case. What type of data would you like to log?

In our initial case, the metadata would be the confidence value, specifically as to whether the flow data is understood to be likely malware. We believe that will help an analyst to triage results. The confidence value would be the result of a machine learning analytic.

Beyond confidence (readily mapped to integer, assuming some concept of precision), I can envision returning malware family. That could be an integer (lookup value) or a string.

Very much appreciate the pointer to the Flow ints and vars. I had looked at them at one point, but not recognized their potential applications for our use case.