Emitting Flow Events From a New Source

Developers
1

If you were at Suricon, you saw me mentioning a project called Bellini, where you could use Suricata from a rust application, by sharing packets using IPC. Here’s the related code for that:

I’m trying to add flow events from suricata as an output in suricata-ipc. When I shutdown suricata, I get flow messages out, but while suricata is running, I’m not getting flow messages out. Alerts and stats work like normal.

Recyclers is set to 3, managers is set to 2.

2020-04-03T20:45:23Z DEBUG broker_poc::process] Stats { decoder: Decoder { pkts: 7675, bytes: 1672792, invalid: 0, ipv4: 7659, ipv6: 0, ethernet: 7675, tcp: 6236, udp: 1423, sctp: 0, icmpv4: 0, icmpv6: 0, vxlan: 0, avg_pkt_size: 217, max_pkt_size: 1506 }, flow: Flow { tcp: 119, udp: 277, emerg_mode_entered: 0, emerg_mode_over: 0, memuse: 8048640 }, tcp: Tcp { sessions: 118, midstream_pickups:
2:48 PM 25, stream_depth_reached: 2, memuse: 6451280, reassembly_memuse: 1716268 } }

I’m also sending in a udp tracer once per minute, so would figure if nothing else, that should generate a flow message, since an alert is being generated. I’m guessing there’s something I’m missing in my source that is preventing the flow manager from seeing an appropriate timestamp for flows being expired.

The source is at src/source-ipc.c

2

Maybe I’m overlooking something, but I don’t see where the packets get a timestamp set? The capture method is supposed to set Packet::ts. This is then used in the Flow Worker to update the global time using TimeSet. The time that is set is used by the management threads, including the Flow Manager, to ‘know’ what the time is.

The runmode registers itself as ‘offline’. If the intention is to use this for live traffic that should probably be changed.

3

Setting the timestamp is somewhat convoluted. When I get the packets from IPC I callback into suricata with the timestamp, data, etc. rust/src/ipc.rs#L104 which eventually resolves to src/decode.c#L679 (forum is not liking the link to actual code right now)

There was a reason why I had it register as offline, but I no longer remember why, so I will try as live and see what happens.

4

This diff causes flow events to be emitted when they occur. Now to figure out why

diff --git a/src/runmode-ipc.c b/src/runmode-ipc.c
index ac699e1f6..5086ca666 100644
--- a/src/runmode-ipc.c
+++ b/src/runmode-ipc.c
@@ -143,7 +143,7 @@ int RunModeIpcAutoFp(void)
     SCLogDebug("server %s", server);

     RunModeInitialize();
-    TimeModeSetOffline();
+//    TimeModeSetOffline();

     /* Available cpus */
     uint16_t ncpus = UtilCpuGetNumProcessorsOnline();
diff --git a/src/runmodes.c b/src/runmodes.c
index ad72a78f3..a6feb19b6 100644
--- a/src/runmodes.c
+++ b/src/runmodes.c
@@ -526,7 +526,7 @@ bool IsRunModeOffline(enum RunModes run_mode_to_check)
         case RUNMODE_ERF_FILE:
         case RUNMODE_ENGINE_ANALYSIS:
         case RUNMODE_UNIX_SOCKET:
-        case RUNMODE_IPC:
+//        case RUNMODE_IPC:
             return true;
             break;
         default:
5

Maybe the first thing to try is see if the timestamp used by the Flow Manager (it calls TimeGet) matches what your packets have. In offline mode I mean.