Evebox esimporter target index

Help
1

I am trying to move all my ES indexes under ILM (Index Lifecycle Management) either by using data streams or fudging it with write aliases.

The key thing is that you always write to the same index and ILM takes care of rolling over the index whenever it meets the criteria you have specified.

One gives evebox an index name and then (I assume) it adds the date to it – any way the indexes appear as evexbox-yyyy.mm.dd.

I need to be able to tell evebox to use evebox-write which is an alias which ES automatically points to the current index.

Is this possible?

2

Not without a code change. I’ll do this soon. Seems to make sense given how ILM works.

What EveBox does right now is more like what Logstash and Filebeat do by default, so I’m curious how they work with ILM, of they need configuration tweaks as well to work with ILM.

3

| Jason Ish ish Suricata Team Member
February 14 |

  • | - |

Not without a code change. I’ll do this soon. Seems to make sense given how ILM works.

That’s what I figured.

What EveBox does right now is more like what Logstash and Filebeat do by default, so I’m curious how they work with ILM, of they need configuration tweaks as well to work with ILM.

logstash and friends now use datastreams which play nicely with ILM.

That is the other way you could go and is much simpler to implement on the back end. For some unknown reason when you write to to a data stream you have to use ‘create’ with bulk rather than ‘index’. Which means that all clients need to be changed to work with datastreams.

https://www.elastic.co/guide/en/elasticsearch/reference/current/use-a-data-stream.html

It is a trivial change.

I my programs I had an option ‘datastream’ for each index and use it to decide wether to use index or create.

I would be happy to contribute to evebox but I have no rust experience and I don’t have enough spare cycles to pick up a new language at the moment. Of course the language is the easy bit it is the mindset that takes time and practice to develop! I have no experience programming asysc systems although I know the theory!

There are also some other reports that I would like to add/extend too :wink:

just looking at the code on github I suspect all you need to do for a datastream is change importer.rs:51 to “create”:{ if the index is a datastream.