Problems with evebox elastic-import

Help
1

I am trying to use evebox elastic-import (tried both 12.0 & 13.0-dev) against an ES server running 7.10.

What happens is that evebox does not connect to the ES server. It waits a minute and then prints an error – 60 sec timeout. Tcpdump show no connection attempt.

sensors@secmonprd10:~$ cat  /etc/evebox/es-import.yaml
elasticsearch: http://10.6.0.67:9200
index: evebox
bookmark: true
bookmark-dir: /data/sensors
bookmark-filename: /data/sensors/eve.json.bookmark
input: /data/sensors/eve.json
sensors@secmonprd10:~$ nc 10.6.0.67 9200
get
HTTP/1.0 400 Bad Request
content-type: application/json; charset=UTF-8
content-length: 203

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"}],"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"},"status":400}get
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"}],"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"},"status":400

which shows we have network connectivity to ES

sensors@secmonprd10:~$ evebox version
EveBox Version 0.13.0-dev (rev 15dadd8); x86_64-unknown-linux-musl

evebox elastic-import -c /etc/evebox/es-import.yaml -vv
2020-12-23 11:18:27  DEBUG evebox::commands::elastic_import: Testing for bookmark directory writability with file: /data/sensors/.evebox
2020-12-23 11:18:27  DEBUG evebox::commands::elastic_import: Bookmark directory is writable: directory="/data/sensors"
2020-12-23 11:19:27  ERROR evebox::commands::elastic_import: Failed to get Elasticsearch version, will try again: error=json: expected value at line 1 column 1
2020-12-23 11:20:28  ERROR evebox::commands::elastic_import: Failed to get Elasticsearch version, will try again: error=json: expected value at line 1 column 1

What am I missing?

Russell

2

Hi Russel, there does appear to be an issue with getting the elasticsearch URL out of the configuration file. A quick work-around is to provide it on the command line with a -e http://10.6.0.67:9200, then your config at least works for me.

I’ll address this in the builds a.s.a.p.

Thanks.

3

I did try that before to no avail. I have now deleted the line from the config altogether and just put the -e but it still behaves the same.

cat  /etc/evebox/es-import.yaml
index: evebox
bookmark: true
bookmark-dir: /data/sensors
bookmark-filename: /data/sensors/eve.json.bookmark
input: /data/sensors/eve.json
sensors@secmonprd10:~$ evebox elastic-import -c /etc/evebox/es-import.yaml -v -e http://10.6.0.67:9200
2020-12-23 12:32:55  DEBUG evebox::commands::elastic_import: Testing for bookmark directory writability with file: /data/sensors/.evebox
2020-12-23 12:32:55  DEBUG evebox::commands::elastic_import: Bookmark directory is writable: directory="/data/sensors"
2020-12-23 12:33:55  ERROR evebox::commands::elastic_import: Failed to get Elasticsearch version, will try again: error=json: expected value at line 1 column 1

BTW I assume you and have fqdn as well as IP addresses. First thing I tried was to change everything to IP addresses.

And what is your recommended way of getting data into evebox from about half a dozen suricata instances (given that we have a ES cluster anyway)

4

solved it! I tried running strace to see what was happening under the hood. Evebox was trying to use the external proxy! Doh!

–noproxy flag ???

5

Hmp!! I have just discovered the noproxy environment var! DOH!!