Hi,
I’m trying to create a new source type for the first time. I’ve been at it all morning and I’m pretty sure I must be missing something fundamental.
The data I’m importing is quite a messy log file. Some events contain XML spread across multiple lines, and where I’m having trouble is keeping the multiline events as one event.
To make life easier (I hope), I’ve added a new column (called “blah”) to the beginning of each event so it’s easier to tell when a new event starts. To help troubleshoot, I’ve created this very simple dummy log file:
blah data
blah a single line 1
blah multiple lines one
multiple lines two
multiple lines three
blah a single line 2
This should appear as a header with two columns called “blah” and “data”, there are three events:
Event 1:
blah a single line 1
Event 2:
blah multiple lines one
multiple lines two
multiple lines three
Event 3:
blah a single line 2
Here’s what the event type looks like in props.conf:
[fsv]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = blah
pulldown_type = true
INDEXED_EXTRACTIONS = tsv
FIELD_DELIMITER=tab
HEADER_FIELD_DELIMITER=tab
KV_MODE = none
category = Structured
description = Fancy seperated value format. Set header and other settings in "Delimited Settings"
I’ve also tried various configurations with LINE_BREAKER but everything I’ve tried still results in one event for each line.
↧