I have events that look like the following:
1pjxVfF7i84nvqrD4p24UVa|2019-05-14 20:41:04.035:[0:T][T1847][PaymentMethodLogoRepositoryImpl][1300][]Fetch logo (consulate_0704c4eb6fb5)
1pjxVfF7i84nvqrD4p24UVa| paymentMethod=Interac
1pjxVfF7i84nvqrD4p24UVa| countryCode=CA
Note the repetition of "1pjxVfF7i84nvqrD4p24UVa|" for every line of the log - sometimes the events are hundreds of lines long. The repetition is wasteful noise.
I want to extract the repeated value into a variable (e.g. transaction_id="1pjxVfF7i84nvqrD4p24UVa". I've used the following for that in props.conf and a SEDCMD to strip the data from _raw
[cbms_merchant_logs]
EXTRACT-transaction_id = ^(?\w{23})\|\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{3}:
SEDCMD-strip-transaction-id = s/\w{23})\|//g
From what I understand, the SEDCMD executes first so the data won't be available for the EXTRACT.
How can I achieve the goals of key-value EXTRACTION and SEDCMD substitution? Multiple TRANSFORMS perhaps? Examples are appreciated.
↧