I have a log that contains multi-line events, some events contain java stack traces. Here is an example log:
INFO 2017-02-06 17:57:36,026 com.loadbalancer.http.SynchHttpProducer [82] - Details of successful transactions:
2017-02-06 17:57:35, 727 [SECU] AUTH date:2017-02-06PST17:57:35-0800 login:loging status:SUCCESS
2017-02-06 17:57:35, 727 [SECU] AUTH date:2017-02-06PST17:57:35-0800 IP:fqdn login:login status:SUCCESS
INFO 2017-02-06 17:57:39,111 com.targethealthcheck.HeartbeatHealthCheckService [374] - TargetHost for IP fqdn marked as AVAILABLE
INFO 2017-02-06 17:58:05,524 com.monitoring.AbstractPrinter [90] - [STAT][PING]
[STAT][PING] Checked by Manager : true
[STAT][PING] Checked by Load Balancer : false
INFO 2017-02-06 17:58:05,525 com.alu.e3.monitoring.AbstractPrinter [90] - [STAT][PROV]
INFO 2017-02-06 17:58:05,526 com.alu.e3.monitoring.AbstractPrinter [90] - [STAT][TOPO]
WARN 2017-02-06 17:58:05,543 com.monitoring.AbstractPrinter [94] - Error during printing stats for category SYNCHRO
java.lang.NullPointerException
at com.monitoring.StatMaker.getVariance(StatMaker.java:89)[494:e3-bundle:6.4.4.1]
at com.monitoring.StatMaker.getSigma(StatMaker.java:101)[494:e3-bundle:6.4.4.1]
at com..monitoring.StatMaker.toString(StatMaker.java:136)[494:e3-bundle:6.4.4.1]
at java.lang.String.valueOf(Unknown Source)[:1.7.0_79]
at java.lang.StringBuilder.append(Unknown Source)[:1.7.0_79]
at com.monitoring.SpeakerSynchronizationTime.logStatsGateway(SpeakerSynchronizationTime.java:235)[494:e3-bundle:6.4.4.1]
For this I used the following props.conf:
[ngeag_e3]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
TRUNCATE = 0
MAX_EVENTS = 2048
BREAK_ONLY_BEFORE = FATAL[\s]+|ERROR[\s]+|INFO[\s]+|WARN[\s]+|DEBUG[\s]+
DATETIME_CONFIG =
TIME_FORMAT = "%Y-%m-%d %H:%M:%S,%3N"
TIME_PREFIX = FATAL[\s]+|ERROR[\s]+|INFO[\s]+|WARN[\s]+|DEBUG[\s]+
MAX_TIMESTAMP_LOOKAHEAD = 23
category = Custom
description = Log file containing AME events
disabled = false
pulldown_type = true
LINE_BREAKER with SHOULD_LINE_MERGE = false did not work for me because the LINE_BREAKER would be:
LINE_BREAKER = (FATAL[\s]+|ERROR[\s]+|INFO[\s]+|WARN[\s]+|DEBUG[\s]+)
This would remove the log type (FATAL|ERROR|INFO|WARN|DEBUG) and also removes the time prefix such that the timestamps don't work. The above config works mostly but my events are sporadically being split unexpectedly. When I show source, I will see strange modifications to the log where the log line gets split mid word:
INFO 2017-02-06 17:57:36,026 com.loadbalancer.http.SynchHttpProducer [82] - D
etails of successful transactions:
2017-02-06 17:57:35, 727 [SECU] AUTH date:2017-02-06PST17:57:35-0800 login:loging status:SUCCESS
2017-02-06 17:57:35, 727 [SECU] AUTH date:2017-02-06PST17:57:35-0800 IP:fqdn login:login status:SUCCESS
INFO 2017-02-06 17:57:39,111 com.targethealthcheck.HeartbeatHealthCheckService [374] - TargetHost for IP fqdn marked as AVA
ILABLE
INFO 2017-02-06 17:58:05,524 com.monitoring.AbstractPrinter [90] - [STAT][PING]
[STAT][PING] Checked by Manager : true
[STAT]
[PING] Checked by Load Balancer : false
Not only is the line getting split, but it is not merging into the prior event. Any ideas why the lines are getting split or why they are not getting merged?
↧