Hi Guys
I have an issue with line breaking. I used data preview in Splunk Web and it breaks line as what I wanted. But it doesn't do the trick when it deploys to props.conf in heavy forwarder.
The props.conf is
SHOULD_LINEMERGE = true
NO_BINARY_CHECK = true
BREAK_ONLY_BEFORE = \[requestID
BREAK_ONLY_BEFORE_DATE = false
CHARSET = UTF-8
MAX_TIMESTAMP_LOOKAHEAD = 12
disabled = false
TIME_FORMAT = %H:%M:%S,%3Q
TIME_PREFIX = X-Forwarded-For\=([^\.]+\.){3}\d{1,3}\]\s
#TZ_ALIAS = EST=AEST
DATETIME_CONFIG =
MAX_EVENTS = 20
pulldown_type = true
category = Application
The log sample lists below.
[requestID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
UA=Apache-HttpClient/android/SM-G900I
rcid=NA
referer=NA
node.no=1
SESSIONID=-xxxxxxxxxx
REMOTEADDRESS=xxx.xxx.xxx.xxx
X-Forwarded-For=xx.xxx.xx.xx] 15:05:31,599 DEBUG utilities.MiddlewareUtils - returning content-type = text/xml
[requestID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
UA=Apache-HttpClient/android/SM-G900I
rcid=NA
referer=NA
node.no=1
SESSIONID=-1067442995
REMOTEADDRESS=168.xxx.xxx.40
X-Forwarded-For=49.xxx.xx.73] 15:05:31,599 DEBUG connectors.ConnectorUtils - isNtlm Authentication Mode false
line count stats in search head.
Top 10 Values Count %
1 520,045 99.516%
8 2,086 0.399%
257 328 0.063%
9 28 0.005%
5 23 0.004%
4 8 0.002%
2 6 0.001%
255 6 0.001%
199 5 0.001%
177 3 0%
I used btool to check props.conf. This is what it shows.
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = \[requestID
BREAK_ONLY_BEFORE_DATE = false
CHARSET = UTF-8
DATETIME_CONFIG =
HEADER_MODE =
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 20
MAX_TIMESTAMP_LOOKAHEAD = 12
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
NO_BINARY_CHECK = true
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = true
TIME_FORMAT = %H:%M:%S,%3Q
TIME_PREFIX = X-Forwarded-For\=([^\.]+\.){3}\d{1,3}\]\s
TRANSFORMS =
TRUNCATE = 10000
category = Application
detect_trailing_nulls = false
disabled = false
maxDist = 100
priority =
pulldown_type = true
sourcetype =
I tried to change SHOULD_LINEMERGE from true to false. Splunk doesn't count each line as individual events and still parses the log in the same way. I also tried to change the sourcetype into new one. It is still the same. Please help.
↧