Hi,
I am working on OS log onboarding data under multiple hostname folders and these hostname folders are located at same file path.
My plan is to dynamically onboard these logs to indexes based on relevant hostname with dynamic sourcetype set based on filename text.
My logs directory structure:
`\opt\myAPP\host1\filename_type1.log`
`\opt\myAPP\host2\filename_type2.log`
`\opt\myAPP\host3\filename_type3.log`
Expected index name from foldername:
`indexname_host1`
`indexname_host2`
`indexname_host3`
Expected sourcetype name from filename :
`sourcetype_type1`
`sourcetype_type2`
`sourcetype_type3`
Following are the configuration am using at inputs.conf , where index=route is just placeholder and no such index is created:
`[monitor:///opt/myAPP/.../*.log]
host_segment = 3
index = route
sourcetype = reroute_1
whitelist = (host1|host4|host5)
[monitor:///opt/myAPP/.../*.log]
host_segment = 3
index = route
sourcetype = reroute_2
whitelist = (host2)
[monitor:///opt/myAPP/.../*.log]
host_segment = 3
index = route
sourcetype = reroute_3
whitelist = (host3)`
At props.conf
`[reroute_1]
TRANSFORMS-sourcetype = overridesourcetype1
TRANSFORMS-index = overrideindex
[reroute_2]
TRANSFORMS-sourcetype = overridesourcetype2
TRANSFORMS-index = overrideindex
[reroute_3]
TRANSFORMS-sourcetype = overridesourcetype3
TRANSFORMS-index = overrideindex
`
at transforms.conf :
`[overridesourcetype1]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype
REGEX = source::\/opt\/myAPP\/\w+\/filename\_(\w+).*
FORMAT = sourcetype::sourcetype_$1
[overridesourcetype2]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype
REGEX = source::\/opt\/myAPP\/\w+\/filename\_(\w+).*
FORMAT = sourcetype::sourcetype_$1
[overridesourcetype3]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype
REGEX = source::\/opt\/myAPP\/\w+\/filename\_(\w+).*
FORMAT = sourcetype::sourcetype_$1
[overrideindex]
SOURCE_KEY = MetaData:Source
REGEX = source::\/opt\/myAPP\/(\w+).*
DEST_KEY = _MetaData:Index
FORMAT = index_$1`
However, all the log files are indexed into the index="indexname_host3".
Is there any way to route this as mentioned under 'Expected'.
Kindly help...
↧
How to dynamically route logs uto multiple indexes and sourcetypes based on file path and filename
↧