Quantcast
Channel: Questions in topic: "props.conf"
Viewing all articles
Browse latest Browse all 1485

How to dynamically route logs uto multiple indexes and sourcetypes based on file path and filename

$
0
0
Hi, I am working on OS log onboarding data under multiple hostname folders and these hostname folders are located at same file path. My plan is to dynamically onboard these logs to indexes based on relevant hostname with dynamic sourcetype set based on filename text. My logs directory structure: `\opt\myAPP\host1\filename_type1.log` `\opt\myAPP\host2\filename_type2.log` `\opt\myAPP\host3\filename_type3.log` Expected index name from foldername: `indexname_host1` `indexname_host2` `indexname_host3` Expected sourcetype name from filename : `sourcetype_type1` `sourcetype_type2` `sourcetype_type3` Following are the configuration am using at inputs.conf , where index=route is just placeholder and no such index is created: `[monitor:///opt/myAPP/.../*.log] host_segment = 3 index = route sourcetype = reroute_1 whitelist = (host1|host4|host5) [monitor:///opt/myAPP/.../*.log] host_segment = 3 index = route sourcetype = reroute_2 whitelist = (host2) [monitor:///opt/myAPP/.../*.log] host_segment = 3 index = route sourcetype = reroute_3 whitelist = (host3)` At props.conf `[reroute_1] TRANSFORMS-sourcetype = overridesourcetype1 TRANSFORMS-index = overrideindex [reroute_2] TRANSFORMS-sourcetype = overridesourcetype2 TRANSFORMS-index = overrideindex [reroute_3] TRANSFORMS-sourcetype = overridesourcetype3 TRANSFORMS-index = overrideindex ` at transforms.conf : `[overridesourcetype1] SOURCE_KEY = MetaData:Source DEST_KEY = MetaData:Sourcetype REGEX = source::\/opt\/myAPP\/\w+\/filename\_(\w+).* FORMAT = sourcetype::sourcetype_$1 [overridesourcetype2] SOURCE_KEY = MetaData:Source DEST_KEY = MetaData:Sourcetype REGEX = source::\/opt\/myAPP\/\w+\/filename\_(\w+).* FORMAT = sourcetype::sourcetype_$1 [overridesourcetype3] SOURCE_KEY = MetaData:Source DEST_KEY = MetaData:Sourcetype REGEX = source::\/opt\/myAPP\/\w+\/filename\_(\w+).* FORMAT = sourcetype::sourcetype_$1 [overrideindex] SOURCE_KEY = MetaData:Source REGEX = source::\/opt\/myAPP\/(\w+).* DEST_KEY = _MetaData:Index FORMAT = index_$1` However, all the log files are indexed into the index="indexname_host3". Is there any way to route this as mentioned under 'Expected'. Kindly help...

Viewing all articles
Browse latest Browse all 1485

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>