Hello,
we want to index files from brtools with information about the archive log backups from oracle /oracle//saparch/arch.log.
This file has following structure and we want to create a report over all logbackups running on each system.
#* FL2 util_file aevgyzlf cds 2017-03-01 08.21.11 2017-03-01 08.21.11 1 ........... 0 0 0 0 ------- 7.20 (13) @0854895704
#
#* FL2 util_file aevgzddv cds 2017-03-01 09.03.03 2017-03-01 09.03.04 1 ........... 0 0 0 0 ------- 7.20 (13) @0854895704
#
#ARCHIVE.. 4637 /oracle/FL2/oraarch/FL2arch1_4637_854895704.dbf 2017-03-01 09.06.35 64463360 252268352 1
#SAVED.... aevgzdmi cds *1488355606 2017-03-01 09.06.48 ........... ............
#COPIED... ........ ... ................. .......... ........ ........... ............
#DELETED.. ........ ... .......... ........
#
#ARCHIVE.. 4638 /oracle/FL2/oraarch/FL2arch1_4638_854895704.dbf 2017-03-01 09.06.38 1024 252310462 1
#SAVED.... aevgzdmi cds *1488355607 2017-03-01 09.06.47 ........... ............
#COPIED... ........ ... ................. .......... ........ ........... ............
#DELETED.. ........ ... .......... ........
#
#* FL2 util_file aevgzdmi cds 2017-03-01 09.06.44 2017-03-01 09.06.49 0 ........... 4637 4638 0 0 ------- 7.20 (13) @0854895704
#
First indexing of this file works correctly.
Then Splunk monitors this file.
Now the problem:
When a logbackup starts it writes this line to this logfile:
#* FL2 util_file aevgzjpc cds 2017-03-01 10.15.32 2017-03-01 10.15.37 9 ........... 0 0 4637 4638 ------- 7.20 (13) @0854895704
#
when logbackup finished some lines will be updated
#ARCHIVE.. 4637 /oracle/FL2/oraarch/FL2arch1_4637_854895704.dbf 2017-03-01 09.06.35 64463360 252268352 1
#SAVED.... aevgzdmi cds *1488355606 2017-03-01 09.06.48 ........... ............
#COPIED... aevgzjpc cds *1488359735 2017-03-01 10.15.36 ........... ............
#DELETED.. *aevgzjpc cds 2017-03-01 10.15.36*
#
#ARCHIVE.. 4638 /oracle/FL2/oraarch/FL2arch1_4638_854895704.dbf 2017-03-01 09.06.38 1024 252310462 1
#SAVED.... aevgzdmi cds *1488355607 2017-03-01 09.06.47 ........... ............
#COPIED... aevgzjpc cds *1488359734 2017-03-01 10.15.34 ........... ............
#DELETED.. aevgzjpc cds 2017-03-01 10.15.34
#
#* FL2 util_file aevgzdmi cds 2017-03-01 09.06.44 2017-03-01 09.06.49 0 ........... 4637 4638 0 0 ------- 7.20 (13) @0854895704
#
#* FL2 util_file aevgzjpc cds 2017-03-01 10.15.32 2017-03-01 10.15.38 0 ........... 0 0 4637 4638 ------- 7.20 (13) @0854895704
#
After updating the lines Splunk wrote following message and reindex complete file.
03-01-2017 10:15:37.957 +0100 INFO WatchedFile - Checksum for seekptr didn't match, will re-read entire file='/oracle/FL2/saparch/archFL2.log'.
03-01-2017 10:15:37.957 +0100 INFO WatchedFile - Will begin reading at offset=0 for file='/oracle/FL2/saparch/archFL2.log'.
For our scenario we index only following lines (other lines are filtered on heavy forwarder)
#* FL2 util_file aevgzdmi cds 2017-03-01 09.06.44 2017-03-01 09.06.49 0 ........... 4637 4638 0 0 ------- 7.20 (13) @0854895704
#
#* FL2 util_file aevgzjpc cds 2017-03-01 10.15.32 2017-03-01 10.15.38 0 ........... 0 0 4637 4638 ------- 7.20 (13) @0854895704
#
So it is sufficient when the last line is reindexed after updating the logfile.
Our settings in inputs.conf on universal forwarder:
[monitor:///oracle/*/saparch/arch*.log]
sourcetype = logbackup_oracle
index = oracle_test
_TCP_ROUTING = splunk_main
disabled = false
Our settings in props.conf on Heavy forwarder
[logbackup_oracle]
TRANSFORMS-filter_archSID_log=setnull,pattern_archSID_log
Our settings in transforms.conf on Heavy forwarder
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[pattern_archSID_log]
REGEX = [A-Z0-9]{3} util_file
DEST_KEY = queue
FORMAT = indexQueue
What are the possibilities to index this file without reindexing after each backup? (same for database backup which can run about 6 hours or longer)
↧