How to find more needles

If you're someone like me who needs to find needles in log haystacks then bash and grep are surely your friends. Here are two tricks I've come to use to make my work easier.

Making errors scream!

Okay so you have four or five terminal tabs going tailing different but related logs and the logs keep on pilling up. A script like the one I got below should be useful (given that you have headphone on, unless you getting weird looks from people around you). 

The script tails a log periodically and checks for the existence of a word(or phrase). If a match is made it plays an audio file(wave) to get your attention.

	#!/bin/bash
	#this script checks the existence of a char sequence in the last X number of lines of a files and plays a wave file if a match is made.
	# sh logchecker.sh <number of lines to tail at one go> <logfile> <char sequence> <wav file to play>
	while [ 1 -gt 0 ]; do
	tail -n $1 $2 | grep -e "$3"
	if [ $? -lt 1 ]; then
	aplay $4 ;
	exit ;
	fi
	sleep 10; #check interval
	done

view raw logchecker.sh hosted with ❤ by GitHub

Usage,

sh logchecker.sh <NO_OF_LINE_TO_TAIL> <LOG_FILE> <WORD_TO_LOOK_FOR> <WAV_FILE> 

Redirect errors to a file

You have a server with debug log level enable at the root. The logs are putting on MBs instead of KBs. Grep out what you need to a separate file to make analysis more manageable.

	#!/bin/bash
	tail -f $1 | grep -e -B $2 -A $3 $4 >> GREPOUT

view raw grepout.sh hosted with ❤ by GitHub

Usage,

sh grepout.sh <LOG_FILE> <NUM_LINES_BEFORE_MATCH> <NUM_LINE_AFTER_MATCH> <WORD_TO_MATCH>