GaSiD.org.uk

When scripting Bash, i’ve often come across problems where using an associative array would make things so much cleaner. Unfortunately Bash has never supported then…. Until now.

With Bash4 I can do this:

#!/bin/bash
declare -A info
info[blah]="Blah"
info[test]="Test"
echo ${info[blah]}
echo ${info[test]}

As you can see, associative arrays are where you use strings as the index of the array, rather than integers. Very useful.

Each time a specific message appeared in a log, I needed to pull out all lines from that same log file that started with a specific identifier. It seemed simple at first, until you realise (or rather someone else tells you!) that the default C library buffers more data when stdin is a pipe rather than your console.

Now usually you wouldn’t notice this as you’d read in a file of a fixed size, you would eventually get an EOF if you don’t reach the limit on the buffer size, then you’d see your data. But if you’re not getting a significant amount of data through, your command can sit there seemingly outputting nothing.

Anyhow, the way to fix it seems to be to tell grep to use line buffering and tell awk to flush its output every time we output something.

tail -f $logfile |
 grep --line-buffered -F "Invalid Response from Server" |
 awk -F '#' '{ fflush(""); print $1 }' |
 while read line; do
  grep -F ${line}# $logfile;
 done