How to get a random line from a file in bash.
Posted: May 2, 2011 | Author: Hilary Mason | Filed under: blog | Tags: bash, code, one-liner, remember, tips | View Comments
I work with a lot of data, and while I’d like to pretend it’s all in upside-down quasi-indexed b-tree rocket ships or some other advanced database, the truth is that much of it is in text files. I often find myself wanting to see a random line from one of these files, just to get a sense of what the data looks like.
I thought there must be an easy bash way to do this, but I couldn’t find it (‘shuf’ isn’t installed on my server), so I turned to twitter, and now I’m pleased to present more methods for finding a random line than you ever expected!
sort -R | head -n 1
If you can use this, do so! If it isn’t available, consider one of the following commands:
@andrewgilmartin suggests using awk:
awk 'BEGIN { srand() } rand() >= 0.5 { print; exit }'
@devinteske offered one of the easiest to solutions to read:
tail -$((RANDOM/(32767/`wc -l</etc/group|tr -d ' '`))) /etc/group|head -1
@terrycojones piped up with this gem:
split -l 1 < file; cat `for i in x*; do echo $RANDOM $i; done | sort -n | cut -f2 -d' ' | head -n 1`; rm x*
@FirefighterBlu3 does sed++:
file=/etc/passwd; lc="$(($RANDOM % $(wc -l $file|awk '{print $1}')))"; sed -n "${lc}p" $file
@burleyarch collects the whole set:
f=YOUR_FILE; n=$(expr $RANDOM \* `cat $f | wc -l` \/ 32768 + 1); head -n $n $f | tail -1
All of the options using $RANDOM should be used with the understanding that the max possible value is 32767, so it will only be random on files that have fewer than 32,767 lines.
@xn with an excellent use of cut:
awk 'BEGIN { OFS="\t"; srand() } { print rand(), $0 }' | sort -n | cut -f2- | head -1
@paulrbrown with a badass example of od:
echo `cat /dev/urandom | od -N4 -An -i `' % '`wc -l < file` | bc | sed 's/-//g' | xargs -I % head -n % file | tail -n 1
And finally, from @alexlines, who actually developed his solution into a blog post:
dd if=file skip=$(expr $(date +%N) \% $(stat -c "%s" file)) ibs=1 count=200 2>/dev/null|sed -n '2{p;q;}'
And, of course, @ceonyc brought some comic relief:
@hmason Good bash one-liner? Take my code, please.
Hilary Mason 