Line buffering (and talking computers and irc meetings)

When I was a kid, I wanted computers to talk to me (and vice versa). Why, decades later, am I still squinting at the screen during hour-long irc meetings? Basta! So the last two weeks I’ve been listening to our hour-long team irc meeting.

There is an excellent screen reader integrated with unity (well, it needs some tweaking to speed up the voice, which you can do in /etc/speech-dispatcher/speechd.conf), but it’s designed for a different purpose: every time you foreground a window it will switch to reading from that window. I want something simpler – I want a stream of text to be converted to speech as it comes in, while I play on other windows and desktops without the speech being affected. So the way I’ve decided to do it is as follows. First I connect using ‘sic’ (the ‘simple irc client’ from to the irc server. Sic uses simple text files for input and output, with all output (for all channels) going to the same file. So I start it as:

   rm -f; touch; tail -f | sic -h -n lurker | tee -a sic.out

and in one window I have a loop into which I can type:

    while read l; do [ -z "$l" ] && break; echo "$l" >>; done

Then I use the following to make it speak:

    tail -n 2 -f ~/sic/sic.out | egrep --line-buffered -v 'ChanServ|TOPIC|JOIN|PART|QUIT|ubottu' | awk -F\  '{ $1=""; $2=""; $3=""; print; fflush(); }' | espeak --stdout -s 220 -ven-us | aplay 2>/dev/null

All this does is filter out some of the stuff I don’t want to hear. (Yes the grep is “superfluous”, deal with it 🙂 But the part which I found interesting enough to blog about was the extra steps in egrep and awk. By default, they do not use line buffers on output, because that could harm performance in normal use. But for what I’m doing, I need every line to be buffered to trigger the next command in the pipeline.

In egrep, this is done simply using –line-buffered. In awk, I do it by adding a fflush() after the print command. I actually had to spend some time looking into this, so I thought I’d share my results.

As a simple example, try the following:

	tail -f /etc/hosts | while read line; do echo $line; done

Does what you’d expect. Now insert awk into the pipeline to print only the hostname:

	tail -f /etc/hosts | awk '{ print $2 }' | while read line; do echo $line; done

There’s no output. That’s because awk is buffering and hasn’t yet flushed its stdout. So the while loop has no output. You can fix this by forcing it to fflush(), as in:

	tail -f /etc/hosts | awk '{ print $2; fflush(); }' | while read line; do echo $line; done

Now there is output.

So there you go. If you want to use something like irssi (which I also have a script for, to listen to hilights window output), you just need to do some tweaking to the grep and awk commands which drop the cruft.

Have fun!

This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to Line buffering (and talking computers and irc meetings)

  1. Amit Kucheria says:

    Very cool. One of those “why didn’t I think of this” ideas. Can you post the irssi script someplace? I’ll give it a go.

    • s3hh says:

      The irssi script is a bit custom to my situation… I won’t mention the ssh oddness I had to throw in. But I do run the hilight script at, which I modified to also write everything it gets into a file ‘hilight.log’. Then I run

      tail -n 2 -f irclogs/hilight.log | sed -u -e ‘s/[^[:print:]]//g’ -e ‘s@8/<gg e@ @g’ | awk -F\ ‘{ print; fflush(); }’ | espeak –stdout -v en-us -s 220 | aplay 2>&1

      You see the ugliness 🙂 I may go back to using pure sic everywhere, or use znc with local sic. (but not right now)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s