gtd/
   done.otl
   next_actions.otl
   someday_maybe.otl
   waiting_on.otl
   Projects/
   Reference/
   tickler/

I’ve discussed the tickler folder before. But while I like having the next_actions.otl file, I feel it’s stopping me from really following the gtd workflow. I tend to ignore some of the things under Projects/, as I have to manually look through those to find a next action.

So I’m going to try something a bit different. I’m going to not have a next_actions file. Rather, I’ll tag items in files under Projects/ with with ‘nextaction:’, and use ~/bin/nextactions which is basically just

grep -r "^nextaction" $HOME/gtd/Projects

Now that doesn’t make me look at files not having next actions, but it keeps me working under ~/gtd/Projects rather than keeping next_actions.otl open in an editor, which otherwise is the temptation. I can also easily print out all top-level files or directories which don’t have a nextaction in or under them,

====================================================
#!/bin/bash

d=$HOME/gtd/Projects

for f in $d/*; do
	grep -rq "^nextaction" $f
	if [ $? -ne 0 ]; then
		echo $f
	fi
done
====================================================

so I can trivially see which project I’ve ignored. Yikes, 23 of them.

We’ll see, I may not like it. For instance, it may seem more awkward cut-pasting things into gtd/done.otl But I think it could improve my workflow.

I did a few tests with that parameter, and saw no instability nor performance degradation. So as of today, qemu-kvm in precise will by default enable nesting on Intel. If you want to turn it off, edit /etc/default/qemu-kvm and set KVM_NESTED=”".

The userspace qemu-kvm doesn’t need any changes to use this, however you do have to pass either ‘-cpu host’ or ‘-cpu qemu64,+vmx’ to the qemu command line options. So, for instance, I was testing with:

kvm -cpu host -drive file=x.img,if=virtio,cache=none,index=0 -m 1024 -redir tcp:2222::22

In that VM I started a nested minimal ubuntu install and compiled a tiny program, ii. With nested kvm that took 0.8s. With it (that is, without passing ‘-cpu host’ to the top level qemu), it took 2.8 seconds.

Enjoy!

For disk performance, I’m going to very closely follow Soren’s suggestion. I’ll use two upstart jobs, one which does heavy read activity (find / -type f and tar zvft of kernel tree) then shuts down, and one which does heavy write activity. Start up the kvm VM and time how long until it is complete. I intend to do all permutations of the following:

host fs: xfs, ext4, ext3, ext2, lvm, lvm snapshot
drive: qcow2, qed, raw, lvm
preallocated (for qcow and qed): yes and no
interface: virtio, ide
cache: unsafe, none writeback, writethrough, directsync
aio: threads, native

For cpu performance, I’ll have an upstart job run kernbench with permutations of the following:

guest memory: 256M, 512M, 1024M, 2048M, 4096M (*1), 8192M (*1)
overcommit: none, ksm, balloon, both
guest swap: none, =memsize, =3xmemsize
smp: 1, 2, 4 (*2), 8 (*2)

Finally, for network performance I’ll probably have an upstart job repeatedly wget kernel tarballs from apache on the host, trying the following options:

nic: tap, user
type: virtio, rtl8139, ne2k_pci, e1000, pcnet

Feedback welcome!

*1 – I won’t do these as my target machine only has 4G ram
*2 – I won’t do these as my target machine only has 4 virtual cpus

The specific test I ran was the phoronix pts/build-linux-kernel
test, which runs 3 compiles and gives you the average time and
standard error and standard deviation.

The only thing I varied was the backing store. I used a raw
partition, and lvm partition, and raw, qcow2, and qed image
files (all un-allocated, that is, sparse). The qcow2 and qed
images I tried both on ext4 and xfs filesystems. The VM in
all cases was partitioned with the default ext4 fs.

I ran the test (with three iterations) twice with each setup.

I had a few snafus with reinstallations, so hopefully next time I can do
this faster, but overall this took me about a week on one laptop with an
intel core2 duo T6670 @ 2.20Ghz. I’m hoping to have two of these in a few
weeks so I can do more tests simultaneously.

I’ll leave analysis out of this post for now, and just dump these numbers.
I know, I’d rather report mean+/-95%CI, for each set of 6, but the numbers
I’m reporting were nicely provided (as a .png) as is by the testsuite,
and I was short on time…

There is nothing new or groundbreaking here. It just records a working setup for future reference.

First, the ‘lab’. I’m sitting away from my router. There are two laptops involved. A is the big one which I use for daily work. It runs a mostly stock desktop install with unity. B is the victim. A is on my wireless network through wlan0. B is connected with a crossover cable to A. Network manager doesn’t seem happy about this, so on A I run a script which does:

	stop network-manager
	iwconfig wlan0 essid myname enc mykey
	iwconfig wlan0 ap apmacaddr
	iwconfig wlan0 freq 2.417
	ifconfig wlan0 up
	dhclient wlan0
	ifconfig eth0 10.42.43.1 netmask 255.255.255.0 up
	/etc/init.d/dnsmasq restart
	/opt/bin/share-eth0-wlan0

/etc/dnsmasq.conf has

	interface=eth0
	dhcp-range=10.42.43.2,10.42.43.100,72h
	bind-interfaces
	dhcp-boot=pxelinux.0,biglap,10.42.43.1

and /opt/bin/share-eth0-wlan0 has:

	#!/bin/sh
	# Share wlan over eth0
	echo 1 > /proc/sys/net/ipv4/ip_forward
	iptables -A FORWARD -o wlan0 -i eth0 -s 10.42.43.0/24 -m conntrack --ctstate NEW -j ACCEPT
	iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
	iptables -A POSTROUTING -t nat -j MASQUERADE
	/etc/init.d/dnsmasq start

‘biglap’ is another name for A.

Packages installed on A include: dnsmasq, nfs-kernel-server, tftpd-hpa, syslinux-common, apt-cacher-ng, and etherwake. When I do ‘sudo etherwake macaddr-of-B’ on A, B wakes up and starts pxeboot. It talks to dnsmasq on A, which tells B to pxeboot from biglap.

The install images are set up under /var/lib/tftboot as follows. First, /var/lib/tftboot/pxelinux.cfg/ has a file ‘default’ which contains:

# Default boot option to use
DEFAULT menu.c32
# Prompt user for selection
PROMPT 0
# Menu Configuration
MENU WIDTH 80
MENU MARGIN 10
MENU PASSWORDMARGIN 3
MENU ROWS 12
MENU TABMSGROW 18
MENU CMDLINEROW 18
MENU ENDROW 24
MENU PASSWORDROW 11
MENU TIMEOUTROW 20
MENU TITLE Main Menu

label linux
  kernel chain.c32
  append hd0

label precise-amd64-mini-preseed
  kernel precise-amd64-mini/linux
  append initrd=precise-amd64-mini/initrd.gz ro netboot=nfs nfsroot=10.42.43.1:/var/lib/tftpboot/precise-amd64-mini boot=casper -- priority=critical locale=en_US file=/preseed-nopart.cfg

label precise-i386-mini-preseed
  kernel precise-i386-mini/linux
  append initrd=precise-i386-mini/initrd.gz ro netboot=nfs nfsroot=10.42.43.1:/var/lib/tftpboot/precise-i386-mini boot=casper -- priority=critical locale=en_US file=/preseed-nopart.cfg

label oneiric-i386-mini-preseed
  kernel oneiric-i386-mini/linux
  append initrd=oneiric-i386-mini/initrd.gz ro netboot=nfs nfsroot=10.42.43.1:/var/lib/tftpboot/oneiric-i386-mini boot=casper -- priority=critical locale=en_US file=/preseed.cfg

For each install image, I grabbed the ‘mini.iso from archive.ubuntu.com.
For instance, for precise-amd64, I did

wget -O precise-amd64-mini.iso \

http://archive.ubuntu.com/ubuntu/dists/precise/main/installer-amd64/current/images/netboot/mini.iso

mount -o loop precise-amd64-mini.iso /mnt
mkdir /var/lib/tftpboot/precise-amd64-mini
cp -a /mnt/. /var/lib/tftpboot/precise-amd64-mini/
umount /mnt

The ‘linux’ target boots from local disk. In order for this to work, I
had to copy chain.c32 from a full install iso (there must be a better
way?) into /var/lib/tftpboot.

In order to export the nfs filesystem I added the following to /etc/exports:

/var/lib/tftpboot/iso 10.42.43.0/24(ro,subtree_check)

and did ‘sudo exportfs -a’.

I created a preseed file and copied it (called preseed-nopart.cfg) into each /var/lib/tftpboot/*-mini/ directory. Mine looks like this, obviously based on Dustin’s :

# Ubuntu Server Quick Install
# by Dustin Kirkland
#  * Documentation: http://bit.ly/uquick-doc 

d-i	hw-detect/load_firmware boolean false

# mirroring
d-i mirror/country string enter information manually
d-i mirror/http/hostname string 10.42.43.1:3142
d-i mirror/http/directory string /us.archive.ubuntu.com/ubuntu

d-i	debian-installer/locale	string en_US.UTF-8
d-i	debian-installer/splash boolean false
d-i	console-setup/ask_detect	boolean false
d-i	console-setup/layoutcode	string us
d-i	console-setup/variantcode	string
d-i	netcfg/get_nameservers	string
d-i     netcfg/get_hostname string stack
d-i	netcfg/get_ipaddress	string
d-i	netcfg/get_netmask	string 255.255.255.0
d-i	netcfg/get_gateway	string
d-i	netcfg/confirm_static	boolean true
d-i     time/zone string America/Chicago
d-i 	clock-setup/utc boolean true
d-i	clock-setup/ntp	boolean true
d-i	clock-setup/ntp-server	string ntp.ubuntu.com
d-i	base-installer/kernel/image	string linux-server
d-i	passwd/root-login	boolean false
d-i	passwd/make-user	boolean true
d-i	passwd/user-fullname string Ubuntu
d-i	passwd/username string ubuntu
d-i	passwd/user-password password ubuntu
d-i	passwd/user-password-again password ubuntu
d-i     user-setup/allow-password-weak boolean true
d-i	user-setup/encrypt-home	boolean false
d-i	apt-setup/services-select	multiselect security
d-i	apt-setup/security_host	string security.ubuntu.com
d-i	apt-setup/security_path	string /ubuntu
d-i	debian-installer/allow_unauthenticated	string false
d-i	pkgsel/upgrade	select safe-upgrade
d-i	pkgsel/language-packs	multiselect
d-i	pkgsel/update-policy	select none
d-i	pkgsel/updatedb	boolean true
d-i	grub-installer/skip	boolean false
d-i	lilo-installer/skip	boolean false
d-i	grub-installer/only_debian	boolean true
d-i	grub-installer/with_other_os	boolean true
d-i	finish-install/keep-consoles	boolean false
d-i	finish-install/reboot_in_progress	note
d-i	cdrom-detect/eject	boolean true
d-i	debian-installer/exit/halt	boolean false
d-i	debian-installer/exit/poweroff	boolean false
d-i	pkgsel/include string byobu vim openssh-server
byobu	byobu/launch-by-default boolean true
d-i debian-installer/exit/poweroff boolean true

The partitioning steps were taken out of there because I wanted to keep my partitioning on B for the moment, but I’ll be adding it again soon to make the whole thing fully automated.

Note that I set 10.42.43.1:3142, in other words laptop A, as archive mirror. I installed apt-cacher-ng on A, which also serves as mirror for all of my schroots, containers, VMs, and for that matter my other laptops, to save a bit of bandwidth from the house.

Now, with all that, I’m able to do repeated automated installs without any changes to house or lab network. For that matter, I’ll be able to do this from a coffee shop or a car tethered over 3G. You know, if I wanted to.

While all of it should be doable using a calendar, I somehow find the tickler nicer to work with. I’ve looked several times for good tools to use for a tickler, but have never found anything that suited my needs. In particular, I want to quickly be able to manipulate using text files, be able to specify events repeating based on day of week or day of month, and be able to very simply, on any platform I use, be able to move things back and forth between tickler files and my next-actions file. For instance, if I decide something on my list has to be put off, I can very quickly put it into the file for next Monday. Since I’m most proficient with vi bindings, that means I need plaintext files.

So a few years ago I whipped up the scripts (at lp:~serge-hallyn/+junk/tickler) which I still use today. The README in that tree, excerpted here, shows how I use it.

Here is how I use this. I have a folder (under gtd/) called ‘tickler. In there, I have files named 01 through 31. These have entries for that date, during the current month. If I get an action item which I want to work on on the 13th, I add it to 13. At the start of the day, I move the contents of that day’s file (i.e. 05) into my gtd/next_actions.otl file.

The tickler folder also has files named January through December. If I get an action item I want to work on in September, I add it to the September file. On September 1, I open the September file and move all items into 01-31 files (or next_actions.otl). If I know in August that I will want to do something on September 17th, then I just prefix the item in the September file with ’17: ‘.

Now here is where the scripts come in. The tickler folder also contains a folder called ‘repeating’. In there, I keep files with repeating entries. These take the following forms:

D 15 Do something on the 15th
A March 10 This happens anually on March 10
W Monday 3pm 4pm Some meeting every monday from 3-4pm

Two scripts sit in my tickler/ folder. On the first of every month, I run ‘update-repeating.sh’. This inserts relevant entries for the current month into the right 01-31 files. These entries start and end with ‘XXX’. When update-repeating.sh starts, it removes any entries which start with ‘XXX’, but keeps other (hand-inserted) entries, so that it is safe to run update-repeating.sh repeatedly.

There you go, full tickler functionality in two small scripts. I’ve used many calender apps, but for my daily activities this is still my system of choice.

For years I used irssi under multiple screen sessions to connect to irc. For the past few months, I’ve been using znc and connecting to it with various clients (xchat, colloqui, irssi, quassel, etc). But, also for several months, I’ve been wanting to try out ii. It’s a minimalistic irc client – less than 500 lines of code – by the folks who gave us dwm and wmii. It creates a directory ($HOME/irc/network-name) for each network, a directory for each channel and PM under there, and, in each of those as well as the network directories, an ‘out’ file and an ‘in’ FIFO. So, for instance, I might

irc -n nick -s irc.freenode.net

and join #ubuntu-server with

echo “/j #ubuntu-server” > ~/irc/irc.freenode.net/in

after which ii will create in and out under ~/irc/irc.freenode.net/#ubuntu-server/.

As most people I’ve mentioned this to, my first reaction was “really neat, very scriptable, but not very usable.” But the thing is, whatever tools you would like to have to make it more useable, you can trivially write them. Here are the ones I use (mostly taken from http://tools.suckless.org/ii/usage):

pcw

Pcw (“popup chat windows”) uses inotify to monitor file activity by ii. By default it creates an xterm for each channel and pm, popping up new ones if one is not already open for that channel. Each xterm runs srw (see below) to give you a nice interface to the channel.

A neat thing about pcw is that it uses an external script, pcw_win.sh, to fire off the program to interact with each channel. To change it, just write your own pcw_win.sh and make it show up first in your PATH.

For instance, when I use pcw, I use it with byobu-tmux. The pcw package at ppa:serge-hallyn/vimprobable has the script needed to do that under /usr/lib/pcw/. So I’ll open a different window for each network, and do (an alias for):

PATH=/usr/lib/pcw:$PATH pcw -n nick ~/irc/irc.freenode.net

Now each channel and PM will show up as separate panes on the same tmux window.

plain srw

If I just want to focus on one channel (for instance a meeting on #ubuntu-meeting), I run srw by hand:

	cd ~/irc/irc.freenode.net/#ubuntu-meeting
	srw -p "#ubuntu-meeting: " cw out in

Note that pcw ships with a script, cw_color.sh, which should provide an even nicer (color-highlighted) interface, but I’ve not played with it yet.

I haven’t done it yet, but am considering writing a little ‘channel bar’ gui which would list each channel along with time of last message. Clicking on a channel would bring up srw for that channel. (It’s too bad that inotify doesn’t work over sshfs).

google chat interaction

As I’ve blogged here, it’s not too much trouble having irssi or znc forward chat messages to your google chat account. However, responding via google chat, that I’ve not figured out how to do. With ii, it’s trivial! The pcw package (again at ppa:serge-hallyn/vimprobable) contains both ‘ii-to-gmail’ which forwards any private messages or channel traffic with your nick, to your google chat. The gchat-ii.pl script does the inverse! If I sent a message to my irc google chat account saying ‘i #ubuntu-server hi’, then ‘hi’ will be sent to #ubuntu-server on freenode. There is certainly room to improve – the scrips sending and receiving gchats should be consolidated into one. Then when sending gchats it could create nonces for each network/channel combination sent, which you could use in responses rather than typing the full chan name. (this is all aimed at reducing typing on my phone). But so far it’s working, and working great!

multitail+vim

I generally don’t actually run pcw, at least while working. Rather, I follow this example at nion.modprobe.de. I run multitail only on the channels I feel I need to watch, and let ii-to-gmail let me know if there are PMs or activity relevant to me on another channel. For the vimrc shortcuts, I prefix them with ‘;’ (i.e. ‘;f1′ to write the line to channel 1 in freenode). It’s actually a very nice interface!

Oh, one other thing – ii upstream doesn’t support ssl. There is a tiny patch which is supposed to add that support, but it didn’t work for me. But, that’s ok – I use it with stunnel and it works great!

All in all, I’m loving this.

I’ve had it packaged for a while now, but it had a few bugs keeping me from evangelizing it. The worst was that, frequently, clicking on an input field would not enable insert mode, and thereafter switching to insert mode seemed impossible. That is now fixed, and I see no remaining nuisances.

So, I believe the package is ready for use. It ships with an apparmor profile, and I use this config

	set homepage=file:///home/serge/homepage.html
	set plugins=false
	set scripts=false
	set java=false
	set images=true
	set caret=true
	set proxy=false
	map <C-p>=:set plugins=true
	map <C-s>=:set scripts=true
	map <C-j>=:set java=true
	set defaultsearch=d

to disable javascript and plugins by default, with ctrl-s turning javascript back on. I will probably ITP it for debian soon, but for now it is packaged at ppa:serge-hallyn/vimprobable.

At this point, I use pentadactyl (firefox) to watch tv and for launchpad and wiki interaction. That’s because ctrl-I brings a text field up in gvim, which is just too handy, and I’ve not coded that for vimprobable. But I use vimprobable for everything else.

ykpersonalize is a very powerful tool for personalizing yubikeys. However it’s always challenging figuring out how to use it. I just tried the gui tool from http://yubico.com/personalization-tool and it’s great! I picked “challenge-response mode”, “HMAC-SHA1″, generated (and stored under pasaffe) a secret key, and committed that to slot 2 on two yubikeys Then i did ‘ykchalresp -2 somepassphrase to generate a long random string like:

c7257ac92b313c98e0e77e30d02822c8a0035163

which i can prepend to hand-typed passphrases.

The things I like about this are that:

* I can re-specify the same secret key later on if I need a new yubikey
* I can store the strings resulting from ‘ykchalresp -2 passphrase’ if I ever want to stop using the yubikeys, without losing access to data protected with those keys
* if I lose the yubikey, nothing should be compromised

I need to think more about whether it’s worth it and whether there are any holes in my reasoning, of course. In the end it’ll only be worth it if it’s really more convenient than alternatives. A little pop-up gui which takes a passphrase without echoing it, feeds it to ykchalresp, and copies the output into xclip -i, and then maybe hangs around waiting for another confirmation after which it wipes it from the clipboard , might help a lot.

Heck, like this:

{{{
#!/bin/bash

# needs dialog, ykchalresp, and xclip
pw=$(dialog –passwordbox “Yubikey challenge” 10 20 –stdout)
output=`ykchalresp -2 $pw`
echo -n $output | xclip -i
echo “hit return to clear the clipboard”
read x
echo | xclip -i
}}}

I think I may encrypt some redundant data this way and test this out on
my next trip, as a test.

Luckily, it’s trivial to use google chat over irssi! It actually took me a few minutes to figure out some of the details, because noone seems to describe exactly how they do it. So here goes:

Install the xmpp irssi plugin. On ubuntu, just “sudo apt-get install irssi-plugin-xmpp”. Start up irssi, and load the plugin with “/load xmpp”. (Or you can put that into ~/.irssi/startup).

To log in, use “/xmppconnect -host talk.google.com user.name@gmail.com yourpassword”. You can see who is on with “/roster”. Then to talk with someone, say “/query other.name@gmail.com”. That’s it!