Debian ¶

Installer Inside QEMU nographic ¶

This ought to have been obvious, but I’ve never seen it written down anywhere. Debian uses GNU screen to provide multiple panes on the installer interface (the TUI, shells, logs, &c); screen uses Ctrl-A as its command prefix. When QEMU is multiplexing the guest serial stream and its monitor, it also uses Ctrl-A as its command prefix. You’ll thus need to use Ctrl-A Ctrl-A 2 to switch to the shell in the installer, for example.

Upgrade With Low Disk Space ¶

The official recommendation is to grab additional storage and temporarily put /var/cache/apt/archives there. Something like the following may also help; the goal here is to upgrade the packages that are already marked as manually installed so that other packages do not get so marked. This may reduce the disk requirements for a subsequent upgrade.

apt-mark showmanual > amsm
apt-get --just-print upgrade \
  | awk '/^Conf/{ print $2 }' \
  | (while read pkg; do grep -e "^$pkg\$" amsm; done) \
  | (while read pkg; do sudo apt-get -y install $pkg; sudo apt-get clean; done)

QEMU Virtio 9P Root ¶

Now there’s a phrase, isn’t it? This is going to give us a chroot-like experience, whereby files are directly accessible on the host (though probably best not done concurrently with the guest running) but the system is running inside qemu. This avoids the need to make disk images on the host and may make it easier to maintain long-term?

To do this properly, we’d build a custom debian-installer, but that’s really excessive just to see if it works. So, to fake it, we’re going to cheat. Go grab the debian netinst kernel (vmlinux or vmlinuz) and initramfs (initrd.gz) files from, e.g. https://www.debian.org/releases/stretch/debian-installer/. List the initramfs contents and figure out which kernel version is packaged therein:

KREV=$(zcat initrd.gz | cpio -it 2>/dev/null | sed -ne 's/lib\/modules\/\(.*\)\/kernel/\1/p' | head -n 1)
echo ${KREV}

Go grab the corresponding Debian kernel image and unpack it:

dget linux-image-${KREV}
dpkg --extract linux-image-${KREV}*.deb kernel

Now, append the 9p kernel modules to the initramfs archive; the virtio modules are, thankfully, already included. Note that we don’t extract the archive and repack it because it contains device nodes, and mknod might be beyond our reach.

gunzip initrd.gz
(cd kernel; find lib -name fscache\* -o name 9p\* | cpio -o --format newc --append -F ../initrd)
gzip initrd

And then boot the installer’s kernel and our modified initramfs and a 9P virtio device; priority=low boots the installer to expert mode:

qemu-system-... ... \
  -initrd initrd.gz -kernel vmlinux -append priority=low \
  -fsdev local,id=root,security_model=mapped-xattr,path=$PWD/root \
  -device virtio-9p-pci,fsdev=root,mount_tag=root

Walk the installer to the point of partitioning disks, which will fail to detect any and “Finish setting up partitions”, which will fail. Now, at the installer shell, mount the system:

depmod
modprobe 9p
modprobe 9pnet_virtio
mkdir /target
mount -t 9p root /target

While ordinarily the installer will balk at the idea of not configuring the disks, since we are running in expert mode, you should be back at the main menu and should be able to select “Install base system” and go from there. The installer will go through almost everything now just fine.

When asked, tell the system to build a “generic” initramfs because otherwise it’s going to fail to find the modules for the root filesystem. After that, press enter repeatedly to finish the installation. Before shutting down, run, at the installer shell:

echo 9p           >> /target/etc/initramfs-tools/modules
echo 9pnet_virtio >> /target/etc/initramfs-tools/modules
echo virtio_pci   >> /target/etc/initramfs-tools/modules

sed -i -e 's/^MODULES=.*$/MODULES=list/' /target/etc/initramfs-tools/initramfs.conf

mount -o rbind /sys  /target/sys
mount -o rbind /proc /target/proc
mount -o rbind /dev  /target/dev
chroot /target update-initramfs -k all -u

Then go ahead and halt the installer.

When booting the system next, you’ll tell qemu to boot from the in-filesystem kernel and initramfs; unfortunately, symlinks are mapped to plaintext files, so there’s a little indirection, and the initramfs needs to be told what’s going on (via the -append option here):

qemu-system-.. ... \
  -initrd "$PWD/root/boot/$(cat $PWD/root/boot/initrd.img)" \
  -kernel "$PWD/root/boot/$(cat $PWD/root/boot/vmlinux)" \
  -append "rootfstype=9p root=root rw" \
  -fsdev local,id=root,security_model=mapped-xattr,path=$PWD/root \
  -device virtio-9p-pci,fsdev=root,mount_tag=root

Ubuntu ¶

Live CD Persistence By Default ¶

Sometimes we want to boot systems persistently by default. Yes, yes, we’re weird. Grab the ISO; for the purposes of this exercise, we used lubuntu-15.10-desktop-amd64.iso. Extract the boot menu configuration:

osirrox -indev lubuntu-15.10-desktop-amd64.iso -cdi /isolinux -extract_single ./txt.cfg txt.cfg

Apply this diff to remove the install option and append persistent:

--- txt.cfg.orig    2015-10-21 12:39:29.000000000 -0400
+++ txt.cfg 2016-02-03 04:37:12.442310613 -0500
@@ -2,11 +2,7 @@
 label live
   menu label ^Try Lubuntu without installing
   kernel /casper/vmlinuz.efi
-  append  file=/cdrom/preseed/lubuntu.seed boot=casper initrd=/casper/initrd.lz quiet splash ---
-label live-install
-  menu label ^Install Lubuntu
-  kernel /casper/vmlinuz.efi
-  append  file=/cdrom/preseed/lubuntu.seed boot=casper only-ubiquity initrd=/casper/initrd.lz quiet splash ---
+  append  file=/cdrom/preseed/lubuntu.seed boot=casper initrd=/casper/initrd.lz quiet splash persistent ---
 label check
   menu label ^Check disc for defects
   kernel /casper/vmlinuz.efi

Repack the ISO image:

xorriso -indev lubuntu-15.10-desktop-amd64.iso -outdev custom.iso -boot_image isolinux keep -cdi /isolinux -cpr txt.cfg .

At boot, now, the livecd will scan for (among other options) a filesystem whose label is casper-rw; you can make such a thing by running, within the live system, for example:

mkfs.ext4 -L casper-rw /dev/sda1

File Transfer ¶

Help, all I have is a shell ¶

Pipe the output of this to your shell. This is exceptionally not fast

hexdump -e '"echo -e '\''" 120/1 "Y%03o" "'Z\'' >> xetc.tgz\n"' xetc.tgz \
  | sed 's;Y;\\0;g;s/Z/\\c/;s/\\0 *\\0/\\c/;1s/>/ /'

SLIP ¶

Ah, the Serial Line Internet Protocol. Here’s a worked example. On the host:

slattach -L -l -n -s 38400 -p slip /dev/tty_dgrp_a2_6
ifconfig sl0 172.29.8.1 pointopoint 172.29.8.2
echo 1 > /proc/sys/net/ipv4/conf/sl0/forwarding
iptables ...

On the guest:

slattach -L -d -m -p slip -s 38400 /dev/ttySC1 &
ifconfig sl0 172.29.8.2 pointopoint 172.29.8.1

ZFS ¶

Large File Deletion ¶

ZFS would occasionally stall a machine if you ask it to delete a large file from a deduplicated data set as it would have engage in huge transactions involving the DDT. Now that https://github.com/zfsonlinux/zfs/issues/3725 is fixed, this should almost certainly not matter; the code here is for historical reference only.

You can hold its hand and slow the process down, which should eliminate IO stalls:

for i in `seq $(($(stat -f %z $FILE)/1024/1024)) -1 1`; do  \
        echo $i; \
        truncate -s $((i*1024*1024)) $FILE; \
        sync; \
        sleep 5; \
done; \
rm $FILE

UNIX Shell ¶

Software Watchdog With runit ¶

runit / daemontools / s6 and friends can be used as a kind of watchdog to detect the absence of some event. Use sv restart ... or the equivalent to tickle the watchdog and delay the event, perhaps in another supervised script!

./run creates a new process group:
```
#!/bin/sh
exec setsid ./run2
```
./run2 traps SIGTERM to exit and waits for a timeout before invoking the watchdog response:
```
#!/bin/sh
onTerm() { trap '' TERM; kill -TERM 0; exit; }
trap onTerm TERM

sleep $(cat ./watchdog-period) &
wait
trap '' TERM
exec ./watchdog-fire
```
Where ./watchdog-period contains the interval in seconds during which the watchdog must be reset, and ./watchdog-fire is the script to run when it’s all gone south.

Line-based Parallel Mapping in Shell ¶

“I was nerd-sniped” is really the only defense I have for this particular section’s existence. It’s something of a response to http://catern.com/posts/pipes.html, which goes into a fair bit of detail and to moderate lengths to work around not having message boundaries respected by UNIX pipes. That approach didn’t sit well with me, aesthetically, and I guess I was wanting to get to know my shell better.

Anyway, given a shell pipeline

source | mapper | sink

if mapper’s action is dependent only on each line, but takes a while, you might wish to have multiple mappers running in parallel. For various reasons, existing tools like GNU parallel and xargs are not sufficient (as of this writing). Thus, I present fanout.sh which uses zsh coproc-esses and the zsh/zselect to manage a flock of workers.

Use as in this self-test:

diff <(./fanout.sh 3 cat < /usr/share/dict/words | sort) <(sort /usr/share/dict/words)

If we pass a slow worker, we can see the right thing happening as the pipes first fill to their internal buffer sizes and then things start blocking:

$ ./fanout.sh 3 ./scat.sh < /usr/share/dict/words
A
Iroquoian's
Sutton's
A's
Iroquois
Suva
AA's

Git ¶

Packing Milestones ¶

Left to its own devices, git will occasionally repack the entire repository, coalescing all objects into a single pack. This entails repacking old revisions every time, which, for large repositories, is unlikely to do anyone any good. You can create “kept” pack files containing objects that will not be reconsidered in the future, reducing the churn. A good way to do this is to pick a “milestone” commit, something that certainly won’t go away in the future, and pack everything transitively reachable therefrom. In zsh, this might be as simple as

(B=origin/master P=.git/objects/pack/pack H=$(git pack-objects --honor-pack-keep --revs $P <<< $B) ; echo "$B $(date +%Y%m%d)" > $P-$H.keep)

This command can be rerun whenever the churn set gets too large. If this is the only source of kept packs (git does not create them by default), then the repository has the nice property that its set of kept packs are transitively closed, which (hopefully) will become a useful fact in future versions of git.

Kerberos & AFS ¶

Persistent AFS Tokens for Daemons ¶

If a daemon runs as a particular UID, it’s relatively straightforward to use the AFS Unix CM’s default UID-based PAG to ensure that that service has authenticated access to AFS, even without that UID having access to the Kerberos keytab. For example, using runit and k5start, a run script of

#!/bin/sh
exec keyctl session - $PWD/run2

creates a new “session” kernel keyring and runs run2

#!/bin/sh
export KRB5CCNAME=KEYRING:session
export AKLOG=$PWD/aklog.sh
exec k5start -F -P -K 240 -f /etc/krb5/service.keytab -U -t

which configures Kerberos to use said session keyring for ticket storage and uses k5start to get tickets from the service.keytab file. The aklog.sh script then enters the target UID PAG, by setting its UID, but brings the session and environment variables with it:

#!/bin/sh
exec chpst -u ${service_uid} aklog

Replace ${service_uid} appropriately.

nwf's Ramblings

Miscellaneous Bits of System Administration