I am a big believer in repeatable builds, especially for embedded widgets. While not perfect, OpenWRT’s build system tends to produce tiny Linuxes that more or less do what I want, and it does so in a pretty reasonable way.

My Divergences

I occasionally push my changes to openwrt and its package tree to github:

ccache Without Configuration

One of my changes allows OpenWRT to grab a CCACHE_DIR environment variable if it’s set before invoking make and just use that, rather than its own. You will probably want to run something like:

PATH=/usr/lib/ccache:$PATH CCACHE_DIR=~/.ccache CONFIG_DEVEL=y CONFIG_CCACHE=y make -j8

The CONFIG_DEVEL=y CONFIG_CCACHE=y is necessary since the build system at present uses the configuration management system to decide whether to use ccache; passing it in via the environment forces its hand, but you may wish to just set these via .config.

Pogoplug V4 support

My patches include some tweaks to the upstream pogoplug v4 support. In particular, the U-Boot is more capable (see Notes on the Pogoplug Mobile / V4 for some examples) and more kinds of images are generated:

  • openwrt-kirkwood-cloudengines_pogoplugv4-squashfs-factory.bin is the image you probably want, and the one closest to what upstream builds. It is an UBI partition containing four volumes: kernel, DTB, squashfs, and ubifs. The ubifs is mounted as an overlay over the squashfs image, as is typical of OpenWRT.

  • openwrt-kirkwood-cloudengines_pogoplugv4-ext4-sdcard.img.gz is a MSDOS partition structure containing a FAT filesystem (for the kernel and DTB) and an ext4 partition for the root filesystem without overlay.

  • openwrt-kirkwood-cloudenginges_pogoplugv4-squashfs-sdcard.img is a MSDOS partition structure containing FAT (as above) and a squashfs root filesystem. OpenWRT mounts a tmpfs overlay, so the sdcard may be entirely readonly, unless steps are taken to create and mount another partition.

  • openwrt-kirkwood-cloudengines_pogoplugv4-ubifs-factory.bin is an UBI partition that does not have the squashfs component, and so does not have an overlay.

Unfortunately, because OpenWRT believes image construction is always a Cartesian product of options, there are numerous other images built that are of less utility, containing, for example, ubifs file systems inside MSDOS partition structures.

Hotplug Debugging

I find it very useful to have hotplug debugging messages from time to time. These can be achieved by landing files in /etc/hotplug.d, for example /etc/hotplug.d/net/00-debug:

#!/bin/sh
(echo -n 'Netdev: ' ; env | tr '\n' ' ') | logger -t hotplug-debug

The result will be something like this in the logs:

hotplug-debug: Netdev: USER=root ACTION=add SHLVL=1 HOME=/ SEQNUM=1234 HOTPLUG_TYPE=net IFINDEX=40 DEVPATH=/devices/virtual/net/tun-rash DEVICENAME=tun-rash LOGNAME=root TERM=linux SUBSYSTEM=net PATH=/usr/sbin:/usr/bin:/sbin:/bin INTERFACE=tun-rash PWD=/

Stunts with DNS

Tracking OpenVPN Clients

I use OpenVPN quite a bit and want to export its connection state into DNS, so that connected clients can be resolved and I don’t have to maintain multiple authoritative configuration files. This is fairly straightforward.

  1. Teach dnsmasq that it has a hostdir by adding to /etc/dnsmasq.conf

    hostsdir=/tmp/dnsmasq.d/vpnhosts

  2. Teach openvpn to write status files in version 3; in its config file(s), say

    status-version 3

  3. Use inotifyd to watch the status file; I run this under runsv-style supervision using the busybox utilities, but whatever floats your boat

    #!/bin/sh

set -e -u

HOSTSDIR=/tmp/dnsmasq.d/vpnhosts
INDIR=/tmp/run

mkdir -p ${HOSTSDIR}

lastfn=""
inotifyd - ${INDIR}:cM | { while read ev dn fn; do

  if [ "${fn}" = "${lastfn}" ]; then continue; fi
  lastfn=${fn}

  case "${fn}" in
  openvpn.foo.status) SFX=foo.example.com ;;
  openvpn.bar.status) SFX=bar.example.com ;;
  *) continue ;;
  esac

  awk -f ./stat2host.awk sfx=${SFX} < "${dn}/${fn}" > ${HOSTSDIR}/${SFX}

  killall -HUP dnsmasq

done; }

    With stat2host.awk being just

    /^CLIENT_LIST/ {
  if ($4 != "") { print $4 "\t" $2 " " $2 "." sfx; }
  if ($5 != "") { print $5 "\t" $2 " " $2 "." sfx; }
}

Adding a USB I2C RTC

I have a i2c_tiny_usb attached to my primary gateway at home and have a DS1303 RTC with battery backup hanging off the resulting I2C bus, as otherwise there’s an annoying period during boot before it’s got its interfaces up that it doesn’t have a good idea of the time. Making this fly on OpenWRT/LEDE is pretty easy, once you know how.

You’ll want to enable the drivers in .config. (If your platform isn’t already a RTC_SUPPORT OpenWRT platform, this is harder than it should be; nag upstream.) In any case, in my case, this meant:

CONFIG_PACKAGE_kmod-rtc-ds1307=y
CONFIG_BUSYBOX_CONFIG_HWCLOCK=y

The next thing to do is to land some hotplug.d scripts around. We need to walk the system through discovery, starting from the auto-scanned USB, kicking off discovery of the I2C device, and last, responding to the appearance of a new RTC. Because my RTC is apparently somewhat flaky, I have added some retry logic to the I2C and RTC parts of this dance. Anyway, without further ado: