AFS Server Setup¶

Miscellany¶

Software¶

apt-get install sudo openafs-fileserver openafs-client krb5-user

# Our AFS cell is acm.jhu.edu

CellServDB¶

Ideally the /afs/acm.jhu.edu/group/admins.pub/CellServDB.server file should be updated (if you’re adding a VLDB) and should be all you need server-side.

If you’re doing something funny (such as being behind a NAT, or setting up a sub-cell), you might need a custom CellServDB client-side.

keytab¶

Easiest is to copy the /etc/openafs/server/rxkad.keytab file from another server. If you do decide to grab it from the KDC, please ensure that you use the -norandkey argument to the xst command, or else all the other servers will be broken.

UserList¶

For update instructions, see AFS.

Todo

We should be hosting the UserList on our highly-replicated group.admins.pub volume; instead, it’s still local on each machine. It’s possible that something like the below could be made to work reliably?

Note that because the VLDB servers will consult the UserList as part of their operation, the VLDB servers (and only the VLDB servers) need to down-weight themselves in their own client’s operation (so that callback breaks, but especially callback breaks for the group.admins.pub volume, result in them asking another VLDB). This can be accomplished by ensuring that a call like fs setserverprefs -vlservers `hostname` 50000 happens on client startup.

Note

If the VLDB server’s AFS client knows of its own VLDB server on a different address (e.g. localhost) then the use of hostname above should change.

NetInfo¶

If the host is behind NAT, it needs a NetInfo file. This, oddly enough, is NOT in /etc/openafs/server, but rather /var/lib/openafs/local. The contents in some alternate world should probably be
private.address.dotted.quad
f public.address.dotted.quad
but in this one I think the right answer is
f public.address.dotted.quad
You can check that the right thing happened, once the server is up, with
vos listaddrs -cell acm.jhu.edu -printuuid -noresolve
There appears to be no easy way to get the uuid, but scan
od -h /var/lib/openafs/local/sysid

NetRestrict¶

If your host is going to listen on addresses that you do not wish it to publish, you must enumerate each address in /var/lib/openafs/local/NetRestrict.

Note that BeagleBones do this by default, as they make private addresses for their USB gadgets, and so will definitely need a NetRestrict file if they are functioning as AFS servers.

Configure the server using BOS¶

PRDB

bos create `hostname` ptserver simple \
    /usr/lib/openafs/ptserver -localauth

VLDB

bos create `hostname` vlserver simple
    /usr/lib/libexec/openafs/vlserver -localauth

File server

bos create `hostname` dafs dafs `
         /usr/lib/openafs/dafileserver  \
         /usr/lib/openafs/davolserver \
         /usr/lib/openafs/salvageserver \
         /usr/lib/openafs/dasalvager -localauth

remctld and afs-backend¶

Install the AFS::PAG perl module; it should be as simple as:

apt-get install libafs-pag-perl

Grab /afs/ir.stanford.edu/service/afs/scripts/vol*, http://archives.eyrie.org/software/afs/afs-backend-acl, and http://archives.eyrie.org/software/afs/afs-backend.

Modify afs-backend-acl to set:

$ACL      = '/afs/acm.jhu.edu/readonly/group/admins.pub/afs-backend.acl';
$REMCTL   = '/etc/remctl/acl/afs-backend';
$DOMAIN   = 'acm.jhu.edu';
$K5_REALM = 'acm.jhu.edu';

Patch pts_expand to pass -expandgroups to pts, as we use supergroups in our cell. Patch remctl_acl_write to be

sub remctl_acl_write {
    my ($fh, @users) = @_;
    for (@users) {
        my ($princ, $realm) = split /@/;
        $realm = $K5_REALM if not defined $realm;
        $princ =~ s%^rcmd\.%host/%;
        if ($princ =~ m%^(host|webauth)/(.+?)(|\.?$DOMAIN)$%) {
            $princ = "$1/$2.$DOMAIN";
        } else {
            $princ =~ tr%.%/%;
        }
        print $fh "$princ\@$realm\n";
    }
}

Modify afs-backend

$ENV{KRB5CCNAME} = '/tmp/krb5cc_afs-backend';
$ACL        = '/afs/acm.jhu.edu/readonly/group/admins.pub/afs-backend.acl';
$AKLOG      = '/usr/bin/aklog';
$REALM      = 'acm.jhu.edu';
@RULES      = ( );
$VOLCREATE  = '/root/bin/volcreate';
$VOLNUKE    = '/root/bin/volnuke';
$VOLRELEASE = '/root/bin/volrelease';

Patch pts_expand to again pass -expandgroups.

Patch out the use AFS::Utils in favor of the supported AFS::PAG. Only the use line needs to change.

Comment out $ADDRESS and the various lines for manipulating the MAIL file handle, because we don’t want to get that much mail.

Add to /etc/inetd.conf the line:

remctl stream tcp nowait root /usr/sbin/tcpd /usr/sbin/remctld

Drop a k5start runit service in /etc/service:

mkdir /etc/sv/k5start_afs-backend
cat <<HERE >/etc/sv/k5start_afs-backend/run
#!/bin/sh
exec k5start -U -f /etc/krb5.keytab -k /tmp/krb5cc_afs-backend -K 240
HERE
chmod +x /etc/sv/k5start_afs-backend/run
ln -s /etc/sv/k5start_afs-backend /etc/service

And make sure that the host is in the UserList and all that.

Other Useful References¶

Take a look at the Openstack AFS notes: http://docs.openstack.org/infra/system-config/afs.html.