Using Stow and Git for Config Files

Want to skip my ramblings and go straight to the meat of using stow for configurations? Click here

A year or so ago my homelab was a rather over-powered Dell R720 I had gotten my hands on from my former employer when they decommisioned a data center. I ran Proxmox on it so services were on a mix of VMs and LXC containers. I downsized the server to a HP Microserver for mostly noise reasons. Turns out stuff built for the data center have other priorities than small-ish urban appartments. Who would have thought?

In setting up everything on the new server I had a new priority - keeping things simple. My son was due in August and I figured that having stuff Just Work™ was a top priority. With that in mind I settled for a good old bare metal installation of the current Ubuntu 22.04 LTS. I’ve been a daily-ish Ubuntu user since 10.04 on both servers and laptops so at this point I’m pretty comfortable in it and I’m also all in on ZFS for file storage and management which makes Ubuntu a really nice plug&play distro for me.

So Ubuntu was to be the base, and I decided that for everything not in the base repositories I’d try to standardize on docker. This has worked out pretty well and I’d say almost all of the complexity of configuration for my server is nicely contained within a collection of docker-compose files and a few complementary service-spesific configs - all managed in a Git repository. Still, some things “have to” live outside that. Mostly the file management stuff, namely my Samba config for my local network shares, and my Sanoid config as well as the corresponding sanoid and syncoid systemd services and timers orchastrating the whole thing. Then there’s a few other bits and bobs here and there.

The correct way of managing these things is of course using something like Ansible to do the whole configuration as code thing. Machines are cattle, not pets etc. But in reality the effectiveness of manually editing a config file, testing the change, and moving on to other things when something is fixed is pretty unrivaled. I have used Ansible a bit, but the truth is that it is complete overkill for my needs. Navigating playbooks and writing yaml files just to use it as a glorified scp wrapper is just not worth it. Add to that the overhead that “doing things properly” with abstractions, variables, secret management and all that jazz and it’s clear that I’m not using the best tool for my needs.

So for a while it was just editing config files manually, doing backups of the whole system and hoping I remember what was setup and what files I need to pull from the backup the day I need to reinstall from scratch. Now usually that’s not something I would really need to think about for quite a few years, but as described in another post I’m looking at doing this sooner rather than later. Even if that wasn’t the case I’m not thrilled about not keeping track of what files have been manually edited on the system - and it’s really far from being neatly kept track of by Git as I would prefer.

Before getting to the point I’ll just mention that I’m somewhat curious about Nix and NixOS, but the aforementioned son has me firmly in “least effort” mode for the forseable future :-)

Dead simple config file management

Some time ago I stumbled upon a post describing using GNU Stow for managing dotfiles. I don’t really remember whose post but if you search for that you should find a few good resources. It dawned on me that my needs are pretty close to how a lot of people do dotfiles management. I want to define config files that somehow then ends up in the right place in the file system hierarchy.

Enter Stow

Stow seems to have been written to manage multiple versions of applications and it does that using symlinks. In essence it’s a symlink orchestrator. It puts files from one directory structure and neatly nests it into another.

Stow seems to be packaged in most directories so just go ahead and install it with your preffered package manager - for me that is apt.

sudo apt install stow

Look at my server-config directory at the time of writing:

➜  server-config git:(main) tree root
root
├── etc
│ ├── netplan
│ │ └── 01-netcfg.yaml
│ ├── samba
│ │ └── smb.conf
│ ├── sanoid
│ │ └── sanoid.conf
│ └── systemd
│     └── system
│         ├── healthcheck@.service
│         ├── sanoid-health.service
│         ├── sanoid-health.timer
│         ├── syncoid.service
│         ├── syncoid.timer
│         └── zfs_exporter.service
└── usr
    ├── local
    │ └── bin
    │     └── zfs_exporter
    └── sbin
        └── sanoid-health.sh

Now, running

sudo stow --target / root

will create symlinks to each of those files in the corresponding directory of my file system.

I can now edit my config files straight in that directory and keep track of everything with Git.

File permissions

Git doesn’t keep track of file permissions outside of the executable bit. Nor does it save the user and group owners of the files. For configs this can be a bit of a problem. There is something called git-cache-meta but this Stack Overflow post suggests another way where you create a .permissions script to restore file properties. Inspired by that I now have a Git pre-commit hook (simply a the file .git/hooks/pre-commit) that looks like this:

#!/usr/bin/env bash

echo -n "Backing-up file permissions... "

cd "$(git rev-parse --show-toplevel)"

git ls-files | sort | xargs -I {} stat --format='chmod %a %N' {} > .permissions && \
git ls-files | sort | xargs -I {} stat --format='chown %U:%G %N' {} >> .permissions

git add .permissions

echo done.

This saves the file permissions and owner information as chmod and chown commands to be applied when running the script and makes sure that it is part of the commit when adding new files.

Makefile

.PHONY: all stow-config restore-permissions

all:
        echo "Read files for commands"

stow-config:
        sudo stow --target / root

restore-permissions:
        sudo ./.permissions

I also put in a small Makefile. Usually I have them in these kinds of repositories as self documentation. I ommited actually doing anything in the all command so that I don’t accidentally seed a bunch of config files on a system without being explicit about it.

Future plans

There’s a few paths to explore here. One could be to encrypt the file contents (potentially with git-secret) so that I could just host the repo wherever. For now, just getting these files under version control is a big improvement.

Another thing to consider would be how to best apply this method to organize config files for a few different machines and also plain old dotfiles (stow has a dotfiles flag btw. Make sure to check it out if you want to use it for those!). Folders, branches or submodules are all under consideration.