Deploy Your Applications With Watchtower
By hernil
Automated deploys, continous integration and continous deployments are neat but can often be somewhat complex to set up just right. That overhead might be a bit much - especially in a homelab environment.
So why not simplify the process and KISS? Let’s explain some background, or jump straight down to the answer!
Watchtower 101
Watchtower describes itself as:
A container-based solution for automating Docker container base image updates.
and is often already deployed in a homelab environment to help keep services up to date. Either automatically or by monitoring and notifying. A typical docker-compose config might look like this:
version: '3'
services:
watchtower:
image: containrrr/watchtower
container_name: watchtower
restart: always
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /etc/timezone:/etc/timezone:ro
command: --schedule "0 0 6 * * *"
environment:
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_LABEL_ENABLE=true
- WATCHTOWER_INCLUDE_RESTARTING=true
labels:
- com.centurylinklabs.watchtower.enable=true
On you various services you can add the following label and voila - your service should be automatically updated once a day at 6.
- com.centurylinklabs.watchtower.enable=true
There’s a bit more to it than that, but not much. Check out the complete docs. Watchtower is actually pretty dumb. But dumb often means simple and simple means easy.
Okay, but what about my own applications?
Running your own applications in a homelab environment is not really a must. You can easily just benefit from the amazing world of open source services out there. But say you have some custom tweaks, or simply a personal blog you want to host? If the rest of your homelab is docker based then it might be a good idea to stay in familiar territory. But you might not want to have to deploy you toy project to Dockerhub or whatever public repository. And integrating with a private one somewhere might get complex pretty quick.
The secret sauce: Locally hosted Docker registry
Running a local Docker registry on your homelab server allows you to publish your docker images there and use that to spin up your personal applications or projects.
version: "3"
networks:
web:
external: true
services:
registry:
image: registry:2
container_name: registry
hostname: registry.example.com
ports:
- 5000:5000
environment:
- REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/data
- REGISTRY_HTTP_RELATIVEURLS=true
- REGISTRY_STORAGE_DELETE_ENABLED=true
restart: unless-stopped
volumes:
- /mnt/tank/docker/registry/data:/data
labels:
- traefik.http.routers.registry.rule=Host(`registry.example.com`)
- traefik.http.routers.registry.tls=true
- traefik.http.routers.registry.tls.certresolver=lets-encrypt
- traefik.http.routers.registry.middlewares=lan-only@file,simpleAuth@file
- traefik.port=5000
- com.centurylinklabs.watchtower.enable=true
networks:
- web
in your traefik.toml you need a middleware defined like so
[http.middlewares.simpleAuth.basicAuth]
users = [
"myUsername:$djsaklfjdisofowjoemwmfewøjoøadhAPPVen0kG",
]
Look at the docs for the specifics.
The volume we define is where we want the image data the registry handles to be stored.
Then there’s a few docker labels that relate to Traefik to expose the registry with a valid TLS certificate and http basic auth (which is required - you cannot host a publically available registry outside localhost without https as far as I have found). In other words you need Traefik running, and a domain to your name. Otherwise you could still only refer to the registry by IP but that gets a bit tedious. Note that we, in addition to protecting the registry behind auth, also use a lan-only middleware so the service is not exposed by traefik outside our lan. See this post for more info.
Two Towers
Assuming you got you registry up and running - and maybe you already have Watchtower running to update your generic homelab services you might now be running into the question of what schedule to run this whole thing on. Daily update checks is not much of a deploy strategy. On the flip side constantly pestering docker hub with requests might hit some sort of rate limit. Another quirk is that if you have set up some services to only notify - not auto update - on new images, Watchtower has no concept of state so it’ll just keep notifying you that an update is available every five minutes or whatever you have defined.
So what you want is two seperate schedules for two types of containers. Your third party services and your own applications that you controll. Furtunately Watchtower can run multiple instances with scopes. You have Two (Watch)Towers!
The setup
version: '3'
services:
watchtower-local:
image: containrrr/watchtower
container_name: watchtower_local
restart: always
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /etc/timezone:/etc/timezone:ro
- ./config.json:/config.json
command: --interval 30 --scope internal
environment:
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_LABEL_ENABLE=true
- WATCHTOWER_INCLUDE_RESTARTING=true
- WATCHTOWER_NOTIFICATIONS=shoutrrr
labels:
- com.centurylinklabs.watchtower.enable=true
- com.centurylinklabs.watchtower.scope=internal
watchtower:
image: containrrr/watchtower
container_name: watchtower
restart: always
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /etc/timezone:/etc/timezone:ro
command: --schedule "0 0 6 * * *" --scope none
environment:
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_LABEL_ENABLE=true
- WATCHTOWER_INCLUDE_RESTARTING=true
labels:
- com.centurylinklabs.watchtower.enable=true
- com.centurylinklabs.watchtower.scope=none
Note that the local watchtower is running in it’s own scope internal. If you omit this Watchtower complains as two running instances in the same scope conflict with each other.
Also, to let Watchtower read from the local docker registry you need a config.json that looks like this. See the docs for details.
{
"auths": {
"registry.example.com": {
"auth": "fjkdlsjfiwejfwemølfjwsj=="
}
}
}
Now you can setup watchtower labels like this (aka no change) for third parties:
- com.centurylinklabs.watchtower.enable=true
and for your internal applications you do it like this
- com.centurylinklabs.watchtower.enable=true
- com.centurylinklabs.watchtower.scope=internal
Deploy an image to the registry
I’ll share the Dockerfile of this blog, as well as the docker-compose for running it on the server just as an example to track.
Dockerfile:
FROM nginxinc/nginx-unprivileged:1.25-alpine
MAINTAINER hernil
COPY ./public/ /var/www
COPY ./nginx.conf /etc/nginx/conf.d/default.conf
docker-compose
version: "3.8"
networks:
web:
external: true
services:
frontend:
image: registry.example.com/yvn/devblog:latest
container_name: yvn_devblog
expose:
- 8080
labels:
- traefik.http.routers.yvn-devblog.rule=Host(`devblog.yvn.no`)
- traefik.http.routers.yvn-devblog.tls=true
- traefik.http.routers.yvn-devblog.tls.certresolver=lets-encrypt
- com.centurylinklabs.watchtower.enable=true
- com.centurylinklabs.watchtower.scope=internal
networks:
- web
To keep things simple we run the building of the docker image on our local workstation.
First authenticate with the registry
docker login registry.example.com
then build the image
docker build . --platform=linux/amd64 --tag registry.example.com/yvn/devblog:latest
then push the tag to the registry
docker push registry.example.com/yvn/devblog:latest
That’s it. Assuming you got everything right you’ll have your new image deployed within about 30 seconds.
You might also want to use Portainer or something similar that exposes a nice gui to browse your registry. This way you can experiment with tagging images with more meaningful tags than just latest and keep a history for convenient roll back should you need it.
Gotchas
Running Watchtower for third party services with auto updates can be a bit hazardous. Some application explicitely recommends against it as it can take down the service if there are breaking changes you haven’t handled. Tracking latest for services you are not in control of is fairly optimistic. A good middle ground here could be to track spesific versions of an application ie. the 2 tag like we do for the registry images further up. Assuming the project follows semantic versioning this should keep you out of most trouble and then you can update manually from time to time between major versions.
Follow ups
There’s a few potential follow ups to this I’d like to write at some point. I’ll update the post here if that gets done.
- How to setup Traefik for your Docker homelab
- How to set up Ntfy
Reply via email!