Homelab Ingress With Deterministic IPv6

Last updated 2026-08-11, 10 min read

In my last post I talked in general about how my homelab is set up. In this piece I'm going to explore one of the more interesting aspects of the stack in more depth: my web ingress.

The canonical homelab Docker-based ingress is Caddy, Traefik, or Nginx Proxy Manager sitting on a shared Docker bridge with every other web-facing container. This is easy and understandable and also a security trap.

I wanted a harder network boundary between my stacks so I run Caddy with host networking and reach services with deterministic IPv6 addresses. I trust Caddy significantly more than I trust whatever random container I spun up most recently.

Ingress in a Nutshell

Most server software targeting hobbyists and dilettantes has a web interface. You spin up the container, you expose port 8080, you point your browser at zaphod.local:8080 and you're done.

Except, what about HTTPS? What about nice-looking names and subdomains?

Something has to deal with that essential complexity. Should every random container handle it? Do you really trust the latest slop app at the top of /r/selfhosted with your DNS credentials so it can generate a certificate? I sure don't.

There are an uncountable number of solutions in this problem space. In the past I've run nginx, Apache, Traefik, and probably more I've forgotten, but for now I've settled on Caddy.

Every stack gets its own Docker bridge network. Services only join other stack networks when they truly need to.

So, how does Caddy get at containers? It runs in host network mode. That means it can reach apps through the stacks' networks because the host inherently has routes into those bridges. No shared bridge necessary.

Deterministic IPv6

The thing that makes this trick work is deterministic IPv6. The lab as a whole has a shared /48 ULA prefix. For each stack, we calculate two hashes: the host name and the stack name. Then, we fill out this formula to determine the stack's /96 prefix:

<48 bit ULA prefix>:<16 bit host>:<32 bit stack>::

Each service in a stack gets a /128 address inside that /96, and to get the final 32 bits we hash the stack name and the service name together along with tags, then fill in the formula again:

<48 bit ULA prefix>:<16 bit host>:<32 bit stack>:<32 bit service>

Here's a worked example:

ULA = IPAddr.new("2001:db8:1234::").mask(48)
hostname = "zaphod"
stack_name = "whoami"
container_name = "whoami"

# this is handwavey pseudocode
hostname_bits = SHA256(["host", hostname].join(":")).first_bits(16)
stack_bits = SHA256(["stack", stack_name].join(":")).first_bits(32)
container_bits = SHA256(["stack", stack_name, "container", container_name].join(":")).first_bits(32)

stack_prefix = ULA.to_i
stack_prefix |= hostname_bits << 64
stack_prefix |= stack_bits << 32
stack_addr = IPAddr.new(stack_prefix, Socket::AF_INET6).mask(96)
#=> IPAddr.new("2001:db8:1234:70ef:55a:c88c::")

container_addr_bits = stack_prefix
container_addr_bits |= container_bits
container_addr = IPAddr.new(container_addr_bits, Socket::AF_INET6)
#=> IPAddr.new("2001:db8:1234:70ef:55a:c88c:301d:58fe")

I need these addresses because Docker's internal DNS only works on bridges, not in host mode, so I can't use it from Caddy.

What does work is extra_hosts, a standard Docker Compose feature that injects hostname to address mappings directly into a container's /etc/hosts file. I use this to statically map every deterministic IPv6 to a hostname like service_name.stack_name.docker.internal. The config generator defaults to just using the IPv6 literal but the names are available for hand-written routes.

There are other ways to avoid a shared bridge. I could publish ports on 127.0.0.1, for example, but then I'd have to figure out a fleet-wide port allocation scheme. Been there, done that, IPv6 is cleaner and way more interesting.

How My Ingress Works

If you read the previous piece you might recall that I have my lab structured around the idea of static allocation. What that means is that there's no scheduler determining where to place containers. There are no leaders or Raft or consensus of any sort. Every piece is determined at build time and then tested, linted, and pushed out to the various hosts in the lab.

Ingress is no different. One of the hooks that runs during build examines every stack for a Compose extension named x-web. The hook builds one or more WebRoute objects for each x-web, which then get passed to an ERB template. The template spits out a config.json file for Caddy. Every host has a different config based on what's actually running there.

WebRoute objects also play a starring role in DNS and TLS certificate setup. Every WebRoute can contain one or more fully qualified domain names that get injected into my DNS config and applied with dnscontrol. They also get normalized and dumped to a text file for my certificate script to consume, which runs lego daily and whenever I add a new domain name. Caddy consumes the certs via a read-only bind mount from the host.

I use lego on a cron instead of Caddy's built-in cert management because I want to use the certs in other contexts and lego + cron makes that easy. This may be paranoia but I also feel more comfortable not handing the public facing edge process DNS API credentials, even tightly scoped ones.

Caddy Stages

A typical Caddy config will just terminate TLS and then bounce to the upstream. Here I split into two stages:

  1. Terminate TLS
    • listens on 80 and 443
    • redirects HTTP to HTTPS
    • rejects requests from non-local IPs unless the route sets public: true
    • forwards to an intermediary (anubis: true or auth: true) or directly to stage 2
  2. Forward to upstream
    • listens on loopback on port 8888
    • redirects to a canonical hostname OR
    • proxies to a remote upstream if public_ingress config is set
    • proxies to a local upstream

"Stage" in this context describes a role rather than strict ordering of Caddy hops. Normally a request will travel through stage 1, stage 2, and then to an upstream container. However, when public_ingress is in play the stage 2 on the edge host will proxy to stage 2 on another server via Tailscale.

The original reason for having two stages was that Anubis, the Web AI Firewall Utility, can only proxy back to a single upstream and I planned on running multiple applications through the same Anubis instance.

In my case, that upstream is another Caddy server block running in the same process listening on localhost:8888 and tailnet:8888. Anubis is also running with host networking and listening on localhost so it can talk to localhost:8888 without exposing either to the outside world.

The split also gives a convenient point to shim in auth via oauth2-proxy.

Example Configs

Here's the simplest possible x-web in the context of a whole docker-compose.yml file. This is something I actually run in my lab, lightly edited for clarity:

services:
  whoami:
    hostname: whoami
    image: traefik/whoami
    restart: unless-stopped
    command: ["--verbose"]
    x-web:
      port: 80
      auth: false
      public: true

Requests to the whoami service follow this path:

Client Browser
HTTPS :443
Homelab host
Caddy · stage 1 TLS + hostname
HTTP · loopback :8888
Caddy · stage 2 Route + proxy
HTTP :80
Docker network whoami

Route Overlays

This next example serves a static site that I built to help me plan a catio along with a tiny Go service.

The stack-level x-web declares two routes. The first mounts localsites-kv at https://catio.example.com/_kv and the second mounts a static HTML site at https://catio.example.com/. The files block tells the deploy system how to fetch the files. It refreshes every 10 minutes and when a Forgejo webhook announces a new package version.

services:
  localsites-kv:
    image: git.example.com/pete/homelab/localsites-kv:${STACK_SOURCE_SHA}
    hostname: localsites-kv
    build:
      context: ../../go
      dockerfile: ../stacks/localsites/Dockerfile
    restart: unless-stopped
    environment:
      - LOCALSITES_KV_DB=/data/kv.sqlite3
      - PORT=9292
    volumes:
      - /data/localsites-kv:/data

x-web:
  - fqdn: catio.example.com
    auth: true
    public: true
    routes:
      - path: /_kv
        upstream: http://localsites-kv.localsites.docker.internal:9292

  - fqdn: catio.example.com
    files:
      sources:
        - id: site
          fetch_script: fetch-static-site.sh
          mount: /
          schedule: "0 */10 * * * *"
          webhook:
            forgejo:
              owner: pete
              repository: catio
              package: catio
              type: generic
    auth: true
    public: true
Client Browser catio.example.com
HTTPS :443
Homelab host
Caddy · stage 1 TLS + hostname
Authentication oauth2-proxy
Caddy · stage 2 Match request path
Path: /_kv
Docker network localsites-kv
Path: everything else
Caddy file server Static site

Public Ingress

Finally, this is the docker-compose.yaml for my Forgejo server. It spans two hosts and exercises most of the ingress system. First, it sets anubis: true, which triggers the Anubis path discussed above. Next, it has public_ingress set to another host within the fleet. This results in ord-router's stage 2 forwarding to nibbler's stage 2, where this stack is deployed.

Finally, it has an extra route that rewrites a status code for one very specific path. The Minecraft container sends a HEAD request to check for updates. It will retry on a 403 but Forgejo sends a 405, so we remap that one response.

services:
  forgejo:
    image: codeberg.org/forgejo/forgejo:15-rootless
    container_name: forgejo
    environment:
      - USER_UID=1000
      - USER_GID=1000
    restart: unless-stopped
    volumes:
      - /data/forgejo:/var/lib/gitea
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    x-web:
      hostname: git
      public_ingress: ord-router
      public: true
      anubis: true
      auth: false
      routes:
        - path: /api/packages/pete/generic/minecraft-floodgate-spigot
          port: 3000
          proxy:
            replace_status:
              "405": 403
        - path: /
          port: 3000
Public client Browser or updater git.example.com
ord-router · public ingress
Caddy · stage 1 TLS + hostname
AI firewall Anubis
Caddy · stage 2 Remote upstream
nibbler · serving host
Caddy · stage 2 Local route + policy
Docker network Forgejo

This Seems Like A Lot

You're not wrong. This is A Lot.

There are a bunch of cool things that this setup brings to the table, though. The vast majority of stacks in my fleet have an x-web just like the whoami stack: declare the port, declare auth, done. The x-web extension keeps the simple things simple and the config right next to the Compose service it pertains to.

Caddy and x-web also make the harder cases tractable. For example, combining static sites with sprinkles of backend service is trivial with the layered routes syntax. The status-code rewrite is a one-off but the route model gives it a home without complicating ordinary stacks.

I also have a few convenience features set up, some of which are demonstrated above:

  • Monitoring via Gatus is automatic
  • Authentication provided by oauth2-proxy is one boolean
  • Anubis, the Web AI Firewall Utility, is another boolean
  • Rejecting non-local clients is another one

The tradeoff is that Caddy with host networking is even more trusted than in a conventional setup. It also only works with static allocation. Dynamic allocation would require a scheduler, taking me several agonizing steps back toward Kubernetes.

Should You Do This?

Look, I'm not advocating for anyone to follow my path. As I explained in the previous piece, I have grown this idiosyncratic system over many years. It works for me. It could maybe work for you too, but the thing I want to get across is that designs like this are possible and worth exploring.

I automatically publish a sanitized snapshot of my homelab config on my Forgejo. Feel free to poke around and let me know what you think!

Posted in: Home Lab  

Tagged: Homelab Projects