WIP: lindberg: Add jellyfin nixos container #140

Draft
fabianhauser wants to merge 13 commits from jellyfin into main
Owner
No description provided.
Lets per-host configs (NixOS containers) disable backup-client,
outgoing-server-mail, and telegraf with a plain assignment.
Container IPs are looked up from the lindberg-containers-nat network,
analogous to how VM domains use backplane. Also replaces 'with lib;' with
explicit inherits to satisfy CODESTYLE.
Sets up jellyfin via nixflix with the reverse proxy through nixflix's
nginx module. Layers per-host ACME and kTLS on top of the auto-generated
vhost. Admin password is read from /run/jellyfin-admin-password, which a
oneshot service materializes from a systemd credential so that both
jellyfin-setup-wizard and jellyfin-users-config can share the file.
Add secrets
All checks were successful
CI / build (push) Successful in 5m52s
CI / deploy (docs-ops.qo.is) (push) Has been skipped
CI / deploy (system-physical) (push) Has been skipped
CI / deploy (system-vm) (push) Has been skipped
CI / deploy-ci (push) Has been skipped
b56d3d835f
@ -5,0 +4,4 @@
qois.backup-client.includePaths = [
"/mnt/data"
"/var/lib/jellyfin"
"/var/lib/nixos-containers"
Author
Owner

TODO: Verify that there is only the machine id etc. in this folder, and nothing else that we might not want to backup.

TODO: Verify that there is only the machine id etc. in this folder, and nothing else that we might not want to backup.
@ -49,1 +50,3 @@
internalInterfaces = [ "vms-nat" ];
internalInterfaces = [
"vms-nat"
"ve-jellyfin"
Author
Owner

Change this to "ve-*" so that it works when we add more containers.

Change this to "ve-*" so that it works when we add more containers.
fabianhauser marked this conversation as resolved
@ -63,6 +68,11 @@ in
allowedTCPPorts = [ 53 ];
};
networking.firewall.interfaces."ve-jellyfin" = {
Author
Owner

Create a config that maps the nat.internalInterfaces, so that interfaces don't have to be specified manually.

Create a config that maps the `nat.internalInterfaces`, so that interfaces don't have to be specified manually.
fabianhauser marked this conversation as resolved
@ -0,0 +1,63 @@
# Jellyfin
Jellyfin media server running as a NixOS container (systemd-nspawn) on `lindberg`, configured via [nixflix](https://kiriwalawren.github.io/nixflix/reference/).
Author
Owner

This module should be hos agnostic - remove lindberg references.
Do, however, explain about the subdomains of the primary domain.

This module should be hos agnostic - remove lindberg references. Do, however, explain about the subdomains of the primary domain.
fabianhauser marked this conversation as resolved
@ -0,0 +32,4 @@
Before deploying, create both secrets on lindberg:
```bash
sops private/nixos-configurations/lindberg/secrets.sops.yaml
Author
Owner

Explain this without referencing lindberg - show the sops set commands to do this without manual copy-paste (see nixos-configurations/setup.md on how to do that).

Explain this without referencing lindberg - show the `sops set` commands to do this without manual copy-paste (see nixos-configurations/setup.md on how to do that).
fabianhauser marked this conversation as resolved
@ -0,0 +34,4 @@
# Admin user: password is read at runtime from /run/jellyfin-admin-password,
# which is materialized by jellyfin-credential-setup.service below.
nixflix.jellyfin.users.admin = {
password._secret = "/run/jellyfin-admin-password";
Author
Owner

TODO: Explain in the readme how to get this login as an admin (sops get ...)

TODO: Explain in the readme how to get this login as an admin (sops get ...)
fabianhauser marked this conversation as resolved
@ -0,0 +64,4 @@
};
# API key read from systemd credential passed by the container host via --load-credential.
# Create: sops private/nixos-configurations/lindberg/secrets.sops.yaml
Author
Owner

Already documented in README.md, remove here.

Already documented in README.md, remove here.
fabianhauser marked this conversation as resolved
@ -0,0 +73,4 @@
"jellyfin-api-key:jellyfin-api-key"
];
# Reverse proxy via nixflix's nginx module: it builds the virtual host
Author
Owner

Remove this self-explanory comments...

Remove this self-explanory comments...
fabianhauser marked this conversation as resolved
@ -0,0 +76,4 @@
# Reverse proxy via nixflix's nginx module: it builds the virtual host
# "${subdomain}.${domain}" with proxyPass, websockets, buffering off, and
# forceSSL via mkVirtualHost, and auto-derives knownProxies/localNetworkAddresses.
# We layer per-host ACME (instead of nixflix's wildcard useACMEHost pattern) and kTLS on top.
Author
Owner

Why not nixflix useACMEHost? It should do the same?

Why not nixflix useACMEHost? It should do the same?
fabianhauser marked this conversation as resolved
@ -0,0 +39,4 @@
sslCertificateKey = "${certs}/${jellyfinDomain}.key.pem";
};
networking.extraHosts = "127.0.0.1 ${jellyfinDomain}";
Author
Owner

Set with networking.hosts. Check if nixflix or our module doesn't do this out of the box.

Set with `networking.hosts`. Check if nixflix or our module doesn't do this out of the box.
fabianhauser marked this conversation as resolved
@ -99,2 +114,4 @@
};
containerDomains = mkOption {
description = "Full domain to container-name mappings; IPs taken from lindberg-containers-nat network";
Author
Owner

TODO: This doesn't currently work in some cases, as we also run a loadbalancer on cyprianspitz. Can cyprianspitz route these ip's over the backplane?

TODO: This doesn't currently work in some cases, as we also run a loadbalancer on cyprianspitz. Can cyprianspitz route these ip's over the backplane?
fabianhauser marked this conversation as resolved
Author
Owner

Addressed the review feedback in commits d34ad10..5481642.

What changed

Review Resolution Commit
#1177, #1178 Dropped self-explanatory comments in nixos-modules/jellyfin/default.nix d34ad10
#1173, #1174, #1175 Rewrote nixos-modules/jellyfin/README.md host-agnostic, switched to sops set with a $SECRETS_FILE placeholder, added an Admin Login section 386d64e
#1179 Replaced networking.extraHosts in the test with nixflix.nginx.addHostsEntries = true so nixflix's mkVirtualHosts adds the entry itself 18c2aa8
#1171, #1172 Changed internalInterfaces and networking.firewall.interfaces to the iptables wildcard ve-+ so any future container's veth is matched automatically 0867212
#1180 Mirror of the vpn.qo.is / cyprianspitz-nginx pattern: added qois.loadbalancer.containerHost (default lindberg); on the container host containerDomains backends still resolve to the container IP, on every other LB they forward to containerHost's backplane IP, so cyprianspitz transparently proxies to lindberg's LB. Avoids extending wireguard AllowedIPs (wgautomesh hard-codes address/32 and overwrites any local override on every endpoint change). 5481642
#1176 Kept per-host enableACME = true to stay consistent with cloud, vault, git, attic, nixpkgs-cache, grafana, static-page, vpn-server. nixflix's nixflix.nginx.enableACME resolves to useACMEHost = "${nixflix.nginx.domain}" (nixflix/lib/mkVirtualHosts.nix:50), which expects a separately-configured security.acme.certs."media.qo.is" (wildcard via DNS-01). We do not have wildcard ACME for media.qo.is, so the per-host HTTP-01 cert avoids new ACME plumbing.
#1170 Not fixed in code; needs a one-time check after deploy (see below).

Deployment order

# Lindberg first: introduces the ve-+ NAT/firewall match and uses the LB module's
# new branch where containerHost == hostName (no behaviour change on lindberg).
deploy --skip-checks .#lindberg.system-physical

# Then cyprianspitz: switches its containerDomains backends from the unreachable
# 10.246.0.2 to lindberg's backplane IP on :80/:443.
deploy --skip-checks .#cyprianspitz.system-physical

Post-deploy checks

On lindberg:

# #1170: verify /var/lib/nixos-containers/jellyfin/ holds only machine-id.
# Expected: an etc/ tree with machine-id and not much else (the rest is volatile,
# real state is bind-mounted from /var/lib/jellyfin and /mnt/data/media).
ls -laR /var/lib/nixos-containers/jellyfin/

# Sanity: firewall now matches the wildcard.
iptables -L nixos-fw -v -n | grep -E 've-'

If anything beyond machine-id shows up under /var/lib/nixos-containers/jellyfin/, narrow qois.backup-client.includePaths or add to excludePaths.

On cyprianspitz:

# Container domain now resolves to lindberg's HAProxy.
curl -kI --resolve jellyfin.media.qo.is:443:127.0.0.1 https://jellyfin.media.qo.is/

# HAProxy stats — confirm the lindberg-jellyfin backend points at lindberg's
# backplane IP (10.250.0.2:443 / :80), not the container IP.
curl -s http://127.0.0.1:8404/metrics | grep -A 4 lindberg-jellyfin

End to end, from an off-network client (resolving the domain to cyprianspitz's public IP):

curl -kI https://jellyfin.media.qo.is/
# Expected: HTTP/2 302 -> /web/index.html (Jellyfin login redirect).

Rollback

deploy-rs magic-rollback is enabled by default — failed activations revert automatically after 30 s. Manual rollback: deploy --rollback.

Addressed the review feedback in commits d34ad10..5481642. ## What changed | Review | Resolution | Commit | | --- | --- | --- | | #1177, #1178 | Dropped self-explanatory comments in `nixos-modules/jellyfin/default.nix` | d34ad10 | | #1173, #1174, #1175 | Rewrote `nixos-modules/jellyfin/README.md` host-agnostic, switched to `sops set` with a `$SECRETS_FILE` placeholder, added an Admin Login section | 386d64e | | #1179 | Replaced `networking.extraHosts` in the test with `nixflix.nginx.addHostsEntries = true` so nixflix's `mkVirtualHosts` adds the entry itself | 18c2aa8 | | #1171, #1172 | Changed `internalInterfaces` and `networking.firewall.interfaces` to the iptables wildcard `ve-+` so any future container's veth is matched automatically | 0867212 | | #1180 | Mirror of the `vpn.qo.is` / `cyprianspitz-nginx` pattern: added `qois.loadbalancer.containerHost` (default `lindberg`); on the container host `containerDomains` backends still resolve to the container IP, on every other LB they forward to `containerHost`'s backplane IP, so cyprianspitz transparently proxies to lindberg's LB. Avoids extending wireguard AllowedIPs (wgautomesh hard-codes `address/32` and overwrites any local override on every endpoint change). | 5481642 | | #1176 | Kept per-host `enableACME = true` to stay consistent with `cloud`, `vault`, `git`, `attic`, `nixpkgs-cache`, `grafana`, `static-page`, `vpn-server`. nixflix's `nixflix.nginx.enableACME` resolves to `useACMEHost = "${nixflix.nginx.domain}"` (`nixflix/lib/mkVirtualHosts.nix:50`), which expects a separately-configured `security.acme.certs."media.qo.is"` (wildcard via DNS-01). We do not have wildcard ACME for `media.qo.is`, so the per-host HTTP-01 cert avoids new ACME plumbing. | — | | #1170 | Not fixed in code; needs a one-time check after deploy (see below). | — | ## Deployment order ```bash # Lindberg first: introduces the ve-+ NAT/firewall match and uses the LB module's # new branch where containerHost == hostName (no behaviour change on lindberg). deploy --skip-checks .#lindberg.system-physical # Then cyprianspitz: switches its containerDomains backends from the unreachable # 10.246.0.2 to lindberg's backplane IP on :80/:443. deploy --skip-checks .#cyprianspitz.system-physical ``` ## Post-deploy checks On `lindberg`: ```bash # #1170: verify /var/lib/nixos-containers/jellyfin/ holds only machine-id. # Expected: an etc/ tree with machine-id and not much else (the rest is volatile, # real state is bind-mounted from /var/lib/jellyfin and /mnt/data/media). ls -laR /var/lib/nixos-containers/jellyfin/ # Sanity: firewall now matches the wildcard. iptables -L nixos-fw -v -n | grep -E 've-' ``` If anything beyond machine-id shows up under `/var/lib/nixos-containers/jellyfin/`, narrow `qois.backup-client.includePaths` or add to `excludePaths`. On `cyprianspitz`: ```bash # Container domain now resolves to lindberg's HAProxy. curl -kI --resolve jellyfin.media.qo.is:443:127.0.0.1 https://jellyfin.media.qo.is/ # HAProxy stats — confirm the lindberg-jellyfin backend points at lindberg's # backplane IP (10.250.0.2:443 / :80), not the container IP. curl -s http://127.0.0.1:8404/metrics | grep -A 4 lindberg-jellyfin ``` End to end, from an off-network client (resolving the domain to cyprianspitz's public IP): ```bash curl -kI https://jellyfin.media.qo.is/ # Expected: HTTP/2 302 -> /web/index.html (Jellyfin login redirect). ``` ## Rollback `deploy-rs` magic-rollback is enabled by default — failed activations revert automatically after 30 s. Manual rollback: `deploy --rollback`.
Forward containerDomains via backplane on non-container hosts
All checks were successful
CI / build (push) Successful in 4m49s
CI / deploy (docs-ops.qo.is) (push) Has been skipped
CI / deploy (system-physical) (push) Has been skipped
CI / deploy (system-vm) (push) Has been skipped
CI / deploy-ci (push) Has been skipped
548164204a
On hosts other than containerHost, containerDomains backends now point
at containerHost's backplane IP (mirror of vpn.qo.is/cyprianspitz-nginx
pattern) so cyprianspitz's loadbalancer can serve container domains end
to end through lindberg's loadbalancer.
All checks were successful
CI / build (push) Successful in 4m49s
CI / deploy (docs-ops.qo.is) (push) Has been skipped
CI / deploy (system-physical) (push) Has been skipped
CI / deploy (system-vm) (push) Has been skipped
CI / deploy-ci (push) Has been skipped
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin jellyfin:jellyfin
git switch jellyfin
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
qo.is/infrastructure!140
No description provided.