docs: from-zero node runbook for the share service
CI / compile (pull_request) Successful in 4s
CI / unit (pull_request) Successful in 22s
CI / integration (pull_request) Successful in 24s
build / build (push) Successful in 7s
CI / compile (push) Successful in 5s
CI / unit (push) Successful in 22s
CI / integration (push) Successful in 24s
CI / compile (pull_request) Successful in 4s
CI / unit (pull_request) Successful in 22s
CI / integration (pull_request) Successful in 24s
build / build (push) Successful in 7s
CI / compile (push) Successful in 5s
CI / unit (push) Successful in 22s
CI / integration (push) Successful in 24s
FILE_SHARING.md explains how the feature works and assumes you already know the node story; there was nothing describing what a fresh box has to provide. SHARE_NODE_SETUP.md fills that: the four host directories, the uid-33 readability requirement on the media library, deploying the musician+share stack (Portainer or CLI), forwarding /share from the reverse proxy, and the musician-side CONJURER_SHARE_* wiring - each with its verification command. Calls out the three couplings that actually break it in practice: the media must be mounted at the SAME container path on both containers (index paths and symlinks are absolute), the link/index dirs must be shared, and CONJURER_SHARE_BASE_URL must match the real public URL - it is not set in the bundled stack file and silently falls back to czernobog.pl. Also records what the node does NOT need: no host Apache, no cron, no Python, no hand-placed scripts - Apache, the vhost, scan_shares.py and revoke_shares.py are all baked into the image. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit was merged in pull request #24.
This commit is contained in:
@@ -10,8 +10,9 @@ Runbook for running Conjurer as Docker containers across Proxmox VMs:
|
||||
| **Share** (optional) | VM-musician or its own | `conjurer-share` | 8081 → 80 | `docker/Dockerfile.share` |
|
||||
|
||||
The share service publishes short-lived file links over Apache and is documented
|
||||
separately in [FILE_SHARING.md](FILE_SHARING.md) — it shares two volumes with the
|
||||
musician, so set it up after the musician is running.
|
||||
separately: [SHARE_NODE_SETUP.md](SHARE_NODE_SETUP.md) to stand a node up from
|
||||
zero, [FILE_SHARING.md](FILE_SHARING.md) for how the feature works. It shares two
|
||||
volumes with the musician, so set it up after the musician is running.
|
||||
|
||||
The three talk to each other over HTTP on the Proxmox LAN. Direction of calls:
|
||||
|
||||
|
||||
@@ -7,6 +7,10 @@ Historically only the Python half of this lived in the repo; the Apache config
|
||||
and the cron entries were placed on the host by hand. This document plus
|
||||
`docker/Dockerfile.share` close that gap.
|
||||
|
||||
> Standing it up on a fresh node (directories, permissions, reverse proxy,
|
||||
> verification)? Start with **[SHARE_NODE_SETUP.md](SHARE_NODE_SETUP.md)** — this
|
||||
> document explains how the feature works and how it is secured.
|
||||
|
||||
## What it actually does
|
||||
|
||||
1. A **scanner** walks the media library and writes a JSON index of every path.
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
# Share node setup — from zero
|
||||
|
||||
Everything the share service needs is **inside the image**: Apache, the vhost,
|
||||
the index scanner and the link revoker. The node needs no Apache, no cron, no
|
||||
Python and no copies of the old hand-placed scripts.
|
||||
|
||||
What the node actually provides is four directories, the media library, and a
|
||||
way in from the internet.
|
||||
|
||||
> Feature docs (how links work, security model, TTL semantics) live in
|
||||
> [FILE_SHARING.md](FILE_SHARING.md). This file is only "how to stand it up on a
|
||||
> fresh box".
|
||||
|
||||
## What runs where
|
||||
|
||||
| Piece | Where it lives | Notes |
|
||||
|---|---|---|
|
||||
| Apache + vhost | in the image | rendered from `share-vhost.conf.tpl` at start |
|
||||
| `scan_shares.py` (index) | in the image | sleep loop, not cron |
|
||||
| `revoke_shares.py` (expiry) | in the image | sleep loop, not cron |
|
||||
| the symlinks | host `/srv/share/links` | **created by the musician**, served here |
|
||||
| the index | host `/srv/share/db` | written here, **read by the musician** |
|
||||
| TLS / public name | your reverse proxy | container speaks plain HTTP |
|
||||
|
||||
The single most important fact: **the musician creates the links, this service
|
||||
serves them.** They must see the same directories, at the same paths.
|
||||
|
||||
## 1. Directories
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /srv/share/{media,links,db,logs}
|
||||
```
|
||||
|
||||
| Path | Contents | Mounted as |
|
||||
|---|---|---|
|
||||
| `/srv/share/media` | the media library | `/mnt/shares` (read-only) |
|
||||
| `/srv/share/links` | published symlinks | `/var/www/html/share` |
|
||||
| `/srv/share/db` | `share_scan.json` | `/srv/share/db` |
|
||||
| `/srv/share/logs` | Apache logs incl. `share_access.log` | `/var/log/apache2` |
|
||||
|
||||
## 2. Media library and permissions
|
||||
|
||||
Put the library at `/srv/share/media` (bind mount, NFS mount, whatever — it is
|
||||
only ever read). Apache serves as **`www-data`, uid 33 inside the container**, so
|
||||
that uid must be able to traverse and read it:
|
||||
|
||||
```bash
|
||||
sudo chmod -R o+rX /srv/share/media # simplest; or use ACLs/group instead
|
||||
sudo -u '#33' test -r /srv/share/media/<some-file> && echo "readable by www-data"
|
||||
```
|
||||
|
||||
A library that root can read but uid 33 cannot is the classic "every link
|
||||
404s / 403s" cause.
|
||||
|
||||
## 3. Deploy the stack
|
||||
|
||||
The musician and the share service share a filesystem, so the supported layout
|
||||
is **both on the same node**, from one stack:
|
||||
|
||||
- Portainer → **Stacks → Add stack** → paste `docker/compose.musician-share.stack.yaml`
|
||||
- or CLI: `docker compose -f docker/compose.musician-share.stack.yaml up -d`
|
||||
|
||||
That stack already wires the four mounts on both containers. Set
|
||||
`CONJURER_SHARE_SERVER_NAME` to your public hostname.
|
||||
|
||||
Share-only node (musician elsewhere): use `docker/compose.share.yaml` and put
|
||||
`/srv/share/links` + `/srv/share/db` on storage **both** hosts mount — otherwise
|
||||
the musician cheerfully creates links this container cannot see.
|
||||
|
||||
## 4. Way in from the internet
|
||||
|
||||
The container listens on **8081 → 80**, plain HTTP by design; TLS stays on the
|
||||
reverse proxy you already run. Forward the `/share/` prefix **unchanged** — the
|
||||
links are `https://<host>/share/<token>`:
|
||||
|
||||
nginx:
|
||||
```nginx
|
||||
location /share/ {
|
||||
proxy_pass http://<share-node-ip>:8081/share/;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
}
|
||||
```
|
||||
|
||||
Apache (as reverse proxy):
|
||||
```apache
|
||||
ProxyPass /share/ http://<share-node-ip>:8081/share/
|
||||
ProxyPassReverse /share/ http://<share-node-ip>:8081/share/
|
||||
```
|
||||
|
||||
Then: DNS for the public name points at the proxy, and the node's firewall lets
|
||||
the proxy reach 8081 (nothing else needs to).
|
||||
|
||||
## 5. Tell the musician where to publish
|
||||
|
||||
The musician builds the URLs it posts to Discord. In its env:
|
||||
|
||||
```ini
|
||||
CONJURER_SHARE_DIR=/var/www/html/share
|
||||
CONJURER_SHARE_DB=/srv/share/db/share_scan.json
|
||||
CONJURER_SHARE_BASE_URL=https://czernobog.pl/share
|
||||
```
|
||||
|
||||
`CONJURER_SHARE_BASE_URL` is **not** set in the bundled stack file — it falls back
|
||||
to `https://czernobog.pl/share`. If your public name differs, set it explicitly
|
||||
or every posted link points at the wrong host.
|
||||
|
||||
## The three couplings that break it
|
||||
|
||||
1. **Same container path for the media.** The index records absolute paths and
|
||||
the symlinks are absolute. Both containers must mount the library at
|
||||
`/mnt/shares`. Mount it elsewhere on one side and every link dangles.
|
||||
2. **Same link dir and index dir** for musician and share (same host, or shared
|
||||
storage).
|
||||
3. **`CONJURER_SHARE_BASE_URL` must equal your real public `/share` URL.**
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# index built (should be non-trivial JSON)
|
||||
sudo head -c 200 /srv/share/db/share_scan.json; echo
|
||||
|
||||
# the two jobs and Apache are alive
|
||||
docker logs conjurer-share | tail -20 # "[share] serving ... scan every ...s"
|
||||
|
||||
# directory listing MUST fail (403) - it would leak every live token
|
||||
curl -sI http://<share-node-ip>:8081/share/ | head -1
|
||||
|
||||
# revoker state files must NOT be served
|
||||
curl -sI http://<share-node-ip>:8081/share/.downloads.json | head -1
|
||||
|
||||
# a real link: publish one from Discord, then
|
||||
curl -sI https://<public-host>/share/<token> | head -1 # 200
|
||||
```
|
||||
|
||||
After the first download of a link, `revoke_shares.py` removes it once
|
||||
`CONJURER_SHARE_TTL_SECONDS` (default 1h) has passed. A link nobody downloads is
|
||||
never revoked by that job.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---|---|---|
|
||||
| every link 404 | media mounted at a different container path than `/mnt/shares`, or the symlink target is gone | align the mounts on both containers |
|
||||
| every link 403 | media not readable by uid 33 | step 2 |
|
||||
| links created but not served | musician and share not sharing `/srv/share/links` | step 3 |
|
||||
| `/get_share_list` empty | index missing/not shared | check `/srv/share/db/share_scan.json` and the musician's `CONJURER_SHARE_DB` |
|
||||
| links never expire | revoker cannot see the access log, or nobody downloaded them | check `/srv/share/logs/share_access.log` exists and grows |
|
||||
| wrong host in posted links | `CONJURER_SHARE_BASE_URL` | step 5 |
|
||||
Reference in New Issue
Block a user