share: dockerize the file-share pipeline and document it

The Python half of the short-lived share links lived in the repo; the Apache
config and the cron entries that make it work were hand-placed on the host, so
the feature could not be rebuilt from a checkout. This adds the missing half.

Scripts (defaults unchanged, so existing bare-metal cron keeps working):
* scan_shares.py / revoke_shares.py take their paths from CONJURER_SHARE_*
  instead of hardcoding the Pi layout, and create their parent dirs;
* the revoke TTL is now CONJURER_SHARE_TTL_SECONDS. Its --help claimed "2min"
  while the code used a hardcoded 3600 - the help text now reports the real,
  configured value.

New share service:
* docker/Dockerfile.share - Apache + the two jobs, reusing the same scripts
  rather than forking copies;
* docker/share-vhost.conf.tpl - the previously undocumented Apache half.
  Three settings are load-bearing and commented as such: +FollowSymLinks (the
  shares ARE symlinks), -Indexes (a listing would expose every live token), and
  a deny rule for dotfiles (revoke_shares.py keeps .downloads.json - a map of
  every live token - inside the served directory);
* docker/entrypoint.share.sh - renders the vhost, seeds the index on first run,
  then runs scanner/revoker in sleep loops beside Apache (no cron, so their
  output shows up in docker logs);
* compose + env example, incl. the two volumes that MUST be shared with the
  musician (it creates the links and reads the index).

docs/deployment/FILE_SHARING.md documents the mechanism, both deployment routes
(docker and existing host Apache), the Discord command, why each Apache setting
matters, verification commands, and the known limitations - notably that a link
nobody ever downloads is never revoked, since the TTL starts at first download.

Verified: scanner and revoker exercised end-to-end against temp dirs (index
built; token recorded from a combined-format log line and the symlink unlinked).
Compose file not validated - no docker on this machine.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Michal Tuszowski
2026-07-19 22:16:49 +02:00
parent dc449dd74c
commit 33ec1c0e35
9 changed files with 437 additions and 10 deletions
+36
View File
@@ -0,0 +1,36 @@
# Conjurer share service: Apache publishing short-lived symlink shares, plus
# the two periodic jobs that build the search index and revoke expired links.
#
# This is the piece that used to live only as hand-placed scripts + hand-edited
# Apache config on the host. Everything it needs is now in the image; the state
# it shares with the musician (the link dir and the index) comes in as volumes.
#
# Build from the repository root:
# docker build -f docker/Dockerfile.share -t conjurer-share .
FROM debian:bookworm-slim
# apache2 serves the links; python3 runs the scanner/revoker (both stdlib-only,
# so there is nothing to pip install).
RUN apt-get update && apt-get install -y --no-install-recommends \
apache2 python3 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# Same scripts the bare-metal deployment uses - no forked copies.
COPY conjurer_musician/scan_shares.py ./scan_shares.py
COPY conjurer_musician/revoke_shares.py ./revoke_shares.py
COPY docker/share-vhost.conf.tpl ./share-vhost.conf.tpl
COPY docker/entrypoint.share.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
# /mnt/shares - media library (mount read-only; symlink targets)
# /var/www/html/share - the symlinks themselves (SHARED with the musician,
# which is what creates them)
# /srv/share/db - share_scan.json (SHARED with the musician, which
# searches it)
# /var/log/apache2 - access log the revoker tails
VOLUME ["/mnt/shares", "/var/www/html/share", "/srv/share/db", "/var/log/apache2"]
EXPOSE 80
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+43
View File
@@ -0,0 +1,43 @@
# Share VM/host: Apache publishing short-lived file links + the scan/revoke jobs.
# Run from the repository root:
# cp docker/env/share.env.example docker/env/share.env # then edit
# docker compose -f docker/compose.share.yaml up -d --build
#
# IMPORTANT - this service shares two paths with the musician, which is the
# process that actually creates the links:
#
# /srv/share/links -> the symlinks (musician writes, Apache reads)
# /srv/share/db -> share_scan.json (this service writes, musician reads)
#
# So the musician must mount the same two host paths (see the CONJURER_SHARE_*
# block in docker/env/musician.env.example). If the musician runs on a different
# machine, put both on shared storage - otherwise it will happily create links
# that this container cannot see.
#
# The media library must be mounted here at the SAME absolute path that the
# index recorded, because the symlink targets are absolute. Default: /mnt/shares.
services:
conjurer-share:
build:
context: ..
dockerfile: docker/Dockerfile.share
image: conjurer-share:latest
container_name: conjurer-share
restart: unless-stopped
env_file:
- env/share.env
ports:
# Plain HTTP. Terminate TLS for czernobog.pl on your existing reverse
# proxy and forward /share here.
- "8081:80"
volumes:
# Media library the links point at. Read-only: this service only ever
# serves these files, it never writes them.
- /srv/share/media:/mnt/shares:ro
# The published symlinks - shared with the musician.
- /srv/share/links:/var/www/html/share
# The search index - shared with the musician.
- /srv/share/db:/srv/share/db
# Access log the revoker tails; kept on the host so link history survives
# a container rebuild.
- /srv/share/logs:/var/log/apache2
+67
View File
@@ -0,0 +1,67 @@
#!/bin/sh
# Entrypoint for the Conjurer share container: renders the vhost, seeds the
# share index, then runs the two periodic jobs alongside Apache.
#
# Cron is deliberately not used. Both scripts are already written to be safe to
# re-run (the scanner writes atomically, the revoker persists its log position),
# so plain sleep loops give the same result with one less moving part and with
# their output on the container's stdout where `docker logs` can see it.
set -eu
: "${CONJURER_SHARE_SERVER_NAME:=localhost}"
: "${CONJURER_SHARE_SCAN_PATH:=/mnt/shares}"
: "${CONJURER_SHARE_DB:=/srv/share/db/share_scan.json}"
: "${CONJURER_SHARE_DIR:=/var/www/html/share}"
: "${CONJURER_APACHE_LOG:=/var/log/apache2/share_access.log}"
: "${CONJURER_SHARE_SCAN_INTERVAL:=3600}"
: "${CONJURER_SHARE_REVOKE_INTERVAL:=60}"
: "${CONJURER_SHARE_TTL_SECONDS:=3600}"
export CONJURER_SHARE_SCAN_PATH CONJURER_SHARE_DB CONJURER_SHARE_DIR \
CONJURER_APACHE_LOG CONJURER_SHARE_TTL_SECONDS
# Render the vhost (same placeholder pattern as icecast.xml.tpl).
sed "s|__SERVER_NAME__|${CONJURER_SHARE_SERVER_NAME}|g" \
/app/share-vhost.conf.tpl > /etc/apache2/sites-available/000-default.conf
mkdir -p "$CONJURER_SHARE_DIR" \
"$(dirname "$CONJURER_SHARE_DB")" \
"$(dirname "$CONJURER_APACHE_LOG")" \
/var/run/apache2
chown -R www-data:www-data "$CONJURER_SHARE_DIR"
# revoke_shares.py seeks to a stored byte offset, so the log must exist before
# the first pass or it starts by reporting an error.
touch "$CONJURER_APACHE_LOG"
# Seed the index immediately when absent - otherwise /get_share_list returns
# nothing until the first scheduled scan, which looks like a broken feature.
if [ ! -f "$CONJURER_SHARE_DB" ]; then
echo "[share] no index at $CONJURER_SHARE_DB - running initial scan of $CONJURER_SHARE_SCAN_PATH"
python3 /app/scan_shares.py || echo "[share] initial scan failed, continuing"
fi
# Periodic index rebuild.
(
while true; do
sleep "$CONJURER_SHARE_SCAN_INTERVAL"
echo "[share] rescanning $CONJURER_SHARE_SCAN_PATH"
python3 /app/scan_shares.py || echo "[share] scan failed"
done
) &
SCAN_PID=$!
# Periodic revocation of links whose TTL expired after first download.
(
while true; do
sleep "$CONJURER_SHARE_REVOKE_INTERVAL"
python3 /app/revoke_shares.py || echo "[share] revoke failed"
done
) &
REVOKE_PID=$!
trap 'kill "$SCAN_PID" "$REVOKE_PID" 2>/dev/null || true' INT TERM
echo "[share] serving $CONJURER_SHARE_DIR as ${CONJURER_SHARE_SERVER_NAME}/share"
echo "[share] scan every ${CONJURER_SHARE_SCAN_INTERVAL}s, revoke sweep every ${CONJURER_SHARE_REVOKE_INTERVAL}s, TTL ${CONJURER_SHARE_TTL_SECONDS}s after first download"
exec apache2ctl -D FOREGROUND
+14
View File
@@ -15,3 +15,17 @@ CONJURER_MUSIC_FOLDER=/music
# Runtime paths (mounted volume) — playlists/logs the service writes.
CONJURER_MUSICIAN_BASE=/data
CONJURER_LOGSTORE=/data/logs
# --- File sharing (optional; see docs/deployment/FILE_SHARING.md) --------
# The musician SEARCHES the index and CREATES the symlinks; the share container
# builds the index, serves the links and revokes them. Both must therefore point
# at the same two host paths — mount them into this container as well:
# - /srv/share/db:/srv/share/db
# - /srv/share/links:/var/www/html/share
# Leave these unset to keep the feature off (the endpoints then just find
# nothing).
# CONJURER_SHARE_DB=/srv/share/db/share_scan.json
# CONJURER_SHARE_DIR=/var/www/html/share
# Public URL prefix handed back to Discord; host half must match the share
# container's CONJURER_SHARE_SERVER_NAME.
# CONJURER_SHARE_BASE_URL=https://czernobog.pl/share
+49
View File
@@ -0,0 +1,49 @@
# Apache vhost for the Conjurer file-share links.
#
# The entrypoint substitutes __SERVER_NAME__ from CONJURER_SHARE_SERVER_NAME and
# writes the result to /etc/apache2/sites-available/000-default.conf (same
# pattern as docker/icecast.xml.tpl).
#
# What this serves: media_search_functions.publish() creates a symlink named
# after a random 32-hex token inside the share dir, pointing at a file in the
# media library. Apache hands that file out at /share/<token>; revoke_shares.py
# later unlinks it. Nothing is ever copied.
<VirtualHost *:80>
ServerName __SERVER_NAME__
DocumentRoot /var/www/html
# The share links ARE symlinks into the media library, so FollowSymLinks is
# mandatory. SymLinksIfOwnerMatch is deliberately NOT used: the targets are
# owned by whoever owns the media, not by www-data, so it would break every
# link. -Indexes is equally mandatory - a directory listing here would hand
# out every currently live token at once.
<Directory /var/www/html/share>
Options +FollowSymLinks -Indexes
AllowOverride None
Require all granted
# revoke_shares.py keeps its state (.downloads.json, .logpos) in this
# very directory. .downloads.json maps every live token to its download
# time, so serving it would leak the entire active link set.
<FilesMatch "^\.">
Require all denied
</FilesMatch>
</Directory>
# Nothing outside /share is published. The more specific block above wins
# for the share dir itself.
<Directory /var/www/html>
Options -Indexes -FollowSymLinks
AllowOverride None
Require all denied
</Directory>
# Dedicated access log. revoke_shares.py tails exactly this file and matches
# the standard "%r" request line against:
# "\s*GET\s+/share/([0-9a-f]{32})\s+HTTP/
# so the format must stay `combined` (or any format containing %r).
SetEnvIf Request_URI "^/share/" conjurer_share
CustomLog ${APACHE_LOG_DIR}/share_access.log combined env=conjurer_share
ErrorLog ${APACHE_LOG_DIR}/share_error.log
</VirtualHost>