share: dockerize the file-share pipeline and document it
The Python half of the short-lived share links lived in the repo; the Apache config and the cron entries that make it work were hand-placed on the host, so the feature could not be rebuilt from a checkout. This adds the missing half. Scripts (defaults unchanged, so existing bare-metal cron keeps working): * scan_shares.py / revoke_shares.py take their paths from CONJURER_SHARE_* instead of hardcoding the Pi layout, and create their parent dirs; * the revoke TTL is now CONJURER_SHARE_TTL_SECONDS. Its --help claimed "2min" while the code used a hardcoded 3600 - the help text now reports the real, configured value. New share service: * docker/Dockerfile.share - Apache + the two jobs, reusing the same scripts rather than forking copies; * docker/share-vhost.conf.tpl - the previously undocumented Apache half. Three settings are load-bearing and commented as such: +FollowSymLinks (the shares ARE symlinks), -Indexes (a listing would expose every live token), and a deny rule for dotfiles (revoke_shares.py keeps .downloads.json - a map of every live token - inside the served directory); * docker/entrypoint.share.sh - renders the vhost, seeds the index on first run, then runs scanner/revoker in sleep loops beside Apache (no cron, so their output shows up in docker logs); * compose + env example, incl. the two volumes that MUST be shared with the musician (it creates the links and reads the index). docs/deployment/FILE_SHARING.md documents the mechanism, both deployment routes (docker and existing host Apache), the Discord command, why each Apache setting matters, verification commands, and the known limitations - notably that a link nobody ever downloads is never revoked, since the TTL starts at first download. Verified: scanner and revoker exercised end-to-end against temp dirs (index built; token recorded from a combined-format log line and the symlink unlinked). Compose file not validated - no docker on this machine. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,36 @@
|
||||
# Conjurer share service: Apache publishing short-lived symlink shares, plus
|
||||
# the two periodic jobs that build the search index and revoke expired links.
|
||||
#
|
||||
# This is the piece that used to live only as hand-placed scripts + hand-edited
|
||||
# Apache config on the host. Everything it needs is now in the image; the state
|
||||
# it shares with the musician (the link dir and the index) comes in as volumes.
|
||||
#
|
||||
# Build from the repository root:
|
||||
# docker build -f docker/Dockerfile.share -t conjurer-share .
|
||||
FROM debian:bookworm-slim
|
||||
|
||||
# apache2 serves the links; python3 runs the scanner/revoker (both stdlib-only,
|
||||
# so there is nothing to pip install).
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
apache2 python3 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
# Same scripts the bare-metal deployment uses - no forked copies.
|
||||
COPY conjurer_musician/scan_shares.py ./scan_shares.py
|
||||
COPY conjurer_musician/revoke_shares.py ./revoke_shares.py
|
||||
COPY docker/share-vhost.conf.tpl ./share-vhost.conf.tpl
|
||||
COPY docker/entrypoint.share.sh /usr/local/bin/entrypoint.sh
|
||||
RUN chmod +x /usr/local/bin/entrypoint.sh
|
||||
|
||||
# /mnt/shares - media library (mount read-only; symlink targets)
|
||||
# /var/www/html/share - the symlinks themselves (SHARED with the musician,
|
||||
# which is what creates them)
|
||||
# /srv/share/db - share_scan.json (SHARED with the musician, which
|
||||
# searches it)
|
||||
# /var/log/apache2 - access log the revoker tails
|
||||
VOLUME ["/mnt/shares", "/var/www/html/share", "/srv/share/db", "/var/log/apache2"]
|
||||
|
||||
EXPOSE 80
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
|
||||
@@ -0,0 +1,43 @@
|
||||
# Share VM/host: Apache publishing short-lived file links + the scan/revoke jobs.
|
||||
# Run from the repository root:
|
||||
# cp docker/env/share.env.example docker/env/share.env # then edit
|
||||
# docker compose -f docker/compose.share.yaml up -d --build
|
||||
#
|
||||
# IMPORTANT - this service shares two paths with the musician, which is the
|
||||
# process that actually creates the links:
|
||||
#
|
||||
# /srv/share/links -> the symlinks (musician writes, Apache reads)
|
||||
# /srv/share/db -> share_scan.json (this service writes, musician reads)
|
||||
#
|
||||
# So the musician must mount the same two host paths (see the CONJURER_SHARE_*
|
||||
# block in docker/env/musician.env.example). If the musician runs on a different
|
||||
# machine, put both on shared storage - otherwise it will happily create links
|
||||
# that this container cannot see.
|
||||
#
|
||||
# The media library must be mounted here at the SAME absolute path that the
|
||||
# index recorded, because the symlink targets are absolute. Default: /mnt/shares.
|
||||
services:
|
||||
conjurer-share:
|
||||
build:
|
||||
context: ..
|
||||
dockerfile: docker/Dockerfile.share
|
||||
image: conjurer-share:latest
|
||||
container_name: conjurer-share
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
- env/share.env
|
||||
ports:
|
||||
# Plain HTTP. Terminate TLS for czernobog.pl on your existing reverse
|
||||
# proxy and forward /share here.
|
||||
- "8081:80"
|
||||
volumes:
|
||||
# Media library the links point at. Read-only: this service only ever
|
||||
# serves these files, it never writes them.
|
||||
- /srv/share/media:/mnt/shares:ro
|
||||
# The published symlinks - shared with the musician.
|
||||
- /srv/share/links:/var/www/html/share
|
||||
# The search index - shared with the musician.
|
||||
- /srv/share/db:/srv/share/db
|
||||
# Access log the revoker tails; kept on the host so link history survives
|
||||
# a container rebuild.
|
||||
- /srv/share/logs:/var/log/apache2
|
||||
Executable
+67
@@ -0,0 +1,67 @@
|
||||
#!/bin/sh
|
||||
# Entrypoint for the Conjurer share container: renders the vhost, seeds the
|
||||
# share index, then runs the two periodic jobs alongside Apache.
|
||||
#
|
||||
# Cron is deliberately not used. Both scripts are already written to be safe to
|
||||
# re-run (the scanner writes atomically, the revoker persists its log position),
|
||||
# so plain sleep loops give the same result with one less moving part and with
|
||||
# their output on the container's stdout where `docker logs` can see it.
|
||||
set -eu
|
||||
|
||||
: "${CONJURER_SHARE_SERVER_NAME:=localhost}"
|
||||
: "${CONJURER_SHARE_SCAN_PATH:=/mnt/shares}"
|
||||
: "${CONJURER_SHARE_DB:=/srv/share/db/share_scan.json}"
|
||||
: "${CONJURER_SHARE_DIR:=/var/www/html/share}"
|
||||
: "${CONJURER_APACHE_LOG:=/var/log/apache2/share_access.log}"
|
||||
: "${CONJURER_SHARE_SCAN_INTERVAL:=3600}"
|
||||
: "${CONJURER_SHARE_REVOKE_INTERVAL:=60}"
|
||||
: "${CONJURER_SHARE_TTL_SECONDS:=3600}"
|
||||
export CONJURER_SHARE_SCAN_PATH CONJURER_SHARE_DB CONJURER_SHARE_DIR \
|
||||
CONJURER_APACHE_LOG CONJURER_SHARE_TTL_SECONDS
|
||||
|
||||
# Render the vhost (same placeholder pattern as icecast.xml.tpl).
|
||||
sed "s|__SERVER_NAME__|${CONJURER_SHARE_SERVER_NAME}|g" \
|
||||
/app/share-vhost.conf.tpl > /etc/apache2/sites-available/000-default.conf
|
||||
|
||||
mkdir -p "$CONJURER_SHARE_DIR" \
|
||||
"$(dirname "$CONJURER_SHARE_DB")" \
|
||||
"$(dirname "$CONJURER_APACHE_LOG")" \
|
||||
/var/run/apache2
|
||||
chown -R www-data:www-data "$CONJURER_SHARE_DIR"
|
||||
|
||||
# revoke_shares.py seeks to a stored byte offset, so the log must exist before
|
||||
# the first pass or it starts by reporting an error.
|
||||
touch "$CONJURER_APACHE_LOG"
|
||||
|
||||
# Seed the index immediately when absent - otherwise /get_share_list returns
|
||||
# nothing until the first scheduled scan, which looks like a broken feature.
|
||||
if [ ! -f "$CONJURER_SHARE_DB" ]; then
|
||||
echo "[share] no index at $CONJURER_SHARE_DB - running initial scan of $CONJURER_SHARE_SCAN_PATH"
|
||||
python3 /app/scan_shares.py || echo "[share] initial scan failed, continuing"
|
||||
fi
|
||||
|
||||
# Periodic index rebuild.
|
||||
(
|
||||
while true; do
|
||||
sleep "$CONJURER_SHARE_SCAN_INTERVAL"
|
||||
echo "[share] rescanning $CONJURER_SHARE_SCAN_PATH"
|
||||
python3 /app/scan_shares.py || echo "[share] scan failed"
|
||||
done
|
||||
) &
|
||||
SCAN_PID=$!
|
||||
|
||||
# Periodic revocation of links whose TTL expired after first download.
|
||||
(
|
||||
while true; do
|
||||
sleep "$CONJURER_SHARE_REVOKE_INTERVAL"
|
||||
python3 /app/revoke_shares.py || echo "[share] revoke failed"
|
||||
done
|
||||
) &
|
||||
REVOKE_PID=$!
|
||||
|
||||
trap 'kill "$SCAN_PID" "$REVOKE_PID" 2>/dev/null || true' INT TERM
|
||||
|
||||
echo "[share] serving $CONJURER_SHARE_DIR as ${CONJURER_SHARE_SERVER_NAME}/share"
|
||||
echo "[share] scan every ${CONJURER_SHARE_SCAN_INTERVAL}s, revoke sweep every ${CONJURER_SHARE_REVOKE_INTERVAL}s, TTL ${CONJURER_SHARE_TTL_SECONDS}s after first download"
|
||||
|
||||
exec apache2ctl -D FOREGROUND
|
||||
Vendored
+14
@@ -15,3 +15,17 @@ CONJURER_MUSIC_FOLDER=/music
|
||||
# Runtime paths (mounted volume) — playlists/logs the service writes.
|
||||
CONJURER_MUSICIAN_BASE=/data
|
||||
CONJURER_LOGSTORE=/data/logs
|
||||
|
||||
# --- File sharing (optional; see docs/deployment/FILE_SHARING.md) --------
|
||||
# The musician SEARCHES the index and CREATES the symlinks; the share container
|
||||
# builds the index, serves the links and revokes them. Both must therefore point
|
||||
# at the same two host paths — mount them into this container as well:
|
||||
# - /srv/share/db:/srv/share/db
|
||||
# - /srv/share/links:/var/www/html/share
|
||||
# Leave these unset to keep the feature off (the endpoints then just find
|
||||
# nothing).
|
||||
# CONJURER_SHARE_DB=/srv/share/db/share_scan.json
|
||||
# CONJURER_SHARE_DIR=/var/www/html/share
|
||||
# Public URL prefix handed back to Discord; host half must match the share
|
||||
# container's CONJURER_SHARE_SERVER_NAME.
|
||||
# CONJURER_SHARE_BASE_URL=https://czernobog.pl/share
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# Apache vhost for the Conjurer file-share links.
|
||||
#
|
||||
# The entrypoint substitutes __SERVER_NAME__ from CONJURER_SHARE_SERVER_NAME and
|
||||
# writes the result to /etc/apache2/sites-available/000-default.conf (same
|
||||
# pattern as docker/icecast.xml.tpl).
|
||||
#
|
||||
# What this serves: media_search_functions.publish() creates a symlink named
|
||||
# after a random 32-hex token inside the share dir, pointing at a file in the
|
||||
# media library. Apache hands that file out at /share/<token>; revoke_shares.py
|
||||
# later unlinks it. Nothing is ever copied.
|
||||
|
||||
<VirtualHost *:80>
|
||||
ServerName __SERVER_NAME__
|
||||
DocumentRoot /var/www/html
|
||||
|
||||
# The share links ARE symlinks into the media library, so FollowSymLinks is
|
||||
# mandatory. SymLinksIfOwnerMatch is deliberately NOT used: the targets are
|
||||
# owned by whoever owns the media, not by www-data, so it would break every
|
||||
# link. -Indexes is equally mandatory - a directory listing here would hand
|
||||
# out every currently live token at once.
|
||||
<Directory /var/www/html/share>
|
||||
Options +FollowSymLinks -Indexes
|
||||
AllowOverride None
|
||||
Require all granted
|
||||
|
||||
# revoke_shares.py keeps its state (.downloads.json, .logpos) in this
|
||||
# very directory. .downloads.json maps every live token to its download
|
||||
# time, so serving it would leak the entire active link set.
|
||||
<FilesMatch "^\.">
|
||||
Require all denied
|
||||
</FilesMatch>
|
||||
</Directory>
|
||||
|
||||
# Nothing outside /share is published. The more specific block above wins
|
||||
# for the share dir itself.
|
||||
<Directory /var/www/html>
|
||||
Options -Indexes -FollowSymLinks
|
||||
AllowOverride None
|
||||
Require all denied
|
||||
</Directory>
|
||||
|
||||
# Dedicated access log. revoke_shares.py tails exactly this file and matches
|
||||
# the standard "%r" request line against:
|
||||
# "\s*GET\s+/share/([0-9a-f]{32})\s+HTTP/
|
||||
# so the format must stay `combined` (or any format containing %r).
|
||||
SetEnvIf Request_URI "^/share/" conjurer_share
|
||||
CustomLog ${APACHE_LOG_DIR}/share_access.log combined env=conjurer_share
|
||||
ErrorLog ${APACHE_LOG_DIR}/share_error.log
|
||||
</VirtualHost>
|
||||
Reference in New Issue
Block a user