docker: Proxmox deployment for bot/librarian/musician + code fixes

Containerises the three services (each intended for its own Proxmox VM)
and adds the code changes needed to run cleanly on Linux/Docker.

Code fixes:
- constants.py: CONJURER_DATA_DIR roots all writable bot state under one
  mounted volume (per-variable overrides still win; native Pi unaffected)
- conjurer_librarian/search_bot.py + scrape_bot.py: the hardcoded Windows
  DOI database path (C:\Database\chunks\) is now CONJURER_LIBRARIAN_DB_PATH,
  with CONJURER_LIBRARIAN_MAXTHREADS / _CHUNK also env-overridable

Docker:
- docker/Dockerfile.{bot,librarian,musician} + compose.{bot,librarian,musician}.yaml
- docker/env/*.env.example (force-added; real *.env stays gitignored)
- docker/entrypoint.bot.sh seeds default JSON state into /data only when
  absent, so preserved history is never overwritten
- .dockerignore
- docs/deployment/DOCKER_PROXMOX.md: step-by-step runbook incl. preserving
  the existing command/conversation history and cross-VM auth

The bot image uses the vendored yt_dlp/spotify_dl forks (they win on
sys.path over the pip packages), dropping the old sed patching.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Michal Tuszowski
2026-07-02 08:42:31 +02:00
parent 9c8384b10c
commit 2fd4077955
15 changed files with 528 additions and 12 deletions
+7 -9
View File
@@ -19,22 +19,20 @@ Global Variables:
"""
# TODO: Wpiemdolić to wszystko w klasę z loggerem przysłanym z góry
import os
from queue import Empty, Queue
from threading import Thread
import time
q = Queue()
#TODO: Count number of lines in files and print to approximate on which part of the file search is
# DATA FOR TEST ONLY
# MAXTHREADS = 5
# DATABASE_PATH = r'C:\\Database\\chunks_1\\'
# Deployment data is environment-overridable so the local DOI database can live
# on a mounted volume (Docker/Linux) instead of the hardcoded Windows path.
MAXTHREADS = int(os.getenv("CONJURER_LIBRARIAN_MAXTHREADS", "41"))
DATABASE_PATH = os.getenv("CONJURER_LIBRARIAN_DB_PATH", r"C:\\Database\\chunks\\")
# DEPLOYMENT DATA
MAXTHREADS = 41
DATABASE_PATH = r"C:\\Database\\chunks\\"
ENCODING = "utf-8"
CHUNK = "_chunk.txt"
ENCODING = os.getenv("CONJURER_ENCODING", "utf-8")
CHUNK = os.getenv("CONJURER_LIBRARIAN_CHUNK", "_chunk.txt")
_sentinel = object()
WORK_Q_SIZE = 35500000