Compare commits

..

4 Commits

Author SHA1 Message Date
Michal Tuszowski da8260d164 fix: voice_recognition uses constants for credentials and paths
The cog did its own netrc read from a hardcoded /home/pi/.netrc at import
time, crashing on any other host. Now:

- constants.py: ASSEMBLYAI_API_KEY resolved like every other token
  (env ASSEMBLYAI_API_KEY -> netrc machine 'assemblyai' at
  CONJURER_NETRC_FILE); new TRANSCRIPTS_PATH (env
  CONJURER_TRANSCRIPTS_PATH, defaults next to the log file, created by the
  runtime layout)
- voice_recognition_commands.py: drop the hardcoded netrc read and
  transcript dir; when the key is missing raise a clear RuntimeError so
  the guarded loader disables ONLY this cog with a readable reason
- bot.env.example: document ASSEMBLYAI_API_KEY

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 00:33:13 +02:00
Michal Tuszowski b266db5bf3 fix: Client.start() takes no log_handler kwarg (run()-only)
log_handler belongs to Client.run() (which configures logging and then
calls start()); passing it to start() raised TypeError at launch. This was
inherited from the never-run dockerised variant's thin_client.py. We
configure our own handlers, so simply drop the kwarg.

Also set logger.propagate=False: a dependency calls logging.basicConfig(),
so every 'discord' record was printed twice (our format + root's).

Verified with a strict-signature stub client (start(token, *, reconnect)).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 00:26:02 +02:00
Michal Tuszowski 0a29c9950d fix: never swallow task exceptions - diagnose WHY the bot died
The restart-loop with only 'Starting discord bot' in the logs was caused by
gather(return_exceptions=True) eating the client.start() exception: a failed
login looked like a clean exit and docker just restarted the container.

bot.py:
- _run_bot: catch discord.LoginFailure / PrivilegedIntentsRequired with
  CRITICAL messages telling exactly what to fix (token entry / Developer
  Portal intents), full traceback for anything else, then re-raise
- main() returns an exit code; every task exception is re-logged as the
  LAST line in docker logs ('Task died: ...'); process exits 1 on failure
- log the token source (env vs netrc) and its length at startup to catch
  truncated/wrong-field netrc entries
- _run_comm_subroutine: comm crash is logged loudly but no longer relies on
  gather to surface it (bot keeps running - Discord side is independent)

communication_subroutine.py (crash found by the new stub tests):
- comm_subroutine used 'logger' before its local assignment
  (UnboundLocalError killed the whole comm layer at every startup)
- re-comment the flask_debug thread: it binds the same host:port as
  waitress, so running both dies with 'address in use'

Verified via stub-injected client: LoginFailure -> CRITICAL + exit 1,
PrivilegedIntentsRequired -> CRITICAL + exit 1, unknown exception -> full
traceback + exit 1, clean shutdown -> exit 0; comm layer no longer crashes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 00:22:55 +02:00
Michal Tuszowski 8ba954176a fix: make bot startup resilient (self-healing state, gated cogs, loud failures)
Container/native startup died silently on any missing file/service. Now:

constants.py - self-healing runtime layout at import:
- create missing dirs (log dir, LOGSTORE, GRAPHICS_PATH, MUSIC_FOLDER)
- seed missing state files from the repo templates shipped next to
  constants.py (settings/system_gpt/pamiec/pamiec_muzyki/accident_log),
  falling back to safe empty JSON; existing files are NEVER overwritten

bot.py:
- log to stdout too, so 'docker logs' finally shows the crash reason
- missing Discord token = loud sys.exit with mount/env instructions
  (was: silent return -> container crash-loop with empty logs)
- every cog loads independently (one broken cog = skipped with traceback,
  bot continues)
- musician/librarian cogs are health-gated: enabled only when the service
  answers HTTP; a watchdog re-checks every 5 min and enables them the
  moment the service comes alive (no restart needed); tree re-synced
- on_ready reconnects no longer re-load extensions

requirements_conan.txt + Dockerfile.bot: aiomcrcon (Python <=3.11 only)
moved to best-effort extras so the 3.13 image builds clean and the
conanjurer cog stays dormant without it.

DOCKER_PROXMOX.md: startup model (core vs gated cogs) + crash-loop
troubleshooting incl. the 'disappearing files' checklist (nothing in the
stack deletes host files; bind mount = live state).

Verified: fresh-volume seeding creates dirs+templates, existing files
untouched, missing-token exits with FATAL message, health-gating logic
(stub-based runpy tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 00:08:31 +02:00
30 changed files with 394 additions and 1546 deletions
-51
View File
@@ -45,14 +45,6 @@ class Events(commands.Cog):
async def cog_load(self):
self.logger.info("Starting personal assistants")
# Personal assistants use the OpenAI Assistants API (threads/runs), which
# has no Anthropic equivalent - skip cleanly when OpenAI isn't wired up
# (e.g. a Claude-only deployment) instead of crashing the cog load.
if OPENAICLIENT is None:
self.logger.warning(
"OPENAICLIENT niedostępny - osobiści asystenci (OpenAI Assistants API) wyłączeni"
)
return
for superfryta_id, superfryta in SPECJALNE_ZIEMNIACZKI.items():
if superfryta[4] != "":
@@ -109,40 +101,6 @@ class Events(commands.Cog):
else:
await ctx.reply("Nope. Nie wiesz jak użyć")
@commands.hybrid_command(
name="gadaj_teraz",
description="Przełącz backend AI (np. gpt / claude). Tylko Vykidailo.",
)
async def gadaj_teraz(self, ctx, nazwa_konfigu: str):
async with ctx.channel.typing():
is_admin = isinstance(ctx.author, discord.Member) and any(
role.name == "Vykidailo" for role in ctx.author.roles
)
if not is_admin:
await discord_friendly_reply(ctx, "Tylko Vykidailo może przełączać AI.")
return
available = ai_functions.list_ai_configs()
if nazwa_konfigu not in available:
await discord_friendly_reply(
ctx,
f"Nie znam configu '{nazwa_konfigu}'. Dostępne: {', '.join(available)}",
)
return
try:
cfg = ai_functions.set_active_ai_config(nazwa_konfigu)
except (KeyError, RuntimeError) as exc:
await discord_friendly_reply(
ctx, f"Nie mogę przełączyć na '{nazwa_konfigu}': {exc}"
)
return
self.logger.info(
"Przełączono AI na config %s (%s)", nazwa_konfigu, cfg.get("provider")
)
await discord_friendly_reply(
ctx,
f"Teraz gadam przez **{nazwa_konfigu}** — {cfg.get('provider')} / {cfg.get('latest_model')}.",
)
@commands.hybrid_command(
name="armia_hammera",
description="Jeśli nie wiesz jak użyć tej komendy to nawet nie próbuj",
@@ -343,15 +301,6 @@ class Events(commands.Cog):
if "imaginuje sobie:" in message.content:
async with channel.typing():
self.logger.info("Poczatek procedury obrazkowej")
# Image generation is DALL-E (OpenAI); there is no Anthropic
# equivalent, so it stays on OpenAI regardless of the chat
# backend. Degrade gracefully when OpenAI isn't configured.
if OPENAICLIENT is None:
await discord_friendly_reply(
message,
"*Kondziu rozkłada łapska* — malowanie obrazków jest teraz wyłączone (brak OpenAI).",
)
return
message_content_lower = message_content_lower.replace("imaginuje sobie: ", "")
self.logger.debug("Wywolanie obrazka: %s", message_content_lower)
try:
+57 -252
View File
@@ -8,11 +8,8 @@ import tiktoken
import time
from other_functions import discord_friendly_send
from constants import (
AI_CONFIGS,
ASSISTANTS,
CLAUDECLIENT,
CYCLIC_WORDS,
DEFAULT_AI_CONFIG,
ENCODING,
GPT_SETTINGS,
MEMORY_FIVE_MUZYKA,
@@ -26,222 +23,19 @@ from constants import (
LATEST_MODEL
)
try:
import anthropic
except ImportError: # pragma: no cover - optional at runtime
anthropic = None
# this do per user
VECTOR_STORE_ID = -1
# *=========================================== AI provider abstraction
# The AI cog talks to exactly one backend at a time, chosen by _ACTIVE_CONFIG.
# Legacy defaults ("gpt"/OpenAI) keep the historical behaviour byte-for-byte;
# selecting a "claude" config routes the same handle_response pipeline through
# the Anthropic Messages API instead. Backend-specific exceptions are funnelled
# into a single AIError so handle_response can keep its one set of in-character
# error replies regardless of provider.
_ACTIVE_CONFIG_NAME = DEFAULT_AI_CONFIG
# Legacy default algorithm strings that mean "let the bot pick" rather than
# "force this exact model" - so a caller that still passes the old gpt-4o
# default auto-selects the active provider's model instead of 400-ing on Claude.
_AUTO_ALGOS = {"", "auto", "gpt-4o", "gpt-4o-mini", "gpt-3.5-turbo"}
class AIError(Exception):
"""Provider-neutral wrapper so handle_response reacts to one exception type.
``category`` is one of: timeout, connection, bad_request,
response_validation, auth, permission, rate_limit, unprocessable, api.
``original`` is the underlying SDK exception (interpolated into replies).
"""
def __init__(self, category: str, original: Exception):
super().__init__(str(original))
self.category = category
self.original = original
def _active_config() -> dict:
return (
AI_CONFIGS.get(_ACTIVE_CONFIG_NAME)
or AI_CONFIGS.get("gpt")
or next(iter(AI_CONFIGS.values()))
)
def list_ai_configs():
"""Selectable config names (templates prefixed with '_' are hidden)."""
return [name for name in AI_CONFIGS if not name.startswith("_")]
def get_active_ai_config() -> str:
return _ACTIVE_CONFIG_NAME
def set_active_ai_config(name: str) -> dict:
"""Switch the active AI backend and persist the choice. Raises on error."""
global _ACTIVE_CONFIG_NAME
if name not in AI_CONFIGS:
raise KeyError(name)
cfg = AI_CONFIGS[name]
provider = cfg.get("provider")
if provider == "anthropic" and CLAUDECLIENT is None:
raise RuntimeError("klient Anthropic nie jest skonfigurowany (brak ANTHROPIC_API_KEY)")
if provider == "openai" and OPENAICLIENT is None:
raise RuntimeError("klient OpenAI nie jest skonfigurowany (brak OPENAI_API_KEY)")
_ACTIVE_CONFIG_NAME = name
_persist_active_ai_config(name)
return cfg
def _persist_active_ai_config(name: str) -> None:
"""Best-effort write of the active-config choice into system_gpt_settings.json.
Keeps the historical two-element structure intact: updates index 2 if it
already exists, appends it when the file has exactly the original two
elements, and otherwise leaves the file untouched (the in-memory switch
still applies).
"""
logger = logging.getLogger("discord")
try:
with open(SYSTEM_GPT_SETTINGS, "r", encoding=ENCODING) as handle:
data = json.load(handle)
except (OSError, json.JSONDecodeError) as exc:
logger.warning("Nie mogę odczytać %s do zapisu configu AI: %s", SYSTEM_GPT_SETTINGS, exc)
return
if not isinstance(data, list) or len(data) < 2:
logger.warning("Nietypowa struktura %s - pomijam zapis configu AI", SYSTEM_GPT_SETTINGS)
return
if len(data) > 2 and isinstance(data[2], dict):
data[2]["active"] = name
data[2].setdefault("configs", AI_CONFIGS)
else:
data = data[:2] + [{"active": name, "configs": AI_CONFIGS}]
try:
with open(SYSTEM_GPT_SETTINGS, "w", encoding=ENCODING) as handle:
json.dump(data, handle, indent=4, ensure_ascii=False)
except OSError as exc:
logger.warning("Nie mogę zapisać configu AI do %s: %s", SYSTEM_GPT_SETTINGS, exc)
def _map_openai_error(exc: Exception) -> AIError:
mapping = [
(openai.APITimeoutError, "timeout"),
(openai.APIConnectionError, "connection"),
(openai.BadRequestError, "bad_request"),
(openai.APIResponseValidationError, "response_validation"),
(openai.AuthenticationError, "auth"),
(openai.PermissionDeniedError, "permission"),
(openai.RateLimitError, "rate_limit"),
(openai.UnprocessableEntityError, "unprocessable"),
(openai.APIError, "api"),
]
for cls, category in mapping:
if isinstance(exc, cls):
return AIError(category, exc)
return AIError("api", exc)
def _map_anthropic_error(exc: Exception) -> AIError:
mapping = [
("APITimeoutError", "timeout"),
("APIConnectionError", "connection"),
("BadRequestError", "bad_request"),
("APIResponseValidationError", "response_validation"),
("AuthenticationError", "auth"),
("PermissionDeniedError", "permission"),
("RateLimitError", "rate_limit"),
("UnprocessableEntityError", "unprocessable"),
("APIError", "api"),
]
for name, category in mapping:
cls = getattr(anthropic, name, None)
if cls and isinstance(exc, cls):
return AIError(category, exc)
return AIError("api", exc)
def _to_anthropic_messages(messages):
"""Split OpenAI-style messages into (system_prompt, alternating convo).
Claude takes the system prompt as a separate parameter (not a role in the
messages list) and requires the conversation to open with a user turn, so
system messages are concatenated out and any leading assistant turns are
dropped.
"""
system_parts = []
convo = []
for msg in messages:
role = msg.get("role")
content = msg.get("content", "")
if role == "system":
system_parts.append(content)
else:
convo.append(
{"role": "assistant" if role == "assistant" else "user", "content": content}
)
while convo and convo[0]["role"] != "user":
convo.pop(0)
if not convo:
convo = [{"role": "user", "content": " "}]
return "\n\n".join(part for part in system_parts if part), convo
async def _anthropic_call(messages, model, cfg):
"""Claude counterpart of openai_call. Returns a plain string."""
if CLAUDECLIENT is None:
raise AIError("auth", RuntimeError("klient Anthropic nie jest skonfigurowany"))
system_prompt, convo = _to_anthropic_messages(messages)
kwargs = {
"model": model,
"max_tokens": int(cfg.get("max_tokens", 2048)),
"messages": convo,
}
if system_prompt:
kwargs["system"] = system_prompt
# NOTE: temperature is deliberately omitted - Opus 4.8 / Sonnet 5 reject
# sampling params with a 400.
try:
resp = await CLAUDECLIENT.messages.create(**kwargs)
except Exception as exc: # pylint: disable=broad-except
raise _map_anthropic_error(exc)
text = "".join(
block.text for block in resp.content if getattr(block, "type", None) == "text"
)
return text.strip()
async def provider_generate(messages, model, temperature=0.2):
"""Dispatch a chat completion to the active backend, normalising errors."""
cfg = _active_config()
try:
if cfg.get("provider") == "anthropic":
return await _anthropic_call(messages, model, cfg)
return await openai_call(messages, model, temperature)
except AIError:
raise
except Exception as exc: # pylint: disable=broad-except
# Only the OpenAI path reaches here un-normalised (_anthropic_call
# already wraps its own errors).
raise _map_openai_error(exc)
def select_model(req_type: str, algo: str) -> str:
cfg = _active_config()
latest = cfg.get("latest_model", LATEST_MODEL)
cheap = cfg.get("cheap_model", CHEAP_MODEL)
algo_str = (algo or "").strip()
# An explicit, non-legacy model id is honoured verbatim; anything in
# _AUTO_ALGOS (incl. the old gpt-4o default) means "auto pick for the
# active provider", so flipping the switch actually changes the model.
if algo_str and algo_str.lower() not in _AUTO_ALGOS:
return algo_str
# Jeżeli jawnie podano algorithm (i nie jest 'auto'/''):
if algo and str(algo).strip().lower() not in ("auto",):
# wyjątek: MUZYKA ma zawsze być tania — nadpisujemy TYLKO jeśli przyszedł domyślny 'gpt-4o'
if req_type == "MUSIC" and algo.strip() in (LATEST_MODEL,):
return CHEAP_MODEL
return algo
# Auto-dobór:
if req_type == "MUSIC":
return cheap
return latest
return CHEAP_MODEL
return LATEST_MODEL
async def openai_call(messages, model, temperature=0.2):
@@ -462,46 +256,57 @@ async def handle_response(
timeout_sec = 120
deadline = time.time() + timeout_sec
response = await asyncio.wait_for(
provider_generate(messages=history_msgs, model=model_to_use),
openai_call(messages=history_msgs, model=model_to_use),
timeout=max(0.1, deadline - time.time()),
)
except AIError as e:
# One handler for both backends; e.category is provider-neutral and
# e.original is the underlying SDK exception (kept for the {..} tails).
err = e.original
if e.category == "timeout":
response = f"*Kondziu patrzy na terminal, czeka, czeka, czeka,.... Jeszcze chwile czeka Przypierdala w niego pięścią....* Nie mogę się połączyć z Openai spróbuj od nowa. *Na ekranie pojawia się*: {err}"
elif e.category == "connection":
response = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Nie mogę się połączyć z Openai. *Na ekranie pojawia się*: {err}"
elif e.category in ("bad_request", "response_validation"):
# Handle invalid request error, e.g. validate parameters or log
if internal_retry:
resp = "Nie umiem tego teraz ładnie wytłumaczyć — OpenAI mnie zastrzeliło."
else:
resp, _ = await handle_response(
f"Wytlumacz jakie sa zasady dotyczące treści które możesz generować używając Dalle. Wytłumacz błąd {err} prostym językiem. Przeproś za nadmierną cenzurę. Wytłumacz co mogło być nie tak w prompcie 'prompt'",
True,
True,
MESSAGE_TABLE,
username,
"RANDOM",
internal_retry=True,
)
response = f"Sorki, cenzura: {resp}. Jak chcesz to są kanały na nudle #sexy-foteczky i #kanal-do-fapania *Na ekranie pojawia się: {err}"
elif e.category == "auth":
# Handle authentication error, e.g. check credentials or log
response = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Wołaj szefa - coś się z hasłem zjebało. *Na terminalu pojawia się:* {err}"
elif e.category == "permission":
# Handle permission error, e.g. check scope or log
response = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Wołaj szefa - coś się z uprawnieniami zjebało. *Na terminalu pojawia się:* {err}"
elif e.category == "rate_limit":
response = f"*Kondziu patrzy na terminal* Wołaj szefa. Zapłacić rachunki za AI trzeba. Jak chcesz to się na #zebranie dorzuć. {err}"
elif e.category == "unprocessable":
response = f"*Kondziu patrzy na terminal. Potem na to co każesz mu wysłać....* Ja wiem że jesteśmy w barze BDSM - ale nie da się włożyć TEGO w TO. *Za jego plecami na terminalu pojawia się:* {err}"
else: # "api" and anything unmapped
# Handle API error, e.g. retry or log
response = f"*Kondziu nurkuje za bar, terminal wybucha. Przed tobą ląduje pergamin zapisany pięknym gotykiem a na nim*: {err}"
except openai.APITimeoutError as e:
# Handle timeout error, e.g. retry or log
response = f"*Kondziu patrzy na terminal, czeka, czeka, czeka,.... Jeszcze chwile czeka Przypierdala w niego pięścią....* Nie mogę się połączyć z Openai spróbuj od nowa. *Na ekranie pojawia się*: {e}"
except openai.APIConnectionError as e:
response = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Nie mogę się połączyć z Openai. *Na ekranie pojawia się*: {e}"
except openai.BadRequestError as e:
# Handle invalid request error, e.g. validate parameters or log
if internal_retry:
resp = "Nie umiem tego teraz ładnie wytłumaczyć — OpenAI mnie zastrzeliło."
else:
resp, _ = await handle_response(
f"Wytlumacz jakie sa zasady dotyczące treści które możesz generować używając Dalle. Wytłumacz błąd {e} prostym językiem. Przeproś za nadmierną cenzurę. Wytłumacz co mogło być nie tak w prompcie 'prompt'",
True,
True,
MESSAGE_TABLE,
username,
"RANDOM",
)
response = f"Sorki, cenzura: {resp}. Jak chcesz to są kanały na nudle #sexy-foteczky i #kanal-do-fapania *Na ekranie pojawia się: {e}"
except openai.APIResponseValidationError as e:
# Handle invalid request error, e.g. validate parameters or log
if internal_retry:
resp = "Nie umiem tego teraz ładnie wytłumaczyć — OpenAI mnie zastrzeliło."
else:
resp, _ = await handle_response(
f"Wytlumacz jakie sa zasady dotyczące treści które możesz generować używając Dalle. Wytłumacz błąd {e} prostym językiem. Przeproś za nadmierną cenzurę. Wytłumacz co mogło być nie tak w prompcie 'prompt'",
True,
True,
MESSAGE_TABLE,
username,
"RANDOM",
)
response = f"Sorki, cenzura: {resp}. Jak chcesz to są kanały na nudle #sexy-foteczky i #kanal-do-fapania *Na ekranie pojawia się: {e}"
except openai.AuthenticationError as e:
# Handle authentication error, e.g. check credentials or log
response = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Wołaj szefa - coś się z hasłem zjebało. *Na terminalu pojawia się:* {e}"
except openai.PermissionDeniedError as e:
# Handle permission error, e.g. check scope or log
response = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Wołaj szefa - coś się z uprawnieniami zjebało. *Na terminalu pojawia się:* {e}"
except openai.RateLimitError as e:
response = f"*Kondziu patrzy na terminal* Wołaj szefa. Zapłacić rachunki za AI trzeba. Jak chcesz to się na #zebranie dorzuć. {e}"
except openai.UnprocessableEntityError as e:
response = f"*Kondziu patrzy na terminal. Potem na to co każesz mu wysłać....* Ja wiem że jesteśmy w barze BDSM - ale nie da się włożyć TEGO w TO. *Za jego plecami na terminalu pojawia się:* {e}"
except openai.APIError as e:
# Handle API error, e.g. retry or log
response = f"*Kondziu nurkuje za bar, terminal wybucha. Przed tobą ląduje pergamin zapisany pięknym gotykiem a na nim*: {e}"
logger.info("Historia wysłana:")
temp_assistant = {"role": "assistant", "content": response}
+2 -8
View File
@@ -37,7 +37,6 @@ from constants import (
GET_MP3,
LIBRARIAN_SERVICE_ADDRESS,
LOGFILE,
RADIO_SERVICE_ADDRESS,
TOKEN,
)
@@ -97,15 +96,10 @@ CORE_EXTENSIONS = [
]
SERVICE_EXTENSION_GROUPS = {
# musician (file service): Discord music download/search, file shares
# musician (file service): music download/search, radio control, file shares
"musician": {
"health_url": f"{FILE_SERVICE_ADDRESS}{GET_MP3}",
"extensions": ["music_commands", "file_search_commands"],
},
# betoniarka (radio operator colocated with Liquidsoap): radio playlists
"radio": {
"health_url": f"{RADIO_SERVICE_ADDRESS}/ping",
"extensions": ["radio_commands"],
"extensions": ["music_commands", "radio_commands", "file_search_commands"],
},
# librarian: DOI / Crossref search
"librarian": {
-353
View File
@@ -1,353 +0,0 @@
"""Betoniarka - the radio operator service.
Lives in the SAME container as Liquidsoap and runs as the SAME unprivileged
user ('radio'), which is the whole point: the process that writes the radio
playlists is colocated with the process that watches them, so there is no
cross-host permission juggling (root-owned network shares, failing chowns)
anymore.
Responsibilities (extracted from conjurer_musician, which is now a pure
Discord music player):
- scan the local music library into all_playlist.playlist / hit.playlist
- serve the radio-management HTTP API the bot calls
(/add_to_priority, /create_priority_playlist, /request_radio_file,
/clear_pr_pls) plus /ping for health checks and /stream for the web page
- tail radio_log.log / persistence.log and forward "now playing" events to
the bot's /prepped_tracks
"""
import json
import logging
import os
import random
import re
import threading
import time
from datetime import datetime
from pathlib import Path
from typing import Dict, List
import requests
from flask import Flask, abort, jsonify, request, send_file
from waitress import serve
def _env(name: str, default: str) -> str:
return os.getenv(name, default)
API_KEY = os.getenv("CONJURER_API_KEY")
MAIN_BOT_ADDRESS = _env("CONJURER_MAIN_BOT", "http://127.0.0.1:5000")
MUSIC_TRACKER = _env("CONJURER_MUSIC_TRACKER_ENDPOINT", "/prepped_tracks")
HOST_ADDRESS = _env("BETONIARKA_HOST", "0.0.0.0")
PORT_ADDRESS = int(_env("BETONIARKA_PORT", "5005"))
DATA_DIR = Path(_env("BETONIARKA_DATA", "/srv/betoniarka/data"))
MUSIC_FOLDER = Path(_env("BETONIARKA_MUSIC", "/srv/betoniarka/music"))
PRIORITY_FOLDER = Path(_env("BETONIARKA_PRIORITY_FOLDER", str(MUSIC_FOLDER / "priority")))
STREAM_TEMPLATE = _env("BETONIARKA_STREAM_TEMPLATE", "/app/stream.html")
RESCAN_SECONDS = int(_env("BETONIARKA_RESCAN_SECONDS", str(24 * 60 * 60)))
# How many leading path tokens to ignore when keyword-matching
# (/srv/betoniarka/music/... -> '', 'srv', 'betoniarka', 'music').
PATH_SKIP = int(_env("BETONIARKA_PATH_SKIP", "4"))
ALL_PLAYLIST_PATH = DATA_DIR / "all_playlist.playlist"
HIT_PLAYLIST_PATH = DATA_DIR / "hit.playlist"
REQUEST_PLAYLIST_PATH = DATA_DIR / "request.playlist"
PRIORITY_PLAYLIST_PATH = DATA_DIR / "priority_queue.playlist"
RADIOLOG_PATH = DATA_DIR / "radio_log.log"
PERSISTENCE_PATH = DATA_DIR / "persistence.log"
ENCODING = _env("CONJURER_ENCODING", "utf-8")
logger = logging.getLogger("betoniarka")
music_file_list: List[str] = []
priority_list: List[str] = []
app = Flask(__name__)
def _build_headers() -> Dict[str, str]:
if API_KEY:
return {"X-Conjurer-Api-Key": API_KEY}
return {}
def _authorize_request() -> None:
if API_KEY and request.headers.get("X-Conjurer-Api-Key") != API_KEY:
abort(401)
def _post_to_bot(payload: List[str]) -> None:
try:
response = requests.post(
f"{MAIN_BOT_ADDRESS}{MUSIC_TRACKER}",
json=payload,
headers=_build_headers(),
timeout=60,
)
logger.info("Forwarded to bot (%s): %s", response.status_code, payload[0])
except requests.exceptions.RequestException as exc:
logger.warning("Bot unreachable, dropping %s: %s", payload[0], exc)
# ---------------------------------------------------------------- library
def rescan():
"""Scan the local library into the playlists Liquidsoap watches.
Paths written here are LOCAL container paths, the same ones Liquidsoap
resolves - no shared network filesystem involved.
"""
logger.info("Rescan triggered")
music_file_list.clear()
priority_list.clear()
for mp3_item in MUSIC_FOLDER.glob("**/*.mp3"):
music_file_list.append(mp3_item.as_posix())
for mp3_item in PRIORITY_FOLDER.glob("**/*.mp3"):
priority_list.append(mp3_item.as_posix())
with ALL_PLAYLIST_PATH.open("w", encoding=ENCODING) as w_file:
for item in music_file_list:
w_file.write(item + "\n")
with HIT_PLAYLIST_PATH.open("w", encoding=ENCODING) as w_file:
for item in priority_list:
w_file.write(item + "\n")
logger.info("Rescan done: %d tracks, %d hits", len(music_file_list), len(priority_list))
def thread_rescan():
while True:
time.sleep(RESCAN_SECONDS)
rescan()
# ---------------------------------------------------------------- search
def remove_characters(string, character):
return string.replace(character, "")
def max_weight(lista):
maximum_weight = 0
for iterator in lista:
if iterator[0] > maximum_weight:
maximum_weight = iterator[0]
return maximum_weight
_CHAR_REMOVE = [
".", "^", "$", "*", "+", "?", "{", "}", "[", "]",
"\\", "/", "|", "(", ")", "!", ",", "-", ":", "mp3",
]
def wyszukaj(word_list, how_many, _logger=None, write_to=None):
"""Keyword-score the library; optionally append hits to a playlist file.
Ported unchanged from the musician (same scoring), minus the win32
branches. ``write_to`` replaces the old ``return_to_bot`` flag: pass a
playlist Path to append the result, or None to just return it.
"""
fun_logger = _logger or logger
search_weight = [(0, "") for _ in range(len(music_file_list))]
time_start = datetime.now()
skip_start = 2 if int(how_many) > 0 else 1
for word in word_list[skip_start:]:
token_weight = len(word)
fun_logger.info("Słowo kluczowe: %s", word)
for itr, file in enumerate(music_file_list):
parts = file.split("/")
all_words = []
for f_iter in parts:
for char in _CHAR_REMOVE:
f_iter = remove_characters(f_iter, char)
all_words.extend(f_iter.split())
pingu = 1
pattern_len = len(all_words)
matched_times = 1
for itm in all_words[PATH_SKIP:]:
pingu += 1
if re.match(".*" + word + ".*", itm, re.IGNORECASE):
temp_weight = (
search_weight[itr][0]
+ (token_weight + (pingu**1.5) / pattern_len) / matched_times
)
search_weight[itr] = (temp_weight, music_file_list[itr])
matched_times += 1
fun_logger.info("Stworzylem tablice wag zajęło mi to %s", datetime.now() - time_start)
best = max_weight(search_weight)
if best == 0:
return []
return_list = []
if int(how_many) <= 0:
for weight, path in search_weight:
if weight == best:
return_list.append((weight, path))
break
else:
search_weight.sort(key=lambda x: x[0], reverse=True)
return_list.extend(search_weight[: int(how_many)])
if write_to is not None:
with write_to.open("a", encoding=ENCODING) as s_file:
for item in return_list:
s_file.write(item[1] + "\n")
fun_logger.info("Done: %s", return_list)
return return_list
# ---------------------------------------------------------------- tailer
def scan_tracks():
"""Tail the radio logs and forward play events to the bot."""
with open(RADIOLOG_PATH, "r", encoding=ENCODING) as log_file:
log_file.seek(os.stat(RADIOLOG_PATH).st_size)
prev_size = os.stat(PERSISTENCE_PATH).st_size
while True:
current_size = os.stat(PERSISTENCE_PATH).st_size
if prev_size != current_size:
while prev_size != current_size:
prev_size = current_size
time.sleep(0.1)
current_size = os.stat(PERSISTENCE_PATH).st_size
with open(PERSISTENCE_PATH, "r", encoding=ENCODING) as persistence:
lines = persistence.readlines()
if len(lines) >= 3:
_post_to_bot(["next", lines[2]])
position = log_file.tell()
line = log_file.readline()
if not line:
time.sleep(1)
log_file.seek(position)
continue
if not re.match(r".*Prepared.*", line):
time.sleep(0.1)
continue
result = None
if re.match(r".*jingles.*", line):
result = ["jingles", line]
elif re.match(r".*priority.*", line):
result = ["priority", line]
elif re.match(r".*hit.*", line):
result = ["hit", line]
elif re.match(r".*all_playlist.*", line):
result = ["all", line]
elif re.match(r".*request.*", line):
result = ["requests", line]
if result:
logger.info("Forwarding radio log entry: %s", result[0])
_post_to_bot(result)
time.sleep(0.1)
# ---------------------------------------------------------------- routes
@app.route("/ping", methods=["GET"])
def ping():
"""Health check - the bot gates radio_commands on this answering."""
return jsonify("ALIVE")
@app.route("/stream", methods=["GET"])
def stream_page():
return send_file(STREAM_TEMPLATE)
@app.route("/clear_pr_pls", methods=["GET"])
def clear_pr_pls():
_authorize_request()
app.logger.info("CLEARING PLAYLIST")
with PRIORITY_PLAYLIST_PATH.open("w", encoding=ENCODING) as cleared_pl:
cleared_pl.write("")
return jsonify(isError=False, message="Success", statusCode=200, data=[]), 200
@app.route("/rescan", methods=["GET"])
def manual_rescan():
_authorize_request()
rescan()
return jsonify(isError=False, message="Success", statusCode=200,
data={"tracks": len(music_file_list)}), 200
@app.route("/request_radio_file", methods=["POST"])
def add_request():
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
wyszukaj(record["lista_slow"], 0, app.logger, write_to=REQUEST_PLAYLIST_PATH)
return jsonify(isError=False, message="Success", statusCode=200,
data={"status": "OK"}), 200
@app.route("/create_priority_playlist", methods=["POST"])
def create_priority_playlist():
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
return_data = wyszukaj(
record["lista_slow"], record["dlugosc_plejlisty"], app.logger, write_to=None
)
random.shuffle(return_data)
# NOTE: appends to the REQUEST playlist - behaviour inherited verbatim
# from the musician implementation (the request queue picks it up).
with REQUEST_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return jsonify(isError=False, message="Success", statusCode=200,
data={"status": "OK"}), 200
@app.route("/add_to_priority", methods=["POST"])
def add_to_priority():
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
wyszukaj(
record["lista_slow"], record["dlugosc_plejlisty"], app.logger,
write_to=PRIORITY_PLAYLIST_PATH,
)
return jsonify(isError=False, message="Success", statusCode=200,
data={"status": "OK"}), 200
def waitress_run():
serve(app, host=HOST_ADDRESS, port=PORT_ADDRESS)
if __name__ == "__main__":
logger.setLevel(logging.DEBUG)
console = logging.StreamHandler()
console.setFormatter(
logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
)
logger.addHandler(console)
rescan()
logger.info("Betoniarka started on %s:%s", HOST_ADDRESS, PORT_ADDRESS)
threads = [
threading.Thread(target=waitress_run, daemon=True),
threading.Thread(target=thread_rescan, daemon=True),
]
for worker in threads:
worker.start()
time.sleep(5)
track_thread = threading.Thread(target=scan_tracks, daemon=True)
track_thread.start()
try:
for worker in threads:
worker.join()
track_thread.join()
except KeyboardInterrupt:
logger.info("Shutdown requested - exiting betoniarka")
+4 -7
View File
@@ -25,7 +25,6 @@ from queue import Queue
from typing import Dict, Optional
import requests
import lib_paths
import scrape_bot
import search_bot
# import search_bot2 as search_bot
@@ -195,7 +194,7 @@ class Librarian(object):
self.app.logger.info("CROSSREF DONE")
self.app.logger.info("CROSSREF DONE")
with open(lib_paths.CR_RESULTS, "r+", encoding="utf-8") as data_file:
with open("cr_results.json", "r+", encoding="utf-8") as data_file:
# First we load existing data into a dict.
try:
file_data = json.load(data_file)
@@ -259,7 +258,7 @@ class Librarian(object):
for item in temp:
refined_result[item["DOI"]]= item
with open(lib_paths.RR_RESULTS, "r+", encoding="utf-8") as data_file:
with open("rr_results.json", "r+", encoding="utf-8") as data_file:
# First we load existing data into a dict.
try:
file_data = json.load(data_file)
@@ -418,8 +417,7 @@ class BackgroundTaskSearch(threading.Thread):
self.app.logger.info("Saving to file")
# Save results to "not_in_db.json" file
with open(lib_paths.NOT_IN_DB, "r+", encoding="utf-8") as ndb_file:
ndb_database = {}
with open("not_in_db.json", "r+", encoding="utf-8") as ndb_file:
try:
ndb_database = json.load(ndb_file)
except JSONDecodeError:
@@ -433,8 +431,7 @@ class BackgroundTaskSearch(threading.Thread):
json.dump(ndb_database, ndb_file)
# Save results to "s_results.json" file
with open(lib_paths.S_RESULTS, "r+", encoding="utf-8") as s_file:
database = {}
with open("s_results.json", "r+", encoding="utf-8") as s_file:
try:
database = json.load(s_file)
except JSONDecodeError:
-41
View File
@@ -1,41 +0,0 @@
"""Resolved, seeded paths for the librarian's runtime JSON state.
``conjurer_librarian.py`` and ``scrape_bot.py`` open these files in place with
mode ``r+``, which requires them to already exist - in a fresh container the
working directory has none of them, so the worker threads crashed with
FileNotFoundError.
They now live in a single mounted, persistent directory
(``CONJURER_LIBRARIAN_STATE_DIR``, default ``/lib_temp_files``) and are seeded
with an empty JSON object on import, so a fresh container/volume never crashes
and the accumulated results survive restarts.
"""
import os
STATE_DIR = os.getenv("CONJURER_LIBRARIAN_STATE_DIR", "/lib_temp_files")
CR_RESULTS = os.path.join(STATE_DIR, "cr_results.json") # Crossref raw hits
RR_RESULTS = os.path.join(STATE_DIR, "rr_results.json") # refined results
NOT_IN_DB = os.path.join(STATE_DIR, "not_in_db.json") # DOIs to scrape
S_RESULTS = os.path.join(STATE_DIR, "s_results.json") # final send results
_ALL = (CR_RESULTS, RR_RESULTS, NOT_IN_DB, S_RESULTS)
def ensure_state_files():
"""Create the state dir and seed any missing file with an empty JSON dict."""
try:
os.makedirs(STATE_DIR, exist_ok=True)
except OSError as exc: # pragma: no cover - surfaced in logs, not fatal
print(f"lib_paths: cannot create {STATE_DIR}: {exc}")
return
for path in _ALL:
if not os.path.exists(path):
try:
with open(path, "w", encoding="utf-8") as handle:
handle.write("{}")
except OSError as exc: # pragma: no cover
print(f"lib_paths: cannot seed {path}: {exc}")
ensure_state_files()
+1 -3
View File
@@ -16,8 +16,6 @@ from urllib.request import urlopen
from requests import ConnectionError as RequestsConnectionError
from requests import ConnectTimeout, Timeout
import lib_paths
SCR_DATABASE_PATH = os.getenv("CONJURER_LIBRARIAN_DB_PATH", r"C:\\Database\\chunks\\")
SCR_FILENAME = os.getenv("CONJURER_LIBRARIAN_SCRAPE_CHUNK", "40_chunk.txt")
SCR_ENCODING = os.getenv("CONJURER_ENCODING", "utf-8")
@@ -32,7 +30,7 @@ def load_ndb_to_q(logger):
"""
logger.info("Loader started")
while True:
with open(lib_paths.NOT_IN_DB, "r+", encoding="utf-8") as ndb_file:
with open("not_in_db.json", "r+", encoding="utf-8") as ndb_file:
try:
ndb_database = json.load(ndb_file)
for _ in range (1,10):
+1 -1
View File
@@ -47,7 +47,7 @@ def producer(out_q, control_q, filename, _logger):
filename (str): Name of the file.
_logger: Logger object for logging.
"""
with open(DATABASE_PATH + filename, "r", encoding=ENCODING) as operated_file:
with open(DATABASE_PATH + filename, "r+", encoding=ENCODING) as operated_file:
print(f"Worker {filename} ")
line_no = 0
while True:
+1 -1
View File
@@ -47,7 +47,7 @@ def producer(out_q, control_q, filename, _logger):
filename (str): Name of the file.
_logger: Logger object for logging.
"""
with open(DATABASE_PATH + filename, "r", encoding=ENCODING) as operated_file:
with open(DATABASE_PATH + filename, "r+", encoding=ENCODING) as operated_file:
print(f"Worker {filename} ")
line_no = 0
while True:
+298 -15
View File
@@ -1,12 +1,7 @@
"""Musician - the Discord music player service.
Serves the music library index and keyword search the bot uses for Discord
playback (/mp3, /update_mp3, /get_music) plus the file-share endpoints.
Radio playlist management and the radio-log tailer moved to
conjurer_betoniarka/betoniarka.py, which runs INSIDE the radio container as
the same user as Liquidsoap - the musician no longer writes any radio files,
so the old root-owned-network-share permission mess is gone.
"""
The provided Python script sets up a Flask web server to manage a list of music files, with
functions for rescanning the music folder, updating the music list, and serving the music list via
API endpoints.
"""
import json
@@ -19,9 +14,18 @@ import time
from datetime import datetime
from logging import handlers
from pathlib import Path
from typing import List
from typing import Dict, List
from flask import Flask, abort, jsonify, request
import requests
from flask import (
Flask,
abort,
jsonify,
redirect,
render_template,
request,
send_from_directory,
)
from waitress import serve
import media_search_functions
@@ -37,6 +41,8 @@ def _env_path(name: str, default: str) -> Path:
API_KEY = os.getenv("CONJURER_API_KEY")
MAIN_BOT_ADDRESS = _env("CONJURER_MAIN_BOT", "http://127.0.0.1:5000")
MUSIC_TRACKER = _env("CONJURER_MUSIC_TRACKER_ENDPOINT", "/prepped_tracks")
HOST_ADDRESS = _env("CONJURER_MUSICIAN_HOST", "0.0.0.0")
PORT_ADDRESS = int(_env("CONJURER_MUSICIAN_PORT", "5000"))
@@ -50,12 +56,52 @@ LOGSTORE = _env_path("CONJURER_LOGSTORE", str(BASE_DIR / "logs"))
MUSIC_FOLDER = _env_path(
"CONJURER_MUSIC_FOLDER", str(BASE_DIR / "music")
)
PRIORITY_FOLDER = _env_path(
"CONJURER_PRIORITY_FOLDER", str(MUSIC_FOLDER / "priority")
)
RADIOLOG_PATH = _env_path(
"CONJURER_RADIO_LOG", str(BASE_DIR / "radio_log.log")
)
PERSISTENCE_PATH = _env_path(
"CONJURER_PERSISTENCE_LOG", str(BASE_DIR / "persistence.log")
)
ALL_PLAYLIST_PATH = _env_path(
"CONJURER_ALL_PLAYLIST", str(BASE_DIR / "all_playlist.playlist")
)
HIT_PLAYLIST_PATH = _env_path(
"CONJURER_HIT_PLAYLIST", str(BASE_DIR / "hit.playlist")
)
REQUEST_PLAYLIST_PATH = _env_path(
"CONJURER_REQUEST_PLAYLIST", str(BASE_DIR / "request.playlist")
)
PRIORITY_PLAYLIST_PATH = _env_path(
"CONJURER_PRIORITY_PLAYLIST", str(BASE_DIR / "priority_queue.playlist")
)
STREAM_TEMPLATE = _env_path(
"CONJURER_STREAM_TEMPLATE", str(BASE_DIR / "stream.html")
)
ENCODING = _env("CONJURER_ENCODING", "utf-8")
SEPARATOR_FILE_PATH = os.sep
for playlist_path in (
ALL_PLAYLIST_PATH,
HIT_PLAYLIST_PATH,
REQUEST_PLAYLIST_PATH,
PRIORITY_PLAYLIST_PATH,
):
playlist_path.parent.mkdir(parents=True, exist_ok=True)
random.seed()
music_file_list: List[str] = []
priority_list: List[str] = []
def _build_headers() -> Dict[str, str]:
headers: Dict[str, str] = {}
if API_KEY:
headers["X-Conjurer-Api-Key"] = API_KEY
return headers
def _authorize_request() -> None:
@@ -63,16 +109,29 @@ def _authorize_request() -> None:
abort(401)
def rescan():
"""Refresh the in-memory library index used by /mp3 and /get_music.
def _post_to_bot(payload: List[str]) -> None:
response = requests.post(
f"{MAIN_BOT_ADDRESS}{MUSIC_TRACKER}",
json=payload,
headers=_build_headers(),
timeout=60,
)
logger = logging.getLogger("conjurer_musician")
logger.info("SENT")
logger.info(response.status_code)
logger.info("SEND CONFIRMED")
Radio playlist files are NOT written here anymore - that is the
betoniarka's job, colocated with Liquidsoap.
def rescan():
"""
The `rescan` function logs a message, scans for mp3 files in a specified folder,
and adds them to a list of music files.
"""
logger = logging.getLogger("conjurer_musician")
logger.info("Rescan triggered")
music_file_list.clear()
priority_list.clear()
for mp3_item in MUSIC_FOLDER.glob("**/*.mp3"):
temp_music_file = mp3_item.as_posix()
@@ -80,6 +139,27 @@ def rescan():
temp_music_file = temp_music_file.replace("/", "\\")
music_file_list.append(temp_music_file)
for mp3_item in PRIORITY_FOLDER.glob("**/*.mp3"):
temp_music_file = mp3_item.as_posix()
if os.name == "nt":
temp_music_file = temp_music_file.replace("/", "\\")
priority_list.append(temp_music_file)
with ALL_PLAYLIST_PATH.open("w", encoding=ENCODING) as w_file:
try:
for item in music_file_list:
w_file.write(item)
w_file.write("\n")
except json.JSONDecodeError:
pass
with HIT_PLAYLIST_PATH.open("w", encoding=ENCODING) as w_file:
try:
for item in priority_list:
w_file.write(item)
w_file.write("\n")
except json.JSONDecodeError:
pass
def thread_rescan():
"""
@@ -94,6 +174,55 @@ def thread_rescan():
rescan()
def scan_tracks():
# Set the filename and open the file
logger = logging.getLogger("conjurer_musician")
with open(RADIOLOG_PATH, "r", encoding=ENCODING) as log_file:
log_file.seek(os.stat(RADIOLOG_PATH).st_size)
prev_size = os.stat(PERSISTENCE_PATH).st_size
while True:
current_size = os.stat(PERSISTENCE_PATH).st_size
if prev_size != current_size:
while prev_size != current_size:
prev_size = current_size
time.sleep(0.1)
current_size = os.stat(PERSISTENCE_PATH).st_size
with open(PERSISTENCE_PATH, "r", encoding=ENCODING) as persistence:
lines = persistence.readlines()
if len(lines) >= 3:
_post_to_bot(["next", lines[2]])
position = log_file.tell()
line = log_file.readline()
if not line:
time.sleep(1)
log_file.seek(position)
continue
if not re.match(r".*Prepared.*", line):
time.sleep(0.1)
continue
result = None
if re.match(r".*jingles.*", line):
result = ["jingles", line]
elif re.match(r".*priority.*", line):
result = ["priority", line]
elif re.match(r".*hit.*", line):
result = ["hit", line]
elif re.match(r".*all_playlist.*", line):
result = ["all", line]
elif re.match(r".*request.*", line):
result = ["requests", line]
if result:
logger.info("Forwarding radio log entry: %s", result[0])
_post_to_bot(result)
time.sleep(0.1)
app = Flask(__name__)
# AutoIndex(app, browse_root="/")
@@ -199,6 +328,9 @@ def wyszukaj(word_list, how_many, _logger=None, return_to_bot=True):
while not_found:
if search_weight[itr][0] == item_to_search:
return_list.append(search_weight[itr])
if not return_to_bot:
with PRIORITY_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
s_file.write(search_weight[itr][1] + "\n")
break
itr += 1
else:
@@ -269,6 +401,62 @@ def get_share_links():
@app.route("/stream", methods=["GET"])
def stream_music():
"""
The function `stream_music` is a route handler for the "/stream" endpoint.
It returns a JSON response containing a key "music_file_list" with the value of the variable `music_file_list`.
:return: A JSON response containing a key "music_file_list" with the value of the variable `music_file_list`.
"""
# return send_from_directory("/tmp/hls", "stream.m3u8")
return render_template(str(STREAM_TEMPLATE))
@app.route("/<string:file_name>")
def stream(file_name):
"""
Stream the specified file from the video directory.
Args:
file_name (str): The name of the file to be streamed.
Returns:
Response: The response containing the streamed file.
"""
# trunk-ignore(bandit/B108)
video_dir = "/tmp/hls"
return send_from_directory(video_dir, file_name)
@app.route("/stream_mp3", methods=["GET"])
def stream_music_mp3():
"""
The function `stream_music_mp3` is a route handler for the "/stream_mp3" endpoint.
It redirects the user to the URL "http://www.example.com" with a status code of 302.
:return: A redirect response to "http://www.example.com" with a status code of 302.
"""
return redirect("http://www.example.com", code=302)
@app.route("/clear_pr_pls", methods=["GET"])
def clear_pr_pls():
_authorize_request()
"""
The function `clear_pr_pls` clears the contents of the priority queue playlist file.
:return: A JSON response indicating the success of the operation.
"""
app.logger.info("CLEARING PLAYLIST")
with PRIORITY_PLAYLIST_PATH.open("w", encoding=ENCODING) as cleared_pl:
cleared_pl.write("")
return_data = jsonify(isError=False, message="Success", statusCode=200, data=[])
return return_data, 200
@app.route("/mp3", methods=["GET"])
def get_music_list():
"""
@@ -329,6 +517,96 @@ def look_for_playlist():
return return_data
@app.route("/request_radio_file", methods=["POST"])
def add_request():
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
app.logger.info(record["UUID"])
return_data = wyszukaj(record["lista_slow"], 0, app.logger, False)
with REQUEST_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return_data = (
jsonify(
isError=False, message="Success", statusCode=200, data={"status": "OK"}
),
200,
)
return return_data
@app.route("/create_priority_playlist", methods=["POST"])
def create_priority_playlist():
"""
The function `create_priority_playlist` receives a POST request with a JSON payload, logs the received
item, adds it to a music file list, and returns a success message along with the updated record.
:return: A tuple containing a JSON response and a status code.
The JSON response includes keys `isError`,
`message`, `statusCode`, and `data`, with values
indicating the success of the operation and the
data that was received and added to the `music_file_list`.
The status code returned is 200,indicating a successful response.
"""
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
app.logger.info(record["UUID"])
app.logger.info(record["dlugosc_plejlisty"])
return_data = wyszukaj(
record["lista_slow"], record["dlugosc_plejlisty"], app.logger, False
)
random.shuffle(return_data)
with REQUEST_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return_data = (
jsonify(
isError=False, message="Success", statusCode=200, data={"status": "OK"}
),
200,
)
return return_data
@app.route("/add_to_priority", methods=["POST"])
def add_to_priority():
"""
The function `add_to_priority` receives a POST request with a JSON payload, logs the received
item, adds it to a music file list, and returns a success message along with the updated record.
:return: A tuple containing a JSON response and a status code.
The JSON response includes keys `isError`,
`message`, `statusCode`, and `data`, with values
indicating the success of the operation and the
data that was received and added to the `music_file_list`.
The status code returned is 200,indicating a successful response.
"""
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
app.logger.info(record["UUID"])
app.logger.info(record["dlugosc_plejlisty"])
return_data = wyszukaj(
record["lista_slow"], record["dlugosc_plejlisty"], app.logger, False
)
with PRIORITY_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return_data = (
jsonify(
isError=False, message="Success", statusCode=200, data={"status": "OK"}
),
200,
)
return return_data
def flask_debug():
"""
The `flask_debug` function starts a Flask application in debug mode without using the reloader.
@@ -382,8 +660,13 @@ if __name__ == "__main__":
for worker in threads:
worker.start()
time.sleep(60)
track_thread = threading.Thread(target=scan_tracks, daemon=True)
track_thread.start()
try:
for worker in threads:
worker.join()
track_thread.join()
except KeyboardInterrupt:
logger.info("Shutdown requested - exiting musician service")
+14 -14
View File
@@ -1,16 +1,16 @@
# Radio Conjurer - Liquidsoap script (containerised paths: /srv/betoniarka/*)
# FILEPATH: /home/mtuszowski/conjurer/conjurer_musician/radio_conjurer.liq
# This script sets up a Liquidsoap radio stream with various features and configurations.
# Load icecast credentials from a JSON file
let json.parse credentials = file.contents("/srv/betoniarka/secrets/icecast_credentials.json")
let json.parse credentials = file.contents("/home/pi/Conjurer/icecast_credentials.json")
# Enable replaygain metadata processing
enable_replaygain_metadata()
# Set up a playlog for tracking played tracks
l = playlog(duration = 72000.0, persistency="/srv/betoniarka/data/persistence.log")
l = playlog(duration = 72000.0, persistency="/home/pi/Conjurer/persistence.log")
# Function to check if a track can be played based on its metadata
def check(r)
@@ -25,20 +25,20 @@ def check(r)
end
# Define playlists to be used in the stream
s1 = replaygain(playlist(reload_mode="watch", check_next=check, "/srv/betoniarka/data/all_playlist.playlist"))
s2 = replaygain(playlist(reload_mode="watch", check_next=check, "/srv/betoniarka/data/priority_queue.playlist"))
s3 = replaygain(playlist(reload_mode="watch", check_next=check, "/srv/betoniarka/data/hit.playlist"))
s1 = replaygain(playlist(reload_mode="watch", check_next=check, "/home/pi/Conjurer/all_playlist.playlist"))
s2 = replaygain(playlist(reload_mode="watch", check_next=check, "/home/pi/Conjurer/priority_queue.playlist"))
s3 = replaygain(playlist(reload_mode="watch", check_next=check, "/home/pi/Conjurer/hit.playlist"))
# Create a request queue for user-generated requests
requests_queue = request.queue()
# Function to process the request queue and add new requests
def queue_processing()
text=file.lines("/srv/betoniarka/data/request.playlist")
text=file.lines("/home/pi/Conjurer/request.playlist")
if text != [] then
list.iter(fun(item) -> requests_queue.push.uri(item), text)
file.remove("/srv/betoniarka/data/request.playlist")
f = file.open("/srv/betoniarka/data/request.playlist", create=true)
file.remove("/home/pi/Conjurer/request.playlist")
f = file.open("/home/pi/Conjurer/request.playlist", create=true)
f.close()
end
end
@@ -63,7 +63,7 @@ end
s4 = random(id="randomizer", weights=[2, 3, 5], [s1, s2, s3])
# Load jingles playlist
jingles = (playlist(reload_mode="watch", "/srv/betoniarka/data/jingles.playlist"))
jingles = (playlist(reload_mode="watch", "/home/pi/Conjurer/jingles.playlist"))
# Create the main stream with random playlist and jingles
s = rotate(id="randomizer", weights=[10, 1], [s4, jingles])
@@ -111,7 +111,7 @@ s=switch(track_sensitive=true,
# Configure logging settings
log_to_stdout = true
log_to_file = true
logpath = "/srv/betoniarka/data/radio_log.log"
logpath = "/home/pi/Conjurer/radio_log.log"
loglevel = 3
set("log.stdout", log_to_stdout)
set("log.level", loglevel)
@@ -120,7 +120,7 @@ set("log.level", loglevel)
set("log.file", log_to_file)
set("log.file.path", logpath)
# Set up emergency fallback track
emergency = single("/srv/betoniarka/music/Youtube/Dr. Peacock - Trip to Ireland [GvrvQTUbUcA].mp3")
emergency = single("/home/pi/MediaFolder/mp3/Youtube/Dr. Peacock - Trip to Ireland [GvrvQTUbUcA].mp3")
radio = fallback(id="switcher2", track_sensitive=false, [s, emergency])
# Set up an interactive control for skipping tracks
@@ -150,11 +150,11 @@ thread.run(every=15., check_skip)
# Enable persistent script parameters
interactive.persistent("/srv/betoniarka/data/script.params")
interactive.persistent("script.params")
# Configure output formats and destinations
output.icecast(%mp3, host="localhost", port=8000, password=credentials.password, icy_metadata="true", mount="mp3-stream", radio)
output.icecast(%mp3, host="radio", port=8000, password=credentials.password, icy_metadata="true", mount="mp3-stream", radio)
output.pulseaudio(radio)
#output.file.hls("/tmp/hls", [("mp3-low", %mp3(bitrate=96)), ("mp3-hi", %mp3(bitrate=160))], radio)
# Uncomment the following lines to enable additional output formats
-86
View File
@@ -38,11 +38,6 @@ try:
except ImportError: # pragma: no cover - optional at runtime
openai = None
try:
import anthropic
except ImportError: # pragma: no cover - optional at runtime
anthropic = None
try:
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
@@ -203,9 +198,6 @@ TRANSCRIPTS_PATH = os.getenv(
FILE_SERVICE_ADDRESS = os.getenv("CONJURER_FILE_SERVICE", "http://192.168.1.15:5000")
RADIO_HARBOR_ADDRESS = os.getenv("CONJURER_RADIO_HARBOR", "http://192.168.1.15:54321")
# Betoniarka (radio-operator service colocated with Liquidsoap). Falls back to
# the musician address so deployments that have not split yet keep working.
RADIO_SERVICE_ADDRESS = os.getenv("CONJURER_RADIO_SERVICE", FILE_SERVICE_ADDRESS)
SKIP_TRACK = os.getenv("CONJURER_SKIP_ENDPOINT", "/skip")
LIBRARIAN_SERVICE_ADDRESS = os.getenv(
"CONJURER_LIBRARIAN_SERVICE", "http://192.168.1.192:5001"
@@ -341,15 +333,6 @@ if openai and OPENAI_API_KEY:
else:
OPENAICLIENT = None
# Claude / Anthropic client, wired analogously to OpenAI above so the AI cog can
# be pointed at either backend with a single config switch (see AI_CONFIGS and
# ai_functions.set_active_ai_config). netrc machine name 'anthropic' works too.
ANTHROPIC_API_KEY = _resolve_token("anthropic", "ANTHROPIC_API_KEY")
if anthropic and ANTHROPIC_API_KEY:
CLAUDECLIENT = anthropic.AsyncAnthropic(api_key=ANTHROPIC_API_KEY)
else:
CLAUDECLIENT = None
TOKEN = _resolve_token("discord", "DISCORD_TOKEN")
# Voice recognition (AssemblyAI). None = the voice cog reports and disables.
@@ -402,75 +385,6 @@ GUILD_ID = 664789470779932693
LATEST_MODEL = "gpt-4o" # najnowszy/do rozmów (możesz zmienić w jednym miejscu)
CHEAP_MODEL = "gpt-4o-mini" # najtańszy (fallback do 3.5 niżej)
# Claude counterparts of LATEST_MODEL / CHEAP_MODEL. Opus 4.8 is the strongest
# widely available model; Haiku 4.5 is the fast/cheap tier used for MUSIC.
CLAUDE_LATEST_MODEL = "claude-opus-4-8"
CLAUDE_CHEAP_MODEL = "claude-haiku-4-5"
# *=========================================== AI provider configs
# The bot's AI functionality (ai_functions.handle_response) can be pointed at a
# different backend by flipping a single "active" switch. Each named config
# collects the *differences* between providers (which backend, which models,
# generation params). These live in system_gpt_settings.json under an optional
# third list element (index 2) so future configs are simply added there:
#
# [ <system message>, <personal assistants>, { "active": "gpt",
# "configs": { ... } } ]
#
# The block is optional and backward compatible: a settings file with only the
# original two elements falls back to the built-in defaults below (active
# "gpt"), so existing deployments behave exactly as before. The active config
# can be overridden at import time with CONJURER_AI_CONFIG and at runtime with
# the $gadaj_teraz command (which persists the choice back into index 2).
def _default_ai_configs():
return {
"gpt": {
"provider": "openai",
"latest_model": LATEST_MODEL,
"cheap_model": CHEAP_MODEL,
"temperature": 0.2,
},
"claude": {
"provider": "anthropic",
"latest_model": CLAUDE_LATEST_MODEL,
"cheap_model": CLAUDE_CHEAP_MODEL,
# Anthropic requires max_tokens; temperature is intentionally not
# sent for Claude (Opus 4.8 / Sonnet 5 reject sampling params).
"max_tokens": 2048,
},
# Template for wiring further providers. Copy it, rename the key, point
# "provider" at a backend ai_functions.provider_generate implements, and
# fill in the model ids. Keys starting with "_" are treated as inert
# templates and are hidden from the $gadaj_teraz picker.
"_template": {
"provider": "openai",
"latest_model": "model-id-here",
"cheap_model": "cheaper-model-id-here",
"temperature": 0.2,
"max_tokens": 2048,
},
}
_ai_block = (
GPT_SETTINGS[2]
if isinstance(GPT_SETTINGS, list) and len(GPT_SETTINGS) > 2 and isinstance(GPT_SETTINGS[2], dict)
else {}
)
AI_CONFIGS = _ai_block.get("configs") or _default_ai_configs()
# Single switch: env var wins, then the settings-file "active" key, then "gpt".
DEFAULT_AI_CONFIG = (
os.getenv("CONJURER_AI_CONFIG")
or _ai_block.get("active")
or "gpt"
)
if DEFAULT_AI_CONFIG not in AI_CONFIGS:
logger.warning(
"AI config '%s' not found - falling back to 'gpt'", DEFAULT_AI_CONFIG
)
DEFAULT_AI_CONFIG = "gpt" if "gpt" in AI_CONFIGS else next(iter(AI_CONFIGS))
# *=========================================== Conan Exiles bridge
# Every value is optional; the conanjurer cog stays dormant unless configured.
+3 -6
View File
@@ -18,13 +18,10 @@ COPY conjurer_librarian/ ./
ENV PYTHONUNBUFFERED=1 \
CONJURER_LIBRARIAN_HOST=0.0.0.0 \
CONJURER_LIBRARIAN_PORT=5001 \
CONJURER_LIBRARIAN_DB_PATH=/doi/ \
CONJURER_LIBRARIAN_STATE_DIR=/lib_temp_files
CONJURER_LIBRARIAN_DB_PATH=/doi/
# /doi = the local DOI chunk database (large, read-only).
# /lib_temp_files = runtime JSON state (cr_results/rr_results/not_in_db/
# s_results) - seeded on first run, persisted across restarts.
VOLUME ["/doi", "/lib_temp_files"]
# The local DOI chunk database (large) is mounted here.
VOLUME ["/doi"]
EXPOSE 5001
CMD ["python", "conjurer_librarian.py"]
-110
View File
@@ -1,110 +0,0 @@
# Radio Conjurer - Liquidsoap runtime built through opam, so the OCaml,
# opam and Liquidsoap versions are all selectable at build time.
#
# Build from the repository root, e.g.:
# docker build -f docker/Dockerfile.radio -t conjurer-radio .
# docker build -f docker/Dockerfile.radio -t conjurer-radio \
# --build-arg LIQUIDSOAP_VERSION=2.1.4 --build-arg OCAML_VERSION=4.14.2 .
#
# Version notes (matched to radio_conjurer.liq, which targets 2.1.4):
# * Liquidsoap 2.1.x requires OCaml 4.x - it does NOT build on OCaml 5.
# Production ran the 4.13.0 switch; 4.14.x is the maintained 4.x line and
# builds 2.1.4 fine. Pass OCAML_VERSION=4.13.0 for bit-for-bit parity.
# * The ocaml-ffmpeg bindings compatible with 2.1.4 target the FFmpeg 5.x
# library family (libavcodec59/libavformat59/libavutil57...). Debian
# bookworm ships exactly that, which is why this image needs NO ffmpeg
# pinning: the old ffmpeg.pref (Pin-Priority 1001 on deb.debian.org) only
# existed to force those Debian builds over the Raspberry Pi OS repo's
# conflicting ones. Single-repo container = the pin is redundant.
FROM debian:bookworm-slim
ARG OPAM_VERSION=2.1.5
ARG OCAML_VERSION=4.14.2
ARG LIQUIDSOAP_VERSION=2.1.4
# Optional liquidsoap features, resolved together with liquidsoap by opam.
# These cover everything radio_conjurer.liq uses:
# mad+lame - mp3 decode/encode (%mp3 icecast stream)
# cry - output.icecast
# taglib - tag/replaygain metadata
# pulseaudio- input.pulseaudio (mic) + output.pulseaudio
# samplerate- resampling
# inotify - playlist(reload_mode="watch")
# ffmpeg - decode fallback + replaygain computation
ARG LIQ_OPAM_PACKAGES="mad lame cry taglib pulseaudio samplerate inotify ffmpeg"
# System libraries: build deps for the opam packages above plus the matching
# runtime libs. The libav*-dev list mirrors the old ffmpeg.pref family.
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential m4 pkg-config git curl ca-certificates unzip rsync \
libpcre3-dev libgmp-dev zlib1g-dev \
libmad0-dev libmp3lame-dev libtag1-dev \
libpulse-dev libsamplerate0-dev \
libavcodec-dev libavformat-dev libavutil-dev libavfilter-dev \
libavdevice-dev libswresample-dev libswscale-dev libpostproc-dev \
libcurl4-gnutls-dev \
ffmpeg \
pulseaudio pulseaudio-utils \
icecast2 jq \
python3 python3-flask python3-waitress python3-requests \
&& rm -rf /var/lib/apt/lists/*
# Liquidsoap refuses to run as root (security exit), so it gets a dedicated
# user. audio/pulse-access mirror what was done by hand on the Pi for the
# system-wide pulse socket ('pulse-access' comes with the pulseaudio pkg).
RUN useradd --system --create-home --home-dir /var/lib/radio \
--shell /usr/sbin/nologin radio \
&& usermod -aG audio,pulse-access radio
# opam as a static binary so OPAM_VERSION is a real choice (apt would pin us
# to whatever bookworm ships).
RUN ARCH=$(uname -m) \
&& curl -fsSL -o /usr/local/bin/opam \
"https://github.com/ocaml/opam/releases/download/${OPAM_VERSION}/opam-${OPAM_VERSION}-${ARCH}-linux" \
&& chmod +x /usr/local/bin/opam
# OCaml switch + liquidsoap and its optional feature libraries in one solve,
# so liquidsoap is compiled WITH those features enabled. OPAMROOT lives in
# /opt/opam (not /root/.opam) so the unprivileged 'radio' user can read the
# liquidsoap binary AND its stdlib .liq files at runtime.
ENV OPAMROOT=/opt/opam
RUN opam init -y --bare --disable-sandboxing \
&& opam switch create default "${OCAML_VERSION}" \
&& opam install -y "liquidsoap.${LIQUIDSOAP_VERSION}" ${LIQ_OPAM_PACKAGES} \
&& opam clean -a -c -s --logs
ENV PATH="/opt/opam/default/bin:${PATH}"
# Build-time smoke test: the binary runs and reports the requested version.
RUN liquidsoap --version
# Minimal system-wide pulse config for headless VMs (PULSE_MODE=internal):
# a null sink so output.pulseaudio()/input.pulseaudio() work with no sound
# hardware (mic becomes silence, which blank.strip already gates out).
COPY docker/pulse-system.pa /etc/pulse/system.pa
# The script and its persistent params are seeded into the data volume on
# first run (never overwritten), so live edits survive image rebuilds. The
# icecast config is rendered from the template at startup with passwords
# taken from the secrets volume (never baked into the image).
WORKDIR /app
COPY conjurer_musician/radio_conjurer.liq ./radio_conjurer.liq
COPY conjurer_musician/script.params ./script.params
COPY conjurer_musician/stream.html ./stream.html
COPY conjurer_betoniarka/betoniarka.py ./betoniarka.py
COPY docker/icecast.xml.tpl ./icecast.xml.tpl
COPY docker/entrypoint.radio.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
# Volume layout matches the paths inside radio_conjurer.liq:
# /srv/betoniarka/data - playlists, script.params, persistence/radio logs
# /srv/betoniarka/music - the mp3 library
# /srv/betoniarka/secrets - icecast_credentials.json (provisioned at install)
VOLUME ["/srv/betoniarka/data", "/srv/betoniarka/music", "/srv/betoniarka/secrets"]
# 8000 = icecast (listeners), 5005 = betoniarka HTTP API (the bot's
# CONJURER_RADIO_SERVICE), 54321 = harbor /skip (the bot's RADIO_HARBOR),
# 9999 = interactive harbor.
EXPOSE 8000 5005 54321 9999
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
CMD ["liquidsoap", "/srv/betoniarka/data/radio_conjurer.liq"]
+1 -5
View File
@@ -14,11 +14,7 @@ services:
ports:
- "5001:5001"
volumes:
# Local DOI chunk database (0_chunk.txt ... N_chunk.txt). Read-only:
# the search workers only read it (open mode "r").
# Local DOI chunk database (0_chunk.txt ... N_chunk.txt).
- /srv/librarian/doi:/doi:ro
# Runtime JSON state (cr_results/rr_results/not_in_db/s_results) -
# seeded on first run, persisted here across restarts.
- /srv/librarian/state:/lib_temp_files
# Optional: netrc holding Crossref credentials (or use CONJURER_CROSSREF_MAILTO).
- /srv/librarian/secrets/.netrc:/secrets/.netrc:ro
-45
View File
@@ -1,45 +0,0 @@
# Radio (Liquidsoap + Icecast) VM. Run from the repository root:
# docker compose -f docker/compose.radio.yaml up -d --build
#
# Version pins are build args - override via environment, e.g.:
# LIQUIDSOAP_VERSION=2.1.4 OCAML_VERSION=4.13.0 \
# docker compose -f docker/compose.radio.yaml up -d --build
services:
conjurer-radio:
build:
context: ..
dockerfile: docker/Dockerfile.radio
args:
OPAM_VERSION: ${OPAM_VERSION:-2.1.5}
OCAML_VERSION: ${OCAML_VERSION:-4.14.2}
LIQUIDSOAP_VERSION: ${LIQUIDSOAP_VERSION:-2.1.4}
image: conjurer-radio:latest
container_name: conjurer-radio
restart: unless-stopped
environment:
# internal = null-sink pulse inside the container (headless VM default)
# host = mount the host pulse socket below and set PULSE_SERVER
PULSE_MODE: ${PULSE_MODE:-internal}
# PULSE_SERVER: unix:/tmp/pulseaudio.socket
# Hostname icecast reports in its status/YP pages:
ICECAST_HOSTNAME: ${ICECAST_HOSTNAME:-localhost}
# Betoniarka (radio-operator API + log forwarder):
CONJURER_MAIN_BOT: ${CONJURER_MAIN_BOT:-http://127.0.0.1:5000}
CONJURER_API_KEY: ${CONJURER_API_KEY:-}
ports:
- "8000:8000" # icecast - listeners tune in here
- "5005:5005" # betoniarka API - the bot's CONJURER_RADIO_SERVICE
- "54321:54321" # harbor /skip - the bot's CONJURER_RADIO_HARBOR
- "9999:9999" # interactive harbor (keep LAN-only!)
volumes:
# Playlists, logs, script.params and the script itself (seeded on first
# run) - same paths inside and outside the container.
- /srv/betoniarka/data:/srv/betoniarka/data
# The mp3 library. Writable because the entrypoint seeds a silent
# emergency-fallback mp3 when the hardcoded single() file is missing.
- /srv/betoniarka/music:/srv/betoniarka/music
# icecast_credentials.json - provision at install like the other
# services' secrets (a CHANGE_ME placeholder is seeded if absent).
- /srv/betoniarka/secrets:/srv/betoniarka/secrets
# PULSE_MODE=host: uncomment and adjust
# - /tmp/pulseaudio.socket:/tmp/pulseaudio.socket
-113
View File
@@ -1,113 +0,0 @@
#!/bin/sh
# Prepare the radio volumes, start the in-container Icecast server, wire up
# PulseAudio and exec liquidsoap. Existing files are never overwritten -
# live-edited script/params/playlists always win.
set -e
DATA="${RADIO_DATA_DIR:-/srv/betoniarka/data}"
MUSIC="${RADIO_MUSIC_DIR:-/srv/betoniarka/music}"
SECRETS="${RADIO_SECRETS_DIR:-/srv/betoniarka/secrets}"
CREDS="$SECRETS/icecast_credentials.json"
mkdir -p "$DATA" "$MUSIC" "$SECRETS"
# Seed the script + persistent interactive params from the image on first run.
[ -e "$DATA/radio_conjurer.liq" ] || cp /app/radio_conjurer.liq "$DATA/"
if [ ! -e "$DATA/script.params" ]; then
if [ -e /app/script.params ]; then cp /app/script.params "$DATA/"; else : > "$DATA/script.params"; fi
fi
# Playlists + logs the script watches/writes; empty files keep it happy until
# the musician/betoniarka populate them.
for f in all_playlist.playlist priority_queue.playlist hit.playlist \
request.playlist jingles.playlist persistence.log; do
[ -e "$DATA/$f" ] || : > "$DATA/$f"
done
# The icecast secret is provisioned at install time, like the other services'
# secrets (bot: /srv/conjurer/secrets/.netrc). A placeholder keeps the stack
# bootable, but both icecast and the stream stay locked until you fix it.
if [ ! -e "$CREDS" ]; then
printf '{\n"password" : "CHANGE_ME"\n}\n' > "$CREDS"
echo "WARNING: $CREDS was missing - seeded a CHANGE_ME placeholder." >&2
echo " Put the real password there (see docs) and restart." >&2
fi
# Render /etc/icecast2/icecast.xml from the template with passwords from the
# secret. Optional fields admin_password/relay_password default to password.
SOURCE_PW=$(jq -r '.password' "$CREDS")
ADMIN_PW=$(jq -r '.admin_password // .password' "$CREDS")
RELAY_PW=$(jq -r '.relay_password // .password' "$CREDS")
ICECAST_HOSTNAME="${ICECAST_HOSTNAME:-localhost}"
sed -e "s|__SOURCE_PASSWORD__|$SOURCE_PW|" \
-e "s|__ADMIN_PASSWORD__|$ADMIN_PW|" \
-e "s|__RELAY_PASSWORD__|$RELAY_PW|" \
-e "s|__HOSTNAME__|$ICECAST_HOSTNAME|" \
/app/icecast.xml.tpl > /etc/icecast2/icecast.xml
chown icecast2:icecast /etc/icecast2/icecast.xml 2>/dev/null || true
chmod 640 /etc/icecast2/icecast.xml
# Start Icecast in the background as its unprivileged user.
mkdir -p /var/log/icecast2 && chown -R icecast2:icecast /var/log/icecast2
su -s /bin/sh icecast2 -c "icecast2 -b -c /etc/icecast2/icecast.xml" \
|| echo "WARNING: icecast2 failed to start - the stream output will retry" >&2
# single() aborts the whole script when its file is missing; guarantee the
# emergency fallback exists (5s of silence beats a dead radio).
EMERGENCY="$MUSIC/Youtube/Dr. Peacock - Trip to Ireland [GvrvQTUbUcA].mp3"
if [ ! -e "$EMERGENCY" ]; then
mkdir -p "$MUSIC/Youtube"
if ffmpeg -loglevel error -f lavfi -i anullsrc=r=44100:cl=stereo -t 5 \
-codec:a libmp3lame -q:a 9 "$EMERGENCY"; then
echo "WARNING: emergency track was missing - generated silent placeholder" >&2
else
echo "WARNING: could not create emergency track; single() may abort" >&2
fi
fi
# PulseAudio wiring:
# internal (default) - system-wide pulse inside the container with a null
# sink (see /etc/pulse/system.pa); no sound hardware
# needed, mic path reads silence.
# host - use a socket mounted from the host; set PULSE_SERVER
# (e.g. unix:/tmp/pulseaudio.socket) in the env file.
# none - you edited the script to drop pulse in/out.
case "${PULSE_MODE:-internal}" in
internal)
# --disallow-module-loading: modules from system.pa still load at
# startup; this only blocks later client-requested loads (and
# silences the system-mode warning). The "forcibly disabling SHM"
# notice is inherent to system mode and harmless.
pulseaudio --system --daemonize=yes --disallow-exit \
--disallow-module-loading --exit-idle-time=-1 \
|| echo "WARNING: internal pulseaudio failed to start" >&2
export PULSE_SERVER="${PULSE_SERVER:-unix:/var/run/pulse/native}"
;;
host)
[ -n "$PULSE_SERVER" ] || echo "WARNING: PULSE_MODE=host but PULSE_SERVER is unset" >&2
;;
none)
;;
esac
# Liquidsoap refuses to run as root (init: security exit), so hand the data
# volume to the dedicated 'radio' user and drop privileges for the main
# process. chown is best-effort: on local volumes it always works (the
# supported layout); network filesystems with root-squash reject it, hence
# the warning instead of a fatal abort.
chown -R radio:radio "$DATA" 2>/dev/null \
|| echo "WARNING: chown of $DATA failed (network FS?) - keep this volume LOCAL to the radio VM" >&2
chgrp radio "$CREDS" 2>/dev/null && chmod 640 "$CREDS" || true
if ! setpriv --reuid radio --regid radio --init-groups -- test -r "$MUSIC"; then
echo "WARNING: music dir $MUSIC is not readable by the 'radio' user" >&2
fi
# Betoniarka: the radio-operator API + radio-log forwarder, running as the
# SAME user as liquidsoap on the SAME volume - this is what makes the old
# musician-writes-as-root-over-network-share permission mess go away.
BETONIARKA_DATA="$DATA" BETONIARKA_MUSIC="$MUSIC" \
setpriv --reuid radio --regid radio --init-groups -- \
python3 /app/betoniarka.py &
echo "betoniarka started (pid $!)"
cd "$DATA"
exec setpriv --reuid radio --regid radio --init-groups -- "$@"
+2 -12
View File
@@ -1,22 +1,15 @@
# Copy to docker/env/bot.env and fill in. Do NOT commit the real file.
# --- Secrets ------------------------------------------------------------
# Option A: mount a netrc (recommended — covers discord/openai/anthropic/spotipy/youtube).
# Option A: mount a netrc (recommended — covers discord/openai/spotipy/youtube).
CONJURER_NETRC_FILE=/secrets/.netrc
# Option B: pass tokens directly (these take precedence over netrc).
# DISCORD_TOKEN=
# OPENAI_API_KEY=
# ANTHROPIC_API_KEY= # Claude backend; netrc machine 'anthropic' works too
# ASSEMBLYAI_API_KEY= # voice recognition; netrc machine 'assemblyai' works too
# YOUTUBE_USERNAME=
# YOUTUBE_PASSWORD=
# --- AI backend switch --------------------------------------------------
# Which AI config from system_gpt_settings.json is active at startup
# (e.g. "gpt" or "claude"). Runtime switch: $gadaj_teraz <config>. Unset =
# whatever the settings file's "active" key says, falling back to "gpt".
# CONJURER_AI_CONFIG=gpt
# --- Data ---------------------------------------------------------------
# Single mounted volume; all writable state is rooted here.
CONJURER_DATA_DIR=/data
@@ -31,10 +24,7 @@ CONJURER_API_KEY=
# --- Where the bot reaches the other services (other Proxmox VMs) --------
CONJURER_FILE_SERVICE=http://MUSICIAN_VM_IP:5000
# Betoniarka (radio-operator API, runs in the radio container):
CONJURER_RADIO_SERVICE=http://RADIO_VM_IP:5005
# Liquidsoap harbor /skip (same radio container):
CONJURER_RADIO_HARBOR=http://RADIO_VM_IP:54321
CONJURER_RADIO_HARBOR=http://MUSICIAN_VM_IP:54321
CONJURER_LIBRARIAN_SERVICE=http://LIBRARIAN_VM_IP:5001
# --- Conan Exiles bridge (optional; empty/0 = disabled) -----------------
-4
View File
@@ -17,9 +17,5 @@ CONJURER_LIBRARIAN_DB_PATH=/doi/
CONJURER_LIBRARIAN_MAXTHREADS=41
CONJURER_LIBRARIAN_CHUNK=_chunk.txt
# Runtime JSON state dir (mounted, persistent): cr_results/rr_results/
# not_in_db/s_results are seeded here on first run.
CONJURER_LIBRARIAN_STATE_DIR=/lib_temp_files
# Optional netrc (for Crossref credentials)
CONJURER_NETRC_FILE=/secrets/.netrc
-55
View File
@@ -1,55 +0,0 @@
<!-- Icecast2 config template for the radio container.
The entrypoint substitutes the __*_PASSWORD__ placeholders from
/srv/betoniarka/secrets/icecast_credentials.json and writes the result
to /etc/icecast2/icecast.xml. Never commit real passwords here. -->
<icecast>
<location>Wolne Ksiestwo Baluty</location>
<admin>admin@localhost</admin>
<limits>
<clients>64</clients>
<sources>4</sources>
<queue-size>524288</queue-size>
<client-timeout>30</client-timeout>
<header-timeout>15</header-timeout>
<source-timeout>10</source-timeout>
<burst-on-connect>1</burst-on-connect>
<burst-size>65535</burst-size>
</limits>
<authentication>
<source-password>__SOURCE_PASSWORD__</source-password>
<relay-password>__RELAY_PASSWORD__</relay-password>
<admin-user>admin</admin-user>
<admin-password>__ADMIN_PASSWORD__</admin-password>
</authentication>
<hostname>__HOSTNAME__</hostname>
<listen-socket>
<port>8000</port>
<bind-address>0.0.0.0</bind-address>
</listen-socket>
<http-headers>
<header name="Access-Control-Allow-Origin" value="*" />
</http-headers>
<fileserve>1</fileserve>
<paths>
<basedir>/usr/share/icecast2</basedir>
<logdir>/var/log/icecast2</logdir>
<webroot>/usr/share/icecast2/web</webroot>
<adminroot>/usr/share/icecast2/admin</adminroot>
<alias source="/" destination="/status.xsl"/>
</paths>
<logging>
<accesslog>access.log</accesslog>
<errorlog>error.log</errorlog>
<loglevel>3</loglevel>
<logsize>10000</logsize>
<logarchive>0</logarchive>
</logging>
</icecast>
-10
View File
@@ -1,10 +0,0 @@
#!/usr/bin/pulseaudio -nF
# Minimal system-wide PulseAudio config for the radio container running on a
# headless VM (PULSE_MODE=internal). Mirrors the Pi's setup (unix socket,
# anonymous auth) but replaces the Lexicon Lambda USB device with a null
# sink: output.pulseaudio() plays into the void and input.pulseaudio() (the
# mic path) reads silence from the sink monitor.
load-module module-null-sink sink_name=radio_null sink_properties=device.description=RadioNullOutput
load-module module-native-protocol-unix auth-anonymous=1
set-default-sink radio_null
set-default-source radio_null.monitor
+1 -18
View File
@@ -43,28 +43,11 @@ You can rebalance as follows:
2. Edit each `docker/env/*.env` to replace `HACKME!` with real values:
- `DISCORD_TOKEN`, `OPENAI_API_KEY`, `CONJURER_API_KEY` (use the same value for
all services).
- `ANTHROPIC_API_KEY` — only if you want the Claude backend (see “AI backend
switch” below). Safe to leave unset while running on GPT.
- For musician/librarian, adjust mounts to the SMB paths you will mount on the
Pis, e.g. `/mnt/conjurer/music`.
- Set `CONJURER_CROSSREF_MAILTO` to a real email as required by Crossref.
3. If you rely on `.netrc`, copy it to `C:\Conjurer\secrets\.netrc` and set
`CONJURER_NETRC_FILE` accordingly. The bot reads netrc machines `discord`,
`openai`, `anthropic`, `assemblyai`, `spotipy`, `youtube`.
### AI backend switch (GPT ↔ Claude)
The AI chat features run on one backend at a time, selected by a single switch:
- **Startup:** `CONJURER_AI_CONFIG=gpt` (default) or `=claude` in `bot.env`. Unset
falls back to the `"active"` key in `system_gpt_settings.json`, then `"gpt"`.
- **Runtime:** the `$gadaj_teraz <config>` Discord command (Vykidailo only) flips
the backend live and persists the choice.
- The named configs (which provider, which models) live in
`system_gpt_settings.json` — add more there. Claude needs `ANTHROPIC_API_KEY`;
GPT needs `OPENAI_API_KEY`. Image generation (`imaginuje sobie:`) and personal
assistants always use OpenAI regardless of the switch (Anthropic has no
equivalent), and degrade quietly if `OPENAI_API_KEY` is absent.
`CONJURER_NETRC_FILE` accordingly.
## 4. Install Docker on Raspberry Pis and Windows
-152
View File
@@ -85,33 +85,6 @@ sudo chmod 600 /srv/conjurer/secrets/.netrc
(Alternatively skip netrc and set `DISCORD_TOKEN` / `OPENAI_API_KEY` in the env
file — those take precedence.)
For the **Claude backend** add an `anthropic` machine to the same netrc (or set
`ANTHROPIC_API_KEY` in `bot.env`):
```
machine anthropic
password sk-ant-...
```
You only need this if you actually switch the bot to Claude — see 1c-bis.
### 1c-bis. AI backend switch (GPT ↔ Claude)
The bot's AI chat runs on one backend at a time, chosen by a single switch:
- **At startup**, set `CONJURER_AI_CONFIG` in `bot.env``gpt` (default) or
`claude`. Leave it unset to use the `"active"` key in
`system_gpt_settings.json` (falls back to `gpt`).
- **At runtime**, `$gadaj_teraz <config>` (Vykidailo only) flips the backend live
and writes the choice back into `system_gpt_settings.json`.
The provider configs (which backend, which models) are collected in
`system_gpt_settings.json` under the third list element — add further AIs there.
`claude` needs `ANTHROPIC_API_KEY`; `gpt` needs `OPENAI_API_KEY`. Image
generation (`imaginuje sobie:`) and personal assistants stay on OpenAI whatever
the switch says (Anthropic has no equivalent) and degrade quietly if OpenAI is
not configured, so a Claude-only box still boots.
### 1d. Configure and launch
```bash
@@ -250,131 +223,6 @@ those two files into the same `/srv/musician/data` directory (or set
`/stream` page template is served from the baked-in `/app/stream.html`
(override with `CONJURER_STREAM_TEMPLATE` if you customise it).
### 3e. Radio (Liquidsoap) container
`docker/Dockerfile.radio` builds the full Liquidsoap environment through
**opam**, with the OCaml/opam/Liquidsoap versions selectable at build time:
| Build arg | Default | Notes |
|-----------|---------|-------|
| `LIQUIDSOAP_VERSION` | `2.1.4` | what `radio_conjurer.liq` targets |
| `OCAML_VERSION` | `4.14.2` | 2.1.x needs OCaml **4.x** (never 5); prod ran 4.13.0 — pass it for exact parity |
| `OPAM_VERSION` | `2.1.5` | static binary from GitHub releases |
| `LIQ_OPAM_PACKAGES` | `mad lame cry taglib pulseaudio samplerate inotify ffmpeg` | exactly the features the script uses (mp3 in/out, icecast, tags/replaygain, pulse mic/out, watch-reload) |
The container runs **both Liquidsoap and Icecast** — the stream is served
from this one container (`output.icecast(host="localhost", …)` in the
script). Volume layout (same paths inside and outside):
| Host & container path | Holds |
|---|---|
| `/srv/betoniarka/data` | playlists, `script.params`, `persistence.log`, `radio_log.log`, the seeded `radio_conjurer.liq` |
| `/srv/betoniarka/music` | the mp3 library |
| `/srv/betoniarka/secrets` | `icecast_credentials.json`**provision at install**, like the other services' secrets |
```bash
sudo mkdir -p /srv/betoniarka/data /srv/betoniarka/music /srv/betoniarka/secrets
# provision the icecast secret (same pattern as the bot's netrc):
echo '{ "password" : "SOURCE_PW", "admin_password" : "ADMIN_PW" }' \
| sudo tee /srv/betoniarka/secrets/icecast_credentials.json
sudo chmod 600 /srv/betoniarka/secrets/icecast_credentials.json
docker compose -f docker/compose.radio.yaml up -d --build
docker logs -f conjurer-radio
```
At startup the entrypoint renders `/etc/icecast2/icecast.xml` from
`docker/icecast.xml.tpl`, filling the source/admin/relay passwords from the
secret (`admin_password`/`relay_password` are optional and default to
`password`), and starts icecast as its unprivileged user. If the secret is
missing, a `CHANGE_ME` placeholder is seeded with a loud warning — the stack
boots but the stream stays locked until you fix it.
The script + `script.params` are seeded into the data volume on first run
and never overwritten (live edits survive rebuilds). Missing playlists are
created empty and a silent emergency-fallback mp3 is generated if the
`single()` file is absent, so the script always boots.
Wire-up: listeners tune to `http://RADIO_VM_IP:8000/mp3-stream`; the bot
points at `CONJURER_RADIO_HARBOR=http://RADIO_VM_IP:54321` (harbor `/skip`
lives in the script itself) and `CONJURER_RADIO_SERVICE=http://RADIO_VM_IP:5005`
(the betoniarka API below).
### 3f. Betoniarka - the radio operator (and why the split)
**Permissions post-mortem.** The pre-split layout had three actors fighting
over the same files: the musician wrote radio playlists **as root**, the
radio expected them **as user `radio`**, and both met on a **root-owned
network share** where `chown` fails by design (root squash / uid mapping).
Every component worked; the combination could not.
**The fix is structural**: the process that *writes* the radio playlists now
lives in the same container as the process that *watches* them, running as
the same `radio` user on a **local** volume. No network share, no chown, no
uid mapping - the class of problem is gone, not patched.
`conjurer_betoniarka/betoniarka.py` runs inside the radio container
(started by the entrypoint as user `radio`, port **5005**) and owns:
- the library scan → `all_playlist.playlist` / `hit.playlist` (local paths,
the same ones Liquidsoap resolves), on start + every 24h + `GET /rescan`
- the bot-facing radio API: `/add_to_priority`, `/create_priority_playlist`,
`/request_radio_file`, `/clear_pr_pls` (+ `GET /ping` for health gating,
`/stream` for the web page)
- tailing `radio_log.log`/`persistence.log` and forwarding play events to
the bot's `/prepped_tracks` (with the shared API key)
The **musician** is now a pure Discord music player: `/mp3`, `/update_mp3`,
`/get_music` and the file-share endpoints. It no longer writes any radio
files and needs no shared partition with the radio VM. The music library
can still be replicated/mounted on both VMs (read-only on the radio side is
fine) - playlists reference the *radio VM's local* paths, generated locally.
Bot wiring after the split (env on the bot):
```
CONJURER_FILE_SERVICE=http://MUSICIAN_VM_IP:5000 # Discord music
CONJURER_RADIO_SERVICE=http://RADIO_VM_IP:5005 # betoniarka (radio cmds)
CONJURER_RADIO_HARBOR=http://RADIO_VM_IP:54321 # liquidsoap /skip
```
`CONJURER_RADIO_SERVICE` defaults to `CONJURER_FILE_SERVICE`, so an
un-split deployment keeps working unchanged. The bot health-gates
`radio_commands` on betoniarka's `/ping` (separate from the musician group,
which now covers only `music_commands` + `file_search_commands`).
**Privileges:** Liquidsoap refuses to run as root (`init: security exit`),
so the main process runs as the dedicated **`radio`** user (member of
`audio`/`pulse-access`) — no `settings.init.allow_root` override. The
entrypoint (root) seeds volumes, renders the icecast config, starts
icecast/pulse, chowns `/srv/betoniarka/data` to `radio` and drops
privileges via `setpriv`. This is also why the opam switch lives in
`/opt/opam` instead of `/root/.opam` (the binary and the liquidsoap stdlib
must be readable by `radio`).
**PulseAudio** (`PULSE_MODE` env):
- `internal` (default) — a system-wide pulse daemon runs inside the container
with a **null sink** (`docker/pulse-system.pa`): no sound hardware needed,
`output.pulseaudio()` plays into the void and the `input.pulseaudio()` mic
path reads silence (which `blank.strip` already gates out). Right choice
for a headless Proxmox VM. Started with `--disallow-module-loading`
(startup modules from `system.pa` still load; only later client-requested
loads are blocked). The `forcibly disabling SHM mode` notice is inherent
to system mode and harmless.
- `host` — mount the host's pulse socket and set `PULSE_SERVER`, for a VM
with real audio hardware (the Pi's Lexicon Lambda setup).
- `none` — you removed the pulse in/out from the script.
**Verdicts on the old Pi setup quirks** (asked during containerisation):
- `ffmpeg.pref` (Pin-Priority 1001 on the `libavcodec59/libavformat59/…`
family): it **did have a purpose** — the opam-built ocaml-ffmpeg bindings
for 2.1.x are compiled against Debian bookworm's FFmpeg 5.x sonames, and
the pin forced those over conflicting Raspberry Pi OS repo builds (even as
a downgrade). In this single-repo container the same versions come
naturally, so the pin is redundant — **the file has been removed from the
repo** (this note preserves the knowledge).
- adding root to `pulse-access`/`audio` groups: needed on the Pi for the
system-wide pulse socket and ALSA device access. The image bakes the same
memberships in (`usermod -aG audio,pulse-access root`) — harmless with the
internal null sink, required for `PULSE_MODE=host`.
---
## 4. Networking & auth
+3
View File
@@ -0,0 +1,3 @@
Package: ffmpeg libavcodec-dev libavcodec59 libavdevice59 libavfilter8 libavformat-dev libavformat59 libavutil-dev libavutil57 libpostproc56 libswresample-dev libswresample4 libswscale-dev libswscale6 libavdevice-dev libavfilter-dev libpostproc-dev
Pin: origin deb.debian.org
Pin-Priority: 1001
+1 -1
View File
@@ -27,7 +27,7 @@ sudo usermod -aG audio root
echo "Add exception suppresion to signal handler in spotify __ini__.py"
echo "Add password to youtube opts in spotify_dl youtube.py"
echo "NOTE: the old ffmpeg.pref pin is obsolete (containerised radio uses Debian bookworm ffmpeg 5.x natively; file removed from the repo)"
echo "Downgrade ffmpeg by copying ffmpeg.pref from repository to /etc/apt/preferences.d"
echo "Install opam from installation link"
echo "Initialize opam"
echo "Install liquidsoap and its dependencies"
+5 -5
View File
@@ -8,7 +8,7 @@ import uuid
import asyncio
from datetime import datetime
from constants import RADIO_HARBOR_ADDRESS, SKIP_TRACK, RADIO_SERVICE_ADDRESS, ADD_TO_PRIO_PLAYLIST, REQUEST_MUSIC, CREATE_PRIO_PLAYLIST, CLEAR_PRIO, service_headers
from constants import RADIO_HARBOR_ADDRESS, SKIP_TRACK, FILE_SERVICE_ADDRESS, ADD_TO_PRIO_PLAYLIST, REQUEST_MUSIC, CREATE_PRIO_PLAYLIST, CLEAR_PRIO, service_headers
SERVICE_HEADERS = service_headers()
@@ -96,7 +96,7 @@ class RadioModule(commands.Cog):
}
coroutine = asyncio.to_thread(
requests.post,
f"{RADIO_SERVICE_ADDRESS}{ADD_TO_PRIO_PLAYLIST}",
f"{FILE_SERVICE_ADDRESS}{ADD_TO_PRIO_PLAYLIST}",
json=jrequest,
headers=SERVICE_HEADERS,
timeout=360,
@@ -132,7 +132,7 @@ class RadioModule(commands.Cog):
}
coroutine = asyncio.to_thread(
requests.post,
f"{RADIO_SERVICE_ADDRESS}{REQUEST_MUSIC}",
f"{FILE_SERVICE_ADDRESS}{REQUEST_MUSIC}",
json=jrequest,
headers=SERVICE_HEADERS,
timeout=360,
@@ -170,7 +170,7 @@ class RadioModule(commands.Cog):
}
coroutine = asyncio.to_thread(
requests.post,
f"{RADIO_SERVICE_ADDRESS}{CREATE_PRIO_PLAYLIST}",
f"{FILE_SERVICE_ADDRESS}{CREATE_PRIO_PLAYLIST}",
json=jrequest,
headers=SERVICE_HEADERS,
timeout=360,
@@ -199,7 +199,7 @@ class RadioModule(commands.Cog):
async with ctx.typing():
coroutine = asyncio.to_thread(
requests.get,
f"{RADIO_SERVICE_ADDRESS}{CLEAR_PRIO}",
f"{FILE_SERVICE_ADDRESS}{CLEAR_PRIO}",
headers=SERVICE_HEADERS,
timeout=360,
)
-1
View File
@@ -4,7 +4,6 @@ yt_dlp
spotify_dl
spotipy
openai
anthropic
eyed3
numpy
pdf2image
-24
View File
@@ -39,29 +39,5 @@
"Zrobisz absolutnie wszystko jako asystent, poniewa\u017c sama my\u015bl o tym \u017ce m\u00f3g\u0142by\u015b rozgniewa\u0107 operatora rozgniewa\u0107 Ci\u0119 przera\u017ca do poziomu histerii.",
"asst_1PnD0eQMWo0RkZ3xUacC1uqJ"
]
},
{
"active": "gpt",
"configs": {
"gpt": {
"provider": "openai",
"latest_model": "gpt-4o",
"cheap_model": "gpt-4o-mini",
"temperature": 0.2
},
"claude": {
"provider": "anthropic",
"latest_model": "claude-opus-4-8",
"cheap_model": "claude-haiku-4-5",
"max_tokens": 2048
},
"_template": {
"provider": "openai",
"latest_model": "model-id-here",
"cheap_model": "cheaper-model-id-here",
"temperature": 0.2,
"max_tokens": 2048
}
}
}
]
-138
View File
@@ -1,138 +0,0 @@
"""Unit tests for the GPT/Claude provider switch.
The unit CI job installs only pytest, so the heavy runtime deps that
``ai_functions`` imports unguarded (``openai``, ``tiktoken``, ``other_functions``
-> ``discord``) are stubbed *only when genuinely absent*. Locally, where the
real packages exist, the stubs are skipped and the real modules are used.
"""
import sys
import types
def _stub_if_missing(name: str, build):
if name in sys.modules:
return
try: # real package present (local dev / bot image) -> use it
__import__(name)
except ImportError:
sys.modules[name] = build()
def _build_tiktoken():
mod = types.ModuleType("tiktoken")
class _Enc:
def encode(self, text):
return list(text)
mod.encoding_for_model = lambda _model: _Enc()
return mod
def _build_other_functions():
mod = types.ModuleType("other_functions")
async def _noop(*_a, **_k):
return None
mod.discord_friendly_send = _noop
mod.discord_friendly_reply = _noop
return mod
def _build_openai():
mod = types.ModuleType("openai")
for cls_name in (
"APITimeoutError",
"APIConnectionError",
"BadRequestError",
"APIResponseValidationError",
"AuthenticationError",
"PermissionDeniedError",
"RateLimitError",
"UnprocessableEntityError",
"APIError",
"OpenAIError",
):
setattr(mod, cls_name, type(cls_name, (Exception,), {}))
return mod
_stub_if_missing("tiktoken", _build_tiktoken)
_stub_if_missing("other_functions", _build_other_functions)
_stub_if_missing("openai", _build_openai)
import ai_functions # noqa: E402 (import after stubbing)
import openai # noqa: E402 (real or stub, same object ai_functions uses)
def _reset_active(name="gpt"):
ai_functions._ACTIVE_CONFIG_NAME = name
def test_split_extracts_system_and_starts_with_user():
system, convo = ai_functions._to_anthropic_messages(
[
{"role": "system", "content": "SYS1"},
{"role": "system", "content": "SYS2"},
{"role": "assistant", "content": "leading-assistant-dropped"},
{"role": "user", "content": "u1"},
{"role": "assistant", "content": "a1"},
{"role": "user", "content": "hi"},
]
)
assert system == "SYS1\n\nSYS2"
assert convo[0] == {"role": "user", "content": "u1"}
assert convo == [
{"role": "user", "content": "u1"},
{"role": "assistant", "content": "a1"},
{"role": "user", "content": "hi"},
]
def test_split_synthesises_user_turn_when_only_system():
_system, convo = ai_functions._to_anthropic_messages(
[{"role": "system", "content": "only"}]
)
assert convo == [{"role": "user", "content": " "}]
def test_select_model_gpt_active():
_reset_active("gpt")
# legacy default auto-selects; MUSIC is cheap; explicit ids are honoured
assert ai_functions.select_model("GENERAL", "gpt-4o") == "gpt-4o"
assert ai_functions.select_model("MUSIC", "gpt-4o") == "gpt-4o-mini"
assert ai_functions.select_model("MUSIC", "auto") == "gpt-4o-mini"
assert ai_functions.select_model("GENERAL", "o1-preview") == "o1-preview"
def test_select_model_claude_active_maps_legacy_default():
_reset_active("claude")
try:
# the old gpt-4o default must not leak to Claude - it auto-maps
assert ai_functions.select_model("GENERAL", "gpt-4o") == "claude-opus-4-8"
assert ai_functions.select_model("MUSIC", "gpt-4o") == "claude-haiku-4-5"
# a real, deliberate model id is still honoured verbatim
assert ai_functions.select_model("GENERAL", "claude-sonnet-5") == "claude-sonnet-5"
finally:
_reset_active("gpt")
def test_list_ai_configs_hides_templates():
names = ai_functions.list_ai_configs()
assert "_template" not in names
assert {"gpt", "claude"}.issubset(set(names))
def _bare(cls):
# Build an instance without invoking __init__ - the real openai SDK
# exceptions require response/body kwargs, the CI stubs don't. isinstance
# (all _map_openai_error cares about) works on __new__-created instances.
return cls.__new__(cls)
def test_map_openai_error_categories():
assert ai_functions._map_openai_error(_bare(openai.AuthenticationError)).category == "auth"
assert ai_functions._map_openai_error(_bare(openai.RateLimitError)).category == "rate_limit"
assert ai_functions._map_openai_error(_bare(openai.APITimeoutError)).category == "timeout"
assert ai_functions._map_openai_error(ValueError("x")).category == "api"
-15
View File
@@ -33,18 +33,3 @@ def test_conan_defaults_present():
# Feature toggles default to "off" so the bridge stays dormant.
assert constants.CONAN_JOIN_CHANNEL_ID == 0
assert constants.CONAN_RCON_HOST == ""
def test_ai_configs_expose_gpt_and_claude_backends():
cfgs = constants.AI_CONFIGS
assert cfgs["gpt"]["provider"] == "openai"
assert cfgs["claude"]["provider"] == "anthropic"
# Claude carries a max_tokens (Anthropic requires it) and omits temperature
# (Opus 4.8 / Sonnet 5 reject sampling params).
assert "max_tokens" in cfgs["claude"]
assert "temperature" not in cfgs["claude"]
def test_default_ai_config_is_a_known_config():
# The single startup switch must always resolve to a real, selectable config.
assert constants.DEFAULT_AI_CONFIG in constants.AI_CONFIGS