gitea/conjurer
1
0
Fork 0
mirror of https://github.com/migatu/conjurer.git synced 2026-07-14 13:34:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

split: betoniarka (radio operator) colocated with liquidsoap; musician goes Discord-only

Browse Source 
Permissions post-mortem that motivated this: the musician wrote radio
playlists AS ROOT onto a ROOT-OWNED network share which liquidsoap then
read AS USER 'radio' - chown fails on such shares by design (root squash /
uid mapping), so the radio came up and died on the playlists. The fix is
structural: the playlist WRITER now lives in the same container as the
READER, as the same user, on a local volume. No shared partition, no
chown, no uid mapping.

New: conjurer_betoniarka/betoniarka.py - runs inside the radio container
(started by the entrypoint as user 'radio', port 5005):
- library scan -> all_playlist/hit playlists with LOCAL container paths
  (start + every 24h + authenticated GET /rescan)
- bot-facing radio API moved from the musician: /add_to_priority,
  /create_priority_playlist, /request_radio_file, /clear_pr_pls,
  plus GET /ping (health) and /stream (web page)
- radio_log/persistence tailer forwarding play events to the bot's
  /prepped_tracks with the shared API key (bot-unreachable = logged, not fatal)

Musician: pure Discord music player now - keeps /mp3, /update_mp3,
/get_music and the file-share endpoints; all radio playlist writing, radio
paths/env and the tailer removed.

Bot: new CONJURER_RADIO_SERVICE (defaults to CONJURER_FILE_SERVICE so
un-split deployments keep working); radio_commands targets it; separate
'radio' health-gate group on betoniarka /ping (musician group now covers
music_commands + file_search_commands only).

Docker: betoniarka baked into the radio image (python3 + flask/waitress/
requests from Debian debs), port 5005 exposed, entrypoint starts it via
setpriv as 'radio'; data-volume chown is now best-effort with a loud
warning (keep the volume local); docs get the post-mortem + wiring.

Verified: py_compile everything; functional stub tests - rescan writes
local-path playlists, wyszukaj scores and appends to priority, auth
401/ok, tailer forward carries the API key.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Michal Tuszowski
2026-07-08 22:14:59 +02:00
committed by Michał Tuszowski
parent f17cf7fdd8
commit 2a21fc9e8c
10 changed files with 453 additions and 312 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conjurer_musician/conjurer_musician.py
+15 -298
View File
@@ -1,7 +1,12 @@
"""
The provided Python script sets up a Flask web server to manage a list of music files, with
functions for rescanning the music folder, updating the music list, and serving the music list via
API endpoints.
"""Musician - the Discord music player service.
Serves the music library index and keyword search the bot uses for Discord
playback (/mp3, /update_mp3, /get_music) plus the file-share endpoints.
Radio playlist management and the radio-log tailer moved to
conjurer_betoniarka/betoniarka.py, which runs INSIDE the radio container as
the same user as Liquidsoap - the musician no longer writes any radio files,
so the old root-owned-network-share permission mess is gone.
"""
import json
@@ -14,18 +19,9 @@ import time
from datetime import datetime
from logging import handlers
from pathlib import Path
from typing import Dict, List
from typing import List
import requests
from flask import (
Flask,
abort,
jsonify,
redirect,
render_template,
request,
send_from_directory,
)
from flask import Flask, abort, jsonify, request
from waitress import serve
import media_search_functions
@@ -41,8 +37,6 @@ def _env_path(name: str, default: str) -> Path:
API_KEY = os.getenv("CONJURER_API_KEY")
MAIN_BOT_ADDRESS = _env("CONJURER_MAIN_BOT", "http://127.0.0.1:5000")
MUSIC_TRACKER = _env("CONJURER_MUSIC_TRACKER_ENDPOINT", "/prepped_tracks")
HOST_ADDRESS = _env("CONJURER_MUSICIAN_HOST", "0.0.0.0")
PORT_ADDRESS = int(_env("CONJURER_MUSICIAN_PORT", "5000"))
@@ -56,52 +50,12 @@ LOGSTORE = _env_path("CONJURER_LOGSTORE", str(BASE_DIR / "logs"))
MUSIC_FOLDER = _env_path(
"CONJURER_MUSIC_FOLDER", str(BASE_DIR / "music")
)
PRIORITY_FOLDER = _env_path(
"CONJURER_PRIORITY_FOLDER", str(MUSIC_FOLDER / "priority")
)
RADIOLOG_PATH = _env_path(
"CONJURER_RADIO_LOG", str(BASE_DIR / "radio_log.log")
)
PERSISTENCE_PATH = _env_path(
"CONJURER_PERSISTENCE_LOG", str(BASE_DIR / "persistence.log")
)
ALL_PLAYLIST_PATH = _env_path(
"CONJURER_ALL_PLAYLIST", str(BASE_DIR / "all_playlist.playlist")
)
HIT_PLAYLIST_PATH = _env_path(
"CONJURER_HIT_PLAYLIST", str(BASE_DIR / "hit.playlist")
)
REQUEST_PLAYLIST_PATH = _env_path(
"CONJURER_REQUEST_PLAYLIST", str(BASE_DIR / "request.playlist")
)
PRIORITY_PLAYLIST_PATH = _env_path(
"CONJURER_PRIORITY_PLAYLIST", str(BASE_DIR / "priority_queue.playlist")
)
STREAM_TEMPLATE = _env_path(
"CONJURER_STREAM_TEMPLATE", str(BASE_DIR / "stream.html")
)
ENCODING = _env("CONJURER_ENCODING", "utf-8")
SEPARATOR_FILE_PATH = os.sep
for playlist_path in (
ALL_PLAYLIST_PATH,
HIT_PLAYLIST_PATH,
REQUEST_PLAYLIST_PATH,
PRIORITY_PLAYLIST_PATH,
):
playlist_path.parent.mkdir(parents=True, exist_ok=True)
random.seed()
music_file_list: List[str] = []
priority_list: List[str] = []
def _build_headers() -> Dict[str, str]:
headers: Dict[str, str] = {}
if API_KEY:
headers["X-Conjurer-Api-Key"] = API_KEY
return headers
def _authorize_request() -> None:
@@ -109,29 +63,16 @@ def _authorize_request() -> None:
abort(401)
def _post_to_bot(payload: List[str]) -> None:
response = requests.post(
f"{MAIN_BOT_ADDRESS}{MUSIC_TRACKER}",
json=payload,
headers=_build_headers(),
timeout=60,
)
logger = logging.getLogger("conjurer_musician")
logger.info("SENT")
logger.info(response.status_code)
logger.info("SEND CONFIRMED")
def rescan():
"""
The `rescan` function logs a message, scans for mp3 files in a specified folder,
and adds them to a list of music files.
"""Refresh the in-memory library index used by /mp3 and /get_music.
Radio playlist files are NOT written here anymore - that is the
betoniarka's job, colocated with Liquidsoap.
"""
logger = logging.getLogger("conjurer_musician")
logger.info("Rescan triggered")
music_file_list.clear()
priority_list.clear()
for mp3_item in MUSIC_FOLDER.glob("**/*.mp3"):
temp_music_file = mp3_item.as_posix()
@@ -139,27 +80,6 @@ def rescan():
temp_music_file = temp_music_file.replace("/", "\\")
music_file_list.append(temp_music_file)
for mp3_item in PRIORITY_FOLDER.glob("**/*.mp3"):
temp_music_file = mp3_item.as_posix()
if os.name == "nt":
temp_music_file = temp_music_file.replace("/", "\\")
priority_list.append(temp_music_file)
with ALL_PLAYLIST_PATH.open("w", encoding=ENCODING) as w_file:
try:
for item in music_file_list:
w_file.write(item)
w_file.write("\n")
except json.JSONDecodeError:
pass
with HIT_PLAYLIST_PATH.open("w", encoding=ENCODING) as w_file:
try:
for item in priority_list:
w_file.write(item)
w_file.write("\n")
except json.JSONDecodeError:
pass
def thread_rescan():
"""
@@ -174,55 +94,6 @@ def thread_rescan():
rescan()
def scan_tracks():
# Set the filename and open the file
logger = logging.getLogger("conjurer_musician")
with open(RADIOLOG_PATH, "r", encoding=ENCODING) as log_file:
log_file.seek(os.stat(RADIOLOG_PATH).st_size)
prev_size = os.stat(PERSISTENCE_PATH).st_size
while True:
current_size = os.stat(PERSISTENCE_PATH).st_size
if prev_size != current_size:
while prev_size != current_size:
prev_size = current_size
time.sleep(0.1)
current_size = os.stat(PERSISTENCE_PATH).st_size
with open(PERSISTENCE_PATH, "r", encoding=ENCODING) as persistence:
lines = persistence.readlines()
if len(lines) >= 3:
_post_to_bot(["next", lines[2]])
position = log_file.tell()
line = log_file.readline()
if not line:
time.sleep(1)
log_file.seek(position)
continue
if not re.match(r".*Prepared.*", line):
time.sleep(0.1)
continue
result = None
if re.match(r".*jingles.*", line):
result = ["jingles", line]
elif re.match(r".*priority.*", line):
result = ["priority", line]
elif re.match(r".*hit.*", line):
result = ["hit", line]
elif re.match(r".*all_playlist.*", line):
result = ["all", line]
elif re.match(r".*request.*", line):
result = ["requests", line]
if result:
logger.info("Forwarding radio log entry: %s", result[0])
_post_to_bot(result)
time.sleep(0.1)
app = Flask(__name__)
# AutoIndex(app, browse_root="/")
@@ -328,9 +199,6 @@ def wyszukaj(word_list, how_many, _logger=None, return_to_bot=True):
while not_found:
if search_weight[itr][0] == item_to_search:
return_list.append(search_weight[itr])
if not return_to_bot:
with PRIORITY_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
s_file.write(search_weight[itr][1] + "\n")
break
itr += 1
else:
@@ -401,62 +269,6 @@ def get_share_links():
@app.route("/stream", methods=["GET"])
def stream_music():
"""
The function `stream_music` is a route handler for the "/stream" endpoint.
It returns a JSON response containing a key "music_file_list" with the value of the variable `music_file_list`.
:return: A JSON response containing a key "music_file_list" with the value of the variable `music_file_list`.
"""
# return send_from_directory("/tmp/hls", "stream.m3u8")
return render_template(str(STREAM_TEMPLATE))
@app.route("/<string:file_name>")
def stream(file_name):
"""
Stream the specified file from the video directory.
Args:
file_name (str): The name of the file to be streamed.
Returns:
Response: The response containing the streamed file.
"""
# trunk-ignore(bandit/B108)
video_dir = "/tmp/hls"
return send_from_directory(video_dir, file_name)
@app.route("/stream_mp3", methods=["GET"])
def stream_music_mp3():
"""
The function `stream_music_mp3` is a route handler for the "/stream_mp3" endpoint.
It redirects the user to the URL "http://www.example.com" with a status code of 302.
:return: A redirect response to "http://www.example.com" with a status code of 302.
"""
return redirect("http://www.example.com", code=302)
@app.route("/clear_pr_pls", methods=["GET"])
def clear_pr_pls():
_authorize_request()
"""
The function `clear_pr_pls` clears the contents of the priority queue playlist file.
:return: A JSON response indicating the success of the operation.
"""
app.logger.info("CLEARING PLAYLIST")
with PRIORITY_PLAYLIST_PATH.open("w", encoding=ENCODING) as cleared_pl:
cleared_pl.write("")
return_data = jsonify(isError=False, message="Success", statusCode=200, data=[])
return return_data, 200
@app.route("/mp3", methods=["GET"])
def get_music_list():
"""
@@ -517,96 +329,6 @@ def look_for_playlist():
return return_data
@app.route("/request_radio_file", methods=["POST"])
def add_request():
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
app.logger.info(record["UUID"])
return_data = wyszukaj(record["lista_slow"], 0, app.logger, False)
with REQUEST_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return_data = (
jsonify(
isError=False, message="Success", statusCode=200, data={"status": "OK"}
),
200,
)
return return_data
@app.route("/create_priority_playlist", methods=["POST"])
def create_priority_playlist():
"""
The function `create_priority_playlist` receives a POST request with a JSON payload, logs the received
item, adds it to a music file list, and returns a success message along with the updated record.
:return: A tuple containing a JSON response and a status code.
The JSON response includes keys `isError`,
`message`, `statusCode`, and `data`, with values
indicating the success of the operation and the
data that was received and added to the `music_file_list`.
The status code returned is 200,indicating a successful response.
"""
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
app.logger.info(record["UUID"])
app.logger.info(record["dlugosc_plejlisty"])
return_data = wyszukaj(
record["lista_slow"], record["dlugosc_plejlisty"], app.logger, False
)
random.shuffle(return_data)
with REQUEST_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return_data = (
jsonify(
isError=False, message="Success", statusCode=200, data={"status": "OK"}
),
200,
)
return return_data
@app.route("/add_to_priority", methods=["POST"])
def add_to_priority():
"""
The function `add_to_priority` receives a POST request with a JSON payload, logs the received
item, adds it to a music file list, and returns a success message along with the updated record.
:return: A tuple containing a JSON response and a status code.
The JSON response includes keys `isError`,
`message`, `statusCode`, and `data`, with values
indicating the success of the operation and the
data that was received and added to the `music_file_list`.
The status code returned is 200,indicating a successful response.
"""
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
app.logger.info(record["UUID"])
app.logger.info(record["dlugosc_plejlisty"])
return_data = wyszukaj(
record["lista_slow"], record["dlugosc_plejlisty"], app.logger, False
)
with PRIORITY_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return_data = (
jsonify(
isError=False, message="Success", statusCode=200, data={"status": "OK"}
),
200,
)
return return_data
def flask_debug():
"""
The `flask_debug` function starts a Flask application in debug mode without using the reloader.
@@ -660,13 +382,8 @@ if __name__ == "__main__":
for worker in threads:
worker.start()
time.sleep(60)
track_thread = threading.Thread(target=scan_tracks, daemon=True)
track_thread.start()
try:
for worker in threads:
worker.join()
track_thread.join()
except KeyboardInterrupt:
logger.info("Shutdown requested - exiting musician service")