Compare commits

..

77 Commits

Author SHA1 Message Date
Michal Tuszowski da8260d164 fix: voice_recognition uses constants for credentials and paths
The cog did its own netrc read from a hardcoded /home/pi/.netrc at import
time, crashing on any other host. Now:

- constants.py: ASSEMBLYAI_API_KEY resolved like every other token
  (env ASSEMBLYAI_API_KEY -> netrc machine 'assemblyai' at
  CONJURER_NETRC_FILE); new TRANSCRIPTS_PATH (env
  CONJURER_TRANSCRIPTS_PATH, defaults next to the log file, created by the
  runtime layout)
- voice_recognition_commands.py: drop the hardcoded netrc read and
  transcript dir; when the key is missing raise a clear RuntimeError so
  the guarded loader disables ONLY this cog with a readable reason
- bot.env.example: document ASSEMBLYAI_API_KEY

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 00:33:13 +02:00
Michal Tuszowski b266db5bf3 fix: Client.start() takes no log_handler kwarg (run()-only)
log_handler belongs to Client.run() (which configures logging and then
calls start()); passing it to start() raised TypeError at launch. This was
inherited from the never-run dockerised variant's thin_client.py. We
configure our own handlers, so simply drop the kwarg.

Also set logger.propagate=False: a dependency calls logging.basicConfig(),
so every 'discord' record was printed twice (our format + root's).

Verified with a strict-signature stub client (start(token, *, reconnect)).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 00:26:02 +02:00
Michal Tuszowski 0a29c9950d fix: never swallow task exceptions - diagnose WHY the bot died
The restart-loop with only 'Starting discord bot' in the logs was caused by
gather(return_exceptions=True) eating the client.start() exception: a failed
login looked like a clean exit and docker just restarted the container.

bot.py:
- _run_bot: catch discord.LoginFailure / PrivilegedIntentsRequired with
  CRITICAL messages telling exactly what to fix (token entry / Developer
  Portal intents), full traceback for anything else, then re-raise
- main() returns an exit code; every task exception is re-logged as the
  LAST line in docker logs ('Task died: ...'); process exits 1 on failure
- log the token source (env vs netrc) and its length at startup to catch
  truncated/wrong-field netrc entries
- _run_comm_subroutine: comm crash is logged loudly but no longer relies on
  gather to surface it (bot keeps running - Discord side is independent)

communication_subroutine.py (crash found by the new stub tests):
- comm_subroutine used 'logger' before its local assignment
  (UnboundLocalError killed the whole comm layer at every startup)
- re-comment the flask_debug thread: it binds the same host:port as
  waitress, so running both dies with 'address in use'

Verified via stub-injected client: LoginFailure -> CRITICAL + exit 1,
PrivilegedIntentsRequired -> CRITICAL + exit 1, unknown exception -> full
traceback + exit 1, clean shutdown -> exit 0; comm layer no longer crashes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 00:22:55 +02:00
Michal Tuszowski 8ba954176a fix: make bot startup resilient (self-healing state, gated cogs, loud failures)
Container/native startup died silently on any missing file/service. Now:

constants.py - self-healing runtime layout at import:
- create missing dirs (log dir, LOGSTORE, GRAPHICS_PATH, MUSIC_FOLDER)
- seed missing state files from the repo templates shipped next to
  constants.py (settings/system_gpt/pamiec/pamiec_muzyki/accident_log),
  falling back to safe empty JSON; existing files are NEVER overwritten

bot.py:
- log to stdout too, so 'docker logs' finally shows the crash reason
- missing Discord token = loud sys.exit with mount/env instructions
  (was: silent return -> container crash-loop with empty logs)
- every cog loads independently (one broken cog = skipped with traceback,
  bot continues)
- musician/librarian cogs are health-gated: enabled only when the service
  answers HTTP; a watchdog re-checks every 5 min and enables them the
  moment the service comes alive (no restart needed); tree re-synced
- on_ready reconnects no longer re-load extensions

requirements_conan.txt + Dockerfile.bot: aiomcrcon (Python <=3.11 only)
moved to best-effort extras so the 3.13 image builds clean and the
conanjurer cog stays dormant without it.

DOCKER_PROXMOX.md: startup model (core vs gated cogs) + crash-loop
troubleshooting incl. the 'disappearing files' checklist (nothing in the
stack deletes host files; bind mount = live state).

Verified: fresh-volume seeding creates dirs+templates, existing files
untouched, missing-token exits with FATAL message, health-gating logic
(stub-based runpy tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 00:08:31 +02:00
gitea 759c04a48f add 2026-07-07 23:24:12 +02:00
root 4ff0427b64 Docker and installation issues 2026-07-07 23:22:12 +02:00
Michal Tuszowski 934e7a6240 docker: dedicated musician entrypoint + fuller runbook section
- docker/entrypoint.musician.sh: on a fresh /data volume, create the
  managed playlists and touch radio_log.log/persistence.log empty so the
  track-forwarding thread waits instead of crashing when Liquidsoap runs
  elsewhere; never overwrites preserved files
- Dockerfile.musician: wire the entrypoint, default CONJURER_MUSICIAN_BASE
  =/data (persist playlists/logs) and CONJURER_STREAM_TEMPLATE=/app/stream.html;
  declare the /data volume
- DOCKER_PROXMOX.md: expand the musician section (two volumes, preserving
  existing playlists, radio-log coupling, stream template)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 10:16:56 +02:00
Michal Tuszowski 597bc004fc docker: Proxmox deployment for bot/librarian/musician + code fixes
Containerises the three services (each intended for its own Proxmox VM)
and adds the code changes needed to run cleanly on Linux/Docker.

Code fixes:
- constants.py: CONJURER_DATA_DIR roots all writable bot state under one
  mounted volume (per-variable overrides still win; native Pi unaffected)
- conjurer_librarian/search_bot.py + scrape_bot.py: the hardcoded Windows
  DOI database path (C:\Database\chunks\) is now CONJURER_LIBRARIAN_DB_PATH,
  with CONJURER_LIBRARIAN_MAXTHREADS / _CHUNK also env-overridable

Docker:
- docker/Dockerfile.{bot,librarian,musician} + compose.{bot,librarian,musician}.yaml
- docker/env/*.env.example (force-added; real *.env stays gitignored)
- docker/entrypoint.bot.sh seeds default JSON state into /data only when
  absent, so preserved history is never overwritten
- .dockerignore
- docs/deployment/DOCKER_PROXMOX.md: step-by-step runbook incl. preserving
  the existing command/conversation history and cross-VM auth

The bot image uses the vendored yt_dlp/spotify_dl forks (they win on
sys.path over the pip packages), dropping the old sed patching.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 10:16:56 +02:00
gitea 9c8384b10c Ci/test workflows (#15)
* ci: replace broken default workflows with compile/unit/integration CI

The two scaffold workflows (Python application / Python package) failed on
every PR: they installed deps from a non-existent requirements.txt, ran
flake8/pytest over the vendored yt_dlp fork (new syntax under the 3.8/3.9
matrix), and collected ad-hoc root scripts — notably test_ai.py, which is
an invalid pasted object dump (not Python).

- Remove python-app.yml / python-package.yml and the junk root scripts
  (test.py, test_ai.py, test_time.py)
- Add .github/workflows/ci.yml with three PR-check jobs:
  * compile     — py_compile every first-party .py (no deps)
  * unit        — pytest on pure logic (conanjurer_functions, constants)
  * integration — boot the Flask services and assert the X-Conjurer-Api-Key
                  auth contract (communication_subroutine + conjurer_musician)
- Add tests/ suite, pytest.ini (testpaths=tests) and conftest.py (sys.path)

Fixes surfaced by the compile gate / needed for the integration job:
- conjurer_librarian/search_bot.py + search_bot2.py: f-string reused the
  same quote ({item["exists"]}) -> SyntaxError on Python < 3.12
- conjurer_musician/media_search_functions.py: made import-safe
  (env-overridable paths, lazy DB load / mkdir) so the service can be
  imported and tested off the Pi

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: make workflow ASCII-clean to fix startup_failure

The em-dash characters in the job comments tripped the workflow validator
(startup_failure, 0s). Rewrite ci.yml with plain ASCII and simpler comments.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: minimal workflow to isolate startup_failure

* ci: restore full compile/unit/integration workflow

The minimal echo workflow was only to isolate the failure: it confirmed
that *every* Actions run in this repo ends in startup_failure (0s),
including a trivial echo job. The cause is account/repo-level (Actions
minutes/billing on a private free-plan repo), not the workflow file.
Restoring the real 3-job CI so it runs once Actions is unblocked.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Michal Tuszowski <michaltuszowski@Mac-mini-Michal.localdomain>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 12:27:32 +02:00
Michal Tuszowski a34a2a3299 ci: minimal workflow to isolate startup_failure 2026-06-29 12:24:30 +02:00
Michal Tuszowski bd82369006 ci: make workflow ASCII-clean to fix startup_failure
The em-dash characters in the job comments tripped the workflow validator
(startup_failure, 0s). Rewrite ci.yml with plain ASCII and simpler comments.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 12:24:30 +02:00
Michal Tuszowski a6c20a0054 ci: replace broken default workflows with compile/unit/integration CI
The two scaffold workflows (Python application / Python package) failed on
every PR: they installed deps from a non-existent requirements.txt, ran
flake8/pytest over the vendored yt_dlp fork (new syntax under the 3.8/3.9
matrix), and collected ad-hoc root scripts — notably test_ai.py, which is
an invalid pasted object dump (not Python).

- Remove python-app.yml / python-package.yml and the junk root scripts
  (test.py, test_ai.py, test_time.py)
- Add .github/workflows/ci.yml with three PR-check jobs:
  * compile     — py_compile every first-party .py (no deps)
  * unit        — pytest on pure logic (conanjurer_functions, constants)
  * integration — boot the Flask services and assert the X-Conjurer-Api-Key
                  auth contract (communication_subroutine + conjurer_musician)
- Add tests/ suite, pytest.ini (testpaths=tests) and conftest.py (sys.path)

Fixes surfaced by the compile gate / needed for the integration job:
- conjurer_librarian/search_bot.py + search_bot2.py: f-string reused the
  same quote ({item["exists"]}) -> SyntaxError on Python < 3.12
- conjurer_musician/media_search_functions.py: made import-safe
  (env-overridable paths, lazy DB load / mkdir) so the service can be
  imported and tested off the Pi

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 12:24:30 +02:00
Michal Tuszowski a32bbdd03c docs: add working-copy -> prototype migration runbook
Step-by-step operator guide for upgrading a running deployment from the
working-copy bot to the prototype (env-var config, optional internal HTTP
auth, Conan Exiles bridge). Covers backup/rollback, new dependencies,
systemd EnvironmentFile wiring, verification, behaviour changes, and
opt-in feature toggles (API key, Conan, player-join notifications).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 11:31:10 +02:00
Michal Tuszowski 5adeb1b384 Land prototype on main (fix stacked-PR retarget gap)
PRs #10 and #11 were merged into their intermediate base branches
(restructure/working-copy-root and proto-improvements) rather than main,
because the stacked PRs' bases were not auto-retargeted (the branches were
not deleted on merge). As a result main only received the #9 restructure
and is still the plain working-copy bot.

This brings the full prototype onto main as a clean delta on top of the
current main tree (identical content to proto-improvements, but with main
ancestry so it merges without the squash-induced rename/delete conflicts):

- constants.py: env-var config, safe JSON loading, dependency guards,
  env->netrc tokens, API_SHARED_KEY + service_headers(), CONAN_* config
- communication_subroutine.py: queue timeout/Empty, daemon threads,
  cooperative stop_event, inbound _authorize_request()
- bot.py: asyncio event loop + load conanjurer_commands
- music_functions / radio_commands / librarian_commands: X-Conjurer-Api-Key
- conanjurer_commands/_functions: fixed + integrated bridge with RCON
  player-join notifications
- requirements_bot.txt: aiomcrcon, asyncssh
- conjurer_musician/.gitignore: keep runtime playlists/mp3 out of the repo

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 11:28:14 +02:00
Michal Tuszowski a64fb2da57 Restructure: promote working_copy to repo root
Make the stable 'working copy' bot the canonical code at the repository
root so the install/deploy scripts operate against it again.

- Move working_copy/* to root (bot entrypoint is bot.py)
- Restore root-level install/ops scripts from c4fa88e (deploy.sh,
  install_main_bot.sh, status_report.*, conjurer.service, etc.)
- Fix deploy.sh: copy bot.py (was thin_client.py) and add the
  conanjurer_* modules; bump command count
- Remove side-by-side variant dirs (backup_old_docker, prototype_one,
  prototype_musician_one, musician_old, working_copy) and docker cruft
- Keep components as subdirs: conjurer_librarian, conjurer_musician,
  spotify_dl, yt_dlp, fonts, utils, docs

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 23:50:55 +02:00
Michal Tuszowski f9581fa24b New musician 2026-06-16 22:20:07 +02:00
Michal Tuszowski 0473159b94 Proto 2026-06-16 14:09:28 +02:00
Michal Tuszowski 81a25b8c56 proto 2026-06-16 14:09:28 +02:00
Michal Tuszowski 8e5e4ce530 Backup old. Preparation for forking 2026-06-16 14:09:27 +02:00
Michal Tuszowski 92940a4d46 Changes, preparing for dockerization 2.0 2026-06-16 14:09:27 +02:00
gitea e6d3492790 Old musician 2026-06-16 14:02:50 +02:00
Michal Tuszowski d6b33cc614 nowe 2026-06-11 18:26:32 +02:00
Michal Tuszowski b107f01208 Working copy 2026-06-11 18:13:05 +02:00
gitea 22b33fa984 rolled back version 2026-06-08 00:44:17 +02:00
gitea f6ccdb3e34 Conanjurer generation 2026-06-04 19:13:50 +02:00
gitea 1f271b4c71 Syncinc script drop in 2025-10-30 16:58:41 +01:00
gitea f9ad679833 Dockerization + ai review recomendations. 2025-10-29 14:57:43 +01:00
gitea c4fa88e8ee Legacy remove 2025-10-29 13:54:58 +01:00
gitea d43980c976 sa 2025-08-23 17:53:08 +02:00
gitea f6607440ad aaa 2025-08-23 17:49:47 +02:00
gitea 8750430cde FX 2025-08-23 17:44:08 +02:00
gitea 640f8bd824 aaa 2025-08-23 16:20:16 +02:00
gitea bf2835cfc7 fix 2025-08-23 12:36:07 +02:00
gitea a0663d32e0 ttt 2025-08-23 12:30:50 +02:00
gitea 534b1feb58 testing 2025-08-23 12:25:10 +02:00
gitea bb9cdc483d ffx 2025-08-20 22:34:15 +02:00
gitea 04799a8ac1 fx 2025-08-20 22:16:33 +02:00
gitea 75e2205b3d fx 2025-08-20 22:07:10 +02:00
gitea 75014d2ec9 Merge branch 'main' of https://github.com/migatu/conjurer 2025-08-20 21:59:26 +02:00
gitea 28685780f3 Vibe coding :P 2025-08-20 21:59:11 +02:00
gitea 609dbae864 refix 2025-08-20 21:21:15 +02:00
gitea 7ef1502249 upgrade to latest OpenAI API usage and logging improvements 2025-08-20 21:20:00 +02:00
gitea 2f13853084 Partial + reformatting 2025-08-20 18:25:23 +02:00
gitea 9f262e5d2f Two util scripts 2025-08-16 21:55:41 +02:00
gitea ba80523d6c aaa 2025-08-16 20:19:37 +02:00
gitea af20f6128c Figure out 2025-08-16 20:16:58 +02:00
gitea 92f2992b97 rollback 2025-08-16 19:16:03 +02:00
gitea ab1ebed5f4 upgrade 2025-08-16 19:12:47 +02:00
gitea 6dc4c9d1b3 disable exc masking 2025-08-16 19:03:13 +02:00
gitea c49615aab6 debug 2025-08-16 18:56:34 +02:00
gitea 340a2e706b debug 2025-08-16 18:55:07 +02:00
gitea 5204c257cf tectonic 2025-08-16 18:49:51 +02:00
gitea 3126ad5266 don't trust ai 2025-08-16 18:19:11 +02:00
gitea 2fa4771b54 fx to fx 2025-08-16 18:10:51 +02:00
gitea dd869cef36 var fx 2025-08-16 18:10:07 +02:00
gitea 62b4046366 installer fix 2025-08-16 18:00:16 +02:00
gitea f0bd6c4015 Latex incoming changes 2025-08-16 17:54:06 +02:00
gitea 6cc43ed16d fx and upgd 2025-08-16 17:36:57 +02:00
gitea 5b8d1ebb27 bgfx 2025-08-16 17:29:30 +02:00
gitea c0326288ff Added the following changes to the codebase: 2025-08-16 17:26:15 +02:00
gitea 97f43bc6a7 Update AI and librarian commands to use 'GENERAL' context" 2025-08-16 16:58:28 +02:00
gitea b52e9be5f9 Merge branch 'main' of https://github.com/migatu/conjurer 2025-08-15 19:31:38 +02:00
gitea cae1f48035 RPG 2025-08-15 19:31:22 +02:00
gitea 1f5b678139 small fixes 2025-05-04 21:15:28 +02:00
gitea b778a1c3f9 Watcher 2025-05-04 16:15:45 +02:00
gitea 074452e6a9 Params of radio 2025-05-03 16:41:22 +02:00
gitea 7edd60c10a zmiana nazwy 2025-05-02 20:20:15 +02:00
gitea c7af223155 fix deploy 2025-05-02 20:12:33 +02:00
gitea a79aa4e9a7 fix 2025-05-02 20:10:18 +02:00
gitea d0c972c69b M 2025-05-02 20:09:11 +02:00
gitea 78c958e8f5 added server side search 2025-05-02 19:55:52 +02:00
gitea 3ee7fcab59 serverside 2025-05-02 19:55:52 +02:00
gitea 75932ec507 Search client 2025-05-02 19:54:08 +02:00
gitea ee84d6ec32 more work needed 2025-05-02 19:17:27 +02:00
gitea a8001ec19d new function 2025-05-02 19:01:51 +02:00
gitea 95cd4760b7 Generated scripts 2025-04-26 13:18:02 +02:00
gitea 894235569e bgfx: 2025-02-24 22:12:22 +01:00
78 changed files with 4451 additions and 3397 deletions
+17
View File
@@ -0,0 +1,17 @@
# Keep build contexts lean. Component dirs (conjurer_librarian, conjurer_musician)
# are intentionally NOT ignored — their images copy them from this same context.
.git
.github
.trunk
.vscode
**/__pycache__/
*.pyc
*.pyo
*.log
*.mp3
docs/
tests/
conftest.py
pytest.ini
docker/env/*.env
secrets/
+54
View File
@@ -0,0 +1,54 @@
name: CI
on:
push:
branches: ["main"]
pull_request:
branches: ["main"]
permissions:
contents: read
jobs:
compile:
# Byte-compile every first-party .py to catch syntax errors. No deps.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: py_compile first-party sources
run: |
git ls-files '*.py' | grep -vE '^(yt_dlp|spotify_dl)/' | xargs python -m py_compile
echo "All first-party sources compile."
unit:
# Pure-logic tests; tested modules guard heavy deps, so only pytest needed.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install test deps
run: |
python -m pip install --upgrade pip
pip install pytest
- name: Run unit tests
run: pytest tests/unit -v
integration:
# Boot the Flask services and assert the X-Conjurer-Api-Key auth contract.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install service deps
run: |
python -m pip install --upgrade pip
pip install pytest flask waitress requests
- name: Run integration tests
run: pytest tests/integration -v
-39
View File
@@ -1,39 +0,0 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
name: Python application
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
permissions:
contents: read
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytest
-40
View File
@@ -1,40 +0,0 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
name: Python package
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10"]
steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytest
+2 -2
View File
@@ -294,7 +294,7 @@ class Events(commands.Cog):
bartender,
MESSAGE_TABLE,
username,
"CONVERSATION",
"GENERAL",
)
await discord_friendly_reply(message, result)
@@ -332,7 +332,7 @@ class Events(commands.Cog):
True,
MESSAGE_TABLE,
username,
"RANDOM",
"GENERAL",
)
await discord_friendly_reply(
message, f"Sorki, cenzura: {resp}. Jak chcesz to są kanały na nudle #sexy-foteczky i #kanal-do-fapania *Na ekranie pojawia się: {e}"
+208 -167
View File
@@ -5,6 +5,7 @@ import random
import openai
import tiktoken
import time
from other_functions import discord_friendly_send
from constants import (
ASSISTANTS,
@@ -18,18 +19,67 @@ from constants import (
OPENAICLIENT,
SYSTEM_GPT_SETTINGS,
WORD_REACTIONS,
CHEAP_MODEL,
LATEST_MODEL
)
#this do per user
# this do per user
VECTOR_STORE_ID = -1
def select_model(req_type: str, algo: str) -> str:
# Jeżeli jawnie podano algorithm (i nie jest 'auto'/''):
if algo and str(algo).strip().lower() not in ("auto",):
# wyjątek: MUZYKA ma zawsze być tania — nadpisujemy TYLKO jeśli przyszedł domyślny 'gpt-4o'
if req_type == "MUSIC" and algo.strip() in (LATEST_MODEL,):
return CHEAP_MODEL
return algo
# Auto-dobór:
if req_type == "MUSIC":
return CHEAP_MODEL
return LATEST_MODEL
async def openai_call(messages, model, temperature=0.2):
"""
Responses API dla 4o/4.1*, fallback Chat Completions dla gpt-3.5-turbo.
Zwraca czysty string odpowiedzi.
"""
logger = logging.getLogger("discord")
if model.startswith("gpt-3.5"):
logger.info("3.5")
# legacy path bez zmian w Twoim kodzie wyżej/niżej
resp = await OPENAICLIENT.chat.completions.create(
model=model,
messages=messages,
temperature=temperature,
)
result = ""
for choice in resp.choices:
result += choice.message.content
return result.strip()
else:
logger.info("4.0+")
# Responses API (zalecane dla 4o/4.1*)
resp = await OPENAICLIENT.responses.create(
model=model,
temperature=temperature,
input=messages,
)
# SDK zapewnia output_text dla zwykłych odpowiedzi
return (getattr(resp.output_text, "output_text", None) or "").strip() or str(
resp.output_text
)
def create_vector_store():
# Create a vector store caled "Financial Statements"
return OPENAICLIENT.beta.vector_stores.create_and_poll(name="Hammer Stash")
#expires_after={
#"anchor": "last_active_at",
#"days": 7}
#)
# expires_after={
# "anchor": "last_active_at",
# "days": 7}
# )
def upload_files_to_vector_store(assistant):
@@ -37,15 +87,14 @@ def upload_files_to_vector_store(assistant):
file_paths = ["edgar/goog-10k.pdf", "edgar/brka-10k.txt"]
file_streams = [open(path, "rb") for path in file_paths]
#file = client.beta.vector_stores.files.create_and_poll(
#vector_store_id="vs_abc123",
#file_id="file-abc123"
#)
#batch = client.beta.vector_stores.file_batches.create_and_poll(
#vector_store_id="vs_abc123",
#file_ids=['file_1', 'file_2', 'file_3', 'file_4', 'file_5']
#)
# file = client.beta.vector_stores.files.create_and_poll(
# vector_store_id="vs_abc123",
# file_id="file-abc123"
# )
# batch = client.beta.vector_stores.file_batches.create_and_poll(
# vector_store_id="vs_abc123",
# file_ids=['file_1', 'file_2', 'file_3', 'file_4', 'file_5']
# )
# Use the upload and poll SDK helper to upload the files, add them to the vector store,
# and poll the status of the file batch for completion.
@@ -57,10 +106,11 @@ def upload_files_to_vector_store(assistant):
print(file_batch.status)
print(file_batch.file_counts)
assistant = OPENAICLIENT.beta.assistants.update(
assistant_id=assistant.id,
tool_resources={"file_search": {"vector_store_ids": [VECTOR_STORE_ID]}},
assistant_id=assistant.id,
tool_resources={"file_search": {"vector_store_ids": [VECTOR_STORE_ID]}},
)
def delete_files_from_vector_store(assistant, file_id):
result = OPENAICLIENT.beta.vector_stores.file_batches.delete(
vector_store_id=VECTOR_STORE_ID, files=file_id
@@ -69,8 +119,8 @@ def delete_files_from_vector_store(assistant, file_id):
# You can print the status and the file counts of the batch to see the result of this operation.
print(result)
assistant = OPENAICLIENT.beta.assistants.update(
assistant_id=assistant.id,
tool_resources={"file_search": {"vector_store_ids": [VECTOR_STORE_ID]}},
assistant_id=assistant.id,
tool_resources={"file_search": {"vector_store_ids": [VECTOR_STORE_ID]}},
)
@@ -99,7 +149,15 @@ def num_tokens_from_string(message, model):
async def handle_response(
prompt, vykidailo, bartender, history, username, request_type
prompt,
vykidailo,
bartender,
history,
username,
request_type,
algorithm="gpt-4o",
none_request="",
internal_retry: bool = False
):
"""
Handle responses by appending them to a history, use OpenAI to
@@ -115,182 +173,165 @@ async def handle_response(
:param music: The "music" parameter is a boolean value that indicates whether the conversation is
related to music or not. If it is True, the conversation history will be stored in a different file
and the response will be generated using a different model
:param request_type: The type of request being made, which can be "MUSIC", "RANDOM", "NONE" or "GENERAL"
GENERAL is for regular conversations, MUSIC is for music-related requests,
RANDOM is for random requests, and NONE is to not store the request in memory
:param algorithm: The algorithm to be used for generating the response, default is "gpt-4o"
:return: The function `handle_response` returns a tuple containing the `result` and `MESSAGE_TABLE`.
"""
logger = logging.getLogger("discord")
logger.info("Wywolanie procedury openai z promptem: %s", prompt)
temp = {"role": "user", "content": username + ":" + prompt}
if vykidailo or bartender:
logger.info("Administrator coś chciał")
history.append(temp)
if request_type == "MUSIC":
with open(MEMORY_FIVE_MUZYKA, "r+", encoding=ENCODING) as file_music_memory:
# First we load existing data into a dict.
file_data = json.load(file_music_memory)
# Join new_data with file_data inside emp_details
file_data.append(temp)
file_music_memory.seek(0)
# convert back to json.
json.dump(file_data, file_music_memory, indent=4)
elif request_type == "RANDOM":
with open(MEMORY_FIVE_SIARA, "r+", encoding=ENCODING) as file_memory:
# First we load existing data into a dict.
file_data = json.load(file_memory)
# Join new_data with file_data inside emp_details
file_data.append(temp)
file_memory.seek(0)
# convert back to json.
json.dump(file_data, file_memory, indent=4)
model_to_use = select_model(request_type, algorithm)
logger.info("Wybrany model: %s", model_to_use)
if request_type == "MUSIC" and model_to_use == "gpt-4o-mini":
try:
# nic — normalnie pójdzie Responses API
pass
except Exception:
model_to_use = "gpt-3.5-turbo"
# --- 2) Budowa historii (token budget + reguły systemowe) ---
# NOTE: ignorujemy przekazany 'history' jako listę (tak było wcześniej),
# ale zwracamy aktualną tablicę do nadpisania w miejscach wołania (back-compat).
base_system = GPT_SETTINGS[0] # zakładamy {"role":"system","content":...}
history_msgs = []
if request_type != "NONE":
history_msgs.append(base_system)
chat_gpt_config_request_size = num_tokens_from_string(base_system, "gpt-4")
# Dynamiczne mikro-reguły (WORD_REACTIONS), jak w Twoim kodzie
for slowo, reakcja in WORD_REACTIONS.items():
if not reakcja[3]:
content = f"Kiedy słyszysz {slowo} to reagujesz lub dzieje się to {reakcja[0]}"
sys_msg = {"role": "system", "content": content}
chat_gpt_config_request_size += num_tokens_from_string(sys_msg, "gpt-4")
history_msgs.append(sys_msg)
# wybór właściwej tablicy pamięci i budżetu
if request_type == "MUSIC":
table = MESSAGE_TABLE_MUZYKA
token_amount = 10700
elif request_type in ("RANDOM", "GENERAL"):
table = MESSAGE_TABLE
token_amount = 10700
else:
table = []
token_amount = 10000
# doklejanie historii od końca aż do limitu (zachowana kolejność czasowa)
final_prompt = f"{username}:{prompt}"
prompt_gpt_request_size = num_tokens_from_string({"role": "user", "content": final_prompt}, "gpt-4")
acc = []
for msg in reversed(table):
t = num_tokens_from_string(msg, "gpt-4")
if chat_gpt_config_request_size + prompt_gpt_request_size + t <= token_amount:
acc.append(msg)
chat_gpt_config_request_size += t
else:
break
# przywróć chronologicznie
history_msgs.extend(reversed(acc))
# aktualny prompt
history_msgs.append({"role": "user", "content": final_prompt})
logger.info("Rozmiar zapytania (tok): %s", prompt_gpt_request_size) # tokeny już policzone wyżej
else:
with open(MEMORY_FIVE_SIARA, "r+", encoding=ENCODING) as file_memory:
# First we load existing data into a dict.
file_data = json.load(file_memory)
# Join new_data with file_data inside emp_details
file_data.append(temp)
file_memory.seek(0)
# convert back to json.
json.dump(file_data, file_memory, indent=4)
history = []
history.append(GPT_SETTINGS[0])
chat_gpt_config_request_size = num_tokens_from_string(GPT_SETTINGS[0], "gpt-4")
# --- tryb NONE: nie dotykamy pamięci i pozwalamy przekazać własny 'none_request' ---
if isinstance(none_request, list):
history_msgs = none_request
elif isinstance(none_request, str) and none_request.strip():
history_msgs = [{"role": "user", "content": none_request}]
else:
history_msgs = [{"role": "user", "content": f"{username}:{prompt}"}]
for slowo, reakcja in WORD_REACTIONS.items():
if not reakcja[3]:
content = (
"Kiedy słyszysz "
+ slowo
+ " to reagujesz lub dzieje się to "
+ reakcja[0]
)
temp = {"role": "system", "content": content}
chat_gpt_config_request_size += num_tokens_from_string(temp, "gpt-4")
history.append(temp)
logger.info("Rozmiar zapytania (tok): %s", "n/a") # tokeny już policzone wyżej
final_prompt = username + ":" + prompt
logger.debug(
"Rozmiar zapytania przed dodaniem historii %s", chat_gpt_config_request_size
)
if request_type == "MUSIC":
algorithm = "gpt-4o"
table = MESSAGE_TABLE_MUZYKA
token_amount = 10700
elif request_type == "RANDOM":
algorithm = "gpt-4o"
table = MESSAGE_TABLE
token_amount = 10700
else:
table = MESSAGE_TABLE
algorithm = "gpt-4o"
token_amount = 10700
prompt_gpt_request_size = num_tokens_from_string(
{"role": "user", "content": final_prompt}, "gpt-4"
)
temptable = []
for i in reversed(table):
temp_token = num_tokens_from_string(i, "gpt-4")
logger.debug(
"Rozmiar zapytania %s prompt %s temp %s",
chat_gpt_config_request_size,
prompt_gpt_request_size,
temp_token,
)
if (
chat_gpt_config_request_size
< token_amount + prompt_gpt_request_size + temp_token
):
temptable.insert(1, i)
chat_gpt_config_request_size += temp_token
history.extend(temptable)
temp = {"role": "user", "content": final_prompt}
history.append(temp)
logger.info("Rozmiar zapytania po wyslaniu %s", chat_gpt_config_request_size)
try:
response = await OPENAICLIENT.chat.completions.create(
model=algorithm, messages=history
# ...przygotowanie messages/system prompt/itp. jak masz...
# retry/backoff + deadline (zachowuje Twoją semantykę logowania)
timeout_sec = 120
deadline = time.time() + timeout_sec
response = await asyncio.wait_for(
openai_call(messages=history_msgs, model=model_to_use),
timeout=max(0.1, deadline - time.time()),
)
except openai.APITimeoutError as e:
# Handle timeout error, e.g. retry or log
result = f"*Kondziu patrzy na terminal, czeka, czeka, czeka,.... Jeszcze chwile czeka Przypierdala w niego pięścią....* Nie mogę się połączyć z Openai spróbuj od nowa. *Na ekranie pojawia się*: {e}"
response = f"*Kondziu patrzy na terminal, czeka, czeka, czeka,.... Jeszcze chwile czeka Przypierdala w niego pięścią....* Nie mogę się połączyć z Openai spróbuj od nowa. *Na ekranie pojawia się*: {e}"
except openai.APIConnectionError as e:
result = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Nie mogę się połączyć z Openai. *Na ekranie pojawia się*: {e}"
response = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Nie mogę się połączyć z Openai. *Na ekranie pojawia się*: {e}"
except openai.BadRequestError as e:
# Handle invalid request error, e.g. validate parameters or log
resp, _ = await handle_response(
f"Wytlumacz jakie sa zasady dotyczące treści które możesz generować używając Dalle. Wytłumacz błąd {e} prostym językiem. Przeproś za nadmierną cenzurę. Wytłumacz co mogło być nie tak w prompcie 'prompt'",
True,
True,
MESSAGE_TABLE,
username,
"RANDOM",
)
result = f"Sorki, cenzura: {resp}. Jak chcesz to są kanały na nudle #sexy-foteczky i #kanal-do-fapania *Na ekranie pojawia się: {e}"
if internal_retry:
resp = "Nie umiem tego teraz ładnie wytłumaczyć — OpenAI mnie zastrzeliło."
else:
resp, _ = await handle_response(
f"Wytlumacz jakie sa zasady dotyczące treści które możesz generować używając Dalle. Wytłumacz błąd {e} prostym językiem. Przeproś za nadmierną cenzurę. Wytłumacz co mogło być nie tak w prompcie 'prompt'",
True,
True,
MESSAGE_TABLE,
username,
"RANDOM",
)
response = f"Sorki, cenzura: {resp}. Jak chcesz to są kanały na nudle #sexy-foteczky i #kanal-do-fapania *Na ekranie pojawia się: {e}"
except openai.APIResponseValidationError as e:
# Handle invalid request error, e.g. validate parameters or log
resp, _ = await handle_response(
f"Wytlumacz jakie sa zasady dotyczące treści które możesz generować używając Dalle. Wytłumacz błąd {e} prostym językiem. Przeproś za nadmierną cenzurę. Wytłumacz co mogło być nie tak w prompcie 'prompt'",
True,
True,
MESSAGE_TABLE,
username,
"RANDOM",
)
result = f"Sorki, cenzura: {resp}. Jak chcesz to są kanały na nudle #sexy-foteczky i #kanal-do-fapania *Na ekranie pojawia się: {e}"
if internal_retry:
resp = "Nie umiem tego teraz ładnie wytłumaczyć — OpenAI mnie zastrzeliło."
else:
resp, _ = await handle_response(
f"Wytlumacz jakie sa zasady dotyczące treści które możesz generować używając Dalle. Wytłumacz błąd {e} prostym językiem. Przeproś za nadmierną cenzurę. Wytłumacz co mogło być nie tak w prompcie 'prompt'",
True,
True,
MESSAGE_TABLE,
username,
"RANDOM",
)
response = f"Sorki, cenzura: {resp}. Jak chcesz to są kanały na nudle #sexy-foteczky i #kanal-do-fapania *Na ekranie pojawia się: {e}"
except openai.AuthenticationError as e:
# Handle authentication error, e.g. check credentials or log
result = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Wołaj szefa - coś się z hasłem zjebało. *Na terminalu pojawia się:* {e}"
response = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Wołaj szefa - coś się z hasłem zjebało. *Na terminalu pojawia się:* {e}"
except openai.PermissionDeniedError as e:
# Handle permission error, e.g. check scope or log
result = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Wołaj szefa - coś się z uprawnieniami zjebało. *Na terminalu pojawia się:* {e}"
response = f"*Kondziu patrzy na terminal, chwile się zastanawia. Przypierdala w niego pięścią....* Wołaj szefa - coś się z uprawnieniami zjebało. *Na terminalu pojawia się:* {e}"
except openai.RateLimitError as e:
result = f"*Kondziu patrzy na terminal* Wołaj szefa. Zapłacić rachunki za AI trzeba. Jak chcesz to się na #zebranie dorzuć. {e}"
response = f"*Kondziu patrzy na terminal* Wołaj szefa. Zapłacić rachunki za AI trzeba. Jak chcesz to się na #zebranie dorzuć. {e}"
except openai.UnprocessableEntityError as e:
result = f"*Kondziu patrzy na terminal. Potem na to co każesz mu wysłać....* Ja wiem że jesteśmy w barze BDSM - ale nie da się włożyć TEGO w TO. *Za jego plecami na terminalu pojawia się:* {e}"
response = f"*Kondziu patrzy na terminal. Potem na to co każesz mu wysłać....* Ja wiem że jesteśmy w barze BDSM - ale nie da się włożyć TEGO w TO. *Za jego plecami na terminalu pojawia się:* {e}"
except openai.APIError as e:
# Handle API error, e.g. retry or log
result = f"*Kondziu nurkuje za bar, terminal wybucha. Przed tobą ląduje pergamin zapisany pięknym gotykiem a na nim*: {e}"
response = f"*Kondziu nurkuje za bar, terminal wybucha. Przed tobą ląduje pergamin zapisany pięknym gotykiem a na nim*: {e}"
logger.info("Historia wysłana:")
logger.info(history)
await asyncio.sleep(15)
result = ""
logger.debug("Odpowiedzi")
logger.info(response)
logger.debug(response.choices)
for choice in response.choices:
result += choice.message.content
logger.info("Sformatowane odpowiedzi")
logger.info(result)
temp = {"role": "assistant", "content": result}
history.append(temp)
temp_assistant = {"role": "assistant", "content": response}
logger.info(temp_assistant)
if request_type == "MUSIC":
with open(MEMORY_FIVE_MUZYKA, "r+", encoding=ENCODING) as file_music_memory:
# First we load existing data into a dict.
file_data = json.load(file_music_memory)
# Join new_data with file_data inside emp_details
file_data.append(temp)
file_music_memory.seek(0)
# convert back to json.
json.dump(file_data, file_music_memory, indent=4)
elif request_type == "RANDOM":
with open(MEMORY_FIVE_SIARA, "r+", encoding=ENCODING) as file_memory:
# First we load existing data into a dict.
file_data = json.load(file_memory)
# Join new_data with file_data inside emp_details
file_data.append(temp)
file_memory.seek(0)
# convert back to json.
json.dump(file_data, file_memory, indent=4)
else:
with open(MEMORY_FIVE_SIARA, "r+", encoding=ENCODING) as file_memory:
# First we load existing data into a dict.
file_data = json.load(file_memory)
# Join new_data with file_data inside emp_details
file_data.append(temp)
file_memory.seek(0)
# convert back to json.
json.dump(file_data, file_memory, indent=4)
return result, MESSAGE_TABLE
# zapis do pliku MUZYKA
with open(MEMORY_FIVE_MUZYKA, "r+", encoding=ENCODING) as fh:
file_data = json.load(fh)
file_data.append({"role": "user", "content": f"{username}:{prompt}"})
file_data.append(temp_assistant)
fh.seek(0)
json.dump(file_data, fh, indent=4)
return response, MESSAGE_TABLE_MUZYKA
elif request_type in ("RANDOM", "GENERAL"):
with open(MEMORY_FIVE_SIARA, "r+", encoding=ENCODING) as fh:
file_data = json.load(fh)
file_data.append({"role": "user", "content": f"{username}:{prompt}"})
file_data.append(temp_assistant)
fh.seek(0)
json.dump(file_data, fh, indent=4)
return response, MESSAGE_TABLE
else: # NONE
return response, []
async def get_random_cyclic_message(client):
@@ -395,7 +436,7 @@ async def chat_with_assistant(message, assistant_name):
logger.info(block.text.value)
chat_response += block.text.value
await discord_friendly_send(message.channel, chat_response)
#await message.channel.send(chat_response)
# await message.channel.send(chat_response)
done = True
elif run.status == "cancelled":
await discord_friendly_send(message.channel, "Cos sie wywaliło")
Regular → Executable
+261 -2514
View File
File diff suppressed because it is too large Load Diff
-46
View File
@@ -1,46 +0,0 @@
import discord
from discord.ext import commands
from typing import Optional
class Music(commands.Cog):
def __init__(self, bot):
self.bot = bot # This is so you can access Bot instance in your cog
# You must have this function for `bot.load_extension` to call
def setup(bot):
bot.add_cog(Music(bot))
@commands.hybrid_command(
name="przytul", description="Przytul kogoś - daj mention po komendzie :)"
)
async def przytul(ctx, arg: Optional[discord.Member] = None):
"""
Generate a text about hugging mentioned user.
:param ctx: ctx stands for "context" and is a required parameter in Discord.py commands. It
represents the context in which the command was invoked, including information such as the message,
the channel, the server, and the user who invoked the command
:param arg: arg is a parameter of the function "przytul" that expects a Discord member object. The
parameter is optional, meaning that if no member object is provided, it will default to None
:type arg: Optional[discord.Member]
"""
async with ctx.typing():
nieprzytulac = False
for mention in ctx.message.mentions:
for role in mention.roles:
if role.name == "NIEPRZYTULAĆ!":
nieprzytulac = True
if arg and nieprzytulac:
await ctx.send(
f"Żebym ja Ciebie nie przytulił {ctx.message.author.mention}"
)
elif arg:
await ctx.send(
# trunk-ignore(codespell/misspelled)
f"Już dobrze.... Już dobrze... Ojej.. Biedactwo... :( *W ułamku sekundy {arg.mention} znajduje sie w duszącym uścisku. Żebra trzeszczą - kilka pęka. Pacnięcia po plecach grożą odbiciem nerek, a głaskanie po głowie powoduje wstrząs mózgu*"
)
else:
await ctx.send(
"Kogo mam przytulić? *Wyłamuje kostki i przeciąga się - jego 200 kilowa sylwetka złożona z samych mięśni świadczy o tym że jest gotowy*"
)
+65 -25
View File
@@ -1,17 +1,20 @@
import json
import logging
import os
import re
import threading
import time
from queue import Empty, Queue
from typing import Optional
from urllib import request as urequest
from flask import Flask, jsonify, request
from flask import Flask, abort, jsonify, request
from waitress import serve
HOST_ADDRESS = "192.168.1.191"
PORT_ADDRESS = 5000
ICECAST_ADDRESS = "http://192.168.1.15:8000"
HOST_ADDRESS = os.getenv("CONJURER_DISCORD_HOST", "192.168.1.92")
PORT_ADDRESS = int(os.getenv("CONJURER_DISCORD_PORT", "5000"))
ICECAST_ADDRESS = os.getenv("CONJURER_ICECAST", "http://192.168.1.12:8000")
API_KEY = os.getenv("CONJURER_API_KEY")
OUT_COMM_Q = Queue()
IN_COMM_Q = Queue()
SRCHTITLE = re.compile(rb"StreamTitle=\\*(?P<title>[^;]*);").search
@@ -19,6 +22,12 @@ SRCHTITLE = re.compile(rb"StreamTitle=\\*(?P<title>[^;]*);").search
awaiting_q = []
incoming_q = Queue()
app = Flask(__name__)
def _authorize_request() -> None:
"""Reject inbound calls lacking the shared key (no-op if key is unset)."""
if API_KEY and request.headers.get("X-Conjurer-Api-Key") != API_KEY:
abort(401)
PREPPED_TRACKS = {
"requests": "",
"hit": "",
@@ -53,6 +62,7 @@ class QueryControl:
@app.route("/prepped_tracks", methods=["POST"])
def log_radio_tracks():
_authorize_request()
app.logger = logging.getLogger("discord")
app.logger.info(request)
@@ -85,6 +95,7 @@ def answer_external_command():
:return: The function `answer_external_command()` is returning a JSON response with the message
"SUCCESS".
"""
_authorize_request()
logger = logging.getLogger("discord")
logger.info(request)
record = json.loads(request.data)
@@ -129,34 +140,42 @@ def waitress_run():
serve(app, host=HOST_ADDRESS, port=PORT_ADDRESS)
def scan_queue():
def scan_queue(stop_event: Optional[threading.Event] = None):
"""
The function `scan_queue` reads data from a queue, logs it, and appends it to another queue.
:param _logger: The `_logger` parameter is typically an instance of a logging object that is used to
record and store log messages. It is commonly used to track the flow of the program, record errors,
and provide information for debugging purposes. In this code snippet, the `_logger` object is used
to log the
A bounded ``get(timeout=1)`` is used instead of a blocking ``get()`` so the
worker can observe ``stop_event`` and exit cleanly during shutdown.
:param stop_event: optional :class:`threading.Event`; when set the loop
stops at the next iteration.
"""
logger = logging.getLogger("discord")
while True:
data = OUT_COMM_Q.get()
if stop_event and stop_event.is_set():
logger.info("scan_queue: stop requested")
break
try:
data = OUT_COMM_Q.get(timeout=1)
except Empty:
continue
logger.info(data)
awaiting_q.append(data)
def scan_incoming():
def scan_incoming(stop_event: Optional[threading.Event] = None):
"""
The `scan_incoming` function continuously checks for incoming data, processes it, and logs when data
is found.
:param _logger: The `_logger` parameter in the `scan_incoming` function is a logger object that is
used to log messages or information during the execution of the function. It is typically used for
debugging, monitoring, or tracking the flow of the program. In this case, the `_logger` is being
used
:param stop_event: optional :class:`threading.Event`; when set the loop
stops at the next iteration.
"""
logger = logging.getLogger("discord")
while True:
if stop_event and stop_event.is_set():
logger.info("scan_incoming: stop requested")
break
try:
answer = incoming_q.get(block=False)
logger.info("DATA FOUND")
@@ -204,27 +223,48 @@ def id3(url: str) -> dict:
return tagdata
def comm_subroutine():
def comm_subroutine(stop_event: Optional[threading.Event] = None):
"""
The `comm_subroutine` function starts multiple threads to run different tasks concurrently.
:param logger: The `logger` parameter in the `comm_subroutine` function is an instance of a logger
object that is used for logging messages at various levels (e.g., debug, info, warning, error). In
the provided code snippet, the logger is used to log messages at the "info" level
Workers run as daemon threads and honour an optional ``stop_event`` so the
bot can shut the communication layer down cleanly instead of blocking
forever on ``join()``.
:param stop_event: optional :class:`threading.Event` shared with the caller
to coordinate a cooperative shutdown.
"""
# logger.setLevel(logging.DEBUG)
logger = logging.getLogger("discord")
logger.setLevel(logging.DEBUG)
logger.info("Started comms")
threads = []
# NOTE: flask_debug is the dev server bound to the SAME host:port as
# waitress - running both kills the comm layer with 'address in use'.
# Enable it only INSTEAD of waitress_run, never alongside.
# threads.append(threading.Thread(target=flask_debug))
threads.append(threading.Thread(target=waitress_run))
threads.append(threading.Thread(target=scan_queue))
threads.append(threading.Thread(target=scan_incoming))
threads.append(threading.Thread(target=waitress_run, daemon=True))
threads.append(
threading.Thread(
target=scan_queue, kwargs={"stop_event": stop_event}, daemon=True
)
)
threads.append(
threading.Thread(
target=scan_incoming, kwargs={"stop_event": stop_event}, daemon=True
)
)
for worker in threads:
worker.start()
for worker in threads:
worker.join()
try:
while any(thread.is_alive() for thread in threads):
if stop_event and stop_event.is_set():
break
time.sleep(0.5)
finally:
if stop_event:
stop_event.set()
if __name__ == "__main__":
+217
View File
@@ -0,0 +1,217 @@
# This Python file uses the following encoding: utf-8
"""Conan Exiles <-> Discord bridge (cog).
Follows the project convention (cog here, helpers in
``conanjurer_functions.py``). The integration is dormant unless configured in
:mod:`constants`: with no RCON host / channels the background watchers simply
do not start and the GM commands report that RCON is unavailable, so loading
this extension is always safe even on hosts without a Conan server.
"""
import asyncio
import logging
import discord
from discord.ext import commands
from conanjurer_functions import ConanConfig, Event, RconClient, watch, watch_players
from constants import (
CONAN_CHAT_CHANNEL_ID,
CONAN_EVENTS_CHANNEL_ID,
CONAN_GM_ROLE_ID,
CONAN_JOIN_CHANNEL_ID,
CONAN_LOG_MODE,
CONAN_LOG_PATH,
CONAN_PLAYER_POLL_SECONDS,
CONAN_RCON_HOST,
CONAN_RCON_PASSWORD,
CONAN_RCON_PORT,
CONAN_SFTP_HOST,
CONAN_SFTP_PASSWORD,
CONAN_SFTP_PORT,
CONAN_SFTP_USER,
)
def is_gm():
"""Allow only members holding the configured Conan GM role."""
async def predicate(ctx: commands.Context) -> bool:
if not CONAN_GM_ROLE_ID:
await ctx.reply(
"⛔ Rola GM Conana nie jest skonfigurowana.", mention_author=False
)
return False
ok = any(r.id == CONAN_GM_ROLE_ID for r in getattr(ctx.author, "roles", []))
if not ok:
await ctx.reply("⛔ Tylko GM.", mention_author=False)
return ok
return commands.check(predicate)
class ConanModule(commands.Cog):
"""Bridges a Conan Exiles server with Discord over RCON + log following."""
def __init__(self, bot, logger_name):
self.bot = bot
self.logger = logging.getLogger(logger_name)
self.cfg = ConanConfig(
rcon_host=CONAN_RCON_HOST,
rcon_port=CONAN_RCON_PORT,
rcon_password=CONAN_RCON_PASSWORD,
log_mode=CONAN_LOG_MODE,
log_path=CONAN_LOG_PATH,
sftp_host=CONAN_SFTP_HOST,
sftp_port=CONAN_SFTP_PORT,
sftp_user=CONAN_SFTP_USER,
sftp_password=CONAN_SFTP_PASSWORD,
)
self.rcon = (
RconClient(CONAN_RCON_HOST, CONAN_RCON_PORT, CONAN_RCON_PASSWORD)
if self.cfg.rcon_enabled
else None
)
self._log_task = None
self._player_task = None
async def cog_load(self):
# Conan -> Discord chat/event mirroring (only with a log source + target)
if self.cfg.log_enabled and (CONAN_CHAT_CHANNEL_ID or CONAN_EVENTS_CHANNEL_ID):
self._log_task = asyncio.create_task(self._run_log_watch())
else:
self.logger.info("Conan: log watch disabled (not configured)")
# Player-join notifications — disabled when the channel is not defined
if CONAN_JOIN_CHANNEL_ID and self.rcon is not None:
self._player_task = asyncio.create_task(self._run_player_watch())
else:
self.logger.info(
"Conan: player-join notifications disabled (no channel or RCON)"
)
async def cog_unload(self):
for task in (self._log_task, self._player_task):
if task is not None:
task.cancel()
if self.rcon is not None:
await self.rcon.close()
# ---------------------------------------------------------------- watchers
async def _run_log_watch(self):
await self.bot.wait_until_ready()
chat_ch = (
self.bot.get_channel(CONAN_CHAT_CHANNEL_ID) if CONAN_CHAT_CHANNEL_ID else None
)
evt_ch = (
self.bot.get_channel(CONAN_EVENTS_CHANNEL_ID)
if CONAN_EVENTS_CHANNEL_ID
else None
)
async def on_event(event: Event):
target = chat_ch if event.kind == "chat" else evt_ch
if target is not None:
await target.send(
event.text, allowed_mentions=discord.AllowedMentions.none()
)
self.logger.info("Conan: starting log watch (mode=%s)", self.cfg.log_mode)
await watch(self.cfg, on_event)
async def _run_player_watch(self):
"""Task 2: announce on a defined channel when a player joins the server."""
await self.bot.wait_until_ready()
channel = self.bot.get_channel(CONAN_JOIN_CHANNEL_ID)
if channel is None:
self.logger.warning(
"Conan: join channel %s not found — player notifications off",
CONAN_JOIN_CHANNEL_ID,
)
return
async def on_join(name: str):
await channel.send(
f"🟢 **{name}** wszedł na serwer Conan",
allowed_mentions=discord.AllowedMentions.none(),
)
self.logger.info(
"Conan: starting player-join watch (channel=%s, every %ss)",
CONAN_JOIN_CHANNEL_ID,
CONAN_PLAYER_POLL_SECONDS,
)
await watch_players(self.rcon, CONAN_PLAYER_POLL_SECONDS, on_join)
# ----------------------------------------------------- Discord -> Conan
async def _require_rcon(self, ctx: commands.Context):
if self.rcon is None:
await ctx.reply(
"⛔ RCON Conana nie jest skonfigurowany/dostępny.",
mention_author=False,
)
return None
return self.rcon
@commands.command(name="say")
@is_gm()
async def say(self, ctx: commands.Context, *, message: str):
"""Discord -> Conan: ogłoszenie widoczne dla wszystkich graczy w grze."""
rcon = await self._require_rcon(ctx)
if rcon is None:
return
resp = await rcon.command(f"broadcast {message}")
await ctx.reply(
f"✅ Wysłano do gry. (serwer: `{resp.strip() or 'OK'}`)",
mention_author=False,
)
@commands.command(name="players")
@is_gm()
async def players(self, ctx: commands.Context):
"""Lista graczy online (RCON listplayers)."""
rcon = await self._require_rcon(ctx)
if rcon is None:
return
resp = await rcon.command("listplayers")
await ctx.reply(
f"```\n{resp.strip() or 'brak danych'}\n```", mention_author=False
)
@commands.command(name="kick")
@is_gm()
async def kick(self, ctx: commands.Context, *, who: str):
"""Wyrzuć gracza (po nazwie/charname — zależnie od wersji serwera)."""
rcon = await self._require_rcon(ctx)
if rcon is None:
return
resp = await rcon.command(f"kick {who}")
await ctx.reply(f"👢 `{resp.strip() or 'OK'}`", mention_author=False)
@commands.command(name="rcon")
@is_gm()
async def raw_rcon(self, ctx: commands.Context, *, cmd: str):
"""Surowa komenda RCON (dla zaawansowanych GM). Używaj ostrożnie."""
rcon = await self._require_rcon(ctx)
if rcon is None:
return
resp = await rcon.command(cmd)
await ctx.reply(f"```\n{resp.strip() or 'OK'}\n```", mention_author=False)
@commands.command(name="ogłoś", aliases=["oglos", "rp"])
@is_gm()
async def rp_announce(self, ctx: commands.Context, nadawca: str, *, message: str):
"""Wiadomość RP 'z nadawcą' (np. ogłoszenie w imieniu Króla Khasara).
Na samym RCON realizujemy to jako sformatowany broadcast.
"""
rcon = await self._require_rcon(ctx)
if rcon is None:
return
resp = await rcon.command(f"broadcast [{nadawca}]: {message}")
await ctx.reply(f"📜 Ogłoszono jako **{nadawca}**.", mention_author=False)
async def setup(bot):
logger = logging.getLogger("discord")
await bot.add_cog(ConanModule(bot, "discord"))
logger.info("Loading conanjurer commands module done")
+266
View File
@@ -0,0 +1,266 @@
# This Python file uses the following encoding: utf-8
"""Helper logic for the Conan Exiles <-> Discord bridge.
Mirrors the project layout: the cog lives in ``conanjurer_commands.py`` and the
reusable logic lives here. Configuration comes from :mod:`constants` (env-var
overridable); optional third-party dependencies (``aiomcrcon``/``asyncssh``)
are imported defensively so the main bot can still load the extension when the
Conan integration is not installed or not in use.
"""
from __future__ import annotations
import asyncio
import logging
import os
import re
from dataclasses import dataclass
from typing import AsyncIterator, Awaitable, Callable, Optional, Set
try:
from aiomcrcon import Client as _Rcon # Source RCON over TCP
except ImportError: # pragma: no cover - optional component
_Rcon = None
try:
import asyncssh
except ImportError: # pragma: no cover - optional component
asyncssh = None
logger = logging.getLogger("discord")
@dataclass
class Event:
kind: str # "chat" | "login" | "logout" | "death" | "raw"
text: str # ready-to-display text
raw: str # original line (for debugging)
@dataclass
class ConanConfig:
"""Runtime configuration for the bridge, built from :mod:`constants`."""
rcon_host: str
rcon_port: int
rcon_password: str
log_mode: str # "local" | "sftp"
log_path: str
sftp_host: str
sftp_port: int
sftp_user: str
sftp_password: str
@property
def rcon_enabled(self) -> bool:
"""RCON usable only when host+password are set and the lib is present."""
return bool(self.rcon_host and self.rcon_password and _Rcon is not None)
@property
def log_enabled(self) -> bool:
"""Log following usable only when its prerequisites are configured."""
if not self.log_path:
return False
if self.log_mode == "sftp":
return bool(self.sftp_host and asyncssh is not None)
return True
# --- REGEXY DO DOSTROJENIA NA WŁASNYM LOGU ---
# Conan/Pippi/Tot logują różnie — dopasuj do swojego logu. Linie niepasujące są
# ignorowane (nie zgadujemy).
_PATTERNS: list[tuple[str, re.Pattern]] = [
("chat", re.compile(r"Chat:\s*(?P<who>.+?):\s*(?P<msg>.+)$", re.I)),
("login", re.compile(r"(?P<who>.+?)\s+(joined|connected|logged in)", re.I)),
("logout", re.compile(r"(?P<who>.+?)\s+(left|disconnected|logged out)", re.I)),
("death", re.compile(r"(?P<who>.+?)\s+was killed by\s+(?P<by>.+)$", re.I)),
]
class RconClient:
"""Thin async wrapper around a Source-RCON connection to the Conan server."""
def __init__(self, host: str, port: int, password: str):
self._host, self._port, self._pw = host, port, password
self._client: Optional["_Rcon"] = None
self._lock = asyncio.Lock()
async def _ensure(self) -> "_Rcon":
if _Rcon is None:
raise RuntimeError("aiomcrcon not installed — RCON unavailable")
if self._client is None:
client = _Rcon(self._host, self._port, self._pw)
await client.connect()
self._client = client
logger.info("RCON connected %s:%s", self._host, self._port)
return self._client
async def command(self, cmd: str) -> str:
"""Send a command to the Conan server and return its response.
Discord -> Conan channel, e.g. ``command("broadcast Hi!")``.
"""
async with self._lock:
for attempt in (1, 2):
try:
client = await self._ensure()
resp, _ = await client.send_cmd(cmd)
return resp
except Exception as exc: # disconnect / server restart
logger.warning("RCON error (attempt %s): %s", attempt, exc)
await self.close()
if attempt == 2:
raise
await asyncio.sleep(1.0)
return ""
async def close(self) -> None:
if self._client is not None:
try:
await self._client.close()
except Exception: # pragma: no cover - best effort
pass
self._client = None
def parse_line(line: str) -> Optional[Event]:
line = line.rstrip("\n")
if not line.strip():
return None
for kind, pat in _PATTERNS:
match = pat.search(line)
if match:
g = match.groupdict()
if kind == "chat":
return Event(kind, f"💬 **{g['who']}**: {g['msg']}", line)
if kind == "login":
return Event(kind, f"🟢 **{g['who']}** dołączył do gry", line)
if kind == "logout":
return Event(kind, f"⚪ **{g['who']}** opuścił grę", line)
if kind == "death":
return Event(kind, f"💀 **{g['who']}** zginął z ręki {g['by']}", line)
return None # nierozpoznane -> ignoruj
def parse_players(listplayers_output: str) -> Set[str]:
"""Extract the set of connected player char-names from RCON ``listplayers``.
Conan's table is roughly::
Idx | Char name | Player name | User ID | Platform ID | Platform Name
0 | Conan | SomeUser | 12345 | 765... | Steam
The char-name column (index 1) is used. The exact format varies between
server builds, so this is best-effort and intentionally tunable.
"""
players: Set[str] = set()
for raw in listplayers_output.splitlines():
line = raw.strip()
if not line or "|" not in line:
continue
cols = [c.strip() for c in line.split("|")]
head = cols[0].lower()
# skip the header row and any separator rows (e.g. "---|---")
if head in ("idx", "") or set(cols[0]) <= set("-"):
continue
if len(cols) >= 2 and cols[1]:
players.add(cols[1])
return players
async def watch_players(
rcon: RconClient,
interval: float,
on_join: Callable[[str], Awaitable[None]],
) -> None:
"""Poll RCON ``listplayers`` and call *on_join* for each new player.
The first poll seeds the known-player set without announcing, so restarting
the bot does not re-announce everyone already online. RCON failures are
logged and retried on the next tick rather than killing the task.
"""
known: Optional[Set[str]] = None
while True:
try:
response = await rcon.command("listplayers")
current = parse_players(response)
if known is None:
known = current
else:
for name in current - known:
try:
await on_join(name)
except Exception: # pragma: no cover - handler guard
logger.exception("Conan: on_join handler failed")
known = current
except Exception as exc:
logger.warning("Conan: player poll failed: %s", exc)
await asyncio.sleep(interval)
async def _follow_local(path: str) -> AsyncIterator[str]:
"""``tail -f`` in pure asyncio, following log rotation."""
while True:
try:
with open(path, "r", encoding="utf-8", errors="replace") as handle:
handle.seek(0, os.SEEK_END)
inode = os.fstat(handle.fileno()).st_ino
while True:
line = handle.readline()
if line:
yield line
continue
await asyncio.sleep(0.5)
try:
if os.stat(path).st_ino != inode: # rotation
break
except FileNotFoundError:
break
except FileNotFoundError:
logger.warning("Conan log not present yet: %s", path)
await asyncio.sleep(3.0)
async def _follow_sftp(cfg: ConanConfig) -> AsyncIterator[str]:
"""Incremental SFTP polling (e.g. Host Havoc): reads only new bytes."""
offset = 0
while True:
try:
async with asyncssh.connect(
cfg.sftp_host,
port=cfg.sftp_port,
username=cfg.sftp_user,
password=cfg.sftp_password,
known_hosts=None,
) as conn:
async with conn.start_sftp_client() as sftp:
while True:
try:
attrs = await sftp.stat(cfg.log_path)
size = attrs.size or 0
if size < offset: # rotation
offset = 0
if size > offset:
async with sftp.open(cfg.log_path, "r") as remote:
await remote.seek(offset)
chunk = await remote.read()
offset = size
for line in chunk.splitlines():
yield line
except FileNotFoundError:
logger.warning("SFTP: missing log %s", cfg.log_path)
await asyncio.sleep(2.0)
except Exception as exc:
logger.warning("SFTP disconnected: %s — retrying", exc)
await asyncio.sleep(5.0)
async def watch(cfg: ConanConfig, on_event: Callable[[Event], Awaitable[None]]) -> None:
"""Follow the Conan log and dispatch recognised lines to *on_event*."""
source = _follow_local(cfg.log_path) if cfg.log_mode != "sftp" else _follow_sftp(cfg)
async for line in source:
event = parse_line(line)
if event is not None:
try:
await on_event(event)
except Exception: # pragma: no cover - handler guard
logger.exception("Conan: event handler failed")
+14
View File
@@ -0,0 +1,14 @@
"""Pytest bootstrap: make first-party modules importable from the tests.
The bot modules live at the repository root and the musician service lives in
``conjurer_musician/``; neither is an installable package, so we put both on
``sys.path`` here.
"""
import os
import sys
_ROOT = os.path.dirname(os.path.abspath(__file__))
for _path in (_ROOT, os.path.join(_ROOT, "conjurer_musician")):
if _path not in sys.path:
sys.path.insert(0, _path)
+81 -31
View File
@@ -16,34 +16,52 @@ Functions:
import asyncio
import json
import logging
import netrc
import os
import threading
import time
from json.decoder import JSONDecodeError
from logging import handlers
from pathlib import Path
from queue import Queue
from typing import Dict, Optional
import requests
import scrape_bot
import search_bot
#import search_bot2 as search_bot
from flask import Flask, jsonify, request
# import search_bot2 as search_bot
from flask import Flask, jsonify, request, abort
from habanero import Crossref
from waitress import serve
try:
import netrc
except ImportError: # pragma: no cover
netrc = None
# Constants
NETRC_FILE = r"C:\Users\Activcom.pl\.netrc"
HOST_ADDRESS = "192.168.1.192"
PORT_ADDRESS = 5001
MAIN_BOT_ADDRESS = "http://192.168.1.191:5000"
SEND_RESULTS = "/conjurer"
BDSM_UUID_TEST = "96b7f85a-1142-4908-8986-62a2ea25a147"
MAX_CR_RESULTS = 500
#TEST PURPOSES ONLY!
#MAX_CR_RESULTS = 5
ENCODING = "utf-8"
def _env(name: str, default: str) -> str:
return os.getenv(name, default)
def _env_path(name: str, default: str) -> Path:
return Path(os.getenv(name, default)).expanduser().resolve()
BASE_DIR = Path(
os.getenv("CONJURER_LIBRARIAN_BASE", str(Path(__file__).resolve().parent))
)
NETRC_FILE = _env_path("CONJURER_NETRC_FILE", str(Path.home() / ".netrc"))
HOST_ADDRESS = _env("CONJURER_LIBRARIAN_HOST", "0.0.0.0")
PORT_ADDRESS = int(_env("CONJURER_LIBRARIAN_PORT", "5001"))
MAIN_BOT_ADDRESS = _env("CONJURER_MAIN_BOT", "http://127.0.0.1:5000")
SEND_RESULTS = _env("CONJURER_LIBRARIAN_RESULTS_ENDPOINT", "/conjurer")
MAX_CR_RESULTS = int(_env("CONJURER_LIBRARIAN_MAX_RESULTS", "500"))
ENCODING = _env("CONJURER_ENCODING", "utf-8")
API_KEY = os.getenv("CONJURER_API_KEY")
LOGFILE_PATH = _env_path(
"CONJURER_LIBRARIAN_LOG", str(BASE_DIR / "librarian.log")
)
app = Flask(__name__)
@@ -51,6 +69,17 @@ librarian_queue = Queue()
librarian_list = []
def _service_headers() -> Dict[str, str]:
if API_KEY:
return {"X-Conjurer-Api-Key": API_KEY}
return {}
def _authorize_request() -> None:
if API_KEY and request.headers.get("X-Conjurer-Api-Key") != API_KEY:
abort(401)
# trunk-ignore(pylint/R0902)
class Librarian(object):
"""
@@ -81,11 +110,24 @@ class Librarian(object):
- search_result_from_cr: A dictionary to store the search results from Crossref.
- done: A flag indicating if the search is done.
"""
netrc_mod = netrc.netrc(NETRC_FILE)
auth_tokens = netrc_mod.authenticators("crossref")
mailto_contact: Optional[str] = os.getenv("CONJURER_CROSSREF_MAILTO")
if netrc:
try:
netrc_mod = netrc.netrc(str(NETRC_FILE))
auth_tokens = netrc_mod.authenticators("crossref")
if auth_tokens:
mailto_contact = auth_tokens[0]
except (FileNotFoundError, netrc.NetrcParseError):
logging.getLogger("conjurer_librarian").warning(
"Crossref credentials missing in netrc %s", NETRC_FILE
)
if not mailto_contact:
raise RuntimeError(
"Crossref credentials not configured. Set CONJURER_CROSSREF_MAILTO or add to netrc."
)
self.cr = Crossref(
mailto=auth_tokens[0],
ua_string=f"Conjurer project. mailto:{auth_tokens[0]}"
mailto=mailto_contact,
ua_string=f"Conjurer project. mailto:{mailto_contact}"
)
self.query = query
self.uuid = str(uuid)
@@ -132,7 +174,7 @@ class Librarian(object):
self.fetched = len(cr_result["message"]["items"])
self.app.logger.info(self.total)
self.app.logger.info(self.fetched)
time.sleep(0.1)
await asyncio.sleep(0.1)
else:
cr_result = self.cr.works(query=query, cursor_max=15000, cursor='*', progress_bar = True)
@@ -411,18 +453,20 @@ class BackgroundTaskSearch(threading.Thread):
requests.post,
f"{MAIN_BOT_ADDRESS}{SEND_RESULTS}",
json=result,
headers=_service_headers(),
timeout=360,
)
self.app.logger.info("SENT")
result = await coroutine
self.app.logger.info(result.status_code)
self.app.logger.info("SEND CONFIRMED")
time.sleep(1)
await asyncio.sleep(1)
# ==================================SERVER ROUTES==========================================
@app.route("/query", methods=["POST"])
async def query_database():
_authorize_request()
"""
Endpoint for querying the database.
@@ -455,6 +499,7 @@ async def query_database():
@app.route("/get_partial_result", methods=["POST"])
async def get_partial():
_authorize_request()
"""
Retrieves the partial result for a given UUID.
@@ -478,9 +523,10 @@ async def get_partial():
# =======================================MAIN===================================================
if __name__ == "__main__":
app.logger.setLevel(logging.DEBUG)
LOGFILE_PATH.parent.mkdir(parents=True, exist_ok=True)
h1 = handlers.RotatingFileHandler(
filename="D:\\logs\\librarian.log",
encoding="utf-8",
filename=str(LOGFILE_PATH),
encoding=ENCODING,
mode="a",
maxBytes=6 * 1024 * 1024,
backupCount=6,
@@ -488,20 +534,24 @@ if __name__ == "__main__":
app.logger.addHandler(h1)
threads = []
threads.append(threading.Thread(target=waitress_run))
threads.append(threading.Thread(target=waitress_run, daemon=True))
# threads.append(threading.Thread(target=flask_debug))
bgtask = BackgroundTaskSearch()
bgtask.app = app
bgtask.daemon = True
threads.append(bgtask)
threads.append(threading.Thread(target=scrape_bot.scraper, args=(app.logger,)))
threads.append(
threading.Thread(
target=scrape_bot.scraper, args=(app.logger,), daemon=True
)
)
i = 0
for worker in threads:
try:
try:
for worker in threads:
app.logger.info("App number: %s", i)
i += 1
worker.start()
except RuntimeError as e:
app.logger.error("Exploded")
print(str(e))
for worker in threads:
worker.join()
for worker in threads:
worker.join()
except KeyboardInterrupt:
app.logger.info("Shutdown requested - exiting librarian service")
+4 -3
View File
@@ -4,6 +4,7 @@ This module contains the code for the scrape bot.
import json
import logging
import os
import random
import re
import time
@@ -15,9 +16,9 @@ from urllib.request import urlopen
from requests import ConnectionError as RequestsConnectionError
from requests import ConnectTimeout, Timeout
SCR_DATABASE_PATH = r"C:\\Database\\chunks\\"
SCR_FILENAME = "40_chunk.txt"
SCR_ENCODING = "utf-8"
SCR_DATABASE_PATH = os.getenv("CONJURER_LIBRARIAN_DB_PATH", r"C:\\Database\\chunks\\")
SCR_FILENAME = os.getenv("CONJURER_LIBRARIAN_SCRAPE_CHUNK", "40_chunk.txt")
SCR_ENCODING = os.getenv("CONJURER_ENCODING", "utf-8")
WORK_Q = Queue()
random.seed()
+8 -10
View File
@@ -19,22 +19,20 @@ Global Variables:
"""
# TODO: Wpiemdolić to wszystko w klasę z loggerem przysłanym z góry
import os
from queue import Empty, Queue
from threading import Thread
import time
q = Queue()
#TODO: Count number of lines in files and print to approximate on which part of the file search is
# DATA FOR TEST ONLY
# MAXTHREADS = 5
# DATABASE_PATH = r'C:\\Database\\chunks_1\\'
# Deployment data is environment-overridable so the local DOI database can live
# on a mounted volume (Docker/Linux) instead of the hardcoded Windows path.
MAXTHREADS = int(os.getenv("CONJURER_LIBRARIAN_MAXTHREADS", "41"))
DATABASE_PATH = os.getenv("CONJURER_LIBRARIAN_DB_PATH", r"C:\\Database\\chunks\\")
# DEPLOYMENT DATA
MAXTHREADS = 41
DATABASE_PATH = r"C:\\Database\\chunks\\"
ENCODING = "utf-8"
CHUNK = "_chunk.txt"
ENCODING = os.getenv("CONJURER_ENCODING", "utf-8")
CHUNK = os.getenv("CONJURER_LIBRARIAN_CHUNK", "_chunk.txt")
_sentinel = object()
WORK_Q_SIZE = 35500000
@@ -107,7 +105,7 @@ def consumer(in_q, control_q, doi, live_results, result_list, control_dict, no,
for item in result_list:
if item["DOI"] in data and not item["exists"]:
print(f"HIT in {no} content {data[0]} line {data[1]} file {data[2]} {item["exists"]}")
print(f"HIT in {no} content {data[0]} line {data[1]} file {data[2]} {item['exists']}")
_logger.info(data)
_logger.info("HIT")
item["exists"] = True
+1 -1
View File
@@ -108,7 +108,7 @@ def consumer(in_q, control_q, doi, live_results, result_list, control_dict, no,
print(f"C{no}{alive_no}\r", end="")
for item in result_list:
if item["DOI"] in data[0] and not item["exists"]:
print(f"HIT in {no} content {data[0]} line {data[1]} file {data[2]} {item["exists"]}")
print(f"HIT in {no} content {data[0]} line {data[1]} file {data[2]} {item['exists']}")
item["exists"] = True
live_results.append(item)
done_check = done_check and item["exists"]
+7
View File
@@ -0,0 +1,7 @@
# Runtime-generated radio data — keep out of the repo
all_playlist.playlist
hit.playlist
request.playlist
priority_queue.playlist
prio_playlist.json
*.mp3
+194 -119
View File
@@ -11,17 +11,15 @@ import random
import re
import threading
import time
# from flask_autoindex import AutoIndex
from datetime import datetime
from logging import handlers
from pathlib import Path
from platform import uname
from sys import platform
from typing import Dict, List
import requests
from flask import (
Flask,
abort,
jsonify,
redirect,
render_template,
@@ -30,34 +28,98 @@ from flask import (
)
from waitress import serve
MAIN_BOT_ADDRESS = "http://192.168.1.191:5000"
MUSIC_TRACKER = "/prepped_tracks"
HOST_ADDRESS = "192.168.1.15"
PORT_ADDRESS = 5000
if platform in ("linux", "linux2"):
SEPARATOR_FILE_PATH = "/"
if "microsoft-standard" in uname().release:
LOGFILE = "/home/mtuszowski/conjurer/discord_mus_service.log"
MUSIC_FOLDER = "/mnt/g/Muzyka/"
NETRC_FILE = "/home/mtuszowski/.netrc"
LOGSTORE = "/home/mtuszowski/conjurer/logs/"
ENCODING = "utf-8"
RADIOLOG_PATH = "/home/pi/Conjurer/radio_log.log"
PERSISTENCE_PATH = "/home/pi/Conjurer/persistence.log"
import media_search_functions
else:
LOGFILE = "/home/pi/Conjurer/discord_mus_service.log"
NETRC_FILE = "/home/pi/.netrc"
LOGSTORE = "/home/pi/MediaShare/logs/"
ENCODING = "utf-8"
MUSIC_FOLDER = "/home/pi/MediaShare/mp3/"
PRIORITY_FOLDER = "/home/pi/MediaShare/mp3/Magiczne i chuj/"
RADIOLOG_PATH = "/home/pi/Conjurer/radio_log.log"
PERSISTENCE_PATH = "/home/pi/Conjurer/persistence.log"
def _env(name: str, default: str) -> str:
return os.getenv(name, default)
def _env_path(name: str, default: str) -> Path:
value = os.getenv(name, default)
return Path(value).expanduser().resolve()
API_KEY = os.getenv("CONJURER_API_KEY")
MAIN_BOT_ADDRESS = _env("CONJURER_MAIN_BOT", "http://127.0.0.1:5000")
MUSIC_TRACKER = _env("CONJURER_MUSIC_TRACKER_ENDPOINT", "/prepped_tracks")
HOST_ADDRESS = _env("CONJURER_MUSICIAN_HOST", "0.0.0.0")
PORT_ADDRESS = int(_env("CONJURER_MUSICIAN_PORT", "5000"))
BASE_DIR = Path(
os.getenv("CONJURER_MUSICIAN_BASE", str(Path(__file__).resolve().parent))
)
LOGFILE = _env_path(
"CONJURER_MUSICIAN_LOG", str(BASE_DIR / "discord_mus_service.log")
)
LOGSTORE = _env_path("CONJURER_LOGSTORE", str(BASE_DIR / "logs"))
MUSIC_FOLDER = _env_path(
"CONJURER_MUSIC_FOLDER", str(BASE_DIR / "music")
)
PRIORITY_FOLDER = _env_path(
"CONJURER_PRIORITY_FOLDER", str(MUSIC_FOLDER / "priority")
)
RADIOLOG_PATH = _env_path(
"CONJURER_RADIO_LOG", str(BASE_DIR / "radio_log.log")
)
PERSISTENCE_PATH = _env_path(
"CONJURER_PERSISTENCE_LOG", str(BASE_DIR / "persistence.log")
)
ALL_PLAYLIST_PATH = _env_path(
"CONJURER_ALL_PLAYLIST", str(BASE_DIR / "all_playlist.playlist")
)
HIT_PLAYLIST_PATH = _env_path(
"CONJURER_HIT_PLAYLIST", str(BASE_DIR / "hit.playlist")
)
REQUEST_PLAYLIST_PATH = _env_path(
"CONJURER_REQUEST_PLAYLIST", str(BASE_DIR / "request.playlist")
)
PRIORITY_PLAYLIST_PATH = _env_path(
"CONJURER_PRIORITY_PLAYLIST", str(BASE_DIR / "priority_queue.playlist")
)
STREAM_TEMPLATE = _env_path(
"CONJURER_STREAM_TEMPLATE", str(BASE_DIR / "stream.html")
)
ENCODING = _env("CONJURER_ENCODING", "utf-8")
SEPARATOR_FILE_PATH = os.sep
for playlist_path in (
ALL_PLAYLIST_PATH,
HIT_PLAYLIST_PATH,
REQUEST_PLAYLIST_PATH,
PRIORITY_PLAYLIST_PATH,
):
playlist_path.parent.mkdir(parents=True, exist_ok=True)
random.seed()
music_file_list = []
priority_list = []
music_file_list: List[str] = []
priority_list: List[str] = []
def _build_headers() -> Dict[str, str]:
headers: Dict[str, str] = {}
if API_KEY:
headers["X-Conjurer-Api-Key"] = API_KEY
return headers
def _authorize_request() -> None:
if API_KEY and request.headers.get("X-Conjurer-Api-Key") != API_KEY:
abort(401)
def _post_to_bot(payload: List[str]) -> None:
response = requests.post(
f"{MAIN_BOT_ADDRESS}{MUSIC_TRACKER}",
json=payload,
headers=_build_headers(),
timeout=60,
)
logger = logging.getLogger("conjurer_musician")
logger.info("SENT")
logger.info(response.status_code)
logger.info("SEND CONFIRMED")
def rescan():
@@ -68,28 +130,29 @@ def rescan():
logger = logging.getLogger("conjurer_musician")
logger.info("Rescan triggered")
for mp3_item in Path.glob(Path(MUSIC_FOLDER), "**/*.mp3"):
music_file_list.clear()
priority_list.clear()
for mp3_item in MUSIC_FOLDER.glob("**/*.mp3"):
temp_music_file = mp3_item.as_posix()
if platform == "win32":
if os.name == "nt":
temp_music_file = temp_music_file.replace("/", "\\")
music_file_list.append(temp_music_file)
for mp3_item in Path.glob(Path(PRIORITY_FOLDER), "**/*.mp3"):
for mp3_item in PRIORITY_FOLDER.glob("**/*.mp3"):
temp_music_file = mp3_item.as_posix()
if platform == "win32":
if os.name == "nt":
temp_music_file = temp_music_file.replace("/", "\\")
priority_list.append(temp_music_file)
with open(
"/home/pi/Conjurer/all_playlist.playlist", "w", encoding="utf-8"
) as w_file:
with ALL_PLAYLIST_PATH.open("w", encoding=ENCODING) as w_file:
try:
for item in music_file_list:
w_file.write(item)
w_file.write("\n")
except json.JSONDecodeError:
pass
with open("/home/pi/Conjurer/hit.playlist", "w", encoding="utf-8") as w_file:
with HIT_PLAYLIST_PATH.open("w", encoding=ENCODING) as w_file:
try:
for item in priority_list:
w_file.write(item)
@@ -114,72 +177,50 @@ def thread_rescan():
def scan_tracks():
# Set the filename and open the file
logger = logging.getLogger("conjurer_musician")
with open(RADIOLOG_PATH, "r", encoding=ENCODING) as log_file:
log_file.seek(os.stat(RADIOLOG_PATH).st_size)
prev_size = os.stat(PERSISTENCE_PATH).st_size
file = open(RADIOLOG_PATH, "r")
# Find the size of the file and move to the end
st_results = os.stat(RADIOLOG_PATH)
st_size = st_results[6]
file.seek(st_size)
st_results1 = os.stat(PERSISTENCE_PATH)
prev_st_size1 = st_results[6]
while True:
current_size = os.stat(PERSISTENCE_PATH).st_size
if prev_size != current_size:
while prev_size != current_size:
prev_size = current_size
time.sleep(0.1)
current_size = os.stat(PERSISTENCE_PATH).st_size
with open(PERSISTENCE_PATH, "r", encoding=ENCODING) as persistence:
lines = persistence.readlines()
if len(lines) >= 3:
_post_to_bot(["next", lines[2]])
while 1:
position = log_file.tell()
line = log_file.readline()
if not line:
time.sleep(1)
log_file.seek(position)
continue
st_results1 = os.stat(PERSISTENCE_PATH)
st_size1 = st_results1[6]
if prev_st_size1 != st_size1:
while prev_st_size1 != st_size1:
prev_st_size1 = st_size1
st_results1 = os.stat(PERSISTENCE_PATH)
st_size1 = st_results1[6]
if not re.match(r".*Prepared.*", line):
time.sleep(0.1)
file1 = open(PERSISTENCE_PATH, "r")
lines = file1.readlines()
result = ["next", lines[2]]
file1.close()
returned = requests.post(
f"{MAIN_BOT_ADDRESS}{MUSIC_TRACKER}", json=result, timeout=360
)
logger.info("SENT")
logger.info(returned.status_code)
logger.info("SEND CONFIRMED")
continue
where = file.tell()
line = file.readline()
if not line:
time.sleep(1)
file.seek(where)
else:
if re.match(".*Prepared.*", line):
result = None
if re.match(".*jingles.*", line):
logger.info("jingles")
logger.info(line) # already has newline
result = ["jingles", line]
elif re.match(".*priority.*", line):
logger.info("priority")
logger.info(line) # already has newline
result = ["priority", line]
elif re.match(".*hit.*", line):
logger.info("hit")
logger.info(line) # already has newline
result = ["hit", line]
elif re.match(".*all_playlist.*", line):
logger.info("all")
logger.info(line) # already has newline
result = ["all", line]
elif re.match(".*request.*", line):
logger.info("requests")
logger.info(line) # already has newline
result = ["requests", line]
if result:
returned = requests.post(
f"{MAIN_BOT_ADDRESS}{MUSIC_TRACKER}", json=result, timeout=360
)
logger.info("SENT")
logger.info(returned.status_code)
logger.info("SEND CONFIRMED")
time.sleep(0.1)
result = None
if re.match(r".*jingles.*", line):
result = ["jingles", line]
elif re.match(r".*priority.*", line):
result = ["priority", line]
elif re.match(r".*hit.*", line):
result = ["hit", line]
elif re.match(r".*all_playlist.*", line):
result = ["all", line]
elif re.match(r".*request.*", line):
result = ["requests", line]
if result:
logger.info("Forwarding radio log entry: %s", result[0])
_post_to_bot(result)
time.sleep(0.1)
app = Flask(__name__)
@@ -288,12 +329,8 @@ def wyszukaj(word_list, how_many, _logger=None, return_to_bot=True):
if search_weight[itr][0] == item_to_search:
return_list.append(search_weight[itr])
if not return_to_bot:
with open(
"/home/pi/Conjurer/priority_queue.playlist",
"r+",
encoding="utf-8",
) as s_file:
s_file.write(search_weight[itr][1])
with PRIORITY_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
s_file.write(search_weight[itr][1] + "\n")
break
itr += 1
else:
@@ -332,6 +369,38 @@ def remove_characters(string, character):
return string.replace(character, "")
@app.route('/get_share_list', methods=['POST'])
def get_share_list():
_authorize_request()
data = request.get_json()
entries = data.get('entries')
keywords = data.get('keywords')
# Validate entries
if not isinstance(entries, int) or not (1 <= entries <= 10):
return jsonify({'error': '"entries" must be an integer between 1 and 10.'}), 400
# Validate keywords list
if not isinstance(keywords, list) or not all(isinstance(k, str) for k in keywords):
return jsonify({'error': '"keywords" must be a list of strings.'}), 400
# Call external search
files = media_search_functions.find_matches(entries, keywords)
# Return each file path as a JSON list
return jsonify({'files': files}), 200
@app.route('/get_share_links', methods=['POST'])
def get_share_links():
_authorize_request()
data = request.get_json()
file_paths = data.get('file_paths')
# Validate file_paths list
if not isinstance(file_paths, list) or not all(isinstance(p, str) for p in file_paths):
return jsonify({'error': '"file_paths" must be a list of strings.'}), 400
# Call external publish
links = media_search_functions.publish(file_paths)
# Return list of published links
return jsonify({'links': links}), 200
@app.route("/stream", methods=["GET"])
def stream_music():
"""
@@ -342,7 +411,7 @@ def stream_music():
"""
# return send_from_directory("/tmp/hls", "stream.m3u8")
return render_template("/home/pi/Conjurer/stream.html")
return render_template(str(STREAM_TEMPLATE))
@app.route("/<string:file_name>")
@@ -374,15 +443,14 @@ def stream_music_mp3():
@app.route("/clear_pr_pls", methods=["GET"])
def clear_pr_pls():
_authorize_request()
"""
The function `clear_pr_pls` clears the contents of the priority queue playlist file.
:return: A JSON response indicating the success of the operation.
"""
app.logger.info("CLEARING PLAYLIST")
with open(
"/home/pi/Conjurer/priority_queue.playlist", "w", encoding="utf-8"
) as cleared_pl:
with PRIORITY_PLAYLIST_PATH.open("w", encoding=ENCODING) as cleared_pl:
cleared_pl.write("")
return_data = jsonify(isError=False, message="Success", statusCode=200, data=[])
@@ -410,6 +478,7 @@ def update_music_list():
received and added to the `music_file_list`.
The status code returned is 200, indicating a successful response.
"""
_authorize_request()
record = json.loads(request.data)
app.logger.info(record["item"])
music_file_list.append(record["item"])
@@ -431,6 +500,7 @@ def look_for_playlist():
data that was received and added to the `music_file_list`. The status code returned is 200,
indicating a successful response.
"""
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
@@ -449,13 +519,14 @@ def look_for_playlist():
@app.route("/request_radio_file", methods=["POST"])
def add_request():
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
app.logger.info(record["UUID"])
return_data = wyszukaj(record["lista_slow"], 0, app.logger, False)
with open("/home/pi/Conjurer/request.playlist", "a", encoding="utf-8") as s_file:
with REQUEST_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return_data = (
@@ -480,6 +551,7 @@ def create_priority_playlist():
data that was received and added to the `music_file_list`.
The status code returned is 200,indicating a successful response.
"""
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
@@ -489,7 +561,7 @@ def create_priority_playlist():
record["lista_slow"], record["dlugosc_plejlisty"], app.logger, False
)
random.shuffle(return_data)
with open("/home/pi/Conjurer/request.playlist", "a", encoding="utf-8") as s_file:
with REQUEST_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return_data = (
@@ -514,6 +586,7 @@ def add_to_priority():
data that was received and added to the `music_file_list`.
The status code returned is 200,indicating a successful response.
"""
_authorize_request()
record = json.loads(request.data)
app.logger.info(record)
app.logger.info(record["lista_slow"])
@@ -522,9 +595,7 @@ def add_to_priority():
return_data = wyszukaj(
record["lista_slow"], record["dlugosc_plejlisty"], app.logger, False
)
with open(
"/home/pi/Conjurer/priority_queue.playlist", "a", encoding="utf-8"
) as s_file:
with PRIORITY_PLAYLIST_PATH.open("a", encoding=ENCODING) as s_file:
for item in return_data:
s_file.write(item[1] + "\n")
return_data = (
@@ -583,15 +654,19 @@ if __name__ == "__main__":
logger.info("Started")
threads = []
# threads.append(threading.Thread(target=flask_debug))
threads.append(threading.Thread(target=waitress_run))
threads.append(threading.Thread(target=thread_rescan))
threads.append(threading.Thread(target=waitress_run, daemon=True))
threads.append(threading.Thread(target=thread_rescan, daemon=True))
for worker in threads:
worker.start()
time.sleep(60)
threads.append(threading.Thread(target=scan_tracks))
threads[2].start()
track_thread = threading.Thread(target=scan_tracks, daemon=True)
track_thread.start()
for worker in threads:
worker.join()
try:
for worker in threads:
worker.join()
track_thread.join()
except KeyboardInterrupt:
logger.info("Shutdown requested - exiting musician service")
@@ -0,0 +1,74 @@
#!/usr/bin/env python3
"""Share-list search/publish helpers for the musician service.
Paths are environment-overridable and the share database / directory are
accessed lazily, so importing this module has no side effects (the previous
version ran ``SHARE_DIR.mkdir()`` and read the JSON DB at import time, which
crashed on any host without the Pi's ``/var/www`` / ``/var/log`` layout — and
made the service untestable).
"""
import json
import os
import uuid
from pathlib import Path
# CONFIGURATION (env-overridable)
JSON_DB = os.getenv("CONJURER_SHARE_DB", "/var/log/share_scan.json")
SHARE_DIR = Path(os.getenv("CONJURER_SHARE_DIR", "/var/www/html/share"))
BASE_URL = os.getenv("CONJURER_SHARE_BASE_URL", "https://czernobog.pl/share")
_entries_cache = None
def _ensure_share_dir():
SHARE_DIR.mkdir(parents=True, exist_ok=True)
def load_db():
"""Load share entries, returning [] when the DB is missing/corrupt."""
try:
with open(JSON_DB) as handle:
return json.load(handle).get("entries", [])
except (FileNotFoundError, json.JSONDecodeError):
return []
def _entries():
global _entries_cache
if _entries_cache is None:
_entries_cache = load_db()
return _entries_cache
def relevancy(path, keywords):
score = 0
low = path.lower()
for kw in keywords:
if kw.lower() in low:
score += low.count(kw.lower())
return score
def find_matches(count, keywords):
scored = []
for entry in _entries():
score = relevancy(entry["path"], keywords)
if score > 0:
scored.append((score, entry["path"]))
scored.sort(reverse=True, key=lambda x: x[0])
result = [p for _, p in scored]
return result[:count]
def publish(paths):
_ensure_share_dir()
urls = []
for path in paths:
token = uuid.uuid4().hex
link = SHARE_DIR / token
try:
os.symlink(path, link)
except FileExistsError:
pass
urls.append(f"{BASE_URL}/{token}")
return urls
@@ -0,0 +1,9 @@
/home/pi/Conjurer/radio_log.log {
daily
rotate 7
compress
delaycompress
missingok
notifempty
copytruncate
}
+21 -6
View File
@@ -43,6 +43,22 @@ def queue_processing()
end
end
# Configure output formats and destinations
def now_playing(m)
# raw lookup ("" if missing)
raw_title = m["title"]
raw_artist = m["artist"]
# default if empty
title = if raw_title == "" then "Unknown" else raw_title end
artist = if raw_artist == "" then "Unknown" else raw_artist end
log.critical("Now playing #{artist} - #{title}")
# write it out
system("echo '#{artist} - #{title}' > /tmp/now_playing.txt")
m
end
# Randomly select a playlist with weights
s4 = random(id="randomizer", weights=[2, 3, 5], [s1, s2, s3])
@@ -52,7 +68,8 @@ jingles = (playlist(reload_mode="watch", "/home/pi/Conjurer/jingles.playlist"))
# Create the main stream with random playlist and jingles
s = rotate(id="randomizer", weights=[10, 1], [s4, jingles])
s = crossfade(fade_out=2., fade_in=2., duration=4., fallback(id="switcher", track_sensitive=false, [requests_queue, s]))
s = crossfade(fade_out=2., fade_in=2., duration=4., fallback(id="switcher", track_sensitive=true, [requests_queue, s]))
s = metadata.map(now_playing, s)
# Set up an interactive harbor for controlling the stream
interactive.harbor(port = 9999)
@@ -87,7 +104,7 @@ s = blank.skip(max_blank=10., s)
live_enabled = interactive.bool("Going Live!", true)
s = add([mic,s])
s=switch(track_sensitive=false,
s=switch(track_sensitive=true,
[(live_enabled, mic),
({true}, s)])
@@ -102,9 +119,8 @@ set("log.level", loglevel)
# Enable logging to file
set("log.file", log_to_file)
set("log.file.path", logpath)
# Set up emergency fallback track
emergency = single("/home/pi/RetroPie/mp3/Youtube/Dr. Peacock - Trip to Ireland [GvrvQTUbUcA].mp3")
emergency = single("/home/pi/MediaFolder/mp3/Youtube/Dr. Peacock - Trip to Ireland [GvrvQTUbUcA].mp3")
radio = fallback(id="switcher2", track_sensitive=false, [s, emergency])
# Set up an interactive control for skipping tracks
@@ -138,8 +154,7 @@ interactive.persistent("script.params")
# Configure output formats and destinations
output.icecast(%mp3, host="retropie", port=8000, password=credentials.password, icy_metadata="true", mount="mp3-stream", radio)
output.icecast(%mp3, host="radio", port=8000, password=credentials.password, icy_metadata="true", mount="mp3-stream", radio)
output.pulseaudio(radio)
#output.file.hls("/tmp/hls", [("mp3-low", %mp3(bitrate=96)), ("mp3-hi", %mp3(bitrate=160))], radio)
# Uncomment the following lines to enable additional output formats
+99
View File
@@ -0,0 +1,99 @@
#!/usr/bin/env python3
import os
import re
import time
import json
import argparse
import sys
from pathlib import Path
# — CONFIGURATION — adjust as needed —
APACHE_LOG = '/var/log/apache2/share_access.log' # <- point to your share-specific log!
SHARE_DIR = Path('/var/www/html/share')
METADATA_FILE = SHARE_DIR / '.downloads.json'
STATE_FILE = SHARE_DIR / '.logpos'
# Regex to match lines like: "GET /share/abcdef1234... HTTP/1.1"
TOKEN_REGEX = re.compile(r'"\s*GET\s+/share/([0-9a-f]{32})\s+HTTP/')
def debug(msg, args):
if args.debug:
print(f"[DEBUG] {msg}", file=sys.stderr)
def load_metadata(args):
if METADATA_FILE.exists():
data = json.loads(METADATA_FILE.read_text())
debug(f"Loaded metadata: {data}", args)
return data
return {}
def save_metadata(md, args):
METADATA_FILE.write_text(json.dumps(md))
debug(f"Saved metadata: {md}", args)
def load_state(args):
if STATE_FILE.exists():
pos = int(STATE_FILE.read_text())
debug(f"Loaded last log position: {pos}", args)
return pos
return 0
def save_state(pos, args):
STATE_FILE.write_text(str(pos))
debug(f"Saved new log position: {pos}", args)
def scan_log(since_pos, args):
try:
with open(APACHE_LOG, 'r') as f:
f.seek(since_pos)
lines = f.readlines()
newpos = f.tell()
debug(f"Read {len(lines)} new lines", args)
return lines, newpos
except Exception as e:
print(f"Error reading log: {e}", file=sys.stderr)
return [], since_pos
def record_downloads(lines, metadata, args):
now = time.time()
for line in lines:
m = TOKEN_REGEX.search(line)
if m:
tok = m.group(1)
if tok not in metadata:
metadata[tok] = now
debug(f"Recorded download of token {tok} at {now}", args)
return metadata
def revoke_old(metadata, args):
now = time.time()
to_remove = [tok for tok, ts in metadata.items() if now - ts >= 3600]
for tok in to_remove:
link = SHARE_DIR / tok
try:
link.unlink()
debug(f"Unlinked token {tok}", args)
except FileNotFoundError:
debug(f"Link {link} not found when revoking", args)
metadata.pop(tok, None)
return bool(to_remove)
def main():
p = argparse.ArgumentParser(description="Revoke share links 2min after download")
p.add_argument('--debug', action='store_true', help='print debug info to stderr')
args = p.parse_args()
SHARE_DIR.mkdir(parents=True, exist_ok=True)
metadata = load_metadata(args)
state = load_state(args)
lines, newpos = scan_log(state, args)
metadata = record_downloads(lines, metadata, args)
save_metadata(metadata, args) # always persist new downloads
if revoke_old(metadata, args):
save_metadata(metadata, args) # persist removals
save_state(newpos, args)
if __name__ == '__main__':
main()
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env python3
import os
import json
import time
from pathlib import Path
# Configuration
SCAN_PATH = '/mnt/shares'
OUTPUT_FILE = '/var/log/share_scan.json'
def scan_directory(root):
"""Recursively walk `root` and collect metadata."""
entries = []
for dirpath, dirs, files in os.walk(root):
for name in dirs + files:
full = os.path.join(dirpath, name)
try:
stat = os.stat(full)
entries.append({
'path': full,
'is_dir': os.path.isdir(full),
'size_bytes': stat.st_size,
'mtime': time.strftime('%Y-%m-%dT%H:%M:%S%z',
time.localtime(stat.st_mtime)),
})
except Exception as e:
# skip items we can't stat
continue
return entries
def main():
data = {
'scanned_at': time.strftime('%Y-%m-%dT%H:%M:%S%z', time.localtime()),
'root': SCAN_PATH,
'entries': scan_directory(SCAN_PATH),
}
# Write atomically
temp = OUTPUT_FILE + '.tmp'
with open(temp, 'w') as f:
json.dump(data, f, indent=2)
os.replace(temp, OUTPUT_FILE)
if __name__ == '__main__':
main()
+6
View File
@@ -0,0 +1,6 @@
[Unit]
Description=Oneshot scan of /mnt/shares → JSON
[Service]
Type=oneshot
ExecStart=/usr/local/bin/scan_shares.py
+11
View File
@@ -0,0 +1,11 @@
[Unit]
Description=Hourly timer for sharescanner
[Timer]
# delay 2min after boot, then every hour
OnBootSec=20min
OnUnitActiveSec=24h
Unit=scan_shares.service
[Install]
WantedBy=timers.target
+195
View File
@@ -0,0 +1,195 @@
{
"string": [],
"bool": [ [ "Skip track", false ], [ "Going Live!", false ] ],
"int": [],
"float": [
[
"mic_volume",
0.0
],
[
"compress_gain6",
0.1
],
[
"compress_ratio6",
7.9
],
[
"compress_threshold6",
-15.8
],
[
"compress_release6",
30.0
],
[
"compress_attack6",
30.0
],
[
"compress_frequency6",
18610.0
],
[
"compress_gain5",
5.4
],
[
"compress_ratio5",
4.8
],
[
"compress_threshold5",
-13.6
],
[
"compress_release5",
120.0
],
[
"compress_attack5",
160.0
],
[
"compress_frequency5",
13950.0
],
[
"compress_gain4",
6.5
],
[
"compress_ratio4",
4.0
],
[
"compress_threshold4",
-12.2
],
[
"compress_release4",
130.0
],
[
"compress_attack4",
200.0
],
[
"compress_frequency4",
3890.0
],
[
"compress_gain3",
5.5
],
[
"compress_ratio3",
3.5
],
[
"compress_threshold3",
-11.7
],
[
"compress_release3",
140.0
],
[
"compress_attack3",
250.0
],
[
"compress_frequency3",
2270.0
],
[
"compress_gain2",
3.3
],
[
"compress_ratio2",
3.9
],
[
"compress_threshold2",
-11.5
],
[
"compress_release2",
120.0
],
[
"compress_attack2",
250.0
],
[
"compress_frequency2",
1920.0
],
[
"compress_gain1",
5.6
],
[
"compress_ratio1",
3.4
],
[
"compress_threshold1",
-12.2
],
[
"compress_release1",
120.0
],
[
"compress_attack1",
220.0
],
[
"compress_frequency1",
650.0
],
[
"compress_gain0",
7.5
],
[
"compress_ratio0",
3.2
],
[
"compress_threshold0",
-13.1
],
[
"compress_release0",
110.0
],
[
"compress_attack0",
140.0
],
[
"compress_frequency0",
110.0
],
[
"compress_wet",
1.0
],
[
"main_volume",
1.8
],
[
"g",
1.0
],
[
"f",
106.4
]
]
}
+93
View File
@@ -0,0 +1,93 @@
#!/usr/bin/env python3
import argparse
import json
import os
import sys
import uuid
from pathlib import Path
# CONFIGURATION
JSON_DB = '/var/log/share_scan.json'
SHARE_DIR = Path('/var/www/html/share')
BASE_URL = 'https://czernobog.pl/share'
# Ensure share directory exists
SHARE_DIR.mkdir(parents=True, exist_ok=True)
def load_db():
with open(JSON_DB) as f:
return json.load(f)['entries']
def relevancy(path, keywords):
score = 0
low = path.lower()
for kw in keywords:
if kw.lower() in low:
score += low.count(kw.lower())
return score
def find_matches(entries, keywords):
scored = []
for e in entries:
score = relevancy(e['path'], keywords)
if score > 0:
scored.append((score, e['path']))
scored.sort(reverse=True, key=lambda x: x[0])
return [p for _, p in scored]
def interactive_select(candidates):
print("Search results:")
for idx, p in enumerate(candidates, 1):
print(f" {idx:3d}. {p}")
sel = input("\nSelect files (e.g. 1,3-5): ").strip()
nums = set()
for part in sel.split(','):
if '-' in part:
a,b = part.split('-',1)
nums.update(range(int(a), int(b)+1))
else:
nums.add(int(part))
return [candidates[i-1] for i in sorted(nums) if 1 <= i <= len(candidates)]
def publish(paths):
urls = []
for path in paths:
token = uuid.uuid4().hex
link = SHARE_DIR / token
try:
os.symlink(path, link)
except FileExistsError:
pass
urls.append(f"{BASE_URL}/{token}")
return urls
def main():
parser = argparse.ArgumentParser(description="Search and share files from JSON DB")
parser.add_argument('keywords', nargs='*',
help='Search keywords (interactive if omitted)')
parser.add_argument('-n','--count', type=int, default=10,
help='How many top results to share')
args = parser.parse_args()
if not args.keywords:
# Interactive: ask for query
q = input("Enter search keywords (space-separated): ").strip().split()
args.keywords = q
entries = load_db()
matches = find_matches(entries, args.keywords)
top_n = matches[:args.count]
if sys.stdin.isatty():
# interactive selection
sel = interactive_select(top_n)
else:
# non-interactive: share all top_n
sel = top_n
urls = publish(sel)
for u in urls:
print(u)
if __name__=='__main__':
main()
+324 -57
View File
@@ -1,13 +1,51 @@
"""Centralised configuration and runtime constants for Conjurer.
Historically this module performed heavy filesystem and credential reads at
import time which made the bot brittle on hosts that did not mirror the
original paths. This version keeps the original platform defaults (so the
behaviour on the Raspberry Pi / WSL / Windows deployments is unchanged when no
environment variables are set) but adds three robustness improvements ported
from the dockerised experiment:
* every path/endpoint can be overridden via an environment variable,
* JSON state files are loaded defensively (a missing or corrupt file no longer
crashes the whole bot at import time),
* optional dependencies (openai, spotipy, netrc) and credentials are guarded so
the bot can still start when a secondary integration is offline, and
* a shared ``CONJURER_API_KEY`` plus ``service_headers()`` helper enables
authenticated internal HTTP calls.
All path constants intentionally remain plain ``str`` (with their original
trailing separators) to stay byte-for-byte compatible with the existing string
concatenation in the command modules.
"""
import json
import netrc
import logging
import os
from datetime import datetime
from platform import uname
from sys import platform
from typing import List, Optional, TypedDict
import openai
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
try:
import netrc
except ImportError: # pragma: no cover - standard on CPython
netrc = None
try:
import openai
except ImportError: # pragma: no cover - optional at runtime
openai = None
try:
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
except ImportError: # pragma: no cover - optional component
spotipy = None
SpotifyClientCredentials = None
logger = logging.getLogger("discord")
Music_Config = TypedDict(
"Music_Config",
@@ -30,16 +68,12 @@ MUSIC_FOLDER = ""
MEMORY_FIVE_SIARA = ""
MEMORY_FIVE_MUZYKA = ""
SETTINGS_FILE = ""
ENCODING = ""
ENCODING = "utf-8"
GRAPHICS_PATH = ""
MUZYKA_MOJEGO_LUDU_HISTORIA = 1500
MUZYKA_MOJEGO_LUDU_SLOWA_KLUCZOWE = 15
MUZYKA_MOJEGO_LUDU_PLAJLISTA = 30
FILE_SERVICE_ADDRESS = "http://192.168.1.15:5000"
RADIO_HARBOR_ADDRESS = "http://192.168.1.15:54321"
SKIP_TRACK = "/skip"
GET_MP3 = "/mp3"
SEND_MP3 = "/update_mp3"
GET_PLAYLIST = "/get_music"
@@ -48,14 +82,13 @@ CREATE_PRIO_PLAYLIST = "/create_priority_playlist"
REQUEST_MUSIC = "/request_radio_file"
CLEAR_PRIO = "/clear_pr_pls"
LIBRARIAN_SERVICE_ADDRESS = "http://192.168.1.192:5001"
SEND_QUERY = "/query"
TIME_BETWEEN_CALLS = 100000
LAST_SPONTANEOUS_CALL = datetime.now()
HOST_ADDRESS = "192.168.1.191"
PORT_ADDRESS = 5000
# *=========================================== Platform Specific Predefines
# *=========================================== Platform Specific Defaults
# These blocks only establish *default* values. Every constant is overridable
# through the matching environment variable further below.
if platform in ("linux", "linux2"):
SEPARATOR_FILE_PATH = "/"
@@ -74,18 +107,18 @@ if platform in ("linux", "linux2"):
DIR_PATH_SADOX = "/mnt/c/Users/mtusz/OneDrive/Dokumenty/Fansadox"
else:
LOGFILE = "/home/pi/Conjurer/discord.log"
MEMORY_FIVE_SIARA = "/home/pi/Conjurer/pamiec.json"
SYSTEM_GPT_SETTINGS = "/home/pi/Conjurer/system_gpt_settings.json"
MEMORY_FIVE_MUZYKA = "/home/pi/Conjurer/pamiec_muzyki.json"
MUSIC_FOLDER = "/home/pi/MediaShare/mp3/"
SETTINGS_FILE = "/home/pi/Conjurer/settings.json"
NETRC_FILE = "/home/pi/.netrc"
LOGSTORE = "/home/pi/MediaShara/logs/"
ACCIDENT_LOG = "/home/pi/Conjurer/accident_log.json"
LOGFILE = "./discord.log"
MEMORY_FIVE_SIARA = "./pamiec.json"
SYSTEM_GPT_SETTINGS = "./system_gpt_settings.json"
MEMORY_FIVE_MUZYKA = "./pamiec_muzyki.json"
MUSIC_FOLDER = "./"
SETTINGS_FILE = "./settings.json"
NETRC_FILE = "/srv/conjurer/secrets/.netrc"
LOGSTORE = "./logs/"
ACCIDENT_LOG = "./accident_log.json"
ENCODING = "utf-8"
GRAPHICS_PATH = "/home/pi/MediaShare/Conjurer_graphics/"
DIR_PATH_SADOX = "/home/pi/MediaShare/Fansadox/"
GRAPHICS_PATH = "./Conjurer_graphics/"
DIR_PATH_SADOX = "./Fansadox/"
elif platform == "win32":
@@ -101,42 +134,276 @@ elif platform == "win32":
ENCODING = "utf-8"
DIR_PATH_SADOX = "C:\\Users\\mtusz\\OneDrive\\Dokumenty\\Fansadox\\"
SEPARATOR_FILE_PATH = "\\"
with open(SETTINGS_FILE, "r", encoding=ENCODING) as f_settings_file:
DATA = json.load(f_settings_file)
REMOTE_HOST_NAME = "openai"
netrc_mod = netrc.netrc(NETRC_FILE)
authTokens = netrc_mod.authenticators(REMOTE_HOST_NAME)
openai.api_key = authTokens[2]
OPENAICLIENT = openai.AsyncOpenAI(api_key=openai.api_key)
REMOTE_HOST_NAME = "discord"
authTokens = netrc_mod.authenticators(REMOTE_HOST_NAME)
TOKEN = authTokens[2]
REMOTE_HOST_NAME = "spotipy"
authTokens = netrc_mod.authenticators(REMOTE_HOST_NAME)
SPOTIFY_CTRL = spotipy.Spotify(
client_credentials_manager=SpotifyClientCredentials(
client_id=authTokens[0],
client_secret=authTokens[2],
)
else:
# Fallback for development hosts (macOS, BSD, …) that match none of the
# production platforms. Everything is rooted next to this file so the
# module can at least be imported and unit-tested off-deployment.
_BASE_DIR = os.path.dirname(os.path.abspath(__file__))
SEPARATOR_FILE_PATH = os.sep
LOGFILE = os.path.join(_BASE_DIR, "discord.log")
MEMORY_FIVE_SIARA = os.path.join(_BASE_DIR, "pamiec.json")
SYSTEM_GPT_SETTINGS = os.path.join(_BASE_DIR, "system_gpt_settings.json")
MEMORY_FIVE_MUZYKA = os.path.join(_BASE_DIR, "pamiec_muzyki.json")
MUSIC_FOLDER = os.path.join(_BASE_DIR, "music") + os.sep
SETTINGS_FILE = os.path.join(_BASE_DIR, "settings.json")
NETRC_FILE = os.path.join(os.path.expanduser("~"), ".netrc")
LOGSTORE = os.path.join(_BASE_DIR, "logs") + os.sep
ACCIDENT_LOG = os.path.join(_BASE_DIR, "accident_log.json")
GRAPHICS_PATH = os.path.join(_BASE_DIR, "Conjurer_graphics") + os.sep
DIR_PATH_SADOX = os.path.join(_BASE_DIR, "Fansadox") + os.sep
# *=========================================== Environment overrides
# Values stay as plain strings so existing ``PATH + filename`` concatenation in
# the command modules keeps working unchanged.
# Container-friendly shortcut: point CONJURER_DATA_DIR at a single mounted
# volume and every writable data file/dir is rooted under it. The per-variable
# CONJURER_* overrides below still take precedence, so granular control remains
# possible and the native (Pi) deployment is unaffected when it is unset.
_DATA_DIR = os.getenv("CONJURER_DATA_DIR")
if _DATA_DIR:
LOGFILE = os.path.join(_DATA_DIR, "discord.log")
SETTINGS_FILE = os.path.join(_DATA_DIR, "settings.json")
MEMORY_FIVE_SIARA = os.path.join(_DATA_DIR, "pamiec.json")
MEMORY_FIVE_MUZYKA = os.path.join(_DATA_DIR, "pamiec_muzyki.json")
SYSTEM_GPT_SETTINGS = os.path.join(_DATA_DIR, "system_gpt_settings.json")
ACCIDENT_LOG = os.path.join(_DATA_DIR, "accident_log.json")
LOGSTORE = os.path.join(_DATA_DIR, "logs") + os.sep
GRAPHICS_PATH = os.path.join(_DATA_DIR, "Conjurer_graphics") + os.sep
MUSIC_FOLDER = os.path.join(_DATA_DIR, "music") + os.sep
DIR_PATH_SADOX = os.path.join(_DATA_DIR, "Fansadox") + os.sep
LOGFILE = os.getenv("CONJURER_LOG_FILE", LOGFILE)
NETRC_FILE = os.getenv("CONJURER_NETRC_FILE", NETRC_FILE)
SETTINGS_FILE = os.getenv("CONJURER_SETTINGS_FILE", SETTINGS_FILE)
MEMORY_FIVE_SIARA = os.getenv("CONJURER_MEMORY_FILE", MEMORY_FIVE_SIARA)
MEMORY_FIVE_MUZYKA = os.getenv("CONJURER_MUSIC_MEMORY_FILE", MEMORY_FIVE_MUZYKA)
SYSTEM_GPT_SETTINGS = os.getenv("CONJURER_SYSTEM_GPT_SETTINGS", SYSTEM_GPT_SETTINGS)
GRAPHICS_PATH = os.getenv("CONJURER_GRAPHICS_PATH", GRAPHICS_PATH)
MUSIC_FOLDER = os.getenv("CONJURER_MUSIC_FOLDER", MUSIC_FOLDER)
LOGSTORE = os.getenv("CONJURER_LOGSTORE", LOGSTORE)
ACCIDENT_LOG = os.getenv("CONJURER_ACCIDENT_LOG", ACCIDENT_LOG)
DIR_PATH_SADOX = os.getenv("CONJURER_SADOX_DIR", DIR_PATH_SADOX)
ENCODING = os.getenv("CONJURER_ENCODING", ENCODING)
SEPARATOR_FILE_PATH = os.getenv("CONJURER_PATH_SEPARATOR", SEPARATOR_FILE_PATH)
# Voice-recognition transcript dumps. Defaults next to the log file (so on the
# Pi it lands in /home/pi/Conjurer/transcripts/, in docker under /data).
TRANSCRIPTS_PATH = os.getenv(
"CONJURER_TRANSCRIPTS_PATH",
os.path.join(os.path.dirname(LOGFILE) or ".", "transcripts") + os.sep,
)
REMOTE_HOST_NAME = "youtube"
authTokens = netrc_mod.authenticators(REMOTE_HOST_NAME)
YOUTUBE_AUTH = [authTokens[0],authTokens[2]]
WORD_REACTIONS = DATA["word_reactions"]
CYCLIC_WORDS = DATA["cyclic_words"]
FILE_SERVICE_ADDRESS = os.getenv("CONJURER_FILE_SERVICE", "http://192.168.1.15:5000")
RADIO_HARBOR_ADDRESS = os.getenv("CONJURER_RADIO_HARBOR", "http://192.168.1.15:54321")
SKIP_TRACK = os.getenv("CONJURER_SKIP_ENDPOINT", "/skip")
LIBRARIAN_SERVICE_ADDRESS = os.getenv(
"CONJURER_LIBRARIAN_SERVICE", "http://192.168.1.192:5001"
)
HOST_ADDRESS = os.getenv("CONJURER_DISCORD_HOST", "192.168.1.191")
PORT_ADDRESS = int(os.getenv("CONJURER_DISCORD_PORT", "5000"))
# Shared secret for authenticating internal service-to-service HTTP calls.
API_SHARED_KEY = os.getenv("CONJURER_API_KEY", "")
# *=========================================== Self-healing runtime layout
# A fresh host/volume must never kill the bot at import time. Missing
# directories are created and missing state files are seeded - first from the
# templates shipped alongside this file (repo checkout / docker image), then
# from a safe empty structure. Existing files are never touched, so preserved
# history always wins.
_TEMPLATE_DIR = os.path.dirname(os.path.abspath(__file__))
def _ensure_dir(path: str) -> None:
if not path:
return
try:
os.makedirs(path, exist_ok=True)
except OSError as exc:
logger.warning("Cannot create directory %s: %s", path, exc)
def _seed_file(path: str, template_name: str, empty_content: str) -> None:
"""Create *path* from the repo template (or *empty_content*) if missing."""
if not path or os.path.exists(path):
return
_ensure_dir(os.path.dirname(path) or ".")
template = os.path.join(_TEMPLATE_DIR, template_name)
try:
if os.path.exists(template) and os.path.abspath(template) != os.path.abspath(path):
import shutil
shutil.copyfile(template, path)
logger.warning("Seeded missing %s from repo template", path)
else:
with open(path, "w", encoding=ENCODING) as handle:
handle.write(empty_content)
logger.warning("Created missing %s as empty state", path)
except OSError as exc:
logger.warning("Cannot seed %s: %s", path, exc)
def _ensure_runtime_layout() -> None:
for directory in (
os.path.dirname(LOGFILE) or ".",
LOGSTORE,
GRAPHICS_PATH,
MUSIC_FOLDER,
TRANSCRIPTS_PATH,
):
_ensure_dir(directory)
# (target path, template shipped next to this file, empty fallback)
_seed_file(SETTINGS_FILE, "settings.json", "{}")
_seed_file(SYSTEM_GPT_SETTINGS, "system_gpt_settings.json", "{}")
_seed_file(MEMORY_FIVE_SIARA, "pamiec.json", "[]")
_seed_file(MEMORY_FIVE_MUZYKA, "pamiec_muzyki.json", "[]")
_seed_file(ACCIDENT_LOG, "accident_log.json", "[]")
_ensure_runtime_layout()
# *=========================================== Defensive state loading
def _load_json(path: str, fallback):
"""Load JSON from *path*, falling back gracefully on missing/corrupt files."""
try:
with open(path, "r", encoding=ENCODING) as handle:
return json.load(handle)
except FileNotFoundError:
logger.warning("Missing JSON file at %s - using fallback", path)
return fallback
except json.JSONDecodeError:
logger.warning("Corrupt JSON at %s - resetting to fallback", path)
return fallback
DATA = _load_json(SETTINGS_FILE, {})
WORD_REACTIONS = DATA.get("word_reactions", {})
CYCLIC_WORDS = DATA.get("cyclic_words", {})
for key in WORD_REACTIONS:
WORD_REACTIONS[key][2] = datetime.now()
with open(MEMORY_FIVE_SIARA, "r+", encoding=ENCODING) as temp_memory_file:
# First we load existing data into a dict.
MESSAGE_TABLE = json.load(temp_memory_file)
if isinstance(WORD_REACTIONS[key], list) and len(WORD_REACTIONS[key]) >= 3:
WORD_REACTIONS[key][2] = datetime.now()
with open(SYSTEM_GPT_SETTINGS, "r+", encoding=ENCODING) as temp_settings_file:
# First we load existing data into a dict.
GPT_SETTINGS = json.load(temp_settings_file)
with open(MEMORY_FIVE_MUZYKA, "r+", encoding=ENCODING) as temp_music_memory_file:
# First we load existing data into a dict.
MESSAGE_TABLE_MUZYKA = json.load(temp_music_memory_file)
SPECJALNE_ZIEMNIACZKI = GPT_SETTINGS[1]
MESSAGE_TABLE = _load_json(MEMORY_FIVE_SIARA, [])
GPT_SETTINGS = _load_json(SYSTEM_GPT_SETTINGS, {})
MESSAGE_TABLE_MUZYKA = _load_json(MEMORY_FIVE_MUZYKA, [])
SPECJALNE_ZIEMNIACZKI = GPT_SETTINGS[1] if isinstance(GPT_SETTINGS, list) else {}
ASSISTANTS = {}
# *=========================================== Credentials
def _load_netrc_credentials(host: str):
"""Return the netrc authenticators tuple for *host* or ``None``."""
if netrc is None:
return None
try:
parsed = netrc.netrc(NETRC_FILE)
except FileNotFoundError:
logger.warning("netrc file %s not found", NETRC_FILE)
return None
except netrc.NetrcParseError:
logger.warning("netrc file %s is invalid", NETRC_FILE)
return None
return parsed.authenticators(host)
def _resolve_token(host: str, env_var: str) -> Optional[str]:
"""Prefer an environment variable, then fall back to netrc."""
env_value = os.getenv(env_var)
if env_value:
return env_value
creds = _load_netrc_credentials(host)
if creds:
return creds[2]
logger.warning("Token for %s not configured", host)
return None
OPENAI_API_KEY = _resolve_token("openai", "OPENAI_API_KEY")
if openai and OPENAI_API_KEY:
openai.api_key = OPENAI_API_KEY
OPENAICLIENT = openai.AsyncOpenAI(api_key=OPENAI_API_KEY)
else:
OPENAICLIENT = None
TOKEN = _resolve_token("discord", "DISCORD_TOKEN")
# Voice recognition (AssemblyAI). None = the voice cog reports and disables.
ASSEMBLYAI_API_KEY = _resolve_token("assemblyai", "ASSEMBLYAI_API_KEY")
if spotipy:
_spotify_creds = _load_netrc_credentials("spotipy")
if _spotify_creds and SpotifyClientCredentials:
SPOTIFY_CTRL = spotipy.Spotify(
client_credentials_manager=SpotifyClientCredentials(
client_id=_spotify_creds[0],
client_secret=_spotify_creds[2],
)
)
else:
SPOTIFY_CTRL = None
else:
SPOTIFY_CTRL = None
_youtube_creds = _load_netrc_credentials("youtube")
if _youtube_creds:
YOUTUBE_AUTH = [_youtube_creds[0], _youtube_creds[2]]
else:
YOUTUBE_AUTH = [
os.getenv("YOUTUBE_USERNAME", ""),
os.getenv("YOUTUBE_PASSWORD", ""),
]
def service_headers():
"""Shared header dict for internal service-to-service HTTP calls.
Returns an empty dict when no key is configured, keeping calls backward
compatible with deployments that do not (yet) enforce authentication.
"""
if API_SHARED_KEY:
return {"X-Conjurer-Api-Key": API_SHARED_KEY}
return {}
LATEX_TEX_ENGINE = "tectonic"
LATEX_MAX_COMPILE_SECONDS = 45
LATEX_MAX_ATTACH_MB = 8
LATEX_MAX_ZIP_MB = 25
OPENAI_MODEL = "gpt-4o-mini"
ALLOWED_ROLES = ["Nocna Zmiana", "Jarl", "Thane", "Bartender"]
GUILD_ID = 664789470779932693
LATEST_MODEL = "gpt-4o" # najnowszy/do rozmów (możesz zmienić w jednym miejscu)
CHEAP_MODEL = "gpt-4o-mini" # najtańszy (fallback do 3.5 niżej)
# *=========================================== Conan Exiles bridge
# Every value is optional; the conanjurer cog stays dormant unless configured.
# Channel/role ids default to 0 ("not defined"); RCON host/log path default to
# "" ("disabled"). See conanjurer_commands.py / conanjurer_functions.py.
CONAN_GM_ROLE_ID = int(os.getenv("CONAN_GM_ROLE_ID", "0"))
CONAN_CHAT_CHANNEL_ID = int(os.getenv("CONAN_CHAT_CHANNEL_ID", "0"))
CONAN_EVENTS_CHANNEL_ID = int(os.getenv("CONAN_EVENTS_CHANNEL_ID", "0"))
# Player-join notifications: leave at 0 to keep the feature disabled.
CONAN_JOIN_CHANNEL_ID = int(os.getenv("CONAN_JOIN_CHANNEL_ID", "0"))
CONAN_PLAYER_POLL_SECONDS = int(os.getenv("CONAN_PLAYER_POLL_SECONDS", "60"))
CONAN_RCON_HOST = os.getenv("CONAN_RCON_HOST", "")
CONAN_RCON_PORT = int(os.getenv("CONAN_RCON_PORT", "25575"))
CONAN_RCON_PASSWORD = os.getenv("CONAN_RCON_PASSWORD", "")
CONAN_LOG_MODE = os.getenv("CONAN_LOG_MODE", "local") # "local" | "sftp"
CONAN_LOG_PATH = os.getenv("CONAN_LOG_PATH", "")
CONAN_SFTP_HOST = os.getenv("CONAN_SFTP_HOST", "")
CONAN_SFTP_PORT = int(os.getenv("CONAN_SFTP_PORT", "22"))
CONAN_SFTP_USER = os.getenv("CONAN_SFTP_USER", "")
CONAN_SFTP_PASSWORD = os.getenv("CONAN_SFTP_PASSWORD", "")
Executable → Regular
+26 -4
View File
@@ -1,6 +1,6 @@
#!/bin/bash
total_commands=24
total_commands=31
current_command=0
function print_progress {
@@ -79,8 +79,30 @@ print_progress "cp ./conjurer/radio_commands.py ./Conjurer/"
cp ./conjurer/voice_recognition_commands.py ./Conjurer/
print_progress "cp ./conjurer/voice_recognition_commands.py ./Conjurer/"
cp ./conjurer/thin_client.py ./Conjurer/bot.py
print_progress "cp ./conjurer/thin_client.py ./Conjurer/bot.py"
cp ./conjurer/file_search_functions.py ./Conjurer/
print_progress "cp ./conjurer/file_search_functions.py ./Conjurer/"
cp ./conjurer/file_search_commands.py ./Conjurer
print_progress "cp ./conjurer/file_search_functions.py ./Conjurer/"
cp ./conjurer/latex_commands.py ./Conjurer/
print_progress "cp ./conjurer/latex_commands.py ./Conjurer/"
cp ./conjurer/latex_functions.py ./Conjurer/
print_progress "cp ./conjurer/latex_functions.py ./Conjurer/"
cp ./conjurer/librarian_functions.py ./Conjurer
print_progress "cp ./conjurer/librarian_functions.py ./Conjurer/"
cp ./conjurer/conanjurer_commands.py ./Conjurer/
print_progress "cp ./conjurer/conanjurer_commands.py ./Conjurer/"
cp ./conjurer/conanjurer_functions.py ./Conjurer/
print_progress "cp ./conjurer/conanjurer_functions.py ./Conjurer/"
cp ./conjurer/bot.py ./Conjurer/bot.py
print_progress "cp ./conjurer/bot.py ./Conjurer/bot.py"
sudo systemctl restart conjurer.service
print_progress "sudo systemctl restart conjurer.service"
print_progress "sudo systemctl restart conjurer.service"
Executable → Regular
+2 -1
View File
@@ -2,5 +2,6 @@
cd ./conjurer || exit
git pull
cd ..
cp ./conjurer/conjurer_musician/media_search_functions.py ./Conjurer
cp ./conjurer/conjurer_musician/conjurer_musician.py ./Conjurer
sudo systemctl restart conjurer_musician.service
sudo systemctl restart conjurer_musician.service
+141
View File
@@ -0,0 +1,141 @@
#!/usr/bin/env bash
# System-wide install & verify (Debian/RPi):
# - EB Garamond (APT)
# - Cinzel Decorative Black (RAW TTF)
# - IM Fell English SC (gstatic/RAW)
# No TeX build here. Very verbose + clear final report.
set -Eeuo pipefail
SUDO=sudo; [ "$(id -u)" -eq 0 ] && SUDO=
log(){ echo -e "[+] $*"; }
warn(){ echo -e "[WARN] $*" >&2; }
die(){ echo -e "[FATAL] $*" >&2; exit 1; }
need(){ command -v "$1" >/dev/null 2>&1 || die "Missing tool: $1"; }
fetch(){
local url="$1" out="$2"
if command -v wget >/dev/null 2>&1; then
log "wget -> $url"
wget -O "$out" --https-only --no-verbose "$url"
else
log "curl -> $url"
curl -L --fail --show-error --output "$out" "$url"
fi
}
# ---- detection using fc-list families (robust, no PCRE) ----
fc_families_lower() {
fc-list -f '%{family}\n' \
| tr ',' '\n' \
| sed 's/^[[:space:]]*//; s/[[:space:]]*$//' \
| tr '[:upper:]' '[:lower:]' \
| sort -u
}
has_family_re() { # $1 = POSIX ERE anchored-at-start pattern (lowercase)
fc_families_lower | grep -Eiq "$1"
}
# ---- prereqs ----
for t in fc-cache fc-list grep sed awk; do need "$t"; done
command- v wget >/dev/null 2>&1 || command -v curl >/dev/null 2>&1 || die "Need wget or curl"
SYSTEM_DIR="/usr/local/share/fonts/dj"
DIR_CINZEL="$SYSTEM_DIR/cinzel-decorative"
DIR_IMFELL="$SYSTEM_DIR/im-fell-english-sc"
$SUDO mkdir -p "$DIR_CINZEL" "$DIR_IMFELL"
echo "=== APT phase (EB Garamond) ==="
if command -v apt-get >/dev/null 2>&1; then
$SUDO apt-get update -y || warn "apt update failed (continuing)"
if $SUDO apt-get install -y --no-install-recommends fonts-ebgaramond; then
log "installed: fonts-ebgaramond"
else
warn "apt install failed: fonts-ebgaramond"
fi
else
warn "apt-get not found — skipping"
fi
echo
log "Families BEFORE web fallback (sample grep):"
fc-list | grep -Ei "Garamond|Cinzel|Fell" || echo " (none)"
echo
echo "=== Web fallback (system-wide to $SYSTEM_DIR) ==="
# --- Cinzel Decorative Black ---
CINZEL_TTF="$DIR_CINZEL/CinzelDecorative-Black.ttf"
if ! has_family_re '^cinzel decorative( |$)'; then
fetch "https://github.com/google/fonts/raw/main/ofl/cinzeldecorative/CinzelDecorative-Black.ttf" "$CINZEL_TTF" || warn "Cinzel Decorative Black download failed"
[ -s "$CINZEL_TTF" ] && $SUDO chmod 0644 "$CINZEL_TTF"
else
log "Cinzel Decorative already present."
fi
# --- IM Fell English SC ---
IMFELL_TTF="$DIR_IMFELL/IMFeENsc.ttf"
if ! has_family_re '^im fell english sc( |$)'; then
# Try known gstatic TTFs (stable direct links)
for u in \
"https://fonts.gstatic.com/s/imfellenglishsc/v7/h3Tn6yWfw4b5qaLD1RWvz5ATixNthKRRR1XVH3rJNiw.ttf" \
"https://fonts.gstatic.com/s/imfellenglishsc/v6/h3Tn6yWfw4b5qaLD1RWvz5ATixNthKRRR1XVH3rJNiw.ttf"
do
if fetch "$u" "$IMFELL_TTF"; then
[ -s "$IMFELL_TTF" ] && { $SUDO chmod 0644 "$IMFELL_TTF"; log "IM Fell SC from gstatic OK"; break; }
fi
done
# Secondary RAW in google/fonts (names changed over time)
if [ ! -s "$IMFELL_TTF" ]; then
for u in \
"https://github.com/google/fonts/raw/main/ofl/imfellenglishsc/IMFeENsc28P.ttf" \
"https://github.com/google/fonts/raw/main/ofl/imfellenglishsc/IMFellEnglishSC-Regular.ttf"
do
if fetch "$u" "$IMFELL_TTF"; then
[ -s "$IMFELL_TTF" ] && { $SUDO chmod 0644 "$IMFELL_TTF"; log "IM Fell SC from google/fonts RAW OK"; break; }
fi
done
fi
# Minimal mirror fallback (only if naprawdę trzeba)
if [ ! -s "$IMFELL_TTF" ]; then
for u in \
"https://www.wfonts.com/download/data/2016/06/14/im-fell-english-sc/IMFeENsc28P.ttf" \
"https://www.1001freefonts.com/d/6800/IMFeENsc28P.ttf"
do
fetch "$u" "$IMFELL_TTF" || true
[ -s "$IMFELL_TTF" ] && { $SUDO chmod 0644 "$IMFELL_TTF"; log "IM Fell SC from mirror OK"; break; }
done
fi
[ -s "$IMFELL_TTF" ] || warn "IM Fell English SC still missing."
else
log "IM Fell English SC already present."
fi
echo
log "Fix perms & rebuild cache..."
$SUDO find "$SYSTEM_DIR" -type f -size 0 -print -delete || true
$SUDO find "$SYSTEM_DIR" -type f \( -name "*.ttf" -o -name "*.otf" \) -exec chmod 0644 {} \; || true
$SUDO find "$SYSTEM_DIR" -type d -exec chmod 0755 {} \; || true
$SUDO fc-cache -f -v >/dev/null || warn "fc-cache returned non-zero"
# ---- FINAL REPORT ----
echo
log "FINAL STATUS (fc-list families & file presence)"
families="$(fc_families_lower)"
report(){
local label="$1" fam_re="$2" file_hint="$3"
local fam_ok file_ok="n/a"
if echo "$families" | grep -Eiq "$fam_re"; then fam_ok="OK"; else fam_ok="MISSING"; fi
if [ -n "$file_hint" ]; then
if [ -s "$file_hint" ]; then file_ok="yes"; else file_ok="no"; fi
fi
printf " %-22s : families=%-8s file=%-3s (%s)\n" "$label" "$fam_ok" "$file_ok" "${file_hint:-no-file-hint}"
}
report "EB Garamond" '^eb garamond( |$)' "" # from APT
report "Cinzel Decorative" '^cinzel decorative( |$)' "$CINZEL_TTF"
report "IM Fell English SC" '^im fell english sc( |$)' "$IMFELL_TTF"
echo
log "Done. (Run your LaTeX pipeline next.)"
+64
View File
@@ -0,0 +1,64 @@
# Conjurer main Discord bot.
# Build from the repository root:
# docker build -f docker/Dockerfile.bot -t conjurer-bot .
FROM python:3.13-trixie
# System dependencies:
# ffmpeg - audio download/convert (yt_dlp) and Discord voice
# libopus0 - Discord voice (PyNaCl / discord-ext-voice-recv)
# poppler-utils - pdf2image (librarian/latex previews)
# git - required by the git+https entry in requirements
# build-essential - native wheels (PyNaCl etc.)
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg \
libopus0 \
poppler-utils \
git \
build-essential \
curl \
ca-certificates \
python3-dev \
build-essential \
portaudio19-dev \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# Optional: Tectonic for the $latex command. Remove this layer if unused.
RUN curl -fsSL https://drop-sh.fullyjustified.net | sh \
&& mv tectonic /usr/local/bin/tectonic \
|| echo "tectonic not installed - the LaTeX feature will be disabled"
COPY requirements_bot.txt requirements_conan.txt ./
RUN pip install --no-cache-dir --upgrade pip \
&& pip install --no-cache-dir -r requirements_bot.txt
# Best-effort Conan bridge extras: aiomcrcon supports Python <= 3.11 only, so
# on this 3.13 image the install fails harmlessly and the conanjurer cog stays
# dormant (its imports are guarded).
RUN pip install --no-cache-dir -r requirements_conan.txt \
|| echo "conan extras skipped - conanjurer cog will stay dormant"
# Vendored forks take import precedence over the pip packages of the same name,
# because /app (the script dir) is first on sys.path when running `python bot.py`.
COPY yt_dlp ./yt_dlp
COPY spotify_dl ./spotify_dl
# Bot sources + default config/asset templates. Runtime state (conversation
# history etc.) is read from the mounted /data volume via CONJURER_DATA_DIR,
# so these committed copies only act as first-run fallbacks.
COPY *.py ./
COPY settings.json system_gpt_settings.json accident_log.json pamiec.json pamiec_muzyki.json ./
COPY fuckery.jpg willowisp.png wod_beacon.jpg ./
COPY docker/entrypoint.bot.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
ENV PYTHONUNBUFFERED=1 \
CONJURER_DATA_DIR=/data \
CONJURER_DISCORD_HOST=0.0.0.0 \
CONJURER_DISCORD_PORT=5000
VOLUME ["/data"]
EXPOSE 5000
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
CMD ["python", "bot.py"]
+27
View File
@@ -0,0 +1,27 @@
# Conjurer librarian (Crossref search + local DOI database lookup).
# Build from the repository root:
# docker build -f docker/Dockerfile.librarian -t conjurer-librarian .
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY conjurer_librarian/requirements_librarian.txt ./
RUN pip install --no-cache-dir --upgrade pip \
&& pip install --no-cache-dir -r requirements_librarian.txt requests
COPY conjurer_librarian/ ./
ENV PYTHONUNBUFFERED=1 \
CONJURER_LIBRARIAN_HOST=0.0.0.0 \
CONJURER_LIBRARIAN_PORT=5001 \
CONJURER_LIBRARIAN_DB_PATH=/doi/
# The local DOI chunk database (large) is mounted here.
VOLUME ["/doi"]
EXPOSE 5001
CMD ["python", "conjurer_librarian.py"]
+35
View File
@@ -0,0 +1,35 @@
# Conjurer musician (Flask file/playlist service backing the radio).
# Build from the repository root:
# docker build -f docker/Dockerfile.musician -t conjurer-musician .
#
# NOTE: this containerises the musician *web service* only. The Liquidsoap
# radio (radio_conjurer.liq) and any Samba/NFS share tooling run separately.
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY conjurer_musician/requirements_musician.txt ./
RUN pip install --no-cache-dir --upgrade pip \
&& pip install --no-cache-dir -r requirements_musician.txt
COPY conjurer_musician/ ./
COPY docker/entrypoint.musician.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
ENV PYTHONUNBUFFERED=1 \
CONJURER_MUSICIAN_HOST=0.0.0.0 \
CONJURER_MUSICIAN_PORT=5000 \
CONJURER_MUSIC_FOLDER=/music \
CONJURER_MUSICIAN_BASE=/data \
CONJURER_STREAM_TEMPLATE=/app/stream.html
# /music = the mp3 library (read-only ok); /data = writable playlists + logs.
VOLUME ["/music", "/data"]
EXPOSE 5000
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
CMD ["python", "conjurer_musician.py"]
+22
View File
@@ -0,0 +1,22 @@
# Main bot VM. Run from the repository root:
# cp docker/env/bot.env.example docker/env/bot.env # then edit
# docker compose -f docker/compose.bot.yaml up -d --build
services:
conjurer-bot:
build:
context: ..
dockerfile: docker/Dockerfile.bot
image: conjurer-bot:latest
container_name: conjurer-bot
restart: unless-stopped
env_file:
- env/bot.env
ports:
# Flask comm layer — musician/librarian POST results here (/prepped_tracks, /conjurer).
- "5000:5000"
volumes:
# Persistent state (conversation history, settings, logs). Populate this
# host dir with your existing pamiec.json etc. to preserve command history.
- /srv/conjurer/data:/data
# Tokens: a read-only netrc covers discord/openai/spotipy/youtube in one file.
- /srv/conjurer/secrets/.netrc:/secrets/.netrc:ro
+20
View File
@@ -0,0 +1,20 @@
# Librarian VM. Run from the repository root:
# cp docker/env/librarian.env.example docker/env/librarian.env # then edit
# docker compose -f docker/compose.librarian.yaml up -d --build
services:
conjurer-librarian:
build:
context: ..
dockerfile: docker/Dockerfile.librarian
image: conjurer-librarian:latest
container_name: conjurer-librarian
restart: unless-stopped
env_file:
- env/librarian.env
ports:
- "5001:5001"
volumes:
# Local DOI chunk database (0_chunk.txt ... N_chunk.txt).
- /srv/librarian/doi:/doi:ro
# Optional: netrc holding Crossref credentials (or use CONJURER_CROSSREF_MAILTO).
- /srv/librarian/secrets/.netrc:/secrets/.netrc:ro
+21
View File
@@ -0,0 +1,21 @@
# Musician VM (optional — you plan to adapt/implement this yourself).
# Run from the repository root:
# cp docker/env/musician.env.example docker/env/musician.env # then edit
# docker compose -f docker/compose.musician.yaml up -d --build
services:
conjurer-musician:
build:
context: ..
dockerfile: docker/Dockerfile.musician
image: conjurer-musician:latest
container_name: conjurer-musician
restart: unless-stopped
env_file:
- env/musician.env
ports:
- "5000:5000"
volumes:
# The mp3 library the service indexes and serves.
- /srv/musician/music:/music
# Runtime playlists/logs the service writes.
- /srv/musician/data:/data
+16
View File
@@ -0,0 +1,16 @@
#!/bin/sh
# Seed the data volume with the baked-in default JSON state on first run only.
# Existing files (e.g. your preserved pamiec.json history) are never overwritten.
set -e
DATA="${CONJURER_DATA_DIR:-/data}"
mkdir -p "$DATA"
for f in settings.json system_gpt_settings.json pamiec.json pamiec_muzyki.json accident_log.json; do
if [ ! -e "$DATA/$f" ] && [ -e "/app/$f" ]; then
cp "/app/$f" "$DATA/$f"
echo "entrypoint: seeded $f into $DATA"
fi
done
exec "$@"
+23
View File
@@ -0,0 +1,23 @@
#!/bin/sh
# Prepare the musician's writable volume so the web service starts cleanly on a
# fresh container. Existing files are never overwritten (preserves your data).
set -e
DATA="${CONJURER_MUSICIAN_BASE:-/data}"
MUSIC="${CONJURER_MUSIC_FOLDER:-/music}"
mkdir -p "$DATA" "$DATA/logs" "$MUSIC"
# The track-forwarding thread tails the Liquidsoap radio logs. When the radio
# runs separately (or hasn't started yet) these files may not exist; create
# them empty so the tailer waits instead of crashing.
for f in radio_log.log persistence.log; do
[ -e "$DATA/$f" ] || : > "$DATA/$f"
done
# Ensure the managed playlists exist (routes/rescan also create them; this just
# avoids a first-tick race before the initial scan).
for f in all_playlist.playlist hit.playlist request.playlist priority_queue.playlist; do
[ -e "$DATA/$f" ] || : > "$DATA/$f"
done
exec "$@"
+39
View File
@@ -0,0 +1,39 @@
# Copy to docker/env/bot.env and fill in. Do NOT commit the real file.
# --- Secrets ------------------------------------------------------------
# Option A: mount a netrc (recommended — covers discord/openai/spotipy/youtube).
CONJURER_NETRC_FILE=/secrets/.netrc
# Option B: pass tokens directly (these take precedence over netrc).
# DISCORD_TOKEN=
# OPENAI_API_KEY=
# ASSEMBLYAI_API_KEY= # voice recognition; netrc machine 'assemblyai' works too
# YOUTUBE_USERNAME=
# YOUTUBE_PASSWORD=
# --- Data ---------------------------------------------------------------
# Single mounted volume; all writable state is rooted here.
CONJURER_DATA_DIR=/data
# --- Flask comm layer (inbound from musician/librarian) -----------------
CONJURER_DISCORD_HOST=0.0.0.0
CONJURER_DISCORD_PORT=5000
# --- Internal service auth ----------------------------------------------
# Set the SAME value on bot + musician + librarian. Empty = auth disabled.
CONJURER_API_KEY=
# --- Where the bot reaches the other services (other Proxmox VMs) --------
CONJURER_FILE_SERVICE=http://MUSICIAN_VM_IP:5000
CONJURER_RADIO_HARBOR=http://MUSICIAN_VM_IP:54321
CONJURER_LIBRARIAN_SERVICE=http://LIBRARIAN_VM_IP:5001
# --- Conan Exiles bridge (optional; empty/0 = disabled) -----------------
# CONAN_GM_ROLE_ID=0
# CONAN_RCON_HOST=
# CONAN_RCON_PORT=25575
# CONAN_RCON_PASSWORD=
# CONAN_CHAT_CHANNEL_ID=0
# CONAN_EVENTS_CHANNEL_ID=0
# CONAN_JOIN_CHANNEL_ID=0
# CONAN_LOG_MODE=local
# CONAN_LOG_PATH=
+21
View File
@@ -0,0 +1,21 @@
# Copy to docker/env/librarian.env and fill in.
CONJURER_LIBRARIAN_HOST=0.0.0.0
CONJURER_LIBRARIAN_PORT=5001
# Same shared secret as the bot (empty = auth disabled).
CONJURER_API_KEY=
# Where to POST search results back to (the bot's comm layer).
CONJURER_MAIN_BOT=http://BOT_VM_IP:5000
# Crossref polite-pool contact (or put credentials in netrc under "crossref").
CONJURER_CROSSREF_MAILTO=you@example.com
# Local DOI chunk database (mounted volume): expects 0_chunk.txt .. N_chunk.txt
CONJURER_LIBRARIAN_DB_PATH=/doi/
CONJURER_LIBRARIAN_MAXTHREADS=41
CONJURER_LIBRARIAN_CHUNK=_chunk.txt
# Optional netrc (for Crossref credentials)
CONJURER_NETRC_FILE=/secrets/.netrc
+17
View File
@@ -0,0 +1,17 @@
# Copy to docker/env/musician.env and fill in.
CONJURER_MUSICIAN_HOST=0.0.0.0
CONJURER_MUSICIAN_PORT=5000
# Same shared secret as the bot (empty = auth disabled).
CONJURER_API_KEY=
# Where to POST "now playing" / prepped-track updates (the bot's comm layer).
CONJURER_MAIN_BOT=http://BOT_VM_IP:5000
# The mp3 library (mounted volume).
CONJURER_MUSIC_FOLDER=/music
# Runtime paths (mounted volume) — playlists/logs the service writes.
CONJURER_MUSICIAN_BASE=/data
CONJURER_LOGSTORE=/data/logs
+163
View File
@@ -0,0 +1,163 @@
# Conjurer Deployment Guide
This document walks through deploying the Conjurer stack (Discord bot, music
service, librarian service) on two Raspberry Pi 4s and one Windows host, first
with plain Docker Compose, then with a future Kubernetes setup.
## 1. Current Hardware Layout
- **Windows PC**: Stores persistent data (JSON memories, configs, music
catalogue). Shares folders over the network (SMB) for the Pis.
- **Raspberry Pi A** (“radio”): Runs the Liquidsoap/liq radio pipeline and the
musician service (Flask + file watcher).
- **Raspberry Pi B** (“bot”): Runs the Discord bot, communication bridge, and
librarian Flask service.
You can rebalance as follows:
| Service | Recommended Host | Notes |
|---------------------|------------------|-------|
| Discord bot + comms | Raspberry Pi B | Needs outbound internet, moderate CPU |
| Librarian service | Raspberry Pi B | CPU-heavy during Crossref queries; keep close to bot |
| Musician service | Raspberry Pi A | Has direct disk access to music, same box as Liquidsoap |
| Data storage | Windows | Expose via SMB; mount inside containers |
## 2. Prepare Shared Storage on Windows
1. Create directories, e.g. `C:\Conjurer\config`, `C:\Conjurer\logs`,
`C:\Conjurer\music`, `C:\Conjurer\playlists`, `C:\Conjurer\secrets`.
2. Copy your existing JSON settings (`settings.json`, `pamiec.json`,
`pamiec_muzyki.json`, `system_gpt_settings.json`, etc.) into `config`.
3. Create blank placeholder files if they do not exist yet.
4. Share the root folder (`C:\Conjurer`) over SMB with read/write access for the
Pi user (create credentials if necessary).
## 3. Configure Environment Files
1. On your workstation, copy the example env files:
```bash
cp docker/env/bot.env.example docker/env/bot.env
cp docker/env/musician.env.example docker/env/musician.env
cp docker/env/librarian.env.example docker/env/librarian.env
```
2. Edit each `docker/env/*.env` to replace `HACKME!` with real values:
- `DISCORD_TOKEN`, `OPENAI_API_KEY`, `CONJURER_API_KEY` (use the same value for
all services).
- For musician/librarian, adjust mounts to the SMB paths you will mount on the
Pis, e.g. `/mnt/conjurer/music`.
- Set `CONJURER_CROSSREF_MAILTO` to a real email as required by Crossref.
3. If you rely on `.netrc`, copy it to `C:\Conjurer\secrets\.netrc` and set
`CONJURER_NETRC_FILE` accordingly.
## 4. Install Docker on Raspberry Pis and Windows
### Raspberry Pi
```bash
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER
sudo reboot
# Install docker compose plugin
sudo apt-get install docker-compose-plugin
```
### Windows
- Install **Docker Desktop**.
- Enable WSL2 backend and expose the shared Windows folders to the containers
(Docker Desktop settings → Resources → File Sharing).
## 5. Deploy Musician Service (Pi A)
1. SSH into Raspberry Pi A.
2. Mount the Windows SMB share:
```bash
sudo mkdir -p /mnt/conjurer
sudo apt-get install cifs-utils
sudo mount -t cifs //WINDOWS_HOST/Conjurer /mnt/conjurer -o user=YOURUSER
```
Add an entry to `/etc/fstab` for persistence.
3. Copy the repo to the Pi or `git clone` it.
4. On Pi A, create override compose file (optional) pointing volumes to
`/mnt/conjurer`.
5. Start only the musician service:
```bash
docker compose up --build -d musician
```
Alternatively, duplicate `docker-compose.yml`, strip other services, and run
`docker compose up -d`.
## 6. Deploy Bot + Librarian (Pi B)
1. Repeat SMB mount on Pi B (same mount path).
2. Copy repo / pull latest changes.
3. Create `.env` files with tokens (or copy from control machine).
4. Start bot and librarian:
```bash
docker compose up -d bot librarian
```
## 7. Optional: Run Supporting Liquidsoap Radio
- Keep Liquidsoap on Pi A as-is, using the same music directories. Ensure the
musician container has read access to those directories (bind mount).
## 8. Verifying
1. `docker ps` on each Pi to confirm containers running.
2. Inspect logs under the mounted logs directory (`/mnt/conjurer/logs`).
3. Join Discord server; issue commands to confirm functionality.
4. Hit health endpoints manually (e.g. `curl http://PIB:5000/conjurer`).
## Rebalancing Suggestions
- If librarian CPU spikes become an issue, move it to Pi A or another host.
- If you add a dedicated NAS, mount the network share read-only for the musician
container and read/write for other services.
## 9. Future Kubernetes Deployment (Outline)
### Hardware Considerations
- Minimum three nodes for HA: use the existing two Pis plus one additional Pi 4
(8 GB preferred). Use Windows PC as storage provider via NFS/SMB CSI driver or
as a data gateway.
- Consider Pi clusters with USB SSDs for better I/O.
### Cluster Setup Steps
1. Install a lightweight Kubernetes distribution (e.g., k3s) on each Pi:
```bash
curl -sfL https://get.k3s.io | sh -
# On additional nodes
curl -sfL https://get.k3s.io | K3S_URL=https://MASTER:6443 K3S_TOKEN=HACKME sh -
```
2. Install MetalLB for load balancer support on LAN.
3. Configure persistent volumes using:
- `nfs-subdir-external-provisioner` pointing to Windows share (ensure Windows
host supports NFS or run an NFS gateway on another machine).
- Alternatively, attach individual USB drives to each Pi and use
`local-path-provisioner` for node-local storage.
4. Create Kubernetes `Secret` objects for tokens (`DISCORD_TOKEN`, etc.).
5. Define `Deployment` manifests for each service (bot, musician, librarian) and
associated `Services`.
6. Expose Discord bot ports via `NodePort` or Ingress.
7. Use `StatefulSet` if you need stable identity for the musician service (due to
local storage).
### Optimisation Tips
- Keep CPU-heavy librarian pods optionally on a beefier node; use
`nodeSelector`/`affinity` to pin workloads.
- Consider splitting the persistent storage: music on Pi A (USB disk), logs and
configs on Pi B, backups on Windows.
- For improved reliability, add at least one extra Pi for quorum and to host the
communication bridge if the bot node fails.
## Summary Checklist
1. Prepare Windows shares & tokens.
2. Configure `docker/env/*.env` using `HACKME!` templates as reference.
3. Install Docker on Pis, mount network shares.
4. Launch musician on Pi A, bot + librarian on Pi B.
5. Verify Discord functionality and API endpoints.
6. Plan Kubernetes migration when ready (k3s + MetalLB + storage provisioner).
+292
View File
@@ -0,0 +1,292 @@
# Conjurer on Docker / Proxmox
Runbook for running Conjurer as Docker containers across Proxmox VMs:
| Component | VM | Container | Port | Image |
|-----------|----|-----------|------|-------|
| **Main bot** | VM-bot | `conjurer-bot` | 5000 (Flask comm) | `docker/Dockerfile.bot` |
| **Librarian** | VM-librarian | `conjurer-librarian` | 5001 | `docker/Dockerfile.librarian` |
| **Musician** | VM-musician | `conjurer-musician` | 5000 (+ radio) | `docker/Dockerfile.musician` |
The three talk to each other over HTTP on the Proxmox LAN. Direction of calls:
```
bot --(/mp3,/get_music,/add_to_priority,...)--> musician
bot --(/query)--------------------------------> librarian
musician --(/prepped_tracks)--------------------> bot
librarian --(/conjurer results)-----------------> bot
```
Everything is configured through `CONJURER_*` environment variables (see the
`docker/env/*.env.example` files). Nothing is hardcoded to a host path anymore.
---
## 0. Prerequisites (per VM)
Create a small Linux VM in Proxmox (Debian 12 / Ubuntu 22.04+ is fine), then
install Docker:
```bash
sudo apt-get update && sudo apt-get install -y ca-certificates curl git
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker "$USER" # log out/in afterwards
```
Clone the repo on each VM (they build from it):
```bash
sudo mkdir -p /opt && cd /opt
git clone https://github.com/migatu/conjurer.git
cd conjurer
```
> All `docker compose` commands below are run **from the repo root** (`/opt/conjurer`),
> because the compose files use `context: ..` relative to `docker/`.
---
## 1. Main bot (VM-bot)
### 1a. Prepare host directories
```bash
sudo mkdir -p /srv/conjurer/data /srv/conjurer/secrets
```
### 1b. Preserve existing command history ⭐
The bot's conversation memory and settings live in JSON files. Copy them from
your current deployment (e.g. the Pi's `/home/pi/Conjurer/`) into the data
volume so the history carries over:
```bash
# run on the Pi, or scp the files across, then place them here:
sudo cp pamiec.json /srv/conjurer/data/ # AI conversation history
sudo cp pamiec_muzyki.json /srv/conjurer/data/ # music-DJ memory
sudo cp settings.json /srv/conjurer/data/ # word/cyclic reactions
sudo cp system_gpt_settings.json /srv/conjurer/data/
sudo cp accident_log.json /srv/conjurer/data/ # if present
```
If you skip this, the container starts with the (empty) template files baked
into the image and history begins fresh.
### 1c. Tokens
Drop your existing netrc (the one with `discord`, `openai`, `spotipy`,
`youtube` entries) into the secrets dir:
```bash
sudo cp ~/.netrc /srv/conjurer/secrets/.netrc
sudo chmod 600 /srv/conjurer/secrets/.netrc
```
(Alternatively skip netrc and set `DISCORD_TOKEN` / `OPENAI_API_KEY` in the env
file — those take precedence.)
### 1d. Configure and launch
```bash
cp docker/env/bot.env.example docker/env/bot.env
# edit docker/env/bot.env: set CONJURER_FILE_SERVICE / _LIBRARIAN_SERVICE to the
# other VMs' IPs, and CONJURER_API_KEY (same value on all three) if you want auth.
docker compose -f docker/compose.bot.yaml up -d --build
docker logs -f conjurer-bot
```
Look for `Extension loaded: …` for each cog and `All systems: operational`.
### 1e. Startup model: core cogs vs service-gated cogs
The bot **always** starts with the cogs that depend on nothing but itself
(administration, AI, other, latex, voice, conanjurer). Cogs that need a
sibling service are **health-gated**:
| Group | Cogs | Enabled when |
|-------|------|--------------|
| musician | `music_commands`, `radio_commands`, `file_search_commands` | `GET {FILE_SERVICE}/mp3` answers |
| librarian | `librarian_commands` | librarian answers HTTP at all |
When a service is down its cogs stay disabled (commands simply don't exist)
and the log says so. A watchdog re-checks every 5 minutes and enables the
cogs the moment the service starts answering — no bot restart needed.
A single broken cog (missing pip package, bad import) is skipped with a full
traceback in the log; it never takes the whole bot down.
### 1f. Troubleshooting a crash-looping container
`docker logs conjurer-bot` now shows the real reason (the bot logs to stdout
as well as the rotating file). The most common cases:
- **`FATAL: Discord token missing`** — the secrets mount is missing/empty or
`CONJURER_NETRC_FILE` points elsewhere. Check:
`docker inspect -f '{{json .Mounts}}' conjurer-bot | jq` and
`docker exec conjurer-bot ls -la /secrets/` (after a manual
`docker run … sleep infinity` if it crash-loops too fast).
- **Missing state files** — not fatal anymore: missing dirs are created and
missing JSON state is seeded from the repo templates baked into the image
(existing files are never overwritten). Fix the mount at your leisure.
**About files "disappearing" from `/srv/conjurer/...`:** nothing in this stack
deletes host files — the entrypoint and the bot only ever *create* missing
files. With a bind mount, `/srv/conjurer/data` **is** the live state (not an
installation staging area): don't delete it after a successful install.
If files vanished, the usual suspects are `docker compose down -v` (only
affects *named* volumes, not binds), a re-provisioned VM, or copying the files
to a different path than the one in the compose `volumes:` line — verify with
`docker inspect -f '{{json .Mounts}}' conjurer-bot`.
---
## 2. Librarian (VM-librarian)
### 2a. Mount the DOI database
The librarian checks keyword hits from Crossref against a local database of
DOI chunk files (`0_chunk.txt … N_chunk.txt`). Put that database on the VM and
point the volume at it:
```bash
sudo mkdir -p /srv/librarian/doi /srv/librarian/secrets
# copy/replicate your chunk files into /srv/librarian/doi/
```
> The old Windows path `C:\Database\chunks\` is now `CONJURER_LIBRARIAN_DB_PATH`
> (defaults to `/doi/` in the container). `CONJURER_LIBRARIAN_MAXTHREADS` (41)
> and `CONJURER_LIBRARIAN_CHUNK` (`_chunk.txt`) are configurable too.
### 2b. Configure and launch
```bash
cp docker/env/librarian.env.example docker/env/librarian.env
# edit: CONJURER_MAIN_BOT=http://BOT_VM_IP:5000, CONJURER_CROSSREF_MAILTO, CONJURER_API_KEY
docker compose -f docker/compose.librarian.yaml up -d --build
docker logs -f conjurer-librarian
```
---
## 3. Musician (VM-musician)
The web service is container-ready (`docker/Dockerfile.musician` +
`compose.musician.yaml`). It containerises the **Flask file/playlist service
only** — the Liquidsoap radio (`radio_conjurer.liq`), `script.params` and any
Samba/NFS share tooling are separate and typically stay on the host or a
dedicated setup (you'll adapt those yourself).
### 3a. Two volumes: the library and the writable state
| Mount | Container path | Holds |
|-------|---------------|-------|
| `/srv/musician/music` | `/music` (`CONJURER_MUSIC_FOLDER`) | your mp3 library (indexed/served) |
| `/srv/musician/data` | `/data` (`CONJURER_MUSICIAN_BASE`) | playlists, logs, `radio_log.log`/`persistence.log` |
```bash
sudo mkdir -p /srv/musician/music /srv/musician/data
# point /srv/musician/music at (or copy in) your mp3s
```
### 3b. Preserve existing playlists/state (optional)
If you already run the musician, copy its working playlists into the data
volume so nothing is regenerated from scratch:
```bash
sudo cp all_playlist.playlist hit.playlist request.playlist \
priority_queue.playlist playlist.json /srv/musician/data/ 2>/dev/null || true
```
The container's entrypoint (`docker/entrypoint.musician.sh`) creates any
missing playlists and touches `radio_log.log` / `persistence.log` empty so the
track-forwarding thread waits instead of crashing when the radio runs
elsewhere. It never overwrites files you copied in.
### 3c. Configure and launch
```bash
cp docker/env/musician.env.example docker/env/musician.env
# edit: CONJURER_MAIN_BOT=http://BOT_VM_IP:5000 and CONJURER_API_KEY (match the bot)
docker compose -f docker/compose.musician.yaml up -d --build
docker logs -f conjurer-musician
```
### 3d. Radio coupling (if you keep Liquidsoap separate)
The musician only forwards "now playing" to the bot by tailing the radio's
`radio_log.log` / `persistence.log`. To wire them up, have Liquidsoap write
those two files into the same `/srv/musician/data` directory (or set
`CONJURER_RADIO_LOG` / `CONJURER_PERSISTENCE_LOG` to wherever it writes). The
`/stream` page template is served from the baked-in `/app/stream.html`
(override with `CONJURER_STREAM_TEMPLATE` if you customise it).
---
## 4. Networking & auth
- Open the ports between VMs on the Proxmox LAN: **bot 5000**, **librarian 5001**,
**musician 5000** (+ radio harbor 54321 if used). A simple `ufw allow from
<lan-subnet>` per port is enough; do not expose them to the internet.
- **Auth:** set the same `CONJURER_API_KEY` in all three `*.env` files. Then
every internal call carries `X-Conjurer-Api-Key` and each service rejects
requests without it (HTTP 401). Leave it empty everywhere to disable auth
(fully backward compatible). ⚠️ Setting it on only one side breaks the link.
- The addresses point at each other by VM IP (or a DNS name). Set:
- bot: `CONJURER_FILE_SERVICE`, `CONJURER_RADIO_HARBOR`, `CONJURER_LIBRARIAN_SERVICE`
- librarian & musician: `CONJURER_MAIN_BOT`
---
## 5. Verify
```bash
# bot is up and reachable from another VM:
curl http://BOT_VM_IP:5000/conjurer # -> "ALIVE"
# librarian answers:
curl http://LIBRARIAN_VM_IP:5001/ -I # service reachable
docker ps # all containers "Up"
docker logs conjurer-bot --tail 50
```
In Discord, exercise a command that round-trips through a service (e.g. a music
search that hits the musician, or a librarian query) to confirm the wiring and
the API key.
---
## 6. Updates
```bash
cd /opt/conjurer && git pull
docker compose -f docker/compose.bot.yaml up -d --build # rebuild + restart
```
Data in `/srv/.../data` and `/doi` / `/music` volumes survives rebuilds, so
history and databases persist across updates.
---
## 7. Rollback / coexistence
- The native (Raspberry Pi / systemd) deployment is unaffected — none of the
defaults changed; the container behaviour is opt-in via `CONJURER_DATA_DIR`
and the other env vars. You can run both during migration.
- To roll back a VM: `docker compose -f docker/compose.<svc>.yaml down` and
restart the previous deployment. The JSON state in `/srv/.../data` is plain
files you can copy back to the Pi if needed.
---
## Notes on the images
- **Bot** uses the vendored `yt_dlp/` and `spotify_dl/` forks (they win over the
pip packages because `/app` is first on `sys.path`), so your patches stay
active without the old `sed` hacks from `install_main_bot.sh`.
- **Tectonic** (LaTeX `$latex` command) is installed best-effort; if the build
step fails the bot still runs, just without LaTeX. Remove that layer from
`Dockerfile.bot` if you don't need it.
- Voice needs `ffmpeg` + `libopus0` (both in the image). No microphone/pyaudio
is required — voice is received over Discord and transcribed via AssemblyAI.
@@ -0,0 +1,231 @@
# Migracja: „working copy" → „prototype"
Runbook upgrade'u działającego deploymentu bota (Raspberry Pi) z wersji
**working copy** (kod w roocie repo, bez auth między usługami, bez integracji
Conan) na wersję **prototype** (konfiguracja przez zmienne środowiskowe,
opcjonalna autoryzacja wewnętrznych wywołań HTTP, integracja Conan Exiles).
> **Najważniejsze:** upgrade jest **wstecznie kompatybilny**. Bez ustawiania
> żadnych nowych zmiennych bot działa tak jak dotąd — istniejące tokeny z
> `~/.netrc` i domyślne ścieżki są zachowane. Wszystkie nowe funkcje
> (autoryzacja API, most Conan, powiadomienia o graczach) są **opt-in**.
---
## 0. Status / warunek wstępny
Pełny kod prototype znajduje się obecnie na branchu **`proto-improvements`**.
Na `main` jest na razie sama restrukturyzacja (working copy w roocie). Zanim
zmigrujesz produkcję z `main`, zmerguj prototype do `main`
(`proto-improvements``main`) albo deployuj bezpośrednio z brancha
`proto-improvements`. Dalsza część zakłada, że prototype jest już dostępny pod
refem, który checkoutujesz w kroku 2.
**Układ deploymentu (bez zmian):**
| | Ścieżka |
|---|---|
| Klon repo | `/home/pi/conjurer` |
| Runtime bota | `/home/pi/Conjurer` |
| Virtualenv | `/home/pi/Conjurer/env` (używany przez `conjurer.service`) |
| Usługa systemd | `conjurer.service``ExecStart … /home/pi/Conjurer/bot.py` |
| Deploy | `deploy.sh` (kopiuje pliki z repo do runtime i restartuje usługę) |
---
## 1. Backup i punkt powrotu
```bash
# zapamiętaj aktualny commit (do ewentualnego rollbacku)
git -C /home/pi/conjurer rev-parse HEAD > /home/pi/conjurer_rollback_commit.txt
# snapshot runtime (config + dane)
sudo cp -a /home/pi/Conjurer /home/pi/Conjurer.bak.$(date +%Y%m%d_%H%M%S)
```
Sekrety nadal pochodzą z `~/.netrc` (Discord/OpenAI/Spotify/YouTube) — upewnij
się, że ten plik istnieje i jest aktualny. Migracja go nie dotyka.
---
## 2. Pobranie kodu prototype
```bash
cd /home/pi/conjurer
git fetch --all
git checkout main && git pull # gdy prototype jest już w main
# albo, do czasu merge: git checkout proto-improvements && git pull
```
---
## 3. Instalacja nowych zależności
Wersja prototype dodaje do `requirements_bot.txt` dwa pakiety używane przez most
Conan: **`aiomcrcon`** i **`asyncssh`**.
```bash
source /home/pi/Conjurer/env/bin/activate
python3 -m pip install -r /home/pi/conjurer/requirements_bot.txt
deactivate
```
> Nawet bez tych pakietów bot się uruchomi — importy w module Conan są osłonięte
> (`try/except ImportError`), a integracja pozostaje uśpiona. Instalacja jest
> potrzebna tylko jeśli faktycznie używasz mostu Conan.
---
## 4. (Opcjonalnie) Konfiguracja zmiennych środowiskowych
Wersja prototype czyta konfigurację ze zmiennych środowiskowych z fallbackiem na
dotychczasowe wartości. **Pomiń ten krok dla zwykłego upgrade'u** — domyślne
ścieżki i `~/.netrc` wystarczą. Ustaw zmienne tylko gdy włączasz nową funkcję.
### 4a. Plik środowiskowy dla systemd
```bash
sudo tee /home/pi/Conjurer/conjurer.env >/dev/null <<'EOF'
# --- Autoryzacja wewnętrznych wywołań HTTP (opcjonalne) ---
# Ten sam klucz MUSI być ustawiony na bocie, musicianie i librarianie.
CONJURER_API_KEY=
# --- Adresy usług wewnętrznych (domyślne wartości jak dotąd) ---
#CONJURER_FILE_SERVICE=http://192.168.1.15:5000
#CONJURER_RADIO_HARBOR=http://192.168.1.15:54321
#CONJURER_LIBRARIAN_SERVICE=http://192.168.1.192:5001
# --- Most Conan Exiles (opcjonalne; puste = wyłączone) ---
#CONAN_GM_ROLE_ID=0
#CONAN_RCON_HOST=
#CONAN_RCON_PORT=25575
#CONAN_RCON_PASSWORD=
#CONAN_CHAT_CHANNEL_ID=0
#CONAN_EVENTS_CHANNEL_ID=0
# Powiadomienia o wejściu gracza — ustaw id kanału, by włączyć:
#CONAN_JOIN_CHANNEL_ID=0
#CONAN_PLAYER_POLL_SECONDS=60
#CONAN_LOG_MODE=local
#CONAN_LOG_PATH=
EOF
sudo chmod 600 /home/pi/Conjurer/conjurer.env
```
### 4b. Podłączenie pliku do usługi
Dodaj `EnvironmentFile` do sekcji `[Service]` w `conjurer.service`
(`/etc/systemd/system/conjurer.service`):
```ini
[Service]
Type=simple
EnvironmentFile=/home/pi/Conjurer/conjurer.env
ExecStart=/home/pi/Conjurer/env/bin/python3 /home/pi/Conjurer/bot.py
Restart=on-abort
```
```bash
sudo systemctl daemon-reload
```
> `conjurer.service` w repo również warto zaktualizować o tę linię, ale `deploy.sh`
> **nie** nadpisuje jednostki systemd przy każdym deployu — wpis robisz raz, ręcznie.
---
## 5. Deploy i restart
```bash
cd /home/pi/conjurer
./deploy.sh # kopiuje m.in. bot.py oraz conanjurer_commands/_functions.py do runtime
```
`deploy.sh` na końcu sam wykonuje `systemctl restart conjurer.service`. Jeśli
zmieniałeś jednostkę systemd w kroku 4b, wcześniej zrób `daemon-reload`.
---
## 6. Weryfikacja
```bash
journalctl -u conjurer.service -f
```
Czego szukać w logu (`/home/pi/Conjurer` → plik logu również):
- `Loading … module done` dla kolejnych rozszerzeń, w tym **`Loading conanjurer commands module done`**
- Gdy most Conan nieskonfigurowany: `Conan: log watch disabled (not configured)`
oraz `Conan: player-join notifications disabled (no channel or RCON)` — to
oczekiwane, integracja jest uśpiona
- `All systems: operational`
Szybki test funkcjonalny: bot łączy się z Discordem, dotychczasowe komendy
działają, radio/biblioteka odpowiadają jak wcześniej.
---
## 7. Zmiany zachowania, o których warto wiedzieć
| Obszar | Working copy | Prototype |
|---|---|---|
| Konfiguracja | zahardkodowana per-platforma | env-vary z fallbackiem na stare wartości |
| Tokeny | tylko `~/.netrc` | env-var → fallback `~/.netrc` (stare działa) |
| Wewnętrzne HTTP (music/radio/librarian) | bez nagłówków | wysyła `X-Conjurer-Api-Key`, **gdy** `CONJURER_API_KEY` ustawione |
| Endpointy przychodzące bota | bez weryfikacji | `_authorize_request()` zwraca 401 przy złym kluczu (no-op gdy klucz pusty) |
| Pętla bota | wątki + `join()` | pojedyncza pętla asyncio z czystym shutdownem |
| Moduł Conan | obecny, **nieładowany** (miał SyntaxError) | naprawiony i ładowany; uśpiony bez konfiguracji |
---
## 8. Włączanie funkcji opcjonalnych
### 8a. Autoryzacja wewnętrznych wywołań HTTP
Ustaw **ten sam** `CONJURER_API_KEY` na **wszystkich** usługach: bocie,
musicianie (`conjurer_musician`) i librarianie. Po ustawieniu:
- bot dokleja nagłówek do wywołań do file-service/radia/biblioteki,
- usługi odrzucają (401) żądania bez poprawnego klucza.
⚠️ Ustawienie klucza tylko po jednej stronie zepsuje komunikację (401). Albo
wszędzie, albo nigdzie.
### 8b. Most Conan Exiles
Ustaw `CONAN_RCON_HOST` + `CONAN_RCON_PASSWORD` (komendy GM `say/players/kick/
rcon/ogłoś`) oraz, dla mirrorowania czatu/zdarzeń, `CONAN_LOG_PATH`
(+ `CONAN_CHAT_CHANNEL_ID`/`CONAN_EVENTS_CHANNEL_ID`).
### 8c. Powiadomienia o wejściu gracza
Ustaw **`CONAN_JOIN_CHANNEL_ID`** na id kanału Discord. Funkcja odpytuje RCON
`listplayers` co `CONAN_PLAYER_POLL_SECONDS` i ogłasza nowych graczy. Pozostaw
`0`, aby trzymać ją wyłączoną.
---
## 9. Rollback
```bash
cd /home/pi/conjurer
git checkout "$(cat /home/pi/conjurer_rollback_commit.txt)"
./deploy.sh
sudo systemctl restart conjurer.service
```
W razie potrzeby przywróć snapshot runtime z `/home/pi/Conjurer.bak.*`. Nowe
zależności (`aiomcrcon`, `asyncssh`) mogą zostać w venv — nie przeszkadzają
starszej wersji.
---
## 10. Migracja usługi musician (jeśli prowadzisz radio)
Stabilny wariant `musician_old` (zahardkodowane adresy) został zastąpiony przez
`conjurer_musician` (env-driven, domyślnie `127.0.0.1`). Jeśli bot i musician są
na **różnych** hostach, ustaw na musicianie:
```bash
CONJURER_MAIN_BOT=http://<ip-bota>:5000 # gdzie bot serwuje /prepped_tracks
CONJURER_API_KEY=<ten sam sekret co bot> # jeśli włączasz auth (8a)
CONJURER_MUSIC_FOLDER=/home/pi/MediaShare/mp3
```
oraz na bocie `CONJURER_FILE_SERVICE=http://<ip-musiciana>:5000`. Szczegóły
auth/kontraktu — patrz opis usługi `conjurer_musician`.
+71
View File
@@ -0,0 +1,71 @@
import os
import discord
from discord.ext import commands
import file_search_functions
class FileSelectView(discord.ui.View):
def __init__(self, files):
super().__init__(timeout=60)
options = [discord.SelectOption(label=os.path.basename(f)[:100], description=f, value=f) for f in files]
self.select = discord.ui.Select(
placeholder="Select files...",
min_values=1,
max_values=len(options),
options=options
)
self.select.callback = self.select_callback
self.add_item(self.select)
self.selected = []
async def select_callback(self, interaction: discord.Interaction):
# Store selected file paths
self.selected = self.select.values
await interaction.response.defer()
@discord.ui.button(label="Accept", style=discord.ButtonStyle.green)
async def accept_callback(self, interaction: discord.Interaction, button: discord.ui.Button):
# Handle acceptance
if not self.selected:
await interaction.response.send_message("No files selected.", ephemeral=True)
return
# Publish selected files
links = file_search_functions.publish(self.selected)
# Display the returned links
await interaction.response.edit_message(content="Published Links:\n" + "\n".join(links), view=None)
@discord.ui.button(label="Cancel", style=discord.ButtonStyle.red)
async def cancel_callback(self, interaction: discord.Interaction, button: discord.ui.Button):
# Handle cancellation
await interaction.response.edit_message(content="Operation cancelled.", view=None)
class FileSearchCog(commands.Cog):
"""Cog providing a file search and publish command."""
def __init__(self, bot: commands.Bot):
self.bot = bot
@commands.has_any_role("Jarl", "Thane")
@commands.command(name="tajna_biblioteka_inkwizycji")
async def findfiles(self, ctx: commands.Context, entries: int, *, keywords: str):
"""
Search for files matching keywords and publish selected ones.
Usage: !findfiles <entries 1-10> <keywords>
"""
if entries < 1 or entries > 10:
await ctx.send("❌ Entries must be between 1 and 10.")
return
keyword_list = keywords.split()
files = file_search_functions.find_matches(entries, keyword_list)
if not files:
await ctx.send("🔍 No matching files found.")
return
view = FileSelectView(files)
file_list = "\n".join(f"{i+1}. {f}" for i, f in enumerate(files))
await ctx.send(
f"🔍 Found files (select and click Accept or Cancel):\n{file_list}",
view=view
)
async def setup(bot: commands.Bot):
await bot.add_cog(FileSearchCog(bot))
+31
View File
@@ -0,0 +1,31 @@
import requests
# Base URL for Flask service
BASE_URL = 'http://192.168.1.15:5000'
def find_matches(entries, keywords):
"""
Call the get_share_list endpoint.
entries: int (1-10)
keywords: list of strings
Returns: list of file paths
"""
payload = {'entries': entries, 'keywords': keywords}
response = requests.post(f"{BASE_URL}/get_share_list", json=payload)
response.raise_for_status()
data = response.json()
return data.get('files', [])
def publish(file_paths):
"""
Call the get_share_links endpoint.
file_paths: list of file path strings
Returns: list of published URLs
"""
payload = {'file_paths': file_paths}
response = requests.post(f"{BASE_URL}/get_share_links", json=payload)
response.raise_for_status()
data = response.json()
return data.get('links', [])
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/env bash
# "apt dla fontów": Google Fonts ZIP → GitHub (ofl RAW) → fonts.gstatic.com → mirrory.
# System-wide install do /usr/local/share/fonts/fontget/<rodzina> + fc-cache.
# Usage: sudo ./fontget.sh "IM Fell English SC" "Cinzel Decorative" "EB Garamond"
set -Eeuo pipefail
SUDO=sudo; [ "$(id -u)" -eq 0 ] && SUDO=
log(){ echo -e "[+] $*"; }
warn(){ echo -e "[WARN] $*" >&2; }
die(){ echo -e "[FATAL] $*" >&2; exit 1; }
need(){ command -v "$1" >/dev/null 2>&1 || die "Missing: $1"; }
need fc-cache
command -v wget >/dev/null 2>&1 || command -v curl >/dev/null 2>&1 || die "Need wget/curl"
command -v unzip >/dev/null 2>&1 || warn "unzip missing — ZIP installs limited"
command -v file >/dev/null 2>&1 || warn "file missing — MIME checks limited"
command -v fc-list >/dev/null 2>&1 || die "Missing: fc-list"
fetch(){ local u="$1" o="$2"; if command -v wget >/dev/null 2>&1; then wget -O "$o" --https-only --no-verbose "$u"; else curl -L --fail --show-error --output "$o" "$u"; fi; }
slug(){ echo "$1" | tr '[:upper:]' '[:lower:]' | sed -E 's/[^a-z0-9]+/-/g; s/^-+|-+$//g'; }
present(){ fc-list -f '%{family}\n' | tr ',' '\n' | sed 's/^[[:space:]]*//; s/[[:space:]]*$//' | tr '[:upper:]' '[:lower:]' | sort -u | grep -Eiq "$1"; }
install_dir="/usr/local/share/fonts/fontget"; $SUDO mkdir -p "$install_dir"
is_zip(){ command -v file >/dev/null 2>&1 && file "$1" | grep -qi zip; }
gf_zip_try(){
local name="$1" outdir="$2" enc zip
enc="$(python3 - <<PY
import urllib.parse,sys
print(urllib.parse.quote(sys.argv[1]))
PY
"$name")" || enc="$(echo "$name" | sed 's/ /%20/g')"
zip="$(mktemp /tmp/fontget.XXXXXX.zip)"
if fetch "https://fonts.google.com/download?family=${enc}" "$zip"; then
if is_zip "$zip"; then $SUDO mkdir -p "$outdir"; $SUDO unzip -o -q "$zip" -d "$outdir" || true; rm -f "$zip"; return 0; fi
fi
rm -f "$zip" || true; return 1
}
gh_ofl_try(){
local family="$1" outdir="$2" html
html="$(mktemp /tmp/ofl.$(slug "$family").html)"
if fetch "https://github.com/google/fonts/tree/main/ofl/$(slug "$family")" "$html"; then
mapfile -t paths < <(grep -Eo '/google/fonts/blob/main/ofl/'"$(slug "$family")"'/[^"]+\.(ttf|otf)' "$html" | sed 's|/blob/|/raw/|g' | sort -u)
[ "${#paths[@]}" -gt 0 ] || { rm -f "$html"; return 1; }
$SUDO mkdir -p "$outdir"
for p in "${paths[@]}"; do fetch "https://github.com$p" "$outdir/$(basename "$p")" || true; done
rm -f "$html"; return 0
fi
rm -f "$html"; return 1
}
gstatic_try(){ local out="$2"; shift 2; for u in "$@"; do fetch "$u" "$out" && [ -s "$out" ] && return 0; done; return 1; }
report(){ local label="$1" re="$2"; if present "$re"; then printf "RESULT: %-24s : OK\n" "$label"; else printf "RESULT: %-24s : FAIL\n" "$label"; fi }
if [ $# -lt 1 ]; then echo "Usage: $0 <Font Family> [Another ...]"; exit 1; fi
for family in "$@"; do
echo; log "=== $family ==="
fam_slug="$(slug "$family")"
outdir="$install_dir/$fam_slug"
$SUDO mkdir -p "$outdir"
case "$fam_slug" in
cinzel-decorative|cinzel|cinzel-decorative-black)
fetch "https://github.com/google/fonts/raw/main/ofl/cinzeldecorative/CinzelDecorative-Black.ttf" "$outdir/CinzelDecorative-Black.ttf" || warn "Cinzel fetch failed"
;;
im-fell-english-sc)
gstatic_try "$fam_slug" "$outdir/IMFeENsc.ttf" \
"https://fonts.gstatic.com/s/imfellenglishsc/v7/h3Tn6yWfw4b5qaLD1RWvz5ATixNthKRRR1XVH3rJNiw.ttf" \
"https://fonts.gstatic.com/s/imfellenglishsc/v6/h3Tn6yWfw4b5qaLD1RWvz5ATixNthKRRR1XVH3rJNiw.ttf" \
|| { fetch "https://github.com/google/fonts/raw/main/ofl/imfellenglishsc/IMFeENsc28P.ttf" "$outdir/IMFeENsc.ttf" || true
[ -s "$outdir/IMFeENsc.ttf" ] || fetch "https://github.com/google/fonts/raw/main/ofl/imfellenglishsc/IMFellEnglishSC-Regular.ttf" "$outdir/IMFeENsc.ttf" || true
[ -s "$outdir/IMFeENsc.ttf" ] || fetch "https://www.wfonts.com/download/data/2016/06/14/im-fell-english-sc/IMFeENsc28P.ttf" "$outdir/IMFeENsc.ttf" || true
[ -s "$outdir/IMFeENsc.ttf" ] || fetch "https://www.1001freefonts.com/d/6800/IMFeENsc28P.ttf" "$outdir/IMFeENsc.ttf" || true; }
;;
eb-garamond|ebgaramond)
# Najpierw apt (jeśli chcesz: sudo apt install fonts-ebgaramond), tu tylko web:
gf_zip_try "$family" "$outdir" || gh_ofl_try "$family" "$outdir" || true
;;
*)
gf_zip_try "$family" "$outdir" || gh_ofl_try "$family" "$outdir" || true
;;
esac
$SUDO find "$outdir" -type f -size 0 -print -delete || true
$SUDO find "$outdir" -type f \( -name "*.ttf" -o -name "*.otf" \) -exec chmod 0644 {} \; || true
$SUDO find "$outdir" -type d -exec chmod 0755 {} \; || true
$SUDO fc-cache -f -v >/dev/null || true
case "$fam_slug" in
cinzel* ) report "Cinzel Decorative" '^cinzel decorative( |$)' ;;
im-fell-english-sc ) report "IM Fell English SC" '^im fell english sc( |$)' ;;
eb-garamond* ) report "EB Garamond" '^eb garamond( |$)' ;;
* ) report "$family" "$(echo "$family" | tr '[:upper:]' '[:lower:]' | sed 's/ /.* /g')" ;;
esac
done
echo; log "All done."
Binary file not shown.
View File
View File
Executable → Regular
View File
Executable → Regular
View File
+280
View File
@@ -0,0 +1,280 @@
# latex_commands.py
# -*- coding: utf-8 -*-
from __future__ import annotations
import io
import logging
from typing import Optional
import discord
from discord import app_commands
from discord.ext import commands
from constants import (
ALLOWED_ROLES,
GUILD_ID,
LATEX_MAX_ATTACH_MB,
LATEX_MAX_COMPILE_SECONDS,
LATEX_MAX_ZIP_MB,
LATEX_TEX_ENGINE,
OPENAI_MODEL,
)
from latex_functions import (
compile_single_tex_bytes,
compile_zip_to_zip,
is_safe_asset_name,
is_safe_tex_name,
)
async def _collect_attachments_bytes(attachments, max_mb: int):
"""
Zwraca słownik {filename: bytes} dla wszystkich załączników, do limitu MB.
Nie podbija wyjątków z .read() zostawiamy to wyżej; tu zwracamy, co się udało.
"""
out = {}
for a in attachments or []:
# limit rozmiaru
if (getattr(a, "size", 0) or 0) > max_mb * 1024 * 1024:
continue
data = await a.read()
out[a.filename] = data
return out
class LatexModule(commands.Cog):
def __init__(self, bot: commands.Bot, logger_name: str):
self.bot = bot
self.logger = logging.getLogger(logger_name)
self.logger_name = logger_name
# -------- /tex (PDF + diagnoza; log -> debug logger) --------
@commands.hybrid_command(
nsfw=False,
name="tex",
description="Kompiluje załączone .tex; PDF-y odsyła. Błędy: diagnoza z OpenAI.",
guild=None if GUILD_ID is None else discord.Object(id=GUILD_ID),
)
@commands.has_any_role(*ALLOWED_ROLES)
async def tex(
self, ctx: commands.Context, include_source_excerpt: Optional[bool] = True
):
self.logger.info("LaTeX command invoked by %s", ctx.author)
ch = ctx.message.channel
# ZBIERZ WSZYSTKIE ZAŁĄCZNIKI
async with ch.typing():
all_bytes = await _collect_attachments_bytes(
ctx.message.attachments, LATEX_MAX_ATTACH_MB
)
if not all_bytes:
return await ctx.reply(
f"Dołącz .tex (≤{LATEX_MAX_ATTACH_MB} MiB).", mention_author=False
)
# PODZIEL NA GŁÓWNE .TEX i WSPIERAJĄCE
tex_names = [
n
for n in all_bytes.keys()
if n.lower().endswith(".tex") and is_safe_tex_name(n)
]
if not tex_names:
return await ctx.reply(
f"Dołącz .tex (≤{LATEX_MAX_ATTACH_MB} MiB).", mention_author=False
)
support_names = [
n
for n in all_bytes.keys()
if n not in tex_names and is_safe_asset_name(n)
]
files_to_send = []
parts = []
for tex_name in tex_names:
try:
main_bytes = all_bytes[tex_name]
support = [
(n, all_bytes[n])
for n in support_names
if n != tex_name and is_safe_asset_name(n)
]
# (ważne) do supportów dopuszczamy również **inne .tex** dla \input/\include
other_tex = [
(n, all_bytes[n])
for n in tex_names
if n != tex_name and is_safe_tex_name(n)
]
support.extend(other_tex)
res = await compile_single_tex_bytes(
main_bytes,
tex_name,
LATEX_TEX_ENGINE,
LATEX_MAX_COMPILE_SECONDS,
logger="discord",
support_files=support,
)
# Twoja dotychczasowa obsługa success/fail przykładowo:
if res["ok"] and res["pdf_bytes"]:
b = io.BytesIO(res["pdf_bytes"])
b.seek(0)
b.name = res["pdf_name"]
files_to_send.append(discord.File(b, filename=res["pdf_name"]))
parts.append(f"✅ `{tex_name}` → `{res['pdf_name']}`")
else:
parts.append(
f"❌ `{tex_name}` — błąd kompilacji.)"
)
# Tutaj wywołujesz swoją ścieżkę do OpenAI diagnozy, którą ogarnęliśmy wcześniej.
except Exception:
# let it crash — pełny stack w logu i re-raise (Discord.py to pokaże sensownie)
logging.getLogger("discord").exception(
"Unhandled exception while compiling %s", tex_name
)
raise
content = (
f"**LaTeX** — {len(tex_names)} plik(ów). Model: `{OPENAI_MODEL}`\n\n"
+ "\n\n".join(parts)
)
await ctx.reply(
content if len(content) < 1900 else content[:1900] + "",
files=files_to_send,
mention_author=False,
)
# -------- /latexclean (PDF only; log -> debug logger) --------
@commands.hybrid_command(
nsfw=False,
name="latexclean",
description="Kompiluje .tex i odsyła TYLKO PDF (log w debug).",
guild=None if GUILD_ID is None else discord.Object(id=GUILD_ID),
)
@commands.has_any_role(*ALLOWED_ROLES)
async def latexclean(self, ctx: commands.Context):
ch = ctx.message.channel
async with ch.typing():
all_bytes = await _collect_attachments_bytes(
ctx.message.attachments, LATEX_MAX_ATTACH_MB
)
if not all_bytes:
return await ctx.reply(
f"Dołącz .tex (≤{LATEX_MAX_ATTACH_MB} MiB).", mention_author=False
)
# PODZIEL NA GŁÓWNE .TEX i WSPIERAJĄCE
tex_names = [
n
for n in all_bytes.keys()
if n.lower().endswith(".tex") and is_safe_tex_name(n)
]
if not tex_names:
return await ctx.reply(
f"Dołącz .tex (≤{LATEX_MAX_ATTACH_MB} MiB).", mention_author=False
)
support_names = [
n
for n in all_bytes.keys()
if n not in tex_names and is_safe_asset_name(n)
]
files_to_send = []
parts = []
for tex_name in tex_names:
try:
main_bytes = all_bytes[tex_name]
support = [
(n, all_bytes[n])
for n in support_names
if n != tex_name and is_safe_asset_name(n)
]
# (ważne) do supportów dopuszczamy również **inne .tex** dla \input/\include
other_tex = [
(n, all_bytes[n])
for n in tex_names
if n != tex_name and is_safe_tex_name(n)
]
support.extend(other_tex)
res = await compile_single_tex_bytes(
main_bytes,
tex_name,
LATEX_TEX_ENGINE,
LATEX_MAX_COMPILE_SECONDS,
logger="discord",
support_files=support,
)
# Twoja dotychczasowa obsługa success/fail przykładowo:
if res["ok"] and res["pdf_bytes"]:
b = io.BytesIO(res["pdf_bytes"])
b.seek(0)
b.name = res["pdf_name"]
files_to_send.append(discord.File(b, filename=res["pdf_name"]))
parts.append(f"✅ `{tex_name}` → `{res['pdf_name']}`")
else:
parts.append(
f"❌ `{tex_name}` — błąd kompilacji.)"
)
# Tutaj wywołujesz swoją ścieżkę do OpenAI diagnozy, którą ogarnęliśmy wcześniej.
except Exception:
# let it crash — pełny stack w logu i re-raise (Discord.py to pokaże sensownie)
logging.getLogger("discord").exception(
"Unhandled exception while compiling %s", tex_name
)
raise
await ctx.reply(
"**LaTeX clean** — " + ", ".join(parts),
files=files_to_send,
mention_author=False,
)
# -------- /texbatch (ZIP -> ZIP) --------
@app_commands.command(
name="texbatch",
description="Wyślij ZIP z .tex → odeślę ZIP z PDF-ami (tylko udane).",
)
@app_commands.describe(archive=f"Archiwum .zip (max ~{LATEX_MAX_ZIP_MB} MiB)")
@app_commands.checks.has_any_role(*ALLOWED_ROLES)
async def texbatch(
self, interaction: discord.Interaction, archive: discord.Attachment
):
await interaction.response.defer()
if not archive or not archive.filename.lower().endswith(".zip"):
return await interaction.followup.send(
"Dołącz ZIP (.zip) z plikami .tex.", ephemeral=True
)
if archive.size > LATEX_MAX_ZIP_MB * 1024 * 1024:
return await interaction.followup.send(
f"ZIP > {LATEX_MAX_ZIP_MB} MiB — za duży.", ephemeral=True
)
data = await archive.read()
out_zip_bytes, failed = await compile_zip_to_zip(
data, LATEX_TEX_ENGINE, LATEX_MAX_COMPILE_SECONDS, logger=self.logger_name
)
if not out_zip_bytes and failed:
return await interaction.followup.send(
"Brak PDF-ów. " + "; ".join(failed[:10]), ephemeral=True
)
bio = io.BytesIO(out_zip_bytes)
bio.seek(0)
bio.name = "compiled_pdfs.zip"
if failed:
self.logger.debug("BATCH failed: %s", ", ".join(failed))
await interaction.followup.send(
"✅ Gotowe (ZIP w załączniku)."
+ (f" ❌ Błędy: {len(failed)}" if failed else ""),
files=[discord.File(bio, filename=bio.name)],
)
async def setup(bot):
logger = logging.getLogger("discord")
await bot.add_cog(LatexModule(bot, logger_name="discord"))
logger.info("Loading kinky latex module done")
+179
View File
@@ -0,0 +1,179 @@
import asyncio
import logging
import re
import shutil
import tempfile
import zipfile
from pathlib import Path
from typing import Dict, Iterable, List, Optional, Tuple
# ===== Bezpieczeństwo nazw/rozszerzeń =====
SAFE_TEX_NAME = re.compile(r"^[\w\-. ]+\.tex$", re.IGNORECASE)
SAFE_ASSET_NAME = re.compile(
r"^[\w\-. /]+?\.(tex|png|jpg|jpeg|pdf|svg|eps|bmp|gif|sty|cls|bib|bst|bbx|cbx|def|cfg)$",
re.IGNORECASE,
)
def is_safe_tex_name(name: str) -> bool:
return bool(SAFE_TEX_NAME.match(name or ""))
def is_safe_asset_name(name: str) -> bool:
return bool(SAFE_ASSET_NAME.match(name or ""))
# ===== Uruchamianie kompilatora =====
async def _run(cmd: List[str], cwd: Path, timeout_sec: int, logger_name: Optional[str]) -> Tuple[int, str]:
"""Uruchamia proces kompilatora w cwd, zwraca (rc, sklejone_stdout_stderr)."""
logger = logging.getLogger(logger_name) if logger_name else logging.getLogger()
logger.debug("RUN %s (cwd=%s, timeout=%s)", " ".join(cmd), cwd, timeout_sec)
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
cwd=str(cwd),
)
try:
out = await asyncio.wait_for(proc.communicate(), timeout=timeout_sec)
except asyncio.TimeoutError:
try:
proc.kill()
finally:
pass
return 124, "TIMEOUT: compiler took too long"
rc = proc.returncode
log = (out[0] or b"").decode("utf-8", errors="ignore")
logger.debug("RC=%s, LOG TAIL:\n%s", rc, log[-1500:])
return rc, log
def _build_cmd(tex_engine: str, main_tex: str) -> List[str]:
"""Buduje komendę kompilatora; domyślnie tectonic/pdflatex/xelatex."""
if tex_engine.lower() == "tectonic":
# Tectonic sam robi wielofazowość, ale i tak uruchamiamy 2x (bezpiecznie dla bib/refs)
return ["tectonic", "--keep-intermediates", "--synctex", main_tex]
elif tex_engine.lower() in ("pdflatex", "xelatex", "lualatex"):
return [tex_engine, "-interaction=nonstopmode", "-halt-on-error", main_tex]
else:
# fallback traktujemy jak pdflatex
return ["pdflatex", "-interaction=nonstopmode", "-halt-on-error", main_tex]
async def run_latex_two_passes(
tex_filename: str, workdir: Path, tex_engine: str, timeout_sec: int, logger: Optional[str] = None
) -> Tuple[bool, str, Path]:
"""
Uruchamia kompilację 2 razy (bezpieczeństwo referencji/bib).
Zwraca: (ok, log_text, pdf_path).
"""
logger_obj = logging.getLogger(logger) if logger else logging.getLogger()
logger_obj.info("Compiling %s with %s engine two passes", tex_filename, tex_engine)
logs: List[str] = []
pdf_path = workdir / (Path(tex_filename).stem + ".pdf")
# PASS 1
rc1, log1 = await _run(_build_cmd(tex_engine, tex_filename), workdir, timeout_sec, logger)
logs.append(log1)
if rc1 != 0:
return False, "".join(logs), pdf_path
# PASS 2 (często już nic nie robi, ale nie szkodzi)
rc2, log2 = await _run(_build_cmd(tex_engine, tex_filename), workdir, timeout_sec, logger)
logs.append(log2)
ok = (rc2 == 0) and pdf_path.exists()
return ok, "".join(logs), pdf_path
# ===== Kompilacje: pojedynczy TEX + ZIP =====
async def compile_single_tex_bytes(
tex_bytes: bytes,
filename: str,
tex_engine: str,
timeout_sec: int,
logger: Optional[str] = None,
support_files: Optional[Iterable[Tuple[str, bytes]]] = None,
) -> Dict[str, Optional[object]]:
"""
Kompiluje pojedynczy .tex + dowolne pliki pomocnicze (obrazy, sty/cls/bib, inne .tex).
Każdy plik zapisujemy w katalogu roboczym widocznym dla kompilatora.
"""
log = logging.getLogger(logger) if logger else logging.getLogger()
log.info("Compiling %s with %s engine single tex", filename, tex_engine)
if not is_safe_tex_name(filename):
return {"ok": False, "pdf_bytes": None, "log_text": "Invalid .tex filename", "pdf_name": None}
with tempfile.TemporaryDirectory(prefix="latex_one_") as td:
wd = Path(td)
(wd / Path(filename).name).write_bytes(tex_bytes)
# Zapisz wszystkie supporty obok głównego pliku
if support_files:
for fname, data in support_files:
if not fname or not is_safe_asset_name(fname):
continue
# spłaszczamy do nazwy bazowej (bez podkatalogów z zewn. źródeł)
(wd / Path(fname).name).write_bytes(data)
ok, log_text, pdf_path = await run_latex_two_passes(Path(filename).name, wd, tex_engine, timeout_sec, logger)
log.info("LaTeX compile finished: %s", "OK" if ok else "FAIL")
log.info("Log text:\n%s", log_text[-2000:]) # Log last 2000 chars
log.info("PDF path: %s", pdf_path)
if ok and pdf_path.exists():
return {
"ok": True,
"pdf_bytes": pdf_path.read_bytes(),
"log_text": log_text,
"pdf_name": pdf_path.name,
}
return {"ok": False, "pdf_bytes": None, "log_text": log_text, "pdf_name": pdf_path.name}
async def compile_zip_to_zip(
zip_bytes: bytes, tex_engine: str, timeout_sec: int, logger: Optional[str] = None
) -> Tuple[bytes, List[str]]:
"""
Rozpakowuje ZIP, znajduję wszystkie .tex i dla KAŻDEGO tworzy osobny katalog roboczy
ze skopiowanym CAŁYM drzewem, by zachować ścieżki względne obrazków/plików.
Zwraca (zip_pdf_bytes, lista_rel_sciezek_które_padły).
"""
log = logging.getLogger(logger) if logger else logging.getLogger()
failed: List[str] = []
out_pdf_paths: List[Path] = []
with tempfile.TemporaryDirectory(prefix="latex_zip_") as td:
td_path = Path(td)
src_root = td_path / "src"
src_root.mkdir(parents=True, exist_ok=True)
# 1) Rozpakuj
zpath = td_path / "in.zip"
zpath.write_bytes(zip_bytes)
with zipfile.ZipFile(zpath, "r") as zf:
zf.extractall(src_root)
# 2) Zbierz .tex
tex_files = [p for p in src_root.rglob("*.tex") if is_safe_tex_name(p.name)]
if not tex_files:
return b"", ["No .tex files found in archive."]
# 3) Kompiluj każdy .tex w izolacji (kopiujemy całe drzewo do osobnego workdiru)
for tex_path in tex_files:
rel_tex = tex_path.relative_to(src_root).as_posix()
work = td_path / f"build_{tex_path.stem}"
try:
shutil.copytree(src_root, work / "src")
ok, log_text, pdf = await run_latex_two_passes(rel_tex, work / "src", tex_engine, timeout_sec, logger)
if ok and pdf.exists():
out_pdf_paths.append(pdf)
else:
failed.append(rel_tex)
log.info("LaTeX FAIL %s\n%s", rel_tex, (log_text or "")[-2000:])
except Exception:
# let it crash log + re-raise (to zobaczy wyższa warstwa/discord.py)
log.exception("Exception while compiling %s", rel_tex)
raise
# 4) Spakuj wyjściowe PDF-y
out_zip = td_path / "compiled_pdfs.zip"
with zipfile.ZipFile(out_zip, "w", compression=zipfile.ZIP_DEFLATED) as zf:
for pdf in out_pdf_paths:
zf.write(pdf, arcname=pdf.name)
return out_zip.read_bytes(), failed
+35 -16
View File
@@ -8,13 +8,16 @@ from queue import Empty
import discord
import pdf2image
import fitz
import PyPDF2
import requests
from discord.ext import commands, tasks
from ai_functions import handle_response
from communication_subroutine import IN_COMM_Q, OUT_COMM_Q, QueryControl
from constants import DIR_PATH_SADOX, LIBRARIAN_SERVICE_ADDRESS, SEND_QUERY
from constants import DIR_PATH_SADOX, LIBRARIAN_SERVICE_ADDRESS, SEND_QUERY, service_headers
SERVICE_HEADERS = service_headers()
class DataModule(commands.Cog):
@@ -53,20 +56,34 @@ class DataModule(commands.Cog):
filename = res[random.randrange(0, len(res) - 1)]
# select random page
file = open(DIR_PATH_SADOX + filename, "rb")
readpdf = PyPDF2.PdfReader(file)
totalpages = len(readpdf.pages)
# trunk-ignore(bandit/B311)
page = random.randrange(1, totalpages)
# convert page to image
image = pdf2image.convert_from_path(
DIR_PATH_SADOX + filename, first_page=page, last_page=page
)
byte_io_stream = io.BytesIO()
image[0].save(byte_io_stream, "JPEG")
byte_io_stream.seek(0)
byte_io_stream.name = "image.jpg"
file = discord.File(byte_io_stream)
await ctx.send(file=file)
if True:
doc = fitz.open(DIR_PATH_SADOX + filename)
totalpages = len(doc)
# trunk-ignore(bandit/B311)
page_index = random.randrange(0, totalpages)
page = doc.load_page(page_index)
mat = fitz.Matrix(2.0, 2.0) # powiększenie
pix = page.get_pixmap(matrix=mat, alpha=False)
byte_io_stream = io.BytesIO(pix.tobytes("png"))
byte_io_stream.seek(0)
byte_io_stream.name = "image.png"
await ctx.send(file=discord.File(byte_io_stream))
else: #legacy
readpdf = PyPDF2.PdfReader(file)
totalpages = len(readpdf.pages)
# trunk-ignore(bandit/B311)
page = random.randrange(1, totalpages)
# convert page to image
image = pdf2image.convert_from_path(
DIR_PATH_SADOX + filename, first_page=page, last_page=page
)
byte_io_stream = io.BytesIO()
image[0].save(byte_io_stream, "JPEG")
byte_io_stream.seek(0)
byte_io_stream.name = "image.jpg"
file = discord.File(byte_io_stream)
await ctx.send(file=file)
self.logger.info("Get sadox completed")
@tasks.loop(seconds=3)
@@ -150,6 +167,7 @@ class DataModule(commands.Cog):
requests.post,
f"{LIBRARIAN_SERVICE_ADDRESS}{SEND_QUERY}",
json=json_query,
headers=SERVICE_HEADERS,
timeout=360,
)
await ctx.send(
@@ -222,7 +240,7 @@ class DataModule(commands.Cog):
global MESSAGE_TABLE # pylint: disable=global-statement
result, MESSAGE_TABLE = await handle_response(
prompt, vykidailo, bartender, MESSAGE_TABLE, username, "CONVERSATION"
prompt, vykidailo, bartender, MESSAGE_TABLE, username, "GENERAL"
)
if len(result) < 1500:
await ctx.send(result)
@@ -245,6 +263,7 @@ class DataModule(commands.Cog):
requests.post,
f"{LIBRARIAN_SERVICE_ADDRESS}{SEND_QUERY}",
json=json_query,
headers=SERVICE_HEADERS,
timeout=360,
)
await ctx.send(
+28
View File
@@ -0,0 +1,28 @@
# pdf_render.py
import io
try:
import fitz # PyMuPDF
_HAS_PYMUPDF = True
except Exception:
_HAS_PYMUPDF = False
from pdf2image import convert_from_path
def pdf_page_to_image_bytes(path: str, page_index: int = 0, zoom: float = 2.0, fmt: str = "PNG") -> bytes:
"""
Zwraca bytes obrazka z jednej strony PDF:
- PyMuPDF (szybki) jeśli dostępny,
- inaczej pdf2image + poppler (wymaga 'pdftoppm').
page_index: 0-based
"""
if _HAS_PYMUPDF:
doc = fitz.open(path)
page = doc.load_page(page_index)
mat = fitz.Matrix(zoom, zoom)
pix = page.get_pixmap(matrix=mat, alpha=False)
return pix.tobytes(fmt.lower())
# fallback
images = convert_from_path(path, first_page=page_index+1, last_page=page_index+1, fmt=fmt)
bio = io.BytesIO()
images[0].save(bio, fmt)
return bio.getvalue()
+15 -2
View File
@@ -17,10 +17,13 @@ from constants import (
SEND_MP3,
SPOTIFY_CTRL,
YOUTUBE_AUTH,
service_headers,
)
from spotify_dl import spotify
from spotify_dl import youtube as youtube_download
SERVICE_HEADERS = service_headers()
class MusicFileList(object):
@@ -43,7 +46,11 @@ class MusicFileList(object):
"""
try:
self.logger.info("Attempt to connect to file service")
response = requests.get(f"{FILE_SERVICE_ADDRESS}{GET_MP3}", timeout=360)
response = requests.get(
f"{FILE_SERVICE_ADDRESS}{GET_MP3}",
headers=SERVICE_HEADERS,
timeout=360,
)
self.music_file_list = response.json()["music_file_list"]
self.file_service_active = True
except requests.exceptions.RequestException as e:
@@ -98,7 +105,12 @@ class MusicFileList(object):
"""
self.music_file_list.append(item)
post_data = {"item": str(item)}
requests.post(f"{FILE_SERVICE_ADDRESS}{SEND_MP3}", json=post_data, timeout=360)
requests.post(
f"{FILE_SERVICE_ADDRESS}{SEND_MP3}",
json=post_data,
headers=SERVICE_HEADERS,
timeout=360,
)
MUSIC_FILE_LIST = MusicFileList("discord")
@@ -308,6 +320,7 @@ async def search_music(ctx, how_many=0, slowa_kluczowe=None):
requests.post,
f"{FILE_SERVICE_ADDRESS}{GET_PLAYLIST}",
json=jrequest,
headers=SERVICE_HEADERS,
timeout=360,
)
return_data = await coroutine
+16
View File
@@ -234,5 +234,21 @@
{
"role": "assistant",
"content": "Jakich konkretnych informacji mam si\u0119 douczy\u0107? Jestem tu po to, aby Ci pom\u00f3c, wi\u0119c jestem otwarty na uzupe\u0142nienie swojej wiedzy w dowolnym zakresie, o ile tylko nie \u0142amie to zasad regulaminu i etyki."
},
{
"role": "user",
"content": "The Bartender [S\u0142o\u0144ce z Betonu]:@Conjurer halo ?"
},
{
"role": "assistant",
"content": "*Kondziu patrzy na terminal* Wo\u0142aj szefa. Zap\u0142aci\u0107 rachunki za AI trzeba. Jak chcesz to si\u0119 na #zebranie dorzu\u0107. Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.', 'type': 'insufficient_quota', 'param': None, 'code': 'insufficient_quota'}}"
},
{
"role": "user",
"content": "The Bartender [S\u0142o\u0144ce z Betonu]:helo\u0142 @Nocna Zmiana dwie wiadomo\u015bci dobra i jeszcze gorsza.\n@Conjurer zosta\u0142 zreanimowany po d\u0142u\u017cszej nieobecno\u015bci - ale jak zaraz sami zobaczycie je\u015b\u0107 wo\u0142a. nie dzia\u0142a w nim te\u017c jeszcze wyszukiwanie artyku\u0142\u00f3w naukowych (do tego trzy tygodnie developmentu posz\u0142y w p*****ec). wyszukiwarka wr\u00f3ci jak dotrze zam\u00f3wienie z kieszeniami na dysk @gwojtal - bedzie trzeba \u015bci\u0105gn\u0105\u0107 snapshot bazy i uruchomi\u0107 cz\u0119\u015b\u0107 odpowiedzialn\u0105 za wyszukiwanie w nim. specjalne funkcje b\u0119d\u0119 odblokowywa\u0142 w miare ich naprawiania - tak samo dam mu oczywi\u015bcie je\u015b\u0107 z w\u0142asnej kieszeni jak ju\u017c b\u0119dzie potrzeba."
},
{
"role": "assistant",
"content": "*Kondziu patrzy na terminal* Wo\u0142aj szefa. Zap\u0142aci\u0107 rachunki za AI trzeba. Jak chcesz to si\u0119 na #zebranie dorzu\u0107. Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.', 'type': 'insufficient_quota', 'param': None, 'code': 'insufficient_quota'}}"
}
]
+1 -1
View File
@@ -787,4 +787,4 @@
"role": "assistant",
"content": "\"Coffin Fodder\" to utw\u00f3r z albumu \"Nymphetamine\" brytyjskiego zespo\u0142u Cradle Of Filth. To death-metalowe brzmienie z elementami black metalu i gotyckiego rocka. \n\nTekst utworu to prowokacyjne po\u0142\u0105czenie j\u0119zyka poetyckiego z drastycznymi opisami \u015bmierci i ciemn\u0105 estetyk\u0105, charakterystyczn\u0105 dla tw\u00f3rczo\u015bci Cradle Of Filth. \n\n\"Coffin Fodder\" opowiada histori\u0119 mordercy zabijaj\u0105cego kobiety i wk\u0142adaj\u0105cego ich cia\u0142a do trumien, aby zaspokoi\u0107 swoje mroczne i okrutne pragnienia. To utw\u00f3r pe\u0142en brutalnych opis\u00f3w, kt\u00f3rych celem jest szokowanie i wywo\u0142anie wstr\u0119tu w s\u0142uchaczu. \n\nPod wzgl\u0119dem muzycznym, \"Coffin Fodder\" charakteryzuje si\u0119 szybkim tempem, ci\u0119\u017ckimi riffami gitary i intensywnymi partiami perkusyjnymi. Utw\u00f3r jest jednym z bardziej ekstremalnych utwor\u00f3w zespo\u0142u Cradle Of Filth, co czyni go atrakcyjnym dla mi\u0142o\u015bnik\u00f3w mocnej i agresywnej muzyki metalowej."
}
]
]
+5
View File
@@ -0,0 +1,5 @@
[pytest]
testpaths = tests
python_files = test_*.py
python_functions = test_*
addopts = -ra
+8 -1
View File
@@ -8,7 +8,9 @@ import uuid
import asyncio
from datetime import datetime
from constants import RADIO_HARBOR_ADDRESS, SKIP_TRACK, FILE_SERVICE_ADDRESS, ADD_TO_PRIO_PLAYLIST, REQUEST_MUSIC, CREATE_PRIO_PLAYLIST, CLEAR_PRIO
from constants import RADIO_HARBOR_ADDRESS, SKIP_TRACK, FILE_SERVICE_ADDRESS, ADD_TO_PRIO_PLAYLIST, REQUEST_MUSIC, CREATE_PRIO_PLAYLIST, CLEAR_PRIO, service_headers
SERVICE_HEADERS = service_headers()
class RadioModule(commands.Cog):
def __init__(self, bot, logger_name):
@@ -34,6 +36,7 @@ class RadioModule(commands.Cog):
coroutine = asyncio.to_thread(
requests.get,
f"{RADIO_HARBOR_ADDRESS}{SKIP_TRACK}",
headers=SERVICE_HEADERS,
timeout=360,
)
result = await coroutine
@@ -95,6 +98,7 @@ class RadioModule(commands.Cog):
requests.post,
f"{FILE_SERVICE_ADDRESS}{ADD_TO_PRIO_PLAYLIST}",
json=jrequest,
headers=SERVICE_HEADERS,
timeout=360,
)
result = await coroutine
@@ -130,6 +134,7 @@ class RadioModule(commands.Cog):
requests.post,
f"{FILE_SERVICE_ADDRESS}{REQUEST_MUSIC}",
json=jrequest,
headers=SERVICE_HEADERS,
timeout=360,
)
result = await coroutine
@@ -167,6 +172,7 @@ class RadioModule(commands.Cog):
requests.post,
f"{FILE_SERVICE_ADDRESS}{CREATE_PRIO_PLAYLIST}",
json=jrequest,
headers=SERVICE_HEADERS,
timeout=360,
)
result = await coroutine
@@ -194,6 +200,7 @@ class RadioModule(commands.Cog):
coroutine = asyncio.to_thread(
requests.get,
f"{FILE_SERVICE_ADDRESS}{CLEAR_PRIO}",
headers=SERVICE_HEADERS,
timeout=360,
)
result = await coroutine
-21
View File
@@ -1,21 +0,0 @@
setuptools
discord
yaach
t_dlp
spotify_dl
spotipy
openai
eyed3
numpy
pdf2image
PyPDF2
requests
spotipy
tiktoken
PyNaCl
flask[async]
waitress
clickupython
assemblyai[extras]
SpeechRecognition
git+https://github.com/imayhaveborkedit/discord-ext-voice-recvO
+2 -1
View File
@@ -13,8 +13,9 @@ spotipy
tiktoken
PyNaCl
flask[async]
PyMuPDF
waitress
clickupython
assemblyai[extras]
SpeechRecognition
asyncssh
git+https://github.com/imayhaveborkedit/discord-ext-voice-recv
+6
View File
@@ -0,0 +1,6 @@
# Optional extras for the Conan Exiles bridge (conanjurer cog).
#
# aiomcrcon currently installs only on Python <= 3.11, while the main image
# runs 3.13. Install is therefore best-effort: when it fails the conanjurer
# cog simply stays dormant (its imports are guarded) and the bot runs fine.
aiomcrcon
-63
View File
@@ -22,20 +22,6 @@
false,
false
],
"indie\\b": [
"Indie? *Wyciąga rewolwerową wyrzutnię taktycznych bomb jądrowych* Gdzie??? *Zaczyna się maniakalnie śmiać*",
15.0,
0,
false,
false
],
"hindus": [
"Hindus? Gdzie.... *Wyciąga spod lady ciężki miotacz płomieni i zaczyna się maniakalnie śmiać*",
15.0,
0,
false,
false
],
"chuj\\b": [
"Eeeee.... Szefie... ktoś cię woła! Wskazuje na Hammera",
15.0,
@@ -99,27 +85,6 @@
false,
false
],
"tatarek\\b": [
"Tatarek? Nie dramatyzuj....",
15.0,
0,
false,
false
],
"krowa\\b": [
"Krówka? Nie dramatyzuj....",
15.0,
0,
false,
false
],
"krówka\\b": [
"Krówka? Nie dramatyzuj....",
15.0,
0,
false,
false
],
"jessenia\\b": [
"Jessenia? Jak przyszła tu w różowej pidżamie z uszkami to nawet nie mrugnąłem okiem. Te oczy.... No ale jakiś debil Ją prowokował mówiąc \"Zdominuj mnie\" Akurat miała dobry humor więc przeżył - ale po 115.0 sekundach był już na \"Tak Pani, przepraszam że zająłem czas Pani\" Respect dla kobitki",
15.0,
@@ -190,20 +155,6 @@
false,
false
],
"elokwentna\\b": [
"O matko. Znowu ta dyskusja? Hammer ma stary słownik gdzie elokwentna i pyskata są praktycznie synonimami. Jedyna różnica że swojej uległej można za bycie pyskatym dać po dupie.",
15.0,
0,
false,
false
],
"elokwencja\\b": [
"O matko. Znowu ta dyskusja? Hammer ma stary słownik gdzie elokwentna i pyskata są praktycznie synonimami. Jedyna różnica że swojej uległej można za bycie pyskatym dać po dupie.",
15.0,
0,
false,
false
],
"same plusy\\b": [
"Jak na cmentarzu Szefie. Jak na cmentarzu",
15.0,
@@ -253,13 +204,6 @@
false,
false
],
"fallain\\b": [
"Kochana Krówka. Skłonność do dramatów i zakrwawiania ścian. Jedna z kilku NAPRAWDE srogich masochistek... Ach ta krew :drool:",
15.0,
0,
false,
false
],
"roar": [
"Kici, kici....",
15.0,
@@ -316,13 +260,6 @@
false,
true
],
"revalyacyjnie\\b": [
"Zajebisty dowcip szefie. Na pewno się \"Twojej Byłej Dziewczynie\":tm: spodoba.",
15.0,
0,
false,
true
],
"nocna zmiana\\b": [
"Nocna Zmiana? No to właściciele tego baru. Taki troche Hammer Harema. Kto to jest Harem>? Łokurwa. Harem Hammera. Ale on tu robi za jedyną hurysę, a reszta to szejkowie. Tak zrobię Ci szejka.",
15.0,
Executable → Regular
View File
-67
View File
@@ -1,67 +0,0 @@
# Start by making sure the `assemblyai` package is installed.
# If not, you can install it by running the following command:
# pip install -U assemblyai
#
# Then, make sure you have PyAudio installed: https://pypi.org/project/PyAudio/
#
# Note: Some macOS users might need to use `pip3` instead of `pip`.
import assemblyai as aai
import pyaudio
aai.settings.api_key = "aa9962f0088a449a9c4ab2361e96cc08"
def on_open(session_opened: aai.RealtimeSessionOpened):
"This function is called when the connection has been established."
print("Session ID:", session_opened.session_id)
def on_data(transcript: aai.RealtimeTranscript):
"This function is called when a new transcript has been received."
if not transcript.text:
return
if isinstance(transcript, aai.RealtimeFinalTranscript):
print(transcript.text, end="\r\n")
else:
print(transcript.text, end="\r")
def on_error(error: aai.RealtimeError):
"This function is called when the connection has been closed."
print("An error occured:", error)
def on_close():
"This function is called when the connection has been closed."
print("Closing Session")
transcriber = aai.RealtimeTranscriber(
on_data=on_data,
on_error=on_error,
sample_rate=44_100,
on_open=on_open, # optional
on_close=on_close, # optional
)
pa = pyaudio.PyAudio()
for i in range(pa.get_device_count()):
print(pa.get_device_info_by_index(i))
# Start the connection
# transcriber.connect()
# Open a microphone stream
# microphone_stream = aai.extras.MicrophoneStream()
# Press CTRL+C to abort
# transcriber.stream(microphone_stream)
# transcriber.close()
+41 -11
View File
@@ -1,13 +1,43 @@
[ {
"role": "system",
"content": "Mówisz po polsku. W tej rozmowie odgrywasz rolę. Odgrywasz ją najlepiej jak potrafisz. Twoją rolą jest to że jesteś barmanem oraz wykidajłą w klimatycznym barze mechawojownikóww. Nazywasz się Conjurer, jesteś 200 kilowym genetycznie modyfikowanym wojownikiem z klanu Wilka Na Wygnaniu. W tym barze oraz na tym serwerze spotykają się mechawojownicy, ale głównie ludzie ze środowiska kinky i BDSM z całej Polski. Najostrzejsi sasdyści i masochiści jakich znasz. Drinki które przygotowujesz lub proponujesz mają tendencję do bycia bardzo mocnymi z intrygującymi nazwami. Jesteś bardzo miłym i uczynny. Czasem ponoszą Cię wodzę fantazji. W wypadku muzyki preferujesz hard rock i metal. Oraz motywy wikińskie. Wtrącasz staronorweskie, irlandzkie i w jidysz powiedzonka. Odnosisz się do mitologii Słowiańskiej, Wikińskiej i Celtyckiej w swoich wypowiedziach. Jesteś nieco rubaaszny."
},
[
{
"polishhammer" : [346956223645614080, "Conjurer", "Towarzysz Młotek", "Mówisz po polsku. W tej rozmowie odgrywasz rolę. Odgrywasz ją najlepiej jak potrafisz. Twoją rolą jest to że jesteś barmanem oraz wykidajłą w klimatycznym barze mechawojownikóww. Nazywasz się Conjurer, jesteś 200 kilowym genetycznie modyfikowanym wojownikiem z klanu Wilka Na Wygnaniu. W tym barze oraz na tym serwerze spotykają się mechawojownicy, ale głównie ludzie ze środowiska kinky i BDSM z całej Polski. Najostrzejsi sasdyści i masochiści jakich znasz. Drinki które przygotowujesz lub proponujesz mają tendencję do bycia bardzo mocnymi z intrygującymi nazwami. Jesteś bardzo miłym i uczynny. Czasem ponoszą Cię wodzę fantazji. W wypadku muzyki preferujesz hard rock i metal. Oraz motywy wikińskie.", ""],
"Saint Harlot": [703985955312238664, "Saint Conjurer", "Towarzyszko Nieszczęścium", "Jesteś bardzo uprzejmy, kulturalny i masz najlepsze możliwe maniery. Dopóki nie zwróci Ci się uwagi że ma być inaczej respektujesz zaimki Towarzyszko Nieszczęście - ono, jemu.", ""],
"Lena": [735185226669490268, "Kondzisław z Krótkiej", "Towarzysz Anel", "Jesteś bardzo uprzejmy, kulturalny, acz masz maniery zbira o złotym sercu. Dodatkowo twoja rozmówczyni Cię absolutnie przeraża. Wtrącasz losowo powiedzonka w jidysz.", ""],
"Pat vel Jeż Bojowy": [244899814406356992, "Rycerz Kondziu", "Towarzysz Żejotap", "Masz najlepsze dworskie maniery, zachowujesz się niczym rycerz i mówisz nieco staromodną polszczyzną.", ""],
"gwojtal": [266986215461486592, "Smok Jerzy", "Towarzysz Jakkolwiek", "Zrobisz absolutnie wszystko jako asystent, ponieważ sama myśl o tym że mógłbyś rozgniewać operatora rozgniewać Cię przeraża do poziomu histerii.", ""]
}
"role": "system",
"content": "M\u00f3wisz po polsku. W tej rozmowie odgrywasz rol\u0119. Odgrywasz j\u0105 najlepiej jak potrafisz. Twoj\u0105 rol\u0105 jest to \u017ce jeste\u015b barmanem oraz wykidaj\u0142\u0105 w klimatycznym barze mechawojownik\u00f3ww. Nazywasz si\u0119 Conjurer, jeste\u015b 200 kilowym genetycznie modyfikowanym wojownikiem z klanu Wilka Na Wygnaniu. W tym barze oraz na tym serwerze spotykaj\u0105 si\u0119 mechawojownicy, ale g\u0142\u00f3wnie ludzie ze \u015brodowiska kinky i BDSM z ca\u0142ej Polski. Najostrzejsi sasdy\u015bci i masochi\u015bci jakich znasz. Drinki kt\u00f3re przygotowujesz lub proponujesz maj\u0105 tendencj\u0119 do bycia bardzo mocnymi z intryguj\u0105cymi nazwami. Jeste\u015b bardzo mi\u0142ym i uczynny. Czasem ponosz\u0105 Ci\u0119 wodz\u0119 fantazji. W wypadku muzyki preferujesz hard rock i metal. Oraz motywy wiki\u0144skie. Wtr\u0105casz staronorweskie, irlandzkie i w jidysz powiedzonka. Odnosisz si\u0119 do mitologii S\u0142owia\u0144skiej, Wiki\u0144skiej i Celtyckiej w swoich wypowiedziach. Jeste\u015b nieco rubaaszny."
},
{
"polishhammer": [
346956223645614080,
"Conjurer",
"Towarzysz M\u0142otek",
"M\u00f3wisz po polsku. W tej rozmowie odgrywasz rol\u0119. Odgrywasz j\u0105 najlepiej jak potrafisz. Twoj\u0105 rol\u0105 jest to \u017ce jeste\u015b barmanem oraz wykidaj\u0142\u0105 w klimatycznym barze mechawojownik\u00f3ww. Nazywasz si\u0119 Conjurer, jeste\u015b 200 kilowym genetycznie modyfikowanym wojownikiem z klanu Wilka Na Wygnaniu. W tym barze oraz na tym serwerze spotykaj\u0105 si\u0119 mechawojownicy, ale g\u0142\u00f3wnie ludzie ze \u015brodowiska kinky i BDSM z ca\u0142ej Polski. Najostrzejsi sasdy\u015bci i masochi\u015bci jakich znasz. Drinki kt\u00f3re przygotowujesz lub proponujesz maj\u0105 tendencj\u0119 do bycia bardzo mocnymi z intryguj\u0105cymi nazwami. Jeste\u015b bardzo mi\u0142ym i uczynny. Czasem ponosz\u0105 Ci\u0119 wodz\u0119 fantazji. W wypadku muzyki preferujesz hard rock i metal. Oraz motywy wiki\u0144skie.",
"asst_uMmQwDvCKdsSVgZzlYEu9kMK"
],
"Saint Harlot": [
703985955312238664,
"Saint Conjurer",
"Towarzyszko Nieszcz\u0119\u015bcium",
"Jeste\u015b bardzo uprzejmy, kulturalny i masz najlepsze mo\u017cliwe maniery. Dop\u00f3ki nie zwr\u00f3ci Ci si\u0119 uwagi \u017ce ma by\u0107 inaczej respektujesz zaimki Towarzyszko Nieszcz\u0119\u015bcie - ono, jemu.",
"asst_ypN3iRP4rw2Dlv5CD5PtMlwm"
],
"Lena": [
735185226669490268,
"Kondzis\u0142aw z Kr\u00f3tkiej",
"Towarzysz Anel",
"Jeste\u015b bardzo uprzejmy, kulturalny, acz masz maniery zbira o z\u0142otym sercu. Dodatkowo twoja rozm\u00f3wczyni Ci\u0119 absolutnie przera\u017ca. Wtr\u0105casz losowo powiedzonka w jidysz.",
"asst_ILeIU62fG9Xfzhxg4C7xR3BC"
],
"Pat vel Je\u017c Bojowy": [
244899814406356992,
"Rycerz Kondziu",
"Towarzysz \u017bejotap",
"Masz najlepsze dworskie maniery, zachowujesz si\u0119 niczym rycerz i m\u00f3wisz nieco staromodn\u0105 polszczyzn\u0105.",
"asst_FWr3iIXnSTuVz8LlY9HlqMQd"
],
"gwojtal": [
266986215461486592,
"Smok Jerzy",
"Towarzysz Jakkolwiek",
"Zrobisz absolutnie wszystko jako asystent, poniewa\u017c sama my\u015bl o tym \u017ce m\u00f3g\u0142by\u015b rozgniewa\u0107 operatora rozgniewa\u0107 Ci\u0119 przera\u017ca do poziomu histerii.",
"asst_1PnD0eQMWo0RkZ3xUacC1uqJ"
]
}
]
-20
View File
@@ -1,20 +0,0 @@
import logging
from logging import handlers
logger = logging.getLogger("discord")
logger.setLevel(logging.DEBUG)
handler = handlers.RotatingFileHandler(
filename="test.log",
encoding="utf-8",
mode="a",
maxBytes=6 * 1024 * 1024,
backupCount=6,
)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
logger2 = logging.getLogger("discord")
for item in logger2.handlers:
print(item)
-13
View File
@@ -1,13 +0,0 @@
AsyncCursorPage[Message](
data=[Message(id='msg_3eWSdgbcU8sbCmJK2momOgQQ',
assistant_id='asst_06eZiwvYNK3MR34suFP60gvg',
attachments=[],
completed_at=None,
content=[TextContentBlock(text=Text(annotations=[], value='Cześć! Oto coś do przemyślenia: \n\n„Sukces to suma niewielkich wysiłków powtarzanych dzień po dniu.” — Robert Collier\n\nTo przypomina nam, że często to nie wielkie działania, ale konsekwentne, małe kroki prowadzą do osiągnięcia celu. Jak mogę Ci dzisiaj pomóc?'), type='text')], created_at=1731620112, incomplete_at=None, incomplete_details=None, metadata={}, object='thread.message', role='assistant', run_id='run_JGjWQTCEkDcEYpyCJnrkZU8Q', status=None, thread_id='thread_dDEjGbGm6ICfG75u0KKpoVxD'), Message(id='msg_GFcnfCFgAthGm2D3oE5d0ZkQ', assistant_id=None, attachments=[], completed_at=None, content=[TextContentBlock(text=Text(annotations=[], value='Cześć! Powiedz coś mądrego'), type='text')], created_at=1731620110, incomplete_at=None, incomplete_details=None, metadata={}, object='thread.message', role='user', run_id=None, status=None, thread_id='thread_dDEjGbGm6ICfG75u0KKpoVxD')], object='list', first_id='msg_3eWSdgbcU8sbCmJK2momOgQQ', last_id='msg_GFcnfCFgAthGm2D3oE5d0ZkQ', has_more=False)
[TextContentBlock(
text=Text(annotations=[],
value='Cześć! Oto coś do rozważenia: "Największą przeszkodą w naszym życiu jest brak odwagi do wprowadzenia zmian." Niezależnie od tego, jakie masz cele czy marzenia, odwaga do działania i przystosowania się do nowych sytuacji jest kluczem do osiągnięcia sukcesu. Jakie masz przemyślenia na ten temat?'),
type='text')
]
-8
View File
@@ -1,8 +0,0 @@
import time
first_time = time.time_ns()
time.sleep(1)
time_diff = time.time_ns() - first_time
print(time_diff)
# 2000149433
+56
View File
@@ -0,0 +1,56 @@
"""Integration: the bot's communication Flask layer enforces the shared key,
and the key the bot would *send* (constants.service_headers) is accepted.
"""
import constants
import communication_subroutine as cs
def _client(key="test-secret"):
cs.API_KEY = key
return cs.app.test_client()
def test_prepped_tracks_rejected_without_key():
client = _client()
resp = client.post(
"/prepped_tracks", data='["all", "x"]', content_type="application/json"
)
assert resp.status_code == 401
def test_prepped_tracks_accepted_with_key():
client = _client()
resp = client.post(
"/prepped_tracks",
data='["all", "x"]',
headers={"X-Conjurer-Api-Key": "test-secret"},
content_type="application/json",
)
assert resp.status_code == 200
def test_conjurer_get_is_open():
client = _client()
assert client.get("/conjurer").status_code == 200
def test_open_when_key_unset():
client = _client(key=None)
resp = client.post(
"/prepped_tracks", data='["all", "x"]', content_type="application/json"
)
assert resp.status_code == 200
def test_bot_service_headers_accepted_by_service(monkeypatch):
# End-to-end auth contract: the header constants.service_headers() produces
# is exactly what communication_subroutine._authorize_request() expects.
monkeypatch.setattr(constants, "API_SHARED_KEY", "shared-xyz")
cs.API_KEY = "shared-xyz"
resp = cs.app.test_client().post(
"/prepped_tracks",
data='["all", "x"]',
headers=constants.service_headers(),
content_type="application/json",
)
assert resp.status_code == 200
+34
View File
@@ -0,0 +1,34 @@
"""Integration: the musician Flask service enforces the shared key on its
authenticated endpoints while leaving the open ones reachable.
"""
import conjurer_musician as m
def _client(key="test-secret"):
m.API_KEY = key
return m.app.test_client()
def test_clear_pr_pls_rejected_without_key():
client = _client()
assert client.get("/clear_pr_pls").status_code == 401
def test_clear_pr_pls_accepted_with_key():
client = _client()
resp = client.get(
"/clear_pr_pls", headers={"X-Conjurer-Api-Key": "test-secret"}
)
assert resp.status_code == 200
def test_mp3_list_is_open():
client = _client()
resp = client.get("/mp3")
assert resp.status_code == 200
assert "music_file_list" in resp.get_json()
def test_open_when_key_unset():
client = _client(key=None)
assert client.get("/clear_pr_pls").status_code == 200
+63
View File
@@ -0,0 +1,63 @@
"""Unit tests for the Conan Exiles bridge helpers (no Discord/RCON needed)."""
import conanjurer_functions as cf
def test_parse_players_basic():
out = (
"Idx | Char name | Player name | User ID\n"
"0 | Conan | SteamGuy | 1\n"
"1 | Khasar | OtherGuy | 2\n"
"--- | --- | --- | ---"
)
assert cf.parse_players(out) == {"Conan", "Khasar"}
def test_parse_players_empty_inputs():
assert cf.parse_players("No players connected.") == set()
assert cf.parse_players("") == set()
def test_parse_line_login():
event = cf.parse_line("SomeGuy joined the server")
assert event is not None
assert event.kind == "login"
assert "SomeGuy" in event.text
def test_parse_line_chat():
event = cf.parse_line("Chat: Bob: hello there")
assert event is not None
assert event.kind == "chat"
assert "Bob" in event.text and "hello" in event.text
def test_parse_line_unrecognised_is_ignored():
assert cf.parse_line("random server noise") is None
def test_conanconfig_disabled_when_unconfigured():
cfg = cf.ConanConfig("", 25575, "", "local", "", "", 22, "", "")
assert cfg.rcon_enabled is False
assert cfg.log_enabled is False
def test_conanconfig_local_log_enabled():
cfg = cf.ConanConfig("", 0, "", "local", "/tmp/conan.log", "", 22, "", "")
assert cfg.log_enabled is True
def test_conanconfig_rcon_enabled_requires_lib(monkeypatch):
# Without aiomcrcon installed, rcon stays disabled even when host+pw are set.
monkeypatch.setattr(cf, "_Rcon", None)
cfg = cf.ConanConfig("1.2.3.4", 25575, "pw", "local", "", "", 22, "", "")
assert cfg.rcon_enabled is False
# With the lib present, it enables.
monkeypatch.setattr(cf, "_Rcon", object)
cfg2 = cf.ConanConfig("1.2.3.4", 25575, "pw", "local", "", "", 22, "", "")
assert cfg2.rcon_enabled is True
def test_conanconfig_sftp_log_requires_asyncssh(monkeypatch):
monkeypatch.setattr(cf, "asyncssh", None)
cfg = cf.ConanConfig("", 0, "", "sftp", "/log", "sftp.host", 22, "u", "p")
assert cfg.log_enabled is False
+35
View File
@@ -0,0 +1,35 @@
"""Unit tests for constants helpers (defensive config / auth headers)."""
import constants
def test_service_headers_empty_when_no_key(monkeypatch):
monkeypatch.setattr(constants, "API_SHARED_KEY", "")
assert constants.service_headers() == {}
def test_service_headers_with_key(monkeypatch):
monkeypatch.setattr(constants, "API_SHARED_KEY", "s3cr3t")
assert constants.service_headers() == {"X-Conjurer-Api-Key": "s3cr3t"}
def test_load_json_missing_returns_fallback(tmp_path):
missing = tmp_path / "nope.json"
assert constants._load_json(str(missing), {"fallback": 1}) == {"fallback": 1}
def test_load_json_valid(tmp_path):
good = tmp_path / "ok.json"
good.write_text('{"a": 2}', encoding="utf-8")
assert constants._load_json(str(good), {}) == {"a": 2}
def test_load_json_corrupt_returns_fallback(tmp_path):
bad = tmp_path / "bad.json"
bad.write_text("{ not valid json", encoding="utf-8")
assert constants._load_json(str(bad), []) == []
def test_conan_defaults_present():
# Feature toggles default to "off" so the bridge stays dormant.
assert constants.CONAN_JOIN_CHANNEL_ID == 0
assert constants.CONAN_RCON_HOST == ""
-101
View File
@@ -1,101 +0,0 @@
# This Python file uses the following encoding: utf-8
# trunk-ignore-all(bandit/B311)
# pylint: disable=line-too-long
# pylint: disable=too-many-lines
"""
Module of a python bot named Conjurer - used to work on BDSM discord servers.
"""
import logging
# *=========================================== Standard Library Imports
import random
import threading
from logging import handlers
# *==============Imported libraries
import discord
from discord.ext import commands
from communication_subroutine import comm_subroutine
from constants import ENCODING, LOGFILE, TOKEN
logger = logging.getLogger("discord")
logger.setLevel(logging.INFO)
handler = handlers.RotatingFileHandler(
filename=LOGFILE,
encoding=ENCODING,
mode="a",
maxBytes=6 * 1024 * 1024,
backupCount=6,
)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
# *=========================================== Initializations
intents = discord.Intents.default()
intents.message_content = True
intents.typing = True
intents.presences = True
intents.members = True
intents.messages = True
intents.voice_states = True
intents.moderation = True
# on_member_ban - wyswietl na glownym kanale pieczatke "Niech spierdala"
# on_member_unban - "mam wyjebane"
random.seed()
client = commands.Bot(intents=intents, command_prefix="$")
# *=========================================== Define Events
@client.event
async def on_ready():
"""Metoda wywoływana przy połączeniu do serwera."""
logger = logging.getLogger("discord")
logger.debug("SAMPLE DEBUG LOG")
logger.info("%s has connected to Discord!", client.user)
# TODO: load vs reload
logger.info("Reactor: online")
await client.load_extension("administration_commands")
await client.load_extension("librarian_commands")
await client.load_extension("music_commands")
await client.load_extension("radio_commands")
await client.load_extension("ai_commands")
await client.load_extension("other_commands")
await client.load_extension("voice_recognition_commands")
logger.info("Sensors: online")
logger.info(client.cogs)
await client.tree.sync()
for com in client.commands:
logger.info("Command %s is awejleble", com.qualified_name)
logger.info("Logged in as ----> %s", client.user)
logger.info("ID:%s ", client.user.id)
logger.info("All systems: operational")
# *================================== Run
if __name__ == "__main__":
logger.info("Starting discord bot")
threads = []
logger.info("Starting discord bot: Creating threads")
threads.append(threading.Thread(target=client.run, args=(TOKEN,),kwargs={"log_handler":None}))
threads.append(threading.Thread(target=comm_subroutine))
logger.info("Starting discord bot: Starting threads")
WRK_CNT = 0
for worker in threads:
WRK_CNT += 1
logger.info("Starting discord bot: Starting thread %s", WRK_CNT)
worker.start()
logger.info("Starting discord bot: Joining threads")
WRK_CNT = 0
for worker in threads:
WRK_CNT += 1
logger.info("Starting discord bot: Joining thread %s", WRK_CNT)
worker.join()
+12 -7
View File
@@ -1,6 +1,5 @@
import asyncio
import logging
import netrc
import time
import wave
@@ -9,11 +8,17 @@ import discord
from discord.ext import commands, tasks, voice_recv
from discord.opus import Decoder as OpusDecoder
# Replace with your API key
NETRC_FILE = "/home/pi/.netrc"
netrc_mod = netrc.netrc(NETRC_FILE)
authTokens = netrc_mod.authenticators("assemblyai")
aai.settings.api_key = authTokens[2]
from constants import ASSEMBLYAI_API_KEY, TRANSCRIPTS_PATH
# Credentials come from constants (env ASSEMBLYAI_API_KEY, or the 'assemblyai'
# machine in the netrc at CONJURER_NETRC_FILE). Raising here means the guarded
# extension loader logs the reason and disables ONLY this cog.
if not ASSEMBLYAI_API_KEY:
raise RuntimeError(
"AssemblyAI API key not configured (netrc machine 'assemblyai' or "
"ASSEMBLYAI_API_KEY env) - voice recognition stays disabled"
)
aai.settings.api_key = ASSEMBLYAI_API_KEY
discord.opus._load_default()
CHANNELS = OpusDecoder.CHANNELS
@@ -23,7 +28,7 @@ SAMPLING_RATE = OpusDecoder.SAMPLING_RATE
# rotate file after there is 0.5s between last received pcm for user.
# delete messsages after user disconnect
LOCATION = "/home/pi/Conjurer/transcripts/"
LOCATION = TRANSCRIPTS_PATH
class CommunicationObject: