So one librarian can serve several bots (test + deploy) instead of firing
every result/pong at a single static CONJURER_MAIN_BOT.
* The bot includes its own callback address (CONJURER_SELF_CALLBACK) in
every /query and /ping.
* The librarian stores that callback with the query (persisted with the
request, so a replay after restart still answers the right bot) and, for
results, in the OUTBOX entry ({target, payload}) so the resender delivers
to the origin bot even across a librarian restart.
* Pongs go back to the pinging bot too - otherwise a second bot's health
check would be ponged to the first and always time out, so it could
never enable its librarian cog.
* Empty callback falls back to MAIN_BOT_ADDRESS, and a legacy OUTBOX entry
(raw payload, pre-callback) is still delivered to the default bot, so the
upgrade is seamless.
Tests: per-origin result delivery + legacy-shape fallback (outbox),
busy/idle pong routed to the callback bot vs default (lifecycle). Suite:
58 unit + 52 integration green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An 8h search result must survive a transient bot outage, an api/address
misroute, or a restart of either side. Make the librarian->bot result
path durably at-least-once with idempotent rendering:
Shared: durable_queue.DiskQueue - a dependency-free, atomically-written,
one-file-per-key disk queue (unit-tested), shared by both images
(added to Dockerfile.librarian; the bot already COPYs *.py).
Librarian (sender): finished results go to a persistent OUTBOX before
sending; delivery retries with backoff; an entry is removed only on a
positive ACK; a resender thread keeps flushing the OUTBOX, so a result
survives a bot outage AND a librarian restart (OUTBOX is on the state
volume) - it simply keeps trying until acked.
Bot (receiver): /conjurer is now idempotent and durable - each result is
persisted to an INBOX before acking and only queued if its uuid was not
already delivered (dropped as a duplicate) or already pending. Once the
cog actually renders it, mark_delivered() records the uuid and clears the
inbox, so the librarian's resends become no-ops. On startup the bot
replays any accepted-but-unrendered result from the INBOX, so a bot crash
mid-flight doesn't lose it. Pongs stay ephemeral.
Together: the librarian keeps a result until the bot confirms it; the bot
keeps it until it is on screen; duplicates never double-render. Combined
with the deploy return-path fix, an expensive result no longer vanishes.
Tests: unit test_durable_queue; integration test_librarian_outbox
(retry/backoff, resend survives outage) and test_result_durable_delivery
(persist, dedup pending, dedup delivered, replay, pong not persisted).
Suite: 55 unit + 39 integration green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The librarian health check was a plain GET to '/', which only proved
Flask was listening - not that the service could actually take a query,
run it through its internal queue+worker, and answer back. So the cog
could load against a librarian whose worker was wedged or that couldn't
reach the bot on the return leg.
Replace it with a ping that travels the SAME path a real search does, on
both sides:
bot: QueryControl -> OUT_COMM_Q -> scan_queue -> awaiting_q
librarian: POST /ping -> librarian_queue -> worker pulls it off
(no Crossref/DOI search) -> pongs back with the same uuid
bot: /conjurer -> incoming_q -> scan_incoming matches uuid, wakes waiter
The cog enables only when that whole loop closes within 3s. This also
proves the librarian->bot return path, which a GET never did.
Safety: uuid is random per ping; the wait and POST are both bounded so
startup can't stall; a pong that finds no waiter is dropped (never
orphaned into IN_COMM_Q, which would make the cog post a bogus 'no
results' message); and a ping whose pong never returns is swept out of
awaiting_q after PING_TTL_SECONDS so nothing leaks. All awaiting_q writes
stay within scan_queue (append) and scan_incoming (remove) - no locks,
no cross-thread mutation.
Integration tests cover: OK round-trip, timeout when accepted-but-no-pong,
unreachable, non-200, orphan-pong-dropped, and that real results still
reach IN_COMM_Q. Suite: 24 integration + 41 unit green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Seven confirmed defects from the code audit, each small and low-risk.
* ai_functions.get_random_cyclic_message: random.randint(0, len(CYCLIC_WORDS))
is inclusive -> could return len -> IndexError. Now randrange(len) + guard on
an empty CYCLIC_WORDS.
* librarian_commands.get_image_sadox: random.randrange(0, len(res)-1) never
picked the last comic and raised ValueError('empty range') on a single file.
Now randrange(len) + an empty-dir guard.
* ai_commands image generation: every DALL-E error branch replied but did not
return, so control fell through to `if response:` with response unbound ->
UnboundLocalError right after the friendly message. Each branch now returns;
response is pre-initialised; and PermissionDeniedError no longer passes a
(message, text) tuple as a single arg.
* search_bot DOI match: `item["DOI"] in data` was a substring test, so a DOI
that is a prefix of a longer one (10.1/1 vs 10.1/12) produced a false 'exists'
hit. Now matches the line's first whitespace token exactly, via an O(1) dict
index built once per consumer (also removes the O(queried-DOIs) per-line scan
- a real win for large databases).
* communication_subroutine.scan_incoming: matched records were never removed
from awaiting_q, so it grew unbounded over uptime and a reused UUID could
re-match a stale record. Matched records are now dropped after dispatch.
* communication_subroutine.id3: (resp.headers.get("icy-name") or "").title()
guards against a stream that omits headers (was AttributeError on None,
500-ing the /prepped_tracks "next" handler).
* betoniarka.scan_tracks: waits for the radio logs to exist instead of dying
with FileNotFoundError on a fresh deploy (which silently killed the
now-playing forwarder until a restart).
Verified: tests/unit/test_search_bot.py gains exact-match and trailing-metadata
cases; full unit job 43 passed. Remaining observations (image-gen stale
/home/pi fallback paths + dead FileNotFoundError-after-OSError branch; tailer
still vulnerable to mid-run log rotation; DOI-first-token assumption) noted for
follow-up - none are crashes on the normal path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two connected features.
1) AI query interface (via the comm layer). communication_subroutine gains an
AI_QUERY_Q, a submit_ai_query() in-process entry point, and an authed
POST /ai_query endpoint ({prompt, channel_id, request_type?, username?}). The
prompt is queued and answered asynchronously by a new tasks.loop worker in the
always-loaded AI cog (Events), which calls handle_response - so it runs on
whichever backend $gadaj_teraz currently selects (GPT or Claude) - and posts the
answer to the requested channel, chunked to Discord's limit. request_type "NONE"
(default) is a clean one-shot: no persona system prompt, no memory write. The
worker starts before the OpenAI guard in cog_load, so it also runs on a
Claude-only box; cog_unload cancels it.
2) Librarian AI review. New command $wyszukaj_z_recenzja mirrors
$wyszukaj_linki_do_dokumentow but sets ai_review=True on the QueryControl, which
rides the round-trip and is matched back by UUID. When the hits return,
check_data_q sends the raw list as before, then - if flagged - hands the same
list (already in Crossref-relevance order) plus the search phrase to the AI
queue for a weighted re-rank and per-source review, delivered to the same
channel. QueryControl gains an ai_review flag (default False, so the orphan path
and all existing callers are unaffected).
Confirmed separately (and noted in the docs): the DOI list the AI receives is
pre-sorted by Crossref relevance - the librarian pipeline only filters (drops
title-less items) and splits (in-db / not-in-db), never re-sorts, and relies on
insertion-ordered dicts (Py 3.7+).
Verified: /ai_query auth (401/200/400/open), submit_ai_query and the queued
dict shape, and the QueryControl flag - via a Flask test client and
tests/integration/test_ai_query_endpoint.py (5 tests, all pass; integration
suite 11 passed, the 3 failures are the pre-existing /clear_pr_pls musician
tests fixed on a separate branch). Full first-party compile clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The restart-loop with only 'Starting discord bot' in the logs was caused by
gather(return_exceptions=True) eating the client.start() exception: a failed
login looked like a clean exit and docker just restarted the container.
bot.py:
- _run_bot: catch discord.LoginFailure / PrivilegedIntentsRequired with
CRITICAL messages telling exactly what to fix (token entry / Developer
Portal intents), full traceback for anything else, then re-raise
- main() returns an exit code; every task exception is re-logged as the
LAST line in docker logs ('Task died: ...'); process exits 1 on failure
- log the token source (env vs netrc) and its length at startup to catch
truncated/wrong-field netrc entries
- _run_comm_subroutine: comm crash is logged loudly but no longer relies on
gather to surface it (bot keeps running - Discord side is independent)
communication_subroutine.py (crash found by the new stub tests):
- comm_subroutine used 'logger' before its local assignment
(UnboundLocalError killed the whole comm layer at every startup)
- re-comment the flask_debug thread: it binds the same host:port as
waitress, so running both dies with 'address in use'
Verified via stub-injected client: LoginFailure -> CRITICAL + exit 1,
PrivilegedIntentsRequired -> CRITICAL + exit 1, unknown exception -> full
traceback + exit 1, clean shutdown -> exit 0; comm layer no longer crashes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PRs #10 and #11 were merged into their intermediate base branches
(restructure/working-copy-root and proto-improvements) rather than main,
because the stacked PRs' bases were not auto-retargeted (the branches were
not deleted on merge). As a result main only received the #9 restructure
and is still the plain working-copy bot.
This brings the full prototype onto main as a clean delta on top of the
current main tree (identical content to proto-improvements, but with main
ancestry so it merges without the squash-induced rename/delete conflicts):
- constants.py: env-var config, safe JSON loading, dependency guards,
env->netrc tokens, API_SHARED_KEY + service_headers(), CONAN_* config
- communication_subroutine.py: queue timeout/Empty, daemon threads,
cooperative stop_event, inbound _authorize_request()
- bot.py: asyncio event loop + load conanjurer_commands
- music_functions / radio_commands / librarian_commands: X-Conjurer-Api-Key
- conanjurer_commands/_functions: fixed + integrated bridge with RCON
player-join notifications
- requirements_bot.txt: aiomcrcon, asyncssh
- conjurer_musician/.gitignore: keep runtime playlists/mp3 out of the repo
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the stable 'working copy' bot the canonical code at the repository
root so the install/deploy scripts operate against it again.
- Move working_copy/* to root (bot entrypoint is bot.py)
- Restore root-level install/ops scripts from c4fa88e (deploy.sh,
install_main_bot.sh, status_report.*, conjurer.service, etc.)
- Fix deploy.sh: copy bot.py (was thin_client.py) and add the
conanjurer_* modules; bump command count
- Remove side-by-side variant dirs (backup_old_docker, prototype_one,
prototype_musician_one, musician_old, working_copy) and docker cruft
- Keep components as subdirs: conjurer_librarian, conjurer_musician,
spotify_dl, yt_dlp, fonts, utils, docs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* New requirement
* SMall fixes for tunnel
* First step towards cogization
* Tmp chng
* Testing of new functions in conjurer_musician.py and radio_conjurer.liq
* Fixes!
* Fix fix
* Mostly working ?
* Requests
* Fixes
* Fix
* Radio requests
* Radio hardkor
* Fix
* Fixes
* Change log to print
* Clearing fetch failed
* Small fix
* sx
* Fix fix fix
* test
* el fixo stupido
* test
* Maybe fix ?
* FIX!
* Fix fix fix
* FIx of volume
gitea
Michal Tuszowski