search_bot conflated MAXTHREADS into two jobs at once - how many chunk files to
read (files 0..MAXTHREADS-1) AND how many producer sentinels to wait for - so
the two had to match exactly. Set too low it silently skipped trailing chunks;
set too high (or with any chunk missing/unreadable) a producer crashed before
emitting its sentinel, the consumers' count never reached the threshold, and
search_for_doi hung on join() forever. The idle-timeout failsafe that was meant
to break a starved consumer was dead code: `if empty_counter > 5: ... elif
empty_counter > 10: break` - >10 implies >5, so the elif never ran.
Fix, three layers:
* auto-discover the chunk files present (discover_chunk_files: <n>_chunk.txt in
numeric order) instead of range(0, MAXTHREADS). All files are read regardless
of count, and no producer is ever pointed at a missing file;
* the sentinel threshold is now the number of producers actually started, so it
can't drift from what's emitted;
* producers emit their sentinel in a finally, so even a crash (missing/unreadable
chunk) can't starve the count; and the idle backstop is reordered so it can
actually fire (>EMPTY_LIMIT seconds) as a last resort.
MAXTHREADS is deprecated and unused (kept only so old env files don't break);
docs/env updated to say chunk files are auto-discovered.
For the reported case (MAXTHREADS=40, files 0..43): before, files 40-43 were
silently never searched, and any run that referenced a missing chunk hung
forever. After, all 44 are searched and it always terminates.
Verified in a pytest-only venv (tests/unit/test_search_bot.py): DOI in a
trailing chunk is found; an unreadable chunk still terminates; empty dir returns
at once; discovery is numeric-sorted. Full unit job 27 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire the bot's AI chat pipeline (ai_functions.handle_response) to talk to
either OpenAI or the Anthropic Messages API, chosen by one active-config
switch. Behaviour on the default "gpt" config is unchanged.
constants.py:
* guarded `import anthropic` + CLAUDECLIENT (mirrors OPENAICLIENT), netrc
machine 'anthropic' / ANTHROPIC_API_KEY;
* CLAUDE_LATEST_MODEL / CLAUDE_CHEAP_MODEL (opus-4-8 / haiku-4-5);
* AI_CONFIGS + DEFAULT_AI_CONFIG loaded from an optional 3rd element of
system_gpt_settings.json (backward compatible - a 2-element file falls
back to built-in defaults, active "gpt"). Single switch: CONJURER_AI_CONFIG
env > settings "active" > "gpt".
ai_functions.py:
* provider_generate() dispatches to OpenAI (unchanged openai_call) or the new
_anthropic_call() (splits system out, alternating messages, max_tokens,
temperature omitted - Opus 4.8 rejects sampling params);
* AIError normalises both SDKs' exceptions into one category set so
handle_response keeps its single set of in-character error replies;
* select_model() reads the active config; legacy "gpt-4o" default auto-maps
to the active provider's model so the switch actually changes the backend;
* set_active_ai_config()/list_ai_configs() with best-effort persistence back
into system_gpt_settings.json index 2.
ai_commands.py:
* $gadaj_teraz <config> hybrid command (Vykidailo-gated) switches backend at
runtime;
* graceful guards when OPENAICLIENT is None: personal assistants (OpenAI
Assistants API) and DALL-E image gen degrade instead of crashing, so a
Claude-only deployment boots.
system_gpt_settings.json: add the configs block (gpt/claude/_template) as the
collection point for future backends. requirements_bot.txt: add anthropic.
bot.env.example: ANTHROPIC_API_KEY + CONJURER_AI_CONFIG. Unit tests cover the
message splitter, model selection, config listing, and error mapping.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The two scaffold workflows (Python application / Python package) failed on
every PR: they installed deps from a non-existent requirements.txt, ran
flake8/pytest over the vendored yt_dlp fork (new syntax under the 3.8/3.9
matrix), and collected ad-hoc root scripts — notably test_ai.py, which is
an invalid pasted object dump (not Python).
- Remove python-app.yml / python-package.yml and the junk root scripts
(test.py, test_ai.py, test_time.py)
- Add .github/workflows/ci.yml with three PR-check jobs:
* compile — py_compile every first-party .py (no deps)
* unit — pytest on pure logic (conanjurer_functions, constants)
* integration — boot the Flask services and assert the X-Conjurer-Api-Key
auth contract (communication_subroutine + conjurer_musician)
- Add tests/ suite, pytest.ini (testpaths=tests) and conftest.py (sys.path)
Fixes surfaced by the compile gate / needed for the integration job:
- conjurer_librarian/search_bot.py + search_bot2.py: f-string reused the
same quote ({item["exists"]}) -> SyntaxError on Python < 3.12
- conjurer_musician/media_search_functions.py: made import-safe
(env-overridable paths, lazy DB load / mkdir) so the service can be
imported and tested off the Pi
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gitea
Michal Tuszowski