5 Commits

Author	SHA1	Message	Date
gitea	868ea96e15	lore: bound pamiec.json by summarising old memory into "Legendy Baru" ... The conversation memory file grows forever (every chat appends a user+assistant pair), so startup load gets slower and the disk fills. New always-loaded cog lore_commands turns that growth into content: a background task summarises the oldest slice into one in-character "legend" via the ACTIVE AI backend, replaces those old messages with the summary (bounding the file, keeping continuity for the next startup's context), archives it to legendy.json, and announces it on #legendy. Safety: the compaction transforms (build_transcript, apply_compaction) are pure and unit-tested. The file rewrite is re-read -> back up -> atomic write with no await in between, so a handle_response append that lands while the summary is being generated can neither be lost (it's in the preserved tail) nor corrupt the file (single-threaded, no interleave). A .bak is kept. Scope note: this bounds the on-disk file (startup/disk); the in-RAM MESSAGE_TABLE is a separate concern left untouched to avoid yanking context from a live conversation. Commands: $zapisz_legende (Vykidailo) forces a compaction now; $legendy recalls a random past legend. All thresholds env-overridable (CONJURER_MEMORY_COMPACT_*, CONJURER_LEGENDS_CHANNEL). constants gains LEGENDS_FILE + config + seed; bot.py registers the cog. Verified: tests/unit/test_lore_commands.py covers prefix-replace/tail-keep, preservation of appends made during summarisation, and transcript formatting + head/tail truncation. Unit job 32 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-31 17:00:34 +02:00
gitea	031c1f8aea	librarian: tolerate bad bytes in chunks; log what the result-send does ... Two field-reported robustness gaps on top of the hang fix. 1) A stray non-UTF-8 byte in a chunk (0x96 in the report) raised UnicodeDecodeError from readline() - which is a ValueError, so the earlier `except OSError` did NOT catch it. The finally-sentinel meant no hang, but the producer died mid-file with a loud traceback and every DOI after the bad byte went unsearched. Now chunks are opened with errors="replace" (bad bytes become U+FFFD; DOIs are ASCII so a match is never affected) so the read runs to EOF, and the producer's except is broadened from OSError to Exception so no per-file error can ever crash the thread - it's logged and the sentinel still fires. 2) The result-send back to the bot (BackgroundTaskSearch._run) now logs exactly what goes out - target URL, uuid, DOI count and the DOI list - so the librarian log plainly shows a result was sent and what was in it. And a failed POST is no longer fatal: a RequestException used to propagate out of the worker loop and kill the thread, stalling every future query until restart; it's now caught and logged, and a non-200 from the bot is logged as a warning. Verified: tests/unit/test_search_bot.py gains a case writing a chunk with a 0x96 byte before a valid DOI and asserting that DOI is still found (file read to completion, not aborted). All 5 search_bot unit tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-30 15:59:07 +02:00
gitea	1a59c9f6c5	librarian: stop the DOI search from hanging on a chunk-count mismatch ... search_bot conflated MAXTHREADS into two jobs at once - how many chunk files to read (files 0..MAXTHREADS-1) AND how many producer sentinels to wait for - so the two had to match exactly. Set too low it silently skipped trailing chunks; set too high (or with any chunk missing/unreadable) a producer crashed before emitting its sentinel, the consumers' count never reached the threshold, and search_for_doi hung on join() forever. The idle-timeout failsafe that was meant to break a starved consumer was dead code: `if empty_counter > 5: ... elif empty_counter > 10: break` - >10 implies >5, so the elif never ran. Fix, three layers: * auto-discover the chunk files present (discover_chunk_files: <n>_chunk.txt in numeric order) instead of range(0, MAXTHREADS). All files are read regardless of count, and no producer is ever pointed at a missing file; * the sentinel threshold is now the number of producers actually started, so it can't drift from what's emitted; * producers emit their sentinel in a finally, so even a crash (missing/unreadable chunk) can't starve the count; and the idle backstop is reordered so it can actually fire (>EMPTY_LIMIT seconds) as a last resort. MAXTHREADS is deprecated and unused (kept only so old env files don't break); docs/env updated to say chunk files are auto-discovered. For the reported case (MAXTHREADS=40, files 0..43): before, files 40-43 were silently never searched, and any run that referenced a missing chunk hung forever. After, all 44 are searched and it always terminates. Verified in a pytest-only venv (tests/unit/test_search_bot.py): DOI in a trailing chunk is found; an unreadable chunk still terminates; empty dir returns at once; discovery is numeric-sorted. Full unit job 27 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-30 00:15:03 +02:00
Michal Tuszowski	3d9d47aa90	ai: single-switch GPT/Claude backend for the chat cog ... Wire the bot's AI chat pipeline (ai_functions.handle_response) to talk to either OpenAI or the Anthropic Messages API, chosen by one active-config switch. Behaviour on the default "gpt" config is unchanged. constants.py: * guarded `import anthropic` + CLAUDECLIENT (mirrors OPENAICLIENT), netrc machine 'anthropic' / ANTHROPIC_API_KEY; * CLAUDE_LATEST_MODEL / CLAUDE_CHEAP_MODEL (opus-4-8 / haiku-4-5); * AI_CONFIGS + DEFAULT_AI_CONFIG loaded from an optional 3rd element of system_gpt_settings.json (backward compatible - a 2-element file falls back to built-in defaults, active "gpt"). Single switch: CONJURER_AI_CONFIG env > settings "active" > "gpt". ai_functions.py: * provider_generate() dispatches to OpenAI (unchanged openai_call) or the new _anthropic_call() (splits system out, alternating messages, max_tokens, temperature omitted - Opus 4.8 rejects sampling params); * AIError normalises both SDKs' exceptions into one category set so handle_response keeps its single set of in-character error replies; * select_model() reads the active config; legacy "gpt-4o" default auto-maps to the active provider's model so the switch actually changes the backend; * set_active_ai_config()/list_ai_configs() with best-effort persistence back into system_gpt_settings.json index 2. ai_commands.py: * $gadaj_teraz <config> hybrid command (Vykidailo-gated) switches backend at runtime; * graceful guards when OPENAICLIENT is None: personal assistants (OpenAI Assistants API) and DALL-E image gen degrade instead of crashing, so a Claude-only deployment boots. system_gpt_settings.json: add the configs block (gpt/claude/_template) as the collection point for future backends. requirements_bot.txt: add anthropic. bot.env.example: ANTHROPIC_API_KEY + CONJURER_AI_CONFIG. Unit tests cover the message splitter, model selection, config listing, and error mapping. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 21:34:59 +02:00
Michal Tuszowski	a6c20a0054	ci: replace broken default workflows with compile/unit/integration CI ... The two scaffold workflows (Python application / Python package) failed on every PR: they installed deps from a non-existent requirements.txt, ran flake8/pytest over the vendored yt_dlp fork (new syntax under the 3.8/3.9 matrix), and collected ad-hoc root scripts — notably test_ai.py, which is an invalid pasted object dump (not Python). - Remove python-app.yml / python-package.yml and the junk root scripts (test.py, test_ai.py, test_time.py) - Add .github/workflows/ci.yml with three PR-check jobs: * compile — py_compile every first-party .py (no deps) * unit — pytest on pure logic (conanjurer_functions, constants) * integration — boot the Flask services and assert the X-Conjurer-Api-Key auth contract (communication_subroutine + conjurer_musician) - Add tests/ suite, pytest.ini (testpaths=tests) and conftest.py (sys.path) Fixes surfaced by the compile gate / needed for the integration job: - conjurer_librarian/search_bot.py + search_bot2.py: f-string reused the same quote ({item["exists"]}) -> SyntaxError on Python < 3.12 - conjurer_musician/media_search_functions.py: made import-safe (env-overridable paths, lazy DB load / mkdir) so the service can be imported and tested off the Pi Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-29 12:24:30 +02:00