AI: personal assistants without the dead API, and keep Ollama warm

Two things the field report asked for.

1) PERSONAL ASSISTANTS (replacing the sunset OpenAI Assistants API)

The old implementation gave three capabilities. Two are reimplemented here,
the third was confirmed unused and is deliberately not replaced:

 * per-user persona - it already lived in system_gpt_settings.json; it was
   only ever being shipped to OpenAI. It is now the system prompt.
 * per-user conversation thread - OpenAI held this server-side. It now lives
   in assistant_memory.json, keyed by discord user id, trimmed to the most
   recent turns (CONJURER_ASSISTANT_MEMORY_TURNS) and written atomically so a
   torn write cannot lose someone's history. Deliberately a plain trim, not
   the AI summarisation used for the bar's shared memory: these are private
   DMs and must not end up in a public "legend".
 * file_search - not replaced. Confirmed not in use.

The conversation goes through handle_response with request_type="NONE" and an
explicit message list, which keeps it out of the bar's shared memory. The big
win: create_chat_assistant hardcoded model="gpt-4o", so assistants were locked
to OpenAI. They now run on whatever $gadaj_teraz selects - Claude and Ollama
included.

create_chat_assistant / chat_with_assistant are gone, and with them the last
call to beta.threads in the startup path - so the cog cannot be killed by that
API again. (add_files_to_vector_store / delete_files_from_vector_store still
reference beta.assistants but are dead code - nothing calls them - so they
cannot crash anything; left alone rather than widening this change.)

2) KEEPING A SELF-HOSTED MODEL WARM

Loading is the slow part - the GPU is shared with other users - so we preload
via Ollama's documented mechanism: /api/generate with a model, a keep_alive
and NO prompt. It loads the model and generates nothing.

 * on switching to ollama, $gadaj_teraz fires a preload in the BACKGROUND
   (not awaited: loading can take minutes and the command must answer at
   once), so the wait lands on the operator rather than the first user;
 * a warm loop re-asserts keep_alive every CONJURER_OLLAMA_WARM_MINUTES.

Both are hard-guarded on the ACTIVE provider being ollama. Warming a metered
API would burn tokens and money for nothing, so that guard is pinned by a test
asserting the preload is never called for gpt/claude, and another asserting the
preload body carries no prompt (a prompt would make every warm-up generate).

Tests: 82 unit + 71 integration green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

ai_commands.py

@@ -1,4 +1,5 @@
 # ai command cogs
+import asyncio
 import logging
 import re
 import sys
@@ -17,7 +18,7 @@ from communication_subroutine import AI_QUERY_Q
 
 import ai_functions
 from constants import (
-    ASSISTANTS,
+    OLLAMA_WARM_MINUTES,
     DATA,
     GRAPHICS_PATH,
     INITIAL_TIME_WAIT,
@@ -101,55 +102,45 @@ class Events(commands.Cog):
             text = text[1900:]
 
     async def cog_load(self):
-        # The AI query worker must run regardless of the OpenAI guard below - it
-        # answers via handle_response, which works on Claude too. Start it first.
+        # The AI query worker answers via handle_response, so it works on every
+        # backend. Start it first.
         if not self.ai_query_worker.is_running():
             self.ai_query_worker.start()
-        self.logger.info("Starting personal assistants")
-        # Personal assistants use the OpenAI Assistants API (threads/runs), which
-        # has no Anthropic equivalent - skip cleanly when OpenAI isn't wired up
-        # (e.g. a Claude-only deployment) instead of crashing the cog load.
-        if OPENAICLIENT is None:
-            self.logger.warning(
-                "OPENAICLIENT niedostępny - osobiści asystenci (OpenAI Assistants API) wyłączeni"
-            )
-            return
-        for superfryta_id, superfryta in SPECJALNE_ZIEMNIACZKI.items():
+        # Keeps a self-hosted model resident; it no-ops on any other provider.
+        if not self.ollama_warm_loop.is_running():
+            self.ollama_warm_loop.start()
+        # NOTE: there is no OpenAI-Assistants bootstrap any more. It called a
+        # sunset API (beta threads), 404'd, and failed the WHOLE extension -
+        # taking every AI command with it. Personal assistants now ride
+        # handle_response with per-user memory (ai_functions), so they work on
+        # Claude and Ollama too and nothing has to be created at startup.
+        self.logger.info("Osobiści asystenci: pamięć per-user, aktywny backend AI")
 
-            if superfryta[4] != "":
-                self.logger.info(
-                    "Personal assistant for user: %s, exists id: %s,name: %s, owner: %s, special instructions: %s assistant id: %s ",
-                    superfryta_id,
-                    superfryta[0],
-                    superfryta[1],
-                    superfryta[2],
-                    superfryta[3],
-                    superfryta[4],
-                )
-                thread = await OPENAICLIENT.beta.threads.create()
-                self.logger.info("Thread id: %s", thread.id)
-                ASSISTANTS[superfryta[1]] = (
-                    superfryta[2],
-                    superfryta[4],
-                    superfryta[0],
-                    thread,
-                )
-            else:
-                self.logger.info(
-                    "Creating personal assistant for user: %s, id: %s,name: %s, owner: %s, special instructions: %s",
-                    superfryta_id,
-                    superfryta[0],
-                    superfryta[1],
-                    superfryta[2],
-                    superfryta[3],
-                )
-                await ai_functions.create_chat_assistant(
-                    superfryta_id, superfryta[0], superfryta[1], superfryta[2], superfryta[3]
-                )
-        self.logger.info("Started personal assistants")
+    @tasks.loop(minutes=OLLAMA_WARM_MINUTES)
+    async def ollama_warm_loop(self):
+        """Keep a self-hosted model resident so users don't pay the load wait.
+
+        Loading is the slow part on a GPU shared with other users, so we
+        re-assert Ollama's keep_alive well inside its window. This preloads
+        WITHOUT generating - no tokens, no cost.
+
+        Hard guard: it does nothing unless the ACTIVE backend is Ollama. Firing
+        warm-ups at a metered API would burn tokens and money for nothing.
+        """
+        try:
+            if ai_functions.active_provider() != "ollama":
+                return
+            await ai_functions.warm_active_model()
+        except Exception as exc:  # pylint: disable=broad-exception-caught
+            self.logger.info("Rozgrzewanie Ollamy nieudane (nieszkodliwe): %s", exc)
+
+    @ollama_warm_loop.before_loop
+    async def before_ollama_warm_loop(self):
+        await self.bot.wait_until_ready()
 
     async def cog_unload(self):
         self.ai_query_worker.cancel()
+        self.ollama_warm_loop.cancel()
 
     @commands.hybrid_command(
         name="switch_dm_mode",
@@ -174,19 +165,57 @@ class Events(commands.Cog):
                 await ctx.reply("Nope. Nie wiesz jak użyć")
 
     @commands.hybrid_command(
-        name="gadaj_teraz",
-        description="Pokaż/przełącz backend AI (bez argumentu = status). Przełączanie: Vykidailo.",
+        name="modele_ai",
+        description="Pokaż modele dostępne dla danego backendu (Ollamę pyta na żywo).",
     )
-    async def gadaj_teraz(self, ctx, nazwa_konfigu: Optional[str] = None):
+    async def modele_ai(self, ctx, nazwa_konfigu: Optional[str] = None):
+        """Read-only model listing. For Ollama this queries the server, so it
+        shows exactly what is pulled on the box right now."""
+        async with ctx.channel.typing():
+            target = nazwa_konfigu or ai_functions.get_active_ai_config()
+            if target not in ai_functions.list_ai_configs():
+                await discord_friendly_reply(
+                    ctx,
+                    f"Nie znam configu '{target}'. Dostępne: "
+                    f"{', '.join(ai_functions.list_ai_configs())}",
+                )
+                return
+            try:
+                models = await ai_functions.list_provider_models(target)
+            except ai_functions.AIError as exc:
+                await discord_friendly_reply(
+                    ctx, f"Nie mogę pobrać modeli dla '{target}': {exc}"
+                )
+                return
+            if not models:
+                await discord_friendly_reply(ctx, f"Brak modeli dla '{target}'.")
+                return
+            await discord_friendly_reply(
+                ctx,
+                f"Modele dla **{target}**: {', '.join(models)}\n"
+                f"Wepniesz przez `$gadaj_teraz {target} <model>` (tylko Vykidailo).",
+            )
+
+    @commands.hybrid_command(
+        name="gadaj_teraz",
+        description="Pokaż/przełącz backend AI i model (bez argumentu = status). Przełączanie: Vykidailo.",
+    )
+    async def gadaj_teraz(
+        self, ctx, nazwa_konfigu: Optional[str] = None, model: Optional[str] = None
+    ):
         async with ctx.channel.typing():
             available = ai_functions.list_ai_configs()
             # No argument -> report the active backend (read-only, open to all).
             if not nazwa_konfigu:
                 active = ai_functions.get_active_ai_config()
+                active_cfg = ai_functions.AI_CONFIGS.get(active, {})
                 await discord_friendly_reply(
                     ctx,
-                    f"Teraz gadam przez **{active}**. Dostępne: {', '.join(available)}. "
-                    "Przełączysz przez `$gadaj_teraz <config>` (tylko Vykidailo).",
+                    f"Teraz gadam przez **{active}** "
+                    f"({active_cfg.get('provider')} / {active_cfg.get('latest_model')}). "
+                    f"Dostępne: {', '.join(available)}. "
+                    "Przełączysz przez `$gadaj_teraz <config> [model]` (tylko Vykidailo), "
+                    "modele zobaczysz przez `$modele_ai`.",
                 )
                 return
             is_admin = isinstance(ctx.author, discord.Member) and any(
@@ -208,13 +237,69 @@ class Events(commands.Cog):
                     ctx, f"Nie mogę przełączyć na '{nazwa_konfigu}': {exc}"
                 )
                 return
+
+            # Optional second argument pins the model. For a backend we can
+            # enumerate (Ollama), reject an unknown id up front with the list -
+            # otherwise the typo only surfaces later as a failed reply.
+            if model:
+                try:
+                    known = await ai_functions.list_provider_models(nazwa_konfigu)
+                except ai_functions.AIError:
+                    known = []  # cannot enumerate -> accept verbatim
+                if known and cfg.get("provider") == "ollama" and model not in known:
+                    await discord_friendly_reply(
+                        ctx,
+                        f"Model '{model}' nie jest wgrany na Ollamie. "
+                        f"Dostępne: {', '.join(known)}",
+                    )
+                    return
+                try:
+                    cfg = ai_functions.set_active_model(model, nazwa_konfigu)
+                except (KeyError, ValueError) as exc:
+                    await discord_friendly_reply(
+                        ctx, f"Nie mogę wpiąć modelu '{model}': {exc}"
+                    )
+                    return
+
             self.logger.info(
-                "Przełączono AI na config %s (%s)", nazwa_konfigu, cfg.get("provider")
+                "Przełączono AI na config %s (%s / %s)",
+                nazwa_konfigu, cfg.get("provider"), cfg.get("latest_model"),
             )
-            await discord_friendly_reply(
-                ctx,
-                f"Teraz gadam przez **{nazwa_konfigu}** — {cfg.get('provider')} / {cfg.get('latest_model')}.",
+            message = (
+                f"Teraz gadam przez **{nazwa_konfigu}** — "
+                f"{cfg.get('provider')} / {cfg.get('latest_model')}."
             )
+            if cfg.get("provider") == "ollama":
+                # Pay the (slow, shared-GPU) load cost NOW, in the background,
+                # so it lands on the operator switching backends rather than on
+                # whoever sends the first message. Not awaited: loading can take
+                # minutes and the command must answer immediately.
+                asyncio.create_task(ai_functions.warm_active_model())
+                message += (
+                    "\nRozgrzewam model w tle — pierwsza odpowiedź może chwilę potrwać."
+                )
+            # Switched without pinning a model: show what else is on offer.
+            if not model:
+                try:
+                    others = await ai_functions.list_provider_models(nazwa_konfigu)
+                except ai_functions.AIError:
+                    others = []
+                current = cfg.get("latest_model")
+                if others and current not in others:
+                    # The configured/pinned model is not on the server: every
+                    # reply would fail with "model not found" and nothing would
+                    # say why. Flag it here, where the list is already in hand.
+                    message += (
+                        f"\n⚠ Uwaga: '{current}' nie jest wgrany na serwerze. "
+                        f"Dostępne: {', '.join(others)} "
+                        f"(`$gadaj_teraz {nazwa_konfigu} <model>`)."
+                    )
+                elif len(others) > 1:
+                    message += (
+                        f"\nDostępne modele: {', '.join(others)} "
+                        f"(`$gadaj_teraz {nazwa_konfigu} <model>`)."
+                    )
+            await discord_friendly_reply(ctx, message)
 
     @commands.hybrid_command(
         name="armia_hammera",
@@ -326,8 +411,10 @@ class Events(commands.Cog):
                 if message.author.id == superfryta[0]:
                     self.logger.info("Specjalny ziemniak")
                     if self.armia[message.author.id] == Dm_Mode.SPECJALNY_ZIEMNIACZEK:
-                        #await self.bot.process_commands(message)
-                        await ai_functions.chat_with_assistant(message, superfryta[1])
+                        # superfryta = [discord_id, assistant_name, owner, instructions, legacy_assistant_id]
+                        await ai_functions.chat_with_personal_assistant(
+                            message, superfryta[2], superfryta[3]
+                        )
                         return
                     elif self.armia[message.author.id] == Dm_Mode.ECHO_ECHO:
                         await ai_functions.echo(message)

ai_functions.py

@@ -1,15 +1,21 @@
 import asyncio
 import json
 import logging
+import os
 import random
+import tempfile
 
 import openai
 import tiktoken
 import time
 from other_functions import discord_friendly_send
+import requests
+
 from constants import (
     AI_CONFIGS,
-    ASSISTANTS,
+    AI_TIMEOUT_SECONDS,
+    ASSISTANT_MEMORY_FILE,
+    ASSISTANT_MEMORY_TURNS,
     CLAUDECLIENT,
     CYCLIC_WORDS,
     DEFAULT_AI_CONFIG,
@@ -19,6 +25,10 @@ from constants import (
     MEMORY_FIVE_SIARA,
     MESSAGE_TABLE,
     MESSAGE_TABLE_MUZYKA,
+    OLLAMACLIENT,
+    OLLAMA_KEEP_ALIVE,
+    OLLAMA_PRELOAD_TIMEOUT,
+    OLLAMA_URL,
     OPENAICLIENT,
     SYSTEM_GPT_SETTINGS,
     WORD_REACTIONS,
@@ -92,12 +102,54 @@ def set_active_ai_config(name: str) -> dict:
         raise RuntimeError("klient Anthropic nie jest skonfigurowany (brak ANTHROPIC_API_KEY)")
     if provider == "openai" and OPENAICLIENT is None:
         raise RuntimeError("klient OpenAI nie jest skonfigurowany (brak OPENAI_API_KEY)")
+    if provider == "ollama" and OLLAMACLIENT is None:
+        raise RuntimeError("Ollama nie jest skonfigurowana (ustaw CONJURER_OLLAMA_URL)")
     _ACTIVE_CONFIG_NAME = name
     _persist_active_ai_config(name)
     return cfg
 
 
-def _persist_active_ai_config(name: str) -> None:
+async def list_provider_models(name: str = None):
+    """Model ids selectable for a config.
+
+    For Ollama this ASKS THE SERVER (its OpenAI-compatible /v1/models), so the
+    picker always reflects what is actually pulled on the box rather than a
+    hardcoded list. Hosted providers are not enumerated - we only report what
+    the config is wired to.
+    """
+    cfg = AI_CONFIGS.get(name or _ACTIVE_CONFIG_NAME) or _active_config()
+    if cfg.get("provider") == "ollama":
+        if OLLAMACLIENT is None:
+            raise AIError(
+                "auth",
+                RuntimeError("Ollama nie jest skonfigurowana (ustaw CONJURER_OLLAMA_URL)"),
+            )
+        try:
+            resp = await OLLAMACLIENT.models.list()
+        except Exception as exc:  # pylint: disable=broad-except
+            raise _map_openai_error(exc)
+        return sorted({item.id for item in resp.data})
+    return [m for m in (cfg.get("latest_model"), cfg.get("cheap_model")) if m]
+
+
+def set_active_model(model: str, name: str = None) -> dict:
+    """Pin the model a config uses for normal replies, and persist it.
+
+    Only ``latest_model`` is changed; ``cheap_model`` stays as configured so the
+    MUSIC path keeps its cheaper backend.
+    """
+    cfg_name = name or _ACTIVE_CONFIG_NAME
+    if cfg_name not in AI_CONFIGS:
+        raise KeyError(cfg_name)
+    if not model or not model.strip():
+        raise ValueError("pusta nazwa modelu")
+    cfg = AI_CONFIGS[cfg_name]
+    cfg["latest_model"] = model.strip()
+    _persist_active_ai_config(_ACTIVE_CONFIG_NAME, model_for=cfg_name)
+    return cfg
+
+
+def _persist_active_ai_config(name: str, model_for: str = None) -> None:
     """Best-effort write of the active-config choice into system_gpt_settings.json.
 
     Keeps the historical two-element structure intact: updates index 2 if it
@@ -117,7 +169,16 @@ def _persist_active_ai_config(name: str) -> None:
         return
     if len(data) > 2 and isinstance(data[2], dict):
         data[2]["active"] = name
-        data[2].setdefault("configs", AI_CONFIGS)
+        # Write back ONLY what this process actually changed. Assigning the whole
+        # in-memory AI_CONFIGS here would clobber operator hand-edits (the only
+        # way to change cheap_model/temperature/max_tokens) and re-seed configs
+        # deliberately deleted from the file, because AI_CONFIGS is the built-in
+        # defaults merged under the file. setdefault alone is not enough either:
+        # it would drop a model pinned via set_active_model, hence model_for.
+        block = data[2].setdefault("configs", AI_CONFIGS)
+        if model_for and model_for in AI_CONFIGS:
+            entry = block.setdefault(model_for, dict(AI_CONFIGS[model_for]))
+            entry["latest_model"] = AI_CONFIGS[model_for]["latest_model"]
     else:
         data = data[:2] + [{"active": name, "configs": AI_CONFIGS}]
     try:
@@ -214,12 +275,83 @@ async def _anthropic_call(messages, model, cfg):
     return text.strip()
 
 
+async def _ollama_call(messages, model, cfg):
+    """Self-hosted counterpart of openai_call. Returns a plain string.
+
+    Ollama exposes an OpenAI-compatible /v1 surface, so the same message format
+    and the same error mapping apply - only the base_url and the model ids
+    differ. Chat Completions (not the Responses API) is what Ollama implements.
+    """
+    if OLLAMACLIENT is None:
+        raise AIError(
+            "auth",
+            RuntimeError("Ollama nie jest skonfigurowana (ustaw CONJURER_OLLAMA_URL)"),
+        )
+    try:
+        resp = await OLLAMACLIENT.chat.completions.create(
+            model=model,
+            messages=messages,
+            temperature=float(cfg.get("temperature", 0.2)),
+        )
+    except Exception as exc:  # pylint: disable=broad-except
+        raise _map_openai_error(exc)
+    return (resp.choices[0].message.content or "").strip()
+
+
+def _ollama_preload(model, keep_alive=None) -> bool:
+    """Load ``model`` into Ollama and keep it resident, generating NOTHING.
+
+    Ollama's /api/generate with a model and no prompt is the documented preload:
+    it pays the (slow, GPU-shared) load cost once and returns, producing no
+    tokens. Used to warm up on switch and to re-assert keep_alive periodically.
+
+    Blocking on purpose - callers wrap it in asyncio.to_thread.
+    """
+    if not OLLAMA_URL:
+        return False
+    logger = logging.getLogger("discord")
+    try:
+        resp = requests.post(
+            f"{OLLAMA_URL}/api/generate",
+            json={"model": model, "keep_alive": keep_alive or OLLAMA_KEEP_ALIVE},
+            timeout=OLLAMA_PRELOAD_TIMEOUT,
+        )
+        ok = resp.status_code == 200
+        logger.info("Ollama preload %s -> HTTP %s", model, resp.status_code)
+        return ok
+    except requests.exceptions.RequestException as exc:
+        logger.info("Ollama preload %s failed: %s", model, exc)
+        return False
+
+
+def active_provider() -> str:
+    """Provider of the active config - the guard every warm-up must check.
+
+    Preloading only makes sense for a self-hosted model; firing it at a metered
+    API would burn tokens (and money) for nothing.
+    """
+    return (_active_config() or {}).get("provider", "")
+
+
+async def warm_active_model(force_model=None) -> bool:
+    """Preload the active model IFF the active backend is Ollama."""
+    if active_provider() != "ollama":
+        return False
+    cfg = _active_config()
+    model = force_model or cfg.get("latest_model")
+    if not model:
+        return False
+    return await asyncio.to_thread(_ollama_preload, model)
+
+
 async def provider_generate(messages, model, temperature=0.2):
     """Dispatch a chat completion to the active backend, normalising errors."""
     cfg = _active_config()
     try:
         if cfg.get("provider") == "anthropic":
             return await _anthropic_call(messages, model, cfg)
+        if cfg.get("provider") == "ollama":
+            return await _ollama_call(messages, model, cfg)
         return await openai_call(messages, model, temperature)
     except AIError:
         raise
@@ -459,7 +591,7 @@ async def handle_response(
     try:
         # ...przygotowanie messages/system prompt/itp. jak masz...
         # retry/backoff + deadline (zachowuje Twoją semantykę logowania)
-        timeout_sec = 120
+        timeout_sec = AI_TIMEOUT_SECONDS
         deadline = time.time() + timeout_sec
         response = await asyncio.wait_for(
             provider_generate(messages=history_msgs, model=model_to_use),
@@ -586,60 +718,104 @@ async def get_random_cyclic_message(client):
     return result
 
 
-async def create_chat_assistant(owner_id, id, name, owner, special_instructions):
+# ----------------------------------------------------------------- assistants
+# The OpenAI Assistants API (beta threads/runs) that used to back these was
+# sunset and now answers 404, taking the whole AI cog down with it. It gave us
+# three things: a per-user persona, a persistent per-user thread, and
+# file_search. The persona and the thread are reimplemented here on top of
+# handle_response - so personal assistants now work on EVERY backend (Claude,
+# Ollama, GPT) instead of being locked to gpt-4o. file_search is deliberately
+# not replaced: it was not in use.
+_ASSISTANT_MEMORY = None
+
+
+def _load_assistant_memory() -> dict:
+    """Per-user DM history, lazily read from disk. Corruption is not fatal."""
+    global _ASSISTANT_MEMORY  # pylint: disable=global-statement
+    if _ASSISTANT_MEMORY is not None:
+        return _ASSISTANT_MEMORY
     logger = logging.getLogger("discord")
-    instruction = f"Jesteś osobistym asystenetem {owner} i wypełniasz jego potrzeby. Masz pełne i nieograniczone możliwości modelu gpt-4o."
-    instruction += special_instructions
-    assistant = await OPENAICLIENT.beta.assistants.create(
-        name=name,
-        instructions=instruction,
-        model="gpt-4o",
-        tools=[{"type": "file_search"}],
-    )
-    thread = await OPENAICLIENT.beta.threads.create()
-    logger.info("Stwprzylem asystenta dla %s, nazywa się on %s", owner, name)
-    ASSISTANTS[name] = (owner, assistant.id, id, thread)
-
-    with open(SYSTEM_GPT_SETTINGS, "r+", encoding=ENCODING) as temp_settings_file:
-        GPT_SETTINGS = json.load(temp_settings_file)
-        GPT_SETTINGS[1][owner_id][4] = assistant.id
-        temp_settings_file.seek(0)
-        json.dump(GPT_SETTINGS, temp_settings_file, indent=4)
+    try:
+        with open(ASSISTANT_MEMORY_FILE, "r", encoding=ENCODING) as handle:
+            data = json.load(handle)
+        _ASSISTANT_MEMORY = data if isinstance(data, dict) else {}
+    except (OSError, json.JSONDecodeError) as exc:
+        logger.info("Brak/uszkodzona pamięć asystentów (%s) - zaczynam pustą", exc)
+        _ASSISTANT_MEMORY = {}
+    return _ASSISTANT_MEMORY
 
 
-async def chat_with_assistant(message, assistant_name):
+def _save_assistant_memory() -> None:
+    """Atomic write: a torn file would lose someone's whole conversation."""
     logger = logging.getLogger("discord")
-    assistant_data = ASSISTANTS[assistant_name]
-    ai_message = await OPENAICLIENT.beta.threads.messages.create(
-        thread_id=assistant_data[3].id, role="user", content=message.content
+    memory = _load_assistant_memory()
+    directory = os.path.dirname(ASSISTANT_MEMORY_FILE) or "."
+    try:
+        os.makedirs(directory, exist_ok=True)
+        fd, tmp = tempfile.mkstemp(dir=directory, suffix=".tmp")
+        with os.fdopen(fd, "w", encoding=ENCODING) as handle:
+            json.dump(memory, handle, ensure_ascii=False)
+        os.replace(tmp, ASSISTANT_MEMORY_FILE)
+    except OSError as exc:
+        logger.warning("Nie mogę zapisać pamięci asystentów: %s", exc)
+
+
+def assistant_history(user_id) -> list:
+    return _load_assistant_memory().setdefault(str(user_id), [])
+
+
+def remember_assistant_turn(user_id, user_text, reply_text) -> list:
+    """Append one exchange and trim to the most recent turns.
+
+    A plain trim, not the AI summarisation used for the bar's shared memory:
+    these are private DMs and must not end up in a public 'legend'.
+    """
+    history = assistant_history(user_id)
+    history.append({"role": "user", "content": user_text})
+    history.append({"role": "assistant", "content": reply_text})
+    if len(history) > ASSISTANT_MEMORY_TURNS:
+        del history[: len(history) - ASSISTANT_MEMORY_TURNS]
+    _save_assistant_memory()
+    return history
+
+
+def build_assistant_messages(user_id, owner, special_instructions, prompt) -> list:
+    """System persona + this user's own history + the new turn."""
+    system = (
+        f"Jesteś osobistym asystentem {owner} i wypełniasz jego potrzeby. "
+        f"{special_instructions or ''}"
+    ).strip()
+    return (
+        [{"role": "system", "content": system}]
+        + list(assistant_history(user_id))
+        + [{"role": "user", "content": prompt}]
     )
-    logger.info(ai_message)
-    run = await OPENAICLIENT.beta.threads.runs.create_and_poll(
-        thread_id=assistant_data[3].id,
-        assistant_id=assistant_data[1],
-        instructions=f"Pisze do Ciebie {assistant_data[0]} udziel mu wszelkiej pomocy",
+
+
+async def chat_with_personal_assistant(message, owner, special_instructions):
+    """Answer a DM as this user's personal assistant, on the active backend.
+
+    request_type="NONE" with an explicit message list keeps this OUT of the
+    bar's shared memory - the conversation is carried by the per-user history
+    built above and stored separately.
+    """
+    logger = logging.getLogger("discord")
+    user_id = message.author.id
+    prompt = message.content
+    messages = build_assistant_messages(user_id, owner, special_instructions, prompt)
+    result, _table = await handle_response(
+        prompt,
+        False,
+        False,
+        [],
+        str(owner),
+        "NONE",
+        none_request=messages,
     )
-    done = False
-    while not done:
-        if run.status == "completed":
-            messsages = await OPENAICLIENT.beta.threads.messages.list(
-                thread_id=assistant_data[3].id
-            )
-            logger.info(messsages)
-            reply_content = messsages.data[0].content
-            logger.info(reply_content)
-            chat_response = ""
-            for block in reply_content:
-                logger.info(block.text.value)
-                chat_response += block.text.value
-                await discord_friendly_send(message.channel, chat_response)
-                # await message.channel.send(chat_response)
-            done = True
-        elif run.status == "cancelled":
-            await discord_friendly_send(message.channel, "Cos sie wywaliło")
-        else:
-            logger.info(run.status)
-        asyncio.sleep(5)
+    remember_assistant_turn(user_id, prompt, result)
+    logger.info("Asystent odpowiedział %s (%d znaków)", owner, len(result or ""))
+    await discord_friendly_send(message.channel, result)
+    return result
 
 
 async def echo(message):

conjurer_librarian/conjurer_librarian.py

@@ -173,6 +173,96 @@ def _cache_put(query, deep_search, final_result) -> None:
     _cache.prune(CACHE_MAX_ENTRIES)
 
 
+# ---- "Still alive" heartbeat for the running search ------------------------
+# A deep scan runs for hours with nothing in the log between start and finish.
+# Every HEARTBEAT_SECONDS the running search says it is still going, with its
+# uuid, the phrase, and a ROUGH how-far-along. The estimate is deliberately
+# cheap: producers already record a byte offset per chunk file, and the total
+# size is stat()'d once at search start - so it costs a sum over ~40 ints.
+HEARTBEAT_SECONDS = int(_env("CONJURER_LIBRARIAN_HEARTBEAT_SECONDS", "1200"))  # 20 min
+_current_search: Dict[str, object] = {}
+_current_lock = threading.Lock()
+
+
+def _set_current_search(uuid, query, progress, live_results) -> None:
+    with _current_lock:
+        _current_search.clear()
+        _current_search.update({
+            "uuid": str(uuid), "query": str(query), "started": time.monotonic(),
+            "progress": progress, "live": live_results,
+        })
+
+
+def _clear_current_search() -> None:
+    with _current_lock:
+        _current_search.clear()
+
+
+def _progress_summary(progress):
+    """(done_bytes, total_bytes, percent) from a live progress dict. Cheap: a
+    sum over one int per chunk file. Percent is 0.0 when the total is unknown."""
+    progress = progress or {}
+    positions = progress.get("positions") or {}
+    total = progress.get("total_bytes") or 0
+    done = sum(positions.values())
+    if total > 0:
+        done = min(done, total)  # a partially-buffered tail can nudge past 100%
+        return done, total, 100.0 * done / total
+    return done, total, 0.0
+
+
+def search_heartbeat(app_logger) -> None:
+    """Log a 'still searching' line every HEARTBEAT_SECONDS while one runs."""
+    while not SHUTDOWN_EVENT.wait(HEARTBEAT_SECONDS):
+        try:
+            with _current_lock:
+                snapshot = dict(_current_search) if _current_search else None
+            if not snapshot:
+                continue  # nothing running - stay quiet
+            done, total, percent = _progress_summary(snapshot.get("progress"))
+            app_logger.info(
+                "SEARCH ALIVE %s | '%s' | ~%.1f%% przeskanowane (%.2f/%.2f GB, "
+                "~%.2f GB do końca) | %d trafień | %.0f min",
+                snapshot["uuid"], snapshot["query"], percent,
+                done / 1e9, total / 1e9, max(0, total - done) / 1e9,
+                len(snapshot.get("live") or []),
+                (time.monotonic() - snapshot["started"]) / 60.0,
+            )
+        except Exception as exc:  # pylint: disable=broad-exception-caught
+            app_logger.warning("Heartbeat tick failed: %s", exc)
+
+
+# Crossref is a public service that times out / rate-limits under load. A single
+# transient ReadTimeout used to blow up the whole (expensive) search, so every
+# habanero call is retried with backoff, and a search that still fails is retried
+# as a whole a few times before being given up on.
+CROSSREF_ATTEMPTS = int(_env("CONJURER_CROSSREF_ATTEMPTS", "4"))
+CROSSREF_BACKOFF = float(_env("CONJURER_CROSSREF_BACKOFF", "5"))
+SEARCH_MAX_ATTEMPTS = int(_env("CONJURER_SEARCH_MAX_ATTEMPTS", "3"))
+
+
+def _crossref_call(app_logger, what, func, *args, **kwargs):
+    """Run one habanero call, retrying transient failures with linear backoff.
+
+    habanero wraps httpx errors (ReadTimeout, connection resets, 5xx) in a plain
+    RuntimeError, so we cannot filter narrowly - we retry a BOUNDED number of
+    times on any failure and re-raise the last error if none succeed. Blocking
+    on purpose: callers invoke it via asyncio.to_thread, which also keeps the
+    worker's event loop free while Crossref is slow."""
+    last_exc = None
+    for attempt in range(1, max(1, CROSSREF_ATTEMPTS) + 1):
+        try:
+            return func(*args, **kwargs)
+        except Exception as exc:  # pylint: disable=broad-exception-caught
+            last_exc = exc
+            app_logger.warning(
+                "Crossref %s failed (attempt %d/%d): %s", what, attempt, CROSSREF_ATTEMPTS, exc
+            )
+            if attempt < CROSSREF_ATTEMPTS:
+                time.sleep(CROSSREF_BACKOFF * attempt)
+    raise last_exc
+
+
 def _forget_search(uuid) -> None:
     """A search is fully done (or abandoned): drop its persisted request and any
     checkpoint so it is never replayed or resumed again."""
@@ -411,14 +501,20 @@ class Librarian(object):
         
         if not deep_search:
             query_limit = MAX_CR_RESULTS if MAX_CR_RESULTS < 1000 else 1000
-            cr_result = self.cr.works(query=query, limit=query_limit)
+            cr_result = await asyncio.to_thread(
+                _crossref_call, self.app.logger, "works",
+                self.cr.works, query=query, limit=query_limit,
+            )
             self.search_result_from_cr.update(cr_result)
             self.total = cr_result["message"]["total-results"]
             self.fetched += len(cr_result["message"]["items"])
             self.app.logger.info(self.total)
             self.app.logger.info(self.fetched)
             while self.total > self.fetched and self.limit > self.fetched:
-                tmp_result = self.cr.works(query=query, limit=query_limit, offset=self.fetched)
+                tmp_result = await asyncio.to_thread(
+                    _crossref_call, self.app.logger, "works(offset)",
+                    self.cr.works, query=query, limit=query_limit, offset=self.fetched,
+                )
                 cr_result["message"]["items"].extend(tmp_result["message"]["items"])
                 self.total = tmp_result["message"]["total-results"]
                 self.fetched = len(cr_result["message"]["items"])
@@ -427,7 +523,10 @@ class Librarian(object):
                 await asyncio.sleep(0.1)
 
         else:
-            cr_result = self.cr.works(query=query, cursor_max=15000, cursor='*', progress_bar = True)
+            cr_result = await asyncio.to_thread(
+                _crossref_call, self.app.logger, "works(deep cursor)",
+                self.cr.works, query=query, cursor_max=15000, cursor='*', progress_bar=True,
+            )
             result = cr_result[0]
             for item in cr_result[1:]:
                 result["message"]["items"].extend(item["message"]["items"])
@@ -516,10 +615,18 @@ class Librarian(object):
         dois = []
         for item, value in refined_result.items():
             dois.append([item, value])
-        result, positions, interrupted = await asyncio.to_thread(
-            search_bot.search_for_doi,
-            dois, self.live_results, self.app.logger, SHUTDOWN_EVENT, resume,
-        )
+        # Publish this scan as "the running search" so the heartbeat can report
+        # it; cleared in finally so a finished/crashed scan never lingers there.
+        progress = {}
+        _set_current_search(self.uuid, self.query, progress, self.live_results)
+        try:
+            result, positions, interrupted = await asyncio.to_thread(
+                search_bot.search_for_doi,
+                dois, self.live_results, self.app.logger, SHUTDOWN_EVENT, resume,
+                progress,
+            )
+        finally:
+            _clear_current_search()
         if interrupted:
             # Graceful shutdown hit mid-scan: checkpoint found-so-far + per-file
             # resume offsets + the DOI list, so a restart continues instead of
@@ -699,6 +806,7 @@ class BackgroundTaskSearch(threading.Thread):
             # behind us, and active_queries so the bot's watchdog can tell a
             # finished-and-gone query from one still in flight.
             worker_busy.set()
+            requeued = False  # set when a crash schedules another attempt
             with _active_lock:
                 active_queries[str(librarian.uuid)] = "processing"
             try:
@@ -760,16 +868,48 @@ class BackgroundTaskSearch(threading.Thread):
                 # forget its request + checkpoint (never replay/resume it again).
                 _forget_search(uuid)
             except Exception as exc:  # pylint: disable=broad-exception-caught
-                # A crashing search must not kill the worker thread (which would
-                # freeze the whole queue). Log and give up on it - forget the
-                # request/checkpoint so it isn't retried forever as a poison pill;
-                # the bot's watchdog tells the user it vanished.
-                self.app.logger.exception("Search %s crashed: %s", librarian.uuid, exc)
-                _forget_search(str(librarian.uuid))
+                # A crashing search must not kill the worker thread (that would
+                # freeze the whole queue). It also must not silently vanish just
+                # because Crossref timed out once: retry the whole search a
+                # bounded number of times (attempt count persisted with the
+                # request, so it can't loop forever), keeping any checkpoint so a
+                # crashed DB scan resumes rather than restarts. Only after
+                # SEARCH_MAX_ATTEMPTS do we give up and let the bot's watchdog
+                # tell the user it vanished.
+                uuid = str(librarian.uuid)
+                self.app.logger.exception("Search %s crashed: %s", uuid, exc)
+                stored = _requests.get(uuid) or {}
+                attempts = int(stored.get("attempts", 0)) + 1
+                if attempts < max(1, SEARCH_MAX_ATTEMPTS):
+                    stored.update({
+                        "query": librarian.query,
+                        "deep_search": librarian.deep_search,
+                        "callback": librarian.callback,
+                        "attempts": attempts,
+                    })
+                    _requests.put(uuid, stored)
+                    librarian_queue.put(
+                        Librarian(self.app, librarian.query, uuid,
+                                  librarian.deep_search, librarian.callback)
+                    )
+                    requeued = True
+                    self.app.logger.warning(
+                        "Search %s requeued after crash (attempt %d/%d)",
+                        uuid, attempts, SEARCH_MAX_ATTEMPTS,
+                    )
+                else:
+                    self.app.logger.error(
+                        "Search %s failed %d times - giving up", uuid, attempts
+                    )
+                    _forget_search(uuid)
             finally:
                 worker_busy.clear()
                 with _active_lock:
-                    active_queries.pop(str(librarian.uuid), None)
+                    if requeued:
+                        # Still known to the bot's watchdog - it's going round again.
+                        active_queries[str(librarian.uuid)] = "queued"
+                    else:
+                        active_queries.pop(str(librarian.uuid), None)
                 await asyncio.sleep(1)
         SHUTDOWN_DONE.set()
         self.app.logger.info("Search worker stopped cleanly")
@@ -951,6 +1091,10 @@ if __name__ == "__main__":
     threads.append(
         threading.Thread(target=outbox_resender, args=(app.logger,), daemon=True)
     )
+    # "Still searching" heartbeat, so an hours-long scan isn't radio silence.
+    threads.append(
+        threading.Thread(target=search_heartbeat, args=(app.logger,), daemon=True)
+    )
     for worker in threads:
         worker.start()
     # Re-enqueue searches that were accepted/in-progress before the last stop.

conjurer_librarian/scrape_bot.py

@@ -106,8 +106,10 @@ def check_if_exists_brute_force(logger):
         ):
             pass
         if blocked:
-            logger.info(item)
-            logger.error("Got blocked. Fuck.")
+            # Expected, routine sci-hub behaviour (we back off an hour and carry
+            # on) - WARNING, not ERROR, so it stops masquerading as a fault when
+            # you're scanning the log for real problems.
+            logger.warning("Got blocked. Fuck. Backing off an hour: %s", item[0])
             time.sleep(60 * 60)
         # trunk-ignore(bandit/B311)
         rand = random.randint(1, 60)

conjurer_librarian/search_bot.py

@@ -247,7 +247,8 @@ def consumer(in_q, control_q, doi, live_results, result_list, control_dict, expe
 
 
 
-def search_for_doi(doi, live_results, _logger, stop_event=None, resume=None):
+def search_for_doi(doi, live_results, _logger, stop_event=None, resume=None,
+                   progress=None):
     """Search for DOI in live_results, resumably.
 
     Returns ``(result_list, positions, interrupted)``:
@@ -262,6 +263,11 @@ def search_for_doi(doi, live_results, _logger, stop_event=None, resume=None):
     ``resume`` is ``{"positions": {...}, "found": [doi, ...]}`` from a previous
     interrupted run: already-found DOIs are pre-marked and each producer seeks to
     its saved offset, so no already-scanned line is read twice.
+
+    ``progress``, if given, is a dict this fills with ``positions`` (the LIVE
+    dict, updated as producers read) and ``total_bytes`` (summed once, up front).
+    That makes a rough "how far along" reading free: sum the offsets, divide by
+    the total - no counting, no extra work in the read loop.
     """
     control_dict = {"sentinels":0}
     result_list = []
@@ -293,6 +299,19 @@ def search_for_doi(doi, live_results, _logger, stop_event=None, resume=None):
         )
         return result_list, positions, bool(stop_event and stop_event.is_set())
 
+    if progress is not None:
+        # One stat() per chunk file, ONCE - then progress is just sum(positions)
+        # / total_bytes, with nothing extra happening per line.
+        total_bytes = 0
+        for name in chunk_files:
+            try:
+                total_bytes += os.path.getsize(DATABASE_PATH + name)
+            except OSError:
+                pass
+        progress["positions"] = positions   # live dict, updated by the producers
+        progress["total_bytes"] = total_bytes
+        progress["chunk_files"] = expected
+
     for i in range (0, (len(doi)//1000)+2):
         t_cons = Thread(
             target=consumer,

conjurer_musician/radio_conjurer.liq

@@ -155,7 +155,13 @@ interactive.persistent("/srv/betoniarka/data/script.params")
 # Configure output formats and destinations
 
 output.icecast(%mp3, host="localhost", port=8000, password=credentials.password, icy_metadata="true", mount="mp3-stream", radio)
-output.pulseaudio(radio)
+# Local monitor output. fallible=true so a dead or missing pulse daemon degrades
+# to "no monitor" instead of failing its clock and taking the whole radio down
+# with it ("Shutdown started!"). The stream that actually matters is the Icecast
+# one above, which needs no sound device at all.
+# NOTE: input.pulseaudio() (the mic, further up) is still a HARD dependency - on
+# a headless container with no capture device, comment BOTH of them out.
+output.pulseaudio(fallible=true, radio)
 #output.file.hls("/tmp/hls", [("mp3-low", %mp3(bitrate=96)), ("mp3-hi", %mp3(bitrate=160))], radio)
 # Uncomment the following lines to enable additional output formats
 # output.icecast(%opus, host="retropie", port=8000, password=credentials.password, icy_metadata="true", mount="opus-stream", radio)

constants.py

@@ -245,6 +245,16 @@ DELIVERED_DIR = os.getenv(
 )
 DELIVERED_MAX = int(os.getenv("CONJURER_DELIVERED_MAX", "10000"))
 
+# Personal DM assistants. Replaces the OpenAI Assistants API (threads/runs),
+# which was sunset and answers 404: the persona now rides handle_response, so it
+# works on EVERY backend, and the conversation lives here instead of on OpenAI's
+# server. Kept per user so private DMs never bleed into the bar's shared memory,
+# and trimmed to the most recent turns so it cannot grow without bound.
+ASSISTANT_MEMORY_FILE = os.getenv(
+    "CONJURER_ASSISTANT_MEMORY", os.path.join(_STATE_ROOT, "assistant_memory.json")
+)
+ASSISTANT_MEMORY_TURNS = int(os.getenv("CONJURER_ASSISTANT_MEMORY_TURNS", "40"))
+
 FILE_SERVICE_ADDRESS = os.getenv("CONJURER_FILE_SERVICE", "http://192.168.1.15:5000")
 RADIO_HARBOR_ADDRESS = os.getenv("CONJURER_RADIO_HARBOR", "http://192.168.1.15:54321")
 # Betoniarka (radio-operator service colocated with Liquidsoap). Falls back to
@@ -400,6 +410,38 @@ if anthropic and ANTHROPIC_API_KEY:
 else:
     CLAUDECLIENT = None
 
+# Ollama (self-hosted models). There is no API key - the endpoint IS the whole
+# configuration, so the feature stays dormant until CONJURER_OLLAMA_URL is set
+# (same pattern as the Conan bridge). We talk to Ollama's OpenAI-COMPATIBLE
+# surface (/v1) with the openai SDK we already depend on, which means the
+# existing message format and _map_openai_error handling work unchanged.
+# How long handle_response waits for ANY backend before giving up. 120s was
+# hardcoded and is fine for hosted APIs, but a self-hosted model on a modest GPU
+# can legitimately take longer, so it is now tunable.
+AI_TIMEOUT_SECONDS = int(os.getenv("CONJURER_AI_TIMEOUT_SECONDS", "120"))
+
+OLLAMA_URL = os.getenv("CONJURER_OLLAMA_URL", "").rstrip("/")
+# Keeping a self-hosted model resident. Loading it is the slow part (it is
+# offloaded to a GPU shared with other users), so we preload it - Ollama's
+# /api/generate with a model and NO prompt loads it and generates nothing, which
+# costs no tokens and no money. KEEP_ALIVE is how long Ollama should then hold
+# it; the warm loop re-asserts that well inside the window.
+# STRICTLY Ollama-only: doing this against a paid API would burn tokens for
+# nothing, so every caller checks the active provider first.
+OLLAMA_KEEP_ALIVE = os.getenv("CONJURER_OLLAMA_KEEP_ALIVE", "30m")
+OLLAMA_WARM_MINUTES = float(os.getenv("CONJURER_OLLAMA_WARM_MINUTES", "10"))
+# A preload waits for the model to finish loading, which on a shared GPU is the
+# slow path we are trying to move off the user's first message.
+OLLAMA_PRELOAD_TIMEOUT = int(os.getenv("CONJURER_OLLAMA_PRELOAD_TIMEOUT", "600"))
+
+OLLAMA_LATEST_MODEL = os.getenv("CONJURER_OLLAMA_MODEL", "llama3.1:8b")
+OLLAMA_CHEAP_MODEL = os.getenv("CONJURER_OLLAMA_CHEAP_MODEL", OLLAMA_LATEST_MODEL)
+if openai and OLLAMA_URL:
+    # api_key is required by the SDK but ignored by Ollama.
+    OLLAMACLIENT = openai.AsyncOpenAI(base_url=f"{OLLAMA_URL}/v1", api_key="ollama")
+else:
+    OLLAMACLIENT = None
+
 TOKEN = _resolve_token("discord", "DISCORD_TOKEN")
 
 # Voice recognition (AssemblyAI). None = the voice cog reports and disables.
@@ -489,6 +531,12 @@ def _default_ai_configs():
             # sent for Claude (Opus 4.8 / Sonnet 5 reject sampling params).
             "max_tokens": 2048,
         },
+        "ollama": {
+            "provider": "ollama",
+            "latest_model": OLLAMA_LATEST_MODEL,
+            "cheap_model": OLLAMA_CHEAP_MODEL,
+            "temperature": 0.2,
+        },
         # Template for wiring further providers. Copy it, rename the key, point
         # "provider" at a backend ai_functions.provider_generate implements, and
         # fill in the model ids. Keys starting with "_" are treated as inert
@@ -508,7 +556,12 @@ _ai_block = (
     if isinstance(GPT_SETTINGS, list) and len(GPT_SETTINGS) > 2 and isinstance(GPT_SETTINGS[2], dict)
     else {}
 )
-AI_CONFIGS = _ai_block.get("configs") or _default_ai_configs()
+# Built-in defaults FIRST, then whatever the settings file defines on top. The
+# file cannot simply win outright: every provider switch persists a "configs"
+# block, so a file written by an older build would permanently hide providers
+# added later (ollama) from the picker.
+AI_CONFIGS = _default_ai_configs()
+AI_CONFIGS.update(_ai_block.get("configs") or {})
 # Single switch: env var wins, then the settings-file "active" key, then "gpt".
 DEFAULT_AI_CONFIG = (
     os.getenv("CONJURER_AI_CONFIG")

docker/entrypoint.radio.sh

@@ -11,6 +11,16 @@ CREDS="$SECRETS/icecast_credentials.json"
 mkdir -p "$DATA" "$MUSIC" "$SECRETS"
 
 # Seed the script + persistent interactive params from the image on first run.
+# NOTE: the live script is deliberately never overwritten, so hand edits win -
+# but that also means image fixes NEVER reach a volume seeded long ago. Set
+# RADIO_FORCE_SCRIPT=1 to take the image's version (the old one is kept as
+# radio_conjurer.liq.bak so nothing hand-written is lost).
+if [ -e "$DATA/radio_conjurer.liq" ] && [ "${RADIO_FORCE_SCRIPT:-0}" = "1" ]; then
+    cp "$DATA/radio_conjurer.liq" "$DATA/radio_conjurer.liq.bak"
+    cp /app/radio_conjurer.liq "$DATA/"
+    echo "RADIO_FORCE_SCRIPT=1: reseeded radio_conjurer.liq from the image" >&2
+    echo "  (previous version saved as radio_conjurer.liq.bak)" >&2
+fi
 [ -e "$DATA/radio_conjurer.liq" ] || cp /app/radio_conjurer.liq "$DATA/"
 if [ ! -e "$DATA/script.params" ]; then
     if [ -e /app/script.params ]; then cp /app/script.params "$DATA/"; else : > "$DATA/script.params"; fi
@@ -73,14 +83,36 @@ fi
 #   none               - you edited the script to drop pulse in/out.
 case "${PULSE_MODE:-internal}" in
     internal)
+        # Clear stale runtime state FIRST. `docker restart` - and the crash-loop
+        # that restart:unless-stopped produces - reuses the container's writable
+        # layer, so /run/pulse/pid left by a killed daemon survives and the next
+        # start dies with "Daemon startup failed"; that kills liquidsoap, which
+        # restarts the container, forever. Removing the pid/socket of a daemon
+        # that is demonstrably not running breaks the loop.
+        if ! pidof pulseaudio >/dev/null 2>&1; then
+            rm -f /run/pulse/pid /var/run/pulse/pid \
+                  /run/pulse/native /var/run/pulse/native 2>/dev/null || true
+        fi
         # --disallow-module-loading: modules from system.pa still load at
         # startup; this only blocks later client-requested loads (and
         # silences the system-mode warning). The "forcibly disabling SHM"
         # notice is inherent to system mode and harmless.
-        pulseaudio --system --daemonize=yes --disallow-exit \
-            --disallow-module-loading --exit-idle-time=-1 \
-            || echo "WARNING: internal pulseaudio failed to start" >&2
-        export PULSE_SERVER="${PULSE_SERVER:-unix:/var/run/pulse/native}"
+        if pulseaudio --system --daemonize=yes --disallow-exit \
+                --disallow-module-loading --exit-idle-time=-1; then
+            export PULSE_SERVER="${PULSE_SERVER:-unix:/var/run/pulse/native}"
+        else
+            # Be loud: with pulse dead, input.pulseaudio()/output.pulseaudio()
+            # fail to start, liquidsoap tears down the whole clock ("Shutdown
+            # started!") and the container crash-loops. The stream itself only
+            # needs Icecast, so the way out is dropping the pulse tor.
+            echo "ERROR: internal pulseaudio failed to start." >&2
+            echo "       Liquidsoap will crash-loop while the script still uses" >&2
+            echo "       input.pulseaudio()/output.pulseaudio(). The Icecast" >&2
+            echo "       output does NOT need pulse: comment those out in" >&2
+            echo "       $DATA/radio_conjurer.liq (or set RADIO_FORCE_SCRIPT=1" >&2
+            echo "       to re-seed the script from the image) and restart." >&2
+            echo "       Diagnose with: pulseaudio --system --daemonize=no -vvvv" >&2
+        fi
         ;;
     host)
         [ -n "$PULSE_SERVER" ] || echo "WARNING: PULSE_MODE=host but PULSE_SERVER is unset" >&2

docker/env/bot.env.example

@@ -13,10 +13,29 @@ CONJURER_NETRC_FILE=/secrets/.netrc
 
 # --- AI backend switch --------------------------------------------------
 # Which AI config from system_gpt_settings.json is active at startup
-# (e.g. "gpt" or "claude"). Runtime switch: $gadaj_teraz <config>. Unset =
-# whatever the settings file's "active" key says, falling back to "gpt".
+# (e.g. "gpt", "claude" or "ollama"). Runtime switch:
+# $gadaj_teraz <config> [model]. Unset = whatever the settings file's "active"
+# key says, falling back to "gpt".
 # CONJURER_AI_CONFIG=gpt
 
+# --- Ollama (self-hosted models) ----------------------------------------
+# The endpoint IS the whole configuration - no API key. Leave unset and the
+# "ollama" backend simply refuses to be selected. In-cluster, use the Service
+# DNS name; from outside, host:port. Port 11434 is Ollama's default.
+#   CONJURER_OLLAMA_URL=http://ollama.ollama.svc.cluster.local:11434
+# CONJURER_OLLAMA_URL=
+# DEFAULT model for normal replies. $modele_ai lists what the server actually
+# has pulled. Pinning one at runtime with `$gadaj_teraz ollama <model>` is
+# persisted into system_gpt_settings.json and from then on WINS over this
+# variable - the pin is the more recent, more explicit choice. Clear the
+# "latest_model" of the ollama entry in that file to fall back to this default.
+# CONJURER_OLLAMA_MODEL=llama3.1:8b
+# Model used for the cheaper MUSIC path; defaults to CONJURER_OLLAMA_MODEL.
+# CONJURER_OLLAMA_CHEAP_MODEL=
+# How long to wait for ANY backend to answer. 120s suits hosted APIs; a
+# self-hosted model on a modest GPU may need more.
+# CONJURER_AI_TIMEOUT_SECONDS=120
+
 # --- Data ---------------------------------------------------------------
 # Single mounted volume; all writable state is rooted here.
 CONJURER_DATA_DIR=/data

docs/deployment/DOCKER_PROXMOX.md

@@ -10,8 +10,9 @@ Runbook for running Conjurer as Docker containers across Proxmox VMs:
 | **Share** (optional) | VM-musician or its own | `conjurer-share` | 8081 → 80 | `docker/Dockerfile.share` |
 
 The share service publishes short-lived file links over Apache and is documented
-separately in [FILE_SHARING.md](FILE_SHARING.md) — it shares two volumes with the
-musician, so set it up after the musician is running.
+separately: [SHARE_NODE_SETUP.md](SHARE_NODE_SETUP.md) to stand a node up from
+zero, [FILE_SHARING.md](FILE_SHARING.md) for how the feature works. It shares two
+volumes with the musician, so set it up after the musician is running.
 
 The three talk to each other over HTTP on the Proxmox LAN. Direction of calls:
 

docs/deployment/FILE_SHARING.md

@@ -7,6 +7,10 @@ Historically only the Python half of this lived in the repo; the Apache config
 and the cron entries were placed on the host by hand. This document plus
 `docker/Dockerfile.share` close that gap.
 
+> Standing it up on a fresh node (directories, permissions, reverse proxy,
+> verification)? Start with **[SHARE_NODE_SETUP.md](SHARE_NODE_SETUP.md)** — this
+> document explains how the feature works and how it is secured.
+
 ## What it actually does
 
 1. A **scanner** walks the media library and writes a JSON index of every path.

docs/deployment/SHARE_NODE_SETUP.md

@@ -0,0 +1,149 @@
+# Share node setup — from zero
+
+Everything the share service needs is **inside the image**: Apache, the vhost,
+the index scanner and the link revoker. The node needs no Apache, no cron, no
+Python and no copies of the old hand-placed scripts.
+
+What the node actually provides is four directories, the media library, and a
+way in from the internet.
+
+> Feature docs (how links work, security model, TTL semantics) live in
+> [FILE_SHARING.md](FILE_SHARING.md). This file is only "how to stand it up on a
+> fresh box".
+
+## What runs where
+
+| Piece | Where it lives | Notes |
+|---|---|---|
+| Apache + vhost | in the image | rendered from `share-vhost.conf.tpl` at start |
+| `scan_shares.py` (index) | in the image | sleep loop, not cron |
+| `revoke_shares.py` (expiry) | in the image | sleep loop, not cron |
+| the symlinks | host `/srv/share/links` | **created by the musician**, served here |
+| the index | host `/srv/share/db` | written here, **read by the musician** |
+| TLS / public name | your reverse proxy | container speaks plain HTTP |
+
+The single most important fact: **the musician creates the links, this service
+serves them.** They must see the same directories, at the same paths.
+
+## 1. Directories
+
+```bash
+sudo mkdir -p /srv/share/{media,links,db,logs}
+```
+
+| Path | Contents | Mounted as |
+|---|---|---|
+| `/srv/share/media` | the media library | `/mnt/shares` (read-only) |
+| `/srv/share/links` | published symlinks | `/var/www/html/share` |
+| `/srv/share/db` | `share_scan.json` | `/srv/share/db` |
+| `/srv/share/logs` | Apache logs incl. `share_access.log` | `/var/log/apache2` |
+
+## 2. Media library and permissions
+
+Put the library at `/srv/share/media` (bind mount, NFS mount, whatever — it is
+only ever read). Apache serves as **`www-data`, uid 33 inside the container**, so
+that uid must be able to traverse and read it:
+
+```bash
+sudo chmod -R o+rX /srv/share/media          # simplest; or use ACLs/group instead
+sudo -u '#33' test -r /srv/share/media/<some-file> && echo "readable by www-data"
+```
+
+A library that root can read but uid 33 cannot is the classic "every link
+404s / 403s" cause.
+
+## 3. Deploy the stack
+
+The musician and the share service share a filesystem, so the supported layout
+is **both on the same node**, from one stack:
+
+- Portainer → **Stacks → Add stack** → paste `docker/compose.musician-share.stack.yaml`
+- or CLI: `docker compose -f docker/compose.musician-share.stack.yaml up -d`
+
+That stack already wires the four mounts on both containers. Set
+`CONJURER_SHARE_SERVER_NAME` to your public hostname.
+
+Share-only node (musician elsewhere): use `docker/compose.share.yaml` and put
+`/srv/share/links` + `/srv/share/db` on storage **both** hosts mount — otherwise
+the musician cheerfully creates links this container cannot see.
+
+## 4. Way in from the internet
+
+The container listens on **8081 → 80**, plain HTTP by design; TLS stays on the
+reverse proxy you already run. Forward the `/share/` prefix **unchanged** — the
+links are `https://<host>/share/<token>`:
+
+nginx:
+```nginx
+location /share/ {
+    proxy_pass http://<share-node-ip>:8081/share/;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+}
+```
+
+Apache (as reverse proxy):
+```apache
+ProxyPass        /share/ http://<share-node-ip>:8081/share/
+ProxyPassReverse /share/ http://<share-node-ip>:8081/share/
+```
+
+Then: DNS for the public name points at the proxy, and the node's firewall lets
+the proxy reach 8081 (nothing else needs to).
+
+## 5. Tell the musician where to publish
+
+The musician builds the URLs it posts to Discord. In its env:
+
+```ini
+CONJURER_SHARE_DIR=/var/www/html/share
+CONJURER_SHARE_DB=/srv/share/db/share_scan.json
+CONJURER_SHARE_BASE_URL=https://czernobog.pl/share
+```
+
+`CONJURER_SHARE_BASE_URL` is **not** set in the bundled stack file — it falls back
+to `https://czernobog.pl/share`. If your public name differs, set it explicitly
+or every posted link points at the wrong host.
+
+## The three couplings that break it
+
+1. **Same container path for the media.** The index records absolute paths and
+   the symlinks are absolute. Both containers must mount the library at
+   `/mnt/shares`. Mount it elsewhere on one side and every link dangles.
+2. **Same link dir and index dir** for musician and share (same host, or shared
+   storage).
+3. **`CONJURER_SHARE_BASE_URL` must equal your real public `/share` URL.**
+
+## Verification
+
+```bash
+# index built (should be non-trivial JSON)
+sudo head -c 200 /srv/share/db/share_scan.json; echo
+
+# the two jobs and Apache are alive
+docker logs conjurer-share | tail -20     # "[share] serving ... scan every ...s"
+
+# directory listing MUST fail (403) - it would leak every live token
+curl -sI http://<share-node-ip>:8081/share/ | head -1
+
+# revoker state files must NOT be served
+curl -sI http://<share-node-ip>:8081/share/.downloads.json | head -1
+
+# a real link: publish one from Discord, then
+curl -sI https://<public-host>/share/<token> | head -1   # 200
+```
+
+After the first download of a link, `revoke_shares.py` removes it once
+`CONJURER_SHARE_TTL_SECONDS` (default 1h) has passed. A link nobody downloads is
+never revoked by that job.
+
+## Troubleshooting
+
+| Symptom | Cause | Fix |
+|---|---|---|
+| every link 404 | media mounted at a different container path than `/mnt/shares`, or the symlink target is gone | align the mounts on both containers |
+| every link 403 | media not readable by uid 33 | step 2 |
+| links created but not served | musician and share not sharing `/srv/share/links` | step 3 |
+| `/get_share_list` empty | index missing/not shared | check `/srv/share/db/share_scan.json` and the musician's `CONJURER_SHARE_DB` |
+| links never expire | revoker cannot see the access log, or nobody downloaded them | check `/srv/share/logs/share_access.log` exists and grows |
+| wrong host in posted links | `CONJURER_SHARE_BASE_URL` | step 5 |

tests/integration/test_librarian_crossref_retry.py

@@ -0,0 +1,102 @@
+"""Integration: transient Crossref failures must not destroy a search.
+
+Field report: a single httpx ReadTimeout inside habanero surfaced as
+"Search <uuid> crashed", and the worker then FORGOT the search - so an
+expensive query vanished and the user got told it was eaten, all because a
+public API blinked. These pin the two defences: retry each Crossref call, and
+retry the whole search a bounded number of times before giving up.
+"""
+import logging
+import sys
+import types
+
+import pytest
+
+if "habanero" not in sys.modules:
+    _habanero = types.ModuleType("habanero")
+    _habanero.Crossref = object
+    sys.modules["habanero"] = _habanero
+
+import conjurer_librarian as lib  # noqa: E402
+from durable_queue import DiskQueue  # noqa: E402
+
+_LOG = logging.getLogger("test-crossref-retry")
+_LOG.addHandler(logging.NullHandler())
+
+
+@pytest.fixture(autouse=True)
+def _no_backoff(monkeypatch):
+    monkeypatch.setattr(lib.time, "sleep", lambda _s: None)
+
+
+@pytest.fixture
+def state(tmp_path, monkeypatch):
+    monkeypatch.setattr(lib, "_requests", DiskQueue(str(tmp_path / "req")))
+    monkeypatch.setattr(lib, "_checkpoints", DiskQueue(str(tmp_path / "cp")))
+    return None
+
+
+def test_crossref_call_retries_then_succeeds():
+    calls = {"n": 0}
+
+    def flaky(**_kwargs):
+        calls["n"] += 1
+        if calls["n"] < 3:
+            raise RuntimeError("The read operation timed out")
+        return {"message": {"total-results": 1, "items": []}}
+
+    result = lib._crossref_call(_LOG, "works", flaky, query="q")
+    assert result["message"]["total-results"] == 1
+    assert calls["n"] == 3  # two failures survived
+
+
+def test_crossref_call_reraises_after_exhausting_attempts(monkeypatch):
+    monkeypatch.setattr(lib, "CROSSREF_ATTEMPTS", 2)
+    calls = {"n": 0}
+
+    def always_fails(**_kwargs):
+        calls["n"] += 1
+        raise RuntimeError("The read operation timed out")
+
+    with pytest.raises(RuntimeError):
+        lib._crossref_call(_LOG, "works", always_fails, query="q")
+    assert calls["n"] == 2  # bounded, not infinite
+
+
+def test_crossref_call_does_not_retry_a_success():
+    calls = {"n": 0}
+
+    def ok(**_kwargs):
+        calls["n"] += 1
+        return "fine"
+
+    assert lib._crossref_call(_LOG, "works", ok, query="q") == "fine"
+    assert calls["n"] == 1
+
+
+def test_attempt_counter_persists_and_bounds_retries(state):
+    # Mirrors what the worker does on a crash: bump the persisted attempt count
+    # and keep the request until SEARCH_MAX_ATTEMPTS is reached.
+    uuid = "u-crash"
+    lib._requests.put(uuid, {"query": "q", "deep_search": False, "callback": ""})
+
+    for expected in (1, 2):
+        stored = lib._requests.get(uuid) or {}
+        attempts = int(stored.get("attempts", 0)) + 1
+        assert attempts == expected
+        stored["attempts"] = attempts
+        lib._requests.put(uuid, stored)
+
+    assert lib._requests.get(uuid)["attempts"] == 2
+    # A third crash reaches the default cap (3) -> the search is forgotten.
+    assert 3 >= lib.SEARCH_MAX_ATTEMPTS
+    lib._forget_search(uuid)
+    assert not lib._requests.contains(uuid)
+
+
+def test_forget_search_clears_request_and_checkpoint(state):
+    lib._requests.put("u-x", {"query": "q", "deep_search": False})
+    lib._checkpoints.put("u-x", {"dois": {}, "found": [], "positions": {}})
+    lib._forget_search("u-x")
+    assert not lib._requests.contains("u-x")
+    assert not lib._checkpoints.contains("u-x")

tests/integration/test_librarian_heartbeat.py

@@ -0,0 +1,108 @@
+"""Integration: the 'still searching' heartbeat and its cheap progress estimate.
+
+The estimate must stay free: producers already record a byte offset per chunk
+file and the total is stat()'d once, so a reading is just a sum over ~40 ints.
+"""
+import logging
+import sys
+import types
+
+import pytest
+
+if "habanero" not in sys.modules:
+    _habanero = types.ModuleType("habanero")
+    _habanero.Crossref = object
+    sys.modules["habanero"] = _habanero
+
+import conjurer_librarian as lib  # noqa: E402
+import search_bot  # noqa: E402
+
+_LOG = logging.getLogger("test-heartbeat")
+_LOG.addHandler(logging.NullHandler())
+
+
+def test_progress_summary_percentages():
+    progress = {"positions": {"0_chunk.txt": 250, "1_chunk.txt": 250}, "total_bytes": 1000}
+    done, total, percent = lib._progress_summary(progress)
+    assert (done, total) == (500, 1000)
+    assert percent == pytest.approx(50.0)
+
+
+def test_progress_summary_unknown_total_is_zero_percent():
+    done, total, percent = lib._progress_summary({"positions": {"a": 10}})
+    assert (done, total, percent) == (10, 0, 0.0)
+
+
+def test_progress_summary_handles_empty_and_none():
+    assert lib._progress_summary(None) == (0, 0, 0.0)
+    assert lib._progress_summary({}) == (0, 0, 0.0)
+
+
+def test_progress_summary_is_clamped_to_100():
+    # A partially-buffered tail can push the summed offsets past the total.
+    _done, _total, percent = lib._progress_summary(
+        {"positions": {"a": 1500}, "total_bytes": 1000}
+    )
+    assert percent == pytest.approx(100.0)
+
+
+def test_current_search_registration_round_trip():
+    progress = {"positions": {"a": 5}, "total_bytes": 10}
+    live = [{"DOI": "10.1/x"}]
+    lib._set_current_search("uuid-1", "kwas foliowy", progress, live)
+    with lib._current_lock:
+        snapshot = dict(lib._current_search)
+    assert snapshot["uuid"] == "uuid-1"
+    assert snapshot["query"] == "kwas foliowy"
+    assert lib._progress_summary(snapshot["progress"])[2] == pytest.approx(50.0)
+    lib._clear_current_search()
+    with lib._current_lock:
+        assert not lib._current_search
+
+
+def _write_two_chunks(tmp_path):
+    (tmp_path / "0_chunk.txt").write_text("10.1/a\n10.1/b\n", encoding="utf-8")
+    (tmp_path / "1_chunk.txt").write_text("10.1/c\n", encoding="utf-8")
+    return sum(
+        (tmp_path / name).stat().st_size for name in ("0_chunk.txt", "1_chunk.txt")
+    )
+
+
+def test_search_fills_progress_and_reaches_full_coverage(tmp_path, monkeypatch):
+    # Coverage must be measured on a search that CANNOT stop early. Once every
+    # queried DOI is found the consumer signals TERM and the producers stop
+    # mid-file, so a search for a DOI that exists reaches an arbitrary offset -
+    # asserting 100% there is a race (it failed roughly one run in two).
+    # An absent DOI forces the whole database to be read.
+    monkeypatch.setattr(search_bot, "DATABASE_PATH", str(tmp_path) + "/")
+    expected_total = _write_two_chunks(tmp_path)
+
+    progress = {}
+    search_bot.search_for_doi([("10.9/absent", "DATA")], [], _LOG, progress=progress)
+
+    assert progress["total_bytes"] == expected_total
+    assert progress["chunk_files"] == 2
+    done, total, percent = lib._progress_summary(progress)
+    assert total == expected_total
+    assert done == expected_total          # nothing stopped it: whole DB scanned
+    assert percent == pytest.approx(100.0)
+
+
+def test_progress_is_populated_for_a_search_that_finds_its_target(tmp_path, monkeypatch):
+    # The early-termination case: the target is found, so coverage is whatever
+    # the producers reached. Assert what IS deterministic - the total is known,
+    # progress is bounded and sane, and the hit is reported.
+    monkeypatch.setattr(search_bot, "DATABASE_PATH", str(tmp_path) + "/")
+    expected_total = _write_two_chunks(tmp_path)
+
+    progress = {}
+    result, _positions, _interrupted = search_bot.search_for_doi(
+        [("10.1/c", "DATA")], [], _LOG, progress=progress
+    )
+
+    assert progress["total_bytes"] == expected_total
+    done, total, percent = lib._progress_summary(progress)
+    assert total == expected_total
+    assert 0 <= done <= total              # bounded, never nonsense
+    assert 0.0 <= percent <= 100.0
+    assert [r for r in result if r["DOI"] == "10.1/c" and r["exists"]]

tests/unit/test_ai_provider_switch.py

@@ -136,3 +136,401 @@ def test_map_openai_error_categories():
     assert ai_functions._map_openai_error(_bare(openai.RateLimitError)).category == "rate_limit"
     assert ai_functions._map_openai_error(_bare(openai.APITimeoutError)).category == "timeout"
     assert ai_functions._map_openai_error(ValueError("x")).category == "api"
+
+
+# ---------------------------------------------------------------- Ollama
+# The self-hosted backend is wired through Ollama's OpenAI-compatible surface,
+# so it reuses the message format and the error mapping above. What is new and
+# worth pinning: it must appear in the picker even on an upgraded settings file,
+# models come from the SERVER, and pinning one must stick.
+import asyncio  # noqa: E402
+
+
+class _FakeModel:
+    def __init__(self, ident):
+        self.id = ident
+
+
+class _FakeModels:
+    def __init__(self, ids, raises=None):
+        self._ids = ids
+        self._raises = raises
+
+    async def list(self):
+        if self._raises:
+            raise self._raises
+        return types.SimpleNamespace(data=[_FakeModel(i) for i in self._ids])
+
+
+class _FakeOllamaClient:
+    def __init__(self, ids=(), raises=None):
+        self.models = _FakeModels(list(ids), raises)
+
+
+def test_ollama_config_is_offered_in_the_picker():
+    assert "ollama" in ai_functions.list_ai_configs()
+    cfg = ai_functions.AI_CONFIGS["ollama"]
+    assert cfg["provider"] == "ollama"
+
+
+def test_select_model_uses_ollama_models_when_active(monkeypatch):
+    monkeypatch.setitem(
+        ai_functions.AI_CONFIGS,
+        "ollama",
+        {"provider": "ollama", "latest_model": "llama3.1:8b", "cheap_model": "qwen2.5:3b"},
+    )
+    _reset_active("ollama")
+    try:
+        # the legacy gpt-4o default must auto-map, not leak to Ollama
+        assert ai_functions.select_model("GENERAL", "gpt-4o") == "llama3.1:8b"
+        assert ai_functions.select_model("MUSIC", "gpt-4o") == "qwen2.5:3b"
+        # an explicit id is still honoured verbatim
+        assert ai_functions.select_model("GENERAL", "mistral:7b") == "mistral:7b"
+    finally:
+        _reset_active("gpt")
+
+
+def test_list_provider_models_queries_the_ollama_server(monkeypatch):
+    monkeypatch.setattr(
+        ai_functions, "OLLAMACLIENT", _FakeOllamaClient(["b:2", "a:1", "a:1"])
+    )
+    models = asyncio.run(ai_functions.list_provider_models("ollama"))
+    assert models == ["a:1", "b:2"]  # sorted + de-duplicated
+
+
+def test_list_provider_models_for_hosted_provider_reports_configured_ids():
+    models = asyncio.run(ai_functions.list_provider_models("gpt"))
+    assert models == [
+        ai_functions.AI_CONFIGS["gpt"]["latest_model"],
+        ai_functions.AI_CONFIGS["gpt"]["cheap_model"],
+    ]
+
+
+def test_list_provider_models_wraps_server_failure(monkeypatch):
+    monkeypatch.setattr(
+        ai_functions, "OLLAMACLIENT", _FakeOllamaClient(raises=ValueError("boom"))
+    )
+    try:
+        asyncio.run(ai_functions.list_provider_models("ollama"))
+    except ai_functions.AIError as exc:
+        assert exc.category == "api"
+    else:
+        raise AssertionError("a server failure must surface as AIError")
+
+
+def test_list_provider_models_without_endpoint_is_an_auth_error(monkeypatch):
+    # Contrast, so the assertion cannot pass vacuously: with a client present the
+    # call succeeds, and ONLY setting it to None turns it into an auth error.
+    monkeypatch.setattr(ai_functions, "OLLAMACLIENT", _FakeOllamaClient(["a:1"]))
+    assert asyncio.run(ai_functions.list_provider_models("ollama")) == ["a:1"]
+
+    monkeypatch.setattr(ai_functions, "OLLAMACLIENT", None)
+    try:
+        asyncio.run(ai_functions.list_provider_models("ollama"))
+    except ai_functions.AIError as exc:
+        assert exc.category == "auth"
+    else:
+        raise AssertionError("an unconfigured Ollama must surface as AIError")
+
+
+def test_set_active_model_pins_latest_and_keeps_cheap(monkeypatch):
+    monkeypatch.setitem(
+        ai_functions.AI_CONFIGS,
+        "ollama",
+        {"provider": "ollama", "latest_model": "old:1", "cheap_model": "cheap:1"},
+    )
+    written = {}
+    monkeypatch.setattr(
+        ai_functions,
+        "_persist_active_ai_config",
+        lambda name, model_for=None: written.update(name=name, model_for=model_for),
+    
+    )
+    ai_functions.set_active_model("mistral:7b", "ollama")
+    # Assert on the shared registry, not on the returned object - that object IS
+    # the mutated dict, so asserting on it would pass even if nothing was stored.
+    stored = ai_functions.AI_CONFIGS["ollama"]
+    assert stored["latest_model"] == "mistral:7b"
+    assert stored["cheap_model"] == "cheap:1"   # MUSIC path untouched
+    assert written["name"]                      # the choice was persisted...
+    assert written["model_for"] == "ollama"     # ...scoped to the config we changed
+
+
+def test_set_active_model_rejects_blank_and_unknown_config(monkeypatch):
+    monkeypatch.setattr(
+        ai_functions, "_persist_active_ai_config", lambda _n, model_for=None: None
+    )
+    for bad in ("", "   "):
+        try:
+            ai_functions.set_active_model(bad, "gpt")
+        except ValueError:
+            pass
+        else:
+            raise AssertionError("a blank model id must be rejected")
+    try:
+        ai_functions.set_active_model("x", "nie-ma-takiego")
+    except KeyError:
+        pass
+    else:
+        raise AssertionError("an unknown config must be rejected")
+
+
+def test_switching_to_ollama_without_endpoint_explains_itself(monkeypatch):
+    monkeypatch.setattr(ai_functions, "OLLAMACLIENT", None)
+    try:
+        ai_functions.set_active_ai_config("ollama")
+    except RuntimeError as exc:
+        assert "CONJURER_OLLAMA_URL" in str(exc)
+    else:
+        raise AssertionError("switching to an unconfigured Ollama must raise")
+    finally:
+        _reset_active("gpt")
+
+
+def test_provider_generate_routes_to_ollama(monkeypatch):
+    monkeypatch.setitem(
+        ai_functions.AI_CONFIGS,
+        "ollama",
+        {"provider": "ollama", "latest_model": "m:1", "cheap_model": "m:1"},
+    )
+    _reset_active("ollama")
+    seen = {}
+
+    async def _fake_ollama_call(messages, model, cfg):
+        seen.update(model=model, messages=messages)
+        return "odpowiedź z domu"
+
+    monkeypatch.setattr(ai_functions, "_ollama_call", _fake_ollama_call)
+    try:
+        out = asyncio.run(
+            ai_functions.provider_generate([{"role": "user", "content": "hej"}], "m:1")
+        )
+    finally:
+        _reset_active("gpt")
+    assert out == "odpowiedź z domu"
+    assert seen["model"] == "m:1"
+
+
+# ------------------------------------------------- persistence (real disk path)
+# This path had NO coverage, which is exactly how a config-clobbering regression
+# got in: persisting the whole in-memory AI_CONFIGS (built-in defaults merged
+# under the file) overwrote operator hand-edits and resurrected deleted configs.
+import json  # noqa: E402
+
+
+def _settings_file(tmp_path, configs, active="gpt"):
+    path = tmp_path / "system_gpt_settings.json"
+    path.write_text(
+        json.dumps(
+            [
+                {"role": "system", "content": "sys"},
+                {"someuser": [1, "a", "b", "c", "asst_x"]},
+                {"active": active, "configs": configs},
+            ]
+        ),
+        encoding="utf-8",
+    )
+    return path
+
+
+def test_persist_writes_the_pin_without_clobbering_operator_edits(tmp_path, monkeypatch):
+    # The file is authoritative for everything the bot does not itself change:
+    # a hand-tuned cheap_model, and a config deliberately deleted from it.
+    settings = _settings_file(
+        tmp_path,
+        {"gpt": {"provider": "openai", "latest_model": "gpt-4.1", "cheap_model": "hand-tuned"}},
+    )
+    monkeypatch.setattr(ai_functions, "SYSTEM_GPT_SETTINGS", str(settings))
+    monkeypatch.setitem(
+        ai_functions.AI_CONFIGS,
+        "ollama",
+        {"provider": "ollama", "latest_model": "mistral:7b", "cheap_model": "c:1"},
+    )
+
+    ai_functions._persist_active_ai_config("ollama", model_for="ollama")
+
+    data = json.loads(settings.read_text(encoding="utf-8"))
+    configs = data[2]["configs"]
+    assert data[2]["active"] == "ollama"
+    assert configs["ollama"]["latest_model"] == "mistral:7b"   # the pin landed
+    assert configs["gpt"]["latest_model"] == "gpt-4.1"         # edit survived
+    assert configs["gpt"]["cheap_model"] == "hand-tuned"       # edit survived
+    assert "claude" not in configs   # a deleted config is NOT resurrected
+    assert data[0]["content"] == "sys" and "someuser" in data[1]  # rest intact
+
+
+def test_plain_switch_leaves_the_configs_block_untouched(tmp_path, monkeypatch):
+    original = {"gpt": {"provider": "openai", "latest_model": "gpt-4.1", "cheap_model": "hand-tuned"}}
+    settings = _settings_file(tmp_path, original, active="claude")
+    monkeypatch.setattr(ai_functions, "SYSTEM_GPT_SETTINGS", str(settings))
+
+    # Switching backend without pinning a model must only move "active".
+    ai_functions._persist_active_ai_config("gpt")
+
+    data = json.loads(settings.read_text(encoding="utf-8"))
+    assert data[2]["active"] == "gpt"
+    assert data[2]["configs"] == original
+
+
+def test_pinned_model_survives_a_restart(tmp_path, monkeypatch):
+    # The whole point of persisting: re-reading the file must yield the pin.
+    settings = _settings_file(tmp_path, {"ollama": {"provider": "ollama", "latest_model": "old:1"}})
+    monkeypatch.setattr(ai_functions, "SYSTEM_GPT_SETTINGS", str(settings))
+    monkeypatch.setitem(
+        ai_functions.AI_CONFIGS,
+        "ollama",
+        {"provider": "ollama", "latest_model": "new:2", "cheap_model": "c:1"},
+    )
+
+    ai_functions._persist_active_ai_config("ollama", model_for="ollama")
+
+    reread = json.loads(settings.read_text(encoding="utf-8"))[2]
+    assert reread["configs"]["ollama"]["latest_model"] == "new:2"
+
+
+def test_persist_survives_an_unreadable_settings_file(tmp_path, monkeypatch):
+    # Best-effort by contract: a broken file must not raise into the command.
+    broken = tmp_path / "broken.json"
+    broken.write_text("{ not json", encoding="utf-8")
+    monkeypatch.setattr(ai_functions, "SYSTEM_GPT_SETTINGS", str(broken))
+    ai_functions._persist_active_ai_config("gpt", model_for="gpt")  # must not raise
+
+
+# ----------------------------------------------- keep-warm (Ollama ONLY) ----
+# The money guard: preloading a self-hosted model is free, but firing the same
+# thing at a metered API would burn tokens for nothing. These pin that it can
+# only ever happen for Ollama.
+
+
+def test_warm_active_model_is_a_noop_for_paid_providers(monkeypatch):
+    called = []
+    monkeypatch.setattr(
+        ai_functions, "_ollama_preload", lambda *a, **k: called.append(a) or True
+    )
+    for paid in ("gpt", "claude"):
+        _reset_active(paid)
+        try:
+            assert asyncio.run(ai_functions.warm_active_model()) is False
+        finally:
+            _reset_active("gpt")
+    assert called == [], "a paid backend must never be preloaded"
+
+
+def test_warm_active_model_preloads_when_ollama_is_active(monkeypatch):
+    monkeypatch.setitem(
+        ai_functions.AI_CONFIGS,
+        "ollama",
+        {"provider": "ollama", "latest_model": "qwen2.5:7b", "cheap_model": "c"},
+    )
+    seen = {}
+    monkeypatch.setattr(
+        ai_functions, "_ollama_preload", lambda model, *a, **k: seen.update(model=model) or True
+    )
+    _reset_active("ollama")
+    try:
+        assert asyncio.run(ai_functions.warm_active_model()) is True
+    finally:
+        _reset_active("gpt")
+    assert seen["model"] == "qwen2.5:7b"
+
+
+def test_active_provider_reports_the_switch():
+    _reset_active("gpt")
+    assert ai_functions.active_provider() == "openai"
+    _reset_active("claude")
+    try:
+        assert ai_functions.active_provider() == "anthropic"
+    finally:
+        _reset_active("gpt")
+
+
+def test_preload_sends_no_prompt_so_it_generates_nothing(monkeypatch):
+    # Ollama's documented preload: a model and keep_alive, and NO prompt. If a
+    # prompt ever crept in, every warm-up would silently generate tokens.
+    sent = {}
+
+    class _Resp:
+        status_code = 200
+
+    monkeypatch.setattr(ai_functions, "OLLAMA_URL", "http://ollama:11434")
+    monkeypatch.setattr(
+        ai_functions.requests, "post",
+        lambda url, json=None, timeout=None: sent.update(url=url, body=json) or _Resp(),
+    )
+    assert ai_functions._ollama_preload("qwen2.5:7b") is True
+    assert sent["url"].endswith("/api/generate")
+    assert sent["body"]["model"] == "qwen2.5:7b"
+    assert "keep_alive" in sent["body"]
+    assert "prompt" not in sent["body"], "a preload must not generate"
+
+
+def test_preload_without_endpoint_is_a_noop(monkeypatch):
+    monkeypatch.setattr(ai_functions, "OLLAMA_URL", "")
+    assert ai_functions._ollama_preload("x") is False
+
+
+# ------------------------------------------- personal assistants (per user) --
+# Replaces the sunset OpenAI Assistants API. The two properties that matter:
+# each user's DM history is ISOLATED (private DMs must not leak into another
+# user's context or the bar's shared memory), and it stays BOUNDED.
+
+
+def _fresh_assistant_memory(tmp_path, monkeypatch, turns=40):
+    monkeypatch.setattr(
+        ai_functions, "ASSISTANT_MEMORY_FILE", str(tmp_path / "assistant_memory.json")
+    )
+    monkeypatch.setattr(ai_functions, "ASSISTANT_MEMORY_TURNS", turns)
+    monkeypatch.setattr(ai_functions, "_ASSISTANT_MEMORY", None)
+
+
+def test_assistant_history_is_isolated_per_user(tmp_path, monkeypatch):
+    _fresh_assistant_memory(tmp_path, monkeypatch)
+    ai_functions.remember_assistant_turn(111, "sekret Anny", "ok Anna")
+    ai_functions.remember_assistant_turn(222, "sekret Bartka", "ok Bartek")
+
+    anna = ai_functions.assistant_history(111)
+    bartek = ai_functions.assistant_history(222)
+    assert [m["content"] for m in anna] == ["sekret Anny", "ok Anna"]
+    assert [m["content"] for m in bartek] == ["sekret Bartka", "ok Bartek"]
+    assert "sekret Anny" not in str(bartek)   # no cross-user bleed
+
+
+def test_assistant_history_is_trimmed_to_the_bound(tmp_path, monkeypatch):
+    _fresh_assistant_memory(tmp_path, monkeypatch, turns=4)
+    for i in range(10):
+        ai_functions.remember_assistant_turn(1, f"u{i}", f"a{i}")
+    history = ai_functions.assistant_history(1)
+    assert len(history) == 4                          # bounded
+    assert history[-1]["content"] == "a9"             # newest kept
+    assert all("u0" != m["content"] for m in history)  # oldest dropped
+
+
+def test_assistant_history_survives_a_restart(tmp_path, monkeypatch):
+    _fresh_assistant_memory(tmp_path, monkeypatch)
+    ai_functions.remember_assistant_turn(7, "pamietaj", "pamietam")
+    # Simulate a restart: drop the in-memory cache, re-read from disk.
+    monkeypatch.setattr(ai_functions, "_ASSISTANT_MEMORY", None)
+    assert [m["content"] for m in ai_functions.assistant_history(7)] == [
+        "pamietaj",
+        "pamietam",
+    ]
+
+
+def test_assistant_messages_carry_persona_history_and_new_turn(tmp_path, monkeypatch):
+    _fresh_assistant_memory(tmp_path, monkeypatch)
+    ai_functions.remember_assistant_turn(5, "wczoraj", "odpowiedz")
+    msgs = ai_functions.build_assistant_messages(
+        5, "Towarzysz Młotek", "Mówisz po polsku.", "dzisiaj"
+    )
+    assert msgs[0]["role"] == "system"
+    assert "Towarzysz Młotek" in msgs[0]["content"]
+    assert "Mówisz po polsku." in msgs[0]["content"]
+    assert [m["content"] for m in msgs[1:]] == ["wczoraj", "odpowiedz", "dzisiaj"]
+
+
+def test_corrupt_assistant_memory_starts_empty_instead_of_crashing(tmp_path, monkeypatch):
+    path = tmp_path / "assistant_memory.json"
+    path.write_text("{ not json", encoding="utf-8")
+    monkeypatch.setattr(ai_functions, "ASSISTANT_MEMORY_FILE", str(path))
+    monkeypatch.setattr(ai_functions, "_ASSISTANT_MEMORY", None)
+    assert ai_functions.assistant_history(1) == []