AI: add a self-hosted Ollama backend, and let the picker choose the model

The bot could talk to OpenAI or Anthropic; this adds Ollama as a third
provider so it can run against models hosted on our own box, and extends
the switch command to pick WHICH model - not just which backend.

Provider: Ollama exposes an OpenAI-compatible /v1 surface, so the client is
just openai.AsyncOpenAI(base_url=OLLAMA_URL + "/v1"). That reuses the
existing message format and the whole _map_openai_error mapping instead of
forking a second error taxonomy. There is no API key - the endpoint IS the
configuration, so the backend stays dormant (and refuses to be selected,
with a message naming the variable) until CONJURER_OLLAMA_URL is set, the
same way the Conan bridge behaves.

Model selection:
* list_provider_models() asks the SERVER for Ollama (/v1/models), so the
  picker shows what is actually pulled on the box rather than a hardcoded
  list. Hosted providers just report what they are wired to.
* set_active_model() pins the config's latest_model and persists it;
  cheap_model is left alone so the MUSIC path keeps its cheaper backend.
* $gadaj_teraz now takes "<config> [model]", and a new read-only $modele_ai
  lists what is available. Pinning an id Ollama does not have is rejected up
  front with the real list - otherwise the typo only surfaces later as a
  failed reply.

Two fixes this exposed:
* AI_CONFIGS now merges built-in defaults with the settings-file block
  instead of letting the file win outright. Every provider switch persists a
  "configs" block, so a file written by an older build would have
  permanently hidden ollama from the picker after an upgrade.
* _persist_active_ai_config assigns "configs" instead of setdefault, so a
  pinned model actually survives a restart.
* the hardcoded 120s response timeout is now CONJURER_AI_TIMEOUT_SECONDS - a
  self-hosted model on a modest GPU can legitimately need longer.

Tests cover: ollama appears in the picker, select_model maps the legacy
gpt-4o default instead of leaking it, model listing (server-queried, sorted,
de-duplicated, failure -> AIError, unconfigured -> auth), pinning (latest
only, blank/unknown rejected), and that provider_generate routes to the new
path. Suite: 68 unit + 70 integration green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-08-21 13:35:03 +02:00
committed by gitea
parent 13d2a04052
commit 6a6b821a0d
5 changed files with 366 additions and 14 deletions
+18 -2
View File
@@ -13,10 +13,26 @@ CONJURER_NETRC_FILE=/secrets/.netrc
# --- AI backend switch --------------------------------------------------
# Which AI config from system_gpt_settings.json is active at startup
# (e.g. "gpt" or "claude"). Runtime switch: $gadaj_teraz <config>. Unset =
# whatever the settings file's "active" key says, falling back to "gpt".
# (e.g. "gpt", "claude" or "ollama"). Runtime switch:
# $gadaj_teraz <config> [model]. Unset = whatever the settings file's "active"
# key says, falling back to "gpt".
# CONJURER_AI_CONFIG=gpt
# --- Ollama (self-hosted models) ----------------------------------------
# The endpoint IS the whole configuration - no API key. Leave unset and the
# "ollama" backend simply refuses to be selected. In-cluster, use the Service
# DNS name; from outside, host:port. Port 11434 is Ollama's default.
# CONJURER_OLLAMA_URL=http://ollama.ollama.svc.cluster.local:11434
# CONJURER_OLLAMA_URL=
# Model used for normal replies. $modele_ai lists what the server actually has
# pulled, and $gadaj_teraz ollama <model> pins one at runtime (persisted).
# CONJURER_OLLAMA_MODEL=llama3.1:8b
# Model used for the cheaper MUSIC path; defaults to CONJURER_OLLAMA_MODEL.
# CONJURER_OLLAMA_CHEAP_MODEL=
# How long to wait for ANY backend to answer. 120s suits hosted APIs; a
# self-hosted model on a modest GPU may need more.
# CONJURER_AI_TIMEOUT_SECONDS=120
# --- Data ---------------------------------------------------------------
# Single mounted volume; all writable state is rooted here.
CONJURER_DATA_DIR=/data