AI: add a self-hosted Ollama backend, and let the picker choose the model

The bot could talk to OpenAI or Anthropic; this adds Ollama as a third
provider so it can run against models hosted on our own box, and extends
the switch command to pick WHICH model - not just which backend.

Provider: Ollama exposes an OpenAI-compatible /v1 surface, so the client is
just openai.AsyncOpenAI(base_url=OLLAMA_URL + "/v1"). That reuses the
existing message format and the whole _map_openai_error mapping instead of
forking a second error taxonomy. There is no API key - the endpoint IS the
configuration, so the backend stays dormant (and refuses to be selected,
with a message naming the variable) until CONJURER_OLLAMA_URL is set, the
same way the Conan bridge behaves.

Model selection:
* list_provider_models() asks the SERVER for Ollama (/v1/models), so the
  picker shows what is actually pulled on the box rather than a hardcoded
  list. Hosted providers just report what they are wired to.
* set_active_model() pins the config's latest_model and persists it;
  cheap_model is left alone so the MUSIC path keeps its cheaper backend.
* $gadaj_teraz now takes "<config> [model]", and a new read-only $modele_ai
  lists what is available. Pinning an id Ollama does not have is rejected up
  front with the real list - otherwise the typo only surfaces later as a
  failed reply.

Two fixes this exposed:
* AI_CONFIGS now merges built-in defaults with the settings-file block
  instead of letting the file win outright. Every provider switch persists a
  "configs" block, so a file written by an older build would have
  permanently hidden ollama from the picker after an upgrade.
* _persist_active_ai_config assigns "configs" instead of setdefault, so a
  pinned model actually survives a restart.
* the hardcoded 120s response timeout is now CONJURER_AI_TIMEOUT_SECONDS - a
  self-hosted model on a modest GPU can legitimately need longer.

Tests cover: ollama appears in the picker, select_model maps the legacy
gpt-4o default instead of leaking it, model listing (server-queried, sorted,
de-duplicated, failure -> AIError, unconfigured -> auth), pinning (latest
only, blank/unknown rejected), and that provider_generate routes to the new
path. Suite: 68 unit + 70 integration green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-08-21 13:35:03 +02:00
committed by gitea
parent 13d2a04052
commit 6a6b821a0d
5 changed files with 366 additions and 14 deletions
+31 -1
View File
@@ -400,6 +400,25 @@ if anthropic and ANTHROPIC_API_KEY:
else:
CLAUDECLIENT = None
# Ollama (self-hosted models). There is no API key - the endpoint IS the whole
# configuration, so the feature stays dormant until CONJURER_OLLAMA_URL is set
# (same pattern as the Conan bridge). We talk to Ollama's OpenAI-COMPATIBLE
# surface (/v1) with the openai SDK we already depend on, which means the
# existing message format and _map_openai_error handling work unchanged.
# How long handle_response waits for ANY backend before giving up. 120s was
# hardcoded and is fine for hosted APIs, but a self-hosted model on a modest GPU
# can legitimately take longer, so it is now tunable.
AI_TIMEOUT_SECONDS = int(os.getenv("CONJURER_AI_TIMEOUT_SECONDS", "120"))
OLLAMA_URL = os.getenv("CONJURER_OLLAMA_URL", "").rstrip("/")
OLLAMA_LATEST_MODEL = os.getenv("CONJURER_OLLAMA_MODEL", "llama3.1:8b")
OLLAMA_CHEAP_MODEL = os.getenv("CONJURER_OLLAMA_CHEAP_MODEL", OLLAMA_LATEST_MODEL)
if openai and OLLAMA_URL:
# api_key is required by the SDK but ignored by Ollama.
OLLAMACLIENT = openai.AsyncOpenAI(base_url=f"{OLLAMA_URL}/v1", api_key="ollama")
else:
OLLAMACLIENT = None
TOKEN = _resolve_token("discord", "DISCORD_TOKEN")
# Voice recognition (AssemblyAI). None = the voice cog reports and disables.
@@ -489,6 +508,12 @@ def _default_ai_configs():
# sent for Claude (Opus 4.8 / Sonnet 5 reject sampling params).
"max_tokens": 2048,
},
"ollama": {
"provider": "ollama",
"latest_model": OLLAMA_LATEST_MODEL,
"cheap_model": OLLAMA_CHEAP_MODEL,
"temperature": 0.2,
},
# Template for wiring further providers. Copy it, rename the key, point
# "provider" at a backend ai_functions.provider_generate implements, and
# fill in the model ids. Keys starting with "_" are treated as inert
@@ -508,7 +533,12 @@ _ai_block = (
if isinstance(GPT_SETTINGS, list) and len(GPT_SETTINGS) > 2 and isinstance(GPT_SETTINGS[2], dict)
else {}
)
AI_CONFIGS = _ai_block.get("configs") or _default_ai_configs()
# Built-in defaults FIRST, then whatever the settings file defines on top. The
# file cannot simply win outright: every provider switch persists a "configs"
# block, so a file written by an older build would permanently hide providers
# added later (ollama) from the picker.
AI_CONFIGS = _default_ai_configs()
AI_CONFIGS.update(_ai_block.get("configs") or {})
# Single switch: env var wins, then the settings-file "active" key, then "gpt".
DEFAULT_AI_CONFIG = (
os.getenv("CONJURER_AI_CONFIG")