AI: add a self-hosted Ollama backend, and let the picker choose the model

The bot could talk to OpenAI or Anthropic; this adds Ollama as a third
provider so it can run against models hosted on our own box, and extends
the switch command to pick WHICH model - not just which backend.

Provider: Ollama exposes an OpenAI-compatible /v1 surface, so the client is
just openai.AsyncOpenAI(base_url=OLLAMA_URL + "/v1"). That reuses the
existing message format and the whole _map_openai_error mapping instead of
forking a second error taxonomy. There is no API key - the endpoint IS the
configuration, so the backend stays dormant (and refuses to be selected,
with a message naming the variable) until CONJURER_OLLAMA_URL is set, the
same way the Conan bridge behaves.

Model selection:
* list_provider_models() asks the SERVER for Ollama (/v1/models), so the
  picker shows what is actually pulled on the box rather than a hardcoded
  list. Hosted providers just report what they are wired to.
* set_active_model() pins the config's latest_model and persists it;
  cheap_model is left alone so the MUSIC path keeps its cheaper backend.
* $gadaj_teraz now takes "<config> [model]", and a new read-only $modele_ai
  lists what is available. Pinning an id Ollama does not have is rejected up
  front with the real list - otherwise the typo only surfaces later as a
  failed reply.

Two fixes this exposed:
* AI_CONFIGS now merges built-in defaults with the settings-file block
  instead of letting the file win outright. Every provider switch persists a
  "configs" block, so a file written by an older build would have
  permanently hidden ollama from the picker after an upgrade.
* _persist_active_ai_config assigns "configs" instead of setdefault, so a
  pinned model actually survives a restart.
* the hardcoded 120s response timeout is now CONJURER_AI_TIMEOUT_SECONDS - a
  self-hosted model on a modest GPU can legitimately need longer.

Tests cover: ollama appears in the picker, select_model maps the legacy
gpt-4o default instead of leaking it, model listing (server-queried, sorted,
de-duplicated, failure -> AIError, unconfigured -> auth), pinning (latest
only, blank/unknown rejected), and that provider_generate routes to the new
path. Suite: 68 unit + 70 integration green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-08-21 13:35:03 +02:00
committed by gitea
parent 13d2a04052
commit 6a6b821a0d
5 changed files with 366 additions and 14 deletions
+74 -2
View File
@@ -9,6 +9,7 @@ import time
from other_functions import discord_friendly_send
from constants import (
AI_CONFIGS,
AI_TIMEOUT_SECONDS,
ASSISTANTS,
CLAUDECLIENT,
CYCLIC_WORDS,
@@ -19,6 +20,7 @@ from constants import (
MEMORY_FIVE_SIARA,
MESSAGE_TABLE,
MESSAGE_TABLE_MUZYKA,
OLLAMACLIENT,
OPENAICLIENT,
SYSTEM_GPT_SETTINGS,
WORD_REACTIONS,
@@ -92,11 +94,53 @@ def set_active_ai_config(name: str) -> dict:
raise RuntimeError("klient Anthropic nie jest skonfigurowany (brak ANTHROPIC_API_KEY)")
if provider == "openai" and OPENAICLIENT is None:
raise RuntimeError("klient OpenAI nie jest skonfigurowany (brak OPENAI_API_KEY)")
if provider == "ollama" and OLLAMACLIENT is None:
raise RuntimeError("Ollama nie jest skonfigurowana (ustaw CONJURER_OLLAMA_URL)")
_ACTIVE_CONFIG_NAME = name
_persist_active_ai_config(name)
return cfg
async def list_provider_models(name: str = None):
"""Model ids selectable for a config.
For Ollama this ASKS THE SERVER (its OpenAI-compatible /v1/models), so the
picker always reflects what is actually pulled on the box rather than a
hardcoded list. Hosted providers are not enumerated - we only report what
the config is wired to.
"""
cfg = AI_CONFIGS.get(name or _ACTIVE_CONFIG_NAME) or _active_config()
if cfg.get("provider") == "ollama":
if OLLAMACLIENT is None:
raise AIError(
"auth",
RuntimeError("Ollama nie jest skonfigurowana (ustaw CONJURER_OLLAMA_URL)"),
)
try:
resp = await OLLAMACLIENT.models.list()
except Exception as exc: # pylint: disable=broad-except
raise _map_openai_error(exc)
return sorted({item.id for item in resp.data})
return [m for m in (cfg.get("latest_model"), cfg.get("cheap_model")) if m]
def set_active_model(model: str, name: str = None) -> dict:
"""Pin the model a config uses for normal replies, and persist it.
Only ``latest_model`` is changed; ``cheap_model`` stays as configured so the
MUSIC path keeps its cheaper backend.
"""
cfg_name = name or _ACTIVE_CONFIG_NAME
if cfg_name not in AI_CONFIGS:
raise KeyError(cfg_name)
if not model or not model.strip():
raise ValueError("pusta nazwa modelu")
cfg = AI_CONFIGS[cfg_name]
cfg["latest_model"] = model.strip()
_persist_active_ai_config(_ACTIVE_CONFIG_NAME)
return cfg
def _persist_active_ai_config(name: str) -> None:
"""Best-effort write of the active-config choice into system_gpt_settings.json.
@@ -117,7 +161,10 @@ def _persist_active_ai_config(name: str) -> None:
return
if len(data) > 2 and isinstance(data[2], dict):
data[2]["active"] = name
data[2].setdefault("configs", AI_CONFIGS)
# Assign (not setdefault): AI_CONFIGS is the in-memory truth and may
# carry a model pinned via set_active_model, which setdefault would
# silently drop on restart.
data[2]["configs"] = AI_CONFIGS
else:
data = data[:2] + [{"active": name, "configs": AI_CONFIGS}]
try:
@@ -214,12 +261,37 @@ async def _anthropic_call(messages, model, cfg):
return text.strip()
async def _ollama_call(messages, model, cfg):
"""Self-hosted counterpart of openai_call. Returns a plain string.
Ollama exposes an OpenAI-compatible /v1 surface, so the same message format
and the same error mapping apply - only the base_url and the model ids
differ. Chat Completions (not the Responses API) is what Ollama implements.
"""
if OLLAMACLIENT is None:
raise AIError(
"auth",
RuntimeError("Ollama nie jest skonfigurowana (ustaw CONJURER_OLLAMA_URL)"),
)
try:
resp = await OLLAMACLIENT.chat.completions.create(
model=model,
messages=messages,
temperature=float(cfg.get("temperature", 0.2)),
)
except Exception as exc: # pylint: disable=broad-except
raise _map_openai_error(exc)
return (resp.choices[0].message.content or "").strip()
async def provider_generate(messages, model, temperature=0.2):
"""Dispatch a chat completion to the active backend, normalising errors."""
cfg = _active_config()
try:
if cfg.get("provider") == "anthropic":
return await _anthropic_call(messages, model, cfg)
if cfg.get("provider") == "ollama":
return await _ollama_call(messages, model, cfg)
return await openai_call(messages, model, temperature)
except AIError:
raise
@@ -459,7 +531,7 @@ async def handle_response(
try:
# ...przygotowanie messages/system prompt/itp. jak masz...
# retry/backoff + deadline (zachowuje Twoją semantykę logowania)
timeout_sec = 120
timeout_sec = AI_TIMEOUT_SECONDS
deadline = time.time() + timeout_sec
response = await asyncio.wait_for(
provider_generate(messages=history_msgs, model=model_to_use),