AI: add a self-hosted Ollama backend, and let the picker choose the model
The bot could talk to OpenAI or Anthropic; this adds Ollama as a third provider so it can run against models hosted on our own box, and extends the switch command to pick WHICH model - not just which backend. Provider: Ollama exposes an OpenAI-compatible /v1 surface, so the client is just openai.AsyncOpenAI(base_url=OLLAMA_URL + "/v1"). That reuses the existing message format and the whole _map_openai_error mapping instead of forking a second error taxonomy. There is no API key - the endpoint IS the configuration, so the backend stays dormant (and refuses to be selected, with a message naming the variable) until CONJURER_OLLAMA_URL is set, the same way the Conan bridge behaves. Model selection: * list_provider_models() asks the SERVER for Ollama (/v1/models), so the picker shows what is actually pulled on the box rather than a hardcoded list. Hosted providers just report what they are wired to. * set_active_model() pins the config's latest_model and persists it; cheap_model is left alone so the MUSIC path keeps its cheaper backend. * $gadaj_teraz now takes "<config> [model]", and a new read-only $modele_ai lists what is available. Pinning an id Ollama does not have is rejected up front with the real list - otherwise the typo only surfaces later as a failed reply. Two fixes this exposed: * AI_CONFIGS now merges built-in defaults with the settings-file block instead of letting the file win outright. Every provider switch persists a "configs" block, so a file written by an older build would have permanently hidden ollama from the picker after an upgrade. * _persist_active_ai_config assigns "configs" instead of setdefault, so a pinned model actually survives a restart. * the hardcoded 120s response timeout is now CONJURER_AI_TIMEOUT_SECONDS - a self-hosted model on a modest GPU can legitimately need longer. Tests cover: ollama appears in the picker, select_model maps the legacy gpt-4o default instead of leaking it, model listing (server-queried, sorted, de-duplicated, failure -> AIError, unconfigured -> auth), pinning (latest only, blank/unknown rejected), and that provider_generate routes to the new path. Suite: 68 unit + 70 integration green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+74
-2
@@ -9,6 +9,7 @@ import time
|
||||
from other_functions import discord_friendly_send
|
||||
from constants import (
|
||||
AI_CONFIGS,
|
||||
AI_TIMEOUT_SECONDS,
|
||||
ASSISTANTS,
|
||||
CLAUDECLIENT,
|
||||
CYCLIC_WORDS,
|
||||
@@ -19,6 +20,7 @@ from constants import (
|
||||
MEMORY_FIVE_SIARA,
|
||||
MESSAGE_TABLE,
|
||||
MESSAGE_TABLE_MUZYKA,
|
||||
OLLAMACLIENT,
|
||||
OPENAICLIENT,
|
||||
SYSTEM_GPT_SETTINGS,
|
||||
WORD_REACTIONS,
|
||||
@@ -92,11 +94,53 @@ def set_active_ai_config(name: str) -> dict:
|
||||
raise RuntimeError("klient Anthropic nie jest skonfigurowany (brak ANTHROPIC_API_KEY)")
|
||||
if provider == "openai" and OPENAICLIENT is None:
|
||||
raise RuntimeError("klient OpenAI nie jest skonfigurowany (brak OPENAI_API_KEY)")
|
||||
if provider == "ollama" and OLLAMACLIENT is None:
|
||||
raise RuntimeError("Ollama nie jest skonfigurowana (ustaw CONJURER_OLLAMA_URL)")
|
||||
_ACTIVE_CONFIG_NAME = name
|
||||
_persist_active_ai_config(name)
|
||||
return cfg
|
||||
|
||||
|
||||
async def list_provider_models(name: str = None):
|
||||
"""Model ids selectable for a config.
|
||||
|
||||
For Ollama this ASKS THE SERVER (its OpenAI-compatible /v1/models), so the
|
||||
picker always reflects what is actually pulled on the box rather than a
|
||||
hardcoded list. Hosted providers are not enumerated - we only report what
|
||||
the config is wired to.
|
||||
"""
|
||||
cfg = AI_CONFIGS.get(name or _ACTIVE_CONFIG_NAME) or _active_config()
|
||||
if cfg.get("provider") == "ollama":
|
||||
if OLLAMACLIENT is None:
|
||||
raise AIError(
|
||||
"auth",
|
||||
RuntimeError("Ollama nie jest skonfigurowana (ustaw CONJURER_OLLAMA_URL)"),
|
||||
)
|
||||
try:
|
||||
resp = await OLLAMACLIENT.models.list()
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
raise _map_openai_error(exc)
|
||||
return sorted({item.id for item in resp.data})
|
||||
return [m for m in (cfg.get("latest_model"), cfg.get("cheap_model")) if m]
|
||||
|
||||
|
||||
def set_active_model(model: str, name: str = None) -> dict:
|
||||
"""Pin the model a config uses for normal replies, and persist it.
|
||||
|
||||
Only ``latest_model`` is changed; ``cheap_model`` stays as configured so the
|
||||
MUSIC path keeps its cheaper backend.
|
||||
"""
|
||||
cfg_name = name or _ACTIVE_CONFIG_NAME
|
||||
if cfg_name not in AI_CONFIGS:
|
||||
raise KeyError(cfg_name)
|
||||
if not model or not model.strip():
|
||||
raise ValueError("pusta nazwa modelu")
|
||||
cfg = AI_CONFIGS[cfg_name]
|
||||
cfg["latest_model"] = model.strip()
|
||||
_persist_active_ai_config(_ACTIVE_CONFIG_NAME)
|
||||
return cfg
|
||||
|
||||
|
||||
def _persist_active_ai_config(name: str) -> None:
|
||||
"""Best-effort write of the active-config choice into system_gpt_settings.json.
|
||||
|
||||
@@ -117,7 +161,10 @@ def _persist_active_ai_config(name: str) -> None:
|
||||
return
|
||||
if len(data) > 2 and isinstance(data[2], dict):
|
||||
data[2]["active"] = name
|
||||
data[2].setdefault("configs", AI_CONFIGS)
|
||||
# Assign (not setdefault): AI_CONFIGS is the in-memory truth and may
|
||||
# carry a model pinned via set_active_model, which setdefault would
|
||||
# silently drop on restart.
|
||||
data[2]["configs"] = AI_CONFIGS
|
||||
else:
|
||||
data = data[:2] + [{"active": name, "configs": AI_CONFIGS}]
|
||||
try:
|
||||
@@ -214,12 +261,37 @@ async def _anthropic_call(messages, model, cfg):
|
||||
return text.strip()
|
||||
|
||||
|
||||
async def _ollama_call(messages, model, cfg):
|
||||
"""Self-hosted counterpart of openai_call. Returns a plain string.
|
||||
|
||||
Ollama exposes an OpenAI-compatible /v1 surface, so the same message format
|
||||
and the same error mapping apply - only the base_url and the model ids
|
||||
differ. Chat Completions (not the Responses API) is what Ollama implements.
|
||||
"""
|
||||
if OLLAMACLIENT is None:
|
||||
raise AIError(
|
||||
"auth",
|
||||
RuntimeError("Ollama nie jest skonfigurowana (ustaw CONJURER_OLLAMA_URL)"),
|
||||
)
|
||||
try:
|
||||
resp = await OLLAMACLIENT.chat.completions.create(
|
||||
model=model,
|
||||
messages=messages,
|
||||
temperature=float(cfg.get("temperature", 0.2)),
|
||||
)
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
raise _map_openai_error(exc)
|
||||
return (resp.choices[0].message.content or "").strip()
|
||||
|
||||
|
||||
async def provider_generate(messages, model, temperature=0.2):
|
||||
"""Dispatch a chat completion to the active backend, normalising errors."""
|
||||
cfg = _active_config()
|
||||
try:
|
||||
if cfg.get("provider") == "anthropic":
|
||||
return await _anthropic_call(messages, model, cfg)
|
||||
if cfg.get("provider") == "ollama":
|
||||
return await _ollama_call(messages, model, cfg)
|
||||
return await openai_call(messages, model, temperature)
|
||||
except AIError:
|
||||
raise
|
||||
@@ -459,7 +531,7 @@ async def handle_response(
|
||||
try:
|
||||
# ...przygotowanie messages/system prompt/itp. jak masz...
|
||||
# retry/backoff + deadline (zachowuje Twoją semantykę logowania)
|
||||
timeout_sec = 120
|
||||
timeout_sec = AI_TIMEOUT_SECONDS
|
||||
deadline = time.time() + timeout_sec
|
||||
response = await asyncio.wait_for(
|
||||
provider_generate(messages=history_msgs, model=model_to_use),
|
||||
|
||||
Reference in New Issue
Block a user