AI: add a self-hosted Ollama backend, and let the picker choose the model
The bot could talk to OpenAI or Anthropic; this adds Ollama as a third provider so it can run against models hosted on our own box, and extends the switch command to pick WHICH model - not just which backend. Provider: Ollama exposes an OpenAI-compatible /v1 surface, so the client is just openai.AsyncOpenAI(base_url=OLLAMA_URL + "/v1"). That reuses the existing message format and the whole _map_openai_error mapping instead of forking a second error taxonomy. There is no API key - the endpoint IS the configuration, so the backend stays dormant (and refuses to be selected, with a message naming the variable) until CONJURER_OLLAMA_URL is set, the same way the Conan bridge behaves. Model selection: * list_provider_models() asks the SERVER for Ollama (/v1/models), so the picker shows what is actually pulled on the box rather than a hardcoded list. Hosted providers just report what they are wired to. * set_active_model() pins the config's latest_model and persists it; cheap_model is left alone so the MUSIC path keeps its cheaper backend. * $gadaj_teraz now takes "<config> [model]", and a new read-only $modele_ai lists what is available. Pinning an id Ollama does not have is rejected up front with the real list - otherwise the typo only surfaces later as a failed reply. Two fixes this exposed: * AI_CONFIGS now merges built-in defaults with the settings-file block instead of letting the file win outright. Every provider switch persists a "configs" block, so a file written by an older build would have permanently hidden ollama from the picker after an upgrade. * _persist_active_ai_config assigns "configs" instead of setdefault, so a pinned model actually survives a restart. * the hardcoded 120s response timeout is now CONJURER_AI_TIMEOUT_SECONDS - a self-hosted model on a modest GPU can legitimately need longer. Tests cover: ollama appears in the picker, select_model maps the legacy gpt-4o default instead of leaking it, model listing (server-queried, sorted, de-duplicated, failure -> AIError, unconfigured -> auth), pinning (latest only, blank/unknown rejected), and that provider_generate routes to the new path. Suite: 68 unit + 70 integration green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Vendored
+18
-2
@@ -13,10 +13,26 @@ CONJURER_NETRC_FILE=/secrets/.netrc
|
||||
|
||||
# --- AI backend switch --------------------------------------------------
|
||||
# Which AI config from system_gpt_settings.json is active at startup
|
||||
# (e.g. "gpt" or "claude"). Runtime switch: $gadaj_teraz <config>. Unset =
|
||||
# whatever the settings file's "active" key says, falling back to "gpt".
|
||||
# (e.g. "gpt", "claude" or "ollama"). Runtime switch:
|
||||
# $gadaj_teraz <config> [model]. Unset = whatever the settings file's "active"
|
||||
# key says, falling back to "gpt".
|
||||
# CONJURER_AI_CONFIG=gpt
|
||||
|
||||
# --- Ollama (self-hosted models) ----------------------------------------
|
||||
# The endpoint IS the whole configuration - no API key. Leave unset and the
|
||||
# "ollama" backend simply refuses to be selected. In-cluster, use the Service
|
||||
# DNS name; from outside, host:port. Port 11434 is Ollama's default.
|
||||
# CONJURER_OLLAMA_URL=http://ollama.ollama.svc.cluster.local:11434
|
||||
# CONJURER_OLLAMA_URL=
|
||||
# Model used for normal replies. $modele_ai lists what the server actually has
|
||||
# pulled, and $gadaj_teraz ollama <model> pins one at runtime (persisted).
|
||||
# CONJURER_OLLAMA_MODEL=llama3.1:8b
|
||||
# Model used for the cheaper MUSIC path; defaults to CONJURER_OLLAMA_MODEL.
|
||||
# CONJURER_OLLAMA_CHEAP_MODEL=
|
||||
# How long to wait for ANY backend to answer. 120s suits hosted APIs; a
|
||||
# self-hosted model on a modest GPU may need more.
|
||||
# CONJURER_AI_TIMEOUT_SECONDS=120
|
||||
|
||||
# --- Data ---------------------------------------------------------------
|
||||
# Single mounted volume; all writable state is rooted here.
|
||||
CONJURER_DATA_DIR=/data
|
||||
|
||||
Reference in New Issue
Block a user