Libr

2026-07-28 13:18:37 +02:00
8 changed files with 8 additions and 371 deletions
@@ -10,10 +10,10 @@ resources:
  - ingress.yaml # wejście po https + przekierowanie z http
 images:
  - name: gitea.czernobog.pl/gitea/astrololo-data
-    newTag: b36b3bee
+    newTag: 623603b1
  - name: gitea.czernobog.pl/gitea/astrololo-logic
-    newTag: b36b3bee
+    newTag: 623603b1
  - name: gitea.czernobog.pl/gitea/astrololo-render
    newTag: latest
  - name: gitea.czernobog.pl/gitea/astrololo-presentation
-    newTag: b36b3bee
+    newTag: 623603b1
@@ -1,104 +0,0 @@
 # Production ("deploy") Conjurer bot
 A second Conjurer bot that runs **alongside** the test `bot` in the `conjurer`
 namespace, sharing the same librarian / musician / radio and the shared
 `CONJURER_API_KEY`. It differs from the test bot only in:
 - its Discord token: the **`deploy-conjurer-netrc`** secret (already created),
 - distinct names/labels (`deploy-bot`) and NodePort **32443**,
 - **`/data` on NFS (RWX)** instead of a block PVC — so you can seed/upload files
  while it runs and back it up concurrently.
 Files: `deploy-bot.yaml` (Deployment + Service), `deploy-bot-backup.yaml`
 (daily backup CronJob). Both are wired into `kustomization.yaml`.
 ## Update channel — only on `[deploy]`
 Unlike the test bot (which tracks every build), the production bot updates **only
 when you promote a version**. It runs a **separate image**,
 `conjurer-bot-deploy`, which the conjurer CI tags **only when the commit message
 contains `[deploy]`** (it re-tags the already-built `conjurer-bot:<sha>` — same
 bytes). The image-updater's `deploy-bot` alias then bumps this bot's tag.
 So: normal commits update the test bot + librarian; a commit with `[deploy]` in
 its message is the one that also rolls the production bot.
 **Bootstrap (first run):** `conjurer-bot-deploy` doesn't exist until the first
 `[deploy]` build. Either land one commit with `[deploy]` in the message, or seed
 it once by hand:
 ```bash
 docker pull  gitea.czernobog.pl/gitea/conjurer-bot:ac16b77f
 docker tag   gitea.czernobog.pl/gitea/conjurer-bot:ac16b77f \
             gitea.czernobog.pl/gitea/conjurer-bot-deploy:ac16b77f
 docker push  gitea.czernobog.pl/gitea/conjurer-bot-deploy:ac16b77f
 ```
 (match the tag in `kustomization.yaml`). Note the librarian is shared, so it
 still tracks latest — mind large bot⇄librarian version skews.
 ## One-time setup
 1. **Secret** — already exists as `deploy-conjurer-netrc` (a netrc carrying the
   deploy Discord token, key `.netrc`). Nothing to do; the Deployment mounts it
   at `/secrets/.netrc`.
 2. **NFS export** — create the data + backup dirs on the NFS server
   (`192.168.1.34`, adjust to your real export):
   ```
   /mnt/Tank1/conjurer_swap/deploy-bot/data
   /mnt/Tank1/conjurer_swap/deploy-bot/backups
   ```
 ## Seeding / uploading config & state into /data
 Because `/data` is a plain NFS export, you upload files by copying them onto the
 share — **no `kubectl cp`, no scaling the bot down**. From the PVE box that holds
 `/srv/data` (mount the same export there, or rsync over it):
 ```bash
 # on a host that can see the NFS export:
 rsync -a /srv/data/ 192.168.1.34:/mnt/Tank1/conjurer_swap/deploy-bot/data/
 ```
 The bot roots everything under `CONJURER_DATA_DIR=/data`, so these land where it
 expects them:
 ```
 accident_log.json  Conjurer_graphics/  music/  pamiec.json
 pamiec_muzyki.json  settings.json  system_gpt_settings.json  transcripts/
 ```
 Upload as much or as little as you like — the bot seeds any missing JSON on
 first run. Live-editing `pamiec.json` while the bot writes to it can race; prefer
 seeding before first start or during a brief `kubectl -n conjurer scale
 deploy/deploy-bot --replicas=0` window.
 ## Backups (production only)
 `deploy-bot-backup` runs daily (04:17) and writes to the `backups` export:
 - `current/` — a full rsync mirror of `/data` (incl. music/graphics), always
  the latest state,
 - `snapshots/deploy-state-<date>.tgz` — dated archives of the critical small
  state (both memories, settings, accident log, transcripts), kept
  `RETENTION_DAYS` (30) days.
 The test bot is deliberately **not** backed up (amnesia there is fine).
 Restore example:
 ```bash
 tar xzf snapshots/deploy-state-2026-08-03-0417.tgz -C /mnt/.../deploy-bot/data/
 ```
 ## Routing librarian / musician callbacks
 **Librarian — handled automatically.** Each bot advertises its own
 `CONJURER_SELF_CALLBACK` (test `…:32442`, this bot `…:32443`) on every query and
 ping, and the librarian answers each result/pong back to the bot that asked. One
 librarian serves both bots — no `CONJURER_MAIN_BOT` repointing needed. (The
 librarian keeps `CONJURER_MAIN_BOT: http://bot:5000` only as a fallback for a bot
 that doesn't send a callback.)
 **Musician — still single-target.** The musician pushes "now playing" events to
 its one `CONJURER_MAIN_BOT` (currently the test bot, `192.168.1.73:32442`). Only
 one bot gets radio events; repoint the musician's `CONJURER_MAIN_BOT` to
 `192.168.1.73:32443` if the production bot should show them instead.
@@ -1,62 +0,0 @@
 apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: bot
  namespace: conjurer
 spec:
  replicas: 1                     # NIGDY więcej — jedna sesja gateway na token
  strategy: { type: Recreate }    # NIE RollingUpdate — dwa pody = wojna o sesję Discord
  selector: { matchLabels: { app: bot } }
  template:
    metadata: { labels: { app: bot } }
    spec:
      imagePullSecrets: [{ name: gitea-registry }]
      containers:
        - name: bot
          image: gitea.czernobog.pl/gitea/conjurer-bot:c8aae106
          ports: [{ containerPort: 5000 }]
          env:
            - { name: CONJURER_DATA_DIR,      value: "/data" }
            - { name: CONJURER_DISCORD_HOST,  value: "0.0.0.0" }
            - { name: CONJURER_DISCORD_PORT,  value: "5000" }
            - { name: CONJURER_API_KEY,  value: "d97008b3-7a5a-11f1-acd1-000b0e0f00ed" }
            - { name: CONJURER_NETRC_FILE, value: "/secrets/.netrc" }
            # ↓ NAZWY PODSTAW WG WYNIKU grepa z 0.2 ↓
            - { name: CONJURER_LIBRARIAN_SERVICE, value: "http://librarian:5001" }
            - { name: CONJURER_MUSICIAN_SERVICE,  value: "http://192.168.1.89:5000" }
            - { name: CONJURER_RADIO_SERVICE,     value: "http://192.168.1.79:5005" }
            - { name: CONJURER_FILE_SERVICE,     value: "http://192.168.1.89:5000" }
            - { name: CONJURER_RADIO_HARBOR,      value: "http://192.168.1.79:54321" }
            # Where the librarian sends THIS bot's results/pongs back to (its own
            # NodePort). Lets one librarian serve both bots - see deploy-bot.yaml.
            - { name: CONJURER_SELF_CALLBACK,     value: "http://192.168.1.73:32442" }
          volumeMounts:
            - { name: data,  mountPath: /data }
            - { name: netrc, mountPath: /secrets, readOnly: true }
          resources:
            requests: { cpu: "200m", memory: "256Mi" }
            limits:   { cpu: "2",    memory: "1Gi" }
      volumes:
        - name: data
          persistentVolumeClaim: { claimName: bot-data }
        - name: netrc
          secret: { secretName: conjurer-netrc }
 ---
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata: { name: bot-data, namespace: conjurer }
 spec:
  accessModes: [ReadWriteOnce]
  resources: { requests: { storage: 2Gi } }
 ---
 apiVersion: v1
 kind: Service
 metadata: { name: bot, namespace: conjurer }
 spec:
  type: NodePort                  # radio/musician z Dockera muszą go dosięgnąć
  selector: { app: bot }
  # nodePort PRZYPIĘTY na sztywno. Bez tego k8s losuje port z 30000-32767 przy
  # (od)tworzeniu Service, a musician/betoniarka z Dockera adresują bota po
  # http://192.168.1.73:32442 na sztywno - rozjazd = wyniki/eventy giną w locie.
  ports: [{ port: 5000, targetPort: 5000, nodePort: 32442 }]
@@ -1,22 +0,0 @@
 apiVersion: argocd-image-updater.argoproj.io/v1alpha1
 kind: ImageUpdater
 metadata:
  name: conjurer
  namespace: argocd
 spec:
  commonUpdateSettings:
    updateStrategy: "newest-build"
  writeBackConfig:
    method: "git:secret:argocd/git-creds"
    gitConfig:
      repository: "https://gitea.czernobog.pl/gitea/deploy.git"
      branch: "master"                       # ← gałąź repo deploy
      writeBackTarget: "kustomization:/conjurer"
  applicationRefs:
    - namePattern: "conjurer"
      images:
        - { alias: "librarian",  imageName: "gitea.czernobog.pl/gitea/conjurer-librarian" }
        - { alias: "bot",        imageName: "gitea.czernobog.pl/gitea/conjurer-bot" }
        # Production bot tracks its own image, which only gets new tags on
        # [deploy] commits (conjurer CI) - so it updates only on promoted builds.
        - { alias: "deploy-bot", imageName: "gitea.czernobog.pl/gitea/conjurer-bot-deploy" }
@@ -1,65 +0,0 @@
 # Daily backup of the PRODUCTION ("deploy") bot's /data.
 #
 # Only production is backed up - the test bot's amnesia is fine, this bot's is
 # not (pamiec.json / pamiec_muzyki.json are the conversation & music memory).
 # Both volumes are NFS (RWX), so this runs while the bot is live - no RWO clash.
 #
 # Two-part backup:
 #   * a full, space-efficient MIRROR (rsync --delete) of everything, incl. music
 #     and graphics - always the current state,
 #   * dated SNAPSHOTS of just the small critical state (the memories, settings,
 #     accident log, transcripts) so you can roll back to a point in time.
 # Snapshots older than RETENTION_DAYS are pruned.
 apiVersion: batch/v1
 kind: CronJob
 metadata:
  name: deploy-bot-backup
  namespace: conjurer
 spec:
  schedule: "17 4 * * *"          # every day at 04:17
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      backoffLimit: 2
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: backup
              image: alpine:3.20
              command: ["/bin/sh", "-c"]
              args:
                - |
                  set -eu
                  apk add --no-cache rsync >/dev/null
                  STAMP="$(date +%F-%H%M)"
                  mkdir -p /backup/current /backup/snapshots
                  echo "[$STAMP] mirroring /data -> /backup/current"
                  rsync -a --delete /data/ /backup/current/
                  echo "[$STAMP] snapshotting critical state"
                  tar czf "/backup/snapshots/deploy-state-$STAMP.tgz" -C /data \
                      accident_log.json pamiec.json pamiec_muzyki.json \
                      settings.json system_gpt_settings.json transcripts \
                      2>/dev/null || echo "  (some files absent - skipped)"
                  echo "[$STAMP] pruning snapshots older than ${RETENTION_DAYS}d"
                  find /backup/snapshots -name 'deploy-state-*.tgz' -type f \
                      -mtime +"${RETENTION_DAYS}" -delete
                  echo "[$STAMP] backup done"
              env:
                - { name: RETENTION_DAYS, value: "30" }
              volumeMounts:
                - { name: data,   mountPath: /data,   readOnly: true }
                - { name: backup, mountPath: /backup }
          volumes:
            # Same export the bot mounts (read-only here).
            - name: data
              nfs:
                server: 192.168.1.34
                path: /mnt/Tank1/conjurer_swap/deploy-bot/data
            # Backup target - ADJUST to your export.
            - name: backup
              nfs:
                server: 192.168.1.34
                path: /mnt/Tank1/conjurer_swap/deploy-bot/backups
@@ -1,80 +0,0 @@
 # Production ("deploy") Conjurer bot.
 #
 # Same infrastructure as the test `bot` (shares librarian / musician / radio and
 # the CONJURER_API_KEY), with three deliberate differences:
 #   1. its Discord token comes from the `deploy-conjurer-netrc` secret,
 #   2. distinct names/labels so it coexists with the test bot in this namespace,
 #   3. /data is an NFS volume (RWX) instead of a block PVC - so config/state can
 #      be uploaded while the bot runs (just copy onto the share) and backed up
 #      concurrently (see deploy-bot-backup.yaml). The test bot keeps its RWO PVC.
 #
 # ┌─ IMPORTANT: librarian & musician call ONE bot back ─────────────────────────┐
 # │ The librarian (CONJURER_MAIN_BOT) and musician push search results and      │
 # │ "now playing" events to a SINGLE bot address - currently the test bot's     │
 # │ NodePort 192.168.1.73:32442. Only one bot can receive them. To make the     │
 # │ PRODUCTION bot the one that gets librarian results / radio events, repoint   │
 # │ those services' CONJURER_MAIN_BOT to this bot's NodePort (…:32443 below).    │
 # └─────────────────────────────────────────────────────────────────────────────┘
 apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: deploy-bot
  namespace: conjurer
 spec:
  replicas: 1                     # NIGDY więcej — jedna sesja gateway na token
  strategy: { type: Recreate }    # NIE RollingUpdate — dwa pody = wojna o sesję Discord
  selector: { matchLabels: { app: deploy-bot } }
  template:
    metadata: { labels: { app: deploy-bot } }
    spec:
      imagePullSecrets: [{ name: gitea-registry }]
      containers:
        - name: bot
          # SEPARATE image from the test bot: conjurer-bot-deploy only gets a new
          # tag when a commit message contains [deploy] (see the conjurer CI), so
          # this bot updates only on versions you explicitly promote. Tag is
          # managed by the image-updater (kustomization images:).
          image: gitea.czernobog.pl/gitea/conjurer-bot-deploy:c8aae106
          ports: [{ containerPort: 5000 }]
          env:
            - { name: CONJURER_DATA_DIR,      value: "/data" }
            - { name: CONJURER_DISCORD_HOST,  value: "0.0.0.0" }
            - { name: CONJURER_DISCORD_PORT,  value: "5000" }
            - { name: CONJURER_API_KEY,  value: "d97008b3-7a5a-11f1-acd1-000b0e0f00ed" }
            - { name: CONJURER_NETRC_FILE, value: "/secrets/.netrc" }
            - { name: CONJURER_LIBRARIAN_SERVICE, value: "http://librarian:5001" }
            - { name: CONJURER_MUSICIAN_SERVICE,  value: "http://192.168.1.89:5000" }
            - { name: CONJURER_RADIO_SERVICE,     value: "http://192.168.1.79:5005" }
            - { name: CONJURER_FILE_SERVICE,      value: "http://192.168.1.89:5000" }
            - { name: CONJURER_RADIO_HARBOR,      value: "http://192.168.1.79:54321" }
            # This bot's own callback (its NodePort). The librarian records it per
            # query and answers results/pongs HERE - so it serves this bot AND the
            # test bot from one instance, no CONJURER_MAIN_BOT repointing needed.
            - { name: CONJURER_SELF_CALLBACK,     value: "http://192.168.1.73:32443" }
          volumeMounts:
            - { name: data,  mountPath: /data }
            - { name: netrc, mountPath: /secrets, readOnly: true }
          resources:
            requests: { cpu: "200m", memory: "256Mi" }
            limits:   { cpu: "2",    memory: "1Gi" }
      volumes:
        # /data on NFS (RWX): lets you seed/upload files while the bot runs (copy
        # straight onto the export from the PVE box that holds /srv/data) and lets
        # the backup CronJob read it concurrently. ADJUST server/path to your
        # actual export - mirrors the librarian's 192.168.1.34 Tank1 layout.
        - name: data
          nfs:
            server: 192.168.1.34
            path: /mnt/Tank1/conjurer_swap/deploy-bot/data
        - name: netrc
          secret: { secretName: deploy-conjurer-netrc }
 ---
 apiVersion: v1
 kind: Service
 metadata: { name: deploy-bot, namespace: conjurer }
 spec:
  type: NodePort                  # so librarian/musician COULD reach it if repointed
  selector: { app: deploy-bot }
  # Distinct pinned nodePort (test bot owns 32442). Point librarian/musician
  # CONJURER_MAIN_BOT at 192.168.1.73:32443 to route their callbacks here.
  ports: [{ port: 5000, targetPort: 5000, nodePort: 32443 }]
@@ -2,15 +2,4 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
  - namespace.yaml
-  - librarian.yaml
+  - librarian.yaml
  - bot.yaml
  - deploy-bot.yaml
  - deploy-bot-backup.yaml
 images:
  - name: gitea.czernobog.pl/gitea/conjurer-librarian
    newTag: ac16b77f
  - name: gitea.czernobog.pl/gitea/conjurer-bot
    newTag: ac16b77f
  # Production bot channel - only bumped when a [deploy]-tagged build appears.
  - name: gitea.czernobog.pl/gitea/conjurer-bot-deploy
    newTag: ac16b77f
@@ -10,45 +10,26 @@ spec:
    metadata: { labels: { app: librarian } }
    spec:
      imagePullSecrets: [{ name: gitea-registry }]
      # On SIGTERM the librarian checkpoints the running search and exits within
      # CONJURER_LIBRARIAN_GRACEFUL_TIMEOUT (default 45s). Give k8s enough grace
      # to let that finish before it SIGKILLs - otherwise a long scan's checkpoint
      # is cut short and the search restarts from scratch instead of resuming.
      terminationGracePeriodSeconds: 60
      containers:
        - name: librarian
-          image: gitea.czernobog.pl/gitea/conjurer-librarian:e243777e
+          image: gitea.czernobog.pl/gitea/conjurer-librarian:v0.1.0
          ports: [{ containerPort: 5001 }]
          env:
            - { name: CONJURER_LIBRARIAN_HOST,      value: "0.0.0.0" }
            - { name: CONJURER_LIBRARIAN_PORT,      value: "5001" }
            - { name: CONJURER_LIBRARIAN_DB_PATH,   value: "/doi/" }
            - { name: CONJURER_API_KEY,  value: "d97008b3-7a5a-11f1-acd1-000b0e0f00ed" }
            - { name: CONJURER_LIBRARIAN_MAXTHREADS,  value: "40" }
            - { name: CONJURER_CROSSREF_MAILTO,  value: "mtuszowski@gmail.com" }
            - { name: CONJURER_LIBRARIAN_CHUNK,  value: "_chunk.txt" }
            # Librarian jest W TYM SAMYM klastrze/namespace co bot - gada z nim
            # po DNS Service'u, nie przez nodePort węzła (ten zostaje tylko dla
            # zewnętrznych musician/betoniarka z Dockera). Odporne na przetasowania.
            - { name: CONJURER_MAIN_BOT,  value: "http://bot:5000" }
            - { name: CONJURER_LIBRARIAN_STATE_DIR, value: "/lib_temp_files" }
          volumeMounts:
            - { name: doi,   mountPath: /doi, readOnly: true }
            - { name: state, mountPath: /lib_temp_files }
          resources:
-            requests: { cpu: "100m", memory: "512Mi" }
+            requests: { cpu: "100m", memory: "256Mi" }
-            # Headroom bump: the work-queue OOM is fixed in code, but a deep
+            limits:   { cpu: "1",    memory: "1Gi" }
            # search still pulls up to 15000 Crossref records into RAM and the
            # accumulating result JSONs are loaded whole. 2Gi gives margin so a
            # big search isn't OOM-killed. Tune down if the node is tight.
            limits:   { cpu: "1",    memory: "2Gi" }
      volumes:
        - name: doi
          nfs:
            server: 192.168.1.34
-            path: /mnt/Tank1/conjurer_swap/librarian_data/base_it1
+            path: /mnt/Tank1/conjurer_doi
        - name: state
          persistentVolumeClaim: { claimName: librarian-state }
 ---