mirror of
https://github.com/migatu/conjurer.git
synced 2026-07-14 21:38:38 +00:00
Dockerization + ai review recomendations.
This commit is contained in:
@@ -0,0 +1,163 @@
|
||||
# Conjurer Deployment Guide
|
||||
|
||||
This document walks through deploying the Conjurer stack (Discord bot, music
|
||||
service, librarian service) on two Raspberry Pi 4s and one Windows host, first
|
||||
with plain Docker Compose, then with a future Kubernetes setup.
|
||||
|
||||
## 1. Current Hardware Layout
|
||||
|
||||
- **Windows PC**: Stores persistent data (JSON memories, configs, music
|
||||
catalogue). Shares folders over the network (SMB) for the Pis.
|
||||
- **Raspberry Pi A** (“radio”): Runs the Liquidsoap/liq radio pipeline and the
|
||||
musician service (Flask + file watcher).
|
||||
- **Raspberry Pi B** (“bot”): Runs the Discord bot, communication bridge, and
|
||||
librarian Flask service.
|
||||
|
||||
You can rebalance as follows:
|
||||
|
||||
| Service | Recommended Host | Notes |
|
||||
|---------------------|------------------|-------|
|
||||
| Discord bot + comms | Raspberry Pi B | Needs outbound internet, moderate CPU |
|
||||
| Librarian service | Raspberry Pi B | CPU-heavy during Crossref queries; keep close to bot |
|
||||
| Musician service | Raspberry Pi A | Has direct disk access to music, same box as Liquidsoap |
|
||||
| Data storage | Windows | Expose via SMB; mount inside containers |
|
||||
|
||||
## 2. Prepare Shared Storage on Windows
|
||||
|
||||
1. Create directories, e.g. `C:\Conjurer\config`, `C:\Conjurer\logs`,
|
||||
`C:\Conjurer\music`, `C:\Conjurer\playlists`, `C:\Conjurer\secrets`.
|
||||
2. Copy your existing JSON settings (`settings.json`, `pamiec.json`,
|
||||
`pamiec_muzyki.json`, `system_gpt_settings.json`, etc.) into `config`.
|
||||
3. Create blank placeholder files if they do not exist yet.
|
||||
4. Share the root folder (`C:\Conjurer`) over SMB with read/write access for the
|
||||
Pi user (create credentials if necessary).
|
||||
|
||||
## 3. Configure Environment Files
|
||||
|
||||
1. On your workstation, copy the example env files:
|
||||
```bash
|
||||
cp docker/env/bot.env.example docker/env/bot.env
|
||||
cp docker/env/musician.env.example docker/env/musician.env
|
||||
cp docker/env/librarian.env.example docker/env/librarian.env
|
||||
```
|
||||
2. Edit each `docker/env/*.env` to replace `HACKME!` with real values:
|
||||
- `DISCORD_TOKEN`, `OPENAI_API_KEY`, `CONJURER_API_KEY` (use the same value for
|
||||
all services).
|
||||
- For musician/librarian, adjust mounts to the SMB paths you will mount on the
|
||||
Pis, e.g. `/mnt/conjurer/music`.
|
||||
- Set `CONJURER_CROSSREF_MAILTO` to a real email as required by Crossref.
|
||||
3. If you rely on `.netrc`, copy it to `C:\Conjurer\secrets\.netrc` and set
|
||||
`CONJURER_NETRC_FILE` accordingly.
|
||||
|
||||
## 4. Install Docker on Raspberry Pis and Windows
|
||||
|
||||
### Raspberry Pi
|
||||
```bash
|
||||
curl -fsSL https://get.docker.com | sh
|
||||
sudo usermod -aG docker $USER
|
||||
sudo reboot
|
||||
|
||||
# Install docker compose plugin
|
||||
sudo apt-get install docker-compose-plugin
|
||||
```
|
||||
|
||||
### Windows
|
||||
- Install **Docker Desktop**.
|
||||
- Enable WSL2 backend and expose the shared Windows folders to the containers
|
||||
(Docker Desktop settings → Resources → File Sharing).
|
||||
|
||||
## 5. Deploy Musician Service (Pi A)
|
||||
|
||||
1. SSH into Raspberry Pi A.
|
||||
2. Mount the Windows SMB share:
|
||||
```bash
|
||||
sudo mkdir -p /mnt/conjurer
|
||||
sudo apt-get install cifs-utils
|
||||
sudo mount -t cifs //WINDOWS_HOST/Conjurer /mnt/conjurer -o user=YOURUSER
|
||||
```
|
||||
Add an entry to `/etc/fstab` for persistence.
|
||||
3. Copy the repo to the Pi or `git clone` it.
|
||||
4. On Pi A, create override compose file (optional) pointing volumes to
|
||||
`/mnt/conjurer`.
|
||||
5. Start only the musician service:
|
||||
```bash
|
||||
docker compose --profile musician up --build -d musician
|
||||
```
|
||||
Alternatively, duplicate `docker-compose.yml`, strip other services, and run
|
||||
`docker compose up -d`.
|
||||
|
||||
## 6. Deploy Bot + Librarian (Pi B)
|
||||
|
||||
1. Repeat SMB mount on Pi B (same mount path).
|
||||
2. Copy repo / pull latest changes.
|
||||
3. Create `.env` files with tokens (or copy from control machine).
|
||||
4. Start bot and librarian:
|
||||
```bash
|
||||
docker compose up -d bot librarian
|
||||
```
|
||||
|
||||
## 7. Optional: Run Supporting Liquidsoap Radio
|
||||
|
||||
- Keep Liquidsoap on Pi A as-is, using the same music directories. Ensure the
|
||||
musician container has read access to those directories (bind mount).
|
||||
|
||||
## 8. Verifying
|
||||
|
||||
1. `docker ps` on each Pi to confirm containers running.
|
||||
2. Inspect logs under the mounted logs directory (`/mnt/conjurer/logs`).
|
||||
3. Join Discord server; issue commands to confirm functionality.
|
||||
4. Hit health endpoints manually (e.g. `curl http://PIB:5000/conjurer`).
|
||||
|
||||
## Rebalancing Suggestions
|
||||
|
||||
- If librarian CPU spikes become an issue, move it to Pi A or another host.
|
||||
- If you add a dedicated NAS, mount the network share read-only for the musician
|
||||
container and read/write for other services.
|
||||
|
||||
## 9. Future Kubernetes Deployment (Outline)
|
||||
|
||||
### Hardware Considerations
|
||||
|
||||
- Minimum three nodes for HA: use the existing two Pis plus one additional Pi 4
|
||||
(8 GB preferred). Use Windows PC as storage provider via NFS/SMB CSI driver or
|
||||
as a data gateway.
|
||||
- Consider Pi clusters with USB SSDs for better I/O.
|
||||
|
||||
### Cluster Setup Steps
|
||||
|
||||
1. Install a lightweight Kubernetes distribution (e.g., k3s) on each Pi:
|
||||
```bash
|
||||
curl -sfL https://get.k3s.io | sh -
|
||||
# On additional nodes
|
||||
curl -sfL https://get.k3s.io | K3S_URL=https://MASTER:6443 K3S_TOKEN=HACKME sh -
|
||||
```
|
||||
2. Install MetalLB for load balancer support on LAN.
|
||||
3. Configure persistent volumes using:
|
||||
- `nfs-subdir-external-provisioner` pointing to Windows share (ensure Windows
|
||||
host supports NFS or run an NFS gateway on another machine).
|
||||
- Alternatively, attach individual USB drives to each Pi and use
|
||||
`local-path-provisioner` for node-local storage.
|
||||
4. Create Kubernetes `Secret` objects for tokens (`DISCORD_TOKEN`, etc.).
|
||||
5. Define `Deployment` manifests for each service (bot, musician, librarian) and
|
||||
associated `Services`.
|
||||
6. Expose Discord bot ports via `NodePort` or Ingress.
|
||||
7. Use `StatefulSet` if you need stable identity for the musician service (due to
|
||||
local storage).
|
||||
|
||||
### Optimisation Tips
|
||||
|
||||
- Keep CPU-heavy librarian pods optionally on a beefier node; use
|
||||
`nodeSelector`/`affinity` to pin workloads.
|
||||
- Consider splitting the persistent storage: music on Pi A (USB disk), logs and
|
||||
configs on Pi B, backups on Windows.
|
||||
- For improved reliability, add at least one extra Pi for quorum and to host the
|
||||
communication bridge if the bot node fails.
|
||||
|
||||
## Summary Checklist
|
||||
|
||||
1. Prepare Windows shares & tokens.
|
||||
2. Configure `docker/env/*.env` using `HACKME!` templates as reference.
|
||||
3. Install Docker on Pis, mount network shares.
|
||||
4. Launch musician on Pi A, bot + librarian on Pi B.
|
||||
5. Verify Discord functionality and API endpoints.
|
||||
6. Plan Kubernetes migration when ready (k3s + MetalLB + storage provisioner).
|
||||
Reference in New Issue
Block a user