Durable result delivery: OUTBOX + idempotent INBOX so results never die
An 8h search result must survive a transient bot outage, an api/address misroute, or a restart of either side. Make the librarian->bot result path durably at-least-once with idempotent rendering: Shared: durable_queue.DiskQueue - a dependency-free, atomically-written, one-file-per-key disk queue (unit-tested), shared by both images (added to Dockerfile.librarian; the bot already COPYs *.py). Librarian (sender): finished results go to a persistent OUTBOX before sending; delivery retries with backoff; an entry is removed only on a positive ACK; a resender thread keeps flushing the OUTBOX, so a result survives a bot outage AND a librarian restart (OUTBOX is on the state volume) - it simply keeps trying until acked. Bot (receiver): /conjurer is now idempotent and durable - each result is persisted to an INBOX before acking and only queued if its uuid was not already delivered (dropped as a duplicate) or already pending. Once the cog actually renders it, mark_delivered() records the uuid and clears the inbox, so the librarian's resends become no-ops. On startup the bot replays any accepted-but-unrendered result from the INBOX, so a bot crash mid-flight doesn't lose it. Pongs stay ephemeral. Together: the librarian keeps a result until the bot confirms it; the bot keeps it until it is on screen; duplicates never double-render. Combined with the deploy return-path fix, an expensive result no longer vanishes. Tests: unit test_durable_queue; integration test_librarian_outbox (retry/backoff, resend survives outage) and test_result_durable_delivery (persist, dedup pending, dedup delivered, replay, pong not persisted). Suite: 55 unit + 39 integration green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit was merged in pull request #12.
This commit is contained in:
+13
-1
@@ -15,7 +15,13 @@ import requests
|
||||
from discord.ext import commands, tasks
|
||||
|
||||
from ai_functions import handle_response
|
||||
from communication_subroutine import IN_COMM_Q, OUT_COMM_Q, QueryControl, submit_ai_query
|
||||
from communication_subroutine import (
|
||||
IN_COMM_Q,
|
||||
OUT_COMM_Q,
|
||||
QueryControl,
|
||||
mark_delivered,
|
||||
submit_ai_query,
|
||||
)
|
||||
from constants import (
|
||||
DIR_PATH_SADOX,
|
||||
LIBRARIAN_SERVICE_ADDRESS,
|
||||
@@ -169,6 +175,12 @@ class DataModule(commands.Cog):
|
||||
await ctx.send(message)
|
||||
message = ""
|
||||
|
||||
# The result is now on screen: mark it delivered so the
|
||||
# librarian's resends become no-ops and it is dropped from the
|
||||
# durable inbox (never replayed again). Done after the core
|
||||
# render but before the optional AI review, which is a bonus.
|
||||
mark_delivered(str(fresh_data.uuid))
|
||||
|
||||
# Optional AI pass: re-rank the (already Crossref-relevance-
|
||||
# sorted) DOI list and review the sources. Enqueued to the AI
|
||||
# worker so it runs on whatever backend $gadaj_teraz selected;
|
||||
|
||||
Reference in New Issue
Block a user