Files
conjurer/tests/integration
gitea 9f22dbf94b
CI / compile (pull_request) Successful in 18s
CI / unit (pull_request) Successful in 39s
CI / integration (pull_request) Failing after 1m2s
CI / compile (push) Successful in 14s
CI / unit (push) Successful in 34s
CI / integration (push) Successful in 37s
build / build (push) Successful in 33s
Librarian: 'still searching' heartbeat every 20 min
A deep scan runs for hours with nothing in the log between start and
finish, so it's impossible to tell a working search from a wedged one.
Every CONJURER_LIBRARIAN_HEARTBEAT_SECONDS (default 1200 = 20 min) a
running search now logs that it is still going, with its uuid, the search
phrase, hits so far, elapsed minutes, and a rough how-far-along.

The estimate is deliberately cheap: the producers ALREADY record a byte
offset per chunk file (the resume watermarks), and the total size is
stat()'d once per search when the chunk list is discovered. A reading is
then just a sum over ~40 ints - nothing extra happens per line, and no
cycles are spent estimating how many cycles are left.

search_for_doi takes an optional progress dict it fills with the live
positions dict + total_bytes; the librarian publishes the running search
(uuid/query/progress/live hits) while the scan runs and clears it in
finally. Nothing running => the heartbeat stays quiet.

Tests: percentage maths incl. unknown-total and >100% clamping, the
register/clear round-trip, and an end-to-end check that a real scan fills
progress so the offsets cover the chunk files on disk. Suite: 58 unit +
65 integration green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-04 12:09:46 +02:00
..