test_search_fills_progress_with_live_positions_and_total asserted 100%
coverage after searching for a DOI that EXISTS. Once every queried DOI is
found the consumer signals TERM and the producers stop mid-file, so the
recorded offsets reach an arbitrary point - the assertion was racing the
scan and failed roughly one full-suite run in two.
Split into the two things that are actually deterministic: coverage is now
measured with an ABSENT DOI (nothing can stop the scan early, so 100% is
guaranteed), and the found-target case asserts what holds regardless of
where the producers stopped - the total is known, progress is bounded and
sane, and the hit is reported.
Verified: 5 consecutive runs of the file and 3 consecutive full integration
runs, all green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A deep scan runs for hours with nothing in the log between start and
finish, so it's impossible to tell a working search from a wedged one.
Every CONJURER_LIBRARIAN_HEARTBEAT_SECONDS (default 1200 = 20 min) a
running search now logs that it is still going, with its uuid, the search
phrase, hits so far, elapsed minutes, and a rough how-far-along.
The estimate is deliberately cheap: the producers ALREADY record a byte
offset per chunk file (the resume watermarks), and the total size is
stat()'d once per search when the chunk list is discovered. A reading is
then just a sum over ~40 ints - nothing extra happens per line, and no
cycles are spent estimating how many cycles are left.
search_for_doi takes an optional progress dict it fills with the live
positions dict + total_bytes; the librarian publishes the running search
(uuid/query/progress/live hits) while the scan runs and clears it in
finally. Nothing running => the heartbeat stays quiet.
Tests: percentage maths incl. unknown-total and >100% clamping, the
register/clear round-trip, and an end-to-end check that a real scan fills
progress so the offsets cover the chunk files on disk. Suite: 58 unit +
65 integration green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gitea