librarian: stop the DOI search from hanging on a chunk-count mismatch

search_bot conflated MAXTHREADS into two jobs at once - how many chunk files to
read (files 0..MAXTHREADS-1) AND how many producer sentinels to wait for - so
the two had to match exactly. Set too low it silently skipped trailing chunks;
set too high (or with any chunk missing/unreadable) a producer crashed before
emitting its sentinel, the consumers' count never reached the threshold, and
search_for_doi hung on join() forever. The idle-timeout failsafe that was meant
to break a starved consumer was dead code: `if empty_counter > 5: ... elif
empty_counter > 10: break` - >10 implies >5, so the elif never ran.

Fix, three layers:
* auto-discover the chunk files present (discover_chunk_files: <n>_chunk.txt in
  numeric order) instead of range(0, MAXTHREADS). All files are read regardless
  of count, and no producer is ever pointed at a missing file;
* the sentinel threshold is now the number of producers actually started, so it
  can't drift from what's emitted;
* producers emit their sentinel in a finally, so even a crash (missing/unreadable
  chunk) can't starve the count; and the idle backstop is reordered so it can
  actually fire (>EMPTY_LIMIT seconds) as a last resort.

MAXTHREADS is deprecated and unused (kept only so old env files don't break);
docs/env updated to say chunk files are auto-discovered.

For the reported case (MAXTHREADS=40, files 0..43): before, files 40-43 were
silently never searched, and any run that referenced a missing chunk hung
forever. After, all 44 are searched and it always terminates.

Verified in a pytest-only venv (tests/unit/test_search_bot.py): DOI in a
trailing chunk is found; an unreadable chunk still terminates; empty dir returns
at once; discovery is numeric-sorted. Full unit job 27 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

tests/unit/test_search_bot.py

@@ -0,0 +1,89 @@
+"""Unit tests for the librarian DOI search - specifically that it always
+terminates.
+
+The producer/consumer search used to hang whenever the number of chunk files it
+was told to read (MAXTHREADS) did not exactly match the files on disk: too few
+and it silently skipped trailing chunks, too many and a producer pointed at a
+missing file crashed before emitting its sentinel, starving the consumers'
+termination count forever. These tests pin down the fix: chunk files are
+auto-discovered, the sentinel threshold equals the number of producers actually
+started, and every producer emits its sentinel even on error.
+"""
+import logging
+import threading
+
+import search_bot
+
+_LOG = logging.getLogger("test-search-bot")
+_LOG.addHandler(logging.NullHandler())
+
+
+def _write_chunks(directory, count, target=None, target_index=None):
+    """Create <n>_chunk.txt files; optionally drop `target` into one of them."""
+    for n in range(count):
+        lines = [f"10.0000/decoy-{n}-a\n", f"10.0000/decoy-{n}-b\n"]
+        if target is not None and n == target_index:
+            lines.append(target + "\n")
+        (directory / f"{n}_chunk.txt").write_text("".join(lines), encoding="utf-8")
+
+
+def _run_bounded(dois, timeout=20):
+    """Run search_for_doi in a thread; return (finished_in_time, result)."""
+    box = {}
+    live = []
+    worker = threading.Thread(
+        target=lambda: box.update(result=search_bot.search_for_doi(dois, live, _LOG)),
+        daemon=True,
+    )
+    worker.start()
+    worker.join(timeout)
+    return (not worker.is_alive()), box.get("result")
+
+
+def test_finds_doi_in_trailing_chunk(tmp_path, monkeypatch):
+    # Target lives in the LAST chunk - the one the old MAXTHREADS=N-too-low would
+    # never have read. Auto-discovery must read every chunk present.
+    monkeypatch.setattr(search_bot, "DATABASE_PATH", str(tmp_path) + "/")
+    target = "10.1234/target.in.trailing.chunk"
+    _write_chunks(tmp_path, count=6, target=target, target_index=5)
+
+    finished, result = _run_bounded([(target, "DATA"), ("10.9999/absent", "DATA")])
+
+    assert finished, "search hung instead of terminating"
+    hit = [r for r in result if r["DOI"] == target and r["exists"]]
+    assert hit, "DOI in the trailing chunk was not found"
+
+
+def test_terminates_when_a_chunk_is_unreadable(tmp_path, monkeypatch):
+    # A chunk that exists at discovery time but cannot be opened (here: it is a
+    # directory) makes its producer raise. The finally-sentinel must still fire
+    # so the consumers' count completes and the search does not deadlock.
+    monkeypatch.setattr(search_bot, "DATABASE_PATH", str(tmp_path) + "/")
+    _write_chunks(tmp_path, count=3)
+    (tmp_path / "9_chunk.txt").mkdir()  # discovered as a chunk, un-openable
+
+    finished, _ = _run_bounded([("10.0000/decoy-0-a", "DATA")])
+
+    assert finished, "an unreadable chunk deadlocked the search"
+
+
+def test_no_chunks_returns_immediately(tmp_path, monkeypatch):
+    # Empty database dir: return an (all-not-found) result at once, never hang.
+    monkeypatch.setattr(search_bot, "DATABASE_PATH", str(tmp_path) + "/")
+
+    finished, result = _run_bounded([("10.0/x", "DATA")], timeout=10)
+
+    assert finished
+    assert result == [{"DOI": "10.0/x", "exists": False, "data": "DATA"}]
+
+
+def test_discover_chunk_files_sorted_numerically(tmp_path, monkeypatch):
+    monkeypatch.setattr(search_bot, "DATABASE_PATH", str(tmp_path) + "/")
+    for n in (0, 2, 10, 1):
+        (tmp_path / f"{n}_chunk.txt").write_text("x\n", encoding="utf-8")
+    (tmp_path / "notes.txt").write_text("ignore me\n", encoding="utf-8")
+
+    found = search_bot.discover_chunk_files(_LOG)
+
+    # Numeric order (10 after 2, not lexicographic), and non-chunk files ignored.
+    assert found == ["0_chunk.txt", "1_chunk.txt", "2_chunk.txt", "10_chunk.txt"]