gitea
ae1bd67772
CI / compile (pull_request) Successful in 12s
CI / unit (pull_request) Successful in 29s
CI / integration (pull_request) Successful in 31s
build / build (push) Successful in 33s
CI / compile (push) Successful in 15s
CI / unit (push) Successful in 30s
CI / integration (push) Successful in 34s
Field report: one httpx ReadTimeout inside habanero surfaced as 'Search <uuid> crashed', and the worker's crash handler then FORGOT the request - so an expensive search vanished and the user was told it was eaten, because a public API blinked once. Two defences: * Every habanero call goes through _crossref_call, which retries with linear backoff (CONJURER_CROSSREF_ATTEMPTS, default 4; backoff CONJURER_CROSSREF_BACKOFF, 5s). habanero wraps httpx errors in a plain RuntimeError so we can't filter narrowly - retries are simply bounded and the last error is re-raised. They now also run via asyncio.to_thread, so a slow Crossref no longer blocks the worker's event loop. * A crashed search is no longer dropped on the first failure: the attempt count is persisted with the request and the search is requeued (keeping any checkpoint, so a crashed DB scan resumes rather than restarts) until CONJURER_SEARCH_MAX_ATTEMPTS (default 3). It stays 'queued' for the bot's watchdog while retrying, and only after the cap is it forgotten. Also: scrape_bot's 'Got blocked' is routine sci-hub behaviour (it backs off an hour and carries on) - log it as WARNING, not ERROR, so it stops looking like a fault when scanning for real problems. Tests: retry-then-succeed, bounded re-raise, no retry on success, the persisted attempt counter, and forget-on-give-up. Suite: 58 unit + 70 integration green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>