A pipeline that reads a local corpus of AI papers, proposes candidate research gaps in a topic, and judges whether each is solvable today with low effort. Every claim is grounded in retrieved evidence rather than model opinion, and the system reports its own degradation instead of hiding it.
Role Sole designer and engineer. Period 2026. Scope Architecture, pipeline, local model serving, web interface, system documentation.
Gapfinder takes a research topic in plain English and returns a ranked list of specific unanswered questions in that topic, each paired with evidence that it is genuinely unanswered and an estimate of the work needed to close it. It reads a local corpus of roughly 164,000 AI papers, described on its own project page, plus live web search.
The commitment that shapes everything else is that the language model is never the source of truth about whether a gap is real. A model proposes; retrieved text decides. That sounds pedantic until you notice the failure it prevents: a system that confidently reports an open problem solved two years ago is worse than no system at all.
A run is eight stages. It expands the topic into search queries and pulls papers from the corpus, repeating until the fraction of newly discovered papers drops below a threshold. That is a measured stopping rule, not the model announcing it has read enough. On the most recent stored run, the topic “Small Specialized Language Models” pulled 147 papers, then 30, then 24, at which point the new-paper share had fallen to 12 percent and retrieval halted at 201.
Those papers are mined two ways. Explicit gaps are missing pieces somebody wrote down, in a limitations section or in a later paper describing what an earlier one failed to do. Implicit gaps are inferred from structure alone: two well-studied ideas that no paper in the corpus has connected. That run produced 474 explicit and 63 implicit.
Everything after is subtraction. A gate checks each candidate against the full corpus and a required web search, dropping it if something already covers it. A feasibility stage looks for existing techniques that could be combined to close the gap, and for comparable papers disclosing what hardware and data they needed. An evidence check then drops candidates where no evidence survived a relevance judgement. Nine survived, in just over six minutes.
The pipeline is one async orchestrator over four things: the corpus behind a lexical search index, a generation model, an embedding model, and a web search.
Retrieval asks the model for six short keyword queries per round, runs each for 25 results, unions and dedupes them, then shows the model what it has so the next round can aim at uncovered facets. Whether to continue is not the model’s call. The loop stops when a round’s genuinely new papers fall below 15 percent of what it found, or at a hard cap of four rounds.
Extraction reads a 14,000-character window of each paper for stated limitations. The implicit mechanisms each run the four-lens swarm and union their output. Near-duplicates are then merged by embedding similarity before the Gate, so swarm width never multiplies the cost of the required web search, and a stratified selection picks which candidates go forward rather than truncating the list blindly.
The Gate examines eight results per check against the full corpus and the web, with adversarial queries written from three angles of attack. A grounded pass scores each result into one of four closeness bins against the text actually fetched, and only a full match drops the candidate; substantial and partial pass through flagged. The web check runs against a plain HTML search endpoint with no API key, and marks the run degraded when it fails rather than claiming a check it did not perform. Feasibility abstracts the gap into domain-neutral language before searching, so it surfaces cross-domain analogues instead of the papers retrieval already returned, and fills the resource fields only where a paper states them. Candidates with no surviving evidence drop at a separate step, ranking is by feasibility alone, and the output is capped at 30.
The figure splits one run into three bands. Read the middle for the counts as they narrow, then read up and down from any stage to see what the model contributed there and what the retrieved text was allowed to decide.
The obvious way to raise confidence with language models is to run several and count agreement. This system refuses to, and the refusal rests on local measurement rather than principle.
An earlier, now shelved project in the same repository ran exactly that experiment and kept the artifacts. Two model judges scored the same 16 comparisons and agreed on 4, which is 25 percent, below the 50 percent you would get by coin flip. They were systematically inverting each other, and one configuration scored zero out of sixteen against its reference.
Agreement among models sharing a base and a prompt is one judgement repeated with noise, not independent corroboration. So models are used only where diversity is a pure win, generating candidates, and judgement stays a single pass over retrieved text.
Four analytical stances, called lenses, propose candidates independently: same underlying problem in different vocabulary, structurally similar machinery under different names, same data constraints, same failure mode. Four mechanisms realise the search underneath: a concept graph, embedding geometry, query-time bridging, and an agentic variant that dispatches one researcher per lens with a budget of twenty tool calls. That last one is deliberately denied web access, because the intended evaluation masks the corpus at a historical date and a web-connected agent could read the future.
The default runs all four and merges them. Generation runs on a 4B open-weight model served locally on one consumer GPU, with a small embedding model on a second, so a run costs electricity rather than per-token billing.
A run is then minutes of silence unless you build against it: the extraction stage on a 192-paper run took 133 seconds and originally emitted one progress event, where it now emits 195. The interface is a single static bundle with no build step, drawing a canvas map of the run’s papers.
The first complete run produced output that looked plausible and was not. All eight candidates came back rated medium effort, which is a rubric that has learned to say nothing, and the method evidence for a speculative decoding gap proposed “causal loop diagrams” and “graph ODEs”.
The root cause was one overshoot. To let retrieval reach across fields, queries are stripped of domain vocabulary; over a corpus this size a fully abstracted query matches almost anything, and the relevance judge saw only the abstraction, so it could not tell a real match from one sharing an abstract shape. Fixing both ends moved ratings to two low and five medium, and made the evidence check start rejecting.
One planned feature was cut on measurement: mining gaps that later papers frame about earlier ones cost a live citation lookup per paper and yielded zero usable pairs from forty papers.
Working prototype, not in continued development. The last commit was 25 July 2026 and nothing is currently running: the model server is down and both GPUs sit idle. Twelve runs exist in total, eleven of them complete.
The honest headline is the one the system documentation states about itself. The retrospective evaluation that would compare the four detection mechanisms is fully specified and was never implemented, so all four run end to end and none is justified as better. There is no automated test suite. The gate has dropped a candidate as already solved on exactly one run, which either means these gaps are genuinely open or that the gate is too permissive. Those caveats are printed by the system rather than discovered by a reader, which is the part worth defending.