Refresh papers and define Agent evaluation

docs/07-exploration-history.md

@@ -1,6 +1,6 @@
 # Exploration History
 
-last_reviewed: 2026-07-12
+last_reviewed: 2026-07-27
 
 This is the decision history, including failed routes. Preserve it so future work does not rediscover the same lessons.
 
@@ -152,8 +152,74 @@ This audit is the current research baseline. Read `research/memory/findings.md`
 
 Relevant commit: `ab758e7`.
 
-## 9. Current Position
+## 9. Position After the Memory Audit
 
 The repository now has a strong collection substrate, a functioning exploration prototype, and one evidence-led field audit. It does not yet have the stateful personal learning/internalization layer that motivated the project.
 
-The next meaningful experiment is not another graph redesign or another bulk paper sweep. It is one end-to-end learning path that records a user's claims, evidence judgments, contradictions, applications, and later recall, using the memory audit as its first evidence base.
+At this point, the proposed next experiment was an end-to-end learning path that records a user's claims,
+evidence judgments, contradictions, applications, and later recall.
+
+## 10. From Learning Page to Knowledge Funnel
+
+The user then made the throughput problem more precise: the target is not a nicer way to read papers one
+by one. It is to preserve as much of an expert's understanding as possible while reducing the time needed
+to process roughly 1,000 papers, ideally to one concentrated day.
+
+The important expert behavior is:
+
+```text
+paper -> what problem / why now / prior method / actual change
+-> evidence / failure boundary / place in the field
+-> compare with neighboring work
+-> notice missing methods and generate new hypotheses
+```
+
+This reframing changes the primary artifact from a course or browser to a paper compiler:
+
+1. extract compact, source-linked paper fingerprints;
+2. collapse repeated work into research moves and lineages;
+3. compare evidence and regressions inside each move;
+4. compile a field grammar that the user can inspect, challenge, and apply;
+5. persist the user's judgments and delayed review only after the compiled knowledge is useful.
+
+Decision retained: the frontend is optional at this stage. The first proof should be an inspectable data
+pipeline and a high-density research artifact, not another navigation surface.
+
+## 11. Incremental Refresh and Agent Evaluation Design
+
+On 2026-07-27 the paper boundary was refreshed from July 9 through July 27:
+
+- 249 unique arXiv candidates;
+- 157 high-relevance selected records;
+- 151 new queued paper items;
+- 1,125 total paper items after indexing.
+
+The arXiv export API timed out and returned 503, so the collector gained transient-error retries,
+configurable timeouts, progress output, and an official search-page fallback. The successful run used
+that fallback. This is an operational improvement, not evidence that the selected papers were read.
+
+The same work inspected `zk-data-agent` at commit `e2e7a7b` and the Zero product flow. The requested
+output was clarified as an Agent evaluation scheme, not a new Agent implementation.
+
+What the target already had:
+
+- isolated real workspaces and layered tests;
+- versioned run events and complete tool lifecycle traces;
+- evidence-gated completion;
+- one real-model live evaluation script.
+
+What it lacked:
+
+- a versioned task suite with fixtures and deterministic validators;
+- repeated paired baseline/variant runs;
+- complete reproducibility manifests;
+- unified failure labels and explicit false-completion checks;
+- separate gain and regression accounting.
+
+The resulting v1 contract lives under `research/evaluation/`, with ten task contracts and a JSONL scorer.
+No real-model comparison was run, and most tasks correctly remain marked as requiring target-repository
+runner work.
+
+Decision retained: evaluation exists to decide whether one controlled Agent change should ship. Do not
+compress safety, false completion, gains, regressions, latency, and cost into one decorative score, and
+do not build a dashboard before the runner and validators exist.