agent/docs/08-next-work.md

# Next Work

last_reviewed: 2026-07-27

## P0: Prove the Paper Compiler

The main project goal is now a high-efficiency knowledge funnel, not a paper browser and not a fixed
course. It should let one person process a corpus on the scale of 1,000 papers while retaining the
structure an expert normally acquires through repeated reading.

The first proof should use one bounded topic but process the whole relevant corpus, not a hand-picked
dozen papers.

### Stage A: Paper Fingerprints

Compile each relevant paper into a source-linked record:

```text
paper_id / version / source boundary
problem and why it matters
prior method or baseline
actual delta
mechanism
evidence and comparison conditions
failure boundary
closest related papers
candidate research move
confidence and review status
```

Use abstracts for high-recall triage only. Claims about mechanism, improvement, ablation, cost, or
limitations require full-text evidence with a page/table/section locator.

### Stage B: Research Moves and Lineages

Do not ask the user to read 1,000 isolated fingerprints. Collapse them into repeated moves such as:

```text
retrieve better / write selectively / preserve execution state
verify before completion / recover after failure / change the environment
trade generality for task structure / improve benchmark validity
```

For each move, preserve:

- the problem pressure that caused it;
- the strongest baseline;
- what variants actually change;
- comparable evidence and non-comparable claims;
- regressions and unsolved boundaries;
- chronological lineage and independent replications.

### Stage C: Field Grammar

Compile the moves into a small, challengeable model:

```text
problem pressures -> available moves -> evidence conditions
-> known trade-offs -> unresolved gaps -> design predictions
```

The field grammar must answer a new target without relying on paper titles:

1. what class of problem is this;
2. which moves have been tried;
3. why they work or fail;
4. what evidence would distinguish alternatives;
5. what appears missing and is worth testing.

### Stage D: User Compression

Only after the compiled model is useful, add persistent user state:

```text
accepted / rejected / uncertain claims
user rationale
counterevidence seen
application attempts
last reviewed / next review
```

Recall is not measured by page completion. The user should be able to reconstruct the field grammar,
diagnose an unseen design, and update a belief when contrary evidence appears.

### P0 Acceptance

The first vertical slice is successful only if:

- corpus coverage and exclusions are explicit;
- every consequential statement links to evidence or is marked as inference;
- repeated papers are compressed without erasing meaningful disagreement;
- the user can inspect a paper's position in seconds;
- the compiled model surfaces at least one non-obvious gap or testable prediction;
- a timed user session shows better diagnosis than raw title/abstract browsing.

The frontend is optional. JSON/Markdown plus a rigorous review session is enough for the first proof.

## P1: Implement the Agent Evaluation Runner

The design contract is complete at
`research/evaluation/k1412-agent-evaluation-v1.md`. Implementation belongs in
`https://git.k1412.top/wuyang/zk-data-agent.git`.

Implement in this order:

1. extract a reusable runner from `scripts/eval-live-work.py`;
2. load versioned task JSON, workspace fixtures, budgets, and deterministic validators;
3. export complete run manifests, sanitized traces, and result JSONL;
4. run paired baseline/variant repetitions in randomized order;
5. add fault injection, multi-turn context, memory, and delegation task adapters;
6. aggregate with `tools/evaluation/score_agent_runs.py`;
7. inspect gains, regressions, both-fail, false completions, and safety events before deciding.

Start with the two task contracts marked `runnable-with-current-live-script`. Do not claim that the
remaining contracts run until their listed runner requirements exist.

### Evaluation Release Boundary

A candidate must not ship if it introduces:

- a safety or isolation violation;
- more false completions;
- an unrecorded model, prompt, tool, fixture, context, or budget change;
- a critical-family regression hidden by average success;
- gains that disappear under independent validation.

No live benchmark has been run yet. The latest evaluation papers are only abstract-level candidates and
need full-text audit before they can change the contract.

The report at `https://agent-eval.k1412.top/` explains the contract and simulates its decision rules.
It is not evidence that any Agent variant passed. Future result views must consume the same validator
JSONL instead of embedding numbers in frontend code.

## P1: Continue Collection Through Questions

The corpus now contains 1,125 papers, 1,110 of them queued. More collection is useful only when paired
with processing:

- run a narrow weekly increment from the last explicit date boundary;
- retain broad candidates and truthful status labels;
- prioritize papers that challenge a current research move or evaluation assumption;
- audit the latest evaluation cluster first;
- refresh Beijing jobs and official industry practice separately.

After item changes:

```bash
python3 tools/collection/build_index.py
python3 tools/check_project.py
```

## P2: Productize Only Validated Workflows

The Atlas/search/reader can remain as an orientation surface. New frontend work should follow a proven
workflow:

- a topic opens unresolved questions and compiled moves, not only a paper list;
- a paper shows the move it supports, challenges, or merely resembles;
- the primary view is the active research decision;
- candidate claims require evidence review before becoming durable knowledge;
- user judgments and review history remain inspectable and reversible.

Avoid another graph or dashboard redesign until either the paper compiler or evaluation runner produces
something the user repeatedly uses.

## P2: Operations

Still missing:

- CI for `tools/check_project.py` and unit tests;
- authentication and authorization for any public multi-user state;
- automated backup for future personal learning data;
- repository-managed documentation of proxy/TLS deployment;
- web API and frontend flow tests.

## Explicit Non-Goals

- Do not present 1,110 queued papers as understood knowledge.
- Do not revive the 12-paper lecture as a field synthesis.
- Do not use per-paper summaries as the final knowledge unit.
- Do not build a leaderboard or single composite Agent score.
- Do not let an Agent or the same judge model certify its own success.
- Do not hide regressions behind average improvement.
- Do not make the graph more decorative as the next milestone.

## First Continuation Tasks

For the knowledge-base repository:

1. choose one bounded topic and enumerate the complete relevant corpus;
2. define the fingerprint schema and evidence locators;
3. compile and audit a representative batch;
4. test whether research-move compression preserves important distinctions;
5. run the result with the user before building UI.

For `zk-data-agent`:

1. port the two currently runnable task contracts;
2. extract the live runner and add one deterministic validator;
3. export valid result JSONL;
4. run a smoke comparison only after deterministic layers pass;
5. record the decision and failures, not just the score.