wuyang/agent
2
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Audit Agent completion and recovery research

Browse Source
This commit is contained in:
wuyang
2026-07-28 00:06:57 +08:00
parent 5373d564ec
commit 3b55bfaad2
12 changed files with 840 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/06-project-status.md
+36 -4
View File
@@ -1,7 +1,7 @@
# Project Status
last_reviewed: 2026-07-27
baseline_before_update: `8e4ff37`
baseline_before_update: `5373d56`
branch: `main`
remote: `https://git.k1412.top/wuyang/agent.git`
@@ -47,6 +47,21 @@ The memory research snapshot contains:
The strongest current conclusion is that recent progress is concentrated in better diagnosis/evaluation and task-specific execution-state memory, not a universal memory architecture. Read `research/memory/findings.md` before making memory design claims.
The Agent completion/verification research snapshot contains:
| Stage | Count |
| --- | ---: |
| full-text problem-driven pool | 40 |
| papers promoted into the evidence ledger | 37 |
| historical context only | 3 |
Its strongest conclusion is that Agent completion is an evidence contract over intended effects,
forbidden effects, required procedure, and remaining uncertainty. State checks, trace rules, learned
judges, and humans have different jobs; none is a universal oracle. Verification and recovery are
separate capabilities, and recovery must distinguish detection, localization, routing, restoration,
and re-verification. Read `research/completion-verification/findings.md` before extending the Agent
evaluation contract.
## Artifact Map
| Area | Source of truth | What it contains |
@@ -62,13 +77,13 @@ The strongest current conclusion is that recent progress is concentrated in bett
| industry | `industry/` | official reports, blogs, products, benchmarks |
| collection runs | `collection-runs/` | queries, scope, counts, method notes |
| collection tools | `tools/collection/` | item creation, arXiv collection, promotion, indexing, URL checks |
| research | `research/memory/` | field findings and evidence ledger |
| research | `research/memory/`, `research/completion-verification/` | field findings and evidence ledgers |
| Agent evaluation | `research/evaluation/` | executable design contract for task suites, validators, paired comparisons, and release gates |
| evaluation data/tools | `data/evaluation/`, `tools/evaluation/` | versioned task contracts and JSONL result scorer |
| evaluation report | `evaluation-site/` | Chinese interactive explanation of the contract, decision gates, tasks, and artifacts |
| research data | `data/research/` | committed corpus, screening, landscape and CSV ledger |
| research tools | `tools/research/` | paginated search, abstract fetch, screening, classification, landscape build |
| experiments | `experiments/knowledge-compilation/` | failed pilot and completed field audit |
| experiments | `experiments/knowledge-compilation/` | failed pilot and completed full-text research audits |
| learning prototype | `learning/agent-memory/` | preserved failed 12-paper learning module |
| web application | `web/` | Atlas, search agent, paper reader/chat, Ollama actions |
| project integrity | `tools/check_project.py` | offline structural and data checks |
@@ -99,6 +114,20 @@ paginated arXiv queries + local memory-tagged papers
Do not merge these two lines silently. The first is a broad browsing corpus; the second is an evidence audit with explicit boundaries.
The Agent completion audit follows the same evidence discipline without a model-assisted screening
stage:
```text
problem-driven arXiv candidates
-> 40 downloaded and text-located full papers
-> 37 audited experiment/ablation/limitation records
-> research/completion-verification/evidence-ledger.md
-> research/completion-verification/findings.md
```
The PDF and extracted-text working set was temporary. The committed evidence ledger preserves source
IDs, key numbers, claim boundaries, and exclusions so another researcher can refetch the sources.
The Agent evaluation design follows a third line:
```text
@@ -111,7 +140,10 @@ zk-data-agent runtime/events + Zero decision-loop constraints
```
Only the task contracts and scorer exist in this repository. The reusable runner, fixtures, validators,
and real-model comparison still need to be implemented in `zk-data-agent`.
and real-model comparison still need to be implemented in `zk-data-agent`. That implementation is
intentionally paused while the research findings are reviewed. Before implementation, task contracts
need separate outcome/invariant/procedure/semantic-residual layers, non-binary
`ambiguous/unverifiable` states, and explicit recovery-stage fields.
`evaluation-site/` visualizes this contract but does not add evidence. Its adjustable decision values are
explicitly labeled as a method simulation, not a real benchmark.