diff --git a/AGENTS.md b/AGENTS.md index 4c4d792..365f821 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -42,8 +42,9 @@ The current Atlas is a corpus exploration prototype, not a validated solution to The current immediate research question is Agent completion, verification, and recovery. Read `research/completion-verification/findings.md` and `evidence-ledger.md` before extending the Agent -evaluation work. Experiments and web redesign are intentionally paused until the user challenges and -accepts or revises those conclusions. +evaluation work. A document-style web rendering now exists at `https://agent-eval.k1412.top/`; +the old v1 evaluation report is archived at `/evaluation-v1.html`. Agent experiments and product-level +web redesign remain paused until the user challenges and accepts or revises the conclusions. ## Truth Hierarchy @@ -120,7 +121,11 @@ python3 tools/collection/collect_arxiv.py --backend search-html --help # Aggregate Agent evaluation result JSONL python3 tools/evaluation/score_agent_runs.py --help -# Preview the static Agent evaluation report +# Regenerate/check the completion research evidence browser +python3 evaluation-site/build_research_data.py +python3 evaluation-site/build_research_data.py --check + +# Preview the research document and archived v1 report python3 -m http.server 4173 --directory evaluation-site ``` diff --git a/README.md b/README.md index ab7b072..99023d9 100644 --- a/README.md +++ b/README.md @@ -40,8 +40,10 @@ evaluator 如何误判以及失败后如何恢复，当前基线是 完成合同、验证器边界、失败定位、恢复路由和停止条件 - [Agent Evaluation v1](research/evaluation/k1412-agent-evaluation-v1.md): 面向 `zk-data-agent` 的任务契约、独立判分、版本对照和发布门禁 -- [Agent Evaluation Report](https://agent-eval.k1412.top/): 中文交互式结论页、发布决策演算器、 - 评测层次、任务契约和产物入口 +- [Agent Completion Research Document](https://agent-eval.k1412.top/): 中文研究正文、完成与恢复 + 模型、发展脉络和 37 篇全文证据索引 +- [Archived Agent Evaluation v1](https://agent-eval.k1412.top/evaluation-v1.html): 旧任务契约与 + 发布决策演算器，仅作历史设计参考 - [Learning](learning/README.md): 从论文编译出的认知地图、讲义、证据和主动回忆模块 - [Industry](industry/README.md): 大公司 Agent 工作、技术报告和工程博客 - [Projects](projects/README.md): 项目目录、项目记录和复盘 diff --git a/docs/06-project-status.md b/docs/06-project-status.md index fae655b..e668940 100644 --- a/docs/06-project-status.md +++ b/docs/06-project-status.md @@ -1,7 +1,7 @@ # Project Status -last_reviewed: 2026-07-27 -baseline_before_update: `5373d56` +last_reviewed: 2026-07-28 +baseline_before_update: `4c88faa` branch: `main` remote: `https://git.k1412.top/wuyang/agent.git` @@ -80,7 +80,7 @@ evaluation contract. | research | `research/memory/`, `research/completion-verification/` | field findings and evidence ledgers | | Agent evaluation | `research/evaluation/` | executable design contract for task suites, validators, paired comparisons, and release gates | | evaluation data/tools | `data/evaluation/`, `tools/evaluation/` | versioned task contracts and JSONL result scorer | -| evaluation report | `evaluation-site/` | Chinese interactive explanation of the contract, decision gates, tasks, and artifacts | +| research document | `evaluation-site/` | completion research reading view, generated evidence index, and archived v1 evaluation report | | research data | `data/research/` | committed corpus, screening, landscape and CSV ledger | | research tools | `tools/research/` | paginated search, abstract fetch, screening, classification, landscape build | | experiments | `experiments/knowledge-compilation/` | failed pilot and completed full-text research audits | @@ -145,8 +145,9 @@ intentionally paused while the research findings are reviewed. Before implementa need separate outcome/invariant/procedure/semantic-residual layers, non-binary `ambiguous/unverifiable` states, and explicit recovery-stage fields. -`evaluation-site/` visualizes this contract but does not add evidence. Its adjustable decision values are -explicitly labeled as a method simulation, not a real benchmark. +`evaluation-site/` now uses the completion/verification audit as its homepage. The old v1 contract +visualization remains at `/evaluation-v1.html`; its adjustable decision values are still explicitly +labeled as a method simulation, not a real benchmark. ## Web Application @@ -185,7 +186,7 @@ On 2026-07-27 both the Tailscale endpoint and `https://lab.k1412.top/api/health` The live Ollama inventory checked on 2026-07-12 included `ChatGPT-5.6:Luna`, `Terra`, `Sol`, `auto`, `embed`, and `gpt-4o:latest`. The web defaults were updated from stale `light/fast/large` tags to configurable Luna/Terra/Sol tiers. -The static Agent evaluation report is packaged separately: +The static Agent completion research document is packaged separately: ```text source: evaluation-site/ @@ -193,15 +194,24 @@ public mapping: https://agent-eval.k1412.top/ container port: 8080 health: /health deployment: Unraid Compose Manager, app slug agent-eval +archived v1: /evaluation-v1.html ``` -It uses no Ollama or runtime API. The source report remains usable even when the research application is -offline. +It uses no Ollama or runtime API. The homepage presents the eight completion/verification/recovery +conclusions, capability decomposition, evidence/recovery chains, 2023-2026 progression, the explicitly +unvalidated system synthesis, unresolved questions, and a searchable/filterable 40-paper source pool +containing 37 audited records. -The 2026-07-27 deployment uses immutable image -`docker.k1412.top/wuyang/agent-eval:20260727T084425Z-cd1eacb` on NAS host port 12005. Container health, -NAS/VPS Tailscale reachability, HTTP-to-HTTPS redirect, HTTP/2, Let's Encrypt SAN, static-asset hashes, -public desktop/mobile rendering, and the homepage entry were all verified. +The evidence browser is deterministically generated from +`research/completion-verification/evidence-ledger.md` by +`evaluation-site/build_research_data.py`. `tools/check_project.py` verifies the source hash and the +40/37/26 pool, audited, and core counts. + +The 2026-07-28 deployment uses immutable image +`docker.k1412.top/wuyang/agent-eval:20260727T165727Z-4c88faa` on NAS host port 12005. Container health, +Compose Manager ownership, NAS/VPS Tailscale reachability, HTTP-to-HTTPS redirect, HTTP/2, Let's Encrypt +SAN, static-asset hashes, public desktop/mobile rendering, evidence filters, and the archived page were +verified. The existing `AGENT EVAL` homepage entry remains under `projects` at order 130. ## Runtime Constraints @@ -221,6 +231,7 @@ public desktop/mobile rendering, and the homepage entry were all verified. Offline validation: ```bash +python3 evaluation-site/build_research_data.py --check python3 tools/check_project.py python3 -m unittest tools.evaluation.test_score_agent_runs git diff --check diff --git a/docs/07-exploration-history.md b/docs/07-exploration-history.md index be57c53..b37e610 100644 --- a/docs/07-exploration-history.md +++ b/docs/07-exploration-history.md @@ -1,6 +1,6 @@ # Exploration History -last_reviewed: 2026-07-27 +last_reviewed: 2026-07-28 This is the decision history, including failed routes. Preserve it so future work does not rediscover the same lessons. @@ -298,3 +298,30 @@ Current source of truth: - `research/completion-verification/findings.md` - `research/completion-verification/evidence-ledger.md` - `experiments/knowledge-compilation/2026-07-27-agent-completion-research-audit.md` + +## 14. Research Document, Not a New Product Shape + +The user asked to read the completion research as a web page because the Markdown artifacts were +harder to traverse. This did not reopen the larger website/product-design question. The implementation +therefore treats the page as a document rendering of committed research: + +- the first view states the conclusion and evidence boundary; +- eight conclusions lead into capability decomposition and two distinct chains for closure and recovery; +- each argument exposes its direct numbers and non-extrapolation boundary progressively; +- the 2023-2026 timeline separates measurement progress from unsolved general capability; +- the completion certificate, failure object, and recovery router remain labeled as system inference; +- all 40 source papers are searchable, with 37 audited and 3 context-only records; +- evidence data is generated from the Markdown ledger and checked by source hash. + +The former v1 evaluation explanation page remains at `/evaluation-v1.html`. It was not deleted or +silently rewritten. + +Desktop 1440×1000 and mobile 390×844 Chrome checks found no horizontal overflow or console/page +errors. Core filtering returned 26 records; recovery filtering returned 12; public evidence search and +expansion worked. The production image +`docker.k1412.top/wuyang/agent-eval:20260727T165727Z-4c88faa` was deployed on the existing NAS port +12005 and existing HTTPS route. + +Decision retained: use `https://agent-eval.k1412.top/` to challenge the research with the user. +Do not interpret successful document presentation as validation of the proposed evaluation system or +as permission to resume runner implementation. diff --git a/docs/08-next-work.md b/docs/08-next-work.md index 27a54a2..0baf3ad 100644 --- a/docs/08-next-work.md +++ b/docs/08-next-work.md @@ -1,12 +1,13 @@ # Next Work -last_reviewed: 2026-07-27 +last_reviewed: 2026-07-28 ## P0: Challenge the Completion Research -The immediate work is not an Agent experiment and not a new evaluation page. Review -`research/completion-verification/findings.md` with the user and determine whether its problem -decomposition is actually useful. +The immediate work is not an Agent experiment or a product redesign. Review the document at +`https://agent-eval.k1412.top/` with the user and determine whether the problem decomposition is +actually useful. The Markdown sources remain +`research/completion-verification/findings.md` and `evidence-ledger.md`. The review should test whether the research can answer, without falling back to paper titles: @@ -24,6 +25,10 @@ does not prove. Only after this review should the candidate completion certificate, failure object, and recovery router be translated into task contracts. They are currently system inferences, not validated implementations. +The old v1 evaluation explanation remains available at +`https://agent-eval.k1412.top/evaluation-v1.html`. Use it only to compare the premature contract-first +route with the current evidence-first research, not as a benchmark result. + ## P1: Prove the Paper Compiler The main project goal is now a high-efficiency knowledge funnel, not a paper browser and not a fixed @@ -222,7 +227,7 @@ Still missing: For the knowledge-base repository: -1. read `research/completion-verification/findings.md` with the user; +1. read `https://agent-eval.k1412.top/` with the user; 2. collect objections, ambiguous terms and missing counterexamples; 3. tighten or reject conclusions against the evidence ledger; 4. only then convert the accepted model into a minimal task-contract schema; diff --git a/evaluation-site/README.md b/evaluation-site/README.md index f815cff..4520bcc 100644 --- a/evaluation-site/README.md +++ b/evaluation-site/README.md @@ -47,5 +47,5 @@ archived v1 report: http://127.0.0.1:4173/evaluation-v1.html - public URL: `https://agent-eval.k1412.top/` - deployment: k1412 Unraid Compose Manager - NAS host port: `12005` - -生产部署必须使用不可变镜像，并在这里和项目状态文档中记录最终 tag。 +- immutable image: + `docker.k1412.top/wuyang/agent-eval:20260727T165727Z-4c88faa` diff --git a/experiments/README.md b/experiments/README.md index 3e60f23..c87a617 100644 --- a/experiments/README.md +++ b/experiments/README.md @@ -12,6 +12,7 @@ | OPS-001 | 完整仓库交接可以让无会话上下文的 AI 继续工作 | completed | 增加状态、历史、下一步、完整性检查、模型校验和 systemd 服务管理 | [记录](operations/2026-07-12-project-handoff-audit.md) | | EVAL-001 | 现有运行事件和隔离工作区可扩展为可决策的 Agent 评测体系 | completed | 形成任务契约、分层评测、独立判分、gain/regression 对照和评分器；尚未运行真实模型 | [记录](agent-evaluation/2026-07-27-k1412-agent-evaluation-design.md) | | OPS-002 | 评估结论可以用交互报告展示而不伪装成真实跑分 | completed | 中文报告、决策演算器、任务筛选、响应式和镜像验证通过，已部署 HTTPS 并注册首页 | [记录](operations/2026-07-27-agent-eval-site.md) | +| OPS-003 | 全文研究产物可以先作为网页文档阅读，而不提前设计知识产品 | completed | 完成研究正文、双链模型、37 篇证据索引和旧报告归档，并更新原 HTTPS 部署 | [记录](operations/2026-07-28-completion-research-document.md) | ## Experiment Rules diff --git a/experiments/operations/2026-07-28-completion-research-document.md b/experiments/operations/2026-07-28-completion-research-document.md new file mode 100644 index 0000000..4b24d7c --- /dev/null +++ b/experiments/operations/2026-07-28-completion-research-document.md @@ -0,0 +1,110 @@ +# 实验：Agent 完成研究文档网页 + +status: completed +date: 2026-07-28 +owner: Codex + user + +## 问题 + +能否把 `research/completion-verification/` 的研究结论和证据账本做成更容易连续阅读和深入核查 +的网页，同时不提前决定未来知识产品的网站形态？ + +## 边界 + +- 这是文档呈现，不是新的 Agent 产品。 +- 不运行 Agent benchmark，不生成新的实验数字。 +- 不让网页结论强于 Markdown 事实源。 +- 旧的 v1 评测说明页必须保留。 +- 系统推论必须继续标记为未验证。 + +## 实现 + +首页按阅读顺序组织： + +```text +核心判断 +-> 八条结论 +-> 具体能力拆解 +-> 完成证明 / 失败恢复双链 +-> 八条论证与边界 +-> 2023-2026 进展 +-> 未验证系统推论 +-> 37 篇证据账本 +-> 未知问题和可反驳预测 +``` + +证据浏览器支持： + +- 中文/英文/数字全文搜索； +- `core / support / context` 级别筛选； +- 完成证明、协议审计、失败恢复、风险可靠性主题筛选； +- 单条或当前结果批量展开； +- 每篇直接打开 arXiv 原文。 + +`evaluation-site/build_research_data.py` 从 Markdown 账本生成浏览器数据。 +`tools/check_project.py` 检查源文件 SHA-256，以及： + +```text +full-text pool: 40 +audited: 37 +core: 26 +support: 11 +context: 3 +``` + +旧页面保留在 `/evaluation-v1.html`。 + +## 本地验证 + +真实 Google Chrome： + +```text +desktop viewport: 1440 x 1000 +mobile viewport: 390 x 844 +desktop overflow: 0 +mobile overflow: 0 +console/page errors: 0 +rendered evidence records: 40 +core filter records: 26 +search "恢复" within core: 6 +batch expand: 6/6 +archived page: HTTP 200 +``` + +本地生产镜像： + +```text +base: nginx:1.29.5-alpine +container port: 8080 +health: healthy +root/archive/assets: HTTP 200 +``` + +## 公开部署 + +```text +app slug: agent-eval +immutable image: docker.k1412.top/wuyang/agent-eval:20260727T165727Z-4c88faa +NAS mapping: 12005 -> 8080 +public URL: https://agent-eval.k1412.top/ +archive URL: https://agent-eval.k1412.top/evaluation-v1.html +health: /health +``` + +已验证： + +- Unraid Compose Manager 管理标签与容器健康； +- VPS 到 NAS Tailscale 返回 200； +- HTTP 到 HTTPS 返回 301； +- HTTPS 使用 HTTP/2 并返回 200； +- Let's Encrypt SAN 为 `agent-eval.k1412.top`； +- 三个新静态资源的公网 SHA-256 与仓库一致； +- 公网页面桌面/手机无横向溢出和浏览器错误； +- 公网 recovery 主题为 12 条，进一步搜索“根因”为 2 条，展开 2/2； +- 旧报告返回 200； +- 首页条目保持 `AGENT EVAL / projects / order 130`。 + +## 当前判断 + +网页解决了“如何阅读本轮产物”，没有解决“未来知识系统应该是什么产品形态”。下一步仍是让 +用户使用该文档质疑结论、补反例和收窄边界，而不是继续增加交互或恢复评测实现。 diff --git a/research/completion-verification/README.md b/research/completion-verification/README.md index aba1003..8ba8051 100644 --- a/research/completion-verification/README.md +++ b/research/completion-verification/README.md @@ -13,6 +13,11 @@ 3. [研究过程记录](../../experiments/knowledge-compilation/2026-07-27-agent-completion-research-audit.md)： 候选池、审计方法、排除项和本轮没有执行的工作。 +网页阅读版： + +- [Agent completion research document](https://agent-eval.k1412.top/) +- [Archived Agent Evaluation v1](https://agent-eval.k1412.top/evaluation-v1.html) + ## Boundary - 研究结论是当前的证据基线。 diff --git a/research/evaluation/README.md b/research/evaluation/README.md index 1509004..046e267 100644 --- a/research/evaluation/README.md +++ b/research/evaluation/README.md @@ -22,3 +22,6 @@ re-verification 分开记录。 这些是研究约束，不表示 runner、validator 或真实 benchmark 已经完成。 + +旧的 v1 交互说明页保留在 +[Archived Agent Evaluation v1](https://agent-eval.k1412.top/evaluation-v1.html)，不再作为站点首页。