Record completion research site deployment
This commit is contained in:
wuyang
+23
-12
@@ -1,7 +1,7 @@
|
||||
# Project Status
|
||||
|
||||
last_reviewed: 2026-07-27
|
||||
baseline_before_update: `5373d56`
|
||||
last_reviewed: 2026-07-28
|
||||
baseline_before_update: `4c88faa`
|
||||
branch: `main`
|
||||
remote: `https://git.k1412.top/wuyang/agent.git`
|
||||
|
||||
@@ -80,7 +80,7 @@ evaluation contract.
|
||||
| research | `research/memory/`, `research/completion-verification/` | field findings and evidence ledgers |
|
||||
| Agent evaluation | `research/evaluation/` | executable design contract for task suites, validators, paired comparisons, and release gates |
|
||||
| evaluation data/tools | `data/evaluation/`, `tools/evaluation/` | versioned task contracts and JSONL result scorer |
|
||||
| evaluation report | `evaluation-site/` | Chinese interactive explanation of the contract, decision gates, tasks, and artifacts |
|
||||
| research document | `evaluation-site/` | completion research reading view, generated evidence index, and archived v1 evaluation report |
|
||||
| research data | `data/research/` | committed corpus, screening, landscape and CSV ledger |
|
||||
| research tools | `tools/research/` | paginated search, abstract fetch, screening, classification, landscape build |
|
||||
| experiments | `experiments/knowledge-compilation/` | failed pilot and completed full-text research audits |
|
||||
@@ -145,8 +145,9 @@ intentionally paused while the research findings are reviewed. Before implementa
|
||||
need separate outcome/invariant/procedure/semantic-residual layers, non-binary
|
||||
`ambiguous/unverifiable` states, and explicit recovery-stage fields.
|
||||
|
||||
`evaluation-site/` visualizes this contract but does not add evidence. Its adjustable decision values are
|
||||
explicitly labeled as a method simulation, not a real benchmark.
|
||||
`evaluation-site/` now uses the completion/verification audit as its homepage. The old v1 contract
|
||||
visualization remains at `/evaluation-v1.html`; its adjustable decision values are still explicitly
|
||||
labeled as a method simulation, not a real benchmark.
|
||||
|
||||
## Web Application
|
||||
|
||||
@@ -185,7 +186,7 @@ On 2026-07-27 both the Tailscale endpoint and `https://lab.k1412.top/api/health`
|
||||
|
||||
The live Ollama inventory checked on 2026-07-12 included `ChatGPT-5.6:Luna`, `Terra`, `Sol`, `auto`, `embed`, and `gpt-4o:latest`. The web defaults were updated from stale `light/fast/large` tags to configurable Luna/Terra/Sol tiers.
|
||||
|
||||
The static Agent evaluation report is packaged separately:
|
||||
The static Agent completion research document is packaged separately:
|
||||
|
||||
```text
|
||||
source: evaluation-site/
|
||||
@@ -193,15 +194,24 @@ public mapping: https://agent-eval.k1412.top/
|
||||
container port: 8080
|
||||
health: /health
|
||||
deployment: Unraid Compose Manager, app slug agent-eval
|
||||
archived v1: /evaluation-v1.html
|
||||
```
|
||||
|
||||
It uses no Ollama or runtime API. The source report remains usable even when the research application is
|
||||
offline.
|
||||
It uses no Ollama or runtime API. The homepage presents the eight completion/verification/recovery
|
||||
conclusions, capability decomposition, evidence/recovery chains, 2023-2026 progression, the explicitly
|
||||
unvalidated system synthesis, unresolved questions, and a searchable/filterable 40-paper source pool
|
||||
containing 37 audited records.
|
||||
|
||||
The 2026-07-27 deployment uses immutable image
|
||||
`docker.k1412.top/wuyang/agent-eval:20260727T084425Z-cd1eacb` on NAS host port 12005. Container health,
|
||||
NAS/VPS Tailscale reachability, HTTP-to-HTTPS redirect, HTTP/2, Let's Encrypt SAN, static-asset hashes,
|
||||
public desktop/mobile rendering, and the homepage entry were all verified.
|
||||
The evidence browser is deterministically generated from
|
||||
`research/completion-verification/evidence-ledger.md` by
|
||||
`evaluation-site/build_research_data.py`. `tools/check_project.py` verifies the source hash and the
|
||||
40/37/26 pool, audited, and core counts.
|
||||
|
||||
The 2026-07-28 deployment uses immutable image
|
||||
`docker.k1412.top/wuyang/agent-eval:20260727T165727Z-4c88faa` on NAS host port 12005. Container health,
|
||||
Compose Manager ownership, NAS/VPS Tailscale reachability, HTTP-to-HTTPS redirect, HTTP/2, Let's Encrypt
|
||||
SAN, static-asset hashes, public desktop/mobile rendering, evidence filters, and the archived page were
|
||||
verified. The existing `AGENT EVAL` homepage entry remains under `projects` at order 130.
|
||||
|
||||
## Runtime Constraints
|
||||
|
||||
@@ -221,6 +231,7 @@ public desktop/mobile rendering, and the homepage entry were all verified.
|
||||
Offline validation:
|
||||
|
||||
```bash
|
||||
python3 evaluation-site/build_research_data.py --check
|
||||
python3 tools/check_project.py
|
||||
python3 -m unittest tools.evaluation.test_score_agent_runs
|
||||
git diff --check
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Exploration History
|
||||
|
||||
last_reviewed: 2026-07-27
|
||||
last_reviewed: 2026-07-28
|
||||
|
||||
This is the decision history, including failed routes. Preserve it so future work does not rediscover the same lessons.
|
||||
|
||||
@@ -298,3 +298,30 @@ Current source of truth:
|
||||
- `research/completion-verification/findings.md`
|
||||
- `research/completion-verification/evidence-ledger.md`
|
||||
- `experiments/knowledge-compilation/2026-07-27-agent-completion-research-audit.md`
|
||||
|
||||
## 14. Research Document, Not a New Product Shape
|
||||
|
||||
The user asked to read the completion research as a web page because the Markdown artifacts were
|
||||
harder to traverse. This did not reopen the larger website/product-design question. The implementation
|
||||
therefore treats the page as a document rendering of committed research:
|
||||
|
||||
- the first view states the conclusion and evidence boundary;
|
||||
- eight conclusions lead into capability decomposition and two distinct chains for closure and recovery;
|
||||
- each argument exposes its direct numbers and non-extrapolation boundary progressively;
|
||||
- the 2023-2026 timeline separates measurement progress from unsolved general capability;
|
||||
- the completion certificate, failure object, and recovery router remain labeled as system inference;
|
||||
- all 40 source papers are searchable, with 37 audited and 3 context-only records;
|
||||
- evidence data is generated from the Markdown ledger and checked by source hash.
|
||||
|
||||
The former v1 evaluation explanation page remains at `/evaluation-v1.html`. It was not deleted or
|
||||
silently rewritten.
|
||||
|
||||
Desktop 1440×1000 and mobile 390×844 Chrome checks found no horizontal overflow or console/page
|
||||
errors. Core filtering returned 26 records; recovery filtering returned 12; public evidence search and
|
||||
expansion worked. The production image
|
||||
`docker.k1412.top/wuyang/agent-eval:20260727T165727Z-4c88faa` was deployed on the existing NAS port
|
||||
12005 and existing HTTPS route.
|
||||
|
||||
Decision retained: use `https://agent-eval.k1412.top/` to challenge the research with the user.
|
||||
Do not interpret successful document presentation as validation of the proposed evaluation system or
|
||||
as permission to resume runner implementation.
|
||||
|
||||
+10
-5
@@ -1,12 +1,13 @@
|
||||
# Next Work
|
||||
|
||||
last_reviewed: 2026-07-27
|
||||
last_reviewed: 2026-07-28
|
||||
|
||||
## P0: Challenge the Completion Research
|
||||
|
||||
The immediate work is not an Agent experiment and not a new evaluation page. Review
|
||||
`research/completion-verification/findings.md` with the user and determine whether its problem
|
||||
decomposition is actually useful.
|
||||
The immediate work is not an Agent experiment or a product redesign. Review the document at
|
||||
`https://agent-eval.k1412.top/` with the user and determine whether the problem decomposition is
|
||||
actually useful. The Markdown sources remain
|
||||
`research/completion-verification/findings.md` and `evidence-ledger.md`.
|
||||
|
||||
The review should test whether the research can answer, without falling back to paper titles:
|
||||
|
||||
@@ -24,6 +25,10 @@ does not prove.
|
||||
Only after this review should the candidate completion certificate, failure object, and recovery router
|
||||
be translated into task contracts. They are currently system inferences, not validated implementations.
|
||||
|
||||
The old v1 evaluation explanation remains available at
|
||||
`https://agent-eval.k1412.top/evaluation-v1.html`. Use it only to compare the premature contract-first
|
||||
route with the current evidence-first research, not as a benchmark result.
|
||||
|
||||
## P1: Prove the Paper Compiler
|
||||
|
||||
The main project goal is now a high-efficiency knowledge funnel, not a paper browser and not a fixed
|
||||
@@ -222,7 +227,7 @@ Still missing:
|
||||
|
||||
For the knowledge-base repository:
|
||||
|
||||
1. read `research/completion-verification/findings.md` with the user;
|
||||
1. read `https://agent-eval.k1412.top/` with the user;
|
||||
2. collect objections, ambiguous terms and missing counterexamples;
|
||||
3. tighten or reject conclusions against the evidence ledger;
|
||||
4. only then convert the accepted model into a minimal task-contract schema;
|
||||
|
||||
Reference in New Issue
Block a user