agent/docs/06-project-status.md

# Project Status

last_reviewed: 2026-07-28
baseline_before_update: `4c88faa`
branch: `main`
remote: `https://git.k1412.top/wuyang/agent.git`

## Purpose

The repository began as a durable Agent knowledge base and now has two layers:

1. a collection layer for papers, Beijing jobs, and company Agent work;
2. an experimental knowledge interface and research layer intended to turn the collection into evidence-backed understanding.

The long-term product is not “a place to browse papers.” It should help the user retain claims, connect them into a personal model, test that model, and use it to reason about unfamiliar Agent problems.

## Current Snapshot

The deterministic index currently contains 1,140 items:

| Collection | Count | State |
| --- | ---: | --- |
| papers | 1,125 | 1,110 queued, 15 skimmed |
| jobs | 6 | 4 active, 2 unknown |
| industry | 9 | 8 analyzed, 1 queued |

The 2026-07-08 expanded arXiv run saw 1,506 candidates, promoted 970 high-recall candidates,
and retained 536 reserve candidates. The 2026-07-27 incremental refresh covered July 9-27, saw
249 unique candidates, selected 157 high-relevance records, skipped 6 existing papers, and added
151 paper items. The newest submission returned was 2026-07-24. These are collection counts, not
quality or reading-completion counts.

The memory topic has two intentionally different counts:

- 336 indexed items now have the broad `memory` topic, including non-paper streams;
- 293 local paper items were resolved for the dedicated memory audit.

The memory research snapshot contains:

| Stage | Count |
| --- | ---: |
| paginated arXiv candidates | 516 |
| lexical title-central upper bound | 379 |
| recent-six-month title-central candidates | 292 |
| PDFs downloaded and text-located during the audit | 32 |
| papers promoted into the evidence ledger | 23 |

The strongest current conclusion is that recent progress is concentrated in better diagnosis/evaluation and task-specific execution-state memory, not a universal memory architecture. Read `research/memory/findings.md` before making memory design claims.

The Agent completion/verification research snapshot contains:

| Stage | Count |
| --- | ---: |
| full-text problem-driven pool | 40 |
| papers promoted into the evidence ledger | 37 |
| historical context only | 3 |

Its strongest conclusion is that Agent completion is an evidence contract over intended effects,
forbidden effects, required procedure, and remaining uncertainty. State checks, trace rules, learned
judges, and humans have different jobs; none is a universal oracle. Verification and recovery are
separate capabilities, and recovery must distinguish detection, localization, routing, restoration,
and re-verification. Read `research/completion-verification/findings.md` before extending the Agent
evaluation contract.

## Artifact Map

| Area | Source of truth | What it contains |
| --- | --- | --- |
| entry/navigation | `README.md` | human-facing repository map |
| AI continuation | `AGENTS.md` | read order, contracts, commands |
| stable concepts | `docs/00-05*.md`, `docs/99-glossary.md` | knowledge map, principles, architecture, practice, evaluation, collection |
| current state | `docs/06-project-status.md` | counts, runtime, artifact inventory |
| exploration history | `docs/07-exploration-history.md` | accepted and rejected routes |
| continuation plan | `docs/08-next-work.md` | priorities and definition of done |
| jobs | `jobs/` | Beijing-first rolling six-month JD stream |
| papers | `papers/` | paper metadata, queue, notes, corpus summary |
| industry | `industry/` | official reports, blogs, products, benchmarks |
| collection runs | `collection-runs/` | queries, scope, counts, method notes |
| collection tools | `tools/collection/` | item creation, arXiv collection, promotion, indexing, URL checks |
| research | `research/memory/`, `research/completion-verification/` | field findings and evidence ledgers |
| Agent evaluation | `research/evaluation/` | executable design contract for task suites, validators, paired comparisons, and release gates |
| evaluation data/tools | `data/evaluation/`, `tools/evaluation/` | versioned task contracts and JSONL result scorer |
| research document | `evaluation-site/` | completion research reading view, generated evidence index, and archived v1 evaluation report |
| research data | `data/research/` | committed corpus, screening, landscape and CSV ledger |
| research tools | `tools/research/` | paginated search, abstract fetch, screening, classification, landscape build |
| experiments | `experiments/knowledge-compilation/` | failed pilot and completed full-text research audits |
| learning prototype | `learning/agent-memory/` | preserved failed 12-paper learning module |
| web application | `web/` | Atlas, search agent, paper reader/chat, llama.cpp actions |
| project integrity | `tools/check_project.py` | offline structural and data checks |

`projects/` and `references/` are still mostly scaffolding. They do not yet contain substantive project or general-reference catalogs.

## Data Lineage

```text
official/web/arXiv sources
  -> jobs|papers|industry/items/*.md
  -> tools/collection/build_index.py
  -> data/index.json + data/summary.json
  -> web/app.py
  -> Atlas / search / paper reading UI
```

The dedicated memory audit is separate:

```text
paginated arXiv queries + local memory-tagged papers
  -> data/research/* corpus snapshots
  -> model-assisted triage + deterministic statistics
  -> selected full-text experiment checks
  -> research/memory/evidence-ledger.md
  -> research/memory/findings.md
```

Do not merge these two lines silently. The first is a broad browsing corpus; the second is an evidence audit with explicit boundaries.

The Agent completion audit follows the same evidence discipline without a model-assisted screening
stage:

```text
problem-driven arXiv candidates
  -> 40 downloaded and text-located full papers
  -> 37 audited experiment/ablation/limitation records
  -> research/completion-verification/evidence-ledger.md
  -> research/completion-verification/findings.md
```

The PDF and extracted-text working set was temporary. The committed evidence ledger preserves source
IDs, key numbers, claim boundaries, and exclusions so another researcher can refetch the sources.

The Agent evaluation design follows a third line:

```text
zk-data-agent runtime/events + Zero decision-loop constraints
  -> versioned task contracts
  -> isolated repeated baseline/variant runs
  -> independent validators + trace rules
  -> JSONL result records
  -> gain/regression/safety decision
```

Only the task contracts and scorer exist in this repository. The reusable runner, fixtures, validators,
and real-model comparison still need to be implemented in `zk-data-agent`. That implementation is
intentionally paused while the research findings are reviewed. Before implementation, task contracts
need separate outcome/invariant/procedure/semantic-residual layers, non-binary
`ambiguous/unverifiable` states, and explicit recovery-stage fields.

`evaluation-site/` now uses the completion/verification audit as its homepage. The old v1 contract
visualization remains at `/evaluation-v1.html`; its adjustable decision values are still explicitly
labeled as a method simulation, not a real benchmark.

## Web Application

Capabilities currently implemented:

- topic Atlas with ECharts and SVG fallback;
- routes, timeline, topic focus and paper filtering;
- desktop/mobile responsive reader mode;
- local keyword search plus an Agent search loop;
- Chinese query interpretation and English keyword expansion;
- asynchronous search activity, aggregate statistics and AI brief;
- follow-up questions on a search result;
- per-paper abstract fetch, summary, translation, deep analysis and chat;
- single-concurrency llama.cpp access and local caches.

Run it with:

```bash
web/manage.sh start
web/manage.sh status
```

Expected addresses:

```text
local/Tailscale: http://100.114.68.27:18080
public mapping:  https://lab.k1412.top/
health:          /api/health
```

At the beginning of the 2026-07-12 handoff audit, no process was listening on 18080 and the public domain returned 502. The handoff added and enabled a user-level systemd service through `web/manage.sh`;
local and public health then returned 200. Runtime status is external state and must always be checked
live. After a corpus update, restart the service because the index is loaded at process startup.
On 2026-07-27 both the Tailscale endpoint and `https://lab.k1412.top/api/health` returned the refreshed
1,125-paper count. After the 2026-07-28 migration, readiness is reported as `model_api_ready`.

On 2026-07-28, inference moved from Ollama to two resident llama.cpp servers. Luna runs as Qwen3 0.6B Q8 on the NAS Radeon 780M through Vulkan. Terra and Sol share one Qwen3.6 27B Q4_K_M runner on the V100 through CUDA 12, preserving three public tiers without loading a second large model. The web application calls the two internal OpenAI-compatible endpoints directly.

The static Agent completion research document is packaged separately:

```text
source:          evaluation-site/
public mapping:  https://agent-eval.k1412.top/
container port:  8080
health:          /health
deployment:      Unraid Compose Manager, app slug agent-eval
archived v1:     /evaluation-v1.html
```

It uses no Ollama or runtime API. The homepage presents the eight completion/verification/recovery
conclusions, capability decomposition, evidence/recovery chains, 2023-2026 progression, the explicitly
unvalidated system synthesis, unresolved questions, and a searchable/filterable 40-paper source pool
containing 37 audited records.

The evidence browser is deterministically generated from
`research/completion-verification/evidence-ledger.md` by
`evaluation-site/build_research_data.py`. `tools/check_project.py` verifies the source hash and the
40/37/26 pool, audited, and core counts.

The 2026-07-28 deployment uses immutable image
`docker.k1412.top/wuyang/agent-eval:20260727T165727Z-4c88faa` on NAS host port 12005. Container health,
Compose Manager ownership, NAS/VPS Tailscale reachability, HTTP-to-HTTPS redirect, HTTP/2, Let's Encrypt
SAN, static-asset hashes, public desktop/mobile rendering, evidence filters, and the archived page were
verified. The existing `AGENT EVAL` homepage entry remains under `projects` at order 130.

## Runtime Constraints

- Python standard library only; no package installation is required for the current web server and collection scripts.
- llama.cpp endpoints default to `http://192.168.1.10:11435/v1` for Luna and `http://192.168.1.10:11436/v1` for Terra/Sol.
- Luna uses the NAS AMD iGPU; one V100 32G serves the shared Terra/Sol runner. AI calls are serialized.
- Context is fixed server-side at 8K for Luna and 32K for Terra/Sol.
- `Sol` is the expensive tier and should require an explicit user action.
- The user mentioned an unlimited company GPT-5.5 data-agent API as a possible stronger offline worker, but no endpoint/authentication contract is committed. It is not currently integrated.
- The public site has no repository-managed authentication or TLS configuration; the domain mapping is external infrastructure.
- The committed systemd unit assumes the checkout path `%h/Code/agent`; edit the unit if a future checkout uses a different path.
- Full PDFs and extracted text used during the memory audit were temporary working files and are not committed. Their arXiv IDs and claim boundaries are preserved in `research/memory/evidence-ledger.md` so they can be fetched again.
- `web/cache/` contents, including arXiv responses, AI outputs, service environment and logs, are intentionally not committed.

## Validation

Offline validation:

```bash
python3 evaluation-site/build_research_data.py --check
python3 tools/check_project.py
python3 -m unittest tools.evaluation.test_score_agent_runs
git diff --check
```

Live validation:

```bash
web/manage.sh status
curl -fsS http://100.114.68.27:18080/api/health | jq
curl -fsS https://lab.k1412.top/api/health | jq
curl -fsS https://agent-eval.k1412.top/health
```

The health payload reports app/assets versions, paper count, available/default models, and missing default model tags.