agent/docs/06-project-status.md

Project Status

last_reviewed: 2026-07-27 baseline_before_update: 5373d56 branch: main remote: https://git.k1412.top/wuyang/agent.git

Purpose

The repository began as a durable Agent knowledge base and now has two layers:

1. a collection layer for papers, Beijing jobs, and company Agent work;
2. an experimental knowledge interface and research layer intended to turn the collection into evidence-backed understanding.

The long-term product is not “a place to browse papers.” It should help the user retain claims, connect them into a personal model, test that model, and use it to reason about unfamiliar Agent problems.

Current Snapshot

The deterministic index currently contains 1,140 items:

Collection	Count	State
papers	1,125	1,110 queued, 15 skimmed
jobs	6	4 active, 2 unknown
industry	9	8 analyzed, 1 queued

The 2026-07-08 expanded arXiv run saw 1,506 candidates, promoted 970 high-recall candidates, and retained 536 reserve candidates. The 2026-07-27 incremental refresh covered July 9-27, saw 249 unique candidates, selected 157 high-relevance records, skipped 6 existing papers, and added 151 paper items. The newest submission returned was 2026-07-24. These are collection counts, not quality or reading-completion counts.

The memory topic has two intentionally different counts:

• 336 indexed items now have the broad memory topic, including non-paper streams;
• 293 local paper items were resolved for the dedicated memory audit.

The memory research snapshot contains:

Stage	Count
paginated arXiv candidates	516
lexical title-central upper bound	379
recent-six-month title-central candidates	292
PDFs downloaded and text-located during the audit	32
papers promoted into the evidence ledger	23

The strongest current conclusion is that recent progress is concentrated in better diagnosis/evaluation and task-specific execution-state memory, not a universal memory architecture. Read research/memory/findings.md before making memory design claims.

The Agent completion/verification research snapshot contains:

Stage	Count
full-text problem-driven pool	40
papers promoted into the evidence ledger	37
historical context only	3

Its strongest conclusion is that Agent completion is an evidence contract over intended effects, forbidden effects, required procedure, and remaining uncertainty. State checks, trace rules, learned judges, and humans have different jobs; none is a universal oracle. Verification and recovery are separate capabilities, and recovery must distinguish detection, localization, routing, restoration, and re-verification. Read research/completion-verification/findings.md before extending the Agent evaluation contract.

Artifact Map

Area	Source of truth	What it contains
entry/navigation	README.md	human-facing repository map
AI continuation	AGENTS.md	read order, contracts, commands
stable concepts	docs/00-05*.md, docs/99-glossary.md	knowledge map, principles, architecture, practice, evaluation, collection
current state	docs/06-project-status.md	counts, runtime, artifact inventory
exploration history	docs/07-exploration-history.md	accepted and rejected routes
continuation plan	docs/08-next-work.md	priorities and definition of done
jobs	jobs/	Beijing-first rolling six-month JD stream
papers	papers/	paper metadata, queue, notes, corpus summary
industry	industry/	official reports, blogs, products, benchmarks
collection runs	collection-runs/	queries, scope, counts, method notes
collection tools	tools/collection/	item creation, arXiv collection, promotion, indexing, URL checks
research	research/memory/, research/completion-verification/	field findings and evidence ledgers
Agent evaluation	research/evaluation/	executable design contract for task suites, validators, paired comparisons, and release gates
evaluation data/tools	data/evaluation/, tools/evaluation/	versioned task contracts and JSONL result scorer
evaluation report	evaluation-site/	Chinese interactive explanation of the contract, decision gates, tasks, and artifacts
research data	data/research/	committed corpus, screening, landscape and CSV ledger
research tools	tools/research/	paginated search, abstract fetch, screening, classification, landscape build
experiments	experiments/knowledge-compilation/	failed pilot and completed full-text research audits
learning prototype	learning/agent-memory/	preserved failed 12-paper learning module
web application	web/	Atlas, search agent, paper reader/chat, Ollama actions
project integrity	tools/check_project.py	offline structural and data checks

projects/ and references/ are still mostly scaffolding. They do not yet contain substantive project or general-reference catalogs.

Data Lineage

official/web/arXiv sources
  -> jobs|papers|industry/items/*.md
  -> tools/collection/build_index.py
  -> data/index.json + data/summary.json
  -> web/app.py
  -> Atlas / search / paper reading UI

The dedicated memory audit is separate:

paginated arXiv queries + local memory-tagged papers
  -> data/research/* corpus snapshots
  -> model-assisted triage + deterministic statistics
  -> selected full-text experiment checks
  -> research/memory/evidence-ledger.md
  -> research/memory/findings.md

Do not merge these two lines silently. The first is a broad browsing corpus; the second is an evidence audit with explicit boundaries.

The Agent completion audit follows the same evidence discipline without a model-assisted screening stage:

problem-driven arXiv candidates
  -> 40 downloaded and text-located full papers
  -> 37 audited experiment/ablation/limitation records
  -> research/completion-verification/evidence-ledger.md
  -> research/completion-verification/findings.md

The PDF and extracted-text working set was temporary. The committed evidence ledger preserves source IDs, key numbers, claim boundaries, and exclusions so another researcher can refetch the sources.

The Agent evaluation design follows a third line:

zk-data-agent runtime/events + Zero decision-loop constraints
  -> versioned task contracts
  -> isolated repeated baseline/variant runs
  -> independent validators + trace rules
  -> JSONL result records
  -> gain/regression/safety decision

Only the task contracts and scorer exist in this repository. The reusable runner, fixtures, validators, and real-model comparison still need to be implemented in zk-data-agent. That implementation is intentionally paused while the research findings are reviewed. Before implementation, task contracts need separate outcome/invariant/procedure/semantic-residual layers, non-binary ambiguous/unverifiable states, and explicit recovery-stage fields.

evaluation-site/ visualizes this contract but does not add evidence. Its adjustable decision values are explicitly labeled as a method simulation, not a real benchmark.

Web Application

Capabilities currently implemented:

• topic Atlas with ECharts and SVG fallback;
• routes, timeline, topic focus and paper filtering;
• desktop/mobile responsive reader mode;
• local keyword search plus an Agent search loop;
• Chinese query interpretation and English keyword expansion;
• asynchronous search activity, aggregate statistics and AI brief;
• follow-up questions on a search result;
• per-paper abstract fetch, summary, translation, deep analysis and chat;
• single-concurrency Ollama access and local caches.

Run it with:

web/manage.sh start
web/manage.sh status

Expected addresses:

local/Tailscale: http://100.114.68.27:18080
public mapping:  https://lab.k1412.top/
health:          /api/health

At the beginning of the 2026-07-12 handoff audit, no process was listening on 18080 and the public domain returned 502. The handoff added and enabled a user-level systemd service through web/manage.sh; local and public health then returned 200. Runtime status is external state and must always be checked live. After a corpus update, restart the service because the index is loaded at process startup. On 2026-07-27 both the Tailscale endpoint and https://lab.k1412.top/api/health returned the refreshed 1,125-paper count with ollama_ready: true.

The live Ollama inventory checked on 2026-07-12 included ChatGPT-5.6:Luna, Terra, Sol, auto, embed, and gpt-4o:latest. The web defaults were updated from stale light/fast/large tags to configurable Luna/Terra/Sol tiers.

The static Agent evaluation report is packaged separately:

source:          evaluation-site/
public mapping:  https://agent-eval.k1412.top/
container port:  8080
health:          /health
deployment:      Unraid Compose Manager, app slug agent-eval

It uses no Ollama or runtime API. The source report remains usable even when the research application is offline.

The 2026-07-27 deployment uses immutable image docker.k1412.top/wuyang/agent-eval:20260727T084425Z-cd1eacb on NAS host port 12005. Container health, NAS/VPS Tailscale reachability, HTTP-to-HTTPS redirect, HTTP/2, Let's Encrypt SAN, static-asset hashes, public desktop/mobile rendering, and the homepage entry were all verified.

Runtime Constraints

• Python standard library only; no package installation is required for the current web server and collection scripts.
• Ollama endpoint defaults to http://192.168.1.10:11434.
• One V100 32G serves all local models; AI calls are serialized.
• Do not send num_ctx or keep_alive from clients. Server-side context and residency must remain authoritative.
• Sol is the expensive tier and should require an explicit user action.
• The user mentioned an unlimited company GPT-5.5 data-agent API as a possible stronger offline worker, but no endpoint/authentication contract is committed. It is not currently integrated.
• The public site has no repository-managed authentication or TLS configuration; the domain mapping is external infrastructure.
• The committed systemd unit assumes the checkout path %h/Code/agent; edit the unit if a future checkout uses a different path.
• Full PDFs and extracted text used during the memory audit were temporary working files and are not committed. Their arXiv IDs and claim boundaries are preserved in research/memory/evidence-ledger.md so they can be fetched again.
• web/cache/ contents, including arXiv responses, AI outputs, service environment and logs, are intentionally not committed.

Validation

Offline validation:

python3 tools/check_project.py
python3 -m unittest tools.evaluation.test_score_agent_runs
git diff --check

Live validation:

web/manage.sh status
curl -fsS http://100.114.68.27:18080/api/health | jq
curl -fsS https://lab.k1412.top/api/health | jq
curl -fsS https://agent-eval.k1412.top/health

The health payload reports app/assets versions, paper count, available/default models, and missing default model tags.