agent/docs/06-project-status.md

# Project Status

last_reviewed: 2026-07-27
baseline_before_update: `8e4ff37`
branch: `main`
remote: `https://git.k1412.top/wuyang/agent.git`

## Purpose

The repository began as a durable Agent knowledge base and now has two layers:

1. a collection layer for papers, Beijing jobs, and company Agent work;
2. an experimental knowledge interface and research layer intended to turn the collection into evidence-backed understanding.

The long-term product is not “a place to browse papers.” It should help the user retain claims, connect them into a personal model, test that model, and use it to reason about unfamiliar Agent problems.

## Current Snapshot

The deterministic index currently contains 1,140 items:

| Collection | Count | State |
| --- | ---: | --- |
| papers | 1,125 | 1,110 queued, 15 skimmed |
| jobs | 6 | 4 active, 2 unknown |
| industry | 9 | 8 analyzed, 1 queued |

The 2026-07-08 expanded arXiv run saw 1,506 candidates, promoted 970 high-recall candidates,
and retained 536 reserve candidates. The 2026-07-27 incremental refresh covered July 9-27, saw
249 unique candidates, selected 157 high-relevance records, skipped 6 existing papers, and added
151 paper items. The newest submission returned was 2026-07-24. These are collection counts, not
quality or reading-completion counts.

The memory topic has two intentionally different counts:

- 336 indexed items now have the broad `memory` topic, including non-paper streams;
- 293 local paper items were resolved for the dedicated memory audit.

The memory research snapshot contains:

| Stage | Count |
| --- | ---: |
| paginated arXiv candidates | 516 |
| lexical title-central upper bound | 379 |
| recent-six-month title-central candidates | 292 |
| PDFs downloaded and text-located during the audit | 32 |
| papers promoted into the evidence ledger | 23 |

The strongest current conclusion is that recent progress is concentrated in better diagnosis/evaluation and task-specific execution-state memory, not a universal memory architecture. Read `research/memory/findings.md` before making memory design claims.

## Artifact Map

| Area | Source of truth | What it contains |
| --- | --- | --- |
| entry/navigation | `README.md` | human-facing repository map |
| AI continuation | `AGENTS.md` | read order, contracts, commands |
| stable concepts | `docs/00-05*.md`, `docs/99-glossary.md` | knowledge map, principles, architecture, practice, evaluation, collection |
| current state | `docs/06-project-status.md` | counts, runtime, artifact inventory |
| exploration history | `docs/07-exploration-history.md` | accepted and rejected routes |
| continuation plan | `docs/08-next-work.md` | priorities and definition of done |
| jobs | `jobs/` | Beijing-first rolling six-month JD stream |
| papers | `papers/` | paper metadata, queue, notes, corpus summary |
| industry | `industry/` | official reports, blogs, products, benchmarks |
| collection runs | `collection-runs/` | queries, scope, counts, method notes |
| collection tools | `tools/collection/` | item creation, arXiv collection, promotion, indexing, URL checks |
| research | `research/memory/` | field findings and evidence ledger |
| Agent evaluation | `research/evaluation/` | executable design contract for task suites, validators, paired comparisons, and release gates |
| evaluation data/tools | `data/evaluation/`, `tools/evaluation/` | versioned task contracts and JSONL result scorer |
| research data | `data/research/` | committed corpus, screening, landscape and CSV ledger |
| research tools | `tools/research/` | paginated search, abstract fetch, screening, classification, landscape build |
| experiments | `experiments/knowledge-compilation/` | failed pilot and completed field audit |
| learning prototype | `learning/agent-memory/` | preserved failed 12-paper learning module |
| web application | `web/` | Atlas, search agent, paper reader/chat, Ollama actions |
| project integrity | `tools/check_project.py` | offline structural and data checks |

`projects/` and `references/` are still mostly scaffolding. They do not yet contain substantive project or general-reference catalogs.

## Data Lineage

```text
official/web/arXiv sources
  -> jobs|papers|industry/items/*.md
  -> tools/collection/build_index.py
  -> data/index.json + data/summary.json
  -> web/app.py
  -> Atlas / search / paper reading UI
```

The dedicated memory audit is separate:

```text
paginated arXiv queries + local memory-tagged papers
  -> data/research/* corpus snapshots
  -> model-assisted triage + deterministic statistics
  -> selected full-text experiment checks
  -> research/memory/evidence-ledger.md
  -> research/memory/findings.md
```

Do not merge these two lines silently. The first is a broad browsing corpus; the second is an evidence audit with explicit boundaries.

The Agent evaluation design follows a third line:

```text
zk-data-agent runtime/events + Zero decision-loop constraints
  -> versioned task contracts
  -> isolated repeated baseline/variant runs
  -> independent validators + trace rules
  -> JSONL result records
  -> gain/regression/safety decision
```

Only the task contracts and scorer exist in this repository. The reusable runner, fixtures, validators,
and real-model comparison still need to be implemented in `zk-data-agent`.

## Web Application

Capabilities currently implemented:

- topic Atlas with ECharts and SVG fallback;
- routes, timeline, topic focus and paper filtering;
- desktop/mobile responsive reader mode;
- local keyword search plus an Agent search loop;
- Chinese query interpretation and English keyword expansion;
- asynchronous search activity, aggregate statistics and AI brief;
- follow-up questions on a search result;
- per-paper abstract fetch, summary, translation, deep analysis and chat;
- single-concurrency Ollama access and local caches.

Run it with:

```bash
web/manage.sh start
web/manage.sh status
```

Expected addresses:

```text
local/Tailscale: http://100.114.68.27:18080
public mapping:  https://lab.k1412.top/
health:          /api/health
```

At the beginning of the 2026-07-12 handoff audit, no process was listening on 18080 and the public domain returned 502. The handoff added and enabled a user-level systemd service through `web/manage.sh`;
local and public health then returned 200. Runtime status is external state and must always be checked
live. After a corpus update, restart the service because the index is loaded at process startup.
On 2026-07-27 both the Tailscale endpoint and `https://lab.k1412.top/api/health` returned the refreshed
1,125-paper count with `ollama_ready: true`.

The live Ollama inventory checked on 2026-07-12 included `ChatGPT-5.6:Luna`, `Terra`, `Sol`, `auto`, `embed`, and `gpt-4o:latest`. The web defaults were updated from stale `light/fast/large` tags to configurable Luna/Terra/Sol tiers.

## Runtime Constraints

- Python standard library only; no package installation is required for the current web server and collection scripts.
- Ollama endpoint defaults to `http://192.168.1.10:11434`.
- One V100 32G serves all local models; AI calls are serialized.
- Do not send `num_ctx` or `keep_alive` from clients. Server-side context and residency must remain authoritative.
- `Sol` is the expensive tier and should require an explicit user action.
- The user mentioned an unlimited company GPT-5.5 data-agent API as a possible stronger offline worker, but no endpoint/authentication contract is committed. It is not currently integrated.
- The public site has no repository-managed authentication or TLS configuration; the domain mapping is external infrastructure.
- The committed systemd unit assumes the checkout path `%h/Code/agent`; edit the unit if a future checkout uses a different path.
- Full PDFs and extracted text used during the memory audit were temporary working files and are not committed. Their arXiv IDs and claim boundaries are preserved in `research/memory/evidence-ledger.md` so they can be fetched again.
- `web/cache/` contents, including arXiv responses, AI outputs, service environment and logs, are intentionally not committed.

## Validation

Offline validation:

```bash
python3 tools/check_project.py
python3 -m unittest tools.evaluation.test_score_agent_runs
git diff --check
```

Live validation:

```bash
web/manage.sh status
curl -fsS http://100.114.68.27:18080/api/health | jq
curl -fsS https://lab.k1412.top/api/health | jq
```

The health payload reports app/assets versions, paper count, available/default models, and missing default model tags.