zk-data-agent/docs/project-history.md

Project history and decisions

中文 · English

This record summarizes the rebuild that began on 2026-07-26. Git remains the source of truth for exact changes; this document captures product intent and architecture decisions that are otherwise difficult to reconstruct from individual commits.

1. Clean-slate web Agent

The existing business/data-specific application was declared a new project with no backward-compatibility requirement. Business skills, domain workflows, and user-configurable provider settings were removed from scope. The retained core requirement was general coding-Agent capability with an independently evolvable loop.

Decision:

• build one general multi-user Web Agent;
• preserve coding, shell, file, Git, process, planning, memory, and delegation capabilities;
• keep infrastructure and product shell separate from Agent intelligence.

2. Open WebUI as product shell

Instead of maintaining a second full account/chat frontend, Open WebUI v0.9.6 was pinned and lightly patched. It supplies registration, login, RBAC, administrator approval, conversation history, and familiar chat interaction.

K1412 kept ownership of the Agent loop, provider routing, memory, tools, scheduling, events, and workspace execution. This reduced the cost of Agent experiments without making the loop dependent on Open WebUI internals.

3. One Agent path, not Chat versus Work

An early design considered a switch between Open WebUI's native Chat loop and the K1412 Work loop. The product was simplified to one Agent path because:

• users should not need to understand two orchestration engines;
• conversation continuation between modes creates ambiguous state;
• two loops double frontend states and verification behavior;
• the custom loop is the main research asset.

The remaining interface is an Agent-first chat with server-owned model selection.

4. Model and thinking semantics

The original “light / medium / high reasoning” labels conflated different models with adjustable effort. The UI and backend now distinguish model identity from thinking capability:

• Luna, Terra, and Sol are separate local K1412/Ollama models;
• each exposes thinking as enabled but without adjustable effort levels;
• DeepSeek V4 Pro is the cloud extreme tier with maximum reasoning effort.

Provider model IDs, URLs, credentials, and tuning remain server-side.

5. Multi-user remote workspaces

Each user received a dedicated Docker container, network, and persistent volume derived from a hashed immutable user ID. Execution moved from the NAS to the physical home-node-itx through SSH Docker.

The physical node was prepared with:

• an ext4 500 GB data disk mounted at /srv/k1412-data;
• Docker root under /srv/k1412-data/docker;
• strict SSH host verification;
• no application/database secrets.

The local and SSH execution providers retain one interface so the host can be replaced without changing Agent behavior.

6. File delivery and focused UI

The frontend was simplified around the Agent:

• K1412 product title and assets;
• a compact model selector with separate thinking status;
• no exposed provider/tool/system-prompt settings;
• an authenticated workspace file browser;
• single-file download and streamed directory archive.

Open WebUI attribution and license requirements remain documented.

7. Loop hardening through failed-task analysis

Real sorting-script/report tasks exposed model weaknesses and platform bugs. The loop was progressively hardened to:

• require actual files rather than textual claims;
• require executable source separately from reports;
• run the source before writing a report;
• require measured benchmark values;
• reject invalid Python and escaped-layout corruption;
• recover from failed verification;
• prevent unchanged retry loops and duplicate batches;
• preserve failures even when output is piped through another command;
• bound iteration, tool-batch, model-output, and retry budgets.

These constraints are not assumed permanently optimal. They are the current baseline and should be removed or changed only through measured experiments.

8. Python environment and long-run reliability

Production debugging found three platform problems:

1. the container root was read-only while pip defaulted to /home/agent;
2. /tmp was noexec;
3. shell pipelines returned the last command's status, hiding failed installs.

The fix moved home, caches, user packages, and virtual environments to the persistent workspace, enabled Bash pipefail, extended tool timeouts to 900 seconds, and taught the Agent the supported .venv workflow.

The same investigation found that a five-minute identity token was reused for every tool in a long run. Runtime now signs a fresh short-lived identity for each Gateway call after the original request identity has been verified.

Production validation created a real .venv, installed and imported psutil 7.2.2, downloaded the generated proof file through the Web API, and then removed the temporary proof file.

9. Documentation and experiment portal

The repository documentation was reorganized around architecture, Agent implementation, security, operations, history, development, and experiments. The curated public portal is shipped inside the existing Web image at /doc/, avoiding a second service or proxy rule.

Current direction

The platform is now suitable for controlled Agent-loop experimentation. The next work should prioritize evaluation data, context/memory experiments, scheduler design, delegation contracts, cost accounting, and hardening for hostile multi-tenant use rather than adding business-specific workflows.