wuyang/agent
2
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

migrate knowledge atlas to llama.cpp

Browse Source
This commit is contained in:
wuyang
2026-07-28 17:16:34 +08:00
parent 88fb212985
commit efbd82532f
9 changed files with 152 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/06-project-status.md
+7 -7
View File
@@ -85,7 +85,7 @@ evaluation contract.
| research tools | `tools/research/` | paginated search, abstract fetch, screening, classification, landscape build |
| experiments | `experiments/knowledge-compilation/` | failed pilot and completed full-text research audits |
| learning prototype | `learning/agent-memory/` | preserved failed 12-paper learning module |
| web application | `web/` | Atlas, search agent, paper reader/chat, Ollama actions |
| web application | `web/` | Atlas, search agent, paper reader/chat, llama.cpp actions |
| project integrity | `tools/check_project.py` | offline structural and data checks |
`projects/` and `references/` are still mostly scaffolding. They do not yet contain substantive project or general-reference catalogs.
@@ -161,7 +161,7 @@ Capabilities currently implemented:
- asynchronous search activity, aggregate statistics and AI brief;
- follow-up questions on a search result;
- per-paper abstract fetch, summary, translation, deep analysis and chat;
- single-concurrency Ollama access and local caches.
- single-concurrency llama.cpp access and local caches.
Run it with:
@@ -182,9 +182,9 @@ At the beginning of the 2026-07-12 handoff audit, no process was listening on 18
local and public health then returned 200. Runtime status is external state and must always be checked
live. After a corpus update, restart the service because the index is loaded at process startup.
On 2026-07-27 both the Tailscale endpoint and `https://lab.k1412.top/api/health` returned the refreshed
1,125-paper count with `ollama_ready: true`.
1,125-paper count. After the 2026-07-28 migration, readiness is reported as `model_api_ready`.
The live Ollama inventory checked on 2026-07-12 included `ChatGPT-5.6:Luna`, `Terra`, `Sol`, `auto`, `embed`, and `gpt-4o:latest`. The web defaults were updated from stale `light/fast/large` tags to configurable Luna/Terra/Sol tiers.
On 2026-07-28, inference moved from Ollama to two resident llama.cpp servers. Luna runs as Qwen3 0.6B Q8 on the NAS Radeon 780M through Vulkan. Terra and Sol share one Qwen3.6 27B Q4_K_M runner on the V100 through CUDA 12, preserving three public tiers without loading a second large model. The web application calls the two internal OpenAI-compatible endpoints directly.
The static Agent completion research document is packaged separately:
@@ -216,9 +216,9 @@ verified. The existing `AGENT EVAL` homepage entry remains under `projects` at o
## Runtime Constraints
- Python standard library only; no package installation is required for the current web server and collection scripts.
- Ollama endpoint defaults to `http://192.168.1.10:11434`.
- One V100 32G serves all local models; AI calls are serialized.
- Do not send `num_ctx` or `keep_alive` from clients. Server-side context and residency must remain authoritative.
- llama.cpp endpoints default to `http://192.168.1.10:11435/v1` for Luna and `http://192.168.1.10:11436/v1` for Terra/Sol.
- Luna uses the NAS AMD iGPU; one V100 32G serves the shared Terra/Sol runner. AI calls are serialized.
- Context is fixed server-side at 8K for Luna and 32K for Terra/Sol.
- `Sol` is the expensive tier and should require an explicit user action.
- The user mentioned an unlimited company GPT-5.5 data-agent API as a possible stronger offline worker, but no endpoint/authentication contract is committed. It is not currently integrated.
- The public site has no repository-managed authentication or TLS configuration; the domain mapping is external infrastructure.
docs/07-exploration-history.md
+10
View File
@@ -98,6 +98,16 @@ Decision retained: always verify `/api/tags`; never infer deployed model aliases
Relevant commits: `47c301c`, `ac6e0e0`, and the 2026-07-12 handoff update.
On 2026-07-28 the runtime moved to llama.cpp after repeated Ollama runner churn and
resource-residency ambiguity. Luna was isolated on the NAS AMD iGPU through Vulkan,
while Terra and Sol became two API behaviors over one fixed 27B CUDA runner on the
V100. The web app now uses the servers' OpenAI-compatible endpoints, fixes context
at deployment time, and controls thinking by tier.
Decision retained: product tiers do not require separate physical model copies.
Keep resource ownership in the serving layer and let Terra/Sol share one resident
runner when their weights are identical.
## 6. Learning-System Reframing
The user clarified the real objective with three levels: