Implemented the next parity slice: prompt-budget preflight and context collapse.

Core changes:

  - Added claw-code/src/token_budget.py for projected prompt size, chat-framing overhead, output reserve, and soft/hard input limits.
  - Wired preflight prompt-length validation and auto-compact/context collapse into claw-code/src/agent_runtime.py.
  - Extended claw-code/src/compact.py so compaction reports usage back to the runtime.
  - Added inspection surfaces in claw-code/src/agent_slash_commands.py and claw-code/src/main.py:
      - /token-budget and /budget
      - token-budget
  - Hardened claw-code/src/tokenizer_runtime.py so arbitrary simple model names fall back cleanly instead of trying a slow Transformers
    lookup.
  - Exported the new helpers in claw-code/src/__init__.py.

  Docs and tracking:

  - Updated claw-code/PARITY_CHECKLIST.md to mark prompt-length validation, token-budget calculation, and auto-compact/context collapse as
    done.
  - Updated claw-code/README.md and claw-code/TESTING_GUIDE.md with the new commands and behavior.

  Tests:

  - Added claw-code/tests/test_token_budget.py.
  - Updated claw-code/tests/test_agent_runtime.py, claw-code/tests/test_agent_slash_commands.py, claw-code/tests/test_main.py, and claw-code/
    tests/test_agent_context_usage.py.
  - Verified with:
      - /data/fs201059/aa17626/miniconda3/bin/python3 -m compileall src tests
      - /data/fs201059/aa17626/miniconda3/bin/python3 -m unittest -v tests.test_token_budget
        tests.test_agent_runtime.AgentRuntimeTests.test_agent_rejects_prompt_before_backend_when_preflight_input_budget_is_exceeded
        tests.test_agent_runtime.AgentRuntimeTests.test_agent_auto_compacts_context_before_next_model_call tests.test_agent_slash_commands
        tests.test_main tests.test_compact tests.test_tokenizer_runtime tests.test_agent_context_usage
      - Result: 71 tests, OK
This commit is contained in:
Abdelrahman Abdallah
2026-04-11 01:38:19 +02:00
parent aacf0a212a
commit b5e5824a56
23 changed files with 2641 additions and 10 deletions
+70
View File
@@ -91,6 +91,7 @@ python3 -m unittest tests.test_plan_runtime -v
python3 -m unittest tests.test_background_runtime -v
python3 -m unittest tests.test_remote_runtime -v
python3 -m unittest tests.test_config_runtime -v
python3 -m unittest tests.test_lsp_runtime -v
python3 -m unittest tests.test_account_runtime -v
python3 -m unittest tests.test_ask_user_runtime -v
python3 -m unittest tests.test_team_runtime -v
@@ -309,6 +310,30 @@ cat > ./test_cases_notebooks/demo.ipynb <<'EOF'
EOF
```
### 4.2e LSP fixture
```bash
cat > ./test_cases/sample.py <<'EOF'
def helper(value):
"""Double a numeric value."""
return value * 2
def orchestrate(item):
return helper(item)
class Greeter:
def greet(self, name):
return helper(len(name))
EOF
cat > ./test_cases/broken.py <<'EOF'
def broken(:
pass
EOF
```
### 4.3 Remote fixtures
```bash
@@ -568,6 +593,8 @@ python3 -m src.main agent "/context" --cwd ./test_cases
python3 -m src.main agent "/usage summarize current session" --cwd ./test_cases
python3 -m src.main agent "/context-raw" --cwd ./test_cases
python3 -m src.main agent "/env" --cwd ./test_cases
python3 -m src.main agent "/token-budget" --cwd ./test_cases
python3 -m src.main agent "/budget" --cwd ./test_cases
python3 -m src.main agent "/prompt" --cwd ./test_cases
python3 -m src.main agent "/system-prompt" --cwd ./test_cases
python3 -m src.main agent "/permissions" --cwd ./test_cases
@@ -650,6 +677,7 @@ python3 -m src.main agent "/mcp (MCP)" --cwd ./test_cases_mcp
python3 -m src.main agent-prompt --cwd ./test_cases
python3 -m src.main agent-context --cwd ./test_cases
python3 -m src.main agent-context-raw --cwd ./test_cases
python3 -m src.main token-budget --cwd ./test_cases
```
### 6.2 Extra working directories and `CLAUDE.md` toggle
@@ -680,6 +708,7 @@ python3 -m src.main agent-prompt \
```bash
python3 -m src.main agent "/status" --cwd ./test_cases
python3 -m src.main agent-context --cwd ./test_cases
python3 -m src.main token-budget --cwd ./test_cases
```
Override the tokenizer backend:
@@ -691,6 +720,7 @@ export CLAW_CODE_TOKENIZER_MODEL=Qwen/Qwen3-Coder-30B-A3B-Instruct
python3 -m src.main agent "/status" --cwd ./test_cases
python3 -m src.main agent-context --cwd ./test_cases
python3 -m src.main token-budget --cwd ./test_cases
```
If no tokenizer backend is available, the runtime will fall back to the heuristic counter and `/status` will report that.
@@ -1228,6 +1258,46 @@ python3 -m src.main agent \
--show-transcript
```
## 14B. LSP Runtime
### 14B.1 CLI status and code intelligence reports
```bash
python3 -m src.main lsp-status --cwd ./test_cases
python3 -m src.main lsp-symbols sample.py --cwd ./test_cases
python3 -m src.main lsp-workspace-symbols helper --cwd ./test_cases
python3 -m src.main lsp-definition sample.py 6 12 --cwd ./test_cases
python3 -m src.main lsp-references sample.py 6 12 --cwd ./test_cases
python3 -m src.main lsp-hover sample.py 1 5 --cwd ./test_cases
python3 -m src.main lsp-diagnostics --cwd ./test_cases
python3 -m src.main lsp-diagnostics --cwd ./test_cases --file-path broken.py
python3 -m src.main lsp-call-hierarchy sample.py 6 12 --cwd ./test_cases
python3 -m src.main lsp-incoming-calls sample.py 1 5 --cwd ./test_cases
python3 -m src.main lsp-outgoing-calls sample.py 6 12 --cwd ./test_cases
```
### 14B.2 Slash commands
```bash
python3 -m src.main agent "/lsp" --cwd ./test_cases
python3 -m src.main agent "/lsp symbols sample.py" --cwd ./test_cases
python3 -m src.main agent "/lsp workspace helper" --cwd ./test_cases
python3 -m src.main agent "/lsp definition sample.py 6 12" --cwd ./test_cases
python3 -m src.main agent "/lsp references sample.py 6 12" --cwd ./test_cases
python3 -m src.main agent "/lsp hover sample.py 1 5" --cwd ./test_cases
python3 -m src.main agent "/lsp diagnostics broken.py" --cwd ./test_cases
python3 -m src.main agent "/lsp hierarchy sample.py 6 12" --cwd ./test_cases
```
### 14B.3 Real tool loop
```bash
python3 -m src.main agent \
"Use the LSP tool to find the definition of helper in sample.py, then summarize the result." \
--cwd ./test_cases \
--show-transcript
```
## 14A. Ask-user Runtime
### 14A.1 CLI status and history