diff --git a/README.md b/README.md index fef61e1..8d05182 100644 --- a/README.md +++ b/README.md @@ -76,6 +76,330 @@ Built on the public porting workspace from [instructkr/claw-code](https://github - [ ] Some deeper runtime details from the npm source - [ ] Cost tracking and budget limits + +--- + +## ๐Ÿ—๏ธ Architecture + +```text +claw-code/ +โ”œโ”€โ”€ README.md +โ”œโ”€โ”€ pyproject.toml +โ”œโ”€โ”€ .gitignore +โ”œโ”€โ”€ images/ +โ”‚ โ””โ”€โ”€ logo.png +โ”œโ”€โ”€ src/ # Python implementation +โ”‚ โ”œโ”€โ”€ main.py # CLI entry point & argument parsing +โ”‚ โ”œโ”€โ”€ agent_runtime.py # Core agent loop (LocalCodingAgent) +โ”‚ โ”œโ”€โ”€ agent_tools.py # Tool definitions & execution engine +โ”‚ โ”œโ”€โ”€ agent_prompting.py # System prompt assembly +โ”‚ โ”œโ”€โ”€ agent_context.py # Context building & CLAUDE.md discovery +โ”‚ โ”œโ”€โ”€ agent_context_usage.py # Context usage estimation & reporting +โ”‚ โ”œโ”€โ”€ agent_session.py # Session state management +โ”‚ โ”œโ”€โ”€ agent_slash_commands.py # Local slash command processing +โ”‚ โ”œโ”€โ”€ agent_types.py # Shared dataclasses & type definitions +โ”‚ โ”œโ”€โ”€ openai_compat.py # OpenAI-compatible API client +โ”‚ โ”œโ”€โ”€ session_store.py # Session serialization & persistence +โ”‚ โ”œโ”€โ”€ permissions.py # Tool permission filtering +โ”‚ โ”œโ”€โ”€ tools.py # Mirrored tool inventory +โ”‚ โ”œโ”€โ”€ commands.py # Mirrored command inventory +โ”‚ โ”œโ”€โ”€ ... # 75+ modules across 30+ packages +โ”‚ โ”œโ”€โ”€ plugins/ # Plugin subsystem (WIP) +โ”‚ โ”œโ”€โ”€ hooks/ # Hook system (WIP) +โ”‚ โ”œโ”€โ”€ remote/ # Remote runtime modes (WIP) +โ”‚ โ”œโ”€โ”€ voice/ # Voice mode (WIP) +โ”‚ โ””โ”€โ”€ vim/ # VIM mode (WIP) +โ””โ”€โ”€ tests/ # Unit tests + โ”œโ”€โ”€ test_agent_runtime.py + โ”œโ”€โ”€ test_agent_context.py + โ”œโ”€โ”€ test_agent_context_usage.py + โ”œโ”€โ”€ test_agent_prompting.py + โ”œโ”€โ”€ test_agent_slash_commands.py + โ””โ”€โ”€ test_porting_workspace.py +``` + +--- + +## ๐Ÿ“ฆ Requirements + +| Requirement | Details | +|-------------|---------| +| ๐Ÿ Python | `3.10` or higher | +| ๐Ÿ–ฅ๏ธ Model Server | `vLLM`, `Ollama`, or `LiteLLM Proxy`, with tool calling support | +| ๐Ÿง  Model | [`Qwen/Qwen3-Coder-30B-A3B-Instruct`](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct) (recommended) | + +--- + +## ๐Ÿš€ Quick Start + +### 1. Start vLLM with Qwen3-Coder + +vLLM must be started with automatic tool choice enabled. Use the `qwen3_xml` parser for Qwen3-Coder tool calling: + +```bash +python -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen3-Coder-30B-A3B-Instruct \ + --host 127.0.0.1 \ + --port 8000 \ + --enable-auto-tool-choice \ + --tool-call-parser qwen3_xml +``` + +Verify the server is running: + +```bash +curl http://127.0.0.1:8000/v1/models +``` + +> ๐Ÿ“š **References:** [vLLM Tool Calling Docs](https://docs.vllm.ai/en/v0.13.0/features/tool_calling/) ยท [OpenAI-Compatible Server](https://docs.vllm.ai/en/v0.13.0/serving/openai_compatible_server.html) + +### Optional: Use Ollama Instead of vLLM + +`claw-code-agent` can also work with Ollama because the runtime targets an OpenAI-compatible API. Use a model that supports tool calling well. + +Example: + +```bash +ollama serve +ollama pull qwen3 +``` + +Then configure: + +```bash +export OPENAI_BASE_URL=http://127.0.0.1:11434/v1 +export OPENAI_API_KEY=ollama +export OPENAI_MODEL=qwen3 +``` + +Notes: + +- prefer tool-capable models such as `qwen3` +- plain chat-only models are not enough for full agent behavior +- Ollama does not use the `vLLM` parser flags shown above + +> ๐Ÿ“š **References:** [Ollama OpenAI Compatibility](https://docs.ollama.com/api/openai-compatibility) ยท [Ollama Tool Calling](https://docs.ollama.com/capabilities/tool-calling) + +### Optional: Use LiteLLM Proxy + +`claw-code-agent` can also work through LiteLLM Proxy because the runtime targets an OpenAI-compatible chat completions API. The routed model still needs to support tool calling for full agent behavior. + +Quick start example: + +```bash +pip install 'litellm[proxy]' +litellm --model ollama/qwen3 +``` + +LiteLLM Proxy runs on port `4000` by default. Then configure: + +```bash +export OPENAI_BASE_URL=http://127.0.0.1:4000 +export OPENAI_API_KEY=anything +export OPENAI_MODEL=ollama/qwen3 +``` + +Notes: + +- LiteLLM Proxy gives you an OpenAI-style gateway in front of many providers +- tool use still depends on the underlying routed model and provider behavior +- if you configure a LiteLLM master key, use that instead of `anything` + +> ๐Ÿ“š **References:** [LiteLLM Docs](https://docs.litellm.ai/) ยท [LiteLLM Proxy Quick Start](https://docs.litellm.ai/) + +### 2. Configure Environment + +```bash +export OPENAI_BASE_URL=http://127.0.0.1:8000/v1 +export OPENAI_API_KEY=local-token +export OPENAI_MODEL=Qwen/Qwen3-Coder-30B-A3B-Instruct +``` + +### Use Another Model With vLLM + +If you want to try another model, keep the same `vLLM` server setup and change the `--model` value when you launch `vLLM`. + +Example: + +```bash +python -m vllm.entrypoints.openai.api_server \ + --model your-model-name \ + --host 127.0.0.1 \ + --port 8000 \ + --enable-auto-tool-choice \ + --tool-call-parser your_parser +``` + +Then update: + +```bash +export OPENAI_MODEL=your-model-name +``` + +Notes: + +- the documented path in this repository is `vLLM` +- the model must support tool calling well enough for agent use +- some model families require a different `--tool-call-parser` +- slash commands such as `/help`, `/context`, and `/tools` are local and do not require the model server + +### 3. Run the Agent + +```bash +# Read-only question +python3 -m src.main agent \ + "Read src/agent_runtime.py and summarize how the loop works." \ + --cwd . + +# Write-enabled task +python3 -m src.main agent \ + "Create TEST_QWEN_AGENT.md with one line: test ok" \ + --cwd . --allow-write + +# Shell-enabled task +python3 -m src.main agent \ + "Run pwd and ls src, then summarize the result." \ + --cwd . --allow-shell +``` + +--- + +## ๐Ÿ› ๏ธ Usage + +### Agent Commands + +| Command | Description | +|---------|-------------| +| `agent ` | Run the agent with a prompt | +| `agent-prompt` | Show the assembled system prompt | +| `agent-context` | Show estimated context usage | +| `agent-context-raw` | Show the raw context snapshot | +| `agent-resume ` | Resume a saved session | + +### CLI Flags + +| Flag | Description | +|------|-------------| +| `--cwd ` | Set the workspace directory | +| `--model ` | Override the model name | +| `--base-url ` | Override the API base URL | +| `--allow-write` | Allow the agent to modify files | +| `--allow-shell` | Allow the agent to execute shell commands | +| `--unsafe` | Allow destructive shell operations | +| `--show-transcript` | Print the full message transcript | +| `--system-prompt ` | Set a custom system prompt | +| `--append-system-prompt ` | Append to the system prompt | +| `--add-dir ` | Add extra directories to context | + +### Slash Commands + +These are handled **locally** before the model loop: + +| Command | Aliases | Description | +|---------|---------|-------------| +| `/help` | `/commands` | Show built-in slash commands | +| `/context` | `/usage` | Show estimated session context usage | +| `/context-raw` | `/env` | Show raw environment & context snapshot | +| `/prompt` | `/system-prompt` | Render the effective system prompt | +| `/permissions` | โ€” | Show active tool permission mode | +| `/model` | โ€” | Show or update the active model | +| `/tools` | โ€” | List registered tools with permission status | +| `/memory` | โ€” | Show loaded CLAUDE.md memory bundle | +| `/status` | `/session` | Show runtime/session status summary | +| `/clear` | โ€” | Clear ephemeral runtime state | + +```bash +python3 -m src.main agent "/help" +python3 -m src.main agent "/context" --cwd . +python3 -m src.main agent "/tools" --cwd . +python3 -m src.main agent "/status" --cwd . +``` + +### Utility Commands + +```bash +python3 -m src.main summary # Workspace summary +python3 -m src.main manifest # Workspace manifest +python3 -m src.main commands --limit 10 # Command inventory +python3 -m src.main tools --limit 10 # Tool inventory +``` + +--- + + +## ๐Ÿ”ง Built-in Tools + +The agent has access to 7 core tools: + +| Tool | Description | Permission | +|------|-------------|------------| +| `list_dir` | List files and directories | ๐ŸŸข Always | +| `read_file` | Read file contents (with line ranges) | ๐ŸŸข Always | +| `write_file` | Write or create files | ๐ŸŸก `--allow-write` | +| `edit_file` | Edit files via exact string matching | ๐ŸŸก `--allow-write` | +| `glob_search` | Find files by glob pattern | ๐ŸŸข Always | +| `grep_search` | Search file contents by regex | ๐ŸŸข Always | +| `bash` | Execute shell commands | ๐Ÿ”ด `--allow-shell` | + +--- + +## ๐Ÿ”„ Session Persistence + +Each `agent` run automatically saves a resumable session: + +```text +session_id=4f2c8c6f9c0e4d7c9c7b1b2a3d4e5f67 +session_path=.port_sessions/agent/4f2c8c6f... +``` + +Resume a previous session: + +```bash +python3 -m src.main agent-resume \ + 4f2c8c6f9c0e4d7c9c7b1b2a3d4e5f67 \ + "Continue the previous task and finish the missing parts." +``` + +Inspect saved sessions: + +```bash +ls -lt .port_sessions/agent +``` + +> **Note:** Run `agent-resume` from the same `claw-code/` directory where the session was created. A resumed session continues from the saved transcript, not from scratch. + +--- + +## ๐Ÿงช Testing + +Run the full test suite: + +```bash +python3 -m unittest discover -s tests -v +``` + +Smoke tests: + +```bash +python3 -m src.main agent "/help" +python3 -m src.main agent-context --cwd . +python3 -m src.main agent \ + "Read src/agent_session.py and summarize the message flow." \ + --cwd . +``` + +--- + +## ๐Ÿ” Permission Model + +Claw Code Agent uses a **tiered permission system** to keep the agent safe by default: + +| Tier | Capability | Flag Required | +|------|-----------|---------------| +| **Read-only** | List, read, glob, grep | None (default) | +| **Write** | + file creation and editing | `--allow-write` | +| **Shell** | + shell command execution | `--allow-shell` | +| **Unsafe** | + destructive shell operations | `--unsafe` | ## ๐Ÿ”Ž Detailed Parity Status Against npm `src` This section tracks what is already implemented in Python and what is still missing compared with the upstream npm runtime in [`/src`](/data/fs201059/aa17626/claude_code_source/src). @@ -429,330 +753,6 @@ Mirrored inventory / scaffold areas that still need real implementation work: --- -## ๐Ÿ—๏ธ Architecture - -```text -claw-code/ -โ”œโ”€โ”€ README.md -โ”œโ”€โ”€ pyproject.toml -โ”œโ”€โ”€ .gitignore -โ”œโ”€โ”€ images/ -โ”‚ โ””โ”€โ”€ logo.png -โ”œโ”€โ”€ src/ # Python implementation -โ”‚ โ”œโ”€โ”€ main.py # CLI entry point & argument parsing -โ”‚ โ”œโ”€โ”€ agent_runtime.py # Core agent loop (LocalCodingAgent) -โ”‚ โ”œโ”€โ”€ agent_tools.py # Tool definitions & execution engine -โ”‚ โ”œโ”€โ”€ agent_prompting.py # System prompt assembly -โ”‚ โ”œโ”€โ”€ agent_context.py # Context building & CLAUDE.md discovery -โ”‚ โ”œโ”€โ”€ agent_context_usage.py # Context usage estimation & reporting -โ”‚ โ”œโ”€โ”€ agent_session.py # Session state management -โ”‚ โ”œโ”€โ”€ agent_slash_commands.py # Local slash command processing -โ”‚ โ”œโ”€โ”€ agent_types.py # Shared dataclasses & type definitions -โ”‚ โ”œโ”€โ”€ openai_compat.py # OpenAI-compatible API client -โ”‚ โ”œโ”€โ”€ session_store.py # Session serialization & persistence -โ”‚ โ”œโ”€โ”€ permissions.py # Tool permission filtering -โ”‚ โ”œโ”€โ”€ tools.py # Mirrored tool inventory -โ”‚ โ”œโ”€โ”€ commands.py # Mirrored command inventory -โ”‚ โ”œโ”€โ”€ ... # 75+ modules across 30+ packages -โ”‚ โ”œโ”€โ”€ plugins/ # Plugin subsystem (WIP) -โ”‚ โ”œโ”€โ”€ hooks/ # Hook system (WIP) -โ”‚ โ”œโ”€โ”€ remote/ # Remote runtime modes (WIP) -โ”‚ โ”œโ”€โ”€ voice/ # Voice mode (WIP) -โ”‚ โ””โ”€โ”€ vim/ # VIM mode (WIP) -โ””โ”€โ”€ tests/ # Unit tests - โ”œโ”€โ”€ test_agent_runtime.py - โ”œโ”€โ”€ test_agent_context.py - โ”œโ”€โ”€ test_agent_context_usage.py - โ”œโ”€โ”€ test_agent_prompting.py - โ”œโ”€โ”€ test_agent_slash_commands.py - โ””โ”€โ”€ test_porting_workspace.py -``` - ---- - -## ๐Ÿ“ฆ Requirements - -| Requirement | Details | -|-------------|---------| -| ๐Ÿ Python | `3.10` or higher | -| ๐Ÿ–ฅ๏ธ Model Server | `vLLM`, `Ollama`, or `LiteLLM Proxy`, with tool calling support | -| ๐Ÿง  Model | [`Qwen/Qwen3-Coder-30B-A3B-Instruct`](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct) (recommended) | - ---- - -## ๐Ÿš€ Quick Start - -### 1. Start vLLM with Qwen3-Coder - -vLLM must be started with automatic tool choice enabled. Use the `qwen3_xml` parser for Qwen3-Coder tool calling: - -```bash -python -m vllm.entrypoints.openai.api_server \ - --model Qwen/Qwen3-Coder-30B-A3B-Instruct \ - --host 127.0.0.1 \ - --port 8000 \ - --enable-auto-tool-choice \ - --tool-call-parser qwen3_xml -``` - -Verify the server is running: - -```bash -curl http://127.0.0.1:8000/v1/models -``` - -> ๐Ÿ“š **References:** [vLLM Tool Calling Docs](https://docs.vllm.ai/en/v0.13.0/features/tool_calling/) ยท [OpenAI-Compatible Server](https://docs.vllm.ai/en/v0.13.0/serving/openai_compatible_server.html) - -### Optional: Use Ollama Instead of vLLM - -`claw-code-agent` can also work with Ollama because the runtime targets an OpenAI-compatible API. Use a model that supports tool calling well. - -Example: - -```bash -ollama serve -ollama pull qwen3 -``` - -Then configure: - -```bash -export OPENAI_BASE_URL=http://127.0.0.1:11434/v1 -export OPENAI_API_KEY=ollama -export OPENAI_MODEL=qwen3 -``` - -Notes: - -- prefer tool-capable models such as `qwen3` -- plain chat-only models are not enough for full agent behavior -- Ollama does not use the `vLLM` parser flags shown above - -> ๐Ÿ“š **References:** [Ollama OpenAI Compatibility](https://docs.ollama.com/api/openai-compatibility) ยท [Ollama Tool Calling](https://docs.ollama.com/capabilities/tool-calling) - -### Optional: Use LiteLLM Proxy - -`claw-code-agent` can also work through LiteLLM Proxy because the runtime targets an OpenAI-compatible chat completions API. The routed model still needs to support tool calling for full agent behavior. - -Quick start example: - -```bash -pip install 'litellm[proxy]' -litellm --model ollama/qwen3 -``` - -LiteLLM Proxy runs on port `4000` by default. Then configure: - -```bash -export OPENAI_BASE_URL=http://127.0.0.1:4000 -export OPENAI_API_KEY=anything -export OPENAI_MODEL=ollama/qwen3 -``` - -Notes: - -- LiteLLM Proxy gives you an OpenAI-style gateway in front of many providers -- tool use still depends on the underlying routed model and provider behavior -- if you configure a LiteLLM master key, use that instead of `anything` - -> ๐Ÿ“š **References:** [LiteLLM Docs](https://docs.litellm.ai/) ยท [LiteLLM Proxy Quick Start](https://docs.litellm.ai/) - -### 2. Configure Environment - -```bash -export OPENAI_BASE_URL=http://127.0.0.1:8000/v1 -export OPENAI_API_KEY=local-token -export OPENAI_MODEL=Qwen/Qwen3-Coder-30B-A3B-Instruct -``` - -### Use Another Model With vLLM - -If you want to try another model, keep the same `vLLM` server setup and change the `--model` value when you launch `vLLM`. - -Example: - -```bash -python -m vllm.entrypoints.openai.api_server \ - --model your-model-name \ - --host 127.0.0.1 \ - --port 8000 \ - --enable-auto-tool-choice \ - --tool-call-parser your_parser -``` - -Then update: - -```bash -export OPENAI_MODEL=your-model-name -``` - -Notes: - -- the documented path in this repository is `vLLM` -- the model must support tool calling well enough for agent use -- some model families require a different `--tool-call-parser` -- slash commands such as `/help`, `/context`, and `/tools` are local and do not require the model server - -### 3. Run the Agent - -```bash -# Read-only question -python3 -m src.main agent \ - "Read src/agent_runtime.py and summarize how the loop works." \ - --cwd . - -# Write-enabled task -python3 -m src.main agent \ - "Create TEST_QWEN_AGENT.md with one line: test ok" \ - --cwd . --allow-write - -# Shell-enabled task -python3 -m src.main agent \ - "Run pwd and ls src, then summarize the result." \ - --cwd . --allow-shell -``` - ---- - -## ๐Ÿ› ๏ธ Usage - -### Agent Commands - -| Command | Description | -|---------|-------------| -| `agent ` | Run the agent with a prompt | -| `agent-prompt` | Show the assembled system prompt | -| `agent-context` | Show estimated context usage | -| `agent-context-raw` | Show the raw context snapshot | -| `agent-resume ` | Resume a saved session | - -### CLI Flags - -| Flag | Description | -|------|-------------| -| `--cwd ` | Set the workspace directory | -| `--model ` | Override the model name | -| `--base-url ` | Override the API base URL | -| `--allow-write` | Allow the agent to modify files | -| `--allow-shell` | Allow the agent to execute shell commands | -| `--unsafe` | Allow destructive shell operations | -| `--show-transcript` | Print the full message transcript | -| `--system-prompt ` | Set a custom system prompt | -| `--append-system-prompt ` | Append to the system prompt | -| `--add-dir ` | Add extra directories to context | - -### Slash Commands - -These are handled **locally** before the model loop: - -| Command | Aliases | Description | -|---------|---------|-------------| -| `/help` | `/commands` | Show built-in slash commands | -| `/context` | `/usage` | Show estimated session context usage | -| `/context-raw` | `/env` | Show raw environment & context snapshot | -| `/prompt` | `/system-prompt` | Render the effective system prompt | -| `/permissions` | โ€” | Show active tool permission mode | -| `/model` | โ€” | Show or update the active model | -| `/tools` | โ€” | List registered tools with permission status | -| `/memory` | โ€” | Show loaded CLAUDE.md memory bundle | -| `/status` | `/session` | Show runtime/session status summary | -| `/clear` | โ€” | Clear ephemeral runtime state | - -```bash -python3 -m src.main agent "/help" -python3 -m src.main agent "/context" --cwd . -python3 -m src.main agent "/tools" --cwd . -python3 -m src.main agent "/status" --cwd . -``` - -### Utility Commands - -```bash -python3 -m src.main summary # Workspace summary -python3 -m src.main manifest # Workspace manifest -python3 -m src.main commands --limit 10 # Command inventory -python3 -m src.main tools --limit 10 # Tool inventory -``` - ---- - - -## ๐Ÿ”ง Built-in Tools - -The agent has access to 7 core tools: - -| Tool | Description | Permission | -|------|-------------|------------| -| `list_dir` | List files and directories | ๐ŸŸข Always | -| `read_file` | Read file contents (with line ranges) | ๐ŸŸข Always | -| `write_file` | Write or create files | ๐ŸŸก `--allow-write` | -| `edit_file` | Edit files via exact string matching | ๐ŸŸก `--allow-write` | -| `glob_search` | Find files by glob pattern | ๐ŸŸข Always | -| `grep_search` | Search file contents by regex | ๐ŸŸข Always | -| `bash` | Execute shell commands | ๐Ÿ”ด `--allow-shell` | - ---- - -## ๐Ÿ”„ Session Persistence - -Each `agent` run automatically saves a resumable session: - -```text -session_id=4f2c8c6f9c0e4d7c9c7b1b2a3d4e5f67 -session_path=.port_sessions/agent/4f2c8c6f... -``` - -Resume a previous session: - -```bash -python3 -m src.main agent-resume \ - 4f2c8c6f9c0e4d7c9c7b1b2a3d4e5f67 \ - "Continue the previous task and finish the missing parts." -``` - -Inspect saved sessions: - -```bash -ls -lt .port_sessions/agent -``` - -> **Note:** Run `agent-resume` from the same `claw-code/` directory where the session was created. A resumed session continues from the saved transcript, not from scratch. - ---- - -## ๐Ÿงช Testing - -Run the full test suite: - -```bash -python3 -m unittest discover -s tests -v -``` - -Smoke tests: - -```bash -python3 -m src.main agent "/help" -python3 -m src.main agent-context --cwd . -python3 -m src.main agent \ - "Read src/agent_session.py and summarize the message flow." \ - --cwd . -``` - ---- - -## ๐Ÿ” Permission Model - -Claw Code Agent uses a **tiered permission system** to keep the agent safe by default: - -| Tier | Capability | Flag Required | -|------|-----------|---------------| -| **Read-only** | List, read, glob, grep | None (default) | -| **Write** | + file creation and editing | `--allow-write` | -| **Shell** | + shell command execution | `--allow-shell` | -| **Unsafe** | + destructive shell operations | `--unsafe` | - ---- - ## โš ๏ธ Disclaimer - This repository is a **Python reimplementation** inspired by the Claude Code npm architecture.