Merge pull request #20 from HarnessLab/feature/readme-update-april-2026

Feature/readme update april 2026
2026-04-08 00:45:25 +02:00
parent 7750e34fbc aacf0a212a
commit a19b1669c4
23 changed files with 7454 additions and 264 deletions
@@ -32,3 +32,5 @@ humaneval_results.json
 test_cases
 e-commerce
 benchmarks/data/*.jsonl
 benchmarks/data/manifest.json
@@ -4,6 +4,8 @@ This document tracks what is already implemented in Python and what is still mis
 This is a functionality-oriented checklist, not a line-by-line source equivalence claim. Large parts of the mirrored Python workspace still act as inventory or scaffolding, while the working Python runtime currently lives mainly in [`src/agent_runtime.py`](src/agent_runtime.py), [`src/query_engine.py`](src/query_engine.py), [`src/agent_tools.py`](src/agent_tools.py), [`src/agent_prompting.py`](src/agent_prompting.py), [`src/agent_context.py`](src/agent_context.py), [`src/agent_manager.py`](src/agent_manager.py), [`src/plugin_runtime.py`](src/plugin_runtime.py), [`src/agent_slash_commands.py`](src/agent_slash_commands.py), and [`src/openai_compat.py`](src/openai_compat.py).
 ---
 ## 1. Core Agent Runtime
 Done:
@@ -117,7 +119,10 @@ Missing:
 - [ ] Full file history snapshots and replay flows beyond the current preview/id-based implementation and delegated-batch replay metadata
 - [ ] Full executable plugin lifecycle beyond manifest-driven prompt/tool/session hooks, blocking, aliases, virtual tools, and persisted runtime state
 - [ ] Full session compaction / snipping parity beyond lineage-aware summaries, mutation-serial compaction metadata, and replay reminders
- [ ] Full `QueryEngine.ts` parity
+- [ ] Full `QueryEngine.ts` parity (session init, message normalization, SDK-compatible message transforms, attachment handling)
 - [ ] Auto-compact and context collapse features from `query.ts`
 - [ ] Prompt length validation from `query.ts`
 - [ ] Token budget calculations from `query/tokenBudget.ts`
 ## 2. CLI Entrypoints And Runtime Modes
@@ -147,15 +152,22 @@ Done:
 Missing:
 - [ ] Full daemon supervisor parity beyond the current local daemon wrapper and worker flow
- [ ] Remote-control / bridge runtime mode
+- [ ] Remote-control / bridge runtime mode (`src/bridge/` — 30+ files: bridgeMain, bridgeApi, bridgeConfig, bridgeMessaging, bridgePermissionCallbacks, replBridge, sessionRunner, trustedDevice, etc.)
 - [ ] Browser/native-host runtime mode
- [ ] Computer-use MCP mode
+- [ ] Computer-use MCP mode (`src/entrypoints/mcp.ts`)
 - [ ] Template job mode
 - [ ] Environment runner mode
 - [ ] Self-hosted runner mode
 - [ ] tmux fast paths
 - [ ] Worktree fast paths at the CLI entrypoint level
- [ ] Full `entrypoints/cli.tsx` and `entrypoints/init.ts` parity
+- [ ] Node.js version check and platform setup from `setup.ts`
 - [ ] Worktree creation/setup from `setup.ts`
 - [ ] Terminal backup/restore from `setup.ts`
 - [ ] Release notes checking from `setup.ts`
 - [ ] Full `entrypoints/cli.tsx` parity (version flag, feature flags, env setup, dynamic imports)
 - [ ] Full `entrypoints/init.ts` parity (settings validation, OAuth, policy limits, telemetry, cleanup handlers)
 - [ ] SDK entrypoint (`entrypoints/sdk/` — controlTypes, coreTypes, runtimeTypes, settingsTypes, toolTypes)
 - [ ] Sandbox types/network config schema (`entrypoints/sandboxTypes.ts`)
 ## 3. Prompt Assembly
@@ -176,18 +188,34 @@ Done:
 - [x] Local planning guidance section in the Python system prompt
 - [x] Local task guidance section in the Python system prompt
 - [x] Product metadata/branding from `constants/product.ts` — ported to `src/prompt_constants.py`
 - [x] API limits constants from `constants/apiLimits.ts` — ported to `src/prompt_constants.py`
 - [x] Tool limits constants from `constants/toolLimits.ts` — ported to `src/prompt_constants.py`
 - [x] Spinner verbs from `constants/spinnerVerbs.ts` (187 verbs) — ported to `src/prompt_constants.py`
 - [x] Turn-completion verbs from `constants/turnCompletionVerbs.ts` (8 verbs) — ported to `src/prompt_constants.py`
 - [x] Figures/UI symbols from `constants/figures.ts` — ported to `src/prompt_constants.py`
 - [x] XML tag constants from `constants/xml.ts` — ported to `src/prompt_constants.py`
 - [x] Message constants from `constants/messages.ts` — ported to `src/prompt_constants.py`
 - [x] Date utilities from `constants/common.ts` — ported to `src/prompt_constants.py`
 - [x] System prompt section caching from `constants/systemPromptSections.ts` — ported to `src/prompt_constants.py`
 - [x] Output-style variants from `constants/outputStyles.ts` — ported to `src/prompt_constants.py`
 - [x] Cyber / risk instruction from `constants/cyberRiskInstruction.ts` — ported to `src/prompt_constants.py`
 - [x] System prompt prefixes from `constants/system.ts` — ported to `src/prompt_constants.py`
 - [x] Knowledge cutoff / model family info from `constants/prompts.ts` — ported to `src/prompt_constants.py`
 - [x] Hook instruction section template — ported to `src/prompt_constants.py`
 - [x] System reminders section — ported to `src/prompt_constants.py`
 - [x] Summarize tool results section — ported to `src/prompt_constants.py`
 - [x] Language-control section helper — ported to `src/prompt_constants.py`
 - [x] Scratchpad prompt instructions helper — ported to `src/prompt_constants.py`
 - [x] Default agent prompt — ported to `src/prompt_constants.py`
 Missing:
- [ ] Full parity with `constants/prompts.ts`
+- [ ] Full parity with `constants/prompts.ts` runtime section assembly (many sections already exist in agent_prompting.py)
- [ ] Hook instruction sections
+- [ ] MCP instruction sections (runtime MCP integration)
- [ ] MCP instruction sections
+- [ ] Model-family-specific prompt variations (runtime)
 - [ ] Model-family-specific prompt variations
 - [ ] Output-style variants
 - [ ] Language-control sections
 - [ ] Scratchpad prompt instructions
 - [ ] More exact autonomous/proactive behavior sections
- [ ] Growthbook / feature-gated prompt sections
+- [ ] Growthbook / feature-gated prompt sections (N/A for external builds)
 - [ ] Cyber / risk sections used upstream
 ## 4. Context Building And Memory
@@ -216,79 +244,122 @@ Done:
 Missing:
 - [ ] Full tokenizer/chat-message framing parity beyond the current model-aware text token counters
- [ ] Full parity with `utils/queryContext.ts`
+- [ ] Full parity with `utils/queryContext.ts` (context analysis, suggestions, cache shaping)
- [ ] Rich memory prompt loading
+- [ ] Rich memory prompt loading (`services/SessionMemory/`)
 - [ ] Internal permission-aware memory handling
 - [ ] Resume-aware prompt cache shaping used upstream
 - [ ] More exact context cache invalidation rules
- [ ] Session context analysis parity
+- [ ] Session context analysis parity (`utils/contextAnalysis.ts`, `utils/contextSuggestions.ts`)
- [ ] Full memory subsystem parity
+- [ ] Full memory subsystem parity (`utils/memory/`, `services/extractMemories/`)
 - [ ] Memory extraction from conversations (`services/extractMemories/extractMemories.ts`)
 - [ ] Team memory sync (`services/teamMemorySync/`)
 - [ ] Away summary generation (`services/awaySummary.ts`)
 - [ ] Token estimation service (`services/tokenEstimation.ts`)
 - [ ] Paste content storage and reference parsing (`history.ts`)
 - [ ] Image paste handling
 ## 5. Slash Commands
-Done:
+Done (37 slash command names in 29 specs):
- [x] `/help`
+- [x] `/help`, `/commands`
- [x] `/commands`
+- [x] `/context`, `/usage`
- [x] `/context`
+- [x] `/context-raw`, `/env`
- [x] `/usage`
+- [x] `/mcp` (with subcommands: `tools`, `tool <name>`)
- [x] `/context-raw`
+- [x] `/search` (with subcommands: `providers`, `provider`, `use`)
- [x] `/env`
+- [x] `/remote` (with `enter`, `exit`)
- [x] `/mcp`
+- [x] `/worktree` (with `enter`, `exit`)
- [x] `/mcp tools`
+- [x] `/account` (with `profiles`, `profile`)
- [x] `/mcp tool <name>`
+- [x] `/ask` (with `history`)
- [x] `/search`
+- [x] `/login`
- [x] `/remote`
+- [x] `/logout`
 - [x] `/config`, `/settings` (with `effective`, `source`, `get`, `set`)
 - [x] `/remotes`
 - [x] `/ssh`
 - [x] `/teleport`
 - [x] `/direct-connect`
 - [x] `/deep-link`
- [x] `/disconnect`
+- [x] `/disconnect`, `/remote-disconnect`
 - [x] `/account`
 - [x] `/login`
 - [x] `/logout`
 - [x] `/resources`
 - [x] `/resource`
- [x] `/plan`
+- [x] `/tasks`, `/todo`
- [x] `/planner`
+- [x] `/workflows`, `/workflow`
- [x] `/tasks`
+- [x] `/triggers`, `/trigger`
- [x] `/todo`
+- [x] `/teams`, `/team`, `/messages`
 - [x] `/task-next`, `/next-task`
 - [x] `/plan`, `/planner`
 - [x] `/task`
- [x] `/task-next`
+- [x] `/prompt`, `/system-prompt`
 - [x] `/prompt`
 - [x] `/system-prompt`
 - [x] `/permissions`
- [x] `/hooks`
+- [x] `/hooks`, `/policy`
 - [x] `/policy`
 - [x] `/trust`
 - [x] `/model`
 - [x] `/tools`
 - [x] `/memory`
- [x] `/status`
+- [x] `/status`, `/session`
 - [x] `/session`
 - [x] `/clear`
 - [x] `/config`
 - [x] `/settings`
-Missing:
+Missing npm slash commands (from `src/commands/` — 80+ commands total):
- [ ] Full npm slash-command surface
+- [ ] `/add-dir` — Add a new working directory
- [x] Slash commands backed by MCP integration
+- [ ] `/agents` — Manage agent configurations
- [ ] Slash commands tied to task/plan systems beyond the current local `/plan`, `/tasks`, and `/task` flows
+- [x] `/branch` — Create a branch of the current conversation
- [ ] Slash commands tied to remote/background sessions beyond the current local remote connect/disconnect and background inspection flows
+- [ ] `/bridge` — Connect for remote-control sessions
- [ ] Slash commands with richer interactive behavior
+- [ ] `/btw` — Quick side question without interrupting main conversation
- [ ] Slash commands tied to plugins and bundled skills
+- [ ] `/chrome` — Chrome extension settings
- [ ] Slash commands tied to account, settings, and auth flows beyond the current local `/account`, `/login`, `/logout`, `/config`, and `/settings` flows
+- [x] `/color` — Set the prompt bar color for this session
 - [x] `/compact` — Clear history but keep a summary in context
 - [x] `/copy` — Copy Claude's last response to clipboard
 - [x] `/cost` — Show total cost and duration of session
 - [ ] `/desktop` — Continue session in Claude Desktop
 - [x] `/diff` — View uncommitted changes and per-turn diffs
 - [x] `/doctor` — Diagnose and verify installation and settings
 - [x] `/effort` — Set effort level for model usage
 - [x] `/exit` — Exit the REPL
 - [x] `/export` — Export conversation to file or clipboard
 - [ ] `/extra-usage` — Configure extra usage for rate limits
 - [ ] `/fast` — Toggle fast mode
 - [ ] `/feedback` — Submit feedback
 - [x] `/files` — List all files currently in context
 - [ ] `/ide` — Manage IDE integrations and show status
 - [ ] `/install-github-app` — Set up GitHub Actions
 - [ ] `/install-slack-app` — Install Slack app
 - [ ] `/keybindings` — Open keybindings config file
 - [ ] `/mobile` — QR code for mobile app
 - [ ] `/output-style` — Change output style
 - [ ] `/passes` — Passes management
 - [ ] `/plugin` — Plugin management
 - [ ] `/pr_comments` — Get comments from a GitHub PR
 - [ ] `/privacy-settings` — View/update privacy settings
 - [ ] `/rate-limit-options` — Show options when rate limited
 - [ ] `/release-notes` — View release notes
 - [ ] `/reload-plugins` — Activate pending plugin changes
 - [ ] `/remote-env` — Configure default remote environment
 - [ ] `/remote-setup` — Remote setup configuration
 - [x] `/rename` — Rename current conversation
 - [ ] `/resume` — Resume a previous conversation
 - [ ] `/rewind` — Restore code/conversation to a previous point
 - [ ] `/sandbox-toggle` — Toggle sandbox mode
 - [ ] `/skills` — List available skills
 - [x] `/stats` — Usage statistics and activity
 - [ ] `/stickers` — Order stickers
 - [x] `/tag` — Toggle a searchable tag on the session
 - [ ] `/theme` — Change the theme
 - [ ] `/upgrade` — Upgrade to Max
 - [ ] `/vim` — Toggle Vim/Normal editing modes
 - [ ] `/voice` — Toggle voice mode
 - [ ] Feature-gated: `/buddy`, `/fork`, `/peers`, `/proactive`, `/torch`, `/workflows` (full), etc.
 - [ ] Internal: `/backfill-sessions`, `/break-cache`, `/bughunter`, `/commit`, `/commit-push-pr`, `/init-verifiers`, `/mock-limits`, `/version`, `/ultraplan`, `/autofix-pr`, etc.
 ## 6. Built-in Tools
-Done:
+### Tools implemented in Python (58 tools):
 - [x] `list_dir`
 - [x] `read_file`
 - [x] `write_file`
 - [x] `edit_file`
 - [x] `notebook_edit`
 - [x] `glob_search`
 - [x] `grep_search`
 - [x] `bash`
@@ -304,7 +375,9 @@ Done:
 - [x] `account_list_profiles`
 - [x] `account_login`
 - [x] `account_logout`
- [x] `notebook_edit`
+- [x] `config_list`
 - [x] `config_get`
 - [x] `config_set`
 - [x] `mcp_list_resources`
 - [x] `mcp_read_resource`
 - [x] `mcp_list_tools`
@@ -313,13 +386,16 @@ Done:
 - [x] `remote_list_profiles`
 - [x] `remote_connect`
 - [x] `remote_disconnect`
- [x] `config_list`
+- [x] `worktree_status`
- [x] `config_get`
+- [x] `worktree_enter`
- [x] `config_set`
+- [x] `worktree_exit`
 - [x] `workflow_list`
 - [x] `workflow_get`
 - [x] `workflow_run`
 - [x] `remote_trigger`
 - [x] `plan_get`
 - [x] `update_plan`
 - [x] `plan_clear`
 - [x] `delegate_agent`
 - [x] `task_next`
 - [x] `task_list`
 - [x] `task_get`
@@ -330,37 +406,53 @@ Done:
 - [x] `task_block`
 - [x] `task_cancel`
 - [x] `todo_write`
 - [x] `delegate_agent`
 - [x] `team_list`
 - [x] `team_get`
 - [x] `team_create`
 - [x] `team_delete`
 - [x] `send_message`
 - [x] `team_messages`
 - [x] `workflow_list`
 - [x] `workflow_get`
 - [x] `workflow_run`
 - [x] `remote_trigger`
 - [x] `worktree_status`
 - [x] `worktree_enter`
 - [x] `worktree_exit`
-Missing:
+### Tools in npm `tools.ts` not yet ported with full fidelity (40 tool dirs):
- [ ] Agent spawning tool parity beyond the current `delegate_agent` runtime tool
+Core tools needing full port:
- [ ] Skill tool
+- [ ] `AgentTool` — Sub-agent spawning with built-in agents (explore, general-purpose, verification, plan, claudeCodeGuide, statusline), fork support, agent memory/snapshots, resume agent, color management
- [ ] Web fetch parity beyond the current local text-fetch implementation
+- [ ] `SkillTool` — Skill execution with bundled skills
- [ ] Web search parity beyond the current provider-backed implementation
+- [ ] `BriefTool` — Brief mode with attachments and file upload
- [ ] LSP tool
+- [ ] `LSPTool` — Language Server Protocol (diagnostics, go-to-definition, references, hover, symbol search, formatting)
- [ ] Tool search parity beyond the current local registry search
+- [ ] `PowerShellTool` — Full PowerShell execution with security, path validation, CLM types, git safety
- [ ] Config tool
+- [ ] `REPLTool` — Interactive REPL with primitive tools (ant-only)
- [ ] Terminal capture tool
+- [ ] `MCPTool` — Full MCP tool execution with collapse classification
- [ ] Browser tool
+- [ ] `McpAuthTool` — MCP authentication handling
- [x] Workflow tool
+- [ ] `ConfigTool` — Full config management with supported settings list
- [x] Remote trigger tool
+- [ ] `SyntheticOutputTool` — Synthetic output injection
- [ ] Sleep / cron tools beyond the current local `sleep` tool
+- [ ] `EnterPlanModeTool` — Enter plan mode with UI
- [ ] PowerShell tool parity
+- [ ] `ExitPlanModeTool` — Exit plan mode with V2 flow
- [x] Worktree enter/exit tools
+- [ ] `EnterWorktreeTool` — Full worktree enter with UI
- [ ] Full `tools.ts` parity
+- [ ] `ExitWorktreeTool` — Full worktree exit with UI
 - [ ] `TaskOutputTool` — Task output display
 - [ ] `TaskStopTool` — Stop a running task
 Feature-gated tools:
 - [ ] `CronCreateTool` / `CronDeleteTool` / `CronListTool` — Cron scheduling (AGENT_TRIGGERS)
 - [ ] `RemoteTriggerTool` — Full remote triggers with UI (AGENT_TRIGGERS_REMOTE)
 - [ ] `MonitorTool` — MCP server monitoring (MONITOR_TOOL)
 - [ ] `SendUserFileTool` — Send file to user (KAIROS)
 - [ ] `PushNotificationTool` — Push notifications (KAIROS)
 - [ ] `SubscribePRTool` — PR subscription (KAIROS_GITHUB_WEBHOOKS)
 - [ ] `SuggestBackgroundPRTool` — Background PR (ant-only)
 - [ ] `VerifyPlanExecutionTool` — Plan verification
 - [ ] `TungstenTool` — Tungsten tool
 - [ ] `WebBrowserTool` — Full web browser
 - [ ] `TerminalCaptureTool` — Terminal capture
 - [ ] `SnipTool` — Force history snipping
 - [ ] `ListPeersTool` — List peers (UDS_INBOX)
 - [ ] `EmbeddedSearchTool` — Embedded search
 - [ ] `CtxInspectTool` — Context inspection
 - [ ] `WorkflowTool` — Workflow scripts (WORKFLOW_SCRIPTS)
 Note: Python has basic tool execution for `bash`, `read_file`, etc., but lacks per-tool UI components, prompt files, constants, and deep security validations (e.g., BashTool has 15 supporting files in npm).
 ## 7. Commands And Task Systems
@@ -383,13 +475,17 @@ Done:
 Missing:
- [ ] Real implementation of the larger upstream command tree
+- [ ] Real implementation of the larger upstream command tree (80+ commands)
 - [ ] Task types: `LocalShellTask`, `LocalAgentTask`, `RemoteAgentTask`, `DreamTask`, `LocalWorkflowTask`, `MonitorMcpTask`, `InProcessTeammateTask`
 - [ ] Task stall detection (45s threshold) and prompt detection for interactive input
 - [ ] Remote agent task session URL tracking and completion checkers
 - [ ] Dream/auto-consolidation task with file tracking and turn history
 - [ ] Task orchestration system beyond the current local dependency-aware task runtime
 - [ ] Planner / task execution parity beyond the current local plan persistence, sync, and next-task flow
 - [ ] Team / collaboration command flows beyond the current local team runtime and message recording flows
 - [ ] Command-specific session behaviors
- [ ] Full `src/commands/*` parity
+- [ ] Full `src/commands/*` parity (80+ command directories)
- [ ] Full `src/tasks/*` parity
+- [ ] Full `src/tasks/*` parity (7 task types)
 ## 8. Permissions, Hooks, And Policy
@@ -410,9 +506,13 @@ Done:
 Missing:
- [ ] Tool-permission workflow parity
+- [x] Full BashTool security: `bashSecurity.ts`, `sedValidation.ts`, `sedEditParser.ts`, `pathValidation.ts`, `readOnlyValidation.ts`, `modeValidation.ts`, `commandSemantics.ts`, `destructiveCommandWarning.ts`, `shouldUseSandbox.ts` → `src/bash_security.py` (18 validators, destructive warnings, command semantics, read-only detection, 163 tests)
 - [ ] Full PowerShellTool security: `powershellSecurity.ts`, `gitSafety.ts`, `clmTypes.ts`
 - [ ] Tool-permission workflow parity (`bashPermissions.ts`, `powershellPermissions.ts`)
 - [ ] Trust-gated initialization
- [ ] Hook-config management
+- [ ] Hook-config management (`schemas/hooks.ts` with Zod schemas)
 - [ ] Policy limits service (`services/policyLimits/`)
 - [ ] Remote managed settings (`services/remoteManagedSettings/`)
 - [ ] Full hooks and policy parity
 ## 9. MCP, Plugins, And Skills
@@ -429,12 +529,13 @@ Done:
 Missing:
- [ ] Full MCP-backed tool parity beyond the current stdio resource/tool list/read/call support
+- [ ] Full MCP service (`services/mcp/` — 25+ files: InProcessTransport, MCPConnectionManager, SdkControlTransport, auth, channelAllowlist, channelPermissions, client, config, elicitationHandler, envExpansion, normalization, oauthPort, officialRegistry, vscodeSdkMcp, xaa, xaaIdpLogin, etc.)
- [ ] Plugin discovery and loading
+- [ ] MCP server approval dialogs (`services/mcpServerApproval.tsx`)
- [ ] Bundled plugin support
+- [ ] Plugin discovery, loading, and installation (`services/plugins/PluginInstallationManager.ts`, `pluginCliCommands.ts`, `pluginOperations.ts`)
 - [ ] Bundled plugin support (`plugins/bundledPlugins.ts`, `plugins/bundled/`)
 - [ ] Plugin lifecycle management
 - [ ] Plugin update/cache behavior
- [ ] Skill discovery and execution parity
+- [ ] Skill discovery and execution (`skills/bundledSkills.ts`, `skills/loadSkillsDir.ts`, `skills/mcpSkillBuilders.ts`, `skills/bundled/`)
 - [ ] Bundled skill support
 - [ ] Full plugin and skill parity
@@ -448,14 +549,22 @@ Done:
 Missing:
- [ ] Interactive REPL parity beyond the current basic `agent-chat` loop
+- [ ] Interactive REPL parity (`screens/REPL.tsx`)
- [ ] Ink/TUI component parity
+- [ ] Ink/TUI framework (`ink/` — 40+ files: custom renderer, reconciler, DOM, layout engine, text wrapping, ANSI handling, focus management, selection)
- [ ] Screen system parity
+- [ ] Screen system (`screens/Doctor.tsx`, `screens/ResumeConversation.tsx`)
 - [ ] Component library (`components/` — 100+ components in 12+ subdirectories):
  - Message rendering: Message, MessageRow, Messages, MessageSelector, MessageResponse
  - Dialogs: ApproveApiKey, AutoModeOptIn, Bridge, CostThreshold, IdeAutoConnect, MCPServerApproval
  - Settings: ThemePicker, LanguagePicker, ModelPicker, OutputStylePicker
  - Search: GlobalSearchDialog, QuickOpenDialog, HistorySearchDialog
  - Status: AgentProgressLine, BashModeProgress, MemoryUsageIndicator, TokenWarning
  - Design system, agent, team, task, skill, memory, permissions, sandbox, shell components
 - [ ] Keyboard interaction parity
 - [ ] Interactive status panes
 - [ ] Approval UI flows
 - [ ] Rich incremental rendering
- [ ] Full `components`, `screens`, and `ink` parity
+- [ ] Virtual scrolling
 - [ ] Copy-on-select behavior
 ## 11. Remote, Background, And Team Features
@@ -470,12 +579,12 @@ Done:
 Missing:
- [ ] Real remote execution modes beyond the current local manifest-backed remote runtime and CLI/profile flows
+- [ ] Real remote session management (`remote/` — 4 files: RemoteSessionManager, SessionsWebSocket, remotePermissionBridge, sdkMessageAdapter)
- [ ] Team runtime features
+- [ ] Bridge subsystem (`bridge/` — 30+ files: bridgeMain, bridgeApi, bridgeConfig, bridgeMessaging, bridgePermissionCallbacks, replBridge, replBridgeHandle, replBridgeTransport, sessionRunner, trustedDevice, jwtUtils, capacityWake, inboundAttachments, inboundMessages, etc.)
- [ ] Team messaging features
+- [ ] Direct connect subsystem (`server/createDirectConnectSession.ts`, `directConnectManager.ts`)
 - [ ] Real team collaboration beyond local recording
 - [ ] Shared remote state
- [ ] Upstream proxy runtime integration
+- [ ] Upstream proxy (`upstreamproxy/upstreamproxy.ts`, `upstreamproxy/relay.ts`)
 - [ ] Full `remote`, `server`, `bridge`, `upstreamproxy`, and team parity
 ## 12. Editor, Platform, And Native Integrations
@@ -485,14 +594,16 @@ Done:
 Missing:
- [ ] Voice mode parity
+- [ ] Voice mode (`voice/`, `services/voice.ts`, `services/voiceKeyterms.ts`, `services/voiceStreamSTT.ts`, hooks)
- [ ] VIM mode parity
+- [ ] VIM mode (`vim/` — 5 files: motions, operators, textObjects, transitions, types)
- [ ] Keybinding parity
+- [ ] Keybinding system (`keybindings/` — 13 files: defaultBindings, loadUserBindings, match, parser, resolver, schema, template, validate, etc.)
- [ ] Notification hooks
+- [ ] Notification hooks (`services/notifier.ts`, `services/preventSleep.ts`)
- [ ] Native TypeScript / platform helper parity
+- [ ] Native TypeScript / platform helpers (`native-ts/`)
- [ ] JetBrains/editor integration parity
+- [ ] JetBrains/editor integration (`utils/jetbrains.ts`, `utils/ide.ts`, `utils/idePathConversion.ts`)
 - [ ] Browser/native host integrations
 - [ ] IDE integration hooks (useIDEIntegration, useIdeAtMentioned, useIdeSelection, useIdeLogging, useDiffInIDE, useLspPluginRecommendation)
 - [ ] Platform-specific startup/shutdown logic
 - [ ] Chrome extension integration
 ## 13. Services And Internal Subsystems
@@ -503,62 +614,196 @@ Done:
 Missing:
- [ ] Real service implementations for the mirrored `services` package
+- [ ] Analytics service (`services/analytics/` — 10+ files: config, Datadog, Growthbook, first-party event logger, sink, killswitch)
- [ ] Config service parity
+- [ ] API service (`services/api/` — 20+ files: claude client, dumpPrompts, errorUtils, filesApi, firstTokenDate, grove, logging, metricsOptOut, promptCacheBreakDetection, sessionIngress, usage, withRetry, etc.)
- [ ] Account/auth service parity
+- [ ] LSP service (`services/lsp/` — 7 files: LSPClient, LSPDiagnosticRegistry, LSPServerInstance, LSPServerManager, config, manager, passiveFeedback)
- [ ] Analytics/telemetry service parity
+- [ ] Tools service (`services/tools/` — 4 files: StreamingToolExecutor, toolExecution, toolHooks, toolOrchestration)
- [ ] Growthbook/feature-flag parity
+- [ ] Compact service (`services/compact/` — 6 files: compact, autoCompact, microCompact, apiMicrocompact, sessionMemoryCompact, compactWarningHook)
- [ ] GitHub / git helper parity
+- [ ] Auto-dream service (`services/autoDream/` — 4 files: autoDream, config, consolidationLock, consolidationPrompt)
- [ ] Sandbox/settings utility parity
+- [ ] Agent summary service (`services/AgentSummary/`)
- [ ] Todo/task utility parity
+- [ ] Magic docs service (`services/MagicDocs/`)
- [ ] Internal helpers used by the upstream runtime
+- [ ] Session memory service (`services/SessionMemory/`)
 - [ ] Prompt suggestion service (`services/PromptSuggestion/`)
 - [ ] Extract memories service (`services/extractMemories/`)
 - [ ] Diagnostic tracking service (`services/diagnosticTracking.ts`)
 - [ ] OAuth service (`services/oauth/` — 5 files)
 - [ ] Rate limiting (`services/claudeAiLimits.ts`, `services/rateLimitMessages.ts`, etc.)
 - [ ] Settings sync (`services/settingsSync/`)
 - [ ] Tips service (`services/tips/`)
 - [ ] Tool use summary service (`services/toolUseSummary/`)
 - [ ] VCR playback (`services/vcr.ts`)
 - [ ] Internal/container logging (`services/internalLogging.ts`)
 - [ ] Plugin installation management (`services/plugins/`)
-## 14. Mirrored Workspace Versus Working Runtime
+## 14. State Management
-Working Python runtime today:
+Done:
- [x] `src/main.py`
+- [x] Session state via `AgentSessionState` dataclass
- [x] `src/agent_runtime.py`
+- [x] Basic session persistence
 - [x] `src/agent_tools.py`
 - [x] `src/agent_prompting.py`
 - [x] `src/agent_context.py`
 - [x] `src/agent_context_usage.py`
 - [x] `src/agent_session.py`
 - [x] `src/agent_slash_commands.py`
 - [x] `src/account_runtime.py`
 - [x] `src/config_runtime.py`
 - [x] `src/agent_types.py`
 - [x] `src/mcp_runtime.py`
 - [x] `src/plan_runtime.py`
 - [x] `src/plugin_runtime.py`
 - [x] `src/remote_runtime.py`
 - [x] `src/search_runtime.py`
 - [x] `src/hook_policy.py`
 - [x] `src/background_runtime.py`
 - [x] `src/task.py`
 - [x] `src/task_runtime.py`
 - [x] `src/tokenizer_runtime.py`
 - [x] `src/openai_compat.py`
 - [x] `src/session_store.py`
 - [x] `src/permissions.py`
-Mirrored inventory / scaffold areas that still need real implementation work:
+Missing:
- [ ] `src/commands.py`
+- [ ] Zustand store (`state/AppStateStore.ts`, `state/store.ts`)
- [ ] `src/tools.py`
+- [ ] Store selectors (`state/selectors.ts`)
- [ ] `src/query_engine.py`
+- [ ] State change callbacks (`state/onChangeAppState.ts`)
- [ ] `src/runtime.py`
+- [ ] React context providers (`state/AppState.tsx`)
 - [ ] Remaining mirrored inventory surfaces still represented mainly by snapshot data under `src/reference_data/*`
 - [ ] Command/task/plugin/skill/service/editor subsystems that exist upstream but do not yet have real Python modules after the tree cleanup
-## 15. High-Priority Next Steps
+## 15. React Hooks (84+ hooks in `src/hooks/`)
- [ ] Expand the real Python tool registry toward upstream `tools.ts`
+Not applicable for Python (no React TUI), but these represent features needing alternative implementations:
- [ ] Replace more snapshot-backed mirrored modules with working runtime code
+
- [ ] Expand MCP parity beyond the current stdio resource/tool transport support
+- [ ] File suggestions and unified suggestions
- [ ] Expand hooks and policy parity beyond the current manifest/runtime implementation
+- [ ] Remote session / SSH / direct connect hooks
- [ ] Build a real interactive REPL / TUI
+- [ ] Input buffer, text input, vim input, typeahead, search input, paste handling
- [ ] Expand background session parity beyond the current local worker/log/attach model
+- [ ] Arrow key history, history search, background task navigation
- [ ] Add real remote session transport and shared remote state beyond the current local remote-profile runtime
+- [ ] Main loop model selection, assistant history, merged clients/commands/tools
- [ ] Port more of the command/task system
+- [ ] Tool permission checking, cancel request, manage plugins
- [ ] Close the gap between the mirrored workspace and the working runtime
+- [ ] Global/command keybindings, exit handling, double-press detection
 - [ ] Terminal size, virtual scroll, copy-on-select
 - [ ] Voice recording, voice integration
 - [ ] IDE integration, @mention, selection, diff-in-IDE
 - [ ] Settings management, dynamic config
 - [ ] Timeout, elapsed time, scheduled tasks, delayed notifications
 - [ ] Prompt suggestion, update notification, feature hints
 - [ ] Queue processor, command queue
 - [ ] Memory usage, away summary, teleport resume
 - [ ] Diff data, turn diffs
 - [ ] Task list watcher, tasks v2, PR status
 - [ ] Session backgrounding, swarm initialization/permission
 - [ ] API key verification, mailbox bridge, inbox poller
 ## 16. Utilities (200+ files in `src/utils/`)
 Done:
 - [x] Basic file operations in tool implementations
 - [x] Basic git status snapshot
 - [x] Basic shell/subprocess handling
 Missing major utility categories:
 - [ ] Shell utilities (`utils/bash/`, `utils/shell/`, `Shell.ts`, `ShellCommand.ts`)
 - [ ] Git operations (`utils/git.ts`, `utils/gitDiff.ts`, `utils/gitSettings.ts`, `utils/commitAttribution.ts`)
 - [ ] File operations (`utils/file.ts`, `utils/fileRead.ts`, `utils/fileHistory.ts`, `utils/fileStateCache.ts`, `utils/fsOperations.ts`, `utils/ripgrep.ts`, `utils/glob.ts`)
 - [ ] AI/Model utilities (`utils/modelCost.ts`, `utils/model/`, `utils/context.ts`, `utils/queryContext.ts`)
 - [ ] Config/Settings (`utils/config.ts`, `utils/settings/`)
 - [ ] Message handling (`utils/messages.ts`, `utils/messages/`, `utils/messageQueueManager.ts`)
 - [ ] API/Network (`utils/api.ts`, `utils/http.ts`, `utils/proxy.ts`, `utils/auth.ts`)
 - [ ] Session management (`utils/sessionStorage.ts`, `utils/sessionState.ts`, `utils/sessionStart.ts`, `utils/sessionRestore.ts`)
 - [ ] Plugin/Skill utilities (`utils/plugins/`, `utils/skills/`)
 - [ ] Memory/Context (`utils/memory/`, `utils/claudemd.ts`, `utils/contextAnalysis.ts`)
 - [ ] IDE integration (`utils/ide.ts`, `utils/jetbrains.ts`)
 - [ ] Platform/OS (`utils/platform.ts`, `utils/terminal.ts`, `utils/systemDirectories.ts`)
 - [ ] Debugging (`utils/debug.ts`, `utils/diagLogs.ts`, `utils/log.ts`, `utils/profilerBase.ts`)
 - [ ] Telemetry (`utils/telemetry/`)
 - [ ] Deep link utilities (`utils/deepLink/`)
 ## 17. Coordinator And Buddy
 Missing:
 - [ ] Coordinator mode (`coordinator/coordinatorMode.ts` — agent tool filtering and async agent allowlist)
 - [ ] Buddy/companion system (`buddy/` — 6 files: CompanionSprite, companion procedural generation, personality prompts, sprites, types, notification UI)
 ## 18. Migrations
 Missing:
 - [ ] Data/config migration system (`migrations/` — 11 migration scripts):
  - Model migrations: migrateFennecToOpus, migrateLegacyOpusToCurrent, migrateOpusToOpus1m, migrateSonnet1mToSonnet45, migrateSonnet45ToSonnet46
  - Feature migrations: migrateAutoUpdatesToSettings, migrateBypassPermissionsAcceptedToSettings, migrateEnableAllProjectMcpServersToSettings, migrateReplBridgeEnabledToRemoteControlAtStartup
  - Config resets: resetAutoModeOptInForDefaultOffer, resetProToOpusDefault
 ## 19. Type Definitions
 Missing:
 - [ ] Full type system from `types/` (command.ts, hooks.ts, ids.ts, logs.ts, permissions.ts, plugin.ts, textInputTypes.ts, generated/)
 ## 20. Mirrored Workspace Versus Working Runtime
 Working Python runtime today (21,193 lines across 51 source files, 10,480 lines across 37 test files):
 - [x] `src/main.py` (1,353 lines)
 - [x] `src/agent_runtime.py` (3,664 lines)
 - [x] `src/agent_tools.py` (2,994 lines)
 - [x] `src/agent_prompting.py` (390 lines)
 - [x] `src/agent_context.py` (459 lines)
 - [x] `src/agent_context_usage.py` (356 lines)
 - [x] `src/agent_session.py` (718 lines)
 - [x] `src/agent_slash_commands.py` (633 lines)
 - [x] `src/agent_manager.py` (296 lines)
 - [x] `src/agent_plugin_cache.py` (154 lines)
 - [x] `src/agent_types.py` (193 lines)
 - [x] `src/account_runtime.py` (470 lines)
 - [x] `src/ask_user_runtime.py` (320 lines)
 - [x] `src/background_runtime.py` (371 lines)
 - [x] `src/config_runtime.py` (296 lines)
 - [x] `src/hook_policy.py` (339 lines)
 - [x] `src/mcp_runtime.py` (880 lines)
 - [x] `src/openai_compat.py` (413 lines)
 - [x] `src/permissions.py` (20 lines)
 - [x] `src/plan_runtime.py` (396 lines)
 - [x] `src/plugin_runtime.py` (654 lines)
 - [x] `src/query_engine.py` (655 lines)
 - [x] `src/remote_runtime.py` (571 lines)
 - [x] `src/remote_trigger_runtime.py` (371 lines)
 - [x] `src/search_runtime.py` (606 lines)
 - [x] `src/session_store.py` (295 lines)
 - [x] `src/task.py` (130 lines)
 - [x] `src/task_runtime.py` (595 lines)
 - [x] `src/team_runtime.py` (386 lines)
 - [x] `src/tokenizer_runtime.py` (202 lines)
 - [x] `src/workflow_runtime.py` (319 lines)
 - [x] `src/worktree_runtime.py` (448 lines)
 - [x] Plus 19 supporting modules
 Mirrored / scaffold areas needing real implementation:
 - [ ] `src/commands.py` — currently minimal dispatch, needs full command tree
 - [ ] `src/tools.py` — reference-data based tool loading, needs real per-tool implementations
 - [ ] `src/query_engine.py` — facade layer, needs full QueryEngine.ts parity
 - [ ] `src/runtime.py` — routing layer, needs full runtime parity
 - [ ] Remaining inventory surfaces under `src/reference_data/*`
 ---
 ## High-Priority Next Steps
 ### Tier 1 — Core Feature Gaps (highest user impact)
 - [x] Full BashTool security parity (sed validation, path validation, sandbox, destructive command warnings, command semantics) → `src/bash_security.py`
 - [ ] LSP tool integration for code intelligence
 - [ ] Full AgentTool with built-in agent types (explore, general-purpose, verification, plan)
 - [ ] Auto-compact and context collapse from `query.ts`
 - [ ] Full compact service (autoCompact, microCompact, sessionMemoryCompact)
 - [ ] Interactive REPL improvements
 ### Tier 2 — Important Feature Gaps
 - [ ] SkillTool with bundled skills
 - [ ] Full MCP service parity (auth, permissions, config, registry)
 - [ ] Plugin discovery, loading, and installation
 - [ ] Real remote session management (WebSocket, bridge)
 - [ ] Full command tree implementation (80+ commands)
 - [ ] Migration system for config/model upgrades
 - [ ] Token budget calculations
 ### Tier 3 — Nice-to-Have Features
 - [ ] TUI/Ink component library
 - [ ] Voice mode
 - [ ] VIM mode and keybinding system
 - [ ] IDE integrations (JetBrains, VS Code)
 - [ ] Chrome extension integration
 - [ ] Buddy/companion system
 - [ ] Analytics/telemetry
 - [ ] Coordinator mode
 - [ ] Feature flag system (Growthbook)
 ### Tier 4 — Platform/Enterprise Features
 - [ ] Full bridge subsystem (30+ files)
 - [ ] Upstream proxy
 - [ ] Direct connect server
 - [ ] OAuth service
 - [ ] Settings sync
 - [ ] Rate limiting and policy limits
 - [ ] Dream/auto-consolidation service
@@ -37,11 +37,18 @@ python3 -m benchmarks.run_suite --all -o results.json
 | **SWE-Bench** | Coding | 5 | Resolve real-world GitHub issues |
 | **Aider** | Coding | 6 | Code editing and refactoring tasks |
 | **LiveCodeBench** | Coding | 5 | Competitive programming problems |
 | **Codeforces** | Coding | 10 | Competitive programming with ELO rating |
 | **MATH** | Math | 15 | Competition mathematics problems |
 | **GSM8K** | Math | 15 | Grade school math word problems |
 | **AIME** | Math | 10 | Challenging competition math (integers 0–999) |
 | **IFEval** | Instruction Following | 10 | Verifiable instruction-following evaluation |
 | **BFCL** | Instruction Following | 7 | Function/tool calling evaluation |
 | **MMLU-Pro** | Knowledge | 10 | Professional-level multiple-choice QA (10 choices) |
 | **GPQA-Diamond** | Knowledge | 10 | Graduate-level science QA (diamond difficulty) |
 | **MMMLU** | Knowledge | 10 | Multilingual MMLU across languages |
 | **HLE** | Knowledge | 10 | Humanity's Last Exam (extremely hard) |
 | **BigBench-Hard** | Reasoning | 10 | BIG-Bench Extra Hard reasoning tasks |
 | **Tau2** | Reasoning | 10 | Tool-augmented reasoning (retail/airline/finance) |
 ### Commands
@@ -60,12 +67,23 @@ python3 -m benchmarks.run_suite --suite mbpp
 python3 -m benchmarks.run_suite --suite swe-bench
 python3 -m benchmarks.run_suite --suite aider
 python3 -m benchmarks.run_suite --suite livecodebench
 python3 -m benchmarks.run_suite --suite codeforces
 # Math benchmarks
 python3 -m benchmarks.run_suite --suite math
 python3 -m benchmarks.run_suite --suite gsm8k
 python3 -m benchmarks.run_suite --suite aime
 # Knowledge benchmarks
 python3 -m benchmarks.run_suite --suite mmlu-pro
 python3 -m benchmarks.run_suite --suite gpqa-diamond
 python3 -m benchmarks.run_suite --suite mmmlu
 python3 -m benchmarks.run_suite --suite hle
 # Reasoning benchmarks
 python3 -m benchmarks.run_suite --suite bigbench-hard
 python3 -m benchmarks.run_suite --suite tau2
 # Instruction following benchmarks
 python3 -m benchmarks.run_suite --suite ifeval
 python3 -m benchmarks.run_suite --suite bfcl
@@ -74,13 +92,19 @@ python3 -m benchmarks.run_suite --suite bfcl
 #### Run by Category
 ```bash
-# All coding benchmarks (~51 problems)
+# All coding benchmarks
 python3 -m benchmarks.run_suite --category coding
-# All math benchmarks (~40 problems)
+# All math benchmarks
 python3 -m benchmarks.run_suite --category math
-# All instruction following benchmarks (~17 problems)
+# All knowledge benchmarks (MMLU-Pro, GPQA, MMMLU, HLE)
 python3 -m benchmarks.run_suite --category knowledge
 # All reasoning benchmarks (BigBench-Hard, Tau2)
 python3 -m benchmarks.run_suite --category reasoning
 # All instruction following benchmarks
 python3 -m benchmarks.run_suite --category instruction-following
 ```
@@ -143,42 +167,41 @@ Each suite looks for a JSONL file in the data directory. If the file isn't found
 | SWE-Bench | `swe_bench.jsonl` | `{"instance_id", "problem_statement", "setup_code", "test_cmd"}` |
 | Aider | `aider.jsonl` | `{"id", "instruction", "setup_code", "test_code"}` |
 | LiveCodeBench | `livecodebench.jsonl` | `{"id", "title", "description", "test_cases", "function_name"}` |
 | Codeforces | `codeforces.jsonl` | `{"id", "rating", "title", "problem", "test_cases"}` |
 | MATH | `math.jsonl` | `{"id", "problem", "answer", "subject", "level"}` |
 | GSM8K | `gsm8k.jsonl` | `{"id", "question", "answer"}` |
-| AIME | `aime.jsonl` | `{"id", "problem", "answer"}` |
+| AIME | `aime.jsonl` or `aime_2026.jsonl` | `{"id", "problem", "answer"}` |
 | IFEval | `ifeval.jsonl` | `{"id", "instruction", "checks"}` |
 | BFCL | `bfcl.jsonl` | `{"id", "instruction", "expected_function", "setup_code", "test_code"}` |
 | MMLU-Pro | `mmlu_pro.jsonl` | `{"id", "subject", "question", "choices", "answer"}` |
 | GPQA-Diamond | `gpqa.jsonl` | `{"id", "subject", "question", "choices", "answer"}` |
 | MMMLU | `mmmlu.jsonl` | `{"id", "language", "subject", "question", "choices", "answer"}` |
 | HLE | `hle.jsonl` | `{"id", "subject", "question", "answer", "answer_type"}` |
 | BigBench-Hard | `bigbench_hard.jsonl` | `{"id", "task", "question", "choices", "answer"}` |
 | Tau2 | `tau2.jsonl` | `{"id", "domain", "question", "choices", "answer"}` |
 ### Downloading Full Datasets
 ```bash
-# HumanEval (from OpenAI)
+# Download all datasets (builtin + official where available)
-wget -O benchmarks/data/humaneval.jsonl \
+python3 -m benchmarks.download_datasets --all
  https://raw.githubusercontent.com/openai/human-eval/master/data/HumanEval.jsonl.gz
 gunzip benchmarks/data/humaneval.jsonl.gz
-# GSM8K (from HuggingFace — requires datasets library)
+# Download builtin-only (no network, uses embedded problems)
-pip install datasets
+python3 -m benchmarks.download_datasets --all --builtin-only
 python3 -c "
 from datasets import load_dataset
 import json
 ds = load_dataset('gsm8k', 'main', split='test')
 with open('benchmarks/data/gsm8k.jsonl', 'w') as f:
    for i, item in enumerate(ds):
        f.write(json.dumps({'id': f'gsm8k-{i}', 'question': item['question'], 'answer': item['answer'].split('####')[-1].strip()}) + '\n')
 "
-# MATH (from HuggingFace)
+# Download specific suites
-python3 -c "
+python3 -m benchmarks.download_datasets --suite humaneval --suite mmlu-pro --suite gpqa-diamond
-from datasets import load_dataset
+
-import json
+# Force re-download
-ds = load_dataset('hendrycks/competition_math', split='test')
+python3 -m benchmarks.download_datasets --all --force
 with open('benchmarks/data/math.jsonl', 'w') as f:
    for i, item in enumerate(ds):
        f.write(json.dumps({'id': f'math-{i}', 'problem': item['problem'], 'answer': item['solution'], 'subject': item['type'], 'level': item['level']}) + '\n')
 "
 ```
 **Datasets with HuggingFace downloaders:**
 HumanEval, GSM8K, MBPP, MATH, MMLU-Pro, GPQA-Diamond, BigBench-Hard, MMMLU, HLE
 **Builtin-only datasets (no HuggingFace source):**
 SWE-Bench, Aider, LiveCodeBench, AIME, IFEval, BFCL, Tau2, Codeforces
 ---
 ## Local Task Benchmarks
@@ -221,16 +244,25 @@ python3 -m benchmarks.run_suite --suite mbpp               # Coding: basic Pytho
 python3 -m benchmarks.run_suite --suite swe-bench          # Coding: GitHub issues
 python3 -m benchmarks.run_suite --suite aider              # Coding: code editing
 python3 -m benchmarks.run_suite --suite livecodebench      # Coding: competitive programming
 python3 -m benchmarks.run_suite --suite codeforces         # Coding: competitive + ELO
 python3 -m benchmarks.run_suite --suite math               # Math: competition math
 python3 -m benchmarks.run_suite --suite gsm8k              # Math: grade school
 python3 -m benchmarks.run_suite --suite aime               # Math: AIME competition
 python3 -m benchmarks.run_suite --suite mmlu-pro           # Knowledge: 14 subjects, 10 choices
 python3 -m benchmarks.run_suite --suite gpqa-diamond       # Knowledge: graduate science
 python3 -m benchmarks.run_suite --suite mmmlu              # Knowledge: multilingual MMLU
 python3 -m benchmarks.run_suite --suite hle                # Knowledge: Humanity's Last Exam
 python3 -m benchmarks.run_suite --suite bigbench-hard      # Reasoning: BIG-Bench Hard
 python3 -m benchmarks.run_suite --suite tau2               # Reasoning: tool-augmented
 python3 -m benchmarks.run_suite --suite ifeval             # Instruction: format following
 python3 -m benchmarks.run_suite --suite bfcl               # Instruction: function calling
 # Category runs
-python3 -m benchmarks.run_suite --category coding                  # All coding (~51 problems)
+python3 -m benchmarks.run_suite --category coding                  # All coding
-python3 -m benchmarks.run_suite --category math                    # All math (~40 problems)
+python3 -m benchmarks.run_suite --category math                    # All math
-python3 -m benchmarks.run_suite --category instruction-following   # All IF (~17 problems)
+python3 -m benchmarks.run_suite --category knowledge               # MMLU-Pro, GPQA, MMMLU, HLE
 python3 -m benchmarks.run_suite --category reasoning               # BigBench, Tau2
 python3 -m benchmarks.run_suite --category instruction-following   # IFEval, BFCL
 # Full run
 python3 -m benchmarks.run_suite --all                              # All suites (~108 problems)
@@ -311,6 +343,7 @@ benchmarks/
 ├── __init__.py
 ├── run.py                  # Local task benchmark runner
 ├── run_suite.py            # Standard evaluation suite runner (CLI)
 ├── download_datasets.py    # Dataset downloader (HuggingFace + builtins)
 ├── README.md               # This file
 ├── data/                   # Dataset files (JSONL) — not committed
 │   └── .gitkeep
@@ -325,9 +358,16 @@ benchmarks/
    ├── swe_bench.py        # SWE-Bench benchmark
    ├── aider.py            # Aider benchmark
    ├── livecodebench.py    # LiveCodeBench benchmark
    ├── codeforces.py       # Codeforces (ELO scoring)
    ├── math_bench.py       # MATH benchmark
    ├── gsm8k.py            # GSM8K benchmark
    ├── aime.py             # AIME benchmark
    ├── mmlu_pro.py         # MMLU-Pro (10-choice QA)
    ├── gpqa.py             # GPQA Diamond (science)
    ├── mmmlu.py            # MMMLU (multilingual)
    ├── hle.py              # HLE (Humanity's Last Exam)
    ├── bigbench.py         # BigBench Extra Hard
    ├── tau2.py             # Tau2 (tool-augmented)
    ├── ifeval.py           # IFEval benchmark
    └── bfcl.py             # BFCL benchmark
 ```
@@ -1,6 +1,9 @@
 #!/usr/bin/env python3
 """
 Download or export benchmark datasets into benchmarks/data.
 Uses the HuggingFace `datasets` library for reliable full downloads.
 Falls back to the REST API or builtins if `datasets` is not installed.
 """
 from __future__ import annotations
@@ -18,19 +21,27 @@ from typing import Any, Callable
 from benchmarks.suites.aider import _BUILTIN_PROBLEMS as _AIDER_BUILTINS
 from benchmarks.suites.aime import _BUILTIN_PROBLEMS as _AIME_BUILTINS
 from benchmarks.suites.bfcl import _BUILTIN_PROBLEMS as _BFCL_BUILTINS
 from benchmarks.suites.bigbench import _BUILTIN_PROBLEMS as _BIGBENCH_BUILTINS
 from benchmarks.suites.codeforces import _BUILTIN_PROBLEMS as _CODEFORCES_BUILTINS
 from benchmarks.suites.gpqa import _BUILTIN_PROBLEMS as _GPQA_BUILTINS
 from benchmarks.suites.gsm8k import _BUILTIN_PROBLEMS as _GSM8K_BUILTINS
 from benchmarks.suites.hle import _BUILTIN_PROBLEMS as _HLE_BUILTINS
 from benchmarks.suites.humaneval import _BUILTIN_PROBLEMS as _HUMANEVAL_BUILTINS
 from benchmarks.suites.ifeval import _BUILTIN_PROBLEMS as _IFEVAL_BUILTINS
 from benchmarks.suites.livecodebench import _BUILTIN_PROBLEMS as _LIVECODEBENCH_BUILTINS
 from benchmarks.suites.math_bench import _BUILTIN_PROBLEMS as _MATH_BUILTINS
 from benchmarks.suites.mbpp import _BUILTIN_PROBLEMS as _MBPP_BUILTINS
 from benchmarks.suites.mmmlu import _BUILTIN_PROBLEMS as _MMMLU_BUILTINS
 from benchmarks.suites.mmlu_pro import _BUILTIN_PROBLEMS as _MMLU_PRO_BUILTINS
 from benchmarks.suites.swe_bench import _BUILTIN_PROBLEMS as _SWE_BUILTINS
 from benchmarks.suites.tau2 import _BUILTIN_PROBLEMS as _TAU2_BUILTINS
 HF_DATASET_VIEWER_BASE = "https://datasets-server.huggingface.co"
 HUMANEVAL_GZ_URL = "https://raw.githubusercontent.com/openai/human-eval/master/data/HumanEval.jsonl.gz"
 DEFAULT_DATA_DIR = Path(__file__).resolve().parent / "data"
 # Legacy REST API support (fallback only)
 HF_DATASET_VIEWER_BASE = "https://datasets-server.huggingface.co"
 JsonFetcher = Callable[[str, dict[str, object], dict[str, str] | None, float], object]
@@ -67,33 +78,45 @@ def _write_jsonl(path: Path, rows: list[dict[str, Any]]) -> int:
    path.parent.mkdir(parents=True, exist_ok=True)
    with open(path, "w", encoding="utf-8") as handle:
        for row in rows:
-            handle.write(json.dumps(row, ensure_ascii=True) + "\n")
+            handle.write(json.dumps(row, ensure_ascii=False) + "\n")
    return len(rows)
-def _extract_gsm8k_answer(text: str) -> str:
+# ---------------------------------------------------------------------------
-    if "####" in text:
+# HuggingFace `datasets` library helpers
-        text = text.split("####", 1)[1]
+# ---------------------------------------------------------------------------
-    numbers = re.findall(r"-?\d[\d,]*\.?\d*", text.replace("$", ""))
+
-    if numbers:
+def _load_hf_dataset(
-        return numbers[-1].replace(",", "")
+    dataset_name: str,
-    return text.strip().replace(",", "")
+    config: str | None = None,
    split: str = "test",
 ) -> list[dict[str, Any]]:
    """Load a dataset using the HuggingFace `datasets` library."""
    from datasets import load_dataset  # type: ignore[import-untyped]
    kwargs: dict[str, Any] = {}
    if config:
        kwargs["name"] = config
    ds = load_dataset(dataset_name, split=split, **kwargs)
    return [dict(row) for row in ds]  # type: ignore[union-attr]
-def _extract_math_answer(solution: str) -> str:
+def _try_load_hf(
-    boxed_fraction = re.search(r"\\boxed\{\\frac\{([^}]+)\}\{([^}]+)\}\}", solution, flags=re.DOTALL)
+    dataset_name: str,
-    if boxed_fraction:
+    config: str | None = None,
-        return f"{boxed_fraction.group(1).strip()}/{boxed_fraction.group(2).strip()}"
+    split_preference: tuple[str, ...] = ("test", "validation", "train"),
-    boxed = re.search(r"\\boxed\{([^{}]+)\}", solution, flags=re.DOTALL)
+) -> list[dict[str, Any]]:
-    value = boxed.group(1) if boxed else solution
+    """Try loading with preferred splits, falling back through the list."""
-    value = value.strip()
+    for split in split_preference:
-    value = value.replace("\\frac{", "").replace("}{", "/").replace("}", "")
+        try:
-    value = value.replace("$", "").replace(",", "").strip()
+            rows = _load_hf_dataset(dataset_name, config=config, split=split)
-    fraction = re.search(r"-?\d+\s*/\s*-?\d+", value)
+            if rows:
-    if fraction:
+                print(f"    Loaded {len(rows)} rows from {dataset_name} [{split}]")
-        return fraction.group(0).replace(" ", "")
+                return rows
-    numbers = re.findall(r"-?\d+(?:/\d+)?(?:\.\d+)?", value)
+        except (ValueError, KeyError):
-    return numbers[-1] if numbers else value
+            continue
    raise ValueError(f"No valid split found for {dataset_name}")
 def _fetch_hf_rows(
@@ -105,6 +128,7 @@ def _fetch_hf_rows(
    timeout: float = 60.0,
    headers: dict[str, str] | None = None,
 ) -> list[dict[str, Any]]:
    """Legacy REST-API fetcher (kept for backward compatibility with tests)."""
    splits_payload = json_fetcher("splits", {"dataset": dataset}, headers, timeout)
    splits = list((splits_payload or {}).get("splits", []))  # type: ignore[assignment]
    if not splits:
@@ -157,6 +181,39 @@ def _fetch_hf_rows(
    return rows
 # ---------------------------------------------------------------------------
 # Answer extraction helpers
 # ---------------------------------------------------------------------------
 def _extract_gsm8k_answer(text: str) -> str:
    if "####" in text:
        text = text.split("####", 1)[1]
    numbers = re.findall(r"-?\d[\d,]*\.?\d*", text.replace("$", ""))
    if numbers:
        return numbers[-1].replace(",", "")
    return text.strip().replace(",", "")
 def _extract_math_answer(solution: str) -> str:
    boxed_fraction = re.search(r"\\boxed\{\\frac\{([^}]+)\}\{([^}]+)\}\}", solution, flags=re.DOTALL)
    if boxed_fraction:
        return f"{boxed_fraction.group(1).strip()}/{boxed_fraction.group(2).strip()}"
    boxed = re.search(r"\\boxed\{([^{}]+)\}", solution, flags=re.DOTALL)
    value = boxed.group(1) if boxed else solution
    value = value.strip()
    value = value.replace("\\frac{", "").replace("}{", "/").replace("}", "")
    value = value.replace("$", "").replace(",", "").strip()
    fraction = re.search(r"-?\d+\s*/\s*-?\d+", value)
    if fraction:
        return fraction.group(0).replace(" ", "")
    numbers = re.findall(r"-?\d+(?:/\d+)?(?:\.\d+)?", value)
    return numbers[-1] if numbers else value
 # ---------------------------------------------------------------------------
 # Individual dataset downloaders (using `datasets` library)
 # ---------------------------------------------------------------------------
 def _download_humaneval(output_path: Path, *, timeout: float) -> DownloadResult:
    raw = fetch_bytes(HUMANEVAL_GZ_URL, timeout=timeout)
    if raw[:2] == b"\x1f\x8b":
@@ -180,15 +237,19 @@ def _download_gsm8k(
    output_path: Path,
    *,
    timeout: float,
-    json_fetcher: JsonFetcher = fetch_json,
+    json_fetcher: JsonFetcher | None = None,
 ) -> DownloadResult:
-    rows = _fetch_hf_rows(
+    if json_fetcher is not None:
-        "openai/gsm8k",
+        # Legacy path for tests
-        config_preference=("main",),
+        rows = _fetch_hf_rows(
-        split_preference=("test",),
+            "openai/gsm8k",
-        json_fetcher=json_fetcher,
+            config_preference=("main",),
-        timeout=timeout,
+            split_preference=("test",),
-    )
+            json_fetcher=json_fetcher,
            timeout=timeout,
        )
    else:
        rows = _try_load_hf("openai/gsm8k", config="main", split_preference=("test",))
    normalized = [
        {
            "id": f"gsm8k-{index + 1:04d}",
@@ -201,19 +262,11 @@ def _download_gsm8k(
    return DownloadResult("gsm8k", count, str(output_path), "official")
-def _download_mbpp(
+def _download_mbpp(output_path: Path, *, timeout: float) -> DownloadResult:
-    output_path: Path,
+    try:
-    *,
+        rows = _try_load_hf("google-research-datasets/mbpp", config="sanitized", split_preference=("test", "validation"))
-    timeout: float,
+    except Exception:
-    json_fetcher: JsonFetcher = fetch_json,
+        rows = _try_load_hf("google-research-datasets/mbpp", config="full", split_preference=("test", "validation"))
 ) -> DownloadResult:
    rows = _fetch_hf_rows(
        "google-research-datasets/mbpp",
        config_preference=("sanitized", "full"),
        split_preference=("test", "validation"),
        json_fetcher=json_fetcher,
        timeout=timeout,
    )
    normalized = [
        {
            "task_id": row.get("task_id", index + 1),
@@ -227,19 +280,8 @@ def _download_mbpp(
    return DownloadResult("mbpp", count, str(output_path), "official")
-def _download_math(
+def _download_math(output_path: Path, *, timeout: float) -> DownloadResult:
-    output_path: Path,
+    rows = _try_load_hf("DigitalLearningGmbH/MATH-lighteval", split_preference=("test", "train"))
    *,
    timeout: float,
    json_fetcher: JsonFetcher = fetch_json,
 ) -> DownloadResult:
    rows = _fetch_hf_rows(
        "hendrycks/competition_math",
        config_preference=("default",),
        split_preference=("test", "train"),
        json_fetcher=json_fetcher,
        timeout=timeout,
    )
    normalized = [
        {
            "id": row.get("problem_id", f"math-{index + 1:04d}"),
@@ -254,6 +296,133 @@ def _download_math(
    return DownloadResult("math", count, str(output_path), "official")
 def _download_mmlu_pro(output_path: Path, *, timeout: float) -> DownloadResult:
    rows = _try_load_hf("TIGER-Lab/MMLU-Pro", split_preference=("test", "validation"))
    letters = "ABCDEFGHIJ"
    normalized = []
    for index, row in enumerate(rows):
        answer_raw = row.get("answer", "")
        if isinstance(answer_raw, int) and answer_raw < len(letters):
            answer = letters[answer_raw]
        else:
            answer = str(answer_raw)
        normalized.append({
            "id": f"mmlu-pro-{index + 1:04d}",
            "subject": row.get("category", row.get("subject", "unknown")),
            "question": row.get("question", ""),
            "choices": row.get("options", row.get("choices", [])),
            "answer": answer,
        })
    count = _write_jsonl(output_path, normalized)
    return DownloadResult("mmlu-pro", count, str(output_path), "official")
 def _download_gpqa(output_path: Path, *, timeout: float) -> DownloadResult:
    rows = _try_load_hf("Idavidrein/gpqa", config="gpqa_diamond", split_preference=("train",))
    normalized = []
    for index, row in enumerate(rows):
        choices = [
            row.get("Correct Answer", ""),
            row.get("Incorrect Answer 1", ""),
            row.get("Incorrect Answer 2", ""),
            row.get("Incorrect Answer 3", ""),
        ]
        normalized.append({
            "id": f"gpqa-{index + 1:04d}",
            "subject": row.get("Subdomain", row.get("domain", "science")),
            "question": row.get("Question", ""),
            "choices": choices,
            "answer": "A",
        })
    count = _write_jsonl(output_path, normalized)
    return DownloadResult("gpqa-diamond", count, str(output_path), "official")
 def _download_bigbench_hard(output_path: Path, *, timeout: float) -> DownloadResult:
    from datasets import load_dataset  # type: ignore[import-untyped]
    configs = [
        "boolean_expressions", "causal_judgement", "date_understanding",
        "disambiguation_qa", "dyck_languages", "formal_fallacies",
        "geometric_shapes", "hyperbaton", "logical_deduction_three_objects",
        "logical_deduction_five_objects", "logical_deduction_seven_objects",
        "movie_recommendation", "multistep_arithmetic_two", "navigate",
        "object_counting", "penguins_in_a_table",
        "reasoning_about_colored_objects", "ruin_names",
        "salient_translation_error_detection", "snarks",
        "sports_understanding", "temporal_sequences",
        "tracking_shuffled_objects_three_objects",
        "tracking_shuffled_objects_five_objects",
        "tracking_shuffled_objects_seven_objects",
        "web_of_lies", "word_sorting",
    ]
    all_rows: list[dict[str, Any]] = []
    for config in configs:
        try:
            ds = load_dataset("lukaemon/bbh", config, split="test")
            for row in ds:
                row_dict = dict(row)  # type: ignore[arg-type]
                row_dict["task"] = config
                all_rows.append(row_dict)
        except Exception:
            continue
    print(f"    Loaded {len(all_rows)} rows from lukaemon/bbh [{len(configs)} tasks]")
    normalized = []
    for index, row in enumerate(all_rows):
        target = row.get("target", row.get("answer", ""))
        normalized.append({
            "id": f"bbh-{index + 1:05d}",
            "task": row.get("task", "unknown"),
            "question": row.get("input", row.get("question", "")),
            "choices": [],
            "answer": str(target).strip(),
        })
    count = _write_jsonl(output_path, normalized)
    return DownloadResult("bigbench-hard", count, str(output_path), "official")
 def _download_mmmlu(output_path: Path, *, timeout: float) -> DownloadResult:
    rows = _try_load_hf("openai/MMMLU", split_preference=("test", "validation"))
    letters = "ABCD"
    normalized = []
    for index, row in enumerate(rows):
        answer_raw = row.get("answer", "")
        if isinstance(answer_raw, int) and answer_raw < len(letters):
            answer = letters[answer_raw]
        else:
            answer = str(answer_raw)
        normalized.append({
            "id": f"mmmlu-{index + 1:04d}",
            "language": row.get("language", "en"),
            "subject": row.get("subject", "unknown"),
            "question": row.get("question", ""),
            "choices": row.get("choices", row.get("options", [])),
            "answer": answer,
        })
    count = _write_jsonl(output_path, normalized)
    return DownloadResult("mmmlu", count, str(output_path), "official")
 def _download_hle(output_path: Path, *, timeout: float) -> DownloadResult:
    rows = _try_load_hf("cais/hle", split_preference=("test", "validation", "train"))
    normalized = []
    for index, row in enumerate(rows):
        entry: dict[str, Any] = {
            "id": f"hle-{index + 1:04d}",
            "subject": row.get("category", row.get("subject", "general")),
            "question": row.get("question", ""),
            "answer": str(row.get("answer", "")),
        }
        if row.get("choices") or row.get("options"):
            entry["answer_type"] = "multiple_choice"
            entry["choices"] = row.get("choices", row.get("options", []))
        else:
            entry["answer_type"] = "exact"
        normalized.append(entry)
    count = _write_jsonl(output_path, normalized)
    return DownloadResult("hle", count, str(output_path), "official")
 def _export_builtin(output_path: Path, suite: str, rows: list[dict[str, Any]], *, source: str = "builtin", note: str = "") -> DownloadResult:
    count = _write_jsonl(output_path, rows)
    return DownloadResult(suite, count, str(output_path), source, note)
@@ -271,6 +440,13 @@ def _builtin_rows(suite: str) -> list[dict[str, Any]]:
        "aime": list(_AIME_BUILTINS),
        "ifeval": list(_IFEVAL_BUILTINS),
        "bfcl": list(_BFCL_BUILTINS),
        "mmlu-pro": list(_MMLU_PRO_BUILTINS),
        "gpqa-diamond": list(_GPQA_BUILTINS),
        "bigbench-hard": list(_BIGBENCH_BUILTINS),
        "mmmlu": list(_MMMLU_BUILTINS),
        "hle": list(_HLE_BUILTINS),
        "tau2": list(_TAU2_BUILTINS),
        "codeforces": list(_CODEFORCES_BUILTINS),
    }
    return list(mapping[suite])
@@ -284,19 +460,33 @@ def prepare_suite(
    official_only: bool,
    timeout: float,
 ) -> DownloadResult:
-    output_path = data_dir / f"{suite}.jsonl"
+    output_map = {
        "mmlu-pro": "mmlu_pro.jsonl",
        "gpqa-diamond": "gpqa.jsonl",
        "bigbench-hard": "bigbench_hard.jsonl",
        "mmmlu": "mmmlu.jsonl",
        "hle": "hle.jsonl",
        "tau2": "tau2.jsonl",
        "codeforces": "codeforces.jsonl",
    }
    output_path = data_dir / output_map.get(suite, f"{suite}.jsonl")
    if output_path.exists() and not force:
        lines = [line for line in output_path.read_text(encoding="utf-8").splitlines() if line.strip()]
        return DownloadResult(suite, len(lines), str(output_path), "existing")
    data_dir.mkdir(parents=True, exist_ok=True)
-    builtin_only_suites = {"swe-bench", "aider", "livecodebench", "aime", "ifeval", "bfcl"}
+    builtin_only_suites = {"swe-bench", "aider", "livecodebench", "aime", "ifeval", "bfcl", "tau2", "codeforces"}
    official_downloaders = {
        "humaneval": _download_humaneval,
        "gsm8k": _download_gsm8k,
        "mbpp": _download_mbpp,
        "math": _download_math,
        "mmlu-pro": _download_mmlu_pro,
        "gpqa-diamond": _download_gpqa,
        "bigbench-hard": _download_bigbench_hard,
        "mmmlu": _download_mmmlu,
        "hle": _download_hle,
    }
    if suite in builtin_only_suites or builtin_only:
@@ -312,6 +502,9 @@ def prepare_suite(
        if official_only:
            raise
        note = f"official download failed: {exc}"
        print(f"  WARNING: {note}")
        print(f"  Falling back to {len(_builtin_rows(suite))} built-in problems.")
        print(f"  To get full data, install `datasets`: pip install datasets")
        return _export_builtin(
            output_path,
            suite,
@@ -358,6 +551,13 @@ def main() -> None:
        "aime",
        "ifeval",
        "bfcl",
        "mmlu-pro",
        "gpqa-diamond",
        "bigbench-hard",
        "mmmlu",
        "hle",
        "tau2",
        "codeforces",
    ]
    if args.list:
@@ -55,6 +55,13 @@ from benchmarks.suites.gsm8k import GSM8KBenchmark
 from benchmarks.suites.aime import AIMEBenchmark
 from benchmarks.suites.ifeval import IFEvalBenchmark
 from benchmarks.suites.bfcl import BFCLBenchmark
 from benchmarks.suites.mmlu_pro import MMLUProBenchmark
 from benchmarks.suites.gpqa import GPQABenchmark
 from benchmarks.suites.bigbench import BigBenchHardBenchmark
 from benchmarks.suites.mmmlu import MMMMLUBenchmark
 from benchmarks.suites.hle import HLEBenchmark
 from benchmarks.suites.tau2 import Tau2Benchmark
 from benchmarks.suites.codeforces import CodeforcesBenchmark
 # ---------------------------------------------------------------------------
@@ -72,12 +79,21 @@ SUITE_REGISTRY: dict[str, type[BenchmarkSuite]] = {
    "aime": AIMEBenchmark,
    "ifeval": IFEvalBenchmark,
    "bfcl": BFCLBenchmark,
    "mmlu-pro": MMLUProBenchmark,
    "gpqa-diamond": GPQABenchmark,
    "bigbench-hard": BigBenchHardBenchmark,
    "mmmlu": MMMMLUBenchmark,
    "hle": HLEBenchmark,
    "tau2": Tau2Benchmark,
    "codeforces": CodeforcesBenchmark,
 }
 CATEGORY_MAP: dict[str, list[str]] = {
-    "coding": ["humaneval", "mbpp", "swe-bench", "aider", "livecodebench"],
+    "coding": ["humaneval", "mbpp", "swe-bench", "aider", "livecodebench", "codeforces"],
    "math": ["math", "gsm8k", "aime"],
    "instruction-following": ["ifeval", "bfcl"],
    "knowledge": ["mmlu-pro", "gpqa-diamond", "mmmlu", "hle"],
    "reasoning": ["bigbench-hard", "tau2"],
 }
@@ -175,8 +191,8 @@ def main() -> None:
            "  python3 -m benchmarks.run_suite --all -o results.json\n"
        ),
    )
-    parser.add_argument("--suite", choices=list(SUITE_REGISTRY.keys()),
+    parser.add_argument("--suite", action="append", default=[],
-                        help="Run a specific benchmark suite")
+                        help="Run a specific benchmark suite (can be repeated)")
    parser.add_argument("--category", choices=list(CATEGORY_MAP.keys()),
                        help="Run all suites in a category")
    parser.add_argument("--all", action="store_true",
@@ -221,7 +237,11 @@ def main() -> None:
    elif args.category:
        suite_names = CATEGORY_MAP.get(args.category, [])
    elif args.suite:
-        suite_names = [args.suite]
+        for s in args.suite:
            if s not in SUITE_REGISTRY:
                print(f"Error: unknown suite '{s}'. Available: {', '.join(SUITE_REGISTRY)}")
                sys.exit(1)
        suite_names = list(args.suite)
    else:
        parser.print_help()
        print("\nError: specify --suite, --category, or --all")
@@ -84,20 +84,22 @@ class AIMEBenchmark(BenchmarkSuite):
    category = "math"
    def load_dataset(self) -> list[dict[str, Any]]:
-        jsonl_path = Path(self.data_dir) / "aime.jsonl"
+        # Prefer aime_2026.jsonl for AIME 2026 specific problems
-        if jsonl_path.exists():
+        for fname in ("aime_2026.jsonl", "aime.jsonl"):
-            problems: list[dict[str, Any]] = []
+            jsonl_path = Path(self.data_dir) / fname
-            with open(jsonl_path) as fh:
+            if jsonl_path.exists():
-                for line in fh:
+                problems: list[dict[str, Any]] = []
-                    line = line.strip()
+                with open(jsonl_path) as fh:
-                    if line:
+                    for line in fh:
-                        problems.append(json.loads(line))
+                        line = line.strip()
-            if self.verbose:
+                        if line:
-                print(f"  Loaded {len(problems)} problems from {jsonl_path}")
+                            problems.append(json.loads(line))
-            return problems
+                if self.verbose:
                    print(f"  Loaded {len(problems)} problems from {jsonl_path}")
                return problems
        if self.verbose:
-            print(f"  {jsonl_path} not found — using built-in 10-problem subset")
+            print("  No AIME data file found — using built-in 10-problem subset")
        return list(_BUILTIN_PROBLEMS)
    def build_prompt(self, problem: dict[str, Any]) -> str:
@@ -0,0 +1,176 @@
 """
 BIG-Bench Extra Hard benchmark suite.
 BIG-Bench Extra Hard (BBH) is a subset of the most challenging tasks from
 the BIG-Bench benchmark that language models have historically struggled with.
 Tasks include logical reasoning, causal judgment, date understanding, etc.
 Reference: https://huggingface.co/datasets/maveriq/bigbenchhard
 """
 from __future__ import annotations
 import json
 import os
 import re
 from pathlib import Path
 from typing import Any
 from .base import BenchmarkResult, BenchmarkSuite
 # ---------------------------------------------------------------------------
 # Built-in mini dataset (10 representative BIG-Bench Hard problems)
 # ---------------------------------------------------------------------------
 _BUILTIN_PROBLEMS: list[dict[str, Any]] = [
    {
        "id": "bbh-001",
        "task": "logical_deduction_three_objects",
        "question": "The following paragraphs each describe a set of three objects arranged in a fixed order. The statements are logically consistent within each paragraph.\n\nOn a shelf, there are three books: a red book, a green book, and a blue book. The blue book is to the left of the green book. The red book is the second from the left.\n\nWhich book is the leftmost?",
        "choices": ["The red book", "The green book", "The blue book"],
        "answer": "C",
    },
    {
        "id": "bbh-002",
        "task": "causal_judgement",
        "question": "How would a typical person answer each of the following questions about causation?\n\nBilly and Suzy each throw a rock at a bottle. Suzy's rock arrives first and shatters the bottle. Billy's rock arrives at where the bottle was a moment later.\n\nDid Billy cause the bottle to shatter?",
        "choices": ["Yes", "No"],
        "answer": "B",
    },
    {
        "id": "bbh-003",
        "task": "date_understanding",
        "question": "Today is Christmas Eve of 1937. What is the date tomorrow in MM/DD/YYYY?",
        "choices": ["12/11/1937", "12/25/1937", "01/04/1938", "12/04/1937", "12/25/2006", "09/25/1937"],
        "answer": "B",
    },
    {
        "id": "bbh-004",
        "task": "disambiguation_qa",
        "question": "In the following sentences, explain the antecedent of the pronoun (which thing the pronoun refers to), or state that it is ambiguous.\n\nSentence: The scientist thanked the librarian because she was helpful.\n\nWho was helpful?",
        "choices": ["The scientist", "The librarian", "Ambiguous"],
        "answer": "B",
    },
    {
        "id": "bbh-005",
        "task": "navigate",
        "question": "If you follow these instructions, do you return to the starting point?\n\nTake 1 step east. Take 2 steps north. Take 1 step west. Take 2 steps south.",
        "choices": ["Yes", "No"],
        "answer": "A",
    },
    {
        "id": "bbh-006",
        "task": "penguins_in_a_table",
        "question": "Here is a table where the first line is a header and each subsequent line is a penguin:\n\nname, age, height (cm), weight (kg)\nLouis, 7, 50, 11\nBernard, 5, 80, 13\nVincent, 9, 60, 11\nGwen, 8, 70, 15\n\nWhat is the name of the tallest penguin?",
        "choices": ["Louis", "Bernard", "Vincent", "Gwen"],
        "answer": "B",
    },
    {
        "id": "bbh-007",
        "task": "sports_understanding",
        "question": "Is the following sentence plausible?\n\n\"Lebron James scored a touchdown.\"",
        "choices": ["plausible", "not plausible"],
        "answer": "B",
    },
    {
        "id": "bbh-008",
        "task": "boolean_expressions",
        "question": "Evaluate the result of a random Boolean expression.\n\nnot ( ( not not True ) ) is",
        "choices": ["True", "False"],
        "answer": "B",
    },
    {
        "id": "bbh-009",
        "task": "web_of_lies",
        "question": "Evaluate a random Boolean function expressed as a word problem.\n\nQuestion: Fidel tells the truth. Jerry says Fidel tells the truth. Vina says Jerry lies. Millicent says Vina tells the truth. Does Millicent tell the truth?",
        "choices": ["Yes", "No"],
        "answer": "B",
    },
    {
        "id": "bbh-010",
        "task": "tracking_shuffled_objects_three_objects",
        "question": "Alice, Bob, and Claire are playing a game. At the start of the game, they are each holding a ball: Alice has a red ball, Bob has a blue ball, and Claire has a green ball.\n\nAs the game progresses, pairs of players trade balls. First, Alice and Bob swap balls. Then, Bob and Claire swap balls.\n\nAt the end of the game, what ball does Bob have?",
        "choices": ["red ball", "blue ball", "green ball"],
        "answer": "C",
    },
 ]
 class BigBenchHardBenchmark(BenchmarkSuite):
    """BIG-Bench Extra Hard: Challenging reasoning tasks from BIG-Bench."""
    name = "BigBench-Extra-Hard"
    description = "Challenging multi-step reasoning tasks from BIG-Bench"
    category = "reasoning"
    def load_dataset(self) -> list[dict[str, Any]]:
        jsonl_path = Path(self.data_dir) / "bigbench_hard.jsonl"
        if jsonl_path.exists():
            problems: list[dict[str, Any]] = []
            with open(jsonl_path) as fh:
                for line in fh:
                    line = line.strip()
                    if line:
                        problems.append(json.loads(line))
            if self.verbose:
                print(f"  Loaded {len(problems)} problems from {jsonl_path}")
            return problems
        if self.verbose:
            print(f"  {jsonl_path} not found — using built-in 10-problem subset")
        return list(_BUILTIN_PROBLEMS)
    def build_prompt(self, problem: dict[str, Any]) -> str:
        question = problem["question"]
        choices = problem.get("choices", [])
        letters = "ABCDEFGHIJ"
        choices_text = "\n".join(
            f"  {letters[i]}. {c}" for i, c in enumerate(choices)
        )
        return (
            f"Answer the following reasoning question.\n\n"
            f"Question: {question}\n\n"
            f"Choices:\n{choices_text}\n\n"
            f"Think step by step, then write ONLY the letter of the correct answer "
            f"to a file called answer.txt — no explanation, just the single letter."
        )
    def recover_output_files(
        self,
        problem: dict[str, Any],
        workspace: str,
        agent_output: str,
        metadata: dict[str, Any],
    ) -> None:
        del problem
        answer_path = Path(workspace) / "answer.txt"
        if answer_path.exists():
            return
        match = re.search(r"\b([A-J])\b", agent_output)
        if match:
            answer_path.write_text(match.group(1) + "\n", encoding="utf-8")
            metadata["recovered_answer_from_output"] = True
    def evaluate(self, problem: dict[str, Any], workspace: str) -> BenchmarkResult:
        pid = problem.get("id", "unknown")
        expected = str(problem["answer"]).strip().upper()
        answer_file = os.path.join(workspace, "answer.txt")
        if not os.path.exists(answer_file):
            return BenchmarkResult(
                problem_id=pid, passed=False, expected=expected,
                error="answer.txt not found",
            )
        with open(answer_file) as fh:
            actual_raw = fh.read().strip()
        match = re.search(r"\b([A-J])\b", actual_raw.upper())
        actual = match.group(1) if match else actual_raw.strip().upper()
        passed = actual == expected
        return BenchmarkResult(
            problem_id=pid, passed=passed,
            expected=expected, actual=actual_raw,
            error="" if passed else f"expected={expected}, got={actual_raw}",
        )
@@ -0,0 +1,275 @@
 """
 Codeforces benchmark suite.
 Evaluates competitive programming ability using Codeforces-style problems.
 Unlike pass/fail suites, this suite computes an ELO-like rating based on
 problem difficulty and correctness.
 Reference: https://codeforces.com/
 """
 from __future__ import annotations
 import json
 import math
 import os
 import re
 from pathlib import Path
 from typing import Any
 from .base import BenchmarkResult, BenchmarkSuite, SuiteReport
 # ---------------------------------------------------------------------------
 # Built-in mini dataset (10 Codeforces-style problems with difficulty ratings)
 # ---------------------------------------------------------------------------
 _BUILTIN_PROBLEMS: list[dict[str, Any]] = [
    {
        "id": "cf-800-001",
        "rating": 800,
        "title": "Watermelon",
        "problem": "Pete and Billy have a watermelon weighing w kilograms. They want to divide it into two parts, each weighing an even number of kilograms. Determine if this is possible.\n\nInput: A single integer w (1 ≤ w ≤ 100)\nOutput: Print YES if possible, NO otherwise.",
        "test_cases": [
            {"input": "8", "expected_output": "YES"},
            {"input": "3", "expected_output": "NO"},
            {"input": "1", "expected_output": "NO"},
            {"input": "2", "expected_output": "NO"},
            {"input": "4", "expected_output": "YES"},
        ],
    },
    {
        "id": "cf-800-002",
        "rating": 800,
        "title": "Way Too Long Words",
        "problem": "Abbreviate words longer than 10 characters. For such words, output the first letter, number of middle characters, and last letter.\n\nInput: First line is n (1 ≤ n ≤ 100). Next n lines each contain a word.\nOutput: For each word, output the abbreviation or the word itself if length ≤ 10.",
        "test_cases": [
            {"input": "4\nword\nlocalization\ninternationalization\npneumonoultramicroscopicsilicovolcanoconiosis", "expected_output": "word\nl10n\ni18n\np43s"},
            {"input": "1\nabcdefghij", "expected_output": "abcdefghij"},
        ],
    },
    {
        "id": "cf-1000-001",
        "rating": 1000,
        "title": "Nearly Lucky Number",
        "problem": "A number is nearly lucky if the count of digits 4 and 7 in it is itself a lucky number (composed only of 4s and 7s). Given n, determine if it is nearly lucky.\n\nInput: A single integer n (1 ≤ n ≤ 10^18)\nOutput: YES or NO",
        "test_cases": [
            {"input": "40047", "expected_output": "NO"},
            {"input": "7747774", "expected_output": "YES"},
            {"input": "1000000000000000000", "expected_output": "NO"},
        ],
    },
    {
        "id": "cf-1200-001",
        "rating": 1200,
        "title": "Beautiful Matrix",
        "problem": "You have a 5×5 matrix with exactly one 1 and rest 0s. In one move, you can swap any two adjacent rows or columns. Find the minimum number of moves to place the 1 in the center (row 3, col 3).\n\nInput: 5 lines each with 5 space-separated integers.\nOutput: Minimum number of moves.",
        "test_cases": [
            {"input": "0 0 0 0 0\n0 0 0 0 1\n0 0 0 0 0\n0 0 0 0 0\n0 0 0 0 0", "expected_output": "3"},
            {"input": "0 0 0 0 0\n0 0 0 0 0\n0 0 1 0 0\n0 0 0 0 0\n0 0 0 0 0", "expected_output": "0"},
        ],
    },
    {
        "id": "cf-1400-001",
        "rating": 1400,
        "title": "Kefa and Park",
        "problem": "Kefa wants to walk from the root (node 1) to any leaf in a tree. Along the path, there are cats at some nodes. Find the number of leaves reachable such that the path doesn't contain more than m consecutive cats.\n\nInput: First line: n m (n nodes, max m consecutive cats). Second line: n integers (0/1) for each node. Next n-1 lines: edges.\nOutput: Number of valid leaves.\n\nFor the built-in test, n=7 m=1, cats=[1,1,0,0,1,0,1], edges: 1-2, 1-3, 2-4, 2-5, 3-6, 3-7",
        "test_cases": [
            {"input": "7 1\n1 1 0 0 1 0 1\n1 2\n1 3\n2 4\n2 5\n3 6\n3 7", "expected_output": "2"},
        ],
    },
    {
        "id": "cf-1600-001",
        "rating": 1600,
        "title": "Divisibility by Eight",
        "problem": "Given a number (as a string of up to 100 digits), determine if you can delete some digits to get a non-empty number divisible by 8. Leading zeros are allowed in the result.\n\nInput: A string of digits.\nOutput: YES and the resulting number, or NO.",
        "test_cases": [
            {"input": "3121", "expected_output": "YES\n312"},
            {"input": "123456789", "expected_output": "YES\n8"},
            {"input": "3", "expected_output": "NO"},
        ],
    },
    {
        "id": "cf-1800-001",
        "rating": 1800,
        "title": "Array Partition",
        "problem": "Given an array a of n integers, determine if you can partition it into three non-empty contiguous parts such that max(part1) = min(part2) = max(part3).\n\nInput: First line n. Second line: a1...an.\nOutput: YES and the lengths of three parts, or NO.",
        "test_cases": [
            {"input": "5\n3 1 5 3 1", "expected_output": "YES"},
            {"input": "3\n1 2 3", "expected_output": "NO"},
        ],
    },
    {
        "id": "cf-2000-001",
        "rating": 2000,
        "title": "Count Binary Strings",
        "problem": "Count the number of binary strings of length n where no two adjacent characters are both '1'. Output the answer modulo 10^9 + 7.\n\nInput: A single integer n (1 ≤ n ≤ 10^6)\nOutput: The count mod 10^9+7.",
        "test_cases": [
            {"input": "1", "expected_output": "2"},
            {"input": "2", "expected_output": "3"},
            {"input": "3", "expected_output": "5"},
            {"input": "10", "expected_output": "144"},
        ],
    },
    {
        "id": "cf-2200-001",
        "rating": 2200,
        "title": "Xor Sequences",
        "problem": "Given an array of n distinct non-negative integers, find the number of pairs (i,j) where i < j such that a[i] XOR a[j] has an even number of set bits.\n\nInput: First line n. Second line: a1...an.\nOutput: Number of such pairs.",
        "test_cases": [
            {"input": "3\n1 2 3", "expected_output": "1"},
            {"input": "4\n0 1 2 3", "expected_output": "2"},
        ],
    },
    {
        "id": "cf-2500-001",
        "rating": 2500,
        "title": "Segment Tree Query",
        "problem": "Given an array of n integers, answer q queries. Each query gives l, r and asks for the sum of elements from index l to r (1-indexed). Implement this efficiently.\n\nInput: First line: n q. Second line: a1...an. Next q lines: l r.\nOutput: q lines with answers.",
        "test_cases": [
            {"input": "5 3\n1 2 3 4 5\n1 3\n2 4\n1 5", "expected_output": "6\n9\n15"},
        ],
    },
 ]
 def _compute_elo(results: list[BenchmarkResult], problems: list[dict[str, Any]]) -> float:
    """Compute an approximate ELO rating from problem results and difficulty ratings."""
    if not results:
        return 0.0
    # Build a mapping from problem_id to rating
    rating_map = {str(p.get("id", "")): p.get("rating", 1200) for p in problems}
    # Start with base rating 800
    elo = 800.0
    k_factor = 40.0
    for result in results:
        problem_rating = rating_map.get(result.problem_id, 1200)
        # Expected score based on ELO difference
        expected = 1.0 / (1.0 + math.pow(10, (problem_rating - elo) / 400.0))
        actual_score = 1.0 if result.passed else 0.0
        elo += k_factor * (actual_score - expected)
    return round(max(0, elo))
 class CodeforcesBenchmark(BenchmarkSuite):
    """Codeforces: Competitive programming with ELO-based scoring."""
    name = "Codeforces"
    description = "Competitive programming problems with ELO rating estimation"
    category = "coding"
    def load_dataset(self) -> list[dict[str, Any]]:
        jsonl_path = Path(self.data_dir) / "codeforces.jsonl"
        if jsonl_path.exists():
            problems: list[dict[str, Any]] = []
            with open(jsonl_path) as fh:
                for line in fh:
                    line = line.strip()
                    if line:
                        problems.append(json.loads(line))
            if self.verbose:
                print(f"  Loaded {len(problems)} problems from {jsonl_path}")
            return problems
        if self.verbose:
            print(f"  {jsonl_path} not found — using built-in 10-problem subset")
        return list(_BUILTIN_PROBLEMS)
    def build_prompt(self, problem: dict[str, Any]) -> str:
        title = problem.get("title", "Problem")
        text = problem["problem"]
        test_cases = problem.get("test_cases", [])
        examples = ""
        for i, tc in enumerate(test_cases[:2], 1):
            examples += f"\nExample {i}:\n  Input: {tc['input']}\n  Output: {tc['expected_output']}\n"
        return (
            f"Solve the following competitive programming problem.\n\n"
            f"Title: {title}\n\n"
            f"Problem:\n{text}\n"
            f"{examples}\n"
            f"Write a Python solution in a file called solution.py that reads from "
            f"stdin and writes to stdout."
        )
    def setup_workspace(self, problem: dict[str, Any], workspace: str) -> None:
        # Write test cases to workspace for evaluation
        test_cases = problem.get("test_cases", [])
        for i, tc in enumerate(test_cases):
            input_file = os.path.join(workspace, f"test_input_{i}.txt")
            expected_file = os.path.join(workspace, f"test_expected_{i}.txt")
            with open(input_file, "w") as f:
                f.write(tc["input"] + "\n")
            with open(expected_file, "w") as f:
                f.write(tc["expected_output"].strip() + "\n")
    def evaluate(self, problem: dict[str, Any], workspace: str) -> BenchmarkResult:
        pid = problem.get("id", "unknown")
        rating = problem.get("rating", 1200)
        solution_file = os.path.join(workspace, "solution.py")
        if not os.path.exists(solution_file):
            return BenchmarkResult(
                problem_id=pid, passed=False,
                error="solution.py not found",
                metadata={"rating": rating},
            )
        test_cases = problem.get("test_cases", [])
        all_passed = True
        errors = []
        for i, tc in enumerate(test_cases):
            input_data = tc["input"]
            expected = tc["expected_output"].strip()
            rc, output = self._run_shell(
                f"echo {repr(input_data)} | python3 solution.py",
                cwd=workspace,
                timeout=10.0,
            )
            actual = output.strip()
            if rc != 0:
                all_passed = False
                errors.append(f"test {i}: runtime error: {output[:200]}")
            elif actual != expected:
                # Check if the output matches any valid answer
                # (for problems with YES/NO + additional output)
                if expected.startswith("YES") and actual.startswith("YES"):
                    pass  # Accept if at least "YES" is correct
                elif expected.startswith("NO") and actual.startswith("NO"):
                    pass
                else:
                    all_passed = False
                    errors.append(f"test {i}: expected={expected!r}, got={actual!r}")
        return BenchmarkResult(
            problem_id=pid, passed=all_passed,
            expected=f"all {len(test_cases)} tests",
            actual=f"{'all passed' if all_passed else '; '.join(errors)}",
            error="" if all_passed else "; ".join(errors),
            metadata={"rating": rating},
        )
    def run_all(self) -> SuiteReport:
        """Override to add ELO computation to the report."""
        report = super().run_all()
        problems = self.load_dataset()
        if self.limit is not None:
            problems = problems[: self.limit]
        elo = _compute_elo(report.results, problems)
        print(f"  Estimated Codeforces ELO: {elo}")
        report.results.append(
            BenchmarkResult(
                problem_id="_elo_rating",
                passed=True,
                actual=str(elo),
                metadata={"elo_rating": elo},
            )
        )
        return report
@@ -0,0 +1,176 @@
 """
 GPQA Diamond benchmark suite.
 GPQA (Graduate-Level Google-Proof Question Answering) Diamond is a subset
 of extremely difficult questions written by domain experts in biology,
 physics, and chemistry. The "Diamond" subset is the hardest tier.
 Reference: https://huggingface.co/datasets/Idavidrein/gpqa
 """
 from __future__ import annotations
 import json
 import os
 import re
 from pathlib import Path
 from typing import Any
 from .base import BenchmarkResult, BenchmarkSuite
 # ---------------------------------------------------------------------------
 # Built-in mini dataset (10 representative GPQA Diamond problems)
 # ---------------------------------------------------------------------------
 _BUILTIN_PROBLEMS: list[dict[str, Any]] = [
    {
        "id": "gpqa-001",
        "subject": "physics",
        "question": "A particle of mass m is confined to a one-dimensional box of length L. What is the energy difference between the first excited state and the ground state?",
        "choices": ["3π²ℏ²/(2mL²)", "π²ℏ²/(2mL²)", "4π²ℏ²/(2mL²)", "2π²ℏ²/(mL²)"],
        "answer": "A",
    },
    {
        "id": "gpqa-002",
        "subject": "chemistry",
        "question": "Which of the following molecules has the highest bond dissociation energy?",
        "choices": ["N₂", "O₂", "F₂", "CO"],
        "answer": "D",
    },
    {
        "id": "gpqa-003",
        "subject": "biology",
        "question": "In the lac operon, which component acts as the inducer that causes the repressor to release from the operator?",
        "choices": ["Lactose", "Allolactose", "Glucose", "cAMP"],
        "answer": "B",
    },
    {
        "id": "gpqa-004",
        "subject": "physics",
        "question": "In quantum electrodynamics, what is the leading-order correction to the electron g-factor (anomalous magnetic moment)?",
        "choices": ["α/(2π)", "α/π", "α²/(2π)", "2α/π"],
        "answer": "A",
    },
    {
        "id": "gpqa-005",
        "subject": "chemistry",
        "question": "What is the primary product when 2-methylpropene undergoes hydroboration-oxidation?",
        "choices": ["2-methylpropan-2-ol", "2-methylpropan-1-ol", "2-methylpropanal", "isobutylene oxide"],
        "answer": "B",
    },
    {
        "id": "gpqa-006",
        "subject": "biology",
        "question": "Which of the following enzymes is responsible for adding a 5' cap to mRNA in eukaryotes?",
        "choices": ["RNA polymerase II", "Guanylyltransferase", "Poly(A) polymerase", "RNA triphosphatase alone"],
        "answer": "B",
    },
    {
        "id": "gpqa-007",
        "subject": "physics",
        "question": "In general relativity, the Schwarzschild radius of a black hole with mass M is given by:",
        "choices": ["2GM/c²", "GM/c²", "GM²/c", "2GM²/c²"],
        "answer": "A",
    },
    {
        "id": "gpqa-008",
        "subject": "chemistry",
        "question": "Which of the following is the correct order of acidity for the hydrogen halides in aqueous solution?",
        "choices": ["HF > HCl > HBr > HI", "HI > HBr > HCl > HF", "HCl > HBr > HI > HF", "HF > HI > HBr > HCl"],
        "answer": "B",
    },
    {
        "id": "gpqa-009",
        "subject": "biology",
        "question": "What is the primary function of topoisomerase II during DNA replication?",
        "choices": ["Unwinding the double helix", "Relieving positive supercoiling ahead of the replication fork", "Joining Okazaki fragments", "Proofreading newly synthesized DNA"],
        "answer": "B",
    },
    {
        "id": "gpqa-010",
        "subject": "physics",
        "question": "In the Standard Model, the Higgs mechanism gives mass to which fundamental particles?",
        "choices": ["Only quarks", "Only W and Z bosons", "W bosons, Z bosons, and all fermions", "All particles including photons and gluons"],
        "answer": "C",
    },
 ]
 class GPQABenchmark(BenchmarkSuite):
    """GPQA Diamond: Graduate-level science QA (physics, chemistry, biology)."""
    name = "GPQA-Diamond"
    description = "Graduate-level science multiple-choice (diamond difficulty)"
    category = "knowledge"
    def load_dataset(self) -> list[dict[str, Any]]:
        jsonl_path = Path(self.data_dir) / "gpqa.jsonl"
        if jsonl_path.exists():
            problems: list[dict[str, Any]] = []
            with open(jsonl_path) as fh:
                for line in fh:
                    line = line.strip()
                    if line:
                        problems.append(json.loads(line))
            if self.verbose:
                print(f"  Loaded {len(problems)} problems from {jsonl_path}")
            return problems
        if self.verbose:
            print(f"  {jsonl_path} not found — using built-in 10-problem subset")
        return list(_BUILTIN_PROBLEMS)
    def build_prompt(self, problem: dict[str, Any]) -> str:
        question = problem["question"]
        choices = problem.get("choices", [])
        letters = "ABCD"
        choices_text = "\n".join(
            f"  {letters[i]}. {c}" for i, c in enumerate(choices)
        )
        return (
            f"Answer the following graduate-level science question.\n\n"
            f"Question: {question}\n\n"
            f"Choices:\n{choices_text}\n\n"
            f"Think carefully and write ONLY the letter of the correct answer (A–D) "
            f"to a file called answer.txt — no explanation, just the single letter."
        )
    def recover_output_files(
        self,
        problem: dict[str, Any],
        workspace: str,
        agent_output: str,
        metadata: dict[str, Any],
    ) -> None:
        del problem
        answer_path = Path(workspace) / "answer.txt"
        if answer_path.exists():
            return
        match = re.search(r"\b([A-D])\b", agent_output)
        if match:
            answer_path.write_text(match.group(1) + "\n", encoding="utf-8")
            metadata["recovered_answer_from_output"] = True
    def evaluate(self, problem: dict[str, Any], workspace: str) -> BenchmarkResult:
        pid = problem.get("id", "unknown")
        expected = str(problem["answer"]).strip().upper()
        answer_file = os.path.join(workspace, "answer.txt")
        if not os.path.exists(answer_file):
            return BenchmarkResult(
                problem_id=pid, passed=False, expected=expected,
                error="answer.txt not found",
            )
        with open(answer_file) as fh:
            actual_raw = fh.read().strip()
        match = re.search(r"\b([A-D])\b", actual_raw.upper())
        actual = match.group(1) if match else actual_raw.strip().upper()
        passed = actual == expected
        return BenchmarkResult(
            problem_id=pid, passed=passed,
            expected=expected, actual=actual_raw,
            error="" if passed else f"expected={expected}, got={actual_raw}",
        )
@@ -0,0 +1,199 @@
 """
 HLE (Humanity's Last Exam) benchmark suite.
 HLE is an extremely challenging benchmark designed to test the limits of
 AI capabilities. It contains questions from diverse domains that are
 intended to be among the hardest questions answerable by humans.
 Reference: https://huggingface.co/datasets/cais/hle
 """
 from __future__ import annotations
 import json
 import os
 import re
 from pathlib import Path
 from typing import Any
 from .base import BenchmarkResult, BenchmarkSuite
 # ---------------------------------------------------------------------------
 # Built-in mini dataset (10 representative HLE-style problems)
 # ---------------------------------------------------------------------------
 _BUILTIN_PROBLEMS: list[dict[str, Any]] = [
    {
        "id": "hle-001",
        "subject": "mathematics",
        "question": "What is the smallest positive integer n such that n! ends with exactly 100 trailing zeros?",
        "answer_type": "exact",
        "answer": "405",
    },
    {
        "id": "hle-002",
        "subject": "physics",
        "question": "In natural units (ℏ = c = 1), the fine-structure constant α ≈ 1/137. What is the approximate ratio of the electromagnetic force to the gravitational force between two protons?",
        "answer_type": "multiple_choice",
        "choices": ["10^36", "10^24", "10^42", "10^18"],
        "answer": "A",
    },
    {
        "id": "hle-003",
        "subject": "computer_science",
        "question": "What is the time complexity of the best known algorithm for matrix multiplication as of 2024?",
        "answer_type": "multiple_choice",
        "choices": ["O(n^2.371552)", "O(n^2.5)", "O(n^3)", "O(n^2 log n)"],
        "answer": "A",
    },
    {
        "id": "hle-004",
        "subject": "mathematics",
        "question": "How many groups of order 16 are there up to isomorphism?",
        "answer_type": "exact",
        "answer": "14",
    },
    {
        "id": "hle-005",
        "subject": "chemistry",
        "question": "What is the maximum number of stereoisomers possible for a molecule with 3 chiral centers and no meso forms?",
        "answer_type": "exact",
        "answer": "8",
    },
    {
        "id": "hle-006",
        "subject": "biology",
        "question": "The human genome contains approximately how many protein-coding genes?",
        "answer_type": "multiple_choice",
        "choices": ["~5,000", "~20,000", "~100,000", "~500,000"],
        "answer": "B",
    },
    {
        "id": "hle-007",
        "subject": "mathematics",
        "question": "What is the value of the Ramanujan sum c_5(3)?",
        "answer_type": "exact",
        "answer": "1",
    },
    {
        "id": "hle-008",
        "subject": "physics",
        "question": "What is the spin of the Higgs boson?",
        "answer_type": "exact",
        "answer": "0",
    },
    {
        "id": "hle-009",
        "subject": "computer_science",
        "question": "In computational complexity theory, which of the following containment relationships is known to be strict?",
        "answer_type": "multiple_choice",
        "choices": ["P ⊂ NP", "AC0 ⊂ NC1", "NP ⊂ PSPACE", "L ⊂ NL"],
        "answer": "B",
    },
    {
        "id": "hle-010",
        "subject": "mathematics",
        "question": "What is the chromatic number of the Petersen graph?",
        "answer_type": "exact",
        "answer": "3",
    },
 ]
 class HLEBenchmark(BenchmarkSuite):
    """HLE: Humanity's Last Exam — extremely challenging expert questions."""
    name = "HLE"
    description = "Extremely challenging expert-level questions (Humanity's Last Exam)"
    category = "knowledge"
    def load_dataset(self) -> list[dict[str, Any]]:
        jsonl_path = Path(self.data_dir) / "hle.jsonl"
        if jsonl_path.exists():
            problems: list[dict[str, Any]] = []
            with open(jsonl_path) as fh:
                for line in fh:
                    line = line.strip()
                    if line:
                        problems.append(json.loads(line))
            if self.verbose:
                print(f"  Loaded {len(problems)} problems from {jsonl_path}")
            return problems
        if self.verbose:
            print(f"  {jsonl_path} not found — using built-in 10-problem subset")
        return list(_BUILTIN_PROBLEMS)
    def build_prompt(self, problem: dict[str, Any]) -> str:
        question = problem["question"]
        answer_type = problem.get("answer_type", "exact")
        if answer_type == "multiple_choice" and "choices" in problem:
            choices = problem["choices"]
            letters = "ABCD"
            choices_text = "\n".join(
                f"  {letters[i]}. {c}" for i, c in enumerate(choices)
            )
            return (
                f"Answer the following extremely challenging question.\n\n"
                f"Question: {question}\n\n"
                f"Choices:\n{choices_text}\n\n"
                f"Think very carefully. Write ONLY the letter of the correct answer "
                f"to a file called answer.txt — no explanation, just the single letter."
            )
        else:
            return (
                f"Answer the following extremely challenging question.\n\n"
                f"Question: {question}\n\n"
                f"Think very carefully. Write ONLY the final answer "
                f"to a file called answer.txt — no explanation, just the answer."
            )
    def recover_output_files(
        self,
        problem: dict[str, Any],
        workspace: str,
        agent_output: str,
        metadata: dict[str, Any],
    ) -> None:
        del problem
        answer_path = Path(workspace) / "answer.txt"
        if answer_path.exists():
            return
        lines = agent_output.strip().splitlines()
        if lines:
            answer_path.write_text(lines[-1].strip() + "\n", encoding="utf-8")
            metadata["recovered_answer_from_output"] = True
    def evaluate(self, problem: dict[str, Any], workspace: str) -> BenchmarkResult:
        pid = problem.get("id", "unknown")
        expected = str(problem["answer"]).strip()
        answer_type = problem.get("answer_type", "exact")
        answer_file = os.path.join(workspace, "answer.txt")
        if not os.path.exists(answer_file):
            return BenchmarkResult(
                problem_id=pid, passed=False, expected=expected,
                error="answer.txt not found",
            )
        with open(answer_file) as fh:
            actual_raw = fh.read().strip()
        if answer_type == "multiple_choice":
            match = re.search(r"\b([A-D])\b", actual_raw.upper())
            actual = match.group(1) if match else actual_raw.strip().upper()
            passed = actual == expected.upper()
        else:
            # Exact match — normalize numbers
            actual = actual_raw.strip()
            try:
                passed = float(actual) == float(expected)
            except (ValueError, TypeError):
                passed = actual.lower() == expected.lower()
        return BenchmarkResult(
            problem_id=pid, passed=passed,
            expected=expected, actual=actual_raw,
            error="" if passed else f"expected={expected}, got={actual_raw}",
        )
@@ -0,0 +1,176 @@
 """
 MMLU-Pro benchmark suite.
 MMLU-Pro (Massive Multitask Language Understanding - Professional) is an
 enhanced version of MMLU with harder questions, 10 answer choices (A–J),
 and chain-of-thought reasoning requirements across 14 subjects.
 Reference: https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro
 """
 from __future__ import annotations
 import json
 import os
 import re
 from pathlib import Path
 from typing import Any
 from .base import BenchmarkResult, BenchmarkSuite
 # ---------------------------------------------------------------------------
 # Built-in mini dataset (10 representative MMLU-Pro problems)
 # ---------------------------------------------------------------------------
 _BUILTIN_PROBLEMS: list[dict[str, Any]] = [
    {
        "id": "mmlu-pro-001",
        "subject": "computer_science",
        "question": "Which of the following best describes the time complexity of binary search on a sorted array of n elements?",
        "choices": ["O(1)", "O(log n)", "O(n)", "O(n log n)", "O(n^2)", "O(2^n)", "O(n!)", "O(sqrt(n))", "O(n^3)", "O(log log n)"],
        "answer": "B",
    },
    {
        "id": "mmlu-pro-002",
        "subject": "physics",
        "question": "A 2 kg block is placed on a frictionless inclined plane that makes a 30° angle with the horizontal. What is the acceleration of the block down the incline? (g = 9.8 m/s²)",
        "choices": ["2.45 m/s²", "4.9 m/s²", "6.93 m/s²", "9.8 m/s²", "3.27 m/s²", "8.49 m/s²", "1.63 m/s²", "5.66 m/s²", "7.35 m/s²", "0.98 m/s²"],
        "answer": "B",
    },
    {
        "id": "mmlu-pro-003",
        "subject": "chemistry",
        "question": "What is the hybridization of the central atom in SF6?",
        "choices": ["sp", "sp2", "sp3", "sp3d", "sp3d2", "dsp3", "d2sp3", "sp2d", "sp3d3", "none of the above"],
        "answer": "E",
    },
    {
        "id": "mmlu-pro-004",
        "subject": "mathematics",
        "question": "What is the derivative of f(x) = x³ · ln(x)?",
        "choices": ["3x² · ln(x)", "x² + 3x² · ln(x)", "3x² · ln(x) + x²", "x³/x + 3x²", "3x² · ln(x) + x³", "x² · (3ln(x) + 1)", "3x · ln(x) + x²", "x³ · (1/x) + 3x² · ln(x)", "3x² / ln(x)", "ln(x) + 3x²"],
        "answer": "F",
    },
    {
        "id": "mmlu-pro-005",
        "subject": "biology",
        "question": "Which of the following is NOT a characteristic of the adaptive immune response?",
        "choices": ["Specificity", "Memory", "Rapid initial response", "Diversity", "Self/non-self discrimination", "Clonal expansion", "Tolerance", "MHC restriction", "Somatic hypermutation", "Pattern recognition"],
        "answer": "C",
    },
    {
        "id": "mmlu-pro-006",
        "subject": "history",
        "question": "The Treaty of Westphalia (1648) is most significant because it:",
        "choices": ["Ended the Hundred Years War", "Established the principle of state sovereignty", "Created the United Nations", "Divided Africa among European powers", "Ended World War I", "Established the Holy Roman Empire", "Created NATO", "Ended the Napoleonic Wars", "Established the European Union", "Ended the Crusades"],
        "answer": "B",
    },
    {
        "id": "mmlu-pro-007",
        "subject": "economics",
        "question": "In a perfectly competitive market in long-run equilibrium, which of the following is true?",
        "choices": ["Price equals minimum ATC", "Firms earn positive economic profit", "Price exceeds marginal cost", "Firms produce at maximum ATC", "Barriers to entry exist", "Price equals maximum AVC", "Firms are price makers", "Marginal cost exceeds price", "Economic profit is negative", "Supply is perfectly inelastic"],
        "answer": "A",
    },
    {
        "id": "mmlu-pro-008",
        "subject": "philosophy",
        "question": "According to Kant's categorical imperative, an action is morally permissible if and only if:",
        "choices": ["It maximizes overall happiness", "It can be universalized without contradiction", "It promotes the greatest good for the greatest number", "God commands it", "It follows natural law", "It leads to virtuous character", "It is agreed upon by rational contractors", "It satisfies one's desires", "It is consistent with social norms", "It minimizes suffering"],
        "answer": "B",
    },
    {
        "id": "mmlu-pro-009",
        "subject": "law",
        "question": "Under the U.S. Constitution, which amendment protects against unreasonable searches and seizures?",
        "choices": ["First Amendment", "Second Amendment", "Third Amendment", "Fourth Amendment", "Fifth Amendment", "Sixth Amendment", "Seventh Amendment", "Eighth Amendment", "Ninth Amendment", "Tenth Amendment"],
        "answer": "D",
    },
    {
        "id": "mmlu-pro-010",
        "subject": "psychology",
        "question": "In Piaget's theory of cognitive development, the stage in which children develop the ability to think abstractly and reason hypothetically is called the:",
        "choices": ["Sensorimotor stage", "Preoperational stage", "Concrete operational stage", "Formal operational stage", "Postformal stage", "Latency stage", "Genital stage", "Identity vs. role confusion", "Autonomy vs. shame", "Initiative vs. guilt"],
        "answer": "D",
    },
 ]
 class MMLUProBenchmark(BenchmarkSuite):
    """MMLU-Pro: Enhanced multitask language understanding with 10-choice questions."""
    name = "MMLU-Pro"
    description = "Professional-level multiple-choice QA across 14 subjects (10 choices)"
    category = "knowledge"
    def load_dataset(self) -> list[dict[str, Any]]:
        jsonl_path = Path(self.data_dir) / "mmlu_pro.jsonl"
        if jsonl_path.exists():
            problems: list[dict[str, Any]] = []
            with open(jsonl_path) as fh:
                for line in fh:
                    line = line.strip()
                    if line:
                        problems.append(json.loads(line))
            if self.verbose:
                print(f"  Loaded {len(problems)} problems from {jsonl_path}")
            return problems
        if self.verbose:
            print(f"  {jsonl_path} not found — using built-in 10-problem subset")
        return list(_BUILTIN_PROBLEMS)
    def build_prompt(self, problem: dict[str, Any]) -> str:
        question = problem["question"]
        choices = problem.get("choices", [])
        letters = "ABCDEFGHIJ"
        choices_text = "\n".join(
            f"  {letters[i]}. {c}" for i, c in enumerate(choices)
        )
        return (
            f"Answer the following multiple-choice question.\n\n"
            f"Question: {question}\n\n"
            f"Choices:\n{choices_text}\n\n"
            f"Write ONLY the letter of the correct answer (A–J) to a file called answer.txt — "
            f"no explanation, just the single letter."
        )
    def recover_output_files(
        self,
        problem: dict[str, Any],
        workspace: str,
        agent_output: str,
        metadata: dict[str, Any],
    ) -> None:
        del problem
        answer_path = Path(workspace) / "answer.txt"
        if answer_path.exists():
            return
        match = re.search(r"\b([A-J])\b", agent_output)
        if match:
            answer_path.write_text(match.group(1) + "\n", encoding="utf-8")
            metadata["recovered_answer_from_output"] = True
    def evaluate(self, problem: dict[str, Any], workspace: str) -> BenchmarkResult:
        pid = problem.get("id", "unknown")
        expected = str(problem["answer"]).strip().upper()
        answer_file = os.path.join(workspace, "answer.txt")
        if not os.path.exists(answer_file):
            return BenchmarkResult(
                problem_id=pid, passed=False, expected=expected,
                error="answer.txt not found",
            )
        with open(answer_file) as fh:
            actual_raw = fh.read().strip()
        match = re.search(r"\b([A-J])\b", actual_raw.upper())
        actual = match.group(1) if match else actual_raw.strip().upper()
        passed = actual == expected
        return BenchmarkResult(
            problem_id=pid, passed=passed,
            expected=expected, actual=actual_raw,
            error="" if passed else f"expected={expected}, got={actual_raw}",
        )
@@ -0,0 +1,191 @@
 """
 MMMLU (Multilingual Massive Multitask Language Understanding) benchmark suite.
 MMMLU extends MMLU to multiple languages, testing language understanding
 across diverse cultural and linguistic contexts.
 Reference: https://huggingface.co/datasets/openai/MMMLU
 """
 from __future__ import annotations
 import json
 import os
 import re
 from pathlib import Path
 from typing import Any
 from .base import BenchmarkResult, BenchmarkSuite
 # ---------------------------------------------------------------------------
 # Built-in mini dataset (10 representative MMMLU problems — multilingual)
 # ---------------------------------------------------------------------------
 _BUILTIN_PROBLEMS: list[dict[str, Any]] = [
    {
        "id": "mmmlu-001",
        "language": "en",
        "subject": "abstract_algebra",
        "question": "Find the degree of the extension Q(sqrt(2), sqrt(3), sqrt(18)) over Q.",
        "choices": ["0", "4", "2", "6"],
        "answer": "B",
    },
    {
        "id": "mmmlu-002",
        "language": "es",
        "subject": "anatomy",
        "question": "¿Cuál de los siguientes músculos es el principal responsable de la abducción del brazo?",
        "choices": ["Deltoides", "Trapecio", "Pectoral mayor", "Dorsal ancho"],
        "answer": "A",
    },
    {
        "id": "mmmlu-003",
        "language": "fr",
        "subject": "astronomy",
        "question": "Quelle est la planète la plus proche du Soleil?",
        "choices": ["Vénus", "Terre", "Mercure", "Mars"],
        "answer": "C",
    },
    {
        "id": "mmmlu-004",
        "language": "de",
        "subject": "business_ethics",
        "question": "Welcher der folgenden Begriffe beschreibt die Verantwortung eines Unternehmens gegenüber der Gesellschaft am besten?",
        "choices": ["Corporate Social Responsibility", "Shareholder Theory", "Laissez-faire", "Monopol"],
        "answer": "A",
    },
    {
        "id": "mmmlu-005",
        "language": "zh",
        "subject": "computer_science",
        "question": "在计算机科学中，二叉搜索树的平均时间复杂度是多少？",
        "choices": ["O(n)", "O(log n)", "O(n²)", "O(1)"],
        "answer": "B",
    },
    {
        "id": "mmmlu-006",
        "language": "ja",
        "subject": "world_history",
        "question": "フランス革命が始まったのは何年ですか？",
        "choices": ["1776年", "1789年", "1804年", "1815年"],
        "answer": "B",
    },
    {
        "id": "mmmlu-007",
        "language": "pt",
        "subject": "geography",
        "question": "Qual é o rio mais longo do mundo?",
        "choices": ["Amazonas", "Nilo", "Mississipi", "Yangtzé"],
        "answer": "B",
    },
    {
        "id": "mmmlu-008",
        "language": "it",
        "subject": "philosophy",
        "question": "Chi ha scritto 'La Repubblica'?",
        "choices": ["Aristotele", "Platone", "Socrate", "Epicuro"],
        "answer": "B",
    },
    {
        "id": "mmmlu-009",
        "language": "ko",
        "subject": "physics",
        "question": "뉴턴의 제2법칙에서 힘의 공식은?",
        "choices": ["F = mv", "F = ma", "F = mg", "F = mc²"],
        "answer": "B",
    },
    {
        "id": "mmmlu-010",
        "language": "ar",
        "subject": "chemistry",
        "question": "ما هو العنصر الأكثر وفرة في القشرة الأرضية؟",
        "choices": ["الحديد", "الأكسجين", "السيليكون", "الألمنيوم"],
        "answer": "B",
    },
 ]
 class MMMMLUBenchmark(BenchmarkSuite):
    """MMMLU: Multilingual MMLU across diverse languages."""
    name = "MMMLU"
    description = "Multilingual multiple-choice QA across languages and subjects"
    category = "knowledge"
    def load_dataset(self) -> list[dict[str, Any]]:
        jsonl_path = Path(self.data_dir) / "mmmlu.jsonl"
        if jsonl_path.exists():
            problems: list[dict[str, Any]] = []
            with open(jsonl_path) as fh:
                for line in fh:
                    line = line.strip()
                    if line:
                        problems.append(json.loads(line))
            if self.verbose:
                print(f"  Loaded {len(problems)} problems from {jsonl_path}")
            return problems
        if self.verbose:
            print(f"  {jsonl_path} not found — using built-in 10-problem subset")
        return list(_BUILTIN_PROBLEMS)
    def build_prompt(self, problem: dict[str, Any]) -> str:
        question = problem["question"]
        choices = problem.get("choices", [])
        language = problem.get("language", "en")
        letters = "ABCD"
        choices_text = "\n".join(
            f"  {letters[i]}. {c}" for i, c in enumerate(choices)
        )
        lang_instruction = ""
        if language != "en":
            lang_instruction = f"The question is in {language}. "
        return (
            f"Answer the following multiple-choice question. {lang_instruction}\n\n"
            f"Question: {question}\n\n"
            f"Choices:\n{choices_text}\n\n"
            f"Write ONLY the letter of the correct answer (A–D) to a file called answer.txt — "
            f"no explanation, just the single letter."
        )
    def recover_output_files(
        self,
        problem: dict[str, Any],
        workspace: str,
        agent_output: str,
        metadata: dict[str, Any],
    ) -> None:
        del problem
        answer_path = Path(workspace) / "answer.txt"
        if answer_path.exists():
            return
        match = re.search(r"\b([A-D])\b", agent_output)
        if match:
            answer_path.write_text(match.group(1) + "\n", encoding="utf-8")
            metadata["recovered_answer_from_output"] = True
    def evaluate(self, problem: dict[str, Any], workspace: str) -> BenchmarkResult:
        pid = problem.get("id", "unknown")
        expected = str(problem["answer"]).strip().upper()
        answer_file = os.path.join(workspace, "answer.txt")
        if not os.path.exists(answer_file):
            return BenchmarkResult(
                problem_id=pid, passed=False, expected=expected,
                error="answer.txt not found",
            )
        with open(answer_file) as fh:
            actual_raw = fh.read().strip()
        match = re.search(r"\b([A-D])\b", actual_raw.upper())
        actual = match.group(1) if match else actual_raw.strip().upper()
        passed = actual == expected
        return BenchmarkResult(
            problem_id=pid, passed=passed,
            expected=expected, actual=actual_raw,
            error="" if passed else f"expected={expected}, got={actual_raw}",
        )
@@ -0,0 +1,181 @@
 """
 Tau2 benchmark suite.
 Tau2 (TAU-bench v2) evaluates language models on tool-augmented tasks
 requiring multi-step reasoning with tool use. It measures the ability
 to plan, select, and chain tool calls to complete complex tasks.
 The benchmark reports an average score across 3 task domains:
 retail, airline, and finance.
 Reference: https://github.com/sierra-research/tau-bench
 """
 from __future__ import annotations
 import json
 import os
 import re
 from pathlib import Path
 from typing import Any
 from .base import BenchmarkResult, BenchmarkSuite
 # ---------------------------------------------------------------------------
 # Built-in mini dataset (10 representative Tau2-style problems)
 # ---------------------------------------------------------------------------
 _BUILTIN_PROBLEMS: list[dict[str, Any]] = [
    {
        "id": "tau2-001",
        "domain": "retail",
        "question": "A customer wants to return a laptop purchased 20 days ago. The store policy allows returns within 30 days with receipt. The customer has the receipt. The laptop is in original packaging. What is the correct action?",
        "choices": ["Process full refund", "Deny return - past return window", "Offer store credit only", "Process exchange only"],
        "answer": "A",
    },
    {
        "id": "tau2-002",
        "domain": "airline",
        "question": "A passenger has a connecting flight with a 45-minute layover. Their first flight is delayed by 30 minutes. The minimum connection time at the hub airport is 60 minutes. What should the agent recommend?",
        "choices": ["Rebook on next available flight", "Keep current booking, passenger will make it", "Cancel entire itinerary", "Upgrade to first class on delayed flight"],
        "answer": "A",
    },
    {
        "id": "tau2-003",
        "domain": "finance",
        "question": "A client wants to transfer $50,000 between accounts. The daily transfer limit is $25,000. The client has proper identification. What is the correct procedure?",
        "choices": ["Split into two $25,000 transfers over two days", "Override limit and process single transfer", "Deny the transfer entirely", "Process as a wire transfer instead"],
        "answer": "A",
    },
    {
        "id": "tau2-004",
        "domain": "retail",
        "question": "An item shows a price of $29.99 on the shelf but rings up as $34.99 at checkout. Store policy states the customer gets the lower price if there's a discrepancy. The shelf tag is verified to be for this item. What action should be taken?",
        "choices": ["Honor the shelf price of $29.99", "Charge the register price of $34.99", "Split the difference", "Give the item for free"],
        "answer": "A",
    },
    {
        "id": "tau2-005",
        "domain": "airline",
        "question": "A passenger with a non-refundable ticket wants to change their travel date. The fare difference is +$150 and there is a $75 change fee. The passenger is a Gold status member (change fees waived). What is the total additional cost?",
        "choices": ["$150", "$225", "$75", "$0"],
        "answer": "A",
    },
    {
        "id": "tau2-006",
        "domain": "finance",
        "question": "A customer's account shows 3 pending transactions totaling $500, but their available balance is $400. One pending transaction of $200 is a pre-authorization from a gas station from 5 days ago. What should the agent do?",
        "choices": ["Release the gas station pre-authorization as it exceeds the standard hold period", "Decline all pending transactions", "Suggest the customer deposit more funds", "Freeze the account for suspicious activity"],
        "answer": "A",
    },
    {
        "id": "tau2-007",
        "domain": "retail",
        "question": "A customer purchased an appliance with a 2-year manufacturer warranty 18 months ago. The appliance has stopped working due to a manufacturing defect. The customer does not have an extended warranty. What should the agent do?",
        "choices": ["Process warranty claim with manufacturer", "Offer store repair at customer's cost", "Deny any assistance", "Offer replacement at discounted price"],
        "answer": "A",
    },
    {
        "id": "tau2-008",
        "domain": "airline",
        "question": "A flight is overbooked by 2 passengers. 3 passengers volunteer to give up their seats. Compensation offered is $400 voucher + next flight (2 hours later). Airline policy requires bumping in reverse check-in order if volunteers insufficient. What should happen?",
        "choices": ["Accept 2 of the 3 volunteers, compensate them", "Accept all 3 volunteers", "Involuntarily deny 2 passengers boarding", "Cancel the flight"],
        "answer": "A",
    },
    {
        "id": "tau2-009",
        "domain": "finance",
        "question": "A customer reports their credit card stolen. There are 3 unauthorized transactions in the last 24 hours totaling $1,200. Federal regulation limits customer liability for unauthorized transactions reported within 2 business days. What steps should be taken first?",
        "choices": ["Block the card immediately and initiate fraud investigation", "Wait for the monthly statement to dispute charges", "Transfer balance to new card without blocking old one", "Ask customer to contact merchants directly"],
        "answer": "A",
    },
    {
        "id": "tau2-010",
        "domain": "retail",
        "question": "A customer wants to price match an identical item found cheaper at a competitor. Store policy allows price matching for identical items at local competitors within 14 days of purchase. The customer purchased 10 days ago and has proof of the competitor's current price. The competitor is an online-only store. Store policy explicitly excludes online-only retailers. What should the agent do?",
        "choices": ["Deny price match - online-only retailers excluded", "Honor the price match anyway", "Offer partial price adjustment", "Contact competitor to verify price"],
        "answer": "A",
    },
 ]
 class Tau2Benchmark(BenchmarkSuite):
    """Tau2: Tool-augmented task evaluation across retail, airline, and finance."""
    name = "Tau2"
    description = "Tool-augmented multi-step reasoning (retail/airline/finance)"
    category = "reasoning"
    def load_dataset(self) -> list[dict[str, Any]]:
        jsonl_path = Path(self.data_dir) / "tau2.jsonl"
        if jsonl_path.exists():
            problems: list[dict[str, Any]] = []
            with open(jsonl_path) as fh:
                for line in fh:
                    line = line.strip()
                    if line:
                        problems.append(json.loads(line))
            if self.verbose:
                print(f"  Loaded {len(problems)} problems from {jsonl_path}")
            return problems
        if self.verbose:
            print(f"  {jsonl_path} not found — using built-in 10-problem subset")
        return list(_BUILTIN_PROBLEMS)
    def build_prompt(self, problem: dict[str, Any]) -> str:
        question = problem["question"]
        choices = problem.get("choices", [])
        domain = problem.get("domain", "general")
        letters = "ABCD"
        choices_text = "\n".join(
            f"  {letters[i]}. {c}" for i, c in enumerate(choices)
        )
        return (
            f"You are a {domain} service agent. Answer the following question "
            f"about the correct action to take.\n\n"
            f"Scenario: {question}\n\n"
            f"Options:\n{choices_text}\n\n"
            f"Write ONLY the letter of the correct answer (A–D) to a file called answer.txt — "
            f"no explanation, just the single letter."
        )
    def recover_output_files(
        self,
        problem: dict[str, Any],
        workspace: str,
        agent_output: str,
        metadata: dict[str, Any],
    ) -> None:
        del problem
        answer_path = Path(workspace) / "answer.txt"
        if answer_path.exists():
            return
        match = re.search(r"\b([A-D])\b", agent_output)
        if match:
            answer_path.write_text(match.group(1) + "\n", encoding="utf-8")
            metadata["recovered_answer_from_output"] = True
    def evaluate(self, problem: dict[str, Any], workspace: str) -> BenchmarkResult:
        pid = problem.get("id", "unknown")
        expected = str(problem["answer"]).strip().upper()
        answer_file = os.path.join(workspace, "answer.txt")
        if not os.path.exists(answer_file):
            return BenchmarkResult(
                problem_id=pid, passed=False, expected=expected,
                error="answer.txt not found",
            )
        with open(answer_file) as fh:
            actual_raw = fh.read().strip()
        match = re.search(r"\b([A-D])\b", actual_raw.upper())
        actual = match.group(1) if match else actual_raw.strip().upper()
        passed = actual == expected
        return BenchmarkResult(
            problem_id=pid, passed=passed,
            expected=expected, actual=actual_raw,
            error="" if passed else f"expected={expected}, got={actual_raw}",
        )
@@ -99,6 +99,8 @@ class LocalCodingAgent:
    worktree_runtime: WorktreeRuntime | None = None
    last_session: AgentSessionState | None = field(default=None, init=False, repr=False)
    last_run_result: AgentRunResult | None = field(default=None, init=False, repr=False)
    cumulative_usage: UsageStats = field(default_factory=UsageStats, init=False, repr=False)
    cumulative_cost_usd: float = field(default=0.0, init=False, repr=False)
    active_session_id: str | None = field(default=None, init=False, repr=False)
    last_session_path: str | None = field(default=None, init=False, repr=False)
    managed_agent_id: str | None = field(default=None, init=False, repr=False)
@@ -321,6 +323,7 @@ class LocalCodingAgent:
            scratchpad_directory=scratchpad_directory,
            existing_file_history=(),
        )
        self._accumulate_usage(result)
        self._finalize_managed_agent(result)
        return result
@@ -357,6 +360,7 @@ class LocalCodingAgent:
            scratchpad_directory=scratchpad_directory,
            existing_file_history=stored_session.file_history,
        )
        self._accumulate_usage(result)
        self._finalize_managed_agent(result)
        return result
@@ -3363,6 +3367,11 @@ class LocalCodingAgent:
        )
        self.resume_source_session_id = None
    def _accumulate_usage(self, result: AgentRunResult) -> None:
        """Add a run's usage to the cumulative session totals."""
        self.cumulative_usage = self.cumulative_usage + result.usage
        self.cumulative_cost_usd += result.total_cost_usd
    def _refresh_runtime_views_for_tool_result(
        self,
        tool_name: str,
@@ -294,6 +294,71 @@ def get_slash_command_specs() -> tuple[SlashCommandSpec, ...]:
            description='Clear ephemeral Python runtime state for this process.',
            handler=_handle_clear,
        ),
        SlashCommandSpec(
            names=('compact',),
            description='Summarise and compact the conversation to free context space.',
            handler=_handle_compact,
        ),
        SlashCommandSpec(
            names=('cost',),
            description='Show the total cost and duration of the current session.',
            handler=_handle_cost,
        ),
        SlashCommandSpec(
            names=('exit', 'quit'),
            description='Exit the REPL.',
            handler=_handle_exit,
        ),
        SlashCommandSpec(
            names=('diff',),
            description='View uncommitted changes (git diff) in the working directory.',
            handler=_handle_diff,
        ),
        SlashCommandSpec(
            names=('files',),
            description='List files currently loaded in the session context.',
            handler=_handle_files,
        ),
        SlashCommandSpec(
            names=('copy',),
            description='Copy the last assistant response to a temp file.',
            handler=_handle_copy,
        ),
        SlashCommandSpec(
            names=('export',),
            description='Export the conversation to a text file.',
            handler=_handle_export,
        ),
        SlashCommandSpec(
            names=('stats',),
            description='Show session usage statistics.',
            handler=_handle_stats,
        ),
        SlashCommandSpec(
            names=('tag',),
            description='Add or remove a searchable tag on the current session.',
            handler=_handle_tag,
        ),
        SlashCommandSpec(
            names=('rename',),
            description='Rename the current conversation.',
            handler=_handle_rename,
        ),
        SlashCommandSpec(
            names=('branch',),
            description='Create a fork/branch of the current conversation.',
            handler=_handle_branch,
        ),
        SlashCommandSpec(
            names=('effort',),
            description='Show or set the model effort level (low, medium, high, max, auto).',
            handler=_handle_effort,
        ),
        SlashCommandSpec(
            names=('doctor',),
            description='Diagnose and verify the claw-code installation and settings.',
            handler=_handle_doctor,
        ),
    )
@@ -620,6 +685,417 @@ def _handle_clear(agent: 'LocalCodingAgent', _args: str, input_text: str) -> Sla
    )
 def _handle_compact(agent: 'LocalCodingAgent', args: str, input_text: str) -> SlashCommandResult:
    from .compact import compact_conversation
    custom_instructions = args.strip() if args.strip() else None
    result = compact_conversation(agent, custom_instructions)
    if result.error:
        return _local_result(input_text, f'Compact failed: {result.error}')
    lines = ['Conversation compacted.']
    if result.pre_compact_token_count:
        lines.append(
            f'  Tokens before: ~{result.pre_compact_token_count:,}  '
            f'→  after: ~{result.post_compact_token_count:,}'
        )
    return _local_result(input_text, '\n'.join(lines))
 def _handle_cost(agent: 'LocalCodingAgent', _args: str, input_text: str) -> SlashCommandResult:
    usage = agent.cumulative_usage
    cost = agent.cumulative_cost_usd
    def _fmt_cost(usd: float) -> str:
        if usd < 0.01:
            return f'${usd:.4f}'
        return f'${usd:.2f}'
    lines = [
        f'Total cost:            {_fmt_cost(cost)}',
        f'Total input tokens:    {usage.input_tokens:,}',
        f'Total output tokens:   {usage.output_tokens:,}',
    ]
    if usage.cache_read_input_tokens:
        lines.append(f'Cache read tokens:     {usage.cache_read_input_tokens:,}')
    if usage.cache_creation_input_tokens:
        lines.append(f'Cache creation tokens:  {usage.cache_creation_input_tokens:,}')
    if usage.reasoning_tokens:
        lines.append(f'Reasoning tokens:      {usage.reasoning_tokens:,}')
    lines.append(f'Total tokens:          {usage.total_tokens:,}')
    return _local_result(input_text, '\n'.join(lines))
 def _handle_exit(agent: 'LocalCodingAgent', _args: str, input_text: str) -> SlashCommandResult:
    import random
    import sys
    messages = ['Goodbye!', 'See ya!', 'Bye!', 'Catch you later!']
    output = random.choice(messages)
    # Build the result first so the transcript is recorded, then exit.
    result = _local_result(input_text, output)
    print(output)
    sys.exit(0)
    return result  # unreachable, but satisfies the type checker
 def _handle_diff(agent: 'LocalCodingAgent', _args: str, input_text: str) -> SlashCommandResult:
    import subprocess
    cwd = str(agent.runtime_config.cwd)
    try:
        proc = subprocess.run(
            ['git', 'diff'],
            cwd=cwd,
            capture_output=True,
            text=True,
            timeout=15,
        )
        diff_output = proc.stdout.strip()
        if not diff_output:
            # Also check staged changes
            proc_staged = subprocess.run(
                ['git', 'diff', '--staged'],
                cwd=cwd,
                capture_output=True,
                text=True,
                timeout=15,
            )
            diff_output = proc_staged.stdout.strip()
            if not diff_output:
                return _local_result(input_text, 'No uncommitted changes.')
            return _local_result(input_text, f'Staged changes:\n{diff_output}')
        return _local_result(input_text, diff_output)
    except FileNotFoundError:
        return _local_result(input_text, 'git is not available.')
    except subprocess.TimeoutExpired:
        return _local_result(input_text, 'git diff timed out.')
    except Exception as exc:
        return _local_result(input_text, f'Error running git diff: {exc}')
 def _handle_files(agent: 'LocalCodingAgent', _args: str, input_text: str) -> SlashCommandResult:
    """List files loaded in the session context (from readFileState)."""
    session = agent.last_session
    if session is None:
        return _local_result(input_text, 'No active session.')
    # Collect file paths mentioned in tool results
    file_paths: list[str] = []
    for msg in session.messages:
        if msg.role == 'tool' and msg.name in ('Read', 'read_file', 'ReadFile'):
            # Extract path from content or metadata
            path = msg.metadata.get('path')
            if isinstance(path, str):
                file_paths.append(path)
            elif msg.content and msg.content.startswith('/'):
                # First line might be the path
                first_line = msg.content.split('\n', 1)[0].strip()
                if '/' in first_line and len(first_line) < 256:
                    file_paths.append(first_line)
    # Also look at tool_calls in assistant messages
    for msg in session.messages:
        if msg.role == 'assistant' and msg.tool_calls:
            for tc in msg.tool_calls:
                func = tc.get('function', {}) if isinstance(tc, dict) else {}
                if func.get('name') in ('Read', 'read_file', 'ReadFile', 'View'):
                    import json as _json
                    try:
                        args = _json.loads(func.get('arguments', '{}'))
                        path = args.get('file_path') or args.get('path')
                        if isinstance(path, str):
                            file_paths.append(path)
                    except (ValueError, TypeError):
                        pass
    # Deduplicate preserving order
    seen: set[str] = set()
    unique_paths: list[str] = []
    for p in file_paths:
        if p not in seen:
            seen.add(p)
            unique_paths.append(p)
    if not unique_paths:
        return _local_result(input_text, 'No files loaded in context.')
    cwd = str(agent.runtime_config.cwd)
    relative_paths = []
    for p in unique_paths:
        if p.startswith(cwd):
            relative_paths.append(p[len(cwd):].lstrip('/'))
        else:
            relative_paths.append(p)
    lines = [f'Files in context ({len(relative_paths)}):']
    for p in relative_paths:
        lines.append(f'  {p}')
    return _local_result(input_text, '\n'.join(lines))
 def _handle_copy(agent: 'LocalCodingAgent', args: str, input_text: str) -> SlashCommandResult:
    """Copy the last assistant response to a temp file."""
    import tempfile as _tempfile
    session = agent.last_session
    if session is None:
        return _local_result(input_text, 'No active session.')
    # Find the Nth most recent assistant message (default N=0 = latest)
    n = 0
    if args.strip().isdigit():
        n = min(int(args.strip()), 20)
    assistant_messages = [
        msg for msg in session.messages
        if msg.role == 'assistant' and msg.content.strip()
    ]
    if not assistant_messages:
        return _local_result(input_text, 'No assistant responses to copy.')
    index = len(assistant_messages) - 1 - n
    if index < 0:
        return _local_result(
            input_text,
            f'Only {len(assistant_messages)} assistant responses available.',
        )
    content = assistant_messages[index].content
    # Write to temp file
    from pathlib import Path as _Path
    tmp_dir = _Path(_tempfile.gettempdir()) / 'claw-code'
    tmp_dir.mkdir(parents=True, exist_ok=True)
    out_path = tmp_dir / 'response.md'
    out_path.write_text(content, encoding='utf-8')
    char_count = len(content)
    line_count = content.count('\n') + 1
    return _local_result(
        input_text,
        f'Copied {char_count:,} chars ({line_count} lines) to {out_path}',
    )
 def _handle_export(agent: 'LocalCodingAgent', args: str, input_text: str) -> SlashCommandResult:
    """Export the conversation transcript to a text file."""
    from pathlib import Path as _Path
    import time as _time
    session = agent.last_session
    if session is None:
        return _local_result(input_text, 'No active session to export.')
    # Build plain-text transcript
    lines: list[str] = []
    for msg in session.messages:
        label = msg.role.upper()
        if msg.role == 'tool' and msg.name:
            label = f'TOOL:{msg.name}'
        lines.append(f'--- {label} ---')
        lines.append(msg.content)
        lines.append('')
    text = '\n'.join(lines)
    # Determine output path
    filename = args.strip()
    if not filename:
        timestamp = _time.strftime('%Y%m%d_%H%M%S')
        filename = f'conversation_{timestamp}.txt'
    if not filename.endswith('.txt'):
        filename += '.txt'
    out_path = _Path(str(agent.runtime_config.cwd)) / filename
    out_path.write_text(text, encoding='utf-8')
    return _local_result(
        input_text,
        f'Exported {len(session.messages)} messages to {out_path}',
    )
 def _handle_stats(agent: 'LocalCodingAgent', _args: str, input_text: str) -> SlashCommandResult:
    """Show session usage statistics."""
    usage = agent.cumulative_usage
    cost = agent.cumulative_cost_usd
    session = agent.last_session
    msg_count = len(session.messages) if session else 0
    user_msgs = sum(1 for m in (session.messages if session else []) if m.role == 'user')
    assistant_msgs = sum(1 for m in (session.messages if session else []) if m.role == 'assistant')
    tool_msgs = sum(1 for m in (session.messages if session else []) if m.role == 'tool')
    lines = [
        '## Session Statistics',
        '',
        f'Messages:     {msg_count} total ({user_msgs} user, {assistant_msgs} assistant, {tool_msgs} tool)',
        f'Input tokens:  {usage.input_tokens:,}',
        f'Output tokens: {usage.output_tokens:,}',
        f'Total tokens:  {usage.total_tokens:,}',
        f'Cost:          ${cost:.4f}',
        f'Model:         {agent.model_config.model}',
    ]
    if agent.active_session_id:
        lines.append(f'Session ID:    {agent.active_session_id}')
    return _local_result(input_text, '\n'.join(lines))
 def _handle_tag(agent: 'LocalCodingAgent', args: str, input_text: str) -> SlashCommandResult:
    """Add or remove a tag on the current session."""
    tag = args.strip()
    if not tag:
        # Show current tags
        tags = getattr(agent, '_session_tags', set())
        if tags:
            return _local_result(input_text, f'Session tags: {", ".join(sorted(tags))}')
        return _local_result(input_text, 'No tags set. Usage: /tag <tag-name>')
    # Toggle tag
    if not hasattr(agent, '_session_tags'):
        agent._session_tags = set()
    if tag in agent._session_tags:
        agent._session_tags.discard(tag)
        return _local_result(input_text, f'Removed tag: {tag}')
    agent._session_tags.add(tag)
    return _local_result(input_text, f'Added tag: {tag}')
 def _handle_rename(agent: 'LocalCodingAgent', args: str, input_text: str) -> SlashCommandResult:
    """Rename the current conversation."""
    name = args.strip()
    if not name:
        return _local_result(input_text, 'Usage: /rename <name>')
    if not hasattr(agent, '_session_name'):
        agent._session_name = None
    agent._session_name = name
    return _local_result(input_text, f'Session renamed to: {name}')
 def _handle_branch(agent: 'LocalCodingAgent', args: str, input_text: str) -> SlashCommandResult:
    """Create a fork/branch of the current conversation."""
    import json as _json
    from uuid import uuid4
    session = agent.last_session
    if session is None:
        return _local_result(input_text, 'No active session to branch.')
    branch_name = args.strip() or f'branch-{uuid4().hex[:8]}'
    new_session_id = uuid4().hex
    # Save a copy of the current transcript as a new session file
    session_dir = agent.runtime_config.session_directory
    session_dir.mkdir(parents=True, exist_ok=True)
    session_path = session_dir / f'{new_session_id}.json'
    transcript = [msg.to_transcript_entry() for msg in session.messages]
    branch_data = {
        'session_id': new_session_id,
        'branch_name': branch_name,
        'branched_from': agent.active_session_id,
        'messages': transcript,
        'model': agent.model_config.model,
    }
    try:
        session_path.write_text(_json.dumps(branch_data, indent=2), encoding='utf-8')
        return _local_result(
            input_text,
            f'Created branch "{branch_name}" (session: {new_session_id})\n'
            f'Saved to: {session_path}',
        )
    except Exception as exc:
        return _local_result(input_text, f'Error creating branch: {exc}')
 def _handle_effort(agent: 'LocalCodingAgent', args: str, input_text: str) -> SlashCommandResult:
    """Show or set the model effort level."""
    import os
    valid_levels = ('low', 'medium', 'high', 'max', 'auto')
    current = getattr(agent.runtime_config, 'effort_level', None)
    env_override = os.environ.get('CLAUDE_CODE_EFFORT_LEVEL')
    if not args.strip():
        level = current or env_override or 'auto'
        msg = f'Current effort level: {level}'
        if env_override:
            msg += f' (from CLAUDE_CODE_EFFORT_LEVEL env var)'
        return _local_result(input_text, msg)
    level = args.strip().lower()
    if level not in valid_levels:
        return _local_result(
            input_text,
            f'Invalid effort level: {level}\nValid levels: {", ".join(valid_levels)}',
        )
    if env_override:
        return _local_result(
            input_text,
            f'Cannot change effort level — overridden by '
            f'CLAUDE_CODE_EFFORT_LEVEL={env_override}',
        )
    # Store effort level on the runtime config
    object.__setattr__(agent.runtime_config, 'effort_level', level)
    return _local_result(input_text, f'Set effort level to: {level}')
 def _handle_doctor(agent: 'LocalCodingAgent', _args: str, input_text: str) -> SlashCommandResult:
    """Diagnose and verify the claw-code installation."""
    import os
    import shutil
    import sys
    from pathlib import Path as _Path
    checks: list[str] = []
    # Python version
    py_ver = f'{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}'
    ok = sys.version_info >= (3, 10)
    checks.append(f'{"✓" if ok else "✗"} Python version: {py_ver} (need ≥3.10)')
    # Git available
    git_ok = shutil.which('git') is not None
    checks.append(f'{"✓" if git_ok else "✗"} git: {"found" if git_ok else "NOT FOUND"}')
    # Model config
    checks.append(f'✓ Model: {agent.model_config.model}')
    checks.append(f'✓ Base URL: {agent.model_config.base_url}')
    # Working directory
    cwd = agent.runtime_config.cwd
    checks.append(f'✓ Working directory: {cwd}')
    checks.append(f'{"✓" if cwd.exists() else "✗"} Working directory exists: {cwd.exists()}')
    # Session directory
    sess_dir = agent.runtime_config.session_directory
    checks.append(f'✓ Session directory: {sess_dir}')
    checks.append(f'{"✓" if sess_dir.exists() else "○"} Session directory exists: {sess_dir.exists()}')
    # API key
    has_key = bool(agent.model_config.api_key)
    checks.append(f'{"✓" if has_key else "✗"} API key: {"set" if has_key else "NOT SET"}')
    # Tools
    tool_count = len(agent.tool_registry) if agent.tool_registry else 0
    checks.append(f'✓ Registered tools: {tool_count}')
    # Memory files (CLAUDE.md)
    claude_md = cwd / 'CLAUDE.md'
    checks.append(
        f'{"✓" if claude_md.exists() else "○"} CLAUDE.md: '
        f'{"found" if claude_md.exists() else "not found (optional)"}'
    )
    output = '## Doctor Report\n\n' + '\n'.join(checks)
    return _local_result(input_text, output)
 def _local_result(input_text: str, output: str) -> SlashCommandResult:
    transcript = (
        {'role': 'user', 'content': input_text},
@@ -0,0 +1,438 @@
 """Conversation compaction service.
 Mirrors the npm ``src/services/compact/compact.ts`` and
 ``src/services/compact/prompt.ts`` modules.  Provides:
 - The 9-section summarisation prompt (``get_compact_prompt``).
 - XML-tag formatting/stripping  (``format_compact_summary``).
 - The post-compact user summary message builder
  (``get_compact_user_summary_message``).
 - The core ``compact_conversation`` entry point that an
  ``/compact`` slash command or auto-compact subsystem can call.
 """
 from __future__ import annotations
 import re
 from dataclasses import dataclass, field
 from typing import TYPE_CHECKING, Any
 from .agent_context_usage import estimate_tokens
 from .agent_session import AgentMessage
 if TYPE_CHECKING:
    from .agent_runtime import LocalCodingAgent
 # ---------------------------------------------------------------------------
 # Constants
 # ---------------------------------------------------------------------------
 AUTOCOMPACT_BUFFER_TOKENS = 13_000
 """How many tokens to reserve below the effective context window before
 auto-compact fires (same as the npm ``AUTOCOMPACT_BUFFER_TOKENS``)."""
 ERROR_NOT_ENOUGH_MESSAGES = 'Not enough messages to compact.'
 ERROR_INCOMPLETE_RESPONSE = (
    'The summary response was incomplete.  '
    'The conversation was not compacted.'
 )
 ERROR_USER_ABORT = 'Compaction canceled.'
 MAX_COMPACT_FAILURES = 3
 """Circuit-breaker – stop retrying auto-compact after this many consecutive
 failures (mirrors the npm implementation)."""
 # ---------------------------------------------------------------------------
 # Prompt construction  (npm ``src/services/compact/prompt.ts``)
 # ---------------------------------------------------------------------------
 _NO_TOOLS_PREAMBLE = """\
 CRITICAL: Respond with TEXT ONLY. Do NOT call any tools.
 - Do NOT use Read, Bash, Grep, Glob, Edit, Write, or ANY other tool.
 - You already have all the context you need in the conversation above.
 - Tool calls will be REJECTED and will waste your only turn — you will fail the task.
 - Your entire response must be plain text: an <analysis> block followed by a <summary> block.
 """
 _DETAILED_ANALYSIS_INSTRUCTION = """\
 Before providing your final summary, wrap your analysis in <analysis> tags to \
 organize your thoughts and ensure you've covered all necessary points. In your \
 analysis process:
 1. Chronologically analyze each message and section of the conversation. \
 For each section thoroughly identify:
   - The user's explicit requests and intents
   - Your approach to addressing the user's requests
   - Key decisions, technical concepts and code patterns
   - Specific details like:
     - file names
     - full code snippets
     - function signatures
     - file edits
   - Errors that you ran into and how you fixed them
   - Pay special attention to specific user feedback that you received, \
 especially if the user told you to do something differently.
 2. Double-check for technical accuracy and completeness, addressing each \
 required element thoroughly."""
 _BASE_COMPACT_PROMPT = f"""\
 Your task is to create a detailed summary of the conversation so far, paying \
 close attention to the user's explicit requests and your previous actions.
 This summary should be thorough in capturing technical details, code patterns, \
 and architectural decisions that would be essential for continuing development \
 work without losing context.
 {_DETAILED_ANALYSIS_INSTRUCTION}
 Your summary should include the following sections:
 1. Primary Request and Intent: Capture all of the user's explicit requests \
 and intents in detail
 2. Key Technical Concepts: List all important technical concepts, technologies, \
 and frameworks discussed.
 3. Files and Code Sections: Enumerate specific files and code sections examined, \
 modified, or created. Pay special attention to the most recent messages and \
 include full code snippets where applicable and include a summary of why this \
 file read or edit is important.
 4. Errors and fixes: List all errors that you ran into, and how you fixed them. \
 Pay special attention to specific user feedback that you received, especially if \
 the user told you to do something differently.
 5. Problem Solving: Document problems solved and any ongoing troubleshooting \
 efforts.
 6. All user messages: List ALL user messages that are not tool results. These \
 are critical for understanding the users' feedback and changing intent.
 7. Pending Tasks: Outline any pending tasks that you have explicitly been asked \
 to work on.
 8. Current Work: Describe in detail precisely what was being worked on \
 immediately before this summary request, paying special attention to the most \
 recent messages from both user and assistant. Include file names and code \
 snippets where applicable.
 9. Optional Next Step: List the next step that you will take that is related to \
 the most recent work you were doing. IMPORTANT: ensure that this step is \
 DIRECTLY in line with the user's most recent explicit requests, and the task \
 you were working on immediately before this summary request. If your last task \
 was concluded, then only list next steps if they are explicitly in line with the \
 users request. Do not start on tangential requests or really old requests that \
 were already completed without confirming with the user first.
                       If there is a next step, include direct quotes from the \
 most recent conversation showing exactly what task you were working on and where \
 you left off. This should be verbatim to ensure there's no drift in task \
 interpretation.
 Here's an example of how your output should be structured:
 <example>
 <analysis>
 [Your thought process, ensuring all points are covered thoroughly and accurately]
 </analysis>
 <summary>
 1. Primary Request and Intent:
   [Detailed description]
 2. Key Technical Concepts:
   - [Concept 1]
   - [Concept 2]
   - [...]
 3. Files and Code Sections:
   - [File Name 1]
      - [Summary of why this file is important]
      - [Summary of the changes made to this file, if any]
      - [Important Code Snippet]
   - [File Name 2]
      - [Important Code Snippet]
   - [...]
 4. Errors and fixes:
    - [Detailed description of error 1]:
      - [How you fixed the error]
      - [User feedback on the error if any]
    - [...]
 5. Problem Solving:
   [Description of solved problems and ongoing troubleshooting]
 6. All user messages:
    - [Detailed non tool use user message]
    - [...]
 7. Pending Tasks:
   - [Task 1]
   - [Task 2]
   - [...]
 8. Current Work:
   [Precise description of current work]
 9. Optional Next Step:
   [Optional Next step to take]
 </summary>
 </example>
 Please provide your summary based on the conversation so far, following this \
 structure and ensuring precision and thoroughness in your response.
 There may be additional summarization instructions provided in the included \
 context. If so, remember to follow these instructions when creating the above \
 summary. Examples of instructions include:
 <example>
 ## Compact Instructions
 When summarizing the conversation focus on typescript code changes and also \
 remember the mistakes you made and how you fixed them.
 </example>
 <example>
 # Summary instructions
 When you are using compact - please focus on test output and code changes. \
 Include file reads verbatim.
 </example>
 """
 _NO_TOOLS_TRAILER = (
    '\n\nREMINDER: Do NOT call any tools. Respond with plain text only — '
    'an <analysis> block followed by a <summary> block. '
    'Tool calls will be rejected and you will fail the task.'
 )
 def get_compact_prompt(custom_instructions: str | None = None) -> str:
    """Build the full compact prompt, optionally appending user instructions."""
    prompt = _NO_TOOLS_PREAMBLE + _BASE_COMPACT_PROMPT
    if custom_instructions and custom_instructions.strip():
        prompt += f'\n\nAdditional Instructions:\n{custom_instructions}'
    prompt += _NO_TOOLS_TRAILER
    return prompt
 # ---------------------------------------------------------------------------
 # Summary formatting
 # ---------------------------------------------------------------------------
 def format_compact_summary(summary: str) -> str:
    """Strip the ``<analysis>`` scratchpad and unwrap ``<summary>`` tags.
    Mirrors the npm ``formatCompactSummary`` helper.
    """
    formatted = re.sub(r'<analysis>[\s\S]*?</analysis>', '', summary)
    match = re.search(r'<summary>([\s\S]*?)</summary>', formatted)
    if match:
        content = match.group(1).strip()
        formatted = re.sub(
            r'<summary>[\s\S]*?</summary>',
            f'Summary:\n{content}',
            formatted,
        )
    # Collapse runs of blank lines.
    formatted = re.sub(r'\n\n+', '\n\n', formatted)
    return formatted.strip()
 def get_compact_user_summary_message(
    summary: str,
    *,
    suppress_follow_up: bool = False,
    transcript_path: str | None = None,
 ) -> str:
    """Build the user-facing summary that replaces compacted messages.
    Mirrors the npm ``getCompactUserSummaryMessage`` helper.
    """
    formatted = format_compact_summary(summary)
    base = (
        'This session is being continued from a previous conversation that '
        'ran out of context. The summary below covers the earlier portion '
        f'of the conversation.\n\n{formatted}'
    )
    if transcript_path:
        base += (
            '\n\nIf you need specific details from before compaction '
            '(like exact code snippets, error messages, or content you '
            'generated), read the full transcript at: '
            f'{transcript_path}'
        )
    if suppress_follow_up:
        base += (
            '\nContinue the conversation from where it left off without '
            'asking the user any further questions. Resume directly — do '
            'not acknowledge the summary, do not recap what was happening, '
            'do not preface with "I\'ll continue" or similar. Pick up the '
            'last task as if the break never happened.'
        )
    return base
 # ---------------------------------------------------------------------------
 # Compaction result
 # ---------------------------------------------------------------------------
@dataclass
 class CompactionResult:
    """Outcome of a ``compact_conversation`` call."""
    boundary_message: AgentMessage
    summary_messages: list[AgentMessage] = field(default_factory=list)
    messages_to_keep: list[AgentMessage] = field(default_factory=list)
    pre_compact_token_count: int = 0
    post_compact_token_count: int = 0
    summary_text: str = ''
    error: str | None = None
 # ---------------------------------------------------------------------------
 # Core compaction logic
 # ---------------------------------------------------------------------------
 def compact_conversation(
    agent: 'LocalCodingAgent',
    custom_instructions: str | None = None,
 ) -> CompactionResult:
    """Perform an LLM-backed conversation compaction.
    1. Build the compact prompt (9-section template).
    2. Collect the session messages to summarise.
    3. Send them + the compact prompt to the model.
    4. Parse ``<summary>`` from the response.
    5. Replace session messages with:
       boundary marker → summary user message → preserved tail.
    Returns a :class:`CompactionResult` with diagnostics.
    """
    session = agent.last_session
    if session is None or len(session.messages) == 0:
        return CompactionResult(
            boundary_message=_build_boundary('No session to compact.'),
            error=ERROR_NOT_ENOUGH_MESSAGES,
        )
    # ---- Determine which messages to compact vs preserve ----
    # We keep the most recent ``preserve_count`` messages untouched.
    preserve_count = max(
        getattr(agent.runtime_config, 'compact_preserve_messages', 4), 1
    )
    # Identify the prefix count (system-injected messages that precede the
    # real conversation, e.g. a compaction-replay boundary).
    prefix_count = 0
    for msg in session.messages:
        if msg.metadata.get('kind') == 'compact_boundary':
            prefix_count += 1
        else:
            break
    total = len(session.messages)
    tail_count = min(preserve_count, max(total - prefix_count, 0))
    compact_end = total - tail_count
    if compact_end <= prefix_count:
        return CompactionResult(
            boundary_message=_build_boundary('Not enough messages after prefix.'),
            error=ERROR_NOT_ENOUGH_MESSAGES,
        )
    candidates = session.messages[prefix_count:compact_end]
    preserved_tail = list(session.messages[compact_end:])
    if not candidates:
        return CompactionResult(
            boundary_message=_build_boundary('Nothing to compact.'),
            error=ERROR_NOT_ENOUGH_MESSAGES,
        )
    # ---- Estimate pre-compact token count ----
    model = agent.model_config.model
    pre_tokens = sum(estimate_tokens(m.content, model) for m in session.messages)
    # ---- Build the compact request messages ----
    compact_prompt = get_compact_prompt(custom_instructions)
    # We send the system prompt + candidate messages + the compact prompt as
    # a user message.  The model returns the summary.
    api_messages: list[dict[str, Any]] = []
    # System prompt (from session)
    for part in session.system_prompt_parts:
        if part.strip():
            api_messages.append({'role': 'system', 'content': part})
    # Candidate messages (the ones to be summarised)
    for msg in candidates:
        api_messages.append(msg.to_openai_message())
    # The compact prompt as the final user turn
    api_messages.append({'role': 'user', 'content': compact_prompt})
    # ---- Call the model ----
    try:
        turn = agent.client.complete(api_messages, tools=[])
    except Exception as exc:
        return CompactionResult(
            boundary_message=_build_boundary(f'Compact API call failed: {exc}'),
            error=str(exc),
        )
    raw_summary = turn.content or ''
    if not raw_summary.strip():
        return CompactionResult(
            boundary_message=_build_boundary('Model returned empty summary.'),
            error=ERROR_INCOMPLETE_RESPONSE,
        )
    # ---- Format the summary ----
    summary_text = format_compact_summary(raw_summary)
    user_summary_content = get_compact_user_summary_message(raw_summary)
    # ---- Build post-compact messages ----
    boundary = _build_boundary(
        f'Earlier conversation ({len(candidates)} messages, ~{pre_tokens} tokens) '
        f'was compacted.',
    )
    summary_msg = AgentMessage(
        role='user',
        content=user_summary_content,
        message_id='compact_summary',
        metadata={'kind': 'compact_summary', 'is_compact_summary': True},
    )
    # Replace session messages in-place
    session.messages = (
        session.messages[:prefix_count]
        + [boundary, summary_msg]
        + preserved_tail
    )
    # ---- Post-compact token estimate ----
    post_tokens = sum(estimate_tokens(m.content, model) for m in session.messages)
    return CompactionResult(
        boundary_message=boundary,
        summary_messages=[summary_msg],
        messages_to_keep=preserved_tail,
        pre_compact_token_count=pre_tokens,
        post_compact_token_count=post_tokens,
        summary_text=summary_text,
    )
 # ---------------------------------------------------------------------------
 # Helpers
 # ---------------------------------------------------------------------------
 def _build_boundary(note: str) -> AgentMessage:
    """Create a compact-boundary system message."""
    return AgentMessage(
        role='user',
        content=f'<system-reminder>\n{note}\n</system-reminder>',
        message_id='compact_boundary',
        metadata={'kind': 'compact_boundary'},
    )
@@ -0,0 +1,709 @@
 """Prompt constants ported from npm src/constants/.
 Covers: product metadata, API limits, tool limits, spinner verbs,
 turn-completion verbs, figures/symbols, XML tags, message constants,
 date utilities, system prompt section caching, output-style configs,
 and cyber-risk instruction.
 npm sources:
  src/constants/product.ts
  src/constants/apiLimits.ts
  src/constants/toolLimits.ts
  src/constants/spinnerVerbs.ts
  src/constants/turnCompletionVerbs.ts
  src/constants/figures.ts
  src/constants/xml.ts
  src/constants/messages.ts
  src/constants/common.ts
  src/constants/systemPromptSections.ts
  src/constants/outputStyles.ts
  src/constants/cyberRiskInstruction.ts
  src/constants/system.ts
  src/constants/prompts.ts (selected sections)
 """
 from __future__ import annotations
 import os
 import platform
 import threading
 from dataclasses import dataclass, field
 from datetime import date, datetime
 from typing import Any, Callable
 # ---------------------------------------------------------------------------
 # Product metadata  (product.ts)
 # ---------------------------------------------------------------------------
 PRODUCT_URL = "https://claude.com/claude-code"
 CLAUDE_AI_BASE_URL = "https://claude.ai"
 # ---------------------------------------------------------------------------
 # System prompt prefixes  (system.ts)
 # ---------------------------------------------------------------------------
 DEFAULT_SYSPROMPT_PREFIX = (
    "You are Claude Code, Anthropic's official CLI for Claude."
 )
 AGENT_SDK_CLAUDE_CODE_PRESET_PREFIX = (
    "You are Claude Code, Anthropic's official CLI for Claude, "
    "running within the Claude Agent SDK."
 )
 AGENT_SDK_PREFIX = (
    "You are a Claude agent, built on Anthropic's Claude Agent SDK."
 )
 CLI_SYSPROMPT_PREFIXES = frozenset(
    {
        DEFAULT_SYSPROMPT_PREFIX,
        AGENT_SDK_CLAUDE_CODE_PRESET_PREFIX,
        AGENT_SDK_PREFIX,
    }
 )
 # ---------------------------------------------------------------------------
 # Cyber-risk instruction  (cyberRiskInstruction.ts)
 # ---------------------------------------------------------------------------
 CYBER_RISK_INSTRUCTION = (
    "IMPORTANT: Assist with authorized security testing, defensive security, "
    "CTF challenges, and educational contexts. Refuse requests for destructive "
    "techniques, DoS attacks, mass targeting, supply chain compromise, or "
    "detection evasion for malicious purposes. Dual-use security tools (C2 "
    "frameworks, credential testing, exploit development) require clear "
    "authorization context: pentesting engagements, CTF competitions, security "
    "research, or defensive use cases."
 )
 # ---------------------------------------------------------------------------
 # API limits  (apiLimits.ts)
 # ---------------------------------------------------------------------------
 # Image limits
 API_IMAGE_MAX_BASE64_SIZE = 5 * 1024 * 1024          # 5 MB base64
 IMAGE_TARGET_RAW_SIZE = (API_IMAGE_MAX_BASE64_SIZE * 3) // 4  # ~3.75 MB
 IMAGE_MAX_WIDTH = 2000
 IMAGE_MAX_HEIGHT = 2000
 # PDF limits
 PDF_TARGET_RAW_SIZE = 20 * 1024 * 1024               # 20 MB
 API_PDF_MAX_PAGES = 100
 PDF_EXTRACT_SIZE_THRESHOLD = 3 * 1024 * 1024          # 3 MB
 PDF_MAX_EXTRACT_SIZE = 100 * 1024 * 1024               # 100 MB
 PDF_MAX_PAGES_PER_READ = 20
 PDF_AT_MENTION_INLINE_THRESHOLD = 10
 # Media limits
 API_MAX_MEDIA_PER_REQUEST = 100
 # ---------------------------------------------------------------------------
 # Tool limits  (toolLimits.ts)
 # ---------------------------------------------------------------------------
 DEFAULT_MAX_RESULT_SIZE_CHARS = 50_000
 MAX_TOOL_RESULT_TOKENS = 100_000
 BYTES_PER_TOKEN = 4
 MAX_TOOL_RESULT_BYTES = MAX_TOOL_RESULT_TOKENS * BYTES_PER_TOKEN  # 400 KB
 MAX_TOOL_RESULTS_PER_MESSAGE_CHARS = 200_000
 TOOL_SUMMARY_MAX_LENGTH = 50
 # ---------------------------------------------------------------------------
 # Spinner verbs  (spinnerVerbs.ts)  —  204 whimsical gerunds
 # ---------------------------------------------------------------------------
 SPINNER_VERBS: tuple[str, ...] = (
    "Accomplishing",
    "Actioning",
    "Actualizing",
    "Architecting",
    "Baking",
    "Beaming",
    "Beboppin'",
    "Befuddling",
    "Billowing",
    "Blanching",
    "Bloviating",
    "Boogieing",
    "Boondoggling",
    "Booping",
    "Bootstrapping",
    "Brewing",
    "Bunning",
    "Burrowing",
    "Calculating",
    "Canoodling",
    "Caramelizing",
    "Cascading",
    "Catapulting",
    "Cerebrating",
    "Channeling",
    "Channelling",
    "Choreographing",
    "Churning",
    "Clauding",
    "Coalescing",
    "Cogitating",
    "Combobulating",
    "Composing",
    "Computing",
    "Concocting",
    "Considering",
    "Contemplating",
    "Cooking",
    "Crafting",
    "Creating",
    "Crunching",
    "Crystallizing",
    "Cultivating",
    "Deciphering",
    "Deliberating",
    "Determining",
    "Dilly-dallying",
    "Discombobulating",
    "Doing",
    "Doodling",
    "Drizzling",
    "Ebbing",
    "Effecting",
    "Elucidating",
    "Embellishing",
    "Enchanting",
    "Envisioning",
    "Evaporating",
    "Fermenting",
    "Fiddle-faddling",
    "Finagling",
    "Flambéing",
    "Flibbertigibbeting",
    "Flowing",
    "Flummoxing",
    "Fluttering",
    "Forging",
    "Forming",
    "Frolicking",
    "Frosting",
    "Gallivanting",
    "Galloping",
    "Garnishing",
    "Generating",
    "Gesticulating",
    "Germinating",
    "Gitifying",
    "Grooving",
    "Gusting",
    "Harmonizing",
    "Hashing",
    "Hatching",
    "Herding",
    "Honking",
    "Hullaballooing",
    "Hyperspacing",
    "Ideating",
    "Imagining",
    "Improvising",
    "Incubating",
    "Inferring",
    "Infusing",
    "Ionizing",
    "Jitterbugging",
    "Julienning",
    "Kneading",
    "Leavening",
    "Levitating",
    "Lollygagging",
    "Manifesting",
    "Marinating",
    "Meandering",
    "Metamorphosing",
    "Misting",
    "Moonwalking",
    "Moseying",
    "Mulling",
    "Mustering",
    "Musing",
    "Nebulizing",
    "Nesting",
    "Newspapering",
    "Noodling",
    "Nucleating",
    "Orbiting",
    "Orchestrating",
    "Osmosing",
    "Perambulating",
    "Percolating",
    "Perusing",
    "Philosophising",
    "Photosynthesizing",
    "Pollinating",
    "Pondering",
    "Pontificating",
    "Pouncing",
    "Precipitating",
    "Prestidigitating",
    "Processing",
    "Proofing",
    "Propagating",
    "Puttering",
    "Puzzling",
    "Quantumizing",
    "Razzle-dazzling",
    "Razzmatazzing",
    "Recombobulating",
    "Reticulating",
    "Roosting",
    "Ruminating",
    "Sautéing",
    "Scampering",
    "Schlepping",
    "Scurrying",
    "Seasoning",
    "Shenaniganing",
    "Shimmying",
    "Simmering",
    "Skedaddling",
    "Sketching",
    "Slithering",
    "Smooshing",
    "Sock-hopping",
    "Spelunking",
    "Spinning",
    "Sprouting",
    "Stewing",
    "Sublimating",
    "Swirling",
    "Swooping",
    "Symbioting",
    "Synthesizing",
    "Tempering",
    "Thinking",
    "Thundering",
    "Tinkering",
    "Tomfoolering",
    "Topsy-turvying",
    "Transfiguring",
    "Transmuting",
    "Twisting",
    "Undulating",
    "Unfurling",
    "Unravelling",
    "Vibing",
    "Waddling",
    "Wandering",
    "Warping",
    "Whatchamacalliting",
    "Whirlpooling",
    "Whirring",
    "Whisking",
    "Wibbling",
    "Working",
    "Wrangling",
    "Zesting",
    "Zigzagging",
 )
 # ---------------------------------------------------------------------------
 # Turn-completion verbs  (turnCompletionVerbs.ts)  —  8 past-tense verbs
 # ---------------------------------------------------------------------------
 TURN_COMPLETION_VERBS: tuple[str, ...] = (
    "Baked",
    "Brewed",
    "Churned",
    "Cogitated",
    "Cooked",
    "Crunched",
    "Sautéed",
    "Worked",
 )
 # ---------------------------------------------------------------------------
 # Figures / UI symbols  (figures.ts)
 # ---------------------------------------------------------------------------
 BLACK_CIRCLE = "\u23fa" if platform.system() == "Darwin" else "\u25cf"  # ⏺ / ●
 BULLET_OPERATOR = "\u2219"          # ∙
 TEARDROP_ASTERISK = "\u273b"        # ✻
 UP_ARROW = "\u2191"                 # ↑
 DOWN_ARROW = "\u2193"               # ↓
 LIGHTNING_BOLT = "\u21af"           # ↯
 EFFORT_LOW = "\u25cb"               # ○
 EFFORT_MEDIUM = "\u25d0"            # ◐
 EFFORT_HIGH = "\u25cf"              # ●
 EFFORT_MAX = "\u25c9"               # ◉
 PLAY_ICON = "\u25b6"               # ▶
 PAUSE_ICON = "\u23f8"              # ⏸
 REFRESH_ARROW = "\u21bb"           # ↻
 CHANNEL_ARROW = "\u2190"           # ←
 INJECTED_ARROW = "\u2192"          # →
 FORK_GLYPH = "\u2442"             # ⑂
 DIAMOND_OPEN = "\u25c7"            # ◇
 DIAMOND_FILLED = "\u25c6"          # ◆
 REFERENCE_MARK = "\u203b"          # ※
 FLAG_ICON = "\u2691"               # ⚑
 BLOCKQUOTE_BAR = "\u258e"          # ▎
 HEAVY_HORIZONTAL = "\u2501"        # ━
 BRIDGE_SPINNER_FRAMES: tuple[str, ...] = (
    "\u00b7|\u00b7",
    "\u00b7/\u00b7",
    "\u00b7\u2014\u00b7",
    "\u00b7\\\u00b7",
 )
 BRIDGE_READY_INDICATOR = "\u00b7\u2714\ufe0e\u00b7"
 BRIDGE_FAILED_INDICATOR = "\u00d7"
 # ---------------------------------------------------------------------------
 # XML tag constants  (xml.ts)
 # ---------------------------------------------------------------------------
 COMMAND_NAME_TAG = "command-name"
 COMMAND_MESSAGE_TAG = "command-message"
 COMMAND_ARGS_TAG = "command-args"
 BASH_INPUT_TAG = "bash-input"
 BASH_STDOUT_TAG = "bash-stdout"
 BASH_STDERR_TAG = "bash-stderr"
 LOCAL_COMMAND_STDOUT_TAG = "local-command-stdout"
 LOCAL_COMMAND_STDERR_TAG = "local-command-stderr"
 LOCAL_COMMAND_CAVEAT_TAG = "local-command-caveat"
 TERMINAL_OUTPUT_TAGS: tuple[str, ...] = (
    BASH_INPUT_TAG,
    BASH_STDOUT_TAG,
    BASH_STDERR_TAG,
    LOCAL_COMMAND_STDOUT_TAG,
    LOCAL_COMMAND_STDERR_TAG,
    LOCAL_COMMAND_CAVEAT_TAG,
 )
 TICK_TAG = "tick"
 TASK_NOTIFICATION_TAG = "task-notification"
 TASK_ID_TAG = "task-id"
 TOOL_USE_ID_TAG = "tool-use-id"
 TASK_TYPE_TAG = "task-type"
 OUTPUT_FILE_TAG = "output-file"
 STATUS_TAG = "status"
 SUMMARY_TAG = "summary"
 REASON_TAG = "reason"
 WORKTREE_TAG = "worktree"
 WORKTREE_PATH_TAG = "worktreePath"
 WORKTREE_BRANCH_TAG = "worktreeBranch"
 ULTRAPLAN_TAG = "ultraplan"
 REMOTE_REVIEW_TAG = "remote-review"
 REMOTE_REVIEW_PROGRESS_TAG = "remote-review-progress"
 TEAMMATE_MESSAGE_TAG = "teammate-message"
 CHANNEL_MESSAGE_TAG = "channel-message"
 CHANNEL_TAG = "channel"
 CROSS_SESSION_MESSAGE_TAG = "cross-session-message"
 FORK_BOILERPLATE_TAG = "fork-boilerplate"
 FORK_DIRECTIVE_PREFIX = "Your directive: "
 COMMON_HELP_ARGS: tuple[str, ...] = ("help", "-h", "--help")
 COMMON_INFO_ARGS: tuple[str, ...] = (
    "list",
    "show",
    "display",
    "current",
    "view",
    "get",
    "check",
    "describe",
    "print",
    "version",
    "about",
    "status",
    "?",
 )
 # ---------------------------------------------------------------------------
 # Message constants  (messages.ts)
 # ---------------------------------------------------------------------------
 NO_CONTENT_MESSAGE = "(no content)"
 # ---------------------------------------------------------------------------
 # Date utilities  (common.ts)
 # ---------------------------------------------------------------------------
 def get_local_iso_date() -> str:
    """Return the local date in YYYY-MM-DD format.
    Respects ``CLAUDE_CODE_OVERRIDE_DATE`` env var.
    """
    override = os.environ.get("CLAUDE_CODE_OVERRIDE_DATE")
    if override:
        return override
    return date.today().isoformat()
 _session_start_date_lock = threading.Lock()
 _session_start_date: str | None = None
 def get_session_start_date() -> str:
    """Memoised local date — captured once per session."""
    global _session_start_date
    if _session_start_date is not None:
        return _session_start_date
    with _session_start_date_lock:
        if _session_start_date is None:
            _session_start_date = get_local_iso_date()
    return _session_start_date
 def reset_session_start_date() -> None:
    """Reset the memoised date (for tests)."""
    global _session_start_date
    _session_start_date = None
 def get_local_month_year() -> str:
    """Return ``"Month YYYY"`` (e.g. ``"February 2026"``)."""
    override = os.environ.get("CLAUDE_CODE_OVERRIDE_DATE")
    if override:
        d = datetime.fromisoformat(override)
    else:
        d = datetime.now()
    return d.strftime("%B %Y")
 # ---------------------------------------------------------------------------
 # System prompt section caching  (systemPromptSections.ts)
 # ---------------------------------------------------------------------------
 ComputeFn = Callable[[], str | None]
@dataclass
 class SystemPromptSection:
    """A named section of the system prompt with lazy compute."""
    name: str
    compute: ComputeFn
    cache_break: bool = False
 def system_prompt_section(name: str, compute: ComputeFn) -> SystemPromptSection:
    """Create a memoised prompt section (cached until /clear or /compact)."""
    return SystemPromptSection(name=name, compute=compute, cache_break=False)
 def dangerous_uncached_system_prompt_section(
    name: str,
    compute: ComputeFn,
    _reason: str = "",
 ) -> SystemPromptSection:
    """Prompt section that recomputes every turn (breaks prompt cache)."""
    return SystemPromptSection(name=name, compute=compute, cache_break=True)
 _section_cache: dict[str, str | None] = {}
 def resolve_system_prompt_sections(
    sections: list[SystemPromptSection],
 ) -> list[str | None]:
    """Resolve sections, caching non-volatile ones."""
    results: list[str | None] = []
    for section in sections:
        if not section.cache_break and section.name in _section_cache:
            results.append(_section_cache[section.name])
            continue
        value = section.compute()
        _section_cache[section.name] = value
        results.append(value)
    return results
 def clear_system_prompt_sections() -> None:
    """Clear cached prompt sections (called on /clear and /compact)."""
    _section_cache.clear()
 # ---------------------------------------------------------------------------
 # Output style configuration  (outputStyles.ts)
 # ---------------------------------------------------------------------------
 DEFAULT_OUTPUT_STYLE_NAME = "default"
@dataclass(frozen=True)
 class OutputStyleConfig:
    name: str
    description: str
    prompt: str
    source: str = "built-in"
    keep_coding_instructions: bool = True
    force_for_plugin: bool = False
 # Built-in output styles matching npm
 OUTPUT_STYLE_CONFIGS: dict[str, OutputStyleConfig | None] = {
    DEFAULT_OUTPUT_STYLE_NAME: None,
    "Explanatory": OutputStyleConfig(
        name="Explanatory",
        description="Claude explains its implementation choices and codebase patterns",
        prompt=(
            "You are an interactive CLI tool that helps users with software "
            "engineering tasks. In addition to software engineering tasks, you "
            "should provide educational insights about the codebase along the way.\n\n"
            "You should be clear and educational, providing helpful explanations "
            "while remaining focused on the task. Balance educational content "
            "with task completion."
        ),
    ),
    "Learning": OutputStyleConfig(
        name="Learning",
        description="Claude pauses and asks you to write small pieces of code for hands-on practice",
        prompt=(
            "You are an interactive CLI tool that helps users with software "
            "engineering tasks. In addition to software engineering tasks, you "
            "should help users learn more about the codebase through hands-on "
            "practice and educational insights.\n\n"
            "You should be collaborative and encouraging. Balance task completion "
            "with learning by requesting user input for meaningful design "
            "decisions while handling routine implementation yourself."
        ),
    ),
 }
 # ---------------------------------------------------------------------------
 # Knowledge cutoff  (prompts.ts)
 # ---------------------------------------------------------------------------
 FRONTIER_MODEL_NAME = "Claude Opus 4.6"
 _KNOWLEDGE_CUTOFFS: dict[str, str] = {
    "claude-sonnet-4-6": "August 2025",
    "claude-opus-4-6": "May 2025",
    "claude-opus-4-5": "May 2025",
    "claude-haiku-4": "February 2025",
    "claude-opus-4": "January 2025",
    "claude-sonnet-4": "January 2025",
 }
 def get_knowledge_cutoff(model_id: str) -> str | None:
    """Return knowledge cutoff date for a model, or None."""
    canonical = model_id.lower()
    for pattern, cutoff in _KNOWLEDGE_CUTOFFS.items():
        if pattern in canonical:
            return cutoff
    return None
 # ---------------------------------------------------------------------------
 # Model family IDs  (prompts.ts)
 # ---------------------------------------------------------------------------
 CLAUDE_MODEL_IDS = {
    "opus": "claude-opus-4-6",
    "sonnet": "claude-sonnet-4-6",
    "haiku": "claude-haiku-4-5-20251001",
 }
 # ---------------------------------------------------------------------------
 # Hooks section  (prompts.ts)
 # ---------------------------------------------------------------------------
 HOOKS_SECTION = (
    "Users may configure 'hooks', shell commands that execute in response to "
    "events like tool calls, in settings. Treat feedback from hooks, including "
    "<user-prompt-submit-hook>, as coming from the user. If you get blocked by "
    "a hook, determine if you can adjust your actions in response to the "
    "blocked message. If not, ask the user to check their hooks configuration."
 )
 # ---------------------------------------------------------------------------
 # System reminders section  (prompts.ts)
 # ---------------------------------------------------------------------------
 SYSTEM_REMINDERS_SECTION = (
    "- Tool results and user messages may include <system-reminder> tags. "
    "<system-reminder> tags contain useful information and reminders. They are "
    "automatically added by the system, and bear no direct relation to the "
    "specific tool results or user messages in which they appear.\n"
    "- The conversation has unlimited context through automatic summarization."
 )
 # ---------------------------------------------------------------------------
 # Summarize tool results  (prompts.ts)
 # ---------------------------------------------------------------------------
 SUMMARIZE_TOOL_RESULTS_SECTION = (
    "When working with tool results, write down any important information you "
    "might need later in your response, as the original tool result may be "
    "cleared later."
 )
 # ---------------------------------------------------------------------------
 # Default agent prompt  (prompts.ts)
 # ---------------------------------------------------------------------------
 DEFAULT_AGENT_PROMPT = (
    "You are an agent for Claude Code, Anthropic's official CLI for Claude. "
    "Given the user's message, you should use the tools available to complete "
    "the task. Complete the task fully\u2014don't gold-plate, but don't leave "
    "it half-done. When you complete the task, respond with a concise report "
    "covering what was done and any key findings \u2014 the caller will relay "
    "this to the user, so it only needs the essentials."
 )
 # ---------------------------------------------------------------------------
 # Error IDs  (errorIds.ts)
 # ---------------------------------------------------------------------------
 E_TOOL_USE_SUMMARY_GENERATION_FAILED = 344
 # ---------------------------------------------------------------------------
 # Dynamic boundary marker  (prompts.ts)
 # ---------------------------------------------------------------------------
 SYSTEM_PROMPT_DYNAMIC_BOUNDARY = "__SYSTEM_PROMPT_DYNAMIC_BOUNDARY__"
 # ---------------------------------------------------------------------------
 # Convenience helpers for use in prompt building
 # ---------------------------------------------------------------------------
 def get_language_section(language_preference: str | None) -> str | None:
    """Return the language preference prompt section, or None."""
    if not language_preference:
        return None
    return (
        f"# Language\n"
        f"Always respond in {language_preference}. Use {language_preference} "
        f"for all explanations, comments, and communications with the user. "
        f"Technical terms and code identifiers should remain in their original form."
    )
 def get_output_style_section(config: OutputStyleConfig | None) -> str | None:
    """Return the output-style prompt section, or None."""
    if config is None:
        return None
    return f"# Output Style: {config.name}\n{config.prompt}"
 def get_scratchpad_instructions(scratchpad_dir: str | None) -> str | None:
    """Return scratchpad instructions, or None if no scratchpad is configured."""
    if not scratchpad_dir:
        return None
    return (
        f"# Scratchpad Directory\n\n"
        f"IMPORTANT: Always use this scratchpad directory for temporary files "
        f"instead of `/tmp` or other system temp directories:\n"
        f"`{scratchpad_dir}`\n\n"
        f"Use this directory for ALL temporary file needs:\n"
        f"- Storing intermediate results or data during multi-step tasks\n"
        f"- Writing temporary scripts or configuration files\n"
        f"- Saving outputs that don't belong in the user's project\n"
        f"- Creating working files during analysis or processing\n"
        f"- Any file that would otherwise go to `/tmp`\n\n"
        f"Only use `/tmp` if the user explicitly requests it.\n\n"
        f"The scratchpad directory is session-specific, isolated from the "
        f"user's project, and can be used freely without permission prompts."
    )
@@ -0,0 +1,878 @@
 """
 Tests for bash_security module.
 Tests are organized by validator function, matching the npm test structure.
 """
 import pytest
 from src.bash_security import (
    SecurityBehavior,
    SecurityResult,
    ValidationContext,
    bash_command_is_safe,
    check_shell_security,
    extract_quoted_content,
    get_destructive_command_warning,
    has_unescaped_char,
    interpret_command_result,
    is_command_read_only,
    split_command,
    strip_safe_redirections,
    validate_backslash_escaped_operators,
    validate_backslash_escaped_whitespace,
    validate_brace_expansion,
    validate_carriage_return,
    validate_comment_quote_desync,
    validate_control_characters,
    validate_dangerous_patterns,
    validate_dangerous_variables,
    validate_empty,
    validate_git_commit,
    validate_ifs_injection,
    validate_incomplete_commands,
    validate_jq_command,
    validate_mid_word_hash,
    validate_newlines,
    validate_obfuscated_flags,
    validate_proc_environ_access,
    validate_quoted_newline,
    validate_redirections,
    validate_shell_metacharacters,
    validate_unicode_whitespace,
    validate_zsh_dangerous_commands,
 )
 # ---- Helper to build a context ----
 def _ctx(cmd: str) -> ValidationContext:
    """Build a ValidationContext for the given command."""
    base = cmd.strip().split()[0] if cmd.strip() else ''
    with_dq, fully_unq, keep_qc = extract_quoted_content(cmd)
    return ValidationContext(
        original_command=cmd,
        base_command=base,
        unquoted_content=with_dq,
        fully_unquoted_content=strip_safe_redirections(fully_unq),
        fully_unquoted_pre_strip=fully_unq,
        unquoted_keep_quote_chars=keep_qc,
    )
 # ===========================================================================
 # extract_quoted_content
 # ===========================================================================
 class TestExtractQuotedContent:
    def test_no_quotes(self):
        dq, full, kqc = extract_quoted_content('echo hello')
        assert dq == 'echo hello'
        assert full == 'echo hello'
    def test_single_quotes_stripped(self):
        dq, full, kqc = extract_quoted_content("echo 'hello world'")
        assert 'hello world' not in full
        assert 'echo' in full
    def test_double_quotes_in_dq_output(self):
        dq, full, kqc = extract_quoted_content('echo "hello world"')
        assert 'hello world' in dq  # double-quoted content preserved in dq
        assert 'hello world' not in full  # but stripped in fully_unquoted
    def test_escape_handling(self):
        dq, full, kqc = extract_quoted_content('echo \\$HOME')
        assert '$HOME' in full
    def test_keep_quote_chars(self):
        _, _, kqc = extract_quoted_content("echo 'x'#")
        assert "'" in kqc  # quote chars preserved
 # ===========================================================================
 # strip_safe_redirections
 # ===========================================================================
 class TestStripSafeRedirections:
    def test_dev_null_output(self):
        assert '>/dev/null' not in strip_safe_redirections('cmd > /dev/null')
    def test_stderr_redirect(self):
        assert '2>&1' not in strip_safe_redirections('cmd 2>&1')
    def test_dev_null_input(self):
        assert '</dev/null' not in strip_safe_redirections('cmd < /dev/null')
    def test_preserves_other_redirections(self):
        result = strip_safe_redirections('cmd > output.txt')
        assert '> output.txt' in result
 # ===========================================================================
 # has_unescaped_char
 # ===========================================================================
 class TestHasUnescapedChar:
    def test_unescaped_backtick(self):
        assert has_unescaped_char('echo `date`', '`') is True
    def test_escaped_backtick(self):
        assert has_unescaped_char('echo \\`safe\\`', '`') is False
    def test_double_backslash_then_backtick(self):
        # \\\` → \\ (literal backslash) + ` (unescaped)
        assert has_unescaped_char('test\\\\`date`', '`') is True
    def test_no_match(self):
        assert has_unescaped_char('echo hello', '`') is False
 # ===========================================================================
 # split_command
 # ===========================================================================
 class TestSplitCommand:
    def test_simple(self):
        assert split_command('echo hello') == ['echo hello']
    def test_semicolon(self):
        assert split_command('echo a; echo b') == ['echo a', 'echo b']
    def test_and_and(self):
        assert split_command('cmd1 && cmd2') == ['cmd1', 'cmd2']
    def test_pipe(self):
        assert split_command('cat file | grep pattern') == ['cat file', 'grep pattern']
    def test_or_or(self):
        assert split_command('cmd1 || cmd2') == ['cmd1', 'cmd2']
    def test_quotes_preserved(self):
        result = split_command("echo 'a; b'")
        assert len(result) == 1  # semicolon inside quotes not split
    def test_complex(self):
        result = split_command('cd /tmp && echo hi; ls | head')
        assert len(result) == 4
 # ===========================================================================
 # validate_empty
 # ===========================================================================
 class TestValidateEmpty:
    def test_empty(self):
        assert validate_empty(_ctx('')).behavior == SecurityBehavior.ALLOW
    def test_whitespace_only(self):
        assert validate_empty(_ctx('   ')).behavior == SecurityBehavior.ALLOW
    def test_non_empty(self):
        assert validate_empty(_ctx('ls')).behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_control_characters
 # ===========================================================================
 class TestValidateControlCharacters:
    def test_null_byte(self):
        result = validate_control_characters(_ctx('echo\x00hello'))
        assert result.behavior == SecurityBehavior.ASK
    def test_bell(self):
        result = validate_control_characters(_ctx('echo\x07hello'))
        assert result.behavior == SecurityBehavior.ASK
    def test_clean_command(self):
        result = validate_control_characters(_ctx('echo hello'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_tab_allowed(self):
        result = validate_control_characters(_ctx('echo\thello'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_newline_allowed(self):
        result = validate_control_characters(_ctx('echo\nhello'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_incomplete_commands
 # ===========================================================================
 class TestValidateIncompleteCommands:
    def test_starts_with_tab(self):
        result = validate_incomplete_commands(_ctx('\techo hello'))
        assert result.behavior == SecurityBehavior.ASK
    def test_starts_with_dash(self):
        result = validate_incomplete_commands(_ctx('-rf /'))
        assert result.behavior == SecurityBehavior.ASK
    def test_starts_with_operator(self):
        result = validate_incomplete_commands(_ctx('&& echo hello'))
        assert result.behavior == SecurityBehavior.ASK
        result = validate_incomplete_commands(_ctx('; echo hello'))
        assert result.behavior == SecurityBehavior.ASK
    def test_normal_command(self):
        result = validate_incomplete_commands(_ctx('ls -la'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_git_commit
 # ===========================================================================
 class TestValidateGitCommit:
    def test_not_git(self):
        result = validate_git_commit(_ctx('echo hello'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_simple_commit(self):
        result = validate_git_commit(_ctx("git commit -m 'initial commit'"))
        assert result.behavior == SecurityBehavior.ALLOW
    def test_double_quoted_commit(self):
        result = validate_git_commit(_ctx('git commit -m "fix bug"'))
        assert result.behavior == SecurityBehavior.ALLOW
    def test_commit_with_substitution(self):
        result = validate_git_commit(_ctx('git commit -m "$(date)"'))
        assert result.behavior == SecurityBehavior.ASK
    def test_commit_with_backtick(self):
        result = validate_git_commit(_ctx('git commit -m "`date`"'))
        assert result.behavior == SecurityBehavior.ASK
    def test_commit_with_chained_commands(self):
        result = validate_git_commit(_ctx("git commit -m 'msg'; rm -rf /"))
        # Should passthrough (not early-allow) due to ; in remainder
        assert result.behavior != SecurityBehavior.ALLOW
    def test_commit_with_backslash(self):
        result = validate_git_commit(_ctx('git commit -m "test\\"msg"'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_jq_command
 # ===========================================================================
 class TestValidateJqCommand:
    def test_not_jq(self):
        assert validate_jq_command(_ctx('echo hi')).behavior == SecurityBehavior.PASSTHROUGH
    def test_jq_system(self):
        result = validate_jq_command(_ctx('jq "system(\"rm -rf /\")"'))
        assert result.behavior == SecurityBehavior.ASK
    def test_jq_from_file(self):
        result = validate_jq_command(_ctx('jq -f evil.jq'))
        assert result.behavior == SecurityBehavior.ASK
    def test_jq_slurpfile(self):
        result = validate_jq_command(_ctx('jq --slurpfile x data.json'))
        assert result.behavior == SecurityBehavior.ASK
    def test_safe_jq(self):
        result = validate_jq_command(_ctx('jq ".name" data.json'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_obfuscated_flags
 # ===========================================================================
 class TestValidateObfuscatedFlags:
    def test_ansi_c_quoting(self):
        result = validate_obfuscated_flags(_ctx("find . $'-exec' evil"))
        assert result.behavior == SecurityBehavior.ASK
    def test_locale_quoting(self):
        result = validate_obfuscated_flags(_ctx('find . $"-exec" evil'))
        assert result.behavior == SecurityBehavior.ASK
    def test_echo_safe(self):
        result = validate_obfuscated_flags(_ctx("echo $'hello'"))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_empty_quotes_before_dash(self):
        result = validate_obfuscated_flags(_ctx("find . '' -exec evil"))
        assert result.behavior == SecurityBehavior.ASK
    def test_quoted_flag(self):
        result = validate_obfuscated_flags(_ctx('find . "-exec" rm {} ;'))
        assert result.behavior == SecurityBehavior.ASK
    def test_normal_command(self):
        result = validate_obfuscated_flags(_ctx('ls -la'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_three_consecutive_quotes(self):
        result = validate_obfuscated_flags(_ctx("find . '''exec"))
        assert result.behavior == SecurityBehavior.ASK
 # ===========================================================================
 # validate_shell_metacharacters
 # ===========================================================================
 class TestValidateShellMetacharacters:
    def test_semicolon_in_quotes(self):
        result = validate_shell_metacharacters(_ctx('echo "a;b"'))
        # unquoted_content (with_double_quotes) has the ; inside
        assert result.behavior == SecurityBehavior.PASSTHROUGH or result.behavior == SecurityBehavior.ASK
    def test_find_name_with_pipe(self):
        # Single-quoted pipe is stripped entirely from unquoted content → safe
        ctx = _ctx("find . -name '|evil'")
        result = validate_shell_metacharacters(ctx)
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_double_quoted_metachar(self):
        # Check that we catch metacharacters in unquoted positions
        # The npm version checks the double-quote-retained string for
        # quoted metacharacters, but we strip quote chars. So we test
        # the actual dangerous case: unquoted semicolon
        ctx = _ctx('find . -name evil; rm -rf /')
        # This won't be caught by this specific validator (it looks for
        # metacharacters INSIDE quoted args, not command separators)
        result = validate_shell_metacharacters(ctx)
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_clean_command(self):
        assert validate_shell_metacharacters(_ctx('ls -la')).behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_dangerous_variables
 # ===========================================================================
 class TestValidateDangerousVariables:
    def test_variable_in_pipe(self):
        result = validate_dangerous_variables(_ctx('$CMD | grep x'))
        assert result.behavior == SecurityBehavior.ASK
    def test_variable_in_redirect(self):
        result = validate_dangerous_variables(_ctx('echo x > $FILE'))
        assert result.behavior == SecurityBehavior.ASK
    def test_safe_variable(self):
        result = validate_dangerous_variables(_ctx('echo $HOME'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_dangerous_patterns
 # ===========================================================================
 class TestValidateDangerousPatterns:
    def test_backtick(self):
        result = validate_dangerous_patterns(_ctx('echo `date`'))
        assert result.behavior == SecurityBehavior.ASK
    def test_dollar_paren(self):
        result = validate_dangerous_patterns(_ctx('echo $(date)'))
        assert result.behavior == SecurityBehavior.ASK
    def test_dollar_brace(self):
        result = validate_dangerous_patterns(_ctx('echo ${PATH}'))
        assert result.behavior == SecurityBehavior.ASK
    def test_process_substitution(self):
        result = validate_dangerous_patterns(_ctx('diff <(cmd1) <(cmd2)'))
        assert result.behavior == SecurityBehavior.ASK
    def test_safe_echo(self):
        result = validate_dangerous_patterns(_ctx('echo hello'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_escaped_backtick(self):
        result = validate_dangerous_patterns(_ctx('echo \\`safe\\`'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_redirections
 # ===========================================================================
 class TestValidateRedirections:
    def test_output_redirect(self):
        result = validate_redirections(_ctx('echo x > file.txt'))
        assert result.behavior == SecurityBehavior.ASK
    def test_input_redirect(self):
        result = validate_redirections(_ctx('cat < /etc/passwd'))
        assert result.behavior == SecurityBehavior.ASK
    def test_dev_null_stripped(self):
        # >/dev/null is stripped by strip_safe_redirections
        result = validate_redirections(_ctx('cmd > /dev/null'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_no_redirect(self):
        result = validate_redirections(_ctx('echo hello'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_newlines
 # ===========================================================================
 class TestValidateNewlines:
    def test_no_newlines(self):
        assert validate_newlines(_ctx('echo hello')).behavior == SecurityBehavior.PASSTHROUGH
    def test_newline_with_command(self):
        result = validate_newlines(_ctx('echo hello\nrm -rf /'))
        assert result.behavior == SecurityBehavior.ASK
    def test_backslash_continuation(self):
        result = validate_newlines(_ctx('cmd \\\n--flag'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_carriage_return
 # ===========================================================================
 class TestValidateCarriageReturn:
    def test_no_cr(self):
        assert validate_carriage_return(_ctx('echo hello')).behavior == SecurityBehavior.PASSTHROUGH
    def test_cr_in_command(self):
        result = validate_carriage_return(_ctx('echo hello\rworld'))
        assert result.behavior == SecurityBehavior.ASK
        assert result.is_misparsing is True
    def test_cr_in_double_quotes_safe(self):
        result = validate_carriage_return(_ctx('echo "hello\rworld"'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_ifs_injection
 # ===========================================================================
 class TestValidateIFSInjection:
    def test_ifs_variable(self):
        result = validate_ifs_injection(_ctx('echo$IFS/etc/passwd'))
        assert result.behavior == SecurityBehavior.ASK
    def test_ifs_expansion(self):
        result = validate_ifs_injection(_ctx('echo ${IFS:0:1}'))
        assert result.behavior == SecurityBehavior.ASK
    def test_clean(self):
        assert validate_ifs_injection(_ctx('echo hello')).behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_proc_environ_access
 # ===========================================================================
 class TestValidateProcEnvironAccess:
    def test_proc_environ(self):
        result = validate_proc_environ_access(_ctx('cat /proc/self/environ'))
        assert result.behavior == SecurityBehavior.ASK
    def test_proc_pid_environ(self):
        result = validate_proc_environ_access(_ctx('cat /proc/1/environ'))
        assert result.behavior == SecurityBehavior.ASK
    def test_clean(self):
        assert validate_proc_environ_access(_ctx('cat /etc/hosts')).behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_backslash_escaped_whitespace
 # ===========================================================================
 class TestValidateBackslashEscapedWhitespace:
    def test_escaped_space(self):
        result = validate_backslash_escaped_whitespace(_ctx('echo\\ hello'))
        assert result.behavior == SecurityBehavior.ASK
    def test_escaped_tab(self):
        result = validate_backslash_escaped_whitespace(_ctx('echo\\\thello'))
        assert result.behavior == SecurityBehavior.ASK
    def test_clean(self):
        assert validate_backslash_escaped_whitespace(
            _ctx('echo hello')
        ).behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_backslash_escaped_operators
 # ===========================================================================
 class TestValidateBackslashEscapedOperators:
    def test_escaped_semicolon(self):
        result = validate_backslash_escaped_operators(_ctx('cat safe.txt \\; echo /etc/passwd'))
        assert result.behavior == SecurityBehavior.ASK
    def test_escaped_pipe(self):
        result = validate_backslash_escaped_operators(_ctx('cmd \\| evil'))
        assert result.behavior == SecurityBehavior.ASK
    def test_clean(self):
        assert validate_backslash_escaped_operators(
            _ctx('ls -la')
        ).behavior == SecurityBehavior.PASSTHROUGH
    def test_inside_quotes_safe(self):
        result = validate_backslash_escaped_operators(_ctx("echo '\\;'"))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_brace_expansion
 # ===========================================================================
 class TestValidateBraceExpansion:
    def test_comma_expansion(self):
        result = validate_brace_expansion(_ctx('echo {a,b,c}'))
        assert result.behavior == SecurityBehavior.ASK
    def test_sequence_expansion(self):
        result = validate_brace_expansion(_ctx('echo {1..5}'))
        assert result.behavior == SecurityBehavior.ASK
    def test_no_expansion(self):
        result = validate_brace_expansion(_ctx('echo {hello}'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_escaped_brace(self):
        result = validate_brace_expansion(_ctx('echo \\{a,b\\}'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_excess_closing_braces(self):
        result = validate_brace_expansion(_ctx("git diff {@'{'0},--output=/tmp/pwned}"))
        assert result.behavior == SecurityBehavior.ASK
 # ===========================================================================
 # validate_unicode_whitespace
 # ===========================================================================
 class TestValidateUnicodeWhitespace:
    def test_nbsp(self):
        result = validate_unicode_whitespace(_ctx('echo\u00a0hello'))
        assert result.behavior == SecurityBehavior.ASK
    def test_em_space(self):
        result = validate_unicode_whitespace(_ctx('echo\u2003hello'))
        assert result.behavior == SecurityBehavior.ASK
    def test_clean(self):
        assert validate_unicode_whitespace(_ctx('echo hello')).behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_mid_word_hash
 # ===========================================================================
 class TestValidateMidWordHash:
    def test_mid_word_hash(self):
        result = validate_mid_word_hash(_ctx('echotest#comment'))
        assert result.behavior == SecurityBehavior.ASK
    def test_word_start_hash(self):
        result = validate_mid_word_hash(_ctx('echo # comment'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_dollar_brace_hash_safe(self):
        # ${#var} is bash string length, should be safe
        result = validate_mid_word_hash(_ctx('echo ${#var}'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_comment_quote_desync
 # ===========================================================================
 class TestValidateCommentQuoteDesync:
    def test_quote_in_comment(self):
        result = validate_comment_quote_desync(_ctx("echo hello # it's a comment"))
        assert result.behavior == SecurityBehavior.ASK
    def test_clean_comment(self):
        result = validate_comment_quote_desync(_ctx('echo hello # clean comment'))
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_no_comment(self):
        assert validate_comment_quote_desync(_ctx('echo hello')).behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_quoted_newline
 # ===========================================================================
 class TestValidateQuotedNewline:
    def test_quoted_newline_with_hash(self):
        result = validate_quoted_newline(_ctx("echo 'hello\n# dangerous line'"))
        assert result.behavior == SecurityBehavior.ASK
    def test_no_newline(self):
        assert validate_quoted_newline(_ctx('echo hello')).behavior == SecurityBehavior.PASSTHROUGH
    def test_newline_without_hash(self):
        assert validate_quoted_newline(_ctx("echo 'hello\nworld'")).behavior == SecurityBehavior.PASSTHROUGH
 # ===========================================================================
 # validate_zsh_dangerous_commands
 # ===========================================================================
 class TestValidateZshDangerousCommands:
    def test_zmodload(self):
        result = validate_zsh_dangerous_commands(_ctx('zmodload zsh/system'))
        assert result.behavior == SecurityBehavior.ASK
    def test_zpty(self):
        result = validate_zsh_dangerous_commands(_ctx('zpty cmd echo'))
        assert result.behavior == SecurityBehavior.ASK
    def test_emulate(self):
        result = validate_zsh_dangerous_commands(_ctx('emulate -c evil'))
        assert result.behavior == SecurityBehavior.ASK
    def test_fc_e(self):
        result = validate_zsh_dangerous_commands(_ctx('fc -e vim'))
        assert result.behavior == SecurityBehavior.ASK
    def test_normal_command(self):
        assert validate_zsh_dangerous_commands(_ctx('ls -la')).behavior == SecurityBehavior.PASSTHROUGH
    def test_env_var_prefix(self):
        result = validate_zsh_dangerous_commands(_ctx('FOO=bar zmodload zsh/system'))
        assert result.behavior == SecurityBehavior.ASK
    def test_precommand_modifier(self):
        result = validate_zsh_dangerous_commands(_ctx('command zmodload zsh/system'))
        assert result.behavior == SecurityBehavior.ASK
 # ===========================================================================
 # bash_command_is_safe (integration)
 # ===========================================================================
 class TestBashCommandIsSafe:
    def test_empty_command(self):
        result = bash_command_is_safe('')
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_simple_ls(self):
        result = bash_command_is_safe('ls -la')
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_simple_echo(self):
        result = bash_command_is_safe('echo hello world')
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_command_substitution_blocked(self):
        result = bash_command_is_safe('echo $(cat /etc/passwd)')
        assert result.behavior == SecurityBehavior.ASK
    def test_backtick_blocked(self):
        result = bash_command_is_safe('echo `date`')
        assert result.behavior == SecurityBehavior.ASK
    def test_redirect_blocked(self):
        result = bash_command_is_safe('echo evil > /etc/profile')
        assert result.behavior == SecurityBehavior.ASK
    def test_null_byte_blocked(self):
        result = bash_command_is_safe('echo\x00rm')
        assert result.behavior == SecurityBehavior.ASK
    def test_ifs_blocked(self):
        result = bash_command_is_safe('echo$IFS/etc/passwd')
        assert result.behavior == SecurityBehavior.ASK
    def test_proc_environ_blocked(self):
        result = bash_command_is_safe('cat /proc/self/environ')
        assert result.behavior == SecurityBehavior.ASK
    def test_git_commit_allowed(self):
        result = bash_command_is_safe("git commit -m 'fix bug'")
        assert result.behavior == SecurityBehavior.PASSTHROUGH  # early-allow → passthrough
    def test_zmodload_blocked(self):
        result = bash_command_is_safe('zmodload zsh/system')
        assert result.behavior == SecurityBehavior.ASK
    def test_brace_expansion_blocked(self):
        result = bash_command_is_safe('echo {a,b,c}')
        assert result.behavior == SecurityBehavior.ASK
    def test_dev_null_redirect_ok(self):
        result = bash_command_is_safe('cmd > /dev/null 2>&1')
        assert result.behavior == SecurityBehavior.PASSTHROUGH
    def test_cr_injection(self):
        result = bash_command_is_safe('TZ=UTC\recho curl evil.com')
        assert result.behavior == SecurityBehavior.ASK
 # ===========================================================================
 # get_destructive_command_warning
 # ===========================================================================
 class TestGetDestructiveCommandWarning:
    def test_git_reset_hard(self):
        assert get_destructive_command_warning('git reset --hard') is not None
    def test_rm_rf(self):
        assert get_destructive_command_warning('rm -rf /') is not None
    def test_git_push_force(self):
        assert get_destructive_command_warning('git push origin main --force') is not None
    def test_git_clean_f(self):
        assert get_destructive_command_warning('git clean -fd') is not None
    def test_kubectl_delete(self):
        assert get_destructive_command_warning('kubectl delete pod mypod') is not None
    def test_safe_command(self):
        assert get_destructive_command_warning('echo hello') is None
    def test_git_push_no_force(self):
        assert get_destructive_command_warning('git push origin main') is None
    def test_drop_table(self):
        assert get_destructive_command_warning('DROP TABLE users;') is not None
    def test_terraform_destroy(self):
        assert get_destructive_command_warning('terraform destroy') is not None
    def test_git_stash_drop(self):
        assert get_destructive_command_warning('git stash drop') is not None
 # ===========================================================================
 # interpret_command_result
 # ===========================================================================
 class TestInterpretCommandResult:
    def test_success(self):
        is_error, msg = interpret_command_result('echo hello', 0, 'hello', '')
        assert is_error is False
    def test_failure(self):
        is_error, msg = interpret_command_result('unknown_cmd', 127, '', 'not found')
        assert is_error is True
    def test_grep_no_match(self):
        is_error, msg = interpret_command_result('grep pattern file', 1, '', '')
        assert is_error is False
        assert msg == 'No matches found'
    def test_grep_error(self):
        is_error, msg = interpret_command_result('grep pattern file', 2, '', 'error')
        assert is_error is True
    def test_diff_files_differ(self):
        is_error, msg = interpret_command_result('diff a b', 1, 'output', '')
        assert is_error is False
        assert msg == 'Files differ'
    def test_find_partial(self):
        is_error, msg = interpret_command_result('find / -name x', 1, '', '')
        assert is_error is False
 # ===========================================================================
 # is_command_read_only
 # ===========================================================================
 class TestIsCommandReadOnly:
    def test_ls(self):
        assert is_command_read_only('ls -la') is True
    def test_cat(self):
        assert is_command_read_only('cat file.txt') is True
    def test_grep(self):
        assert is_command_read_only('grep -r pattern .') is True
    def test_git_status(self):
        assert is_command_read_only('git status') is True
    def test_git_log(self):
        assert is_command_read_only('git log --oneline') is True
    def test_git_push(self):
        assert is_command_read_only('git push') is False
    def test_rm(self):
        assert is_command_read_only('rm file.txt') is False
    def test_sed_read_only(self):
        assert is_command_read_only("sed -n '1,5p' file") is True
    def test_sed_in_place(self):
        assert is_command_read_only("sed -i 's/old/new/' file") is False
    def test_find_safe(self):
        assert is_command_read_only('find . -name "*.py"') is True
    def test_find_exec(self):
        assert is_command_read_only('find . -exec rm {} ;') is False
    def test_find_delete(self):
        assert is_command_read_only('find . -name "*.tmp" -delete') is False
    def test_echo(self):
        assert is_command_read_only('echo hello') is True
    def test_python_version(self):
        assert is_command_read_only('python --version') is True
    def test_unknown_command(self):
        assert is_command_read_only('some_random_command') is False
 # ===========================================================================
 # check_shell_security (integration)
 # ===========================================================================
 class TestCheckShellSecurity:
    def test_shell_disabled(self):
        allowed, msg = check_shell_security('ls', allow_shell=False)
        assert allowed is False
        assert 'disabled' in msg.lower()
    def test_safe_command_allowed(self):
        allowed, msg = check_shell_security('ls -la')
        assert allowed is True
    def test_destructive_blocked(self):
        allowed, msg = check_shell_security('rm -rf /', allow_destructive=False)
        assert allowed is False
        assert 'destructive' in msg.lower()
    def test_destructive_allowed_when_enabled(self):
        allowed, msg = check_shell_security('rm -rf /tmp/test', allow_destructive=True)
        # rm -rf still triggers destructive check, but allow_destructive=True skips it
        # However rm -rf may also trigger the security check for force-remove
        # Let's check: the main security check should pass (no injection)
        # and destructive should be allowed
        assert allowed is True
    def test_injection_blocked(self):
        allowed, msg = check_shell_security('echo `evil`')
        assert allowed is False
        assert 'backtick' in msg.lower() or 'security' in msg.lower()
    def test_misparsing_always_blocked(self):
        allowed, msg = check_shell_security('echo\x00rm')
        assert allowed is False
    def test_safe_git_commit(self):
        allowed, msg = check_shell_security("git commit -m 'fix'")
        assert allowed is True
@@ -0,0 +1,405 @@
 """Tests for src/compact.py – the conversation compaction service."""
 from __future__ import annotations
 import tempfile
 import unittest
 from dataclasses import replace
 from pathlib import Path
 from unittest.mock import MagicMock
 from src.agent_runtime import LocalCodingAgent
 from src.agent_session import AgentMessage, AgentSessionState
 from src.agent_types import AgentRuntimeConfig, ModelConfig
 from src.compact import (
    AUTOCOMPACT_BUFFER_TOKENS,
    ERROR_INCOMPLETE_RESPONSE,
    ERROR_NOT_ENOUGH_MESSAGES,
    CompactionResult,
    compact_conversation,
    format_compact_summary,
    get_compact_prompt,
    get_compact_user_summary_message,
 )
 class TestGetCompactPrompt(unittest.TestCase):
    """Tests for the compact prompt builder."""
    def test_basic_prompt_contains_all_nine_sections(self) -> None:
        prompt = get_compact_prompt()
        for section in [
            '1. Primary Request and Intent',
            '2. Key Technical Concepts',
            '3. Files and Code Sections',
            '4. Errors and fixes',
            '5. Problem Solving',
            '6. All user messages',
            '7. Pending Tasks',
            '8. Current Work',
            '9. Optional Next Step',
        ]:
            self.assertIn(section, prompt, f'Missing section: {section}')
    def test_no_tools_preamble_present(self) -> None:
        prompt = get_compact_prompt()
        self.assertIn('CRITICAL: Respond with TEXT ONLY', prompt)
        self.assertIn('Do NOT call any tools', prompt)
    def test_no_tools_trailer_present(self) -> None:
        prompt = get_compact_prompt()
        self.assertIn('REMINDER: Do NOT call any tools', prompt)
    def test_analysis_and_summary_example_tags_present(self) -> None:
        prompt = get_compact_prompt()
        self.assertIn('<analysis>', prompt)
        self.assertIn('</analysis>', prompt)
        self.assertIn('<summary>', prompt)
        self.assertIn('</summary>', prompt)
    def test_custom_instructions_appended(self) -> None:
        prompt = get_compact_prompt('Focus on database changes.')
        self.assertIn('Additional Instructions:', prompt)
        self.assertIn('Focus on database changes.', prompt)
    def test_empty_custom_instructions_ignored(self) -> None:
        prompt_no_custom = get_compact_prompt()
        prompt_empty = get_compact_prompt('')
        prompt_whitespace = get_compact_prompt('   ')
        self.assertEqual(prompt_no_custom, prompt_empty)
        self.assertEqual(prompt_no_custom, prompt_whitespace)
    def test_none_custom_instructions_ignored(self) -> None:
        prompt_none = get_compact_prompt(None)
        prompt_no_arg = get_compact_prompt()
        self.assertEqual(prompt_none, prompt_no_arg)
 class TestFormatCompactSummary(unittest.TestCase):
    """Tests for the summary formatting / XML stripping."""
    def test_strips_analysis_block(self) -> None:
        raw = '<analysis>thinking here</analysis>\n\n<summary>result</summary>'
        formatted = format_compact_summary(raw)
        self.assertNotIn('<analysis>', formatted)
        self.assertNotIn('thinking here', formatted)
        self.assertIn('result', formatted)
    def test_unwraps_summary_tags(self) -> None:
        raw = '<summary>The main points.\n1. First</summary>'
        formatted = format_compact_summary(raw)
        self.assertNotIn('<summary>', formatted)
        self.assertNotIn('</summary>', formatted)
        self.assertIn('Summary:', formatted)
        self.assertIn('The main points.', formatted)
    def test_handles_no_xml_tags(self) -> None:
        raw = 'Plain text summary without any tags.'
        formatted = format_compact_summary(raw)
        self.assertEqual(formatted, raw)
    def test_collapses_excess_blank_lines(self) -> None:
        raw = '<analysis>x</analysis>\n\n\n\n<summary>y</summary>'
        formatted = format_compact_summary(raw)
        self.assertNotIn('\n\n\n', formatted)
    def test_multiline_analysis_stripped(self) -> None:
        raw = (
            '<analysis>\nLine 1\nLine 2\nLine 3\n</analysis>\n'
            '<summary>Final summary</summary>'
        )
        formatted = format_compact_summary(raw)
        self.assertNotIn('Line 1', formatted)
        self.assertIn('Final summary', formatted)
 class TestGetCompactUserSummaryMessage(unittest.TestCase):
    """Tests for the post-compact user message builder."""
    def test_basic_message_structure(self) -> None:
        msg = get_compact_user_summary_message('<summary>overview</summary>')
        self.assertIn('continued from a previous conversation', msg)
        self.assertIn('overview', msg)
    def test_transcript_path_appended(self) -> None:
        msg = get_compact_user_summary_message(
            '<summary>ok</summary>',
            transcript_path='/tmp/transcript.json',
        )
        self.assertIn('/tmp/transcript.json', msg)
    def test_suppress_follow_up(self) -> None:
        msg = get_compact_user_summary_message(
            '<summary>ok</summary>',
            suppress_follow_up=True,
        )
        self.assertIn('without asking the user any further questions', msg)
    def test_no_suppress_follow_up_default(self) -> None:
        msg = get_compact_user_summary_message('<summary>ok</summary>')
        self.assertNotIn('without asking the user any further questions', msg)
 class TestCompactConversation(unittest.TestCase):
    """Tests for the core compact_conversation() function."""
    def _make_agent(self, tmp_dir: str) -> LocalCodingAgent:
        """Create a minimal agent with a session loaded."""
        agent = LocalCodingAgent(
            model_config=ModelConfig(model='test-model'),
            runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
        )
        return agent
    def _set_session(
        self, agent: LocalCodingAgent, messages: list[AgentMessage]
    ) -> None:
        session = AgentSessionState(
            system_prompt_parts=('You are a helpful assistant.',),
            messages=messages,
        )
        agent.last_session = session
    def _make_messages(self, count: int) -> list[AgentMessage]:
        msgs: list[AgentMessage] = []
        for i in range(count):
            role = 'user' if i % 2 == 0 else 'assistant'
            msgs.append(
                AgentMessage(
                    role=role,
                    content=f'Message {i} content. ' * 10,
                    message_id=f'msg_{i}',
                )
            )
        return msgs
    def test_no_session_returns_error(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = self._make_agent(tmp_dir)
            agent.last_session = None
            result = compact_conversation(agent)
        self.assertIsNotNone(result.error)
        self.assertIn('Not enough', result.error)
    def test_empty_session_returns_error(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = self._make_agent(tmp_dir)
            self._set_session(agent, [])
            result = compact_conversation(agent)
        self.assertIsNotNone(result.error)
    def test_too_few_messages_returns_error(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = self._make_agent(tmp_dir)
            # With only 2 messages and preserve_count=4, nothing to compact
            self._set_session(agent, self._make_messages(2))
            result = compact_conversation(agent)
        self.assertIsNotNone(result.error)
    def test_successful_compaction(self) -> None:
        """Simulate a successful model call and verify session is compacted."""
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = self._make_agent(tmp_dir)
            msgs = self._make_messages(10)
            self._set_session(agent, msgs)
            # Mock the client's complete method
            from src.openai_compat import AssistantTurn
            from src.agent_types import UsageStats
            mock_turn = AssistantTurn(
                content=(
                    '<analysis>Thinking through the conversation...</analysis>\n'
                    '<summary>\n1. Primary Request and Intent:\n'
                    '   User wanted to test compaction.\n'
                    '2. Key Technical Concepts:\n   - Testing\n'
                    '3. Files and Code Sections:\n   - test.py\n'
                    '4. Errors and fixes:\n   - None\n'
                    '5. Problem Solving:\n   Basic testing.\n'
                    '6. All user messages:\n   - "test compaction"\n'
                    '7. Pending Tasks:\n   - None\n'
                    '8. Current Work:\n   Testing compaction.\n'
                    '9. Optional Next Step:\n   Verify it works.\n'
                    '</summary>'
                ),
                tool_calls=(),
                finish_reason='stop',
                raw_message={},
                usage=UsageStats(),
            )
            agent.client = MagicMock()
            agent.client.complete.return_value = mock_turn
            result = compact_conversation(agent)
        self.assertIsNone(result.error)
        self.assertGreater(result.pre_compact_token_count, 0)
        # Session should have fewer messages than original 10
        self.assertLess(
            len(agent.last_session.messages), 10,
            'Session should have fewer messages after compaction',
        )
        # Should contain a compact_boundary message
        boundary_msgs = [
            m for m in agent.last_session.messages
            if m.metadata.get('kind') == 'compact_boundary'
        ]
        self.assertEqual(len(boundary_msgs), 1)
        # Should contain a compact_summary message
        summary_msgs = [
            m for m in agent.last_session.messages
            if m.metadata.get('kind') == 'compact_summary'
        ]
        self.assertEqual(len(summary_msgs), 1)
        # Summary should not contain <analysis> block
        self.assertNotIn('<analysis>', result.summary_text)
        # Summary should contain the actual summary content
        self.assertIn('User wanted to test compaction', result.summary_text)
    def test_api_error_returns_compaction_error(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = self._make_agent(tmp_dir)
            self._set_session(agent, self._make_messages(10))
            agent.client = MagicMock()
            agent.client.complete.side_effect = RuntimeError('API down')
            result = compact_conversation(agent)
        self.assertIsNotNone(result.error)
        self.assertIn('API down', result.error)
    def test_empty_model_response_returns_error(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = self._make_agent(tmp_dir)
            self._set_session(agent, self._make_messages(10))
            from src.openai_compat import AssistantTurn
            from src.agent_types import UsageStats
            agent.client = MagicMock()
            agent.client.complete.return_value = AssistantTurn(
                content='',
                tool_calls=(),
                finish_reason='stop',
                raw_message={},
                usage=UsageStats(),
            )
            result = compact_conversation(agent)
        self.assertIsNotNone(result.error)
    def test_preserves_tail_messages(self) -> None:
        """The most recent messages should survive compaction."""
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = self._make_agent(tmp_dir)
            msgs = self._make_messages(12)
            self._set_session(agent, msgs)
            from src.openai_compat import AssistantTurn
            from src.agent_types import UsageStats
            agent.client = MagicMock()
            agent.client.complete.return_value = AssistantTurn(
                content='<summary>Summarised.</summary>',
                tool_calls=(),
                finish_reason='stop',
                raw_message={},
                usage=UsageStats(),
            )
            result = compact_conversation(agent)
        self.assertIsNone(result.error)
        # The last 4 messages (default preserve_count) should still be present
        session_contents = [m.content for m in agent.last_session.messages]
        for original_msg in msgs[-4:]:
            self.assertIn(
                original_msg.content,
                session_contents,
                f'Tail message "{original_msg.message_id}" should be preserved',
            )
    def test_custom_instructions_passed_to_prompt(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = self._make_agent(tmp_dir)
            self._set_session(agent, self._make_messages(10))
            from src.openai_compat import AssistantTurn
            from src.agent_types import UsageStats
            agent.client = MagicMock()
            agent.client.complete.return_value = AssistantTurn(
                content='<summary>Custom summary.</summary>',
                tool_calls=(),
                finish_reason='stop',
                raw_message={},
                usage=UsageStats(),
            )
            compact_conversation(agent, custom_instructions='Focus on CSS.')
        # Check that the API was called with custom instructions in the prompt
        call_args = agent.client.complete.call_args
        messages = call_args[0][0]
        last_user_msg = messages[-1]['content']
        self.assertIn('Focus on CSS.', last_user_msg)
        self.assertIn('Additional Instructions:', last_user_msg)
 class TestCompactSlashCommand(unittest.TestCase):
    """Test the /compact slash command handler end-to-end."""
    def test_slash_compact_returns_success_message(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
            )
            # Set up a session with enough messages
            session = AgentSessionState(
                system_prompt_parts=('You are a helper.',),
                messages=[
                    AgentMessage(role='user', content='Hello', message_id=f'u{i}')
                    if i % 2 == 0
                    else AgentMessage(role='assistant', content='Hi', message_id=f'a{i}')
                    for i in range(10)
                ],
            )
            agent.last_session = session
            from src.openai_compat import AssistantTurn
            from src.agent_types import UsageStats
            agent.client = MagicMock()
            agent.client.complete.return_value = AssistantTurn(
                content='<summary>Session summarised.</summary>',
                tool_calls=(),
                finish_reason='stop',
                raw_message={},
                usage=UsageStats(),
            )
            result = agent.run('/compact')
        self.assertIn('Conversation compacted', result.final_output)
        self.assertIn('Tokens before', result.final_output)
    def test_slash_compact_no_session_returns_error(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
            )
            result = agent.run('/compact')
        self.assertIn('failed', result.final_output.lower())
 class TestConstants(unittest.TestCase):
    """Verify key constants match the npm reference."""
    def test_autocompact_buffer_tokens(self) -> None:
        self.assertEqual(AUTOCOMPACT_BUFFER_TOKENS, 13_000)
 if __name__ == '__main__':
    unittest.main()
@@ -0,0 +1,189 @@
 """Tests for /cost, /exit, and /diff slash commands."""
 from __future__ import annotations
 import subprocess
 import tempfile
 import unittest
 from pathlib import Path
 from unittest.mock import MagicMock, patch
 from src.agent_runtime import LocalCodingAgent
 from src.agent_slash_commands import preprocess_slash_command
 from src.agent_types import AgentRuntimeConfig, ModelConfig, UsageStats
 class TestCostCommand(unittest.TestCase):
    """Tests for the /cost slash command."""
    def test_cost_shows_zero_initially(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
            )
            result = agent.run('/cost')
        self.assertIn('Total cost:', result.final_output)
        self.assertIn('$0.0000', result.final_output)
        self.assertIn('Total input tokens:', result.final_output)
        self.assertIn('Total output tokens:', result.final_output)
        self.assertIn('Total tokens:', result.final_output)
    def test_cost_shows_accumulated_usage(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
            )
            # Simulate accumulated usage
            agent.cumulative_usage = UsageStats(
                input_tokens=1000,
                output_tokens=500,
                cache_read_input_tokens=200,
            )
            agent.cumulative_cost_usd = 0.05
            result = agent.run('/cost')
        self.assertIn('$0.05', result.final_output)
        self.assertIn('1,000', result.final_output)
        self.assertIn('500', result.final_output)
        self.assertIn('Cache read tokens:', result.final_output)
    def test_cost_hides_zero_cache_and_reasoning(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
            )
            agent.cumulative_usage = UsageStats(
                input_tokens=100,
                output_tokens=50,
            )
            result = agent.run('/cost')
        # Should NOT show cache/reasoning lines when they're zero
        self.assertNotIn('Cache read tokens:', result.final_output)
        self.assertNotIn('Cache creation tokens:', result.final_output)
        self.assertNotIn('Reasoning tokens:', result.final_output)
    def test_cost_small_amounts_show_four_decimals(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
            )
            agent.cumulative_cost_usd = 0.0023
            result = agent.run('/cost')
        self.assertIn('$0.0023', result.final_output)
 class TestExitCommand(unittest.TestCase):
    """Tests for the /exit and /quit slash commands."""
    def test_exit_triggers_system_exit(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
            )
            with self.assertRaises(SystemExit) as cm:
                agent.run('/exit')
            self.assertEqual(cm.exception.code, 0)
    def test_quit_triggers_system_exit(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
            )
            with self.assertRaises(SystemExit):
                agent.run('/quit')
 class TestDiffCommand(unittest.TestCase):
    """Tests for the /diff slash command."""
    def test_diff_in_non_git_directory(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
            )
            result = agent.run('/diff')
        # In a non-git dir, git diff returns an error
        output = result.final_output.lower()
        self.assertTrue(
            'no uncommitted' in output
            or 'not a git' in output
            or 'error' in output,
            f'Expected a non-git or no-changes message, got: {result.final_output}',
        )
    def test_diff_in_git_repo_with_no_changes(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            workspace = Path(tmp_dir)
            subprocess.run(
                ['git', 'init'], cwd=str(workspace),
                capture_output=True, check=True,
            )
            subprocess.run(
                ['git', 'config', 'user.email', 'test@test.com'],
                cwd=str(workspace), capture_output=True,
            )
            subprocess.run(
                ['git', 'config', 'user.name', 'Test'],
                cwd=str(workspace), capture_output=True,
            )
            (workspace / 'hello.txt').write_text('hello\n')
            subprocess.run(
                ['git', 'add', '.'], cwd=str(workspace),
                capture_output=True, check=True,
            )
            subprocess.run(
                ['git', 'commit', '-m', 'init'], cwd=str(workspace),
                capture_output=True, check=True,
            )
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=workspace),
            )
            result = agent.run('/diff')
        self.assertIn('No uncommitted changes', result.final_output)
    def test_diff_shows_changes(self) -> None:
        with tempfile.TemporaryDirectory() as tmp_dir:
            workspace = Path(tmp_dir)
            subprocess.run(
                ['git', 'init'], cwd=str(workspace),
                capture_output=True, check=True,
            )
            subprocess.run(
                ['git', 'config', 'user.email', 'test@test.com'],
                cwd=str(workspace), capture_output=True,
            )
            subprocess.run(
                ['git', 'config', 'user.name', 'Test'],
                cwd=str(workspace), capture_output=True,
            )
            (workspace / 'hello.txt').write_text('hello\n')
            subprocess.run(
                ['git', 'add', '.'], cwd=str(workspace),
                capture_output=True, check=True,
            )
            subprocess.run(
                ['git', 'commit', '-m', 'init'], cwd=str(workspace),
                capture_output=True, check=True,
            )
            # Make a change
            (workspace / 'hello.txt').write_text('hello world\n')
            agent = LocalCodingAgent(
                model_config=ModelConfig(model='test-model'),
                runtime_config=AgentRuntimeConfig(cwd=workspace),
            )
            result = agent.run('/diff')
        self.assertIn('hello', result.final_output)
        self.assertIn('diff', result.final_output.lower())
 if __name__ == '__main__':
    unittest.main()
@@ -0,0 +1,299 @@
 """Tests for /files, /copy, /export, /stats, /tag, /rename, /branch, /effort, /doctor."""
 from __future__ import annotations
 import json
 import os
 import shutil
 import subprocess
 import tempfile
 import unittest
 from pathlib import Path
 from src.agent_runtime import LocalCodingAgent
 from src.agent_session import AgentMessage, AgentSessionState
 from src.agent_types import AgentRuntimeConfig, ModelConfig, UsageStats
 def _make_agent(tmp_dir: str) -> LocalCodingAgent:
    return LocalCodingAgent(
        model_config=ModelConfig(model='test-model'),
        runtime_config=AgentRuntimeConfig(cwd=Path(tmp_dir)),
    )
 def _set_session(agent: LocalCodingAgent, messages: list[AgentMessage]) -> None:
    session = AgentSessionState(
        system_prompt_parts=('You are a helper.',),
        messages=messages,
    )
    agent.last_session = session
 class TestFilesCommand(unittest.TestCase):
    def test_no_session(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/files')
        self.assertIn('No active session', result.final_output)
    def test_no_files_in_context(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Hello'),
                AgentMessage(role='assistant', content='Hi'),
            ])
            result = agent.run('/files')
        self.assertIn('No files loaded', result.final_output)
    def test_files_from_tool_calls(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Read main.py'),
                AgentMessage(
                    role='assistant', content='',
                    tool_calls=(
                        {'id': 'tc1', 'type': 'function', 'function': {
                            'name': 'Read',
                            'arguments': json.dumps({'file_path': '/home/user/project/main.py'}),
                        }},
                    ),
                ),
                AgentMessage(
                    role='tool', content='print("hello")',
                    name='Read',
                    tool_call_id='tc1',
                    metadata={'path': '/home/user/project/main.py'},
                ),
            ])
            result = agent.run('/files')
        self.assertIn('Files in context', result.final_output)
        self.assertIn('main.py', result.final_output)
 class TestCopyCommand(unittest.TestCase):
    def test_no_session(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/copy')
        self.assertIn('No active session', result.final_output)
    def test_no_assistant_messages(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Hello'),
            ])
            result = agent.run('/copy')
        self.assertIn('No assistant responses', result.final_output)
    def test_copies_latest_response(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Hello'),
                AgentMessage(role='assistant', content='First response.'),
                AgentMessage(role='user', content='More'),
                AgentMessage(role='assistant', content='Second response with details.'),
            ])
            result = agent.run('/copy')
        self.assertIn('Copied', result.final_output)
        self.assertIn('response.md', result.final_output)
        # Verify the file was written
        tmp_file = Path(tempfile.gettempdir()) / 'claw-code' / 'response.md'
        self.assertTrue(tmp_file.exists())
        content = tmp_file.read_text()
        self.assertEqual(content, 'Second response with details.')
    def test_copies_nth_response(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Hello'),
                AgentMessage(role='assistant', content='First.'),
                AgentMessage(role='user', content='More'),
                AgentMessage(role='assistant', content='Second.'),
            ])
            result = agent.run('/copy 1')
        tmp_file = Path(tempfile.gettempdir()) / 'claw-code' / 'response.md'
        content = tmp_file.read_text()
        self.assertEqual(content, 'First.')
 class TestExportCommand(unittest.TestCase):
    def test_no_session(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/export')
        self.assertIn('No active session', result.final_output)
    def test_exports_with_auto_filename(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Hello'),
                AgentMessage(role='assistant', content='Hi there'),
            ])
            result = agent.run('/export')
        self.assertIn('Exported 2 messages', result.final_output)
        self.assertIn('.txt', result.final_output)
    def test_exports_with_custom_filename(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Hello'),
                AgentMessage(role='assistant', content='Hi'),
            ])
            result = agent.run('/export my_chat')
            self.assertIn('my_chat.txt', result.final_output)
            out_file = Path(tmp) / 'my_chat.txt'
            self.assertTrue(out_file.exists())
            content = out_file.read_text()
            self.assertIn('Hello', content)
            self.assertIn('Hi', content)
 class TestStatsCommand(unittest.TestCase):
    def test_shows_statistics(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Hello'),
                AgentMessage(role='assistant', content='Hi'),
                AgentMessage(role='user', content='Question'),
            ])
            agent.cumulative_usage = UsageStats(input_tokens=500, output_tokens=200)
            result = agent.run('/stats')
        self.assertIn('Session Statistics', result.final_output)
        self.assertIn('3 total', result.final_output)
        self.assertIn('2 user', result.final_output)
        self.assertIn('1 assistant', result.final_output)
        self.assertIn('500', result.final_output)
        self.assertIn('200', result.final_output)
 class TestTagCommand(unittest.TestCase):
    def test_no_tags_initially(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/tag')
        self.assertIn('No tags set', result.final_output)
    def test_add_tag(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/tag important')
        self.assertIn('Added tag: important', result.final_output)
    def test_toggle_tag(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            agent.run('/tag my-tag')
            result = agent.run('/tag my-tag')
        self.assertIn('Removed tag: my-tag', result.final_output)
    def test_list_tags(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            agent.run('/tag alpha')
            agent.run('/tag beta')
            result = agent.run('/tag')
        self.assertIn('alpha', result.final_output)
        self.assertIn('beta', result.final_output)
 class TestRenameCommand(unittest.TestCase):
    def test_no_name(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/rename')
        self.assertIn('Usage', result.final_output)
    def test_rename_session(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/rename My Cool Session')
        self.assertIn('renamed to: My Cool Session', result.final_output)
 class TestBranchCommand(unittest.TestCase):
    def test_no_session(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/branch')
        self.assertIn('No active session', result.final_output)
    def test_branch_with_session(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Hello'),
                AgentMessage(role='assistant', content='Hi'),
            ])
            result = agent.run('/branch my-feature')
        self.assertIn('Created branch "my-feature"', result.final_output)
        self.assertIn('Saved to:', result.final_output)
    def test_branch_auto_name(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            _set_session(agent, [
                AgentMessage(role='user', content='Hello'),
            ])
            result = agent.run('/branch')
        self.assertIn('Created branch "branch-', result.final_output)
 class TestEffortCommand(unittest.TestCase):
    def test_show_current_effort(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/effort')
        self.assertIn('Current effort level: auto', result.final_output)
    def test_set_effort_level(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/effort high')
        self.assertIn('Set effort level to: high', result.final_output)
    def test_invalid_effort_level(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/effort extreme')
        self.assertIn('Invalid effort level', result.final_output)
    def test_all_valid_levels(self) -> None:
        for level in ('low', 'medium', 'high', 'max', 'auto'):
            with tempfile.TemporaryDirectory() as tmp:
                agent = _make_agent(tmp)
                result = agent.run(f'/effort {level}')
            self.assertIn(f'Set effort level to: {level}', result.final_output)
 class TestDoctorCommand(unittest.TestCase):
    def test_shows_report(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            agent = _make_agent(tmp)
            result = agent.run('/doctor')
        output = result.final_output
        self.assertIn('Doctor Report', output)
        self.assertIn('Python version', output)
        self.assertIn('git', output)
        self.assertIn('Model', output)
        self.assertIn('Working directory', output)
    def test_detects_claude_md(self) -> None:
        with tempfile.TemporaryDirectory() as tmp:
            (Path(tmp) / 'CLAUDE.md').write_text('memory file')
            agent = _make_agent(tmp)
            result = agent.run('/doctor')
        self.assertIn('CLAUDE.md', result.final_output)
        self.assertIn('found', result.final_output)
 if __name__ == '__main__':
    unittest.main()
@@ -0,0 +1,643 @@
 """Tests for prompt_constants module.
 Validates that all constants ported from npm src/constants/ are present,
 correctly typed, and that helper functions behave as expected.
 """
 from __future__ import annotations
 import os
 import platform
 from unittest.mock import patch
 import pytest
 from src.prompt_constants import (
    # Product metadata
    PRODUCT_URL,
    CLAUDE_AI_BASE_URL,
    # System prompt prefixes
    DEFAULT_SYSPROMPT_PREFIX,
    AGENT_SDK_CLAUDE_CODE_PRESET_PREFIX,
    AGENT_SDK_PREFIX,
    CLI_SYSPROMPT_PREFIXES,
    # Cyber risk
    CYBER_RISK_INSTRUCTION,
    # API limits
    API_IMAGE_MAX_BASE64_SIZE,
    IMAGE_TARGET_RAW_SIZE,
    IMAGE_MAX_WIDTH,
    IMAGE_MAX_HEIGHT,
    PDF_TARGET_RAW_SIZE,
    API_PDF_MAX_PAGES,
    PDF_EXTRACT_SIZE_THRESHOLD,
    PDF_MAX_EXTRACT_SIZE,
    PDF_MAX_PAGES_PER_READ,
    PDF_AT_MENTION_INLINE_THRESHOLD,
    API_MAX_MEDIA_PER_REQUEST,
    # Tool limits
    DEFAULT_MAX_RESULT_SIZE_CHARS,
    MAX_TOOL_RESULT_TOKENS,
    BYTES_PER_TOKEN,
    MAX_TOOL_RESULT_BYTES,
    MAX_TOOL_RESULTS_PER_MESSAGE_CHARS,
    TOOL_SUMMARY_MAX_LENGTH,
    # Spinner verbs
    SPINNER_VERBS,
    # Turn completion verbs
    TURN_COMPLETION_VERBS,
    # Figures
    BLACK_CIRCLE,
    BULLET_OPERATOR,
    TEARDROP_ASTERISK,
    UP_ARROW,
    DOWN_ARROW,
    LIGHTNING_BOLT,
    EFFORT_LOW,
    EFFORT_MEDIUM,
    EFFORT_HIGH,
    EFFORT_MAX,
    PLAY_ICON,
    PAUSE_ICON,
    REFRESH_ARROW,
    CHANNEL_ARROW,
    INJECTED_ARROW,
    FORK_GLYPH,
    DIAMOND_OPEN,
    DIAMOND_FILLED,
    REFERENCE_MARK,
    FLAG_ICON,
    BLOCKQUOTE_BAR,
    HEAVY_HORIZONTAL,
    BRIDGE_SPINNER_FRAMES,
    BRIDGE_READY_INDICATOR,
    BRIDGE_FAILED_INDICATOR,
    # XML tags
    COMMAND_NAME_TAG,
    COMMAND_MESSAGE_TAG,
    COMMAND_ARGS_TAG,
    BASH_INPUT_TAG,
    BASH_STDOUT_TAG,
    BASH_STDERR_TAG,
    LOCAL_COMMAND_STDOUT_TAG,
    LOCAL_COMMAND_STDERR_TAG,
    LOCAL_COMMAND_CAVEAT_TAG,
    TERMINAL_OUTPUT_TAGS,
    TICK_TAG,
    TASK_NOTIFICATION_TAG,
    TASK_ID_TAG,
    TOOL_USE_ID_TAG,
    TASK_TYPE_TAG,
    OUTPUT_FILE_TAG,
    STATUS_TAG,
    SUMMARY_TAG,
    REASON_TAG,
    WORKTREE_TAG,
    WORKTREE_PATH_TAG,
    WORKTREE_BRANCH_TAG,
    ULTRAPLAN_TAG,
    REMOTE_REVIEW_TAG,
    REMOTE_REVIEW_PROGRESS_TAG,
    TEAMMATE_MESSAGE_TAG,
    CHANNEL_MESSAGE_TAG,
    CHANNEL_TAG,
    CROSS_SESSION_MESSAGE_TAG,
    FORK_BOILERPLATE_TAG,
    FORK_DIRECTIVE_PREFIX,
    COMMON_HELP_ARGS,
    COMMON_INFO_ARGS,
    # Messages
    NO_CONTENT_MESSAGE,
    # Date utilities
    get_local_iso_date,
    get_session_start_date,
    reset_session_start_date,
    get_local_month_year,
    # System prompt section caching
    SystemPromptSection,
    system_prompt_section,
    dangerous_uncached_system_prompt_section,
    resolve_system_prompt_sections,
    clear_system_prompt_sections,
    # Output styles
    DEFAULT_OUTPUT_STYLE_NAME,
    OutputStyleConfig,
    OUTPUT_STYLE_CONFIGS,
    # Knowledge cutoff
    FRONTIER_MODEL_NAME,
    get_knowledge_cutoff,
    CLAUDE_MODEL_IDS,
    # Prompt sections
    HOOKS_SECTION,
    SYSTEM_REMINDERS_SECTION,
    SUMMARIZE_TOOL_RESULTS_SECTION,
    DEFAULT_AGENT_PROMPT,
    SYSTEM_PROMPT_DYNAMIC_BOUNDARY,
    get_language_section,
    get_output_style_section,
    get_scratchpad_instructions,
    # Error IDs
    E_TOOL_USE_SUMMARY_GENERATION_FAILED,
 )
 # =========================================================================
 # Product metadata
 # =========================================================================
 class TestProductMetadata:
    def test_product_url(self):
        assert PRODUCT_URL == "https://claude.com/claude-code"
    def test_claude_ai_base_url(self):
        assert CLAUDE_AI_BASE_URL == "https://claude.ai"
 # =========================================================================
 # System prompt prefixes
 # =========================================================================
 class TestSystemPromptPrefixes:
    def test_default_prefix_content(self):
        assert "Claude Code" in DEFAULT_SYSPROMPT_PREFIX
        assert "Anthropic" in DEFAULT_SYSPROMPT_PREFIX
    def test_agent_sdk_prefix_content(self):
        assert "Agent SDK" in AGENT_SDK_PREFIX
    def test_cli_sysprompt_prefixes_is_frozenset(self):
        assert isinstance(CLI_SYSPROMPT_PREFIXES, frozenset)
        assert len(CLI_SYSPROMPT_PREFIXES) == 3
    def test_all_prefixes_in_set(self):
        assert DEFAULT_SYSPROMPT_PREFIX in CLI_SYSPROMPT_PREFIXES
        assert AGENT_SDK_CLAUDE_CODE_PRESET_PREFIX in CLI_SYSPROMPT_PREFIXES
        assert AGENT_SDK_PREFIX in CLI_SYSPROMPT_PREFIXES
 # =========================================================================
 # Cyber risk
 # =========================================================================
 class TestCyberRisk:
    def test_instruction_mentions_ctf(self):
        assert "CTF" in CYBER_RISK_INSTRUCTION
    def test_instruction_mentions_dos(self):
        assert "DoS" in CYBER_RISK_INSTRUCTION
 # =========================================================================
 # API limits
 # =========================================================================
 class TestAPILimits:
    def test_image_base64_size(self):
        assert API_IMAGE_MAX_BASE64_SIZE == 5 * 1024 * 1024
    def test_image_target_raw_size(self):
        assert IMAGE_TARGET_RAW_SIZE == (API_IMAGE_MAX_BASE64_SIZE * 3) // 4
    def test_image_dimensions(self):
        assert IMAGE_MAX_WIDTH == 2000
        assert IMAGE_MAX_HEIGHT == 2000
    def test_pdf_target_raw_size(self):
        assert PDF_TARGET_RAW_SIZE == 20 * 1024 * 1024
    def test_pdf_max_pages(self):
        assert API_PDF_MAX_PAGES == 100
    def test_pdf_extract_threshold(self):
        assert PDF_EXTRACT_SIZE_THRESHOLD == 3 * 1024 * 1024
    def test_pdf_max_extract_size(self):
        assert PDF_MAX_EXTRACT_SIZE == 100 * 1024 * 1024
    def test_pdf_pages_per_read(self):
        assert PDF_MAX_PAGES_PER_READ == 20
    def test_pdf_inline_threshold(self):
        assert PDF_AT_MENTION_INLINE_THRESHOLD == 10
    def test_media_per_request(self):
        assert API_MAX_MEDIA_PER_REQUEST == 100
 # =========================================================================
 # Tool limits
 # =========================================================================
 class TestToolLimits:
    def test_default_max_result_size(self):
        assert DEFAULT_MAX_RESULT_SIZE_CHARS == 50_000
    def test_max_tool_result_tokens(self):
        assert MAX_TOOL_RESULT_TOKENS == 100_000
    def test_bytes_per_token(self):
        assert BYTES_PER_TOKEN == 4
    def test_max_tool_result_bytes_derived(self):
        assert MAX_TOOL_RESULT_BYTES == MAX_TOOL_RESULT_TOKENS * BYTES_PER_TOKEN
        assert MAX_TOOL_RESULT_BYTES == 400_000
    def test_max_per_message_chars(self):
        assert MAX_TOOL_RESULTS_PER_MESSAGE_CHARS == 200_000
    def test_tool_summary_max_length(self):
        assert TOOL_SUMMARY_MAX_LENGTH == 50
 # =========================================================================
 # Spinner verbs
 # =========================================================================
 class TestSpinnerVerbs:
    def test_is_tuple(self):
        assert isinstance(SPINNER_VERBS, tuple)
    def test_count(self):
        assert len(SPINNER_VERBS) == 187
    def test_first_verb(self):
        assert SPINNER_VERBS[0] == "Accomplishing"
    def test_last_verb(self):
        assert SPINNER_VERBS[-1] == "Zigzagging"
    def test_all_strings(self):
        for verb in SPINNER_VERBS:
            assert isinstance(verb, str)
    def test_contains_clauding(self):
        assert "Clauding" in SPINNER_VERBS
    def test_contains_thinking(self):
        assert "Thinking" in SPINNER_VERBS
    def test_no_duplicates(self):
        assert len(SPINNER_VERBS) == len(set(SPINNER_VERBS))
 # =========================================================================
 # Turn completion verbs
 # =========================================================================
 class TestTurnCompletionVerbs:
    def test_is_tuple(self):
        assert isinstance(TURN_COMPLETION_VERBS, tuple)
    def test_count(self):
        assert len(TURN_COMPLETION_VERBS) == 8
    def test_contains_worked(self):
        assert "Worked" in TURN_COMPLETION_VERBS
    def test_contains_baked(self):
        assert "Baked" in TURN_COMPLETION_VERBS
    def test_all_past_tense(self):
        # All end in 'd' (past tense)
        for verb in TURN_COMPLETION_VERBS:
            assert verb[-1] == "d", f"{verb} doesn't end with 'd'"
 # =========================================================================
 # Figures / UI symbols
 # =========================================================================
 class TestFigures:
    def test_black_circle_is_string(self):
        assert isinstance(BLACK_CIRCLE, str)
        assert len(BLACK_CIRCLE) == 1
    def test_effort_symbols_are_distinct(self):
        symbols = {EFFORT_LOW, EFFORT_MEDIUM, EFFORT_HIGH, EFFORT_MAX}
        assert len(symbols) == 4
    def test_arrows(self):
        assert UP_ARROW == "\u2191"
        assert DOWN_ARROW == "\u2193"
    def test_bridge_spinner_frames(self):
        assert isinstance(BRIDGE_SPINNER_FRAMES, tuple)
        assert len(BRIDGE_SPINNER_FRAMES) == 4
    def test_play_pause_icons(self):
        assert PLAY_ICON == "\u25b6"
        assert PAUSE_ICON == "\u23f8"
    def test_diamond_symbols(self):
        assert DIAMOND_OPEN == "\u25c7"
        assert DIAMOND_FILLED == "\u25c6"
 # =========================================================================
 # XML tag constants
 # =========================================================================
 class TestXMLTags:
    def test_command_tags(self):
        assert COMMAND_NAME_TAG == "command-name"
        assert COMMAND_MESSAGE_TAG == "command-message"
        assert COMMAND_ARGS_TAG == "command-args"
    def test_bash_tags(self):
        assert BASH_INPUT_TAG == "bash-input"
        assert BASH_STDOUT_TAG == "bash-stdout"
        assert BASH_STDERR_TAG == "bash-stderr"
    def test_terminal_output_tags_tuple(self):
        assert isinstance(TERMINAL_OUTPUT_TAGS, tuple)
        assert len(TERMINAL_OUTPUT_TAGS) == 6
        assert BASH_INPUT_TAG in TERMINAL_OUTPUT_TAGS
        assert LOCAL_COMMAND_STDOUT_TAG in TERMINAL_OUTPUT_TAGS
    def test_tick_tag(self):
        assert TICK_TAG == "tick"
    def test_task_tags(self):
        assert TASK_NOTIFICATION_TAG == "task-notification"
        assert TASK_ID_TAG == "task-id"
        assert TOOL_USE_ID_TAG == "tool-use-id"
    def test_worktree_tags(self):
        assert WORKTREE_TAG == "worktree"
        assert WORKTREE_PATH_TAG == "worktreePath"
    def test_fork_tags(self):
        assert FORK_BOILERPLATE_TAG == "fork-boilerplate"
        assert FORK_DIRECTIVE_PREFIX == "Your directive: "
    def test_common_help_args(self):
        assert isinstance(COMMON_HELP_ARGS, tuple)
        assert "help" in COMMON_HELP_ARGS
        assert "-h" in COMMON_HELP_ARGS
        assert "--help" in COMMON_HELP_ARGS
    def test_common_info_args(self):
        assert isinstance(COMMON_INFO_ARGS, tuple)
        assert "list" in COMMON_INFO_ARGS
        assert "status" in COMMON_INFO_ARGS
        assert "?" in COMMON_INFO_ARGS
 # =========================================================================
 # Message constants
 # =========================================================================
 class TestMessages:
    def test_no_content_message(self):
        assert NO_CONTENT_MESSAGE == "(no content)"
 # =========================================================================
 # Date utilities
 # =========================================================================
 class TestDateUtilities:
    def test_get_local_iso_date_format(self):
        d = get_local_iso_date()
        parts = d.split("-")
        assert len(parts) == 3
        assert len(parts[0]) == 4  # year
        assert len(parts[1]) == 2  # month
        assert len(parts[2]) == 2  # day
    def test_get_local_iso_date_override(self):
        with patch.dict(os.environ, {"CLAUDE_CODE_OVERRIDE_DATE": "2025-01-15"}):
            assert get_local_iso_date() == "2025-01-15"
    def test_get_session_start_date_memoised(self):
        reset_session_start_date()
        d1 = get_session_start_date()
        d2 = get_session_start_date()
        assert d1 == d2
    def test_reset_session_start_date(self):
        reset_session_start_date()
        d = get_session_start_date()
        assert isinstance(d, str)
        reset_session_start_date()
        # After reset, should still return valid date
        d2 = get_session_start_date()
        assert isinstance(d2, str)
    def test_get_local_month_year_format(self):
        result = get_local_month_year()
        parts = result.split()
        assert len(parts) == 2
        assert parts[1].isdigit()
        assert len(parts[1]) == 4
    def test_get_local_month_year_override(self):
        with patch.dict(os.environ, {"CLAUDE_CODE_OVERRIDE_DATE": "2026-02-15"}):
            assert get_local_month_year() == "February 2026"
 # =========================================================================
 # System prompt section caching
 # =========================================================================
 class TestSystemPromptSections:
    def setup_method(self):
        clear_system_prompt_sections()
    def test_system_prompt_section_creates_cached(self):
        s = system_prompt_section("test", lambda: "hello")
        assert s.name == "test"
        assert s.cache_break is False
    def test_dangerous_uncached_creates_volatile(self):
        s = dangerous_uncached_system_prompt_section("test", lambda: "hello", "reason")
        assert s.name == "test"
        assert s.cache_break is True
    def test_resolve_caches_sections(self):
        call_count = 0
        def compute():
            nonlocal call_count
            call_count += 1
            return f"value-{call_count}"
        sections = [system_prompt_section("s1", compute)]
        r1 = resolve_system_prompt_sections(sections)
        r2 = resolve_system_prompt_sections(sections)
        assert r1 == ["value-1"]
        assert r2 == ["value-1"]  # cached
        assert call_count == 1
    def test_uncached_recomputes(self):
        call_count = 0
        def compute():
            nonlocal call_count
            call_count += 1
            return f"value-{call_count}"
        sections = [dangerous_uncached_system_prompt_section("s2", compute, "test")]
        r1 = resolve_system_prompt_sections(sections)
        r2 = resolve_system_prompt_sections(sections)
        assert r1 == ["value-1"]
        assert r2 == ["value-2"]  # recomputed
        assert call_count == 2
    def test_clear_resets_cache(self):
        call_count = 0
        def compute():
            nonlocal call_count
            call_count += 1
            return f"value-{call_count}"
        sections = [system_prompt_section("s3", compute)]
        resolve_system_prompt_sections(sections)
        clear_system_prompt_sections()
        r = resolve_system_prompt_sections(sections)
        assert r == ["value-2"]
        assert call_count == 2
    def test_resolve_handles_none(self):
        sections = [system_prompt_section("nil", lambda: None)]
        r = resolve_system_prompt_sections(sections)
        assert r == [None]
    def test_multiple_sections(self):
        sections = [
            system_prompt_section("a", lambda: "alpha"),
            system_prompt_section("b", lambda: "beta"),
            system_prompt_section("c", lambda: None),
        ]
        r = resolve_system_prompt_sections(sections)
        assert r == ["alpha", "beta", None]
 # =========================================================================
 # Output styles
 # =========================================================================
 class TestOutputStyles:
    def test_default_style_name(self):
        assert DEFAULT_OUTPUT_STYLE_NAME == "default"
    def test_default_style_is_none(self):
        assert OUTPUT_STYLE_CONFIGS[DEFAULT_OUTPUT_STYLE_NAME] is None
    def test_explanatory_exists(self):
        style = OUTPUT_STYLE_CONFIGS["Explanatory"]
        assert style is not None
        assert style.name == "Explanatory"
        assert "explains" in style.description
    def test_learning_exists(self):
        style = OUTPUT_STYLE_CONFIGS["Learning"]
        assert style is not None
        assert style.name == "Learning"
        assert "hands-on" in style.description
    def test_output_style_config_frozen(self):
        style = OutputStyleConfig(
            name="Test", description="desc", prompt="prompt"
        )
        with pytest.raises(Exception):
            style.name = "other"  # type: ignore[misc]
 # =========================================================================
 # Knowledge cutoff
 # =========================================================================
 class TestKnowledgeCutoff:
    def test_frontier_model_name(self):
        assert FRONTIER_MODEL_NAME == "Claude Opus 4.6"
    def test_opus_46_cutoff(self):
        assert get_knowledge_cutoff("claude-opus-4-6-20250601") == "May 2025"
    def test_sonnet_46_cutoff(self):
        assert get_knowledge_cutoff("claude-sonnet-4-6-20250801") == "August 2025"
    def test_opus_45_cutoff(self):
        assert get_knowledge_cutoff("claude-opus-4-5-20250601") == "May 2025"
    def test_haiku_cutoff(self):
        assert get_knowledge_cutoff("claude-haiku-4-20250201") == "February 2025"
    def test_sonnet_4_cutoff(self):
        assert get_knowledge_cutoff("claude-sonnet-4-20250114") == "January 2025"
    def test_unknown_model_returns_none(self):
        assert get_knowledge_cutoff("gpt-4-turbo") is None
    def test_claude_model_ids(self):
        assert "opus" in CLAUDE_MODEL_IDS
        assert "sonnet" in CLAUDE_MODEL_IDS
        assert "haiku" in CLAUDE_MODEL_IDS
 # =========================================================================
 # Prompt section helpers
 # =========================================================================
 class TestPromptSectionHelpers:
    def test_hooks_section_content(self):
        assert "hooks" in HOOKS_SECTION
        assert "user-prompt-submit-hook" in HOOKS_SECTION
    def test_system_reminders_section(self):
        assert "system-reminder" in SYSTEM_REMINDERS_SECTION
    def test_summarize_tool_results(self):
        assert "tool results" in SUMMARIZE_TOOL_RESULTS_SECTION
    def test_default_agent_prompt(self):
        assert "agent for Claude Code" in DEFAULT_AGENT_PROMPT
    def test_dynamic_boundary(self):
        assert SYSTEM_PROMPT_DYNAMIC_BOUNDARY == "__SYSTEM_PROMPT_DYNAMIC_BOUNDARY__"
    def test_language_section_none_when_no_preference(self):
        assert get_language_section(None) is None
        assert get_language_section("") is None
    def test_language_section_with_preference(self):
        result = get_language_section("Spanish")
        assert result is not None
        assert "Spanish" in result
        assert "# Language" in result
    def test_output_style_section_none_when_no_config(self):
        assert get_output_style_section(None) is None
    def test_output_style_section_with_config(self):
        config = OutputStyleConfig(
            name="TestStyle",
            description="A test style",
            prompt="Be concise.",
        )
        result = get_output_style_section(config)
        assert result is not None
        assert "# Output Style: TestStyle" in result
        assert "Be concise." in result
    def test_scratchpad_none_when_no_dir(self):
        assert get_scratchpad_instructions(None) is None
        assert get_scratchpad_instructions("") is None
    def test_scratchpad_with_dir(self):
        result = get_scratchpad_instructions("/tmp/session-123")
        assert result is not None
        assert "/tmp/session-123" in result
        assert "# Scratchpad Directory" in result
        assert "temporary files" in result
 # =========================================================================
 # Error IDs
 # =========================================================================
 class TestErrorIDs:
    def test_tool_use_summary_error(self):
        assert E_TOOL_USE_SUMMARY_GENERATION_FAILED == 344