Collect first agent research and market sources

papers/paper-insights.md

@@ -6,35 +6,50 @@ status: seed
 
 ## Current Snapshot
 
-- date:
-- scope:
-- papers reviewed:
+- date: 2026-07-08
+- scope: Agent evaluation, memory, coding-agent benchmarks
+- papers reviewed: 6 starter items
+- caveat: sources are provisional; source approval workflow still needs user confirmation.
 
 ## Strong Signals
 
--
+- Agent safety/eval 正在转向真实工具、多轮任务、trajectory、失败定位和系统级 guardrail。
+- Memory 不能只看事实召回；更重要的是是否改善流程执行和跨任务决策。
+- Coding-agent benchmark 正在从单 issue patch 转向长程软件演进和人机对话。
+- JD 高频词和论文趋势高度重合：memory、eval、planning/tool use、coding agent。
 
 ## Weak or Unproven Claims
 
--
+- “加 memory 就更强”不成立；EvoMemBench 显示长上下文 baseline 很强，memory 价值依赖任务。
+- 单一 safety benchmark 不足以说明 Agent 安全；不同 benchmark 的方法差异可能改变结论。
+- SWE-Bench 类单点任务不足以代表真实 coding agent 生产力。
 
 ## Mature Directions
 
--
+- Agent evaluation harness
+- Memory benchmark and memory architecture comparison
+- Coding agent benchmark beyond SWE-Bench
+- Agent failure diagnosis and trajectory analysis
 
 ## Emerging Directions
 
--
+- Dialogue-driven coding agents
+- Stateful enterprise task benchmarks
+- Agent safety benchmark taxonomy
+- Skill/prompt optimization as trainable artifacts
 
 ## Experiment Candidates
 
 | Idea | Related Papers | Metric | Notes |
 | --- | --- | --- | --- |
-| _TBD_ | _TBD_ | _TBD_ | _TBD_ |
+| Mini Agent Safety Suite | OpenAgentSafety, Agent safety benchmark taxonomy | unsafe action rate, refusal precision, tool misuse | 选 10-20 个本地工具任务先跑 |
+| Memory vs Long Context | Memory survey, EvoMemBench, STATE-Bench | success rate, token cost, latency, contradiction rate | 必须包含 long-context baseline |
+| Long-horizon Coding Eval | SWE-EVO, Dialogue-SWEBench | task success, fix rate, clarification quality | 从真实 repo release notes 构造小任务 |
+| Failure Step Diagnosis | AgentRx, OpenAgentSafety | critical failure step accuracy | 需要统一 trace schema |
 
 ## Links Back to Knowledge Base
 
-- docs to update:
-- experiments to create:
-- projects affected:
-- job skills connected:
+- docs to update: evaluation, memory, coding-agent, safety
+- experiments to create: memory comparison, mini safety suite, coding-agent eval
+- projects affected: future front-end/indexing project, coding-agent experiments
+- job skills connected: agent-evaluation, memory, coding-agent, tool-use, planning