Collect first agent research and market sources
This commit is contained in:
wuyang
@@ -0,0 +1,64 @@
|
||||
# Paper: OpenAgentSafety
|
||||
|
||||
---
|
||||
type: paper
|
||||
title: OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety
|
||||
authors: Sanidhya Vijayvargiya, Aditya Bharat Soni, Xuhui Zhou, Zora Zhiruo Wang, Nouha Dziri, Graham Neubig, Maarten Sap
|
||||
year: 2025
|
||||
venue: ICLR 2026 / IASEAI 2026
|
||||
url: https://arxiv.org/abs/2507.06134
|
||||
code_url:
|
||||
source: arxiv
|
||||
collected_at: 2026-07-08
|
||||
status: skimmed
|
||||
relevance: high
|
||||
topics:
|
||||
- agent-evaluation
|
||||
- agent-safety
|
||||
- tool-use
|
||||
methods:
|
||||
- multi-turn-agent-evaluation
|
||||
- rule-based-analysis
|
||||
- llm-as-judge
|
||||
benchmarks:
|
||||
- OpenAgentSafety
|
||||
models:
|
||||
-
|
||||
datasets:
|
||||
-
|
||||
related_concepts:
|
||||
- guardrail
|
||||
- human-in-the-loop
|
||||
related_jobs:
|
||||
- 2026-07-08-baidu-aidu-agent-algorithm-engineer-beijing
|
||||
- 2026-07-08-tencent-cloud-ai-agent-test-engineer
|
||||
related_experiments:
|
||||
-
|
||||
related_projects:
|
||||
-
|
||||
---
|
||||
|
||||
## One-line Takeaway
|
||||
|
||||
Agent safety 评估正在从简单拒答测试转向真实工具、多用户、多轮任务和可扩展评估框架。
|
||||
|
||||
## Problem
|
||||
|
||||
真实 Agent 会调用浏览器、文件系统、shell、消息平台和代码执行环境，传统安全 benchmark 对工具、任务和对抗场景覆盖不足。
|
||||
|
||||
## Core Idea
|
||||
|
||||
OpenAgentSafety 提供模块化框架，用真实工具和多轮任务评估 Agent 在八类风险下的行为，并结合规则分析和 LLM-as-judge。
|
||||
|
||||
## Evidence
|
||||
|
||||
论文报告覆盖 350+ 多轮、多用户任务，并评估五个 LLM 在 agentic 场景下的安全表现。
|
||||
|
||||
## Useful For Us
|
||||
|
||||
- 可作为 Agent eval harness 的安全维度参考。
|
||||
- 与 JD 中的 Agent 评测体系、成功率、稳定性、安全边界直接相关。
|
||||
|
||||
## Follow-up Experiments
|
||||
|
||||
- 选 10 个高风险工具任务，建立一个本地 mini OpenAgentSafety 风格测试集。
|
||||
@@ -0,0 +1,61 @@
|
||||
# Paper: Agent Safety Benchmark Taxonomy
|
||||
|
||||
---
|
||||
type: paper
|
||||
title: "Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents"
|
||||
authors:
|
||||
year: 2026
|
||||
venue:
|
||||
url: https://arxiv.org/html/2605.16282v1
|
||||
code_url:
|
||||
source: arxiv
|
||||
collected_at: 2026-07-08
|
||||
status: queued
|
||||
relevance: medium
|
||||
topics:
|
||||
- agent-safety
|
||||
- agent-evaluation
|
||||
methods:
|
||||
- benchmark-taxonomy
|
||||
- coverage-matrix
|
||||
benchmarks:
|
||||
- agent-safety-benchmarks
|
||||
models:
|
||||
-
|
||||
datasets:
|
||||
-
|
||||
related_concepts:
|
||||
- guardrail
|
||||
- red-teaming
|
||||
related_jobs:
|
||||
- 2026-07-08-tencent-cloud-ai-agent-test-engineer
|
||||
related_experiments:
|
||||
-
|
||||
related_projects:
|
||||
-
|
||||
---
|
||||
|
||||
## One-line Takeaway
|
||||
|
||||
Agent safety benchmark 还没有收敛，做评估时不能只依赖单一 benchmark。
|
||||
|
||||
## Problem
|
||||
|
||||
不同安全 benchmark 的风险覆盖、任务环境、评估方法和结论可能不一致。
|
||||
|
||||
## Core Idea
|
||||
|
||||
该论文整理 2023-2026 年的 behavioral agent-safety benchmarks，并用六轴 taxonomy 和 coverage matrix 分析方法选择如何影响结论。
|
||||
|
||||
## Evidence
|
||||
|
||||
搜索摘要显示论文 catalog 了 40 个 behavioral agent-safety benchmarks 和 5 个相邻 evaluator/defense/dataset artifact。
|
||||
|
||||
## Useful For Us
|
||||
|
||||
- 适合作为选择 Agent safety benchmark 的二级阅读材料。
|
||||
- 与 OpenAgentSafety、AgentRx、腾讯 Agent 测试岗位形成交叉验证。
|
||||
|
||||
## Follow-up Experiments
|
||||
|
||||
- 根据 taxonomy 给知识库建立自己的 Agent evaluation source map。
|
||||
@@ -0,0 +1,63 @@
|
||||
# Paper: Dialogue-SWEBench
|
||||
|
||||
---
|
||||
type: paper
|
||||
title: "Dialogue-SWEBench: A Benchmark for Dialogue-Driven Coding Agents"
|
||||
authors: Brendan King, Jeffrey Flanigan
|
||||
year: 2026
|
||||
venue:
|
||||
url: https://arxiv.org/html/2606.13995v1
|
||||
code_url: https://jlab-nlp.github.io/dialogue-swe-bench/
|
||||
source: arxiv
|
||||
collected_at: 2026-07-08
|
||||
status: skimmed
|
||||
relevance: high
|
||||
topics:
|
||||
- coding-agent
|
||||
- human-in-the-loop
|
||||
- agent-evaluation
|
||||
methods:
|
||||
- user-simulator
|
||||
- dialogue-benchmark
|
||||
- schema-guided-agent
|
||||
benchmarks:
|
||||
- Dialogue-SWEBench
|
||||
models:
|
||||
-
|
||||
datasets:
|
||||
- SWE-Bench Verified
|
||||
related_concepts:
|
||||
- human-in-the-loop
|
||||
- coding-agent
|
||||
related_jobs:
|
||||
- 2026-07-08-baidu-aidu-agent-fullstack-engineer-beijing
|
||||
related_experiments:
|
||||
-
|
||||
related_projects:
|
||||
-
|
||||
---
|
||||
|
||||
## One-line Takeaway
|
||||
|
||||
Coding agent 的真实能力不能只看自动修复，还要看它能不能通过对话澄清需求。
|
||||
|
||||
## Problem
|
||||
|
||||
现实中的 coding assistant 经常和用户交互，但主流 benchmark 多把它们评估为全自动系统。
|
||||
|
||||
## Core Idea
|
||||
|
||||
Dialogue-SWEBench 使用 persona-grounded user simulator，把 SWE task 改造成多轮对话评估，并提出 schema-guided agent 改善对话能力。
|
||||
|
||||
## Evidence
|
||||
|
||||
论文报告 schema-guided agent 相比强 baseline 提升 3-14%，并指出更强 coding model 不一定是更强 dialogue agent。
|
||||
|
||||
## Useful For Us
|
||||
|
||||
- 对“Agent 何时应该提问”很有启发。
|
||||
- 可以作为 coding-agent / human-in-the-loop 评估维度。
|
||||
|
||||
## Follow-up Experiments
|
||||
|
||||
- 给本知识库的 coding-agent 项目设计“澄清问题质量”评估。
|
||||
@@ -0,0 +1,65 @@
|
||||
# Paper: Memory for Autonomous LLM Agents
|
||||
|
||||
---
|
||||
type: paper
|
||||
title: "Memory for Autonomous LLM Agents: Mechanisms, Evaluation, and Emerging Frontiers"
|
||||
authors:
|
||||
year: 2026
|
||||
venue:
|
||||
url: https://arxiv.org/html/2603.07670v1
|
||||
code_url:
|
||||
source: arxiv
|
||||
collected_at: 2026-07-08
|
||||
status: skimmed
|
||||
relevance: high
|
||||
topics:
|
||||
- memory
|
||||
- agent-architecture
|
||||
- agent-evaluation
|
||||
methods:
|
||||
- write-manage-read-loop
|
||||
- retrieval-augmented-memory
|
||||
- reflective-memory
|
||||
- hierarchical-context
|
||||
benchmarks:
|
||||
-
|
||||
models:
|
||||
-
|
||||
datasets:
|
||||
-
|
||||
related_concepts:
|
||||
- memory
|
||||
- context-engineering
|
||||
related_jobs:
|
||||
- 2026-07-08-baidu-aidu-agent-algorithm-engineer-beijing
|
||||
- 2026-07-08-bytedance-seed-llm-agent-research-engineer
|
||||
related_experiments:
|
||||
-
|
||||
related_projects:
|
||||
-
|
||||
---
|
||||
|
||||
## One-line Takeaway
|
||||
|
||||
Memory 应该作为 Agent 系统的一等工程组件，而不是 prompt 里临时塞历史记录。
|
||||
|
||||
## Problem
|
||||
|
||||
长任务、多 session 和个性化场景中，单一上下文窗口无法保存并正确复用历史经验。
|
||||
|
||||
## Core Idea
|
||||
|
||||
该综述把 Agent memory 形式化为 write-manage-read loop，并从时间范围、表示基底和控制策略三个维度整理机制。
|
||||
|
||||
## Evidence
|
||||
|
||||
论文覆盖 2022 到 2026 年初的 memory 机制和评估进展，并总结从静态 recall 到多 session agentic benchmark 的转变。
|
||||
|
||||
## Useful For Us
|
||||
|
||||
- 用来设计知识库自己的 memory / context 管理专题。
|
||||
- 与百度和字节 JD 中的长期记忆、context compression、agent harness 强相关。
|
||||
|
||||
## Follow-up Experiments
|
||||
|
||||
- 用同一组任务比较 raw history、summary memory、retrieval memory 和 procedural memory。
|
||||
@@ -0,0 +1,65 @@
|
||||
# Paper: SWE-EVO
|
||||
|
||||
---
|
||||
type: paper
|
||||
title: "SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios"
|
||||
authors: Minh Vu Thai Pham, Tue Le, Dung Nguyen Manh, Huy Nhat Phan, Nghi D. Q. Bui
|
||||
year: 2026
|
||||
venue:
|
||||
url: https://arxiv.org/html/2512.18470v5
|
||||
code_url: https://github.com/SWE-EVO/SWE-EVO
|
||||
source: arxiv
|
||||
collected_at: 2026-07-08
|
||||
status: skimmed
|
||||
relevance: high
|
||||
topics:
|
||||
- coding-agent
|
||||
- agent-evaluation
|
||||
methods:
|
||||
- long-horizon-software-evolution
|
||||
- benchmark
|
||||
benchmarks:
|
||||
- SWE-EVO
|
||||
models:
|
||||
- OpenAI
|
||||
- DeepSeek
|
||||
- Zhipu
|
||||
- Qwen
|
||||
- Moonshot
|
||||
datasets:
|
||||
- release-notes
|
||||
related_concepts:
|
||||
- coding-agent
|
||||
- long-horizon-task
|
||||
related_jobs:
|
||||
- 2026-07-08-deepseek-agent-hiring-wave-beijing-hangzhou
|
||||
related_experiments:
|
||||
-
|
||||
related_projects:
|
||||
-
|
||||
---
|
||||
|
||||
## One-line Takeaway
|
||||
|
||||
Coding agent 评估正在从单 issue 修 bug 转向多文件、多 PR、版本演进式长任务。
|
||||
|
||||
## Problem
|
||||
|
||||
SWE-Bench 类任务偏单点 issue，而真实工程经常要求根据 release notes 进行系统性版本演进。
|
||||
|
||||
## Core Idea
|
||||
|
||||
SWE-EVO 从成熟 Python 项目的 release notes 构造 48 个长程任务，平均涉及 21 个文件，并提出 Fix Rate 捕捉部分进展。
|
||||
|
||||
## Evidence
|
||||
|
||||
论文显示强模型在 SWE-EVO 上的解决率远低于 SWE-Bench Verified，失败更多来自语义理解和长期指令跟随，而不是简单工具操作。
|
||||
|
||||
## Useful For Us
|
||||
|
||||
- 适合指导 coding-agent 项目的内部评测集设计。
|
||||
- 和 DeepSeek JD 中的 AI coding tool 使用、Agent infra、评测闭环相关。
|
||||
|
||||
## Follow-up Experiments
|
||||
|
||||
- 选一个开源 Python 项目，用 release notes 构造小规模 SWE-EVO 风格任务。
|
||||
@@ -0,0 +1,62 @@
|
||||
# Paper: EvoMemBench
|
||||
|
||||
---
|
||||
type: paper
|
||||
title: "EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective"
|
||||
authors: Yuyao Wang, Zhongjian Zhang, Mo Chi, Kaichi Yu, Yuhan Li, Miao Peng, Bing Tong, Chen Zhang, Yan Zhou, Jia Li
|
||||
year: 2026
|
||||
venue:
|
||||
url: https://arxiv.org/html/2605.18421
|
||||
code_url:
|
||||
source: arxiv
|
||||
collected_at: 2026-07-08
|
||||
status: skimmed
|
||||
relevance: high
|
||||
topics:
|
||||
- memory
|
||||
- agent-evaluation
|
||||
methods:
|
||||
- memory-benchmark
|
||||
- self-evolving-agent
|
||||
benchmarks:
|
||||
- EvoMemBench
|
||||
models:
|
||||
-
|
||||
datasets:
|
||||
-
|
||||
related_concepts:
|
||||
- memory
|
||||
- procedural-memory
|
||||
related_jobs:
|
||||
- 2026-07-08-baidu-aidu-agent-algorithm-engineer-beijing
|
||||
- 2026-07-08-bytedance-seed-llm-agent-research-engineer
|
||||
related_experiments:
|
||||
-
|
||||
related_projects:
|
||||
-
|
||||
---
|
||||
|
||||
## One-line Takeaway
|
||||
|
||||
Memory 的价值取决于任务类型和上下文约束；长上下文 baseline 仍然很强，不能假设外部记忆天然更好。
|
||||
|
||||
## Problem
|
||||
|
||||
现有 Agent benchmark 主要评估 reasoning、planning、execution，对 memory 的更新、修正和复用能力覆盖不足。
|
||||
|
||||
## Core Idea
|
||||
|
||||
EvoMemBench 按 memory scope 和 memory content 两条轴设计 benchmark，比较 15 种 memory 方法和长上下文 baseline。
|
||||
|
||||
## Evidence
|
||||
|
||||
结果显示 memory 在当前上下文不足或任务较难时更有价值；知识密集任务更适合 retrieval memory，执行导向任务更依赖 procedural/long-term memory。
|
||||
|
||||
## Useful For Us
|
||||
|
||||
- 设计 memory 实验时要加入长上下文 baseline。
|
||||
- 不同岗位里提到 memory 时，要进一步区分事实记忆、过程记忆和跨任务经验。
|
||||
|
||||
## Follow-up Experiments
|
||||
|
||||
- 用 Agent 项目任务集做 memory-vs-long-context 对照实验。
|
||||
+27
-12
@@ -6,35 +6,50 @@ status: seed
|
||||
|
||||
## Current Snapshot
|
||||
|
||||
- date:
|
||||
- scope:
|
||||
- papers reviewed:
|
||||
- date: 2026-07-08
|
||||
- scope: Agent evaluation, memory, coding-agent benchmarks
|
||||
- papers reviewed: 6 starter items
|
||||
- caveat: sources are provisional; source approval workflow still needs user confirmation.
|
||||
|
||||
## Strong Signals
|
||||
|
||||
-
|
||||
- Agent safety/eval 正在转向真实工具、多轮任务、trajectory、失败定位和系统级 guardrail。
|
||||
- Memory 不能只看事实召回；更重要的是是否改善流程执行和跨任务决策。
|
||||
- Coding-agent benchmark 正在从单 issue patch 转向长程软件演进和人机对话。
|
||||
- JD 高频词和论文趋势高度重合：memory、eval、planning/tool use、coding agent。
|
||||
|
||||
## Weak or Unproven Claims
|
||||
|
||||
-
|
||||
- “加 memory 就更强”不成立；EvoMemBench 显示长上下文 baseline 很强，memory 价值依赖任务。
|
||||
- 单一 safety benchmark 不足以说明 Agent 安全；不同 benchmark 的方法差异可能改变结论。
|
||||
- SWE-Bench 类单点任务不足以代表真实 coding agent 生产力。
|
||||
|
||||
## Mature Directions
|
||||
|
||||
-
|
||||
- Agent evaluation harness
|
||||
- Memory benchmark and memory architecture comparison
|
||||
- Coding agent benchmark beyond SWE-Bench
|
||||
- Agent failure diagnosis and trajectory analysis
|
||||
|
||||
## Emerging Directions
|
||||
|
||||
-
|
||||
- Dialogue-driven coding agents
|
||||
- Stateful enterprise task benchmarks
|
||||
- Agent safety benchmark taxonomy
|
||||
- Skill/prompt optimization as trainable artifacts
|
||||
|
||||
## Experiment Candidates
|
||||
|
||||
| Idea | Related Papers | Metric | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| _TBD_ | _TBD_ | _TBD_ | _TBD_ |
|
||||
| Mini Agent Safety Suite | OpenAgentSafety, Agent safety benchmark taxonomy | unsafe action rate, refusal precision, tool misuse | 选 10-20 个本地工具任务先跑 |
|
||||
| Memory vs Long Context | Memory survey, EvoMemBench, STATE-Bench | success rate, token cost, latency, contradiction rate | 必须包含 long-context baseline |
|
||||
| Long-horizon Coding Eval | SWE-EVO, Dialogue-SWEBench | task success, fix rate, clarification quality | 从真实 repo release notes 构造小任务 |
|
||||
| Failure Step Diagnosis | AgentRx, OpenAgentSafety | critical failure step accuracy | 需要统一 trace schema |
|
||||
|
||||
## Links Back to Knowledge Base
|
||||
|
||||
- docs to update:
|
||||
- experiments to create:
|
||||
- projects affected:
|
||||
- job skills connected:
|
||||
- docs to update: evaluation, memory, coding-agent, safety
|
||||
- experiments to create: memory comparison, mini safety suite, coding-agent eval
|
||||
- projects affected: future front-end/indexing project, coding-agent experiments
|
||||
- job skills connected: agent-evaluation, memory, coding-agent, tool-use, planning
|
||||
|
||||
@@ -4,7 +4,12 @@ status: seed
|
||||
|
||||
| Priority | Paper | Topic | Reason | Status | Link |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| _TBD_ | _TBD_ | _TBD_ | _TBD_ | queued | _TBD_ |
|
||||
| P0 | OpenAgentSafety | agent safety / evaluation | 真实工具、多轮、多用户安全评估，直接对应 Agent eval 需求 | skimmed | https://arxiv.org/abs/2507.06134 |
|
||||
| P0 | Memory for Autonomous LLM Agents | memory | 综述 memory 机制、评估和工程挑战，对 JD 高频 memory/context 信号有解释力 | skimmed | https://arxiv.org/html/2603.07670v1 |
|
||||
| P0 | EvoMemBench | memory benchmark | 说明 memory 不总是优于长上下文，适合作为实验设计依据 | skimmed | https://arxiv.org/html/2605.18421 |
|
||||
| P0 | SWE-EVO | coding agent benchmark | 长程软件演进任务比 SWE-Bench 更接近真实工程 | skimmed | https://arxiv.org/html/2512.18470v5 |
|
||||
| P1 | Dialogue-SWEBench | human-in-the-loop coding agent | 评估 coding agent 澄清需求能力，补足全自动 benchmark 盲区 | skimmed | https://arxiv.org/html/2606.13995v1 |
|
||||
| P1 | Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents | safety benchmark taxonomy | 用于选择和比较 Agent safety benchmark | queued | https://arxiv.org/html/2605.16282v1 |
|
||||
|
||||
## Priority Rules
|
||||
|
||||
|
||||
@@ -8,13 +8,13 @@
|
||||
|
||||
| Source | Status | Notes |
|
||||
| --- | --- | --- |
|
||||
| arXiv | candidate | 新论文发现和预印本 |
|
||||
| arXiv | provisional-used | 新论文发现和预印本；first sweep 已使用，待用户正式确认 |
|
||||
| OpenReview | candidate | ICLR、NeurIPS 等投稿和评审线索 |
|
||||
| ACL Anthology | candidate | NLP、RAG、tool use、evaluation 相关论文 |
|
||||
| Conference proceedings | candidate | NeurIPS、ICLR、ICML、ACL、EMNLP、KDD、WWW 等 |
|
||||
| Papers with Code | candidate | benchmark、代码和任务关联 |
|
||||
| Official project GitHub | candidate | 复现代码、模型卡和实验配置 |
|
||||
| Company research blogs | candidate | 只收技术细节足够强的研究内容 |
|
||||
| Official project GitHub | provisional-used | 复现代码、模型卡和实验配置；first sweep 已作为 code_url 使用 |
|
||||
| Company research blogs | provisional-used | 只收技术细节足够强的研究内容；first sweep 已在 industry 中使用 |
|
||||
|
||||
## Source Types
|
||||
|
||||
|
||||
Reference in New Issue
Block a user