wuyang
1.8 KiB
1.8 KiB
Paper: Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents
type: paper title: Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents authors: Sina Hajimiri, Masih Aminbeidokhti, Jose Dolz, Ismail Ben Ayed, Issam H. Laradji, Spandana Gella, Nicolas Gontier year: 2026 venue: arXiv url: https://arxiv.org/abs/2606.15017 code_url: source: arxiv collected_at: 2026-07-08 published_at: 2026-06-12 updated_at: 2026-06-12 status: skimmed relevance: high topics:
- agent-evaluation
- computer-use
- memory
- reasoning
- workflow-agent methods:
- budget-matched-baseline
- online-augmentation-audit benchmarks:
- WebArena
- WorkArena-L1 models:
datasets:
- cs.CL related_concepts:
- memory-cost-accounting
- evaluation-hygiene related_jobs:
related_experiments:
- KC-001-agent-memory-pilot related_projects:
- learning/agent-memory collection_score: 14 collection_queries: web-gui-agent
One-line Takeaway
在线 memory 或 skill 的收益必须与“把相同推理预算直接交给基础 Agent”比较，不能只对比相同步数。
Pilot Skim
- problem: augmentation 论文常忽略 induction、retrieval、verification 和 prompt injection 的额外 token。
- method: 用 Vanilla-IB 把近似相同预算用于更多 observe-act steps，并报告多 run 方差。
- evidence: WebArena 三域三模型中，Vanilla-IB 平均 SR 均最高且通常 token 更少；WorkArena-L1 上仍处于 success-cost frontier。
- boundary: 研究对象是 online augmentation；可离线摊销的 skill/memory 需要另一种成本模型。