wuyang
64 lines
1.8 KiB
Markdown
64 lines
1.8 KiB
Markdown
# Paper: Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents
|
|
|
|
---
|
|
type: paper
|
|
title: Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents
|
|
authors: Sina Hajimiri, Masih Aminbeidokhti, Jose Dolz, Ismail Ben Ayed, Issam H. Laradji, Spandana Gella, Nicolas Gontier
|
|
year: 2026
|
|
venue: arXiv
|
|
url: https://arxiv.org/abs/2606.15017
|
|
code_url:
|
|
source: arxiv
|
|
collected_at: 2026-07-08
|
|
published_at: 2026-06-12
|
|
updated_at: 2026-06-12
|
|
status: skimmed
|
|
relevance: high
|
|
topics:
|
|
- agent-evaluation
|
|
- computer-use
|
|
- memory
|
|
- reasoning
|
|
- workflow-agent
|
|
methods:
|
|
- budget-matched-baseline
|
|
- online-augmentation-audit
|
|
benchmarks:
|
|
- WebArena
|
|
- WorkArena-L1
|
|
models:
|
|
-
|
|
datasets:
|
|
- cs.CL
|
|
related_concepts:
|
|
- memory-cost-accounting
|
|
- evaluation-hygiene
|
|
related_jobs:
|
|
-
|
|
related_experiments:
|
|
- KC-001-agent-memory-pilot
|
|
related_projects:
|
|
- learning/agent-memory
|
|
collection_score: 14
|
|
collection_queries: web-gui-agent
|
|
---
|
|
|
|
## One-line Takeaway
|
|
|
|
在线 memory 或 skill 的收益必须与“把相同推理预算直接交给基础 Agent”比较，不能只对比相同步数。
|
|
|
|
## Pilot Skim
|
|
|
|
- problem: augmentation 论文常忽略 induction、retrieval、verification 和 prompt injection 的额外 token。
|
|
- method: 用 Vanilla-IB 把近似相同预算用于更多 observe-act steps，并报告多 run 方差。
|
|
- evidence: WebArena 三域三模型中，Vanilla-IB 平均 SR 均最高且通常 token 更少；WorkArena-L1 上仍处于 success-cost frontier。
|
|
- boundary: 研究对象是 online augmentation；可离线摊销的 skill/memory 需要另一种成本模型。
|
|
|
|
## Used In
|
|
|
|
- [Agent Memory evidence matrix](../../learning/agent-memory/evidence-matrix.md)
|
|
|
|
## Links
|
|
|
|
- arXiv: https://arxiv.org/abs/2606.15017
|