wuyang
71 lines
1.8 KiB
Markdown
71 lines
1.8 KiB
Markdown
# Paper: MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs
|
||
|
||
---
|
||
type: paper
|
||
title: "MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs"
|
||
authors: Junwei Liao, Haoting Shi, Ruiwen Zhou, Jiaqian Wang, Shengtao Zhang, Wei Zhang, Ying Wen, Zhiyu Li, et al.
|
||
year: 2026
|
||
venue: arXiv
|
||
url: https://arxiv.org/abs/2605.08374
|
||
code_url:
|
||
source: arxiv
|
||
collected_at: 2026-07-08
|
||
published_at: 2026-05-08
|
||
updated_at: 2026-05-14
|
||
status: skimmed
|
||
relevance: high
|
||
topics:
|
||
- agent-evaluation
|
||
- computer-use
|
||
- embodied-agent
|
||
- memory
|
||
- rag
|
||
- reasoning
|
||
- tool-use
|
||
methods:
|
||
- provenance-dag
|
||
- td-lambda
|
||
- q-guided-retrieval
|
||
benchmarks:
|
||
- LLAB
|
||
- LiveCodeBench
|
||
- MMMU-Pro
|
||
- ERQA
|
||
- GPQA-Diamond
|
||
- BFCL
|
||
models:
|
||
-
|
||
datasets:
|
||
- cs.AI
|
||
related_concepts:
|
||
- memory-credit-assignment
|
||
- procedural-memory
|
||
related_jobs:
|
||
-
|
||
related_experiments:
|
||
- KC-001-agent-memory-pilot
|
||
related_projects:
|
||
- learning/agent-memory
|
||
collection_score: 20
|
||
collection_queries: function-calling
|
||
---
|
||
|
||
## One-line Takeaway
|
||
|
||
Memory 的长期价值应沿“哪些旧记忆帮助生成了新记忆”的 provenance DAG 反向归因，而不只奖励最后一次检索。
|
||
|
||
## Pilot Skim
|
||
|
||
- problem: 单步 memory value update 忽略跨多次任务形成的经验依赖链。
|
||
- method: 用 TD(lambda) 沿 provenance DAG 回传 credit，结构深度代替时间步距离。
|
||
- evidence: 六个 benchmark 上领先或并列；相对 MemRL 的收益在深链任务最高达到 5.7pp，单步任务最低约 0.77pp。
|
||
- boundary: 假设 memory 单调增长，未解决 consolidation/deletion；DAG 存储和 BFS 更新有持续成本。
|
||
|
||
## Used In
|
||
|
||
- [Agent Memory evidence matrix](../../learning/agent-memory/evidence-matrix.md)
|
||
|
||
## Links
|
||
|
||
- arXiv: https://arxiv.org/abs/2605.08374
|