agent/papers/items/2026-2605-08374-memq-integrating-q-learning-into-self-evolving-memory-agents-over-provenance-dag.md

Paper: MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs

---

type: paper title: "MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs" authors: Junwei Liao, Haoting Shi, Ruiwen Zhou, Jiaqian Wang, Shengtao Zhang, Wei Zhang, Ying Wen, Zhiyu Li, et al. year: 2026 venue: arXiv url: https://arxiv.org/abs/2605.08374 code_url: source: arxiv collected_at: 2026-07-08 published_at: 2026-05-08 updated_at: 2026-05-14 status: skimmed relevance: high topics:

• agent-evaluation
• computer-use
• embodied-agent
• memory
• rag
• reasoning
• tool-use methods:
• provenance-dag
• td-lambda
• q-guided-retrieval benchmarks:
• LLAB
• LiveCodeBench
• MMMU-Pro
• ERQA
• GPQA-Diamond
• BFCL models:

datasets:

• cs.AI related_concepts:
• memory-credit-assignment
• procedural-memory related_jobs:

related_experiments:

• KC-001-agent-memory-pilot related_projects:
• learning/agent-memory collection_score: 20 collection_queries: function-calling

---

One-line Takeaway

Memory 的长期价值应沿“哪些旧记忆帮助生成了新记忆”的 provenance DAG 反向归因，而不只奖励最后一次检索。

Pilot Skim

• problem: 单步 memory value update 忽略跨多次任务形成的经验依赖链。
• method: 用 TD(lambda) 沿 provenance DAG 回传 credit，结构深度代替时间步距离。
• evidence: 六个 benchmark 上领先或并列；相对 MemRL 的收益在深链任务最高达到 5.7pp，单步任务最低约 0.77pp。
• boundary: 假设 memory 单调增长，未解决 consolidation/deletion；DAG 存储和 BFS 更新有持续成本。

Used In

• Agent Memory evidence matrix

Links

• arXiv: https://arxiv.org/abs/2605.08374