# Paper: MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs

---
type: paper
title: "MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs"
authors: Junwei Liao, Haoting Shi, Ruiwen Zhou, Jiaqian Wang, Shengtao Zhang, Wei Zhang, Ying Wen, Zhiyu Li, et al.
year: 2026
venue: arXiv
url: https://arxiv.org/abs/2605.08374
code_url:
source: arxiv
collected_at: 2026-07-08
published_at: 2026-05-08
updated_at: 2026-05-14
status: skimmed
relevance: high
topics:
  - agent-evaluation
  - computer-use
  - embodied-agent
  - memory
  - rag
  - reasoning
  - tool-use
methods:
  - provenance-dag
  - td-lambda
  - q-guided-retrieval
benchmarks:
  - LLAB
  - LiveCodeBench
  - MMMU-Pro
  - ERQA
  - GPQA-Diamond
  - BFCL
models:
  -
datasets:
  - cs.AI
related_concepts:
  - memory-credit-assignment
  - procedural-memory
related_jobs:
  -
related_experiments:
  - KC-001-agent-memory-pilot
related_projects:
  - learning/agent-memory
collection_score: 20
collection_queries: function-calling
---

## One-line Takeaway

Memory 的长期价值应沿“哪些旧记忆帮助生成了新记忆”的 provenance DAG 反向归因，而不只奖励最后一次检索。

## Pilot Skim

- problem: 单步 memory value update 忽略跨多次任务形成的经验依赖链。
- method: 用 TD(lambda) 沿 provenance DAG 回传 credit，结构深度代替时间步距离。
- evidence: 六个 benchmark 上领先或并列；相对 MemRL 的收益在深链任务最高达到 5.7pp，单步任务最低约 0.77pp。
- boundary: 假设 memory 单调增长，未解决 consolidation/deletion；DAG 存储和 BFS 更新有持续成本。

## Used In

- [Agent Memory evidence matrix](../../learning/agent-memory/evidence-matrix.md)

## Links

- arXiv: https://arxiv.org/abs/2605.08374