feat: execute DeepSeek absorbed MLA cache

2026-07-29 14:57:06 +08:00
parent ab19381046
commit 9ba26da545
10 changed files with 2290 additions and 32 deletions
@@ -14,7 +14,7 @@
 | 表示、位置与残差高速公路 | 完成首版 | 81% | 真实 hidden-state / norm traces、长上下文位置外推与深层稳定性消融 |
 | Scaling Laws | 完成首版 | 74% | 真实拟合复现、置信区间与更多模型族对照 |
 | 数据工程与预训练配方 | 完成首版 | 73% | FineWeb / DCLM 逐图精读、真实去重误伤与 mixture traces |
-| DeepSeek 专题 | 三轮实证进行中 | 87% | 真实 latent-cache kernel、更大样本负载、FP8/pipeline 与 R1-like RL 复现 |
+| DeepSeek 专题 | 三轮实证进行中 | 89% | SM90 FlashMLA 优化 kernel、更大样本负载、FP8/pipeline 与 R1-like RL 复现 |
 | 指令微调与人类偏好 | 完成首版 | 75% | 真实偏好分歧、RM 长度偏置与 PPO/DPO 小模型复现 |
 | 推理与测试时扩展 | 完成首版 | 76% | 真实模型采样曲线、PRM 案例与逐篇图表精读 |
 | 工具使用与长程 Agent | 完成首版 | 74% | 真实环境 traces、cross-harness 对照、Agent RL 曲线与安全案例 |
@@ -41,7 +41,7 @@
 - [x] 完成 486 篇关键论文索引，覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。
 - [x] 完成可检索、可按专题筛选的论文库页面。
 - [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。
- [x] 完成 K3 三轴架构、八联报告实验与四联开放工件实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 八联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等七十五个原创交互视图。
+- [x] 完成 K3 三轴架构、八联报告实验与四联开放工件实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 九联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等七十六个原创交互视图。
 - [x] 完成长上下文首版：五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
 - [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文，并建立长上下文研究账本。
 - [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文，并建立 MoE 研究账本。
@@ -174,11 +174,15 @@
 - [x] 独立复跑 31/31 exact：来源、配置、tokenization、逐层 hidden hashes、MLA shapes、aggregate loads 与全部 token routes 完全一致；计时明确不比较。
 - [x] DeepSeek 新四视图本地真实 Chrome 回归通过：逐 token 路由、覆盖/均衡分账、latent/HF eager cache 计算、27 层执行断面、键盘 tabs 与 390px 移动端均无异常。
 - [x] DeepSeek 三轮首个真实权重执行里程碑以源提交 `e864205`、不可变镜像 `20260729T062034Z-e864205` 发布；OCI digest `sha256:c3c37631…91addc`，NAS、VPS/Tailscale、NPM、DNS、HTTPS、证书、门户与十六套生产 Chrome 回归全链路通过；保留 `20260729T053852Z-2ef846f` 回滚。
+- [x] DeepSeek 真实吸收式缓存闭环：同一官方 V2-Lite layer-1 权重与真实 hidden state 依次执行 HF eager、V3 naive 与 V3 absorb；26-token BF16 active cache 从 266,240 B 降至 29,952 B，实际比率 8.8889× / 88.75%。
+- [x] 权重吸收正确性闸门通过：HF eager↔V3 naive BF16 max |Δ| 0.0012207，V3 naive↔absorb BF16 max |Δ| 0.00390625；同一 BF16 权重提升到 FP32 后 naive↔absorb max |Δ| 1.19e-7，所有输出 finite。
+- [x] 独立吸收实验复跑 byte-exact：两份完整 JSON SHA-256 均为 `6b4c714a…e63d`；实验脚本、机器可读结果与正式研究账本进入开源树。
+- [x] FlashMLA 架构边界闭合：固定 revision `15f13e50…a7772c6` 与 CUTLASS submodule；官方矩阵、gencode 与 dense runtime guard 只覆盖 SM90/SM100，本机 SM120 不宣称优化 kernel 已执行；两次隔离构建失败位置如实入账。

 ## 正在进行

 - [ ] K3 三轮下一闸门：获得真实 token hidden states、expert load 与 cache traces，解释或修订 `A_log [128]` 工件冲突，再做 Figure 3/4/5 数值重绘和独立小模型复现。
- [ ] DeepSeek 三轮下一闸门：真实 latent-cache kernel、更大样本 expert-load 置信区间、FP8 / pipeline traces 与 R1-like RL 小模型复现。
+- [ ] DeepSeek 三轮下一闸门：在官方支持的 SM90 环境执行 FlashMLA 优化 kernel、更大样本 expert-load 置信区间、FP8 / pipeline traces 与 R1-like RL 小模型复现。
 - [ ] 表示、位置与残差二轮：真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。
 - [ ] 评测安全二轮：真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。
 - [ ] 推理服务二轮：真实 GPU kernel / workload traces、功耗与成本、跨 vLLM / SGLang / TensorRT-LLM 复现。
@@ -298,6 +302,8 @@
 | 2026-07-29 | MLA 算法状态与框架物化格式分两张账 | V2-Lite latent 576 元素与 HF eager 5,120 元素同时报告；88.75% 不冒充 V2 论文 93.3% |
 | 2026-07-29 | 专家覆盖、负载均衡与语义永久分开 | 60–64/64 used 不推出 balanced；CV/Gini/effective 同报；expert ID 不跨层连线或命名 |
 | 2026-07-29 | DeepSeek 三轮首个真实权重版本以 `20260729T062034Z-e864205` 发布 | OCI digest `sha256:c3c37631…91addc`；复用 `12010→8080`、NPM 31 / cert 41、门户 order 180；保留 `20260729T053852Z-2ef846f` 回滚 |
+| 2026-07-29 | MLA 576 元素缓存从算法合同升级为真实执行 | 官方 V3 naive / absorb 使用同一 V2-Lite 权重与 hidden state；26-token active cache 266,240 B→29,952 B，FP32 代数审计 max \|Δ\| 1.19e-7 |
+| 2026-07-29 | reference absorb 与 FlashMLA optimized kernel 永久分身份 | 前者已在 RTX 5090 执行；pinned FlashMLA 只列 SM90/SM100 并只生成 sm_90a/sm_100f，本机 SM120 明确标为未支持/未执行 |
 | 2026-07-29 | K3 二轮按 32 张对象账与完整报告顺序重建 | total/active、2.5×、KDA state、深度来源、专家路由、视觉目标、轨迹、缓存与评测协议不再压成一页组件摘要 |
 | 2026-07-29 | K3 原生视觉事实回到 §2.4 / §3.3 核验 | 删除“先冻结语言模型再解冻”旧表述；明确 MoonViT-V2 从头训练，视觉/文本从开始共同 NTP |
 | 2026-07-29 | K3 Figure 1–16 / Table 1–5 全部建立课程视觉契约 | 每张图同时写支持范围与不可外推项；作者报告、论文、推导与 toy model 使用 R/P/D/T 标签 |
@@ -19,7 +19,7 @@

 当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读，
 语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化，以及评测与安全深度专题，
-以及 75 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
+以及 76 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
 8 个交互实验和 100 个一手/官方节点，完整覆盖架构、预训练、后训练、系统、评测、案例与附录。
 第三轮已完成开放工件与首个真实 kernel 里程碑：固定官方模型与 FlashKDA revisions，审计 96 个 checkpoint shards、
 497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计，并用 4 个新视图
@@ -28,10 +28,13 @@
 [K3_ARTIFACT_AUDIT.md](./research/K3_ARTIFACT_AUDIT.md) 与
 [checkpoint_probe.py](./experiments/k3/checkpoint_probe.py)、[FlashKDA probe](./experiments/k3/flashkda/)。
 DeepSeek 三轮专题以 24 张问题账、10 次技术转向、
-8 个交互实验和 60 个一手/官方节点，串起 Dense、MoE、MLA、V3 协同、R1 与 V4；
+9 个交互实验和 60 个一手/官方节点，串起 Dense、MoE、MLA、V3 协同、R1 与 V4；
 并固定官方 V2-Lite revision，在 RTX 5090 上连续执行 7/27 层，记录 3,240 次真实专家选择、
-MLA/HF eager cache shapes 与 `31/31` exact 独立复跑。详见
-[DEEPSEEK_V2_LITE_TRACE.md](./research/DEEPSEEK_V2_LITE_TRACE.md)。
+MLA/HF eager cache shapes 与 `31/31` exact 独立复跑；进一步用真实 layer-1 权重执行官方 V3
+naive/absorb 路径，实际写入 576 元素 latent cache，并以 FP32 将两种结合顺序的最大误差压到
+`1.19e-7`。FlashMLA 的 SM90/SM100 官方支持矩阵与本机 SM120 边界单独记账。详见
+[DEEPSEEK_V2_LITE_TRACE.md](./research/DEEPSEEK_V2_LITE_TRACE.md) 与
+[DEEPSEEK_MLA_ABSORB_AUDIT.md](./research/DEEPSEEK_MLA_ABSORB_AUDIT.md)。
 其余专题按进度账本持续扩建。

 ## 本地开发
@@ -0,0 +1,661 @@
+#!/usr/bin/env python3
+"""Execute DeepSeek MLA's naive and weight-absorbed cache paths with real weights.
+
+The probe deliberately separates three things:
+
+1. DeepSeek-V2-Lite's official Hugging Face eager attention;
+2. DeepSeek-V3's official pure-PyTorch ``naive`` and ``absorb`` reference paths;
+3. FlashMLA's optimized-kernel architecture boundary.
+
+The first official checkpoint shard contains the embedding, layer 0, and all
+layer-1 attention weights. We execute layer 0, normalize the real layer-1 input,
+map the unchanged V2-Lite attention tensors into the pinned V3 reference class,
+then perform a 25-token prefill plus one-token incremental decode.
+
+This is a correctness and cache-layout probe, not a serving benchmark.
+"""
+
+from __future__ import annotations
+
+import argparse
+import copy
+import gc
+import hashlib
+import importlib.util
+import json
+import math
+import platform
+import subprocess
+import sys
+import types
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+import torch
+import torch.nn.functional as F
+from safetensors import safe_open
+from transformers import AutoTokenizer, DynamicCache
+
+
+HF_REVISION = "604d5664dddd88a0433dbae533b7fe9472482de0"
+V3_REVISION = "9b4e9788e4a3a731f7567338ed15d3ec549ce03b"
+FLASHMLA_REVISION = "15f13e5030374295491c5ce31b02d7e63a7772c6"
+CUTLASS_REVISION = "147f5673d0c1c3dcf66f78d677fd647e4a020219"
+PROMPT = "用通俗的语言解释，为什么稀疏专家模型可以拥有很多参数，但每个 token 只使用其中一小部分。"
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--artifact-dir", type=Path, required=True)
+    parser.add_argument("--v3-repo", type=Path, required=True)
+    parser.add_argument("--flashmla-repo", type=Path, required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument("--device", default="cuda")
+    parser.add_argument("--cache-slots", type=int, default=32)
+    parser.add_argument("--captured-at", default=None)
+    return parser.parse_args()
+
+
+def sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def git_revision(root: Path) -> str:
+    return subprocess.check_output(
+        ["git", "-C", str(root), "rev-parse", "HEAD"], text=True
+    ).strip()
+
+
+def tensor_bytes(tensor: torch.Tensor) -> int:
+    return tensor.numel() * tensor.element_size()
+
+
+def compare(left: torch.Tensor, right: torch.Tensor) -> dict[str, Any]:
+    if left.shape != right.shape:
+        raise ValueError(f"shape mismatch: {left.shape} != {right.shape}")
+    delta = (left.float() - right.float()).abs().flatten()
+    return {
+        "shape": list(left.shape),
+        "finite": bool(torch.isfinite(left).all() and torch.isfinite(right).all()),
+        "exact_fraction": (left == right).float().mean().item(),
+        "max_abs": delta.max().item(),
+        "mean_abs": delta.mean().item(),
+        "p50_abs": torch.quantile(delta, 0.50).item(),
+        "p90_abs": torch.quantile(delta, 0.90).item(),
+        "p99_abs": torch.quantile(delta, 0.99).item(),
+    }
+
+
+def load_v2_modules(root: Path) -> tuple[Any, Any]:
+    package_name = "deepseek_v2_lite_absorb_official"
+    package = types.ModuleType(package_name)
+    package.__path__ = [str(root)]
+    sys.modules[package_name] = package
+    loaded = {}
+    for leaf in ("configuration_deepseek", "modeling_deepseek"):
+        name = f"{package_name}.{leaf}"
+        spec = importlib.util.spec_from_file_location(name, root / f"{leaf}.py")
+        if spec is None or spec.loader is None:
+            raise RuntimeError(f"cannot import official V2-Lite module: {leaf}")
+        module = importlib.util.module_from_spec(spec)
+        sys.modules[name] = module
+        spec.loader.exec_module(module)
+        loaded[leaf] = module
+    return loaded["configuration_deepseek"], loaded["modeling_deepseek"]
+
+
+def load_v3_module(root: Path) -> Any:
+    inference = root / "inference"
+    sys.path.insert(0, str(inference))
+    spec = importlib.util.spec_from_file_location(
+        "deepseek_v3_absorb_official", inference / "model.py"
+    )
+    if spec is None or spec.loader is None:
+        raise RuntimeError("cannot import official DeepSeek-V3 inference/model.py")
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[spec.name] = module
+    spec.loader.exec_module(module)
+    return module
+
+
+def read_shard_state(
+    shard: Path, prefixes: tuple[str, ...], exact: tuple[str, ...] = ()
+) -> dict[str, torch.Tensor]:
+    with safe_open(shard, framework="pt", device="cpu") as handle:
+        return {
+            key: handle.get_tensor(key)
+            for key in handle.keys()
+            if key in exact or any(key.startswith(prefix) for prefix in prefixes)
+        }
+
+
+def load_real_layer1_input(
+    shard: Path,
+    modeling: Any,
+    config: Any,
+    tokenizer: Any,
+    device: torch.device,
+) -> tuple[torch.Tensor, dict[str, torch.Tensor], list[int]]:
+    encoded = tokenizer(PROMPT, add_special_tokens=True, return_tensors="pt")
+    input_ids = encoded.input_ids
+    token_ids = input_ids[0].tolist()
+    if len(token_ids) != 26:
+        raise RuntimeError(f"prompt contract changed: expected 26 tokens, got {len(token_ids)}")
+
+    state = read_shard_state(
+        shard,
+        ("model.layers.0.", "model.layers.1.self_attn."),
+        ("model.embed_tokens.weight", "model.layers.1.input_layernorm.weight"),
+    )
+    embedding = state.pop("model.embed_tokens.weight")
+    hidden = F.embedding(input_ids, embedding).to(device)
+
+    layer0_prefix = "model.layers.0."
+    layer0_state = {
+        key[len(layer0_prefix) :]: value
+        for key, value in state.items()
+        if key.startswith(layer0_prefix)
+    }
+    with torch.device("meta"):
+        layer0 = modeling.DeepseekV2DecoderLayer(config, 0)
+    layer0.to_empty(device="cpu")
+    layer0.load_state_dict(layer0_state, strict=True, assign=True)
+    layer0 = layer0.to(device=device, dtype=torch.bfloat16).eval()
+
+    sequence = input_ids.shape[1]
+    position_ids = torch.arange(sequence, device=device).unsqueeze(0)
+    attention_mask = torch.ones_like(input_ids, device=device)
+    causal_mask = modeling._prepare_4d_causal_attention_mask(
+        attention_mask,
+        input_ids.shape,
+        hidden,
+        0,
+    )
+    with torch.inference_mode():
+        hidden = layer0(
+            hidden,
+            attention_mask=causal_mask,
+            position_ids=position_ids,
+            past_key_value=DynamicCache(),
+            use_cache=True,
+        )[0]
+
+    layer1_norm = modeling.DeepseekV2RMSNorm(
+        config.hidden_size, eps=config.rms_norm_eps
+    ).to(device=device, dtype=torch.bfloat16)
+    with torch.no_grad():
+        layer1_norm.weight.copy_(
+            state["model.layers.1.input_layernorm.weight"].to(
+                device=device, dtype=torch.bfloat16
+            )
+        )
+    with torch.inference_mode():
+        normalized = layer1_norm(hidden)
+
+    attention_prefix = "model.layers.1.self_attn."
+    attention_state = {
+        key[len(attention_prefix) :]: value
+        for key, value in state.items()
+        if key.startswith(attention_prefix)
+    }
+    del layer0, layer0_state, embedding, hidden, layer1_norm, state
+    gc.collect()
+    torch.cuda.empty_cache()
+    return normalized, attention_state, token_ids
+
+
+def v3_args(v3: Any, config: Any, cache_slots: int) -> Any:
+    rope = config.rope_scaling
+    args = v3.ModelArgs(
+        max_batch_size=1,
+        max_seq_len=cache_slots,
+        dtype="bf16",
+        vocab_size=config.vocab_size,
+        dim=config.hidden_size,
+        n_layers=config.num_hidden_layers,
+        n_heads=config.num_attention_heads,
+        q_lora_rank=0,
+        kv_lora_rank=config.kv_lora_rank,
+        qk_nope_head_dim=config.qk_nope_head_dim,
+        qk_rope_head_dim=config.qk_rope_head_dim,
+        v_head_dim=config.v_head_dim,
+        original_seq_len=rope["original_max_position_embeddings"],
+        rope_theta=config.rope_theta,
+        rope_factor=rope["factor"],
+        beta_fast=rope["beta_fast"],
+        beta_slow=rope["beta_slow"],
+        mscale=rope["mscale"],
+    )
+    # The official class uses max_seq_len both to configure YaRN and to allocate
+    # cache buffers. Keep the workstation cache small, but retain the model's
+    # official 163,840-position YaRN contract for frequencies and scaling.
+    args.rope_reference_max_seq_len = config.max_position_embeddings
+    return args
+
+
+def copy_v2_attention_weights(
+    module: Any,
+    state: dict[str, torch.Tensor],
+    device: torch.device,
+    dtype: torch.dtype,
+) -> None:
+    mapping = {
+        "wq.weight": "q_proj.weight",
+        "wkv_a.weight": "kv_a_proj_with_mqa.weight",
+        "kv_norm.weight": "kv_a_layernorm.weight",
+        "wkv_b.weight": "kv_b_proj.weight",
+        "wo.weight": "o_proj.weight",
+    }
+    target = dict(module.named_parameters())
+    with torch.no_grad():
+        for destination, source in mapping.items():
+            target[destination].copy_(state[source].to(device=device, dtype=dtype))
+
+
+def execute_v3_path(
+    v3: Any,
+    args: Any,
+    state: dict[str, torch.Tensor],
+    normalized: torch.Tensor,
+    device: torch.device,
+    dtype: torch.dtype,
+    implementation: str,
+) -> tuple[torch.Tensor, dict[str, Any]]:
+    v3.attn_impl = implementation
+    v3.Linear.dtype = dtype
+    module = v3.MLA(args).to(device=device, dtype=dtype).eval()
+    copy_v2_attention_weights(module, state, device, dtype)
+    frequency_args = copy.copy(args)
+    frequency_args.max_seq_len = args.rope_reference_max_seq_len
+    frequencies = v3.precompute_freqs_cis(frequency_args).to(device)
+    if frequency_args.max_seq_len > args.original_seq_len:
+        mscale = 0.1 * args.mscale * math.log(args.rope_factor) + 1.0
+        module.softmax_scale = module.qk_head_dim ** -0.5 * mscale * mscale
+    prefill = normalized[:, :-1].to(dtype)
+    decode = normalized[:, -1:].to(dtype)
+    prefill_length = prefill.shape[1]
+    mask = torch.full(
+        (prefill_length, prefill_length),
+        float("-inf"),
+        device=device,
+        dtype=torch.float32,
+    ).triu_(1)
+    with torch.inference_mode():
+        module(prefill, 0, frequencies[:prefill_length], mask)
+        output = module(
+            decode,
+            prefill_length,
+            frequencies[prefill_length : prefill_length + 1],
+            None,
+        )
+
+    if implementation == "naive":
+        active = {
+            "key": {
+                "shape": [1, prefill_length + 1, args.n_heads, args.qk_nope_head_dim + args.qk_rope_head_dim],
+                "bytes": (prefill_length + 1)
+                * args.n_heads
+                * (args.qk_nope_head_dim + args.qk_rope_head_dim)
+                * torch.empty((), dtype=dtype).element_size(),
+            },
+            "value": {
+                "shape": [1, prefill_length + 1, args.n_heads, args.v_head_dim],
+                "bytes": (prefill_length + 1)
+                * args.n_heads
+                * args.v_head_dim
+                * torch.empty((), dtype=dtype).element_size(),
+            },
+        }
+        allocated = {
+            "key_shape": list(module.k_cache.shape),
+            "key_bytes": tensor_bytes(module.k_cache),
+            "value_shape": list(module.v_cache.shape),
+            "value_bytes": tensor_bytes(module.v_cache),
+        }
+    else:
+        active = {
+            "latent": {
+                "shape": [1, prefill_length + 1, args.kv_lora_rank],
+                "bytes": (prefill_length + 1)
+                * args.kv_lora_rank
+                * torch.empty((), dtype=dtype).element_size(),
+            },
+            "rope": {
+                "shape": [1, prefill_length + 1, args.qk_rope_head_dim],
+                "bytes": (prefill_length + 1)
+                * args.qk_rope_head_dim
+                * torch.empty((), dtype=dtype).element_size(),
+            },
+        }
+        allocated = {
+            "latent_shape": list(module.kv_cache.shape),
+            "latent_bytes": tensor_bytes(module.kv_cache),
+            "rope_shape": list(module.pe_cache.shape),
+            "rope_bytes": tensor_bytes(module.pe_cache),
+        }
+
+    result = {
+        "implementation": implementation,
+        "dtype": str(dtype).replace("torch.", ""),
+        "active_cache": active,
+        "active_cache_bytes": sum(item["bytes"] for item in active.values()),
+        "allocated_cache": allocated,
+        "allocated_cache_bytes": sum(
+            value for key, value in allocated.items() if key.endswith("_bytes")
+        ),
+        "decode_finite": bool(torch.isfinite(output).all()),
+    }
+    output = output.detach().cpu()
+    del module, frequencies, prefill, decode, mask
+    gc.collect()
+    torch.cuda.empty_cache()
+    return output, result
+
+
+def execute_hf_eager(
+    modeling: Any,
+    config: Any,
+    state: dict[str, torch.Tensor],
+    normalized: torch.Tensor,
+    device: torch.device,
+) -> tuple[torch.Tensor, dict[str, Any]]:
+    # A standalone DynamicCache is zero-indexed; the weights still come from
+    # decoder layer 1, while this isolated attention module occupies cache slot 0.
+    attention = modeling.DeepseekV2Attention(config, layer_idx=0)
+    attention.load_state_dict(state, strict=True, assign=True)
+    attention = attention.to(device=device, dtype=torch.bfloat16).eval()
+    prefill = normalized[:, :-1]
+    decode = normalized[:, -1:]
+    prefill_length = prefill.shape[1]
+    prefill_mask = modeling._prepare_4d_causal_attention_mask(
+        torch.ones((1, prefill_length), dtype=torch.long, device=device),
+        (1, prefill_length),
+        prefill,
+        0,
+    )
+    cache = DynamicCache()
+    with torch.inference_mode():
+        attention(
+            prefill,
+            attention_mask=prefill_mask,
+            position_ids=torch.arange(prefill_length, device=device).unsqueeze(0),
+            past_key_value=cache,
+            use_cache=True,
+        )
+        output = attention(
+            decode,
+            attention_mask=torch.zeros(
+                (1, 1, 1, prefill_length + 1),
+                device=device,
+                dtype=decode.dtype,
+            ),
+            position_ids=torch.tensor([[prefill_length]], device=device),
+            past_key_value=cache,
+            use_cache=True,
+        )[0]
+    key = cache.key_cache[0]
+    value = cache.value_cache[0]
+    result = {
+        "implementation": "huggingface_eager",
+        "dtype": "bfloat16",
+        "key_shape": list(key.shape),
+        "value_shape": list(value.shape),
+        "active_cache_bytes": tensor_bytes(key) + tensor_bytes(value),
+        "decode_finite": bool(torch.isfinite(output).all()),
+    }
+    output = output.detach().cpu()
+    del attention, cache, key, value, prefill, decode, prefill_mask
+    gc.collect()
+    torch.cuda.empty_cache()
+    return output, result
+
+
+def nvidia_smi() -> dict[str, str]:
+    fields = ["name", "driver_version", "memory.total", "compute_cap"]
+    output = subprocess.check_output(
+        [
+            "nvidia-smi",
+            f"--query-gpu={','.join(fields)}",
+            "--format=csv,noheader,nounits",
+        ],
+        text=True,
+    ).strip()
+    return dict(zip(fields, [item.strip() for item in output.split(",")], strict=True))
+
+
+def flashmla_boundary(root: Path, device: torch.device) -> dict[str, Any]:
+    readme = (root / "README.md").read_text()
+    setup = (root / "setup.py").read_text()
+    dense_header = (root / "csrc/api/dense_decode.h").read_text()
+    required = [
+        "| Dense Decoding | SM90 | MQA | BF16 |",
+        "| Sparse Decoding | SM90 & SM100 | MQA | FP8",
+        '"arch=compute_100f,code=sm_100f"',
+        '"arch=compute_90a,code=sm_90a"',
+        "Dense decode MLA is only supported on SM90a architecture",
+    ]
+    combined = "\n".join((readme, setup, dense_header))
+    missing = [needle for needle in required if needle not in combined]
+    if missing:
+        raise RuntimeError(f"FlashMLA source contract changed: {missing}")
+    capability = torch.cuda.get_device_capability(device)
+    return {
+        "official_revision": FLASHMLA_REVISION,
+        "cutlass_revision": CUTLASS_REVISION,
+        "official_support_matrix": [
+            {"kernel": "dense_decode", "architectures": ["SM90"], "mode": "MQA", "cache": "BF16"},
+            {"kernel": "sparse_decode", "architectures": ["SM90", "SM100"], "mode": "MQA", "cache": "FP8"},
+            {"kernel": "dense_prefill", "architectures": ["SM100"], "mode": "MHA", "cache": None},
+            {"kernel": "sparse_prefill", "architectures": ["SM90", "SM100"], "mode": "MQA", "cache": None},
+        ],
+        "compiled_gencode": ["sm_100f", "sm_90a"],
+        "dense_decode_runtime_guard": "Dense decode MLA is only supported on SM90a architecture",
+        "local_architecture": f"SM{capability[0]}{capability[1]}",
+        "optimized_kernel_executed": False,
+        "why_not": "The pinned official source does not list or generate SM120 kernels; dense decode additionally rejects non-SM90a devices.",
+        "build_attempts": [
+            {
+                "environment": "host CUDA 12.8 / g++ 15.2",
+                "result": "failed before compilation",
+                "evidence": "CUDA 12.8 rejects host compiler versions newer than g++ 13; g++-13 is not installed.",
+            },
+            {
+                "environment": "isolated CUDA 13.0.2 / torch 2.11.0+cu130 / g++ 13.3",
+                "result": "failed during official extension compilation",
+                "evidence": "csrc/api.cpp could not find cuda/std/utility; no wheel or runtime kernel was produced.",
+            },
+        ],
+        "source_sha256": {
+            "readme": sha256(root / "README.md"),
+            "setup": sha256(root / "setup.py"),
+            "dense_decode_header": sha256(root / "csrc/api/dense_decode.h"),
+        },
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    artifact_root = args.artifact_dir.resolve()
+    v3_root = args.v3_repo.resolve()
+    flashmla_root = args.flashmla_repo.resolve()
+    shard = artifact_root / "model-00001-of-000004.safetensors"
+    required = [
+        shard,
+        artifact_root / "config.json",
+        artifact_root / "configuration_deepseek.py",
+        artifact_root / "modeling_deepseek.py",
+        artifact_root / "tokenizer.json",
+        v3_root / "inference/model.py",
+        flashmla_root / "README.md",
+        flashmla_root / "setup.py",
+        flashmla_root / "csrc/api/dense_decode.h",
+    ]
+    missing = [str(path) for path in required if not path.exists()]
+    if missing:
+        raise FileNotFoundError(f"missing required official artifacts: {missing}")
+    if git_revision(v3_root) != V3_REVISION:
+        raise RuntimeError("DeepSeek-V3 revision does not match the probe contract")
+    if git_revision(flashmla_root) != FLASHMLA_REVISION:
+        raise RuntimeError("FlashMLA revision does not match the probe contract")
+    if args.cache_slots < 26:
+        raise ValueError("cache-slots must be at least 26")
+    device = torch.device(args.device)
+    if device.type == "cuda" and not torch.cuda.is_available():
+        raise RuntimeError("CUDA requested but unavailable")
+
+    torch.manual_seed(0)
+    configuration, modeling = load_v2_modules(artifact_root)
+    config = configuration.DeepseekV2Config.from_pretrained(artifact_root)
+    config._attn_implementation = "eager"
+    tokenizer = AutoTokenizer.from_pretrained(
+        artifact_root,
+        trust_remote_code=True,
+        local_files_only=True,
+    )
+    v3 = load_v3_module(v3_root)
+
+    normalized, attention_state, token_ids = load_real_layer1_input(
+        shard, modeling, config, tokenizer, device
+    )
+    hf_output, hf_result = execute_hf_eager(
+        modeling, config, attention_state, normalized, device
+    )
+    model_args = v3_args(v3, config, args.cache_slots)
+    naive_bf16, naive_result = execute_v3_path(
+        v3,
+        model_args,
+        attention_state,
+        normalized,
+        device,
+        torch.bfloat16,
+        "naive",
+    )
+    absorb_bf16, absorb_result = execute_v3_path(
+        v3,
+        model_args,
+        attention_state,
+        normalized,
+        device,
+        torch.bfloat16,
+        "absorb",
+    )
+    naive_fp32, naive_fp32_result = execute_v3_path(
+        v3,
+        model_args,
+        attention_state,
+        normalized,
+        device,
+        torch.float32,
+        "naive",
+    )
+    absorb_fp32, absorb_fp32_result = execute_v3_path(
+        v3,
+        model_args,
+        attention_state,
+        normalized,
+        device,
+        torch.float32,
+        "absorb",
+    )
+
+    naive_bytes = naive_result["active_cache_bytes"]
+    absorb_bytes = absorb_result["active_cache_bytes"]
+    result = {
+        "schema_version": 1,
+        "captured_at": args.captured_at or datetime.now(timezone.utc).isoformat(),
+        "evidence_identity": "X / official V2-Lite weights executed through official HF eager and V3 naive/absorb reference paths",
+        "boundary": {
+            "correctness_probe": True,
+            "serving_benchmark": False,
+            "full_model_generation": False,
+            "optimized_flashmla_kernel_executed": False,
+            "scope": "one real layer-1 attention input; 25-token prefill plus one-token decode",
+        },
+        "provenance": {
+            "huggingface_model": "deepseek-ai/DeepSeek-V2-Lite",
+            "huggingface_revision": HF_REVISION,
+            "deepseek_v3_repository": "deepseek-ai/DeepSeek-V3",
+            "deepseek_v3_revision": V3_REVISION,
+            "flashmla_repository": "deepseek-ai/FlashMLA",
+            "flashmla_revision": FLASHMLA_REVISION,
+            "sha256": {
+                "v2_config": sha256(artifact_root / "config.json"),
+                "v2_modeling": sha256(artifact_root / "modeling_deepseek.py"),
+                "v2_shard_1": sha256(shard),
+                "v3_model": sha256(v3_root / "inference/model.py"),
+            },
+        },
+        "environment": {
+            "python": platform.python_version(),
+            "platform": platform.platform(),
+            "libc": list(platform.libc_ver()),
+            "torch": torch.__version__,
+            "torch_cuda": torch.version.cuda,
+            "transformers": __import__("transformers").__version__,
+            "safetensors": __import__("safetensors").__version__,
+            "device": str(device),
+            "cuda_capability": list(torch.cuda.get_device_capability(device)),
+            "nvidia_smi": nvidia_smi(),
+        },
+        "input": {
+            "prompt": PROMPT,
+            "token_ids": token_ids,
+            "tokens": tokenizer.convert_ids_to_tokens(token_ids),
+            "sequence_tokens": len(token_ids),
+            "prefill_tokens": len(token_ids) - 1,
+            "decode_tokens": 1,
+            "layer0_executed": True,
+            "layer1_input_normalized": True,
+        },
+        "configuration": {
+            "hidden": config.hidden_size,
+            "attention_heads": config.num_attention_heads,
+            "qk_nope_head_dim": config.qk_nope_head_dim,
+            "qk_rope_head_dim": config.qk_rope_head_dim,
+            "v_head_dim": config.v_head_dim,
+            "kv_lora_rank": config.kv_lora_rank,
+            "cache_slots": args.cache_slots,
+        },
+        "execution": {
+            "huggingface_eager": hf_result,
+            "v3_naive_bfloat16": naive_result,
+            "v3_absorb_bfloat16": absorb_result,
+            "v3_naive_float32": naive_fp32_result,
+            "v3_absorb_float32": absorb_fp32_result,
+        },
+        "correctness": {
+            "hf_eager_vs_v3_naive_bfloat16": compare(hf_output, naive_bf16),
+            "v3_naive_vs_absorb_bfloat16": compare(naive_bf16, absorb_bf16),
+            "hf_eager_vs_v3_absorb_bfloat16": compare(hf_output, absorb_bf16),
+            "v3_naive_vs_absorb_float32": compare(naive_fp32, absorb_fp32),
+        },
+        "cache_accounting": {
+            "active_tokens": len(token_ids),
+            "dtype": "BF16",
+            "naive_active_bytes": naive_bytes,
+            "absorb_active_bytes": absorb_bytes,
+            "naive_elements_per_token": config.num_attention_heads
+            * (
+                config.qk_nope_head_dim
+                + config.qk_rope_head_dim
+                + config.v_head_dim
+            ),
+            "absorb_elements_per_token": config.kv_lora_rank
+            + config.qk_rope_head_dim,
+            "naive_over_absorb_ratio": naive_bytes / absorb_bytes,
+            "absorb_reduction_vs_naive": 1 - absorb_bytes / naive_bytes,
+        },
+        "flashmla_boundary": flashmla_boundary(flashmla_root, device),
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n")
+    print(json.dumps(result, ensure_ascii=False, indent=2))
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,420 @@
+# DeepSeek MLA 吸收式缓存：真实权重执行与 FlashMLA 架构边界
+
+> 状态：DeepSeek 第三轮第二里程碑  
+> 证据等级：`O` 官方源码 / `X` 本机真实执行 / `D` 确定推导 / `U` 未覆盖  
+> 结论边界：正确性与缓存格式实验，不是服务吞吐 benchmark
+
+## 0. 先给结论
+
+这轮补上了上一次实验最重要的空白：
+
+- 上一次已经观察到 V2-Lite 的真实压缩投影是 `[4, 27, 576]`；
+- 但 Hugging Face 官方 eager 路径实际缓存展开后的
+  `K [4,16,27,192]` 与 `V [4,16,27,128]`；
+- 这一次把同一组真实 V2-Lite layer-1 权重映射到 DeepSeek-V3 官方
+  `inference/model.py`，真正执行了 `attn_impl="absorb"`；
+- 25-token prefill + 1-token decode 后，官方 buffer 中实际存在：
+  - `kv_cache [1,32,512]`，其中 26-token active slice 为 `26,624 B`；
+  - `pe_cache [1,32,64]`，其中 26-token active slice 为 `3,328 B`；
+  - active 总计 `29,952 B`。
+- 同一官方参考类的 naive 路径实际 active cache 为：
+  - `k_cache`：`159,744 B`；
+  - `v_cache`：`106,496 B`；
+  - active 总计 `266,240 B`。
+- 因而在这个 V2-Lite 配置、BF16、同一 26-token 输入下：
+
+```text
+naive / absorb = 266,240 / 29,952 = 8.888888…
+reduction       = 1 - 29,952 / 266,240 = 88.75%
+```
+
+这不再只是 shape 算术：两组官方 cache buffers 都已在 RTX 5090 上实际分配、写入并用于
+incremental decode。
+
+但是另一条边界同样重要：
+
+- DeepSeek-V3 的官方纯 PyTorch `absorb` 参考路径：**已执行**；
+- DeepSeek FlashMLA 优化 kernel：**未执行**；
+- 原因不是把环境问题猜成性能结论，而是 pinned FlashMLA 官方支持矩阵与源码只覆盖
+  SM90 / SM100，本机 RTX 5090 是 SM120。
+
+## 1. 可复现身份
+
+### 1.1 模型与源码
+
+| 工件 | 固定身份 |
+|---|---|
+| Hugging Face 模型 | `deepseek-ai/DeepSeek-V2-Lite` |
+| HF revision | `604d5664dddd88a0433dbae533b7fe9472482de0` |
+| DeepSeek-V3 仓库 | `deepseek-ai/DeepSeek-V3` |
+| V3 revision | `9b4e9788e4a3a731f7567338ed15d3ec549ce03b` |
+| FlashMLA 仓库 | `deepseek-ai/FlashMLA` |
+| FlashMLA revision | `15f13e5030374295491c5ce31b02d7e63a7772c6` |
+| FlashMLA CUTLASS submodule | `147f5673d0c1c3dcf66f78d677fd647e4a020219` |
+
+官方入口：
+
+- [DeepSeek-V2-Lite 模型](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite)
+- [DeepSeek-V2 仓库](https://github.com/deepseek-ai/DeepSeek-V2)
+- [DeepSeek-V2 技术报告](https://arxiv.org/abs/2405.04434)
+- [DeepSeek-V3 inference/model.py](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py)
+- [FlashMLA](https://github.com/deepseek-ai/FlashMLA)
+
+### 1.2 本机环境
+
+| 字段 | 值 |
+|---|---|
+| GPU | NVIDIA GeForce RTX 5090 |
+| compute capability | `12.0` / SM120 |
+| driver | `595.84` |
+| torch | `2.11.0+cu128` |
+| CUDA runtime | `12.8` |
+| Transformers | 隔离的 `4.41.2` |
+| checkpoint dtype | BF16 |
+
+V2-Lite 的 2024 官方 remote code 不能直接在当前 Transformers 5.5 中导入，因此延续上一轮合同：
+官方模型源码不打补丁，只把依赖固定到隔离目录中的 Transformers 4.41.2。
+
+### 1.3 输出
+
+- 主结果：`src/data/deepseek-v2-lite-absorb.json`
+- 独立复跑：`src/data/deepseek-v2-lite-absorb-repro.json`
+- 两份完整 JSON SHA-256：
+  `6b4c714a470baa6d6b04982cd289e693ec863398b5def4e6df541cb10525e63d`
+- `cmp`：逐字节一致
+
+实验没有计时字段，因此不需要用“排除 timing”掩盖不同；两次输出完整 byte-exact。
+
+## 2. 为什么权重可以“吸收”
+
+### 2.1 普通展开路径
+
+令每个历史 token 的 512 维压缩状态为 `cₜ`。忽略位置分支时，每个 attention head 的
+key/value 可以写成：
+
+```text
+kₜ = Wᴷ cₜ
+vₜ = Wⱽ cₜ
+```
+
+naive 实现先把所有历史 `cₜ` 展开成每个 head 的 `kₜ` 和 `vₜ`，再缓存展开结果。V2-Lite
+有 16 个 head：
+
+```text
+key   = 16 × (128 no-RoPE + 64 RoPE) = 3,072 elements/token
+value = 16 × 128                     = 2,048 elements/token
+total = 5,120 elements/token/layer
+```
+
+问题不是这些张量算错了，而是同一个 512 维内容状态被投影并复制到各个 head 后才进入缓存。
+
+### 2.2 把 key 投影搬到 query
+
+注意力分数里的 no-RoPE 部分：
+
+```text
+qᵀkₜ
+= qᵀ(Wᴷcₜ)
+= (Wᴷᵀq)ᵀcₜ
+```
+
+矩阵乘法结合律允许先把 query 变成 `Wᴷᵀq`，再直接与历史 latent `cₜ` 点积。于是历史状态不必
+缓存每个 head 展开后的 `Wᴷcₜ`。
+
+这里的“吸收”不是删掉参数：
+
+- `Wᴷ` 仍然存在；
+- 它从“写 cache 前作用在每个历史 token 上”，搬到“读 cache 时作用在当前 query 上”；
+- 权重的作用位置改变，数学函数在精确算术中不变。
+
+### 2.3 把 value 投影搬到 attention 后
+
+attention 的 value 聚合：
+
+```text
+Σₜ pₜvₜ
+= Σₜ pₜ(Wⱽcₜ)
+= Wⱽ(Σₜ pₜcₜ)
+```
+
+因此可以先用注意力概率 `pₜ` 对 512 维 latent 做加权和，再把聚合结果通过 `Wⱽ` 展开到各个
+head 的 value 空间。
+
+naive 顺序：
+
+```text
+每个历史 cₜ → Wⱽ → 展开 vₜ → cache → 按 pₜ 加权
+```
+
+absorb 顺序：
+
+```text
+每个历史 cₜ → cache → 按 pₜ 加权 → Wⱽ → 展开输出
+```
+
+### 2.4 为什么 64 维 RoPE key 仍要缓存
+
+RoPE 是位置相关变换。历史 token `t` 的 key 旋转角度依赖位置 `t`，不能像固定线性投影那样无条件
+搬到当前 query 一侧。因此 V2-Lite 的可缓存状态不是只有 512：
+
+```text
+512 KV latent + 64 decoupled RoPE key = 576 elements/token/layer
+```
+
+这也是网站从第一轮起始终把“512 latent”与“576 完整缓存合同”分开的原因。
+
+## 3. 实验怎样避免偷换输入
+
+### 3.1 输入不是随机 hidden state
+
+固定中文 prompt：
+
+```text
+用通俗的语言解释，为什么稀疏专家模型可以拥有很多参数，但每个 token 只使用其中一小部分。
+```
+
+官方 tokenizer 得到 26 个 token。执行链是：
+
+```text
+官方 tokenizer
+  → 官方 embedding 权重
+  → 官方 V2-Lite decoder layer 0 完整 forward
+  → 官方 layer-1 input RMSNorm
+  → 真实 layer-1 attention input
+```
+
+因此 attention 的输入来自 checkpoint 执行轨迹，不是为了让误差更小而构造的随机数。
+
+### 3.2 为什么只测 layer 1 attention
+
+第一 checkpoint 分片完整包含：
+
+- embedding；
+- layer 0 全部参数；
+- layer 1 全部 attention 参数与 input RMSNorm。
+
+本里程碑问的是“吸收路径能否用真实权重兑现 576 元素缓存”，不需要再次运行 MoE。只隔离 layer-1
+attention，可以同时做到：
+
+- 输入由真实前层产生；
+- attention 权重全部来自官方 checkpoint；
+- naive 与 absorb 使用完全相同的输入和参数；
+- 不把未下载的完整模型写成已复现。
+
+### 3.3 权重怎样映射
+
+V2-Lite HF remote code 与 V3 官方 inference reference 的张量语义一致：
+
+| V2-Lite HF | V3 reference |
+|---|---|
+| `q_proj.weight` | `wq.weight` |
+| `kv_a_proj_with_mqa.weight` | `wkv_a.weight` |
+| `kv_a_layernorm.weight` | `kv_norm.weight` |
+| `kv_b_proj.weight` | `wkv_b.weight` |
+| `o_proj.weight` | `wo.weight` |
+
+没有转置 checkpoint、没有重新训练、没有拟合中间 adapter。
+
+### 3.4 三条 BF16 路径
+
+同一 26-token 输入切成：
+
+- prefill：前 25 token；
+- incremental decode：最后 1 token。
+
+执行：
+
+1. 官方 V2-Lite Hugging Face eager attention；
+2. pinned V3 官方 `attn_impl="naive"`；
+3. pinned V3 官方 `attn_impl="absorb"`。
+
+HF eager 与 V3 naive 的对照先验证“权重映射 + YaRN 配置 + causal/incremental 路径”没有发生大幅漂移；
+V3 naive 与 V3 absorb 的对照再隔离权重吸收本身。
+
+## 4. 真实 cache 观测
+
+### 4.1 BF16 active slice
+
+| 路径 | 实际 shape | bytes |
+|---|---|---:|
+| V3 naive key | `[1,26,16,192]` | 159,744 |
+| V3 naive value | `[1,26,16,128]` | 106,496 |
+| **V3 naive total** | 26 × 5,120 BF16 | **266,240** |
+| V3 absorb latent | `[1,26,512]` | 26,624 |
+| V3 absorb RoPE | `[1,26,64]` | 3,328 |
+| **V3 absorb total** | 26 × 576 BF16 | **29,952** |
+
+### 4.2 32-slot 实际 buffer
+
+| 路径 | 分配 shape | allocated bytes |
+|---|---|---:|
+| V3 naive key | `[1,32,16,192]` | 196,608 |
+| V3 naive value | `[1,32,16,128]` | 131,072 |
+| **V3 naive total** |  | **327,680** |
+| V3 absorb latent | `[1,32,512]` | 32,768 |
+| V3 absorb RoPE | `[1,32,64]` | 4,096 |
+| **V3 absorb total** |  | **36,864** |
+
+active 与 allocated 分开记录，是为了不把 32-slot 容器的尾部空位算成 26-token 状态。
+
+## 5. 数值正确性
+
+所有输出 shape 都是 `[1,1,2048]`，全部 finite。
+
+### 5.1 HF eager 与 V3 naive
+
+| 指标 | BF16 |
+|---|---:|
+| max absolute error | 0.001220703125 |
+| mean absolute error | 0.0000888268 |
+| p99 absolute error | 0.0003823861 |
+| exact fraction | 0.284668 |
+
+两份官方实现的张量布局、einsum/matmul 顺序不同，BF16 不能要求逐元素完全相等。这个对照的作用是检查
+映射与配置；它不是“证明两个代码库 bit-exact”。
+
+### 5.2 V3 naive 与 V3 absorb
+
+| 指标 | BF16 |
+|---|---:|
+| max absolute error | 0.00390625 |
+| mean absolute error | 0.0002703113 |
+| p99 absolute error | 0.0009236149 |
+| exact fraction | 0.100098 |
+
+absorb 改变了矩阵乘法结合顺序；有限精度下，先乘什么会改变舍入位置。因此 BF16 输出接近但不
+bit-exact，符合预期。
+
+### 5.3 FP32 代数审计
+
+为了区分“公式不等价”和“BF16 舍入顺序”，实验把同一组 BF16 checkpoint 权重与真实输入提升为 FP32，
+重新执行 V3 naive/absorb：
+
+| 指标 | FP32 |
+|---|---:|
+| max absolute error | `1.1920929e-7` |
+| mean absolute error | `9.4644861e-9` |
+| p99 absolute error | `3.1446120e-8` |
+
+这不是声称存在 FP32 checkpoint，也不是模型精度评测。它是一个代数隔离实验：精度提高后，两种结合
+顺序的误差下降到 FP32 舍入量级。
+
+## 6. FlashMLA：为什么没有在 RTX 5090 上宣称成功
+
+### 6.1 官方支持矩阵
+
+pinned FlashMLA README：
+
+| kernel | GPU architecture | mode | KV cache |
+|---|---|---|---|
+| dense decoding | SM90 | MQA | BF16 |
+| sparse decoding | SM90 & SM100 | MQA | FP8 |
+| dense prefill | SM100 | MHA | — |
+| sparse prefill | SM90 & SM100 | MQA | — |
+
+README 对 MQA mode 的定义是：
+
+```text
+head_dim_k = 576
+head_dim_v = 512
+```
+
+这里的 `head_dim_v=512` 是吸收后的 latent 聚合空间，不应与 naive 路径每个 query head 的
+`v_head_dim=128` 混为一谈。
+
+### 6.2 编译目标与运行时 guard
+
+pinned `setup.py` 只加入：
+
+```text
+arch=compute_100f,code=sm_100f
+arch=compute_90a,code=sm_90a
+```
+
+`csrc/api/dense_decode.h` 还显式检查：
+
+```text
+Dense decode MLA is only supported on SM90a architecture
+```
+
+本机是 SM120。因此即使 CUDA 工具链能把扩展编译完成，也不能从这份官方源码推导出 SM120 dense
+decode 支持。
+
+### 6.3 两次构建尝试
+
+| 环境 | 结果 | 停止位置 |
+|---|---|---|
+| host CUDA 12.8 + g++ 15.2 | 失败 | CUDA 12.8 在正式编译前拒绝过新的 host compiler；本机没有 g++-13 |
+| 隔离 CUDA 13.0.2 + torch 2.11.0+cu130 + g++ 13.3 | 失败 | 官方 extension 编译 `csrc/api.cpp` 时找不到 `cuda/std/utility` |
+
+第二次构建中的其它 CUDA translation units 继续输出日志，是 Ninja 并行任务已经启动的结果；最终
+`ninja: build stopped: subcommand failed`，没有 wheel。
+
+正确的实验状态是：
+
+```text
+official V3 pure-PyTorch absorb reference: EXECUTED
+official FlashMLA optimized kernel on SM120: NOT EXECUTED / NOT LISTED AS SUPPORTED
+```
+
+不能把前者的成功写成后者的成功，也不能把后者的未支持写成“5090 做不了 MLA”。
+
+## 7. 与 V2 报告 93.3% 的关系
+
+本实验得到的 `88.75%` 是：
+
+```text
+V2-Lite 576-element latent contract
+vs
+V2-Lite 5,120-element HF/V3-naive expanded cache
+```
+
+V2 技术报告的 `93.3%` 属于论文中的完整 V2 模型、论文基线与论文配置。两者回答的口径不同。
+
+可以说：
+
+- V2-Lite 本地吸收缓存相对本地展开格式减少 88.75%；
+- 这轮已经从 shape 推导升级为真实 buffer 与 decode 执行；
+- 结果支持 MLA 权重吸收机制。
+
+不能说：
+
+- 本实验“复现了论文 93.3%”；
+- 88.75% 推翻了 93.3%；
+- reference 实现的缓存降幅就是生产吞吐提升；
+- SM120 已运行 FlashMLA。
+
+## 8. 复现命令
+
+```bash
+PYTHONPATH=/tmp/deepseek-v2-lite-pydeps.6ZJAH3 \
+/home/wuyang/.pyenv/versions/navi-router-cu128/bin/python \
+  experiments/deepseek/v2_lite_absorb_probe.py \
+  --artifact-dir /tmp/deepseek-v2-lite-artifacts.OEjfce \
+  --v3-repo /tmp/llm-atlas-deepseek-v3-official \
+  --flashmla-repo /tmp/llm-atlas-flashmla-official \
+  --output src/data/deepseek-v2-lite-absorb.json \
+  --captured-at 2026-07-29T06:35:00Z
+```
+
+第二次改写 `--output` 到
+`src/data/deepseek-v2-lite-absorb-repro.json`，然后：
+
+```bash
+sha256sum \
+  src/data/deepseek-v2-lite-absorb.json \
+  src/data/deepseek-v2-lite-absorb-repro.json
+
+cmp -s \
+  src/data/deepseek-v2-lite-absorb.json \
+  src/data/deepseek-v2-lite-absorb-repro.json
+```
+
+## 9. 下一轮应该做什么
+
+1. 在官方明确支持的 SM90/Hopper 环境执行 FlashMLA dense decode correctness test；
+2. 把同一组 latent cache 接入 serving runtime，分开测 prefill、decode、batch 与 context；
+3. 同时记录吞吐、TTFT、TPOT、峰值显存和数值误差，避免只报一个“快多少”；
+4. 扩展到多个 layer 与多个 prompt，检查误差如何累计；
+5. 若 FlashMLA 官方未来加入 SM120，固定新 revision 后重开支持矩阵审计，不能把未来支持倒写到本次记录。
+
@@ -276,6 +276,29 @@ const artifactCache = await evaluate(`(() => {
  return { trace, million };
 })()`);

+const artifactAbsorb = await evaluate(`(() => {
+  const root = document.querySelector("[data-dsv2-lab]");
+  root.querySelector('[data-artifact-tab="absorb"]').click();
+  return {
+    panel: root.querySelector("[data-artifact-panel]:not([hidden])").dataset.artifactPanel,
+    algebra: root.querySelectorAll(".absorb-algebra article").length,
+    naive: root.querySelector(".absorb-cache-flow .naive header b").textContent.trim(),
+    absorbed: root.querySelector(".absorb-cache-flow .absorbed header b").textContent.trim(),
+    metrics: [...root.querySelectorAll(".absorb-metrics article b")].map((node) => node.textContent.trim()),
+    precisionRows: root.querySelectorAll(".precision-lens > div").length,
+    matrixRows: root.querySelectorAll(".kernel-matrix > div").length,
+    localUnsupported: root.querySelectorAll(".kernel-matrix i.no").length,
+    executionCards: root.querySelectorAll(".execution-split article").length,
+    boundary: root.querySelector('[data-artifact-panel="absorb"] .artifact-boundary').textContent.replaceAll(/\\s+/g, " ").trim(),
+  };
+})()`);
+await evaluate(`(() => {
+  document.querySelector("[data-dsv2-lab]").scrollIntoView({ block: "start", behavior: "instant" });
+  window.scrollBy(0, -82);
+})()`);
+await pause(180);
+await screenshot("/tmp/llm-atlas-deepseek-absorb-desktop.png");
+
 const artifactEvidence = await evaluate(`(() => {
  const root = document.querySelector("[data-dsv2-lab]");
  root.querySelector('[data-artifact-tab="evidence"]').click();
@@ -361,13 +384,15 @@ const mobile = await evaluate(`(() => {
 })()`);
 await evaluate(`(() => {
  document.querySelector("#menu-toggle")?.click();
-  document.querySelector("[data-dsv2-lab]").scrollIntoView({ block: "start", behavior: "instant" });
+  const artifact = document.querySelector("[data-dsv2-lab]");
+  artifact.querySelector('[data-artifact-tab="absorb"]').click();
+  artifact.scrollIntoView({ block: "start", behavior: "instant" });
  window.scrollBy(0, -82);
 })()`);
 await pause(180);
 await screenshot("/tmp/llm-atlas-deepseek-artifact-mobile.png");

-const report = { overview, capacity, cache, codesign, rl, artifactRoute, artifactLoad, artifactCache, artifactEvidence, home, papers, mobile, exceptions };
+const report = { overview, capacity, cache, codesign, rl, artifactRoute, artifactLoad, artifactCache, artifactAbsorb, artifactEvidence, home, papers, mobile, exceptions };
 console.log(JSON.stringify(report, null, 2));

 const numeric = (text) => Number.parseFloat(text.replaceAll(",", ""));
@@ -377,7 +402,7 @@ if (overview.sections !== 26 || overview.tocLinks !== 26) failures.push("二十
 if (overview.ledgers !== 24 || overview.waves !== 10) failures.push("二十四张问题账或十次转向结构异常");
 if (overview.paperLinks !== 60 || overview.branches !== 5 || overview.followups !== 1) failures.push("论文链、旁支或公开后续标记异常");
 if (overview.labTabs !== 4 || overview.labPanels !== 4) failures.push("四联实验结构异常");
-if (overview.artifactTabs !== 4 || overview.artifactPanels !== 4 || overview.artifactLayers !== 27) failures.push("真实权重四联实验结构异常");
+if (overview.artifactTabs !== 5 || overview.artifactPanels !== 5 || overview.artifactLayers !== 27) failures.push("真实权重五联实验结构异常");
 if (overview.navLinks !== 20 || home.navLinks !== 20 || mobile.mobileLinks !== 20 || overview.activeNav !== "DeepSeek") failures.push("全站导航未同步 DeepSeek");
 if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出");
 if (capacity.initial.panel !== "capacity" || capacity.initial.total !== "32.1× FFN" || capacity.initial.active !== "1.13× FFN") failures.push("V3 稀疏容量初始账异常");
@@ -401,12 +426,15 @@ if (artifactLoad.layer1.panel !== "load" || artifactLoad.layer1.used !== "63" ||
 if (artifactLoad.layer2.used !== "64" || artifactLoad.layer2.cv !== "0.549" || artifactLoad.layer4.used !== "62" || artifactLoad.layer4.gini !== "0.417") failures.push("跨层负载统计切换异常");
 if (artifactCache.trace.panel !== "cache" || artifactCache.trace.latent !== "850.50 KiB" || artifactCache.trace.eager !== "7.38 MiB" || artifactCache.trace.ratio !== "8.89×" || artifactCache.trace.reduction !== "88.75%") failures.push("V2-Lite trace 缓存实现账异常");
 if (!artifactCache.million.latent.includes("GiB") || !artifactCache.million.eager.includes("TiB")) failures.push("V2-Lite 百万 Token 缓存外推异常");
+if (artifactAbsorb.panel !== "absorb" || artifactAbsorb.algebra !== 2 || artifactAbsorb.naive !== "260.00 KiB" || artifactAbsorb.absorbed !== "29.25 KiB") failures.push("真实 absorb 缓存执行账异常");
+if (artifactAbsorb.metrics[0] !== "8.8889×" || artifactAbsorb.metrics[1] !== "0.00390625" || artifactAbsorb.metrics[2] !== "1.19e-7" || artifactAbsorb.metrics[3] !== "BYTE-EXACT") failures.push("absorb 数值正确性或复跑闸门异常");
+if (artifactAbsorb.precisionRows !== 3 || artifactAbsorb.matrixRows !== 5 || artifactAbsorb.localUnsupported !== 4 || artifactAbsorb.executionCards !== 3 || !artifactAbsorb.boundary.includes("不是 FlashMLA 性能")) failures.push("FlashMLA SM120 边界结构异常");
 if (artifactEvidence.panel !== "evidence" || artifactEvidence.layers !== 27 || artifactEvidence.executed !== 7 || artifactEvidence.split !== 1 || artifactEvidence.unloaded !== 19 || artifactEvidence.exact !== "31 / 31") failures.push("真实工件执行边界或复跑闸门异常");
 if (!artifactEvidence.dependency.includes("Transformers 5.5") || !artifactEvidence.dependency.includes("4.41.2") || !artifactEvidence.boundary.includes("完整 27 层生成")) failures.push("依赖版本或未覆盖边界异常");
 if (artifactEvidence.keyboardSelected !== "load" || artifactEvidence.keyboardVisible !== "load") failures.push("真实工件实验键盘 tab 导航异常");
 if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/" || home.paperCount !== "486") failures.push("首页 DeepSeek 首发入口或论文数异常");
 if (papers.total !== 486 || !papers.hasFilter || papers.visible < 20 || !papers.hasCoder || !papers.hasEngram) failures.push("论文库 DeepSeek 聚光异常");
-if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4 || mobile.artifactTabs !== 4 || mobile.artifactHeatCells !== 64) failures.push("移动端导航或实验异常");
+if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4 || mobile.artifactTabs !== 5 || mobile.artifactHeatCells !== 64) failures.push("移动端导航或实验异常");
 if (mobile.offenders.length) failures.push(`移动端越界元素：${JSON.stringify(mobile.offenders)}`);
 if (exceptions.length) failures.push(`浏览器异常：${exceptions.join(" | ")}`);

@@ -1,9 +1,18 @@
 ---
 import rawTrace from "@/data/deepseek-v2-lite-trace.json";
 import rawRepro from "@/data/deepseek-v2-lite-trace-repro.json";
+import rawAbsorb from "@/data/deepseek-v2-lite-absorb.json";
+import rawAbsorbRepro from "@/data/deepseek-v2-lite-absorb-repro.json";

 const trace = rawTrace as any;
 const repro = rawRepro as any;
+const absorb = rawAbsorb as any;
+const absorbRepro = rawAbsorbRepro as any;
+const absorbExact = JSON.stringify(absorb) === JSON.stringify(absorbRepro);
+const bytes = (value: number) => value >= 1024
+  ? `${(value / 1024).toFixed(2)} KiB`
+  : `${value.toLocaleString()} B`;
+const scientific = (value: number) => value.toExponential(2);
 const compact = {
  prompts: trace.prompts.map((prompt: any) => ({
    id: prompt.id,
@@ -46,7 +55,7 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    </div>
    <p>
      固定官方 revision、tokenizer、模型代码和 BF16 第一分片；RTX 5090 连续执行 layer 0–6，
-      捕获真实 MLA 状态与 3,240 次 routed-expert 选择。所有结论都带证据身份与停止线。
+      捕获 3,240 次真实路由，并让 layer-1 权重继续走入官方吸收式 cache。所有结论都带证据身份与停止线。
    </p>
  </header>

@@ -67,8 +76,11 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    <button type="button" role="tab" data-artifact-tab="cache" aria-selected="false" tabindex="-1">
      <span>03</span><b>缓存实现账</b><small>latent vs HF eager</small>
    </button>
+    <button type="button" role="tab" data-artifact-tab="absorb" aria-selected="false" tabindex="-1">
+      <span>04</span><b>吸收式执行</b><small>real cache · SM120</small>
+    </button>
    <button type="button" role="tab" data-artifact-tab="evidence" aria-selected="false" tabindex="-1">
-      <span>04</span><b>证据断面</b><small>revision · shards · rerun</small>
+      <span>05</span><b>证据断面</b><small>revision · shards · rerun</small>
    </button>
  </div>

@@ -243,7 +255,114 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi

    <div class="artifact-boundary">
      <b>IMPLEMENTATION CONTRACT</b>
-      <p>算法上“可以缓存 576 元素”与本次框架“实际缓存 5,120 元素”同时为真；生产吞吐还需要真实 latent-cache kernel 与 serving runtime。</p>
+      <p>算法上“可以缓存 576 元素”与 HF eager“实际缓存 5,120 元素”同时为真；下一页继续执行官方 absorb 参考路径，生产吞吐仍需要受支持的优化 kernel 与 serving runtime。</p>
+    </div>
+  </section>
+
+  <section class="artifact-panel" data-artifact-panel="absorb" hidden>
+    <div class="panel-lead">
+      <div><span>X + O / WEIGHT ABSORPTION</span><h4>这一次 576 元素真的进入了缓存</h4></div>
+      <p>
+        同一组真实 V2-Lite layer-1 权重、同一条 26-token hidden-state 轨迹：
+        先用 V3 官方 naive 路径展开 K/V，再用官方 absorb 路径做 25-token prefill + 1-token decode。
+      </p>
+    </div>
+
+    <div class="absorb-algebra" aria-label="MLA 权重吸收的两条代数变换">
+      <article>
+        <span>SCORE / 把 Wᴷ 搬到 query</span>
+        <code>qᵀ(Wᴷc) = (Wᴷᵀq)ᵀc</code>
+        <p>不再为每个历史 token、每个 head 存展开后的 no-RoPE key；query 先变成 512 维，再与 latent <i>c</i> 相乘。</p>
+      </article>
+      <article>
+        <span>VALUE / 把 Wⱽ 搬到 attention 后</span>
+        <code>Σ pₜ(Wⱽcₜ) = Wⱽ(Σ pₜcₜ)</code>
+        <p>先对 512 维 latent 做加权和，再展开成各 head 的 value。含位置的 64 维 RoPE key 不能这样吸收，必须单独缓存。</p>
+      </article>
+    </div>
+
+    <div class="absorb-cache-flow">
+      <article class="naive">
+        <header><span>X / V3 NAIVE · BF16</span><b>{bytes(absorb.cache_accounting.naive_active_bytes)}</b></header>
+        <div>
+          <code>K [1, 26, 16, 192]</code>
+          <code>V [1, 26, 16, 128]</code>
+        </div>
+        <p>26 × 5,120 元素；与 HF eager 是同一种展开状态。</p>
+      </article>
+      <i aria-hidden="true">→</i>
+      <article class="absorbed">
+        <header><span>X / V3 ABSORB · BF16</span><b>{bytes(absorb.cache_accounting.absorb_active_bytes)}</b></header>
+        <div>
+          <code>latent [1, 26, 512]</code>
+          <code>RoPE [1, 26, 64]</code>
+        </div>
+        <p>26 × 576 元素；两个官方 buffer 的真实 active slice。</p>
+      </article>
+    </div>
+
+    <div class="absorb-metrics">
+      <article><span>REAL CACHE RATIO</span><b>{absorb.cache_accounting.naive_over_absorb_ratio.toFixed(4)}×</b><p>展开 K/V ÷ latent + RoPE</p></article>
+      <article><span>BF16 MAX |Δ|</span><b>{absorb.correctness.v3_naive_vs_absorb_bfloat16.max_abs}</b><p>官方 V3 naive ↔ absorb decode</p></article>
+      <article><span>FP32 MAX |Δ|</span><b>{scientific(absorb.correctness.v3_naive_vs_absorb_float32.max_abs)}</b><p>同一 BF16 权重转 FP32 做代数审计</p></article>
+      <article class="exact"><span>INDEPENDENT RERUN</span><b>{absorbExact ? "BYTE-EXACT" : "MISMATCH"}</b><p>完整 JSON SHA-256 6b4c714a…e63d</p></article>
+    </div>
+
+    <div class="precision-lens">
+      <div>
+        <span>HF EAGER ↔ V3 NAIVE / BF16</span>
+        <i><em style={`--error:${absorb.correctness.hf_eager_vs_v3_naive_bfloat16.max_abs / .004}`}></em></i>
+        <b>max {absorb.correctness.hf_eager_vs_v3_naive_bfloat16.max_abs}</b>
+      </div>
+      <div>
+        <span>V3 NAIVE ↔ ABSORB / BF16</span>
+        <i><em style={`--error:${absorb.correctness.v3_naive_vs_absorb_bfloat16.max_abs / .004}`}></em></i>
+        <b>max {absorb.correctness.v3_naive_vs_absorb_bfloat16.max_abs}</b>
+      </div>
+      <div>
+        <span>V3 NAIVE ↔ ABSORB / FP32</span>
+        <i><em style={`--error:${absorb.correctness.v3_naive_vs_absorb_float32.max_abs / .004}`}></em></i>
+        <b>max {scientific(absorb.correctness.v3_naive_vs_absorb_float32.max_abs)}</b>
+      </div>
+      <p>
+        三组输出均 finite。FP32 不是新的 checkpoint 精度，而是把同一组 BF16 权重提升后，
+        隔离矩阵乘法顺序造成的舍入误差。
+      </p>
+    </div>
+
+    <div class="kernel-contract">
+      <div>
+        <span>OFFICIAL FLASHMLA / PINNED {absorb.flashmla_boundary.official_revision.slice(0, 8)}</span>
+        <h5>算法路径已执行，不等于优化 kernel 已执行</h5>
+        <p>官方支持矩阵与编译目标只覆盖 SM90 / SM100；本机 RTX 5090 是 SM120。</p>
+      </div>
+      <div class="kernel-matrix" role="table" aria-label="FlashMLA 官方架构支持矩阵">
+        <div class="head" role="row"><b>KERNEL</b><b>SM90</b><b>SM100</b><b>LOCAL SM120</b></div>
+        <div role="row"><span>dense decode</span><i class="yes">支持</i><i>—</i><i class="no">未支持</i></div>
+        <div role="row"><span>sparse decode</span><i class="yes">支持</i><i class="yes">支持</i><i class="no">未支持</i></div>
+        <div role="row"><span>dense prefill</span><i>—</i><i class="yes">支持</i><i class="no">未支持</i></div>
+        <div role="row"><span>sparse prefill</span><i class="yes">支持</i><i class="yes">支持</i><i class="no">未支持</i></div>
+      </div>
+    </div>
+
+    <div class="execution-split">
+      <article><span>EXECUTED</span><b>官方 V3 pure-PyTorch absorb</b><p>真实权重、真实 latent / RoPE buffers、真实 incremental decode。</p></article>
+      <article><span>BUILD ATTEMPTS</span><b>2 个环境，均未产出 wheel</b><p>host 编译器边界；隔离 CUDA 13 环境缺少 <code>cuda/std/utility</code>。</p></article>
+      <article class="blocked"><span>NOT EXECUTED</span><b>FlashMLA optimized kernel</b><p>源码只生成 <code>sm_90a</code> / <code>sm_100f</code>，dense decode 运行时还拒绝非 SM90a。</p></article>
+    </div>
+
+    <div class="evidence-links">
+      <a href="https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py" rel="noreferrer">官方 V3 absorb 参考实现 ↗</a>
+      <a href="https://github.com/deepseek-ai/FlashMLA" rel="noreferrer">官方 FlashMLA 支持矩阵 ↗</a>
+      <a href="https://arxiv.org/abs/2405.04434" rel="noreferrer">DeepSeek-V2 MLA 报告 ↗</a>
+    </div>
+
+    <div class="artifact-boundary">
+      <b>CORRECT CLAIM</b>
+      <p>
+        本实验复现的是“真实权重 + 官方参考实现中的压缩缓存与 decode 等价性”，不是 FlashMLA 性能、
+        生产 serving 吞吐或 V2 论文完整模型相对 MHA 的 93.3% 缓存降幅。
+      </p>
    </div>
  </section>

@@ -328,15 +447,16 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi

    <div class="artifact-boundary">
      <b>U / STILL OPEN</b>
-      <p>完整 27 层生成、真实 latent-cache kernel、生产服务、训练负载、FP8/pipeline 与 R1-like 训练 trace 仍未覆盖。</p>
+      <p>完整 27 层生成、受支持硬件上的 FlashMLA 优化 kernel、生产服务、训练负载、FP8/pipeline 与 R1-like 训练 trace 仍未覆盖。</p>
    </div>
  </section>

  <figcaption>
    <span>可复现入口</span>
    <code>experiments/deepseek/v2_lite_trace.py</code> ·
+    <code>experiments/deepseek/v2_lite_absorb_probe.py</code> ·
    <code>src/data/deepseek-v2-lite-trace.json</code> ·
-    <code>research/DEEPSEEK_V2_LITE_TRACE.md</code>
+    <code>research/DEEPSEEK_MLA_ABSORB_AUDIT.md</code>
  </figcaption>

  <script is:inline type="application/json" data-dsv2-trace set:html={compactJson}></script>
@@ -575,6 +695,11 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
  .jaccard-block span,
  .observed-cache span,
  .artifact-identity span,
+  .absorb-algebra span,
+  .absorb-cache-flow span,
+  .absorb-metrics span,
+  .kernel-contract span,
+  .execution-split span,
  .repro-gate span,
  .dependency-split span,
  .checksum-grid span {
@@ -631,7 +756,7 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
  .artifact-status b { color: var(--ink); font-size: .72rem; }
  .artifact-tabs {
    display: grid;
-    grid-template-columns: repeat(4, 1fr);
+    grid-template-columns: repeat(5, 1fr);
    background: var(--ink);
  }
  .artifact-tabs button {
@@ -792,6 +917,7 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
  .route-metrics,
  .cache-ratio,
  .load-lessons,
+  .absorb-metrics,
  .artifact-identity,
  .checksum-grid {
    display: grid;
@@ -802,6 +928,7 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
  .route-metrics article,
  .cache-ratio article,
  .load-lessons article,
+  .absorb-metrics article,
  .artifact-identity article,
  .checksum-grid article {
    padding: .8rem;
@@ -810,20 +937,24 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
  .route-metrics article:last-child,
  .cache-ratio article:last-child,
  .load-lessons article:last-child,
+  .absorb-metrics article:last-child,
  .artifact-identity article:last-child,
  .checksum-grid article:last-child { border-right: 0; }
  .route-metrics span,
  .cache-ratio span,
-  .load-lessons span {
+  .load-lessons span,
+  .absorb-metrics span {
    display: block;
    color: var(--blue);
    font: 700 .61rem/1 var(--font-mono);
  }
  .route-metrics b,
-  .cache-ratio b { display: block; margin-top: .35rem; font: 750 1.25rem/1 var(--font-display); }
+  .cache-ratio b,
+  .absorb-metrics b { display: block; margin-top: .35rem; font: 750 1.25rem/1 var(--font-display); }
  .route-metrics p,
  .cache-ratio p,
-  .load-lessons p { margin: .3rem 0 0; color: rgba(32,32,39,.56); font-size: .65rem; line-height: 1.45; }
+  .load-lessons p,
+  .absorb-metrics p { margin: .3rem 0 0; color: rgba(32,32,39,.56); font-size: .65rem; line-height: 1.45; }
  .artifact-boundary {
    display: grid;
    grid-template-columns: 11rem 1fr;
@@ -950,6 +1081,127 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
  .cache-ledger p { margin: .35rem 0 0; color: rgba(32,32,39,.58); font-size: .68rem; line-height: 1.45; }
  .cache-ratio { grid-template-columns: repeat(3, 1fr); }
  .cache-ratio .warning { background: rgba(186,118,44,.09); }
+  .absorb-algebra {
+    display: grid;
+    grid-template-columns: 1fr 1fr;
+    gap: .8rem;
+  }
+  .absorb-algebra article {
+    padding: 1rem;
+    border: 1px solid rgba(32,32,39,.15);
+    background: #fffdf8;
+  }
+  .absorb-algebra code {
+    display: block;
+    margin: .75rem 0;
+    color: var(--ink);
+    font: 750 clamp(.88rem, 1.7vw, 1.2rem)/1.2 var(--font-mono);
+  }
+  .absorb-algebra p,
+  .absorb-cache-flow p,
+  .kernel-contract p,
+  .execution-split p {
+    margin: 0;
+    color: rgba(32,32,39,.62);
+    font-size: .69rem;
+    line-height: 1.55;
+  }
+  .absorb-cache-flow {
+    display: grid;
+    grid-template-columns: 1fr auto 1fr;
+    gap: .8rem;
+    align-items: center;
+    margin-top: .8rem;
+  }
+  .absorb-cache-flow > i { color: var(--teal); font: 800 1.5rem/1 var(--font-display); }
+  .absorb-cache-flow article {
+    padding: 1rem;
+    border: 1px solid rgba(32,32,39,.15);
+  }
+  .absorb-cache-flow article.naive { background: rgba(161,77,77,.07); }
+  .absorb-cache-flow article.absorbed { background: rgba(57,120,110,.09); }
+  .absorb-cache-flow header { display: flex; justify-content: space-between; gap: 1rem; align-items: center; }
+  .absorb-cache-flow header b { font: 760 1.25rem/1 var(--font-display); }
+  .absorb-cache-flow div { display: flex; flex-wrap: wrap; gap: .4rem; margin: .75rem 0; }
+  .absorb-cache-flow code {
+    padding: .35rem .45rem;
+    background: rgba(32,32,39,.08);
+    font-size: .66rem;
+  }
+  .absorb-metrics { grid-template-columns: repeat(4, 1fr); }
+  .absorb-metrics article.exact { background: rgba(57,120,110,.1); }
+  .precision-lens {
+    display: grid;
+    gap: .55rem;
+    margin-top: .8rem;
+    padding: 1rem;
+    background: var(--ink);
+    color: white;
+  }
+  .precision-lens > div {
+    display: grid;
+    grid-template-columns: minmax(13rem, 1.3fr) 1fr minmax(8rem, auto);
+    gap: .8rem;
+    align-items: center;
+  }
+  .precision-lens span,
+  .precision-lens b { font: 650 .62rem/1.3 var(--font-mono); }
+  .precision-lens b { text-align: right; }
+  .precision-lens i { height: 7px; overflow: hidden; background: rgba(255,255,255,.13); }
+  .precision-lens em {
+    display: block;
+    width: max(calc(var(--error) * 100%), 2px);
+    max-width: 100%;
+    height: 100%;
+    background: #b7bce5;
+  }
+  .precision-lens > div:last-of-type em { background: #91cfc1; }
+  .precision-lens > p {
+    margin: .25rem 0 0;
+    padding-top: .7rem;
+    border-top: 1px solid rgba(255,255,255,.15);
+    color: rgba(255,255,255,.58);
+    font-size: .66rem;
+    line-height: 1.5;
+  }
+  .kernel-contract {
+    display: grid;
+    grid-template-columns: .7fr 1.3fr;
+    gap: 1rem;
+    margin-top: .8rem;
+    padding: 1rem;
+    border: 1px solid rgba(32,32,39,.15);
+    background: #e6dfd3;
+  }
+  .kernel-contract h5 { margin: .45rem 0; font: 720 1rem/1.15 var(--font-display); }
+  .kernel-matrix { overflow-x: auto; border: 1px solid rgba(32,32,39,.14); background: #fffdf8; }
+  .kernel-matrix > div {
+    display: grid;
+    grid-template-columns: 1.4fr repeat(3, minmax(5rem, .7fr));
+    min-width: 520px;
+  }
+  .kernel-matrix b,
+  .kernel-matrix span,
+  .kernel-matrix i {
+    padding: .55rem;
+    border-right: 1px solid rgba(32,32,39,.1);
+    border-bottom: 1px solid rgba(32,32,39,.1);
+    font: 650 .61rem/1.2 var(--font-mono);
+  }
+  .kernel-matrix .head { background: var(--ink); color: white; }
+  .kernel-matrix i { color: rgba(32,32,39,.45); font-style: normal; text-align: center; }
+  .kernel-matrix i.yes { color: var(--teal); background: rgba(57,120,110,.08); }
+  .kernel-matrix i.no { color: var(--red); background: rgba(161,77,77,.08); }
+  .execution-split {
+    display: grid;
+    grid-template-columns: repeat(3, 1fr);
+    margin-top: .8rem;
+    border: 1px solid rgba(32,32,39,.14);
+  }
+  .execution-split article { padding: .9rem; border-right: 1px solid rgba(32,32,39,.12); }
+  .execution-split article:last-child { border-right: 0; }
+  .execution-split article.blocked { background: rgba(161,77,77,.08); }
+  .execution-split b { display: block; margin: .4rem 0; font-size: .77rem; }
  .artifact-identity { grid-template-columns: repeat(3, 1fr); }
  .artifact-identity article { display: grid; gap: .35rem; }
  .artifact-identity b { font-size: .78rem; }
@@ -1030,7 +1282,8 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    .panel-lead,
    .heat-head,
    .aggregate-card,
-    .jaccard-block { grid-template-columns: 1fr; }
+    .jaccard-block,
+    .kernel-contract { grid-template-columns: 1fr; }
    .artifact-status { grid-template-columns: 1fr 1fr; }
    .artifact-tabs { grid-template-columns: 1fr 1fr; }
    .route-controls { grid-template-columns: 1fr 1fr; }
@@ -1039,7 +1292,8 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    .route-stage > i { transform: rotate(90deg); justify-self: center; }
    .cache-controls { grid-template-columns: 1fr 1fr; }
    .route-metrics,
-    .checksum-grid { grid-template-columns: 1fr 1fr; }
+    .checksum-grid,
+    .absorb-metrics { grid-template-columns: 1fr 1fr; }
    .layer-evidence { grid-template-columns: repeat(9, 1fr); }
    .repro-gate { grid-template-columns: 1fr 1fr; }
    .repro-gate > p { grid-column: 1 / -1; padding: .8rem 0 0; border-left: 0; border-top: 1px solid rgba(255,255,255,.18); }
@@ -1062,20 +1316,28 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    .route-metrics,
    .cache-ratio,
    .load-lessons,
+    .absorb-metrics,
    .artifact-identity,
    .checksum-grid,
    .cache-ledger { grid-template-columns: 1fr; }
    .route-metrics article,
    .cache-ratio article,
    .load-lessons article,
+    .absorb-metrics article,
    .artifact-identity article,
    .checksum-grid article { border-right: 0; border-bottom: 1px solid rgba(32,32,39,.12); }
    .artifact-boundary { grid-template-columns: 1fr; }
    .load-dials { grid-template-columns: 1fr; }
    .observed-cache,
-    .dependency-split { grid-template-columns: 1fr; }
+    .dependency-split,
+    .absorb-algebra,
+    .absorb-cache-flow,
+    .execution-split { grid-template-columns: 1fr; }
    .observed-cache > i,
    .dependency-split > i { transform: rotate(90deg); justify-self: center; }
+    .absorb-cache-flow > i { transform: rotate(90deg); justify-self: center; }
+    .precision-lens > div { grid-template-columns: 1fr; }
+    .precision-lens b { text-align: left; }
    .layer-evidence { grid-template-columns: repeat(7, 1fr); }
    .repro-gate { grid-template-columns: 1fr; }
    .repro-gate > p { grid-column: auto; }
@@ -0,0 +1,439 @@
+{
+  "schema_version": 1,
+  "captured_at": "2026-07-29T06:35:00Z",
+  "evidence_identity": "X / official V2-Lite weights executed through official HF eager and V3 naive/absorb reference paths",
+  "boundary": {
+    "correctness_probe": true,
+    "serving_benchmark": false,
+    "full_model_generation": false,
+    "optimized_flashmla_kernel_executed": false,
+    "scope": "one real layer-1 attention input; 25-token prefill plus one-token decode"
+  },
+  "provenance": {
+    "huggingface_model": "deepseek-ai/DeepSeek-V2-Lite",
+    "huggingface_revision": "604d5664dddd88a0433dbae533b7fe9472482de0",
+    "deepseek_v3_repository": "deepseek-ai/DeepSeek-V3",
+    "deepseek_v3_revision": "9b4e9788e4a3a731f7567338ed15d3ec549ce03b",
+    "flashmla_repository": "deepseek-ai/FlashMLA",
+    "flashmla_revision": "15f13e5030374295491c5ce31b02d7e63a7772c6",
+    "sha256": {
+      "v2_config": "f346286b0f1c8b044252fd54cb4fa78b9fab6472a6e8bebb9edfe03d414ea03d",
+      "v2_modeling": "7d8e5221095286eea991137760893fd7ba52727c0b4ebf48ec09e8bc56b45b9c",
+      "v2_shard_1": "0d7e9f39bde40111a4c0f390b87497dce4565cf578d916395e6b2c7851f1e8da",
+      "v3_model": "fdb7995bc234badbf0011b5af8cf391be94343abc51305428cb9a04452116f02"
+    }
+  },
+  "environment": {
+    "python": "3.10.14",
+    "platform": "Linux-7.0.0-28-generic-x86_64-with-glibc2.43",
+    "libc": [
+      "glibc",
+      "2.43"
+    ],
+    "torch": "2.11.0+cu128",
+    "torch_cuda": "12.8",
+    "transformers": "4.41.2",
+    "safetensors": "0.8.0",
+    "device": "cuda",
+    "cuda_capability": [
+      12,
+      0
+    ],
+    "nvidia_smi": {
+      "name": "NVIDIA GeForce RTX 5090",
+      "driver_version": "595.84",
+      "memory.total": "32607",
+      "compute_cap": "12.0"
+    }
+  },
+  "input": {
+    "prompt": "用通俗的语言解释，为什么稀疏专家模型可以拥有很多参数，但每个 token 只使用其中一小部分。",
+    "token_ids": [
+      100000,
+      1082,
+      83983,
+      91476,
+      17882,
+      19304,
+      10285,
+      21844,
+      20923,
+      10900,
+      39322,
+      1876,
+      9476,
+      3599,
+      21273,
+      19304,
+      1705,
+      13712,
+      10728,
+      207,
+      2141,
+      5118,
+      6802,
+      51905,
+      5781,
+      398
+    ],
+    "tokens": [
+      "<｜begin▁of▁sentence｜>",
+      "çĶ¨",
+      "éĢļä¿Ĺ",
+      "çļĦè¯Ńè¨Ģ",
+      "è§£éĩĬ",
+      "ï¼Į",
+      "ä¸ºä»Ģä¹Ī",
+      "ç¨Ģ",
+      "çĸı",
+      "ä¸ĵå®¶",
+      "æ¨¡åŀĭ",
+      "åı¯ä»¥",
+      "æĭ¥æľī",
+      "å¾Īå¤ļ",
+      "åıĤæķ°",
+      "ï¼Į",
+      "ä½Ĩ",
+      "æ¯ıä¸ª",
+      "Ġtoken",
+      "Ġ",
+      "åıª",
+      "ä½¿çĶ¨",
+      "åħ¶ä¸Ń",
+      "ä¸Ģå°ı",
+      "éĥ¨åĪĨ",
+      "ãĢĤ"
+    ],
+    "sequence_tokens": 26,
+    "prefill_tokens": 25,
+    "decode_tokens": 1,
+    "layer0_executed": true,
+    "layer1_input_normalized": true
+  },
+  "configuration": {
+    "hidden": 2048,
+    "attention_heads": 16,
+    "qk_nope_head_dim": 128,
+    "qk_rope_head_dim": 64,
+    "v_head_dim": 128,
+    "kv_lora_rank": 512,
+    "cache_slots": 32
+  },
+  "execution": {
+    "huggingface_eager": {
+      "implementation": "huggingface_eager",
+      "dtype": "bfloat16",
+      "key_shape": [
+        1,
+        16,
+        26,
+        192
+      ],
+      "value_shape": [
+        1,
+        16,
+        26,
+        128
+      ],
+      "active_cache_bytes": 266240,
+      "decode_finite": true
+    },
+    "v3_naive_bfloat16": {
+      "implementation": "naive",
+      "dtype": "bfloat16",
+      "active_cache": {
+        "key": {
+          "shape": [
+            1,
+            26,
+            16,
+            192
+          ],
+          "bytes": 159744
+        },
+        "value": {
+          "shape": [
+            1,
+            26,
+            16,
+            128
+          ],
+          "bytes": 106496
+        }
+      },
+      "active_cache_bytes": 266240,
+      "allocated_cache": {
+        "key_shape": [
+          1,
+          32,
+          16,
+          192
+        ],
+        "key_bytes": 196608,
+        "value_shape": [
+          1,
+          32,
+          16,
+          128
+        ],
+        "value_bytes": 131072
+      },
+      "allocated_cache_bytes": 327680,
+      "decode_finite": true
+    },
+    "v3_absorb_bfloat16": {
+      "implementation": "absorb",
+      "dtype": "bfloat16",
+      "active_cache": {
+        "latent": {
+          "shape": [
+            1,
+            26,
+            512
+          ],
+          "bytes": 26624
+        },
+        "rope": {
+          "shape": [
+            1,
+            26,
+            64
+          ],
+          "bytes": 3328
+        }
+      },
+      "active_cache_bytes": 29952,
+      "allocated_cache": {
+        "latent_shape": [
+          1,
+          32,
+          512
+        ],
+        "latent_bytes": 32768,
+        "rope_shape": [
+          1,
+          32,
+          64
+        ],
+        "rope_bytes": 4096
+      },
+      "allocated_cache_bytes": 36864,
+      "decode_finite": true
+    },
+    "v3_naive_float32": {
+      "implementation": "naive",
+      "dtype": "float32",
+      "active_cache": {
+        "key": {
+          "shape": [
+            1,
+            26,
+            16,
+            192
+          ],
+          "bytes": 319488
+        },
+        "value": {
+          "shape": [
+            1,
+            26,
+            16,
+            128
+          ],
+          "bytes": 212992
+        }
+      },
+      "active_cache_bytes": 532480,
+      "allocated_cache": {
+        "key_shape": [
+          1,
+          32,
+          16,
+          192
+        ],
+        "key_bytes": 393216,
+        "value_shape": [
+          1,
+          32,
+          16,
+          128
+        ],
+        "value_bytes": 262144
+      },
+      "allocated_cache_bytes": 655360,
+      "decode_finite": true
+    },
+    "v3_absorb_float32": {
+      "implementation": "absorb",
+      "dtype": "float32",
+      "active_cache": {
+        "latent": {
+          "shape": [
+            1,
+            26,
+            512
+          ],
+          "bytes": 53248
+        },
+        "rope": {
+          "shape": [
+            1,
+            26,
+            64
+          ],
+          "bytes": 6656
+        }
+      },
+      "active_cache_bytes": 59904,
+      "allocated_cache": {
+        "latent_shape": [
+          1,
+          32,
+          512
+        ],
+        "latent_bytes": 65536,
+        "rope_shape": [
+          1,
+          32,
+          64
+        ],
+        "rope_bytes": 8192
+      },
+      "allocated_cache_bytes": 73728,
+      "decode_finite": true
+    }
+  },
+  "correctness": {
+    "hf_eager_vs_v3_naive_bfloat16": {
+      "shape": [
+        1,
+        1,
+        2048
+      ],
+      "finite": true,
+      "exact_fraction": 0.28466796875,
+      "max_abs": 0.001220703125,
+      "mean_abs": 8.882676775101572e-05,
+      "p50_abs": 6.103515625e-05,
+      "p90_abs": 0.000244140625,
+      "p99_abs": 0.00038238614797592163
+    },
+    "v3_naive_vs_absorb_bfloat16": {
+      "shape": [
+        1,
+        1,
+        2048
+      ],
+      "finite": true,
+      "exact_fraction": 0.10009765625,
+      "max_abs": 0.00390625,
+      "mean_abs": 0.00027031131321564317,
+      "p50_abs": 0.000244140625,
+      "p90_abs": 0.00054931640625,
+      "p99_abs": 0.0009236149489879608
+    },
+    "hf_eager_vs_v3_absorb_bfloat16": {
+      "shape": [
+        1,
+        1,
+        2048
+      ],
+      "finite": true,
+      "exact_fraction": 0.1044921875,
+      "max_abs": 0.00390625,
+      "mean_abs": 0.00027483145822770894,
+      "p50_abs": 0.000244140625,
+      "p90_abs": 0.00054931640625,
+      "p99_abs": 0.0009765625
+    },
+    "v3_naive_vs_absorb_float32": {
+      "shape": [
+        1,
+        1,
+        2048
+      ],
+      "finite": true,
+      "exact_fraction": 0.0556640625,
+      "max_abs": 1.1920928955078125e-07,
+      "mean_abs": 9.464486083743395e-09,
+      "p50_abs": 7.450580596923828e-09,
+      "p90_abs": 1.955777406692505e-08,
+      "p99_abs": 3.144612037431216e-08
+    }
+  },
+  "cache_accounting": {
+    "active_tokens": 26,
+    "dtype": "BF16",
+    "naive_active_bytes": 266240,
+    "absorb_active_bytes": 29952,
+    "naive_elements_per_token": 5120,
+    "absorb_elements_per_token": 576,
+    "naive_over_absorb_ratio": 8.88888888888889,
+    "absorb_reduction_vs_naive": 0.8875
+  },
+  "flashmla_boundary": {
+    "official_revision": "15f13e5030374295491c5ce31b02d7e63a7772c6",
+    "cutlass_revision": "147f5673d0c1c3dcf66f78d677fd647e4a020219",
+    "official_support_matrix": [
+      {
+        "kernel": "dense_decode",
+        "architectures": [
+          "SM90"
+        ],
+        "mode": "MQA",
+        "cache": "BF16"
+      },
+      {
+        "kernel": "sparse_decode",
+        "architectures": [
+          "SM90",
+          "SM100"
+        ],
+        "mode": "MQA",
+        "cache": "FP8"
+      },
+      {
+        "kernel": "dense_prefill",
+        "architectures": [
+          "SM100"
+        ],
+        "mode": "MHA",
+        "cache": null
+      },
+      {
+        "kernel": "sparse_prefill",
+        "architectures": [
+          "SM90",
+          "SM100"
+        ],
+        "mode": "MQA",
+        "cache": null
+      }
+    ],
+    "compiled_gencode": [
+      "sm_100f",
+      "sm_90a"
+    ],
+    "dense_decode_runtime_guard": "Dense decode MLA is only supported on SM90a architecture",
+    "local_architecture": "SM120",
+    "optimized_kernel_executed": false,
+    "why_not": "The pinned official source does not list or generate SM120 kernels; dense decode additionally rejects non-SM90a devices.",
+    "build_attempts": [
+      {
+        "environment": "host CUDA 12.8 / g++ 15.2",
+        "result": "failed before compilation",
+        "evidence": "CUDA 12.8 rejects host compiler versions newer than g++ 13; g++-13 is not installed."
+      },
+      {
+        "environment": "isolated CUDA 13.0.2 / torch 2.11.0+cu130 / g++ 13.3",
+        "result": "failed during official extension compilation",
+        "evidence": "csrc/api.cpp could not find cuda/std/utility; no wheel or runtime kernel was produced."
+      }
+    ],
+    "source_sha256": {
+      "readme": "46c001c53800f47148918f4fc2969102492dd5164f710fc015db1622f6f9929a",
+      "setup": "b493b648e4f718e50dd323775834ab53703b155aee6f639e7c812b9e8a2ce48d",
+      "dense_decode_header": "f3b012556eca4c8469bc99f6e299785793e6e6fe4aef090789501696e68e860f"
+    }
+  }
+}
@@ -0,0 +1,439 @@
+{
+  "schema_version": 1,
+  "captured_at": "2026-07-29T06:35:00Z",
+  "evidence_identity": "X / official V2-Lite weights executed through official HF eager and V3 naive/absorb reference paths",
+  "boundary": {
+    "correctness_probe": true,
+    "serving_benchmark": false,
+    "full_model_generation": false,
+    "optimized_flashmla_kernel_executed": false,
+    "scope": "one real layer-1 attention input; 25-token prefill plus one-token decode"
+  },
+  "provenance": {
+    "huggingface_model": "deepseek-ai/DeepSeek-V2-Lite",
+    "huggingface_revision": "604d5664dddd88a0433dbae533b7fe9472482de0",
+    "deepseek_v3_repository": "deepseek-ai/DeepSeek-V3",
+    "deepseek_v3_revision": "9b4e9788e4a3a731f7567338ed15d3ec549ce03b",
+    "flashmla_repository": "deepseek-ai/FlashMLA",
+    "flashmla_revision": "15f13e5030374295491c5ce31b02d7e63a7772c6",
+    "sha256": {
+      "v2_config": "f346286b0f1c8b044252fd54cb4fa78b9fab6472a6e8bebb9edfe03d414ea03d",
+      "v2_modeling": "7d8e5221095286eea991137760893fd7ba52727c0b4ebf48ec09e8bc56b45b9c",
+      "v2_shard_1": "0d7e9f39bde40111a4c0f390b87497dce4565cf578d916395e6b2c7851f1e8da",
+      "v3_model": "fdb7995bc234badbf0011b5af8cf391be94343abc51305428cb9a04452116f02"
+    }
+  },
+  "environment": {
+    "python": "3.10.14",
+    "platform": "Linux-7.0.0-28-generic-x86_64-with-glibc2.43",
+    "libc": [
+      "glibc",
+      "2.43"
+    ],
+    "torch": "2.11.0+cu128",
+    "torch_cuda": "12.8",
+    "transformers": "4.41.2",
+    "safetensors": "0.8.0",
+    "device": "cuda",
+    "cuda_capability": [
+      12,
+      0
+    ],
+    "nvidia_smi": {
+      "name": "NVIDIA GeForce RTX 5090",
+      "driver_version": "595.84",
+      "memory.total": "32607",
+      "compute_cap": "12.0"
+    }
+  },
+  "input": {
+    "prompt": "用通俗的语言解释，为什么稀疏专家模型可以拥有很多参数，但每个 token 只使用其中一小部分。",
+    "token_ids": [
+      100000,
+      1082,
+      83983,
+      91476,
+      17882,
+      19304,
+      10285,
+      21844,
+      20923,
+      10900,
+      39322,
+      1876,
+      9476,
+      3599,
+      21273,
+      19304,
+      1705,
+      13712,
+      10728,
+      207,
+      2141,
+      5118,
+      6802,
+      51905,
+      5781,
+      398
+    ],
+    "tokens": [
+      "<｜begin▁of▁sentence｜>",
+      "çĶ¨",
+      "éĢļä¿Ĺ",
+      "çļĦè¯Ńè¨Ģ",
+      "è§£éĩĬ",
+      "ï¼Į",
+      "ä¸ºä»Ģä¹Ī",
+      "ç¨Ģ",
+      "çĸı",
+      "ä¸ĵå®¶",
+      "æ¨¡åŀĭ",
+      "åı¯ä»¥",
+      "æĭ¥æľī",
+      "å¾Īå¤ļ",
+      "åıĤæķ°",
+      "ï¼Į",
+      "ä½Ĩ",
+      "æ¯ıä¸ª",
+      "Ġtoken",
+      "Ġ",
+      "åıª",
+      "ä½¿çĶ¨",
+      "åħ¶ä¸Ń",
+      "ä¸Ģå°ı",
+      "éĥ¨åĪĨ",
+      "ãĢĤ"
+    ],
+    "sequence_tokens": 26,
+    "prefill_tokens": 25,
+    "decode_tokens": 1,
+    "layer0_executed": true,
+    "layer1_input_normalized": true
+  },
+  "configuration": {
+    "hidden": 2048,
+    "attention_heads": 16,
+    "qk_nope_head_dim": 128,
+    "qk_rope_head_dim": 64,
+    "v_head_dim": 128,
+    "kv_lora_rank": 512,
+    "cache_slots": 32
+  },
+  "execution": {
+    "huggingface_eager": {
+      "implementation": "huggingface_eager",
+      "dtype": "bfloat16",
+      "key_shape": [
+        1,
+        16,
+        26,
+        192
+      ],
+      "value_shape": [
+        1,
+        16,
+        26,
+        128
+      ],
+      "active_cache_bytes": 266240,
+      "decode_finite": true
+    },
+    "v3_naive_bfloat16": {
+      "implementation": "naive",
+      "dtype": "bfloat16",
+      "active_cache": {
+        "key": {
+          "shape": [
+            1,
+            26,
+            16,
+            192
+          ],
+          "bytes": 159744
+        },
+        "value": {
+          "shape": [
+            1,
+            26,
+            16,
+            128
+          ],
+          "bytes": 106496
+        }
+      },
+      "active_cache_bytes": 266240,
+      "allocated_cache": {
+        "key_shape": [
+          1,
+          32,
+          16,
+          192
+        ],
+        "key_bytes": 196608,
+        "value_shape": [
+          1,
+          32,
+          16,
+          128
+        ],
+        "value_bytes": 131072
+      },
+      "allocated_cache_bytes": 327680,
+      "decode_finite": true
+    },
+    "v3_absorb_bfloat16": {
+      "implementation": "absorb",
+      "dtype": "bfloat16",
+      "active_cache": {
+        "latent": {
+          "shape": [
+            1,
+            26,
+            512
+          ],
+          "bytes": 26624
+        },
+        "rope": {
+          "shape": [
+            1,
+            26,
+            64
+          ],
+          "bytes": 3328
+        }
+      },
+      "active_cache_bytes": 29952,
+      "allocated_cache": {
+        "latent_shape": [
+          1,
+          32,
+          512
+        ],
+        "latent_bytes": 32768,
+        "rope_shape": [
+          1,
+          32,
+          64
+        ],
+        "rope_bytes": 4096
+      },
+      "allocated_cache_bytes": 36864,
+      "decode_finite": true
+    },
+    "v3_naive_float32": {
+      "implementation": "naive",
+      "dtype": "float32",
+      "active_cache": {
+        "key": {
+          "shape": [
+            1,
+            26,
+            16,
+            192
+          ],
+          "bytes": 319488
+        },
+        "value": {
+          "shape": [
+            1,
+            26,
+            16,
+            128
+          ],
+          "bytes": 212992
+        }
+      },
+      "active_cache_bytes": 532480,
+      "allocated_cache": {
+        "key_shape": [
+          1,
+          32,
+          16,
+          192
+        ],
+        "key_bytes": 393216,
+        "value_shape": [
+          1,
+          32,
+          16,
+          128
+        ],
+        "value_bytes": 262144
+      },
+      "allocated_cache_bytes": 655360,
+      "decode_finite": true
+    },
+    "v3_absorb_float32": {
+      "implementation": "absorb",
+      "dtype": "float32",
+      "active_cache": {
+        "latent": {
+          "shape": [
+            1,
+            26,
+            512
+          ],
+          "bytes": 53248
+        },
+        "rope": {
+          "shape": [
+            1,
+            26,
+            64
+          ],
+          "bytes": 6656
+        }
+      },
+      "active_cache_bytes": 59904,
+      "allocated_cache": {
+        "latent_shape": [
+          1,
+          32,
+          512
+        ],
+        "latent_bytes": 65536,
+        "rope_shape": [
+          1,
+          32,
+          64
+        ],
+        "rope_bytes": 8192
+      },
+      "allocated_cache_bytes": 73728,
+      "decode_finite": true
+    }
+  },
+  "correctness": {
+    "hf_eager_vs_v3_naive_bfloat16": {
+      "shape": [
+        1,
+        1,
+        2048
+      ],
+      "finite": true,
+      "exact_fraction": 0.28466796875,
+      "max_abs": 0.001220703125,
+      "mean_abs": 8.882676775101572e-05,
+      "p50_abs": 6.103515625e-05,
+      "p90_abs": 0.000244140625,
+      "p99_abs": 0.00038238614797592163
+    },
+    "v3_naive_vs_absorb_bfloat16": {
+      "shape": [
+        1,
+        1,
+        2048
+      ],
+      "finite": true,
+      "exact_fraction": 0.10009765625,
+      "max_abs": 0.00390625,
+      "mean_abs": 0.00027031131321564317,
+      "p50_abs": 0.000244140625,
+      "p90_abs": 0.00054931640625,
+      "p99_abs": 0.0009236149489879608
+    },
+    "hf_eager_vs_v3_absorb_bfloat16": {
+      "shape": [
+        1,
+        1,
+        2048
+      ],
+      "finite": true,
+      "exact_fraction": 0.1044921875,
+      "max_abs": 0.00390625,
+      "mean_abs": 0.00027483145822770894,
+      "p50_abs": 0.000244140625,
+      "p90_abs": 0.00054931640625,
+      "p99_abs": 0.0009765625
+    },
+    "v3_naive_vs_absorb_float32": {
+      "shape": [
+        1,
+        1,
+        2048
+      ],
+      "finite": true,
+      "exact_fraction": 0.0556640625,
+      "max_abs": 1.1920928955078125e-07,
+      "mean_abs": 9.464486083743395e-09,
+      "p50_abs": 7.450580596923828e-09,
+      "p90_abs": 1.955777406692505e-08,
+      "p99_abs": 3.144612037431216e-08
+    }
+  },
+  "cache_accounting": {
+    "active_tokens": 26,
+    "dtype": "BF16",
+    "naive_active_bytes": 266240,
+    "absorb_active_bytes": 29952,
+    "naive_elements_per_token": 5120,
+    "absorb_elements_per_token": 576,
+    "naive_over_absorb_ratio": 8.88888888888889,
+    "absorb_reduction_vs_naive": 0.8875
+  },
+  "flashmla_boundary": {
+    "official_revision": "15f13e5030374295491c5ce31b02d7e63a7772c6",
+    "cutlass_revision": "147f5673d0c1c3dcf66f78d677fd647e4a020219",
+    "official_support_matrix": [
+      {
+        "kernel": "dense_decode",
+        "architectures": [
+          "SM90"
+        ],
+        "mode": "MQA",
+        "cache": "BF16"
+      },
+      {
+        "kernel": "sparse_decode",
+        "architectures": [
+          "SM90",
+          "SM100"
+        ],
+        "mode": "MQA",
+        "cache": "FP8"
+      },
+      {
+        "kernel": "dense_prefill",
+        "architectures": [
+          "SM100"
+        ],
+        "mode": "MHA",
+        "cache": null
+      },
+      {
+        "kernel": "sparse_prefill",
+        "architectures": [
+          "SM90",
+          "SM100"
+        ],
+        "mode": "MQA",
+        "cache": null
+      }
+    ],
+    "compiled_gencode": [
+      "sm_100f",
+      "sm_90a"
+    ],
+    "dense_decode_runtime_guard": "Dense decode MLA is only supported on SM90a architecture",
+    "local_architecture": "SM120",
+    "optimized_kernel_executed": false,
+    "why_not": "The pinned official source does not list or generate SM120 kernels; dense decode additionally rejects non-SM90a devices.",
+    "build_attempts": [
+      {
+        "environment": "host CUDA 12.8 / g++ 15.2",
+        "result": "failed before compilation",
+        "evidence": "CUDA 12.8 rejects host compiler versions newer than g++ 13; g++-13 is not installed."
+      },
+      {
+        "environment": "isolated CUDA 13.0.2 / torch 2.11.0+cu130 / g++ 13.3",
+        "result": "failed during official extension compilation",
+        "evidence": "csrc/api.cpp could not find cuda/std/utility; no wheel or runtime kernel was produced."
+      }
+    ],
+    "source_sha256": {
+      "readme": "46c001c53800f47148918f4fc2969102492dd5164f710fc015db1622f6f9929a",
+      "setup": "b493b648e4f718e50dd323775834ab53703b155aee6f639e7c812b9e8a2ce48d",
+      "dense_decode_header": "f3b012556eca4c8469bc99f6e299785793e6e6fe4aef090789501696e68e860f"
+    }
+  }
+}
@@ -37,7 +37,7 @@ const toc = [

 <BaseLayout
  title="DeepSeek 技术谱系与真实权重深读：从 Dense、MoE、MLA 到 R1 与 V4"
-  description="用二十四张问题账、十次技术转向、八个交互实验、真实 V2-Lite 权重 trace 和六十个一手节点，完整理解 DeepSeek 的 MoE、MLA、FP8、DualPipe、GRPO、R1、V3.2 与 V4。"
+  description="用二十四张问题账、十次技术转向、九个交互实验、真实 V2-Lite 权重与吸收式缓存 trace 和六十个一手节点，完整理解 DeepSeek 的 MoE、MLA、FP8、DualPipe、GRPO、R1、V3.2 与 V4。"
  section="deepseek"
 >
  <header class="page-hero deepseek-hero">
@@ -55,7 +55,7 @@ const toc = [
        <div><dt>SPAN</dt><dd>2024.01 → 2026.06</dd></div>
        <div><dt>LEDGERS</dt><dd>24 张问题账</dd></div>
        <div><dt>LINEAGE</dt><dd>10 次技术转向</dd></div>
-        <div><dt>LABS</dt><dd>8 个可操作实验</dd></div>
+        <div><dt>LABS</dt><dd>9 个可操作实验</dd></div>
        <div><dt>EVIDENCE</dt><dd>60 个一手 / 官方节点</dd></div>
        <div><dt>STATUS</dt><dd>三轮 · 真实权重执行</dd></div>
      </dl>
@@ -768,14 +768,14 @@ const toc = [
        <p class="eyebrow"><span>22</span> OFFICIAL WEIGHTS / EXECUTED</p>
        <h2>从“MLA 与 MoE 的概念”再往前一步：让官方 V2-Lite 权重真的跑起来</h2>
        <p class="lede">
-          前面的四联实验负责建立公式与角色合同；下面的四联工件实验固定官方 revision、tokenizer、
+          前面的四联实验负责建立公式与角色合同；下面的五联工件实验固定官方 revision、tokenizer、
          模型代码和 checkpoint 第一分片，在 RTX 5090 上连续执行 layer 0–6。它把真实观测、shape 推导、
-          实现差距和未覆盖范围放在同一张证据图里。
+          吸收式 latent cache、实现差距和未覆盖范围放在同一张证据图里。
        </p>
        <div class="artifact-callout">
          <article><span>X / FORWARD</span><b>7 / 27 layers</b><p>1 个 dense 层 + 6 个 MoE 层；layer 7 因跨分片停止。</p></article>
          <article><span>X / ROUTES</span><b>3,240</b><p>90 个有效 token × 6 层 × top-6 routed experts。</p></article>
-          <article><span>X + D / CACHE</span><b>576 ↔ 5,120</b><p>latent 合同与 HF eager 实际展开元素，两张账同时保留。</p></article>
+          <article><span>X / ABSORB CACHE</span><b>266,240 → 29,952 B</b><p>同一真实 layer-1 权重的 naive / absorb active buffers。</p></article>
          <article><span>X / RERUN</span><b>31 / 31 exact</b><p>hidden hashes、MLA shapes、loads 与全部 token routes。</p></article>
        </div>
        <DeepSeekArtifactLab />
@@ -15,7 +15,7 @@ const workstreams = [
  { label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
  { label: "Scaling Laws", value: 74, next: "加入真实拟合复现、置信区间与更多模型族对照" },
  { label: "数据工程与预训练配方", value: 73, next: "逐图精读 FineWeb / DCLM，加入真实去重与 mixture traces" },
-  { label: "DeepSeek 专题", value: 87, next: "真实 latent-cache kernel、更大样本负载、FP8/pipeline 与 R1-like RL 复现" },
+  { label: "DeepSeek 专题", value: 89, next: "SM90 FlashMLA 优化 kernel、更大样本负载、FP8/pipeline 与 R1-like RL 复现" },
  { label: "指令微调与人类偏好", value: 75, next: "加入真实偏好分歧样本、RM 长度偏置与 PPO/DPO 小模型复现" },
  { label: "推理与测试时扩展", value: 76, next: "真实模型采样曲线、PRM 案例与逐篇图表精读" },
  { label: "工具使用与长程 Agent", value: 74, next: "补真实环境 traces、cross-harness 对照、Agent RL 训练曲线与安全案例" },
@@ -97,12 +97,12 @@ const workstreams = [
      <article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
      <article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全，包含先修依赖和三条贯穿案例。</p></article>
      <article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
-      <article><span>✓</span><h3>七十五个原创交互视图</h3><p>K3 三轴图、八联报告实验与四联开放工件实验，DeepSeek 四联公式实验与四联真实权重实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
+      <article><span>✓</span><h3>七十六个原创交互视图</h3><p>K3 三轴图、八联报告实验与四联开放工件实验，DeepSeek 四联公式实验与五联真实权重实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
      <article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
      <article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
      <article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
      <article><span>✓</span><h3>表示、位置与残差高速公路深度专题</h3><p>二十张问题账、66 个一手节点、DeepSeek/Kimi 双谱系，以及 Token—位置—Norm—Residual/FFN 四联实验。</p></article>
-      <article><span>✓</span><h3>DeepSeek 三轮真实权重里程碑</h3><p>在二十四张问题账、十次转向与四联公式实验上，新增 V2-Lite 7/27 层连续 forward、3,240 次真实专家选择、MLA/eager cache 实现账与 31/31 exact 复跑四联实验。</p></article>
+      <article><span>✓</span><h3>DeepSeek 三轮真实权重里程碑</h3><p>在二十四张问题账、十次转向与四联公式实验上，新增 V2-Lite 7/27 层连续 forward、3,240 次真实专家选择、MLA/eager cache 实现账、31/31 exact 复跑，以及官方 V3 absorb 的 576 元素真实缓存五联实验。</p></article>
      <article><span>✓</span><h3>Kimi K3 技术报告二轮深读</h3><p>三十二张问题账、Figure 1–16 / Table 1–5 审计、100 节点阅读链，以及 Delta—Decay—AttnRes—LatentMoE—SiTU—QB—MOPD—Cache 八联实验。</p></article>
      <article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions，审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes；四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
      <article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel；6/6 官方参考逐元素相等，并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
@@ -134,7 +134,7 @@ const workstreams = [
    <div class="queue-table">
      <div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
      <div><span>P0</span><strong>K3 三轮</strong><p>开放权重 traces → FlashKDA / AttnRes / MoE 真实行为 → Figure 1–16 数值重绘与独立复现</p><em>运行证据 + 逐图复现</em></div>
-      <div><span>P0</span><strong>DeepSeek 三轮</strong><p>真实 latent-cache kernel / 更大负载样本 → FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
+      <div><span>P0</span><strong>DeepSeek 三轮</strong><p>SM90 FlashMLA 优化 kernel / 更大负载样本 → FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
      <div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
      <div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
      <div><span>P0</span><strong>语言模型前史二轮</strong><p>Kneser–Ney / LSTM / Bahdanau 逐图 → 真实小语料复现 → tokenizer 公平性</p><em>可复现实验 + 逐图笔记</em></div>