feat: add bootstrapped DeepSeek routing corpus

2026-07-29 15:32:03 +08:00
parent b7fef2350f
commit 5bcfd58452
12 changed files with 158485 additions and 32 deletions
@@ -14,7 +14,7 @@
 | 表示、位置与残差高速公路 | 完成首版 | 81% | 真实 hidden-state / norm traces、长上下文位置外推与深层稳定性消融 |
 | Scaling Laws | 完成首版 | 74% | 真实拟合复现、置信区间与更多模型族对照 |
 | 数据工程与预训练配方 | 完成首版 | 73% | FineWeb / DCLM 逐图精读、真实去重误伤与 mixture traces |
-| DeepSeek 专题 | 三轮实证进行中 | 89% | SM90 FlashMLA 优化 kernel、更大样本负载、FP8/pipeline 与 R1-like RL 复现 |
+| DeepSeek 专题 | 三轮实证进行中 | 91% | SM90 FlashMLA kernel、完整 27 层、长度匹配对照、FP8/pipeline 与 R1-like RL 复现 |
 | 指令微调与人类偏好 | 完成首版 | 75% | 真实偏好分歧、RM 长度偏置与 PPO/DPO 小模型复现 |
 | 推理与测试时扩展 | 完成首版 | 76% | 真实模型采样曲线、PRM 案例与逐篇图表精读 |
 | 工具使用与长程 Agent | 完成首版 | 74% | 真实环境 traces、cross-harness 对照、Agent RL 曲线与安全案例 |
@@ -41,7 +41,7 @@
 - [x] 完成 486 篇关键论文索引，覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。
 - [x] 完成可检索、可按专题筛选的论文库页面。
 - [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。
- [x] 完成 K3 三轴架构、八联报告实验与四联开放工件实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 九联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等七十六个原创交互视图。
+- [x] 完成 K3 三轴架构、八联报告实验与四联开放工件实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 十联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等七十七个原创交互视图。
 - [x] 完成长上下文首版：五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
 - [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文，并建立长上下文研究账本。
 - [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文，并建立 MoE 研究账本。
@@ -179,11 +179,15 @@
 - [x] 独立吸收实验复跑 byte-exact：两份完整 JSON SHA-256 均为 `6b4c714a…e63d`；实验脚本、机器可读结果与正式研究账本进入开源树。
 - [x] FlashMLA 架构边界闭合：固定 revision `15f13e50…a7772c6` 与 CUTLASS submodule；官方矩阵、gencode 与 dense runtime guard 只覆盖 SM90/SM100，本机 SM120 不宣称优化 kernel 已执行；两次隔离构建失败位置如实入账。
 - [x] DeepSeek 吸收式缓存里程碑以源提交 `9ba26da`、不可变镜像 `20260729T065822Z-9ba26da` 发布；OCI digest `sha256:0184d7e8…905659`，复用 NAS `12010→8080`、NPM host 31 / cert 41、门户 `LLM ATLAS / projects / 180`，十六套生产 Chrome 回归全通过；保留 `20260729T062034Z-e864205` 回滚。
 - [x] DeepSeek 路由探针从 4 条手写 prompt 扩展为 4 个公开域：WikiText-2、TNEWS、HumanEval、GSM8K 各 32 条；固定 revision、原始文件 SHA-256、哈希选样 salt、source ID、tokenizer 与右截断合同。
 - [x] RTX 5090 执行 128 条 prompt、8,460 个有效 token；layer 1–6 每层产生 50,760 次真实 top-6 选择，六层共 304,560 次，逐 prompt 的 64 维 integer load 全量保留。
 - [x] 统计单位从 token 修正为 prompt：域内有放回重采样 2,000 次，固定 seed `20260729`，同时报告 token-weighted / prompt-balanced 的 CV、Gini、effective experts、top share、64 专家份额与两两 JSD 95% percentile 区间。
 - [x] 多域路由正式运行与独立重跑 byte-exact：两份 1.6 MiB JSON SHA-256 均为 `4678a1d1…a09e4`；六联交互实验可切换层与聚合口径，并永久注明不是训练分布、线上流量、专家语义或显著性检验。
 ## 正在进行
 - [ ] K3 三轮下一闸门：获得真实 token hidden states、expert load 与 cache traces，解释或修订 `A_log [128]` 工件冲突，再做 Figure 3/4/5 数值重绘和独立小模型复现。
- [ ] DeepSeek 三轮下一闸门：在官方支持的 SM90 环境执行 FlashMLA 优化 kernel、更大样本 expert-load 置信区间、FP8 / pipeline traces 与 R1-like RL 小模型复现。
+- [ ] DeepSeek 三轮下一闸门：在官方支持的 SM90 环境执行 FlashMLA 优化 kernel；扩到完整 27 层并做长度/tokenization 匹配对照，再推进 FP8 / pipeline traces 与 R1-like RL 小模型复现。
 - [ ] 表示、位置与残差二轮：真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。
 - [ ] 评测安全二轮：真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。
 - [ ] 推理服务二轮：真实 GPU kernel / workload traces、功耗与成本、跨 vLLM / SGLang / TensorRT-LLM 复现。
@@ -306,6 +310,9 @@
 | 2026-07-29 | MLA 576 元素缓存从算法合同升级为真实执行 | 官方 V3 naive / absorb 使用同一 V2-Lite 权重与 hidden state；26-token active cache 266,240 B→29,952 B，FP32 代数审计 max \|Δ\| 1.19e-7 |
 | 2026-07-29 | reference absorb 与 FlashMLA optimized kernel 永久分身份 | 前者已在 RTX 5090 执行；pinned FlashMLA 只列 SM90/SM100 并只生成 sm_90a/sm_100f，本机 SM120 明确标为未支持/未执行 |
 | 2026-07-29 | DeepSeek 吸收式缓存版本以 `20260729T065822Z-9ba26da` 发布 | OCI digest `sha256:0184d7e8…905659`；复用 `12010→8080`、NPM 31 / cert 41、门户 order 180；十六套生产 Chrome 回归通过，保留 `20260729T062034Z-e864205` 回滚 |
 | 2026-07-29 | 大样本路由使用四个可重建公开域 | WikiText-2 / TNEWS / HumanEval / GSM8K 各 32 条；只用 text / sentence / prompt / question，答案不输入、代码不执行 |
 | 2026-07-29 | 路由区间以 prompt 而非 token 为抽样单位 | 同 prompt token 有共同前缀与主题，不能伪装成独立样本；域内 2,000 次 bootstrap，同时保留 token 加权与 prompt 等权 |
 | 2026-07-29 | 路由分布差异与 expert semantics 永久分开 | Layer 4 中文、Layer 5/6 数学的集中度与域间 JSD 只描述 128-prompt 探针；不命名 expert，不冒充训练/线上总体或显著性检验 |
 | 2026-07-29 | K3 二轮按 32 张对象账与完整报告顺序重建 | total/active、2.5×、KDA state、深度来源、专家路由、视觉目标、轨迹、缓存与评测协议不再压成一页组件摘要 |
 | 2026-07-29 | K3 原生视觉事实回到 §2.4 / §3.3 核验 | 删除“先冻结语言模型再解冻”旧表述；明确 MoonViT-V2 从头训练，视觉/文本从开始共同 NTP |
 | 2026-07-29 | K3 Figure 1–16 / Table 1–5 全部建立课程视觉契约 | 每张图同时写支持范围与不可外推项；作者报告、论文、推导与 toy model 使用 R/P/D/T 标签 |
@@ -19,7 +19,7 @@
 当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读，
 语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化，以及评测与安全深度专题，
-以及 76 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
+以及 77 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
 8 个交互实验和 100 个一手/官方节点，完整覆盖架构、预训练、后训练、系统、评测、案例与附录。
 第三轮已完成开放工件与首个真实 kernel 里程碑：固定官方模型与 FlashKDA revisions，审计 96 个 checkpoint shards、
 497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计，并用 4 个新视图
@@ -28,13 +28,16 @@
 [K3_ARTIFACT_AUDIT.md](./research/K3_ARTIFACT_AUDIT.md) 与
 [checkpoint_probe.py](./experiments/k3/checkpoint_probe.py)、[FlashKDA probe](./experiments/k3/flashkda/)。
 DeepSeek 三轮专题以 24 张问题账、10 次技术转向、
-9 个交互实验和 60 个一手/官方节点，串起 Dense、MoE、MLA、V3 协同、R1 与 V4；
+10 个交互实验和 60 个一手/官方节点，串起 Dense、MoE、MLA、V3 协同、R1 与 V4；
 并固定官方 V2-Lite revision，在 RTX 5090 上连续执行 7/27 层，记录 3,240 次真实专家选择、
 MLA/HF eager cache shapes 与 `31/31` exact 独立复跑；进一步用真实 layer-1 权重执行官方 V3
 naive/absorb 路径，实际写入 576 元素 latent cache，并以 FP32 将两种结合顺序的最大误差压到
-`1.19e-7`。FlashMLA 的 SM90/SM100 官方支持矩阵与本机 SM120 边界单独记账。详见
+`1.19e-7`；再以 WikiText-2、TNEWS、HumanEval、GSM8K 各 32 条固定样本执行 304,560 次
 真实 top-6 路由，用 2,000 次 prompt-level 分层 bootstrap 同报点估计与 95% 区间，独立重跑
 整份 JSON byte-exact。FlashMLA 的 SM90/SM100 官方支持矩阵与本机 SM120 边界单独记账。详见
 [DEEPSEEK_V2_LITE_TRACE.md](./research/DEEPSEEK_V2_LITE_TRACE.md) 与
-[DEEPSEEK_MLA_ABSORB_AUDIT.md](./research/DEEPSEEK_MLA_ABSORB_AUDIT.md)。
+[DEEPSEEK_MLA_ABSORB_AUDIT.md](./research/DEEPSEEK_MLA_ABSORB_AUDIT.md)、
 [DEEPSEEK_ROUTING_CORPUS_AUDIT.md](./research/DEEPSEEK_ROUTING_CORPUS_AUDIT.md)。
 其余专题按进度账本持续扩建。
 ## 本地开发
@@ -7,6 +7,17 @@ server {
  charset utf-8;
  server_tokens off;
  gzip on;
  gzip_comp_level 6;
  gzip_min_length 1024;
  gzip_vary on;
  gzip_types
    text/plain
    text/css
    application/javascript
    application/json
    application/xml
    image/svg+xml;
  location = /healthz {
    access_log off;
@@ -54,3 +54,49 @@ python experiments/deepseek/compare_v2_lite_traces.py \
  --second /path/to/trace-2.json \
  --output src/data/deepseek-v2-lite-trace-repro.json
 ```
 ## Fixed public routing corpus
 `v2_lite_routing_corpus.py` keeps the same official layer 0–6 execution boundary
 but replaces the four authored prompts with 128 source-addressable public
 prompts:
 - 32 WikiText-2 raw validation passages;
 - 32 CLUE TNEWS public-test sentences;
 - 32 OpenAI HumanEval prompts, without solutions/tests or code execution;
 - 32 OpenAI GSM8K test questions, without answers.
 Selection is the ascending SHA-256 rank of a fixed salt, domain, and source ID.
 Inputs are truncated to 96 DeepSeek tokens. The six MoE layers therefore produce
 304,560 actual top-6 routed-expert selections over 8,460 valid tokens.
 The output includes both token-weighted and prompt-balanced distributions. Its
 95% intervals use 2,000 prompt-level bootstrap resamples within each domain,
 rather than treating correlated tokens as independent observations.
 ```bash
 PYTHONPATH=/path/to/transformers-4.41.2-deps \
 python -B experiments/deepseek/v2_lite_routing_corpus.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-corpus.json \
  --per-domain 32 \
  --max-tokens 96 \
  --batch-size 16 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T07:45:00+00:00
 ```
 The committed independent rerun is byte-exact. Both JSON files have SHA-256:
 ```text
 4678a1d15395de93ffba757598cc3642bf35e9e07f71d82ddd87c27fc38a09e4
 ```
 See `research/DEEPSEEK_ROUTING_CORPUS_AUDIT.md` for corpus revisions and hashes,
 metric definitions, interval semantics, results, and claim boundaries.
@@ -0,0 +1,819 @@
 #!/usr/bin/env python3
 """Run a fixed, multi-domain DeepSeek-V2-Lite routing corpus.
 The probe executes the official embedding and decoder layers 0 through 6 from
 the first official BF16 safetensors shard. Layers 1 through 6 are MoE layers.
 Their real top-6 routed-expert IDs are aggregated at prompt level before any
 statistics are computed.
 Sampling is deterministic and source-addressable. Confidence intervals use a
 prompt-level, within-domain bootstrap; tokens from the same prompt are never
 treated as independent observations.
 """
 from __future__ import annotations
 import argparse
 import gc
 import gzip
 import hashlib
 import importlib.util
 import json
 import math
 import platform
 import re
 import subprocess
 import sys
 import types
 from datetime import datetime, timezone
 from itertools import combinations
 from pathlib import Path
 from typing import Any
 import numpy as np
 import pyarrow.parquet as pq
 import torch
 import torch.nn.functional as F
 from safetensors import safe_open
 from transformers import AutoTokenizer
 DOMAIN_ORDER = ("english", "chinese", "code", "math")
 DOMAIN_LABELS = {
    "english": "English encyclopedia",
    "chinese": "中文新闻",
    "code": "Python code",
    "math": "Grade-school math",
 }
 MIN_TOKENS = {
    "english": 24,
    "chinese": 8,
    "code": 24,
    "math": 16,
 }
 SAMPLE_SALT = "llm-atlas-deepseek-routing-v1"
 def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser()
    parser.add_argument("--artifact-dir", type=Path, required=True)
    parser.add_argument("--human-eval", type=Path, required=True)
    parser.add_argument("--gsm8k", type=Path, required=True)
    parser.add_argument("--tnews", type=Path, required=True)
    parser.add_argument("--tnews-archive", type=Path, required=True)
    parser.add_argument("--wikitext", type=Path, required=True)
    parser.add_argument("--output", type=Path, required=True)
    parser.add_argument("--per-domain", type=int, default=32)
    parser.add_argument("--max-tokens", type=int, default=96)
    parser.add_argument("--batch-size", type=int, default=16)
    parser.add_argument("--layers", type=int, default=7)
    parser.add_argument("--bootstrap", type=int, default=2000)
    parser.add_argument("--seed", type=int, default=20260729)
    parser.add_argument("--device", default="cuda")
    parser.add_argument("--captured-at", default=None)
    return parser.parse_args()
 def sha256(path: Path) -> str:
    digest = hashlib.sha256()
    with path.open("rb") as handle:
        for block in iter(lambda: handle.read(1024 * 1024), b""):
            digest.update(block)
    return digest.hexdigest()
 def text_sha256(value: str) -> str:
    return hashlib.sha256(value.encode("utf-8")).hexdigest()
 def git_revision(path: Path) -> str | None:
    current = path.resolve()
    for candidate in (current, *current.parents):
        if (candidate / ".git").exists():
            return subprocess.check_output(
                ["git", "-C", str(candidate), "rev-parse", "HEAD"],
                text=True,
            ).strip()
    return None
 def scoped_seed(seed: int, scope: str) -> int:
    payload = f"{seed}:{scope}".encode()
    return int.from_bytes(hashlib.sha256(payload).digest()[:8], "big")
 def load_official_modules(root: Path) -> tuple[Any, Any]:
    package_name = "deepseek_v2_lite_official_routing_corpus"
    package = types.ModuleType(package_name)
    package.__path__ = [str(root)]
    sys.modules[package_name] = package
    loaded = {}
    for leaf in ("configuration_deepseek", "modeling_deepseek"):
        name = f"{package_name}.{leaf}"
        spec = importlib.util.spec_from_file_location(name, root / f"{leaf}.py")
        if spec is None or spec.loader is None:
            raise RuntimeError(f"cannot load official module: {leaf}")
        module = importlib.util.module_from_spec(spec)
        sys.modules[name] = module
        spec.loader.exec_module(module)
        loaded[leaf] = module
    return loaded["configuration_deepseek"], loaded["modeling_deepseek"]
 def read_jsonl(path: Path) -> list[dict[str, Any]]:
    with path.open(encoding="utf-8") as handle:
        return [json.loads(line) for line in handle if line.strip()]
 def load_candidates(args: argparse.Namespace) -> dict[str, list[dict[str, str]]]:
    with gzip.open(args.human_eval, "rt", encoding="utf-8") as handle:
        human_eval = [json.loads(line) for line in handle if line.strip()]
    gsm8k = read_jsonl(args.gsm8k)
    tnews = read_jsonl(args.tnews)
    wikitext = pq.read_table(args.wikitext, columns=["text"]).column("text").to_pylist()
    return {
        "english": [
            {
                "id": f"wikitext2/raw-validation/{index:04d}",
                "text": str(text).strip(),
            }
            for index, text in enumerate(wikitext)
            if str(text).strip()
            and not re.fullmatch(r"=+\s*.*?\s*=+", str(text).strip())
        ],
        "chinese": [
            {
                "id": f"tnews/test/{row['id']}",
                "text": str(row["sentence"]).strip(),
            }
            for row in tnews
            if str(row.get("sentence", "")).strip()
        ],
        "code": [
            {
                "id": str(row["task_id"]),
                "text": str(row["prompt"]).rstrip(),
            }
            for row in human_eval
            if str(row.get("prompt", "")).strip()
        ],
        "math": [
            {
                "id": f"gsm8k/test/{index:04d}",
                "text": str(row["question"]).strip(),
            }
            for index, row in enumerate(gsm8k)
            if str(row.get("question", "")).strip()
        ],
    }
 def select_corpus(
    tokenizer: Any,
    candidates: dict[str, list[dict[str, str]]],
    per_domain: int,
    max_tokens: int,
 ) -> tuple[list[dict[str, Any]], dict[str, dict[str, int]]]:
    selected: list[dict[str, Any]] = []
    counts: dict[str, dict[str, int]] = {}
    for domain in DOMAIN_ORDER:
        rows = candidates[domain]
        texts = [row["text"] for row in rows]
        encoded: list[list[int]] = []
        for start in range(0, len(texts), 512):
            result = tokenizer(
                texts[start : start + 512],
                add_special_tokens=True,
                truncation=True,
                max_length=max_tokens,
                padding=False,
            )
            encoded.extend(result.input_ids)
        eligible = []
        for row, token_ids in zip(rows, encoded, strict=True):
            if len(token_ids) < MIN_TOKENS[domain]:
                continue
            rank = hashlib.sha256(
                f"{SAMPLE_SALT}|{domain}|{row['id']}".encode()
            ).hexdigest()
            eligible.append(
                {
                    "id": row["id"],
                    "domain": domain,
                    "label": DOMAIN_LABELS[domain],
                    "text": row["text"],
                    "text_sha256": text_sha256(row["text"]),
                    "characters": len(row["text"]),
                    "token_ids": token_ids,
                    "tokens": len(token_ids),
                    "selection_rank": rank,
                }
            )
        eligible.sort(key=lambda row: (row["selection_rank"], row["id"]))
        if len(eligible) < per_domain:
            raise RuntimeError(
                f"{domain} has only {len(eligible)} eligible prompts; need {per_domain}"
            )
        domain_selection = eligible[:per_domain]
        for within_domain_index, row in enumerate(domain_selection):
            row["within_domain_index"] = within_domain_index
        selected.extend(domain_selection)
        counts[domain] = {
            "candidate_records_after_text_filter": len(rows),
            "eligible_records": len(eligible),
            "selected_records": len(domain_selection),
            "valid_tokens": sum(row["tokens"] for row in domain_selection),
        }
    return selected, counts
 def make_batches(
    samples: list[dict[str, Any]],
    batch_size: int,
    pad_token_id: int,
 ) -> list[dict[str, Any]]:
    ordered = sorted(
        samples,
        key=lambda row: (row["tokens"], row["domain"], row["id"]),
    )
    batches = []
    for start in range(0, len(ordered), batch_size):
        rows = ordered[start : start + batch_size]
        sequence = max(row["tokens"] for row in rows)
        input_ids = torch.full(
            (len(rows), sequence),
            pad_token_id,
            dtype=torch.long,
        )
        attention_mask = torch.zeros((len(rows), sequence), dtype=torch.long)
        for index, row in enumerate(rows):
            length = row["tokens"]
            input_ids[index, :length] = torch.tensor(row["token_ids"])
            attention_mask[index, :length] = 1
        batches.append(
            {
                "samples": rows,
                "input_ids": input_ids,
                "attention_mask": attention_mask,
                "padded_sequence": sequence,
            }
        )
    return batches
 def distribution(loads: np.ndarray, mode: str) -> np.ndarray:
    if mode == "token_weighted":
        values = loads.sum(axis=0, dtype=np.float64)
        return values / values.sum()
    if mode == "prompt_balanced":
        prompt_distributions = loads / loads.sum(axis=1, keepdims=True)
        values = prompt_distributions.mean(axis=0, dtype=np.float64)
        return values / values.sum()
    raise ValueError(mode)
 def metric_vector(distributions: np.ndarray) -> dict[str, np.ndarray]:
    values = np.atleast_2d(distributions).astype(np.float64, copy=False)
    expert_count = values.shape[1]
    means = values.mean(axis=1)
    cv = values.std(axis=1) / means
    ordered = np.sort(values, axis=1)
    indices = np.arange(1, expert_count + 1, dtype=np.float64)
    gini = (
        ((2 * indices - expert_count - 1) * ordered).sum(axis=1)
        / (expert_count * ordered.sum(axis=1))
    )
    log_values = np.zeros_like(values)
    np.log(values, out=log_values, where=values > 0)
    entropy = -(values * log_values).sum(axis=1)
    return {
        "cv": cv,
        "gini": gini,
        "effective_experts": np.exp(entropy),
        "top_expert_share": values.max(axis=1),
        "used_experts": (values > 0).sum(axis=1).astype(np.float64),
    }
 def interval(values: np.ndarray) -> list[float]:
    low, high = np.quantile(values, [0.025, 0.975], axis=0)
    if np.ndim(low) == 0:
        return [float(low), float(high)]
    return np.stack([low, high], axis=-1).tolist()
 def bootstrap_domain(
    loads: np.ndarray,
    mode: str,
    replicates: int,
    seed: int,
    scope: str,
 ) -> dict[str, Any]:
    point_distribution = distribution(loads, mode)
    prompt_count = loads.shape[0]
    rng = np.random.default_rng(scoped_seed(seed, scope))
    sampled = rng.integers(
        0,
        prompt_count,
        size=(replicates, prompt_count),
        endpoint=False,
    )
    if mode == "token_weighted":
        bootstrap_loads = loads[sampled].sum(axis=1, dtype=np.float64)
        bootstrap_distributions = bootstrap_loads / bootstrap_loads.sum(
            axis=1, keepdims=True
        )
    else:
        prompt_distributions = loads / loads.sum(axis=1, keepdims=True)
        bootstrap_distributions = prompt_distributions[sampled].mean(axis=1)
        bootstrap_distributions /= bootstrap_distributions.sum(axis=1, keepdims=True)
    point_metrics = metric_vector(point_distribution)
    bootstrap_metrics = metric_vector(bootstrap_distributions)
    metrics = {
        name: {
            "point": float(point_metrics[name][0]),
            "ci95": interval(bootstrap_metrics[name]),
        }
        for name in point_metrics
    }
    order = np.argsort(point_distribution)[::-1][:8]
    return {
        "distribution": point_distribution.tolist(),
        "expert_share_ci95": interval(bootstrap_distributions),
        "metrics": metrics,
        "top_experts": [
            {
                "expert": int(expert),
                "share": float(point_distribution[expert]),
                "ci95": interval(bootstrap_distributions[:, expert]),
            }
            for expert in order
        ],
    }
 def js_divergence(left: np.ndarray, right: np.ndarray) -> np.ndarray:
    left = np.atleast_2d(left).astype(np.float64, copy=False)
    right = np.atleast_2d(right).astype(np.float64, copy=False)
    midpoint = (left + right) / 2
    left_log_ratio = np.zeros_like(left)
    right_log_ratio = np.zeros_like(right)
    left_ratio = np.ones_like(left)
    right_ratio = np.ones_like(right)
    np.divide(left, midpoint, out=left_ratio, where=left > 0)
    np.divide(right, midpoint, out=right_ratio, where=right > 0)
    np.log(left_ratio, out=left_log_ratio, where=left > 0)
    np.log(right_ratio, out=right_log_ratio, where=right > 0)
    return 0.5 * (
        (left * left_log_ratio).sum(axis=1)
        + (right * right_log_ratio).sum(axis=1)
    )
 def bootstrap_pair(
    left: np.ndarray,
    right: np.ndarray,
    mode: str,
    replicates: int,
    seed: int,
    scope: str,
 ) -> dict[str, Any]:
    left_point = distribution(left, mode)
    right_point = distribution(right, mode)
    rng = np.random.default_rng(scoped_seed(seed, scope))
    left_indices = rng.integers(
        0, left.shape[0], size=(replicates, left.shape[0]), endpoint=False
    )
    right_indices = rng.integers(
        0, right.shape[0], size=(replicates, right.shape[0]), endpoint=False
    )
    if mode == "token_weighted":
        left_boot = left[left_indices].sum(axis=1, dtype=np.float64)
        right_boot = right[right_indices].sum(axis=1, dtype=np.float64)
        left_boot /= left_boot.sum(axis=1, keepdims=True)
        right_boot /= right_boot.sum(axis=1, keepdims=True)
    else:
        left_prompt = left / left.sum(axis=1, keepdims=True)
        right_prompt = right / right.sum(axis=1, keepdims=True)
        left_boot = left_prompt[left_indices].mean(axis=1)
        right_boot = right_prompt[right_indices].mean(axis=1)
        left_boot /= left_boot.sum(axis=1, keepdims=True)
        right_boot /= right_boot.sum(axis=1, keepdims=True)
    bootstrap_jsd = js_divergence(left_boot, right_boot)
    return {
        "point": float(js_divergence(left_point, right_point)[0]),
        "ci95": interval(bootstrap_jsd),
        "unit": "nats",
        "upper_bound": math.log(2),
    }
 def layer_statistics(
    prompt_rows: list[dict[str, Any]],
    replicates: int,
    seed: int,
    layer_index: int,
 ) -> dict[str, Any]:
    loads_by_domain = {
        domain: np.asarray(
            [row["load"] for row in prompt_rows if row["domain"] == domain],
            dtype=np.int64,
        )
        for domain in DOMAIN_ORDER
    }
    modes: dict[str, Any] = {}
    for mode in ("token_weighted", "prompt_balanced"):
        domains = {
            domain: bootstrap_domain(
                loads_by_domain[domain],
                mode,
                replicates,
                seed,
                f"layer={layer_index}|mode={mode}|domain={domain}",
            )
            for domain in DOMAIN_ORDER
        }
        pairs = []
        for left, right in combinations(DOMAIN_ORDER, 2):
            pairs.append(
                {
                    "left": left,
                    "right": right,
                    "js_divergence": bootstrap_pair(
                        loads_by_domain[left],
                        loads_by_domain[right],
                        mode,
                        replicates,
                        seed,
                        f"layer={layer_index}|mode={mode}|pair={left}:{right}",
                    ),
                }
            )
        modes[mode] = {"domains": domains, "pairs": pairs}
    return modes
 def gpu_identity(device: torch.device) -> dict[str, Any] | None:
    if device.type != "cuda":
        return None
    index = device.index or 0
    properties = torch.cuda.get_device_properties(index)
    driver = subprocess.check_output(
        [
            "nvidia-smi",
            "--query-gpu=driver_version",
            "--format=csv,noheader,nounits",
            f"--id={index}",
        ],
        text=True,
    ).strip()
    return {
        "name": properties.name,
        "total_memory_bytes": properties.total_memory,
        "compute_capability": list(torch.cuda.get_device_capability(index)),
        "driver": driver,
    }
 def main() -> None:
    args = parse_args()
    root = args.artifact_dir.resolve()
    shard = root / "model-00001-of-000004.safetensors"
    required = [
        root / "config.json",
        root / "configuration_deepseek.py",
        root / "modeling_deepseek.py",
        root / "model.safetensors.index.json",
        root / "tokenizer.json",
        shard,
        args.human_eval,
        args.gsm8k,
        args.tnews,
        args.tnews_archive,
        args.wikitext,
    ]
    missing = [str(path) for path in required if not path.exists()]
    if missing:
        raise FileNotFoundError(f"missing artifacts: {missing}")
    if args.device.startswith("cuda") and not torch.cuda.is_available():
        raise RuntimeError("CUDA requested but unavailable")
    if not 2 <= args.layers <= 7:
        raise ValueError("need layer 0 plus at least one MoE layer; shard ends at layer 6")
    if args.per_domain < 2:
        raise ValueError("per-domain sample must be at least two")
    if args.bootstrap < 100:
        raise ValueError("bootstrap replicates must be at least 100")
    torch.manual_seed(args.seed)
    torch.cuda.manual_seed_all(args.seed)
    torch.backends.cuda.matmul.allow_tf32 = False
    configuration, modeling = load_official_modules(root)
    config = configuration.DeepseekV2Config.from_pretrained(root)
    config._attn_implementation = "eager"
    tokenizer = AutoTokenizer.from_pretrained(
        root,
        trust_remote_code=True,
        local_files_only=True,
    )
    if tokenizer.pad_token_id is None:
        tokenizer.pad_token = tokenizer.eos_token
    tokenizer.padding_side = "right"
    candidates = load_candidates(args)
    samples, corpus_counts = select_corpus(
        tokenizer,
        candidates,
        args.per_domain,
        args.max_tokens,
    )
    batches = make_batches(samples, args.batch_size, tokenizer.pad_token_id)
    device = torch.device(args.device)
    with safe_open(shard, framework="pt", device="cpu") as handle:
        embedding = handle.get_tensor("model.embed_tokens.weight")
    for batch in batches:
        batch["hidden"] = F.embedding(batch["input_ids"], embedding)
    del embedding
    layer_results = []
    for layer_index in range(args.layers):
        prefix = f"model.layers.{layer_index}."
        with safe_open(shard, framework="pt", device="cpu") as handle:
            state = {
                key[len(prefix) :]: handle.get_tensor(key)
                for key in handle.keys()
                if key.startswith(prefix)
            }
        state_numel = sum(value.numel() for value in state.values())
        state_bytes = sum(value.numel() * value.element_size() for value in state.values())
        with torch.device("meta"):
            layer = modeling.DeepseekV2DecoderLayer(config, layer_index)
        layer.to_empty(device="cpu")
        layer.load_state_dict(state, strict=True, assign=True)
        del state
        layer = layer.to(device=device, dtype=torch.bfloat16).eval()
        prompt_rows = []
        next_hidden = []
        for batch in batches:
            hidden = batch["hidden"].to(device)
            attention_mask = batch["attention_mask"].to(device)
            sequence = batch["padded_sequence"]
            position_ids = torch.arange(sequence, device=device).unsqueeze(0)
            causal_mask = modeling._prepare_4d_causal_attention_mask(
                attention_mask,
                batch["input_ids"].shape,
                hidden,
                0,
            )
            captures: dict[str, torch.Tensor] = {}
            hook = None
            if layer_index > 0:
                def capture_gate(_module: Any, _inputs: Any, output: Any) -> None:
                    captures["topk_ids"] = output[0].detach().cpu()
                hook = layer.mlp.gate.register_forward_hook(capture_gate)
            with torch.inference_mode():
                outputs = layer(
                    hidden,
                    attention_mask=causal_mask,
                    position_ids=position_ids,
                    past_key_value=None,
                    use_cache=False,
                )
            next_hidden.append(outputs[0].cpu())
            if hook is not None:
                hook.remove()
                topk_ids = captures["topk_ids"].view(
                    len(batch["samples"]),
                    sequence,
                    config.num_experts_per_tok,
                )
                for sample_index, sample in enumerate(batch["samples"]):
                    length = sample["tokens"]
                    load = torch.bincount(
                        topk_ids[sample_index, :length].flatten(),
                        minlength=config.n_routed_experts,
                    )
                    prompt_rows.append(
                        {
                            "id": sample["id"],
                            "domain": sample["domain"],
                            "tokens": length,
                            "routes": int(load.sum()),
                            "load": load.tolist(),
                        }
                    )
            del hidden, attention_mask, causal_mask, outputs
        for batch, hidden in zip(batches, next_hidden, strict=True):
            batch["hidden"] = hidden
        result: dict[str, Any] = {
            "layer": layer_index,
            "ffn": "dense" if layer_index == 0 else "moe",
            "state_numel": state_numel,
            "state_bytes": state_bytes,
        }
        if layer_index > 0:
            prompt_rows.sort(
                key=lambda row: (
                    DOMAIN_ORDER.index(row["domain"]),
                    row["id"],
                )
            )
            result["valid_tokens"] = sum(row["tokens"] for row in prompt_rows)
            result["routes"] = sum(row["routes"] for row in prompt_rows)
            result["prompts"] = prompt_rows
            result["statistics"] = layer_statistics(
                prompt_rows,
                args.bootstrap,
                args.seed,
                layer_index,
            )
        layer_results.append(result)
        del layer, next_hidden
        gc.collect()
        if device.type == "cuda":
            torch.cuda.empty_cache()
    captured_at = args.captured_at or datetime.now(timezone.utc).isoformat()
    index = json.loads((root / "model.safetensors.index.json").read_text())
    selected_identity = [
        {
            "id": sample["id"],
            "domain": sample["domain"],
            "within_domain_index": sample["within_domain_index"],
            "selection_rank": sample["selection_rank"],
            "text_sha256": sample["text_sha256"],
            "characters": sample["characters"],
            "tokens": sample["tokens"],
        }
        for sample in samples
    ]
    result = {
        "schema_version": 1,
        "captured_at": captured_at,
        "evidence_identity": "X / official BF16 weights, fixed public corpus, local truncated forward",
        "boundary": {
            "model": "DeepSeek-V2-Lite base",
            "executed_layers": list(range(args.layers)),
            "measured_moe_layers": list(range(1, args.layers)),
            "total_model_layers": config.num_hidden_layers,
            "full_model_generation": False,
            "training_or_online_load": False,
            "expert_semantics_inferred": False,
            "population": (
                f"{len(samples)} fixed public prompts across four domains; "
                "not training data, online traffic, or a task-performance benchmark"
            ),
            "code_execution": False,
            "answers_used": False,
        },
        "provenance": {
            "model": {
                "huggingface_model": "deepseek-ai/DeepSeek-V2-Lite",
                "huggingface_revision": "604d5664dddd88a0433dbae533b7fe9472482de0",
                "sha256": {
                    "config": sha256(root / "config.json"),
                    "modeling_code": sha256(root / "modeling_deepseek.py"),
                    "tokenizer": sha256(root / "tokenizer.json"),
                    "index": sha256(root / "model.safetensors.index.json"),
                    "shard_1": sha256(shard),
                },
                "checkpoint_tensor_bytes": index["metadata"]["total_size"],
                "shard_1_bytes": shard.stat().st_size,
            },
            "corpora": {
                "english": {
                    "name": "WikiText-2 raw validation",
                    "url": "https://huggingface.co/datasets/Salesforce/wikitext",
                    "revision": "b08601e04326c79dfdd32d625aee71d232d685c3",
                    "file_sha256": sha256(args.wikitext),
                    "field_used": "text",
                },
                "chinese": {
                    "name": "CLUE TNEWS public test",
                    "url": "https://github.com/CLUEbenchmark/CLUE",
                    "download_url": "https://storage.googleapis.com/cluebenchmark/tasks/tnews_public.zip",
                    "revision": "9e61ddd3659ddb57ed82b4d0ba0a8613dfb55a2e",
                    "archive_sha256": sha256(args.tnews_archive),
                    "file_sha256": sha256(args.tnews),
                    "field_used": "sentence",
                },
                "code": {
                    "name": "OpenAI HumanEval",
                    "url": "https://github.com/openai/human-eval",
                    "revision": git_revision(args.human_eval),
                    "file_sha256": sha256(args.human_eval),
                    "field_used": "prompt",
                },
                "math": {
                    "name": "OpenAI GSM8K test",
                    "url": "https://github.com/openai/grade-school-math",
                    "revision": git_revision(args.gsm8k),
                    "file_sha256": sha256(args.gsm8k),
                    "field_used": "question",
                },
            },
        },
        "environment": {
            "python": platform.python_version(),
            "platform": platform.platform(),
            "torch": torch.__version__,
            "torch_cuda": torch.version.cuda,
            "transformers": __import__("transformers").__version__,
            "safetensors": __import__("safetensors").__version__,
            "numpy": np.__version__,
            "pyarrow": __import__("pyarrow").__version__,
            "device": str(device),
            "gpu": gpu_identity(device),
            "matmul_allow_tf32": torch.backends.cuda.matmul.allow_tf32,
        },
        "configuration": {
            "layers": config.num_hidden_layers,
            "hidden": config.hidden_size,
            "routed_experts": config.n_routed_experts,
            "active_routed_experts": config.num_experts_per_tok,
            "shared_experts": config.n_shared_experts,
            "first_dense_layers": config.first_k_dense_replace,
            "router_scoring": config.scoring_func,
            "router_topk_method": config.topk_method,
            "normalize_selected_weights": config.norm_topk_prob,
        },
        "corpus_contract": {
            "domains": list(DOMAIN_ORDER),
            "domain_labels": DOMAIN_LABELS,
            "sample_salt": SAMPLE_SALT,
            "selection": "ascending SHA256(salt|domain|source_id), then source_id",
            "per_domain": args.per_domain,
            "max_tokens": args.max_tokens,
            "minimum_tokens": MIN_TOKENS,
            "special_tokens": True,
            "truncation": "right",
            "counts": corpus_counts,
            "selected": selected_identity,
        },
        "inference_contract": {
            "batch_size": args.batch_size,
            "batches": len(batches),
            "batch_order": "ascending token count, then domain, then source id",
            "attention": "official eager",
            "dtype": "BF16",
            "total_prompts": len(samples),
            "valid_tokens": sum(sample["tokens"] for sample in samples),
            "routes_per_moe_layer": sum(sample["tokens"] for sample in samples)
            * config.num_experts_per_tok,
        },
        "statistical_contract": {
            "resampling_unit": "prompt",
            "strata": "domain",
            "replicates": args.bootstrap,
            "seed": args.seed,
            "interval": "95% percentile bootstrap",
            "modes": {
                "token_weighted": "sum route counts, so longer prompts contribute more",
                "prompt_balanced": "normalize each prompt first, then give every prompt equal weight",
            },
            "metrics": {
                "cv": "population standard deviation across 64 expert shares divided by their mean",
                "gini": "Gini coefficient across 64 expert shares",
                "effective_experts": "exp(Shannon entropy) in nats",
                "top_expert_share": "largest routed-expert share",
                "used_experts": "experts with nonzero share in the resample",
                "js_divergence": "Jensen-Shannon divergence in nats; bounded by ln(2)",
            },
            "multiple_comparison_correction": False,
        },
        "layers": layer_results,
    }
    args.output.parent.mkdir(parents=True, exist_ok=True)
    args.output.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n")
    print(
        json.dumps(
            {
                "output": str(args.output),
                "prompts": len(samples),
                "tokens": result["inference_contract"]["valid_tokens"],
                "routes_per_moe_layer": result["inference_contract"][
                    "routes_per_moe_layer"
                ],
                "measured_moe_layers": args.layers - 1,
            },
            ensure_ascii=False,
        )
    )
 if __name__ == "__main__":
    main()
@@ -0,0 +1,386 @@
 # DeepSeek-V2-Lite 多域路由语料审计
 > 状态：第三轮真实权重执行 / 固定公开语料 / prompt-level bootstrap  
 > 捕获时间：2026-07-29T07:45:00+00:00  
 > 证据身份：X（本机真实执行）+ O（官方工件）+ S（统计摘要）  
 > 输出：
 > `src/data/deepseek-v2-lite-routing-corpus.json` 与独立重跑
 > `src/data/deepseek-v2-lite-routing-corpus-repro.json`
 ## 1. 这轮到底补了什么
 上一轮 `v2_lite_trace.py` 的任务是做“显微镜”：4 条手写 prompt、90 个有效
 token，逐 token 保存 top-6 expert ID 与 gate weight。它适合解释 router 实际输出
 什么，却不适合回答“换一批文本，这个现象还在不在”。
 本轮把问题改成：
 1. 固定四种可公开重建的文本域；
 2. 每域固定 32 条，共 128 条 prompt；
 3. 对官方 DeepSeek-V2-Lite 的 embedding 与 layer 0–6 做连续 BF16 forward；
 4. 在 layer 1–6 捕获每个有效 token 的真实 top-6 routed-expert ID；
 5. 先聚合到 prompt，再以 prompt 为单位做 2,000 次分层 bootstrap；
 6. 同时报告点估计、95% 区间、两种聚合口径和明确停止线。
 正式运行包含：
 | 数量 | 值 |
 | --- | ---: |
 | prompt | 128 |
 | 有效 token | 8,460 |
 | 每个 MoE 层的 routed-expert 选择 | 50,760 |
 | 实测 MoE 层 | 6 |
 | routed-expert 选择总数 | **304,560** |
 | routed experts / layer | 64 |
 | active routed experts / token | 6 |
 | bootstrap replicates | 2,000 |
 这里的 304,560 是 `8,460 tokens × top-6 × 6 MoE layers`，不是模型参数量、
 训练 token 数或线上请求量。
 ## 2. 语料合同
 ### 2.1 四个公开来源
 | 域 | 公开来源 | 只使用的字段 | 固定 revision / 文件 SHA-256 |
 | --- | --- | --- | --- |
 | 英文百科 | [Salesforce WikiText-2 raw validation](https://huggingface.co/datasets/Salesforce/wikitext/tree/b08601e04326c79dfdd32d625aee71d232d685c3/wikitext-2-raw-v1) | `text` | revision `b08601e…685c3`; parquet `204929b7…11c4c` |
 | 中文新闻 | [CLUE TNEWS](https://github.com/CLUEbenchmark/CLUE/tree/9e61ddd3659ddb57ed82b4d0ba0a8613dfb55a2e) | `sentence` | repo `9e61ddd…55a2e`; archive `77c476e7…33462`; test `74f19932…a0ca8` |
 | Python 代码 | [OpenAI HumanEval](https://github.com/openai/human-eval/tree/6d43fb980f9fee3c892a914eda09951f772ad10d) | `prompt` | revision `6d43fb9…ad10d`; gzip `b796127e…979ef` |
 | 小学数学 | [OpenAI GSM8K](https://github.com/openai/grade-school-math/tree/3101c7d5072418e28b9008a6636bde82a006892c) | `question` | revision `3101c7d…6892c`; test JSONL `3730d312…d14` |
 边界：
 - HumanEval 的 `canonical_solution`、`test` 没有进入模型，也没有执行代码；
 - GSM8K 的 `answer` 没有进入模型；
 - TNEWS 的标签和关键词没有进入模型；
 - WikiText 只取 raw validation 的正文候选，不连接其他 split；
 - 这四个来源被当作“探针语料”，不是任务正确率评测。
 ### 2.2 确定性选样
 固定 salt：
 ```text
 llm-atlas-deepseek-routing-v1
 ```
 每域候选按下式升序排列，再取前 32 条：
 ```text
 SHA256(salt | domain | source_id), then source_id
 ```
 这比 `random.sample` 多保留了两层可复现性：
 - 不依赖 Python / NumPy 随机实现；
 - 每条输出都保留 source ID、selection rank、原文 SHA-256、字符数与 token 数，
  可以从固定源文件重新定位。
 文本使用官方 DeepSeek-V2-Lite tokenizer，加入 special tokens，从右侧截到 96
 tokens。最低长度过滤为：
 | 域 | 最低 token | 文本过滤后的候选 | 长度合格 | 选中 | 选中 token |
 | --- | ---: | ---: | ---: | ---: | ---: |
 | 英文百科 | 24 | 1,841 | 1,655 | 32 | 2,882 |
 | 中文新闻 | 8 | 10,000 | 9,415 | 32 | 573 |
 | Python 代码 | 24 | 164 | 164 | 32 | 2,983 |
 | 小学数学 | 16 | 1,319 | 1,319 | 32 | 2,022 |
 英文与代码经常触到 96-token 截止线；中文新闻标题明显更短。因此后续不能只给
 一份“把所有 token 倒进桶里”的汇总。
 ## 3. 模型与执行边界
 ### 3.1 官方工件
 ```text
 deepseek-ai/DeepSeek-V2-Lite
 revision 604d5664dddd88a0433dbae533b7fe9472482de0
 ```
 关键 SHA-256：
 | 工件 | SHA-256 |
 | --- | --- |
 | `config.json` | `f346286b0f1c8b044252fd54cb4fa78b9fab6472a6e8bebb9edfe03d414ea03d` |
 | `modeling_deepseek.py` | `7d8e5221095286eea991137760893fd7ba52727c0b4ebf48ec09e8bc56b45b9c` |
 | `tokenizer.json` | `41f3bf64213da8c012d8bd0871a58a1fdf70463e8f08f110ddbb1082f529f669` |
 | checkpoint index | `d2cdb2f325f6682cf3ad1ad2526a9f979d857390b579380c0331d975136e0acf` |
 | shard 1 | `0d7e9f39bde40111a4c0f390b87497dce4565cf578d916395e6b2c7851f1e8da` |
 第一分片完整包含 embedding 与 decoder layer 0–6；layer 7 跨分片，因此继续遵守
 原有停止线。Layer 0 是 dense FFN，layer 1–6 是本次统计的六个 MoE 层。
 ### 3.2 批处理不会把 padding 算进统计
 128 条样本按 token 数、domain、source ID 稳定排序，batch size 为 16。每个 batch
 右侧 padding，并使用官方 eager attention 与 causal/padding mask。
 gate hook 会看到 padded positions 的输出，但统计时只切：
 ```text
 topk_ids[sample, :valid_length, :]
 ```
 所以每条 prompt 的 route 总数必须严格等于：
 ```text
 valid_tokens × 6
 ```
 六层均通过这个不变量检查；每层都是 50,760 routes。
 ### 3.3 未执行的对象
 - 未下载或执行 layer 7–26；
 - 未做完整模型生成；
 - 未测回答正确率；
 - 未测训练期间 router load；
 - 未测线上 serving 流量；
 - 未执行 FlashMLA 优化 kernel；
 - 未给 expert ID 赋语义名称。
 ## 4. 为什么 bootstrap 的单位必须是 prompt
 一个 prompt 中相邻 token：
 - 共享主题、语言与格式；
 - 共享相同前缀 hidden state；
 - 在 causal attention 下相互依赖；
 - 不是独立同分布样本。
 如果把 8,460 个 token 当成 8,460 个独立样本，区间会虚假变窄。这里先为每个
 prompt 保存 64 维 route-count vector，再在每个 domain 的 32 条 prompt 内有放回
 抽取 32 条。四个 domain 分层进行，固定：
 ```text
 replicates = 2000
 seed = 20260729
 interval = percentile [2.5%, 97.5%]
 ```
 它回答的是：
 > 如果仍从这个固定探针来源与选样合同附近换一批 prompt，统计量会怎样波动？
 它不回答：
 > 真实训练分布或线上流量的总体参数是什么？
 ## 5. 两种聚合口径
 ### 5.1 Token weighted
 先汇总一个 domain 内所有 prompt 的 route counts，再归一：
 ```text
 p_e = Σ_i count(i,e) / Σ_i Σ_e count(i,e)
 ```
 长 prompt 贡献更多，适合回答“本批实际 token 流向哪里”。
 ### 5.2 Prompt balanced
 先把每条 prompt 的 64 维分布归一，再让 32 条 prompt 等权：
 ```text
 p_e = mean_i [count(i,e) / Σ_e count(i,e)]
 ```
 适合跨格式比较，因为一条短新闻标题与一条被截到 96 token 的代码 prompt 各有一票。
 网站默认展示这一口径，并允许切换。
 在本次四域内部，CV 的两种口径最大绝对差为 0.027（Layer 1 数学）。这说明主要
 层级图景不是某一条特别长 prompt 单独制造的；但两套数字仍完整保留，不能据此假设
 未来语料也不敏感。
 ## 6. 指标是什么
 对当前层、当前 domain 的 64 维 expert share `p`：
 ### CV
 ```text
 CV = population_std(p) / mean(p)
 ```
 越大表示 share 越分散。它不是训练损失，也不直接等于“不均衡惩罚”。
 ### Gini
 64 个 share 的 Gini coefficient。0 表示完全均匀；越大表示质量集中到更少 expert。
 ### Effective experts
 ```text
 H(p) = -Σ p_e log p_e
 effective = exp(H)
 ```
 完全均匀时为 64；越小表示“按熵折算”的有效 expert 数越少。
 ### Top-expert share
 64 个 share 的最大值。完全均匀基线为 `1/64 = 1.5625%`。
 ### Jensen–Shannon divergence
 同层两个 domain 分布 `P,Q`：
 ```text
 M = (P + Q) / 2
 JSD(P,Q) = 1/2 KL(P||M) + 1/2 KL(Q||M)
 ```
 使用自然对数，范围 `[0, ln(2)]`。0 表示两条 64 维分布完全相同。
 ## 7. 主要结果
 以下均为 **prompt-balanced** 点估计；括号内是 effective experts。网站可切换到
 token-weighted 并查看完整 95% 区间。
 | layer | 英文百科 CV（effective） | 中文新闻 CV（effective） | 代码 CV（effective） | 数学 CV（effective） |
 | ---: | ---: | ---: | ---: | ---: |
 | 1 | 0.439（59.0） | 0.754（53.3） | 0.566（55.4） | 0.613（56.3） |
 | 2 | 0.363（60.2） | 0.460（58.4） | 0.400（59.1） | 0.403（59.0） |
 | 3 | 0.398（59.2） | 0.469（57.9） | 0.440（57.9） | 0.525（57.0） |
 | 4 | 0.424（58.6） | **0.910（47.2）** | 0.610（54.0） | 0.527（56.0） |
 | 5 | 0.418（58.5） | 0.551（55.0） | 0.524（56.0） | **0.848（49.7）** |
 | 6 | 0.496（56.7） | 0.594（53.1） | 0.556（54.7） | **0.770（50.7）** |
 ### 7.1 Layer 2 在四域都相对平
 Layer 2 的 prompt-balanced CV 为 0.363–0.460，effective experts 为
 58.4–60.2，是六个被测 MoE 层中四域共同最平的一段。
 “相对平”只指这 64 维 route-count distribution；不等于 expert 权重相同、计算量
 完全相同或训练全局均衡。
 ### 7.2 Layer 4 的中文新闻更集中
 Layer 4 中文新闻：
 ```text
 CV = 0.910, 95% CI [0.857, 0.993]
 effective experts = 47.2, 95% CI [44.4, 48.5]
 top share = E29 9.06%, 95% CI [8.41%, 9.78%]
 ```
 同层英文百科 CV 为 0.424、代码为 0.610、数学为 0.527。正确表述是：
 > 在这 32 条 TNEWS 句子、这个 tokenizer、这个 layer 的固定探针中，route counts
 > 比另外三域更集中。
 错误表述是：
 > E29 是“中文专家”。
 ### 7.3 Layer 5/6 的数学更集中
 Layer 5 数学：
 ```text
 CV = 0.848, 95% CI [0.794, 0.911]
 effective experts = 49.7, 95% CI [48.1, 50.7]
 top share = E14 8.04%, 95% CI [7.38%, 8.64%]
 ```
 Layer 6 数学：
 ```text
 CV = 0.770, 95% CI [0.728, 0.826]
 effective experts = 50.7, 95% CI [49.3, 51.5]
 top share = E28 7.54%, 95% CI [6.79%, 8.26%]
 ```
 E14 与 E28 位于不同层，是不同参数；不能连成一个跨层“数学专家轨迹”。
 ### 7.4 同层域间 JSD
 每层最大的 prompt-balanced pair：
 | layer | 最大 pair | JSD point | 95% bootstrap interval |
 | ---: | --- | ---: | ---: |
 | 1 | 中文新闻 ↔ 代码 | 0.0587 | [0.0549, 0.0703] |
 | 2 | 中文新闻 ↔ 代码 | 0.0345 | [0.0306, 0.0466] |
 | 3 | 中文新闻 ↔ 代码 | 0.0569 | [0.0519, 0.0713] |
 | 4 | **中文新闻 ↔ 代码** | **0.1502** | **[0.1376, 0.1715]** |
 | 5 | 中文新闻 ↔ 数学 | 0.1113 | [0.0990, 0.1368] |
 | 6 | 中文新闻 ↔ 代码 | 0.1161 | [0.1054, 0.1351] |
 Layer 4 的中文新闻↔代码是本探针最大的同层距离，但仍只有理论上界
 `ln(2)≈0.693` 的约 21.7%。
 必须特别注意：普通 percentile bootstrap 是围绕经验分布重采样，不是“两个总体完全
 相同”的 null bootstrap。区间不含 0 不能被偷换成经过校正的显著性检验。
 ## 8. 可复现性
 正式运行和独立重跑的完整 JSON：
 ```text
 4678a1d15395de93ffba757598cc3642bf35e9e07f71d82ddd87c27fc38a09e4
 ```
 两个文件 byte-for-byte identical。确定性范围包括：
 - 128 条选样身份、token 数与原文 SHA-256；
 - 六层、每条 prompt 的 64 维 integer route counts；
 - 两种聚合口径；
 - 2,000 次 bootstrap 导出的所有 metric / expert-share / JSD 区间；
 - 模型、语料、环境与统计合同。
 没有记录 wall-clock timing，因此复跑一致性没有把 GPU warm-up、频率或系统噪声混进
 证据闸门。
 ### 8.1 运行入口
 ```bash
 PYTHONPATH=/path/to/transformers-4.41.2-deps \
 python -B experiments/deepseek/v2_lite_routing_corpus.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-corpus.json \
  --per-domain 32 \
  --max-tokens 96 \
  --batch-size 16 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T07:45:00+00:00
 ```
 官方 2024 remote code 在本机 Transformers 5.5 上会因已移除的
 `is_torch_fx_available` 失败；与前两轮相同，本实验使用隔离的 Transformers 4.41.2
 依赖，并把官方模型代码作为只读本地 package 导入，不给源码打补丁。
 ## 9. 仍然不能说什么
 本实验没有授权以下结论：
 - “DeepSeek 训练时专家全局负载就是这样”；
 - “线上中文请求一定路由到 E29”；
 - “E48 是代码专家、E14 是数学专家”；
 - “某个 domain 的任务性能更好”；
 - “router 的分布差异来自语义，而不是长度、tokenization、格式或共享前缀”；
 - “六个早期 MoE 层代表完整 27 层”；
 - “bootstrap CI 已经完成假设检验或多重比较控制”。
 更稳妥的结论只有：
 > 在固定、可重建的 128 条公开 prompt 上，DeepSeek-V2-Lite 的六个早期 MoE 层
 > 确实呈现可测量的层间与域间 route-count 差异；prompt-level bootstrap 让我们能同时
 > 看见差异和换 prompt 时的波动，但不能把参数索引直接解释成语义角色。
 ## 10. 下一步
 1. 在受支持的 SM90 / SM100 环境执行 FlashMLA 优化 kernel，而不只运行算法参考路径；
 2. 扩展到完整 layer 0–26，需要下载并校验其余 checkpoint shards；
 3. 加入长度匹配与 tokenizer-fragmentation 对照，拆开 domain 和长度因素；
 4. 预注册更少的主要比较，或使用 permutation / null bootstrap 与多重比较校正；
 5. 研究 shared expert 与 routed expert 的激活规模，而不只看 routed IDs；
 6. 若讨论性能，另建生成质量与吞吐合同，不与本路由描述实验混在一起。
@@ -299,6 +299,35 @@ await evaluate(`(() => {
 await pause(180);
 await screenshot("/tmp/llm-atlas-deepseek-absorb-desktop.png");
 const artifactCorpus = await evaluate(`(() => {
  const root = document.querySelector("[data-dsv2-lab]");
  root.querySelector('[data-artifact-tab="corpus"]').click();
  const read = () => ({
    panel: root.querySelector("[data-artifact-panel]:not([hidden])").dataset.artifactPanel,
    rows: root.querySelectorAll("[data-corpus-domain-rows] > div").length,
    heatRows: root.querySelectorAll("[data-corpus-heatmap] > div").length,
    heatCells: root.querySelectorAll("[data-corpus-heatmap] > div > span").length,
    jsdCells: root.querySelectorAll("[data-corpus-jsd] > *").length,
    highest: root.querySelector("[data-corpus-highest-cv]").textContent.trim(),
    largest: root.querySelector("[data-corpus-largest-jsd]").textContent.trim(),
    heatTitle: root.querySelector("[data-corpus-heat-title]").textContent.trim(),
    exact: root.querySelector(".corpus-ledger .exact b").textContent.trim(),
    modeNote: root.querySelector("[data-corpus-mode-note]").textContent.trim(),
  });
  const layer1 = read();
  root.querySelector('[data-corpus-layer="4"]').click();
  const layer4 = read();
  root.querySelector('[data-corpus-mode="token_weighted"]').click();
  const tokenWeighted = read();
  return { layer1, layer4, tokenWeighted };
 })()`);
 await evaluate(`(() => {
  document.querySelector("[data-dsv2-lab]").scrollIntoView({ block: "start", behavior: "instant" });
  window.scrollBy(0, -82);
 })()`);
 await pause(180);
 await screenshot("/tmp/llm-atlas-deepseek-corpus-desktop.png");
 const artifactEvidence = await evaluate(`(() => {
  const root = document.querySelector("[data-dsv2-lab]");
  root.querySelector('[data-artifact-tab="evidence"]').click();
@@ -385,14 +414,14 @@ const mobile = await evaluate(`(() => {
 await evaluate(`(() => {
  document.querySelector("#menu-toggle")?.click();
  const artifact = document.querySelector("[data-dsv2-lab]");
-  artifact.querySelector('[data-artifact-tab="absorb"]').click();
+  artifact.querySelector('[data-artifact-tab="corpus"]').click();
  artifact.scrollIntoView({ block: "start", behavior: "instant" });
  window.scrollBy(0, -82);
 })()`);
 await pause(180);
-await screenshot("/tmp/llm-atlas-deepseek-artifact-mobile.png");
+await screenshot("/tmp/llm-atlas-deepseek-corpus-mobile.png");
-const report = { overview, capacity, cache, codesign, rl, artifactRoute, artifactLoad, artifactCache, artifactAbsorb, artifactEvidence, home, papers, mobile, exceptions };
+const report = { overview, capacity, cache, codesign, rl, artifactRoute, artifactLoad, artifactCache, artifactAbsorb, artifactCorpus, artifactEvidence, home, papers, mobile, exceptions };
 console.log(JSON.stringify(report, null, 2));
 const numeric = (text) => Number.parseFloat(text.replaceAll(",", ""));
@@ -402,7 +431,7 @@ if (overview.sections !== 26 || overview.tocLinks !== 26) failures.push("二十
 if (overview.ledgers !== 24 || overview.waves !== 10) failures.push("二十四张问题账或十次转向结构异常");
 if (overview.paperLinks !== 60 || overview.branches !== 5 || overview.followups !== 1) failures.push("论文链、旁支或公开后续标记异常");
 if (overview.labTabs !== 4 || overview.labPanels !== 4) failures.push("四联实验结构异常");
-if (overview.artifactTabs !== 5 || overview.artifactPanels !== 5 || overview.artifactLayers !== 27) failures.push("真实权重五联实验结构异常");
+if (overview.artifactTabs !== 6 || overview.artifactPanels !== 6 || overview.artifactLayers !== 27) failures.push("真实权重六联实验结构异常");
 if (overview.navLinks !== 20 || home.navLinks !== 20 || mobile.mobileLinks !== 20 || overview.activeNav !== "DeepSeek") failures.push("全站导航未同步 DeepSeek");
 if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出");
 if (capacity.initial.panel !== "capacity" || capacity.initial.total !== "32.1× FFN" || capacity.initial.active !== "1.13× FFN") failures.push("V3 稀疏容量初始账异常");
@@ -429,12 +458,16 @@ if (!artifactCache.million.latent.includes("GiB") || !artifactCache.million.eage
 if (artifactAbsorb.panel !== "absorb" || artifactAbsorb.algebra !== 2 || artifactAbsorb.naive !== "260.00 KiB" || artifactAbsorb.absorbed !== "29.25 KiB") failures.push("真实 absorb 缓存执行账异常");
 if (artifactAbsorb.metrics[0] !== "8.8889×" || artifactAbsorb.metrics[1] !== "0.00390625" || artifactAbsorb.metrics[2] !== "1.19e-7" || artifactAbsorb.metrics[3] !== "BYTE-EXACT") failures.push("absorb 数值正确性或复跑闸门异常");
 if (artifactAbsorb.precisionRows !== 3 || artifactAbsorb.matrixRows !== 5 || artifactAbsorb.localUnsupported !== 4 || artifactAbsorb.executionCards !== 3 || !artifactAbsorb.boundary.includes("不是 FlashMLA 性能")) failures.push("FlashMLA SM120 边界结构异常");
 if (artifactCorpus.layer1.panel !== "corpus" || artifactCorpus.layer1.rows !== 4 || artifactCorpus.layer1.heatRows !== 4 || artifactCorpus.layer1.heatCells !== 256 || artifactCorpus.layer1.jsdCells !== 25) failures.push("128 样本路由区间结构异常");
 if (!artifactCorpus.layer1.highest.includes("中文新闻 · 0.754") || !artifactCorpus.layer1.largest.includes("中文新闻 ↔ Python 代码 · 0.059") || artifactCorpus.layer1.exact !== "BYTE-EXACT") failures.push("Layer 1 多域统计或复跑闸门异常");
 if (!artifactCorpus.layer4.highest.includes("中文新闻 · 0.910") || !artifactCorpus.layer4.largest.includes("中文新闻 ↔ Python 代码 · 0.150") || !artifactCorpus.layer4.heatTitle.includes("layer 4")) failures.push("Layer 4 多域路由切换异常");
 if (!artifactCorpus.tokenWeighted.heatTitle.includes("按 token 加权") || !artifactCorpus.tokenWeighted.modeNote.includes("长 prompt 权重更高")) failures.push("公开语料聚合口径切换异常");
 if (artifactEvidence.panel !== "evidence" || artifactEvidence.layers !== 27 || artifactEvidence.executed !== 7 || artifactEvidence.split !== 1 || artifactEvidence.unloaded !== 19 || artifactEvidence.exact !== "31 / 31") failures.push("真实工件执行边界或复跑闸门异常");
 if (!artifactEvidence.dependency.includes("Transformers 5.5") || !artifactEvidence.dependency.includes("4.41.2") || !artifactEvidence.boundary.includes("完整 27 层生成")) failures.push("依赖版本或未覆盖边界异常");
 if (artifactEvidence.keyboardSelected !== "load" || artifactEvidence.keyboardVisible !== "load") failures.push("真实工件实验键盘 tab 导航异常");
 if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/" || home.paperCount !== "486") failures.push("首页 DeepSeek 首发入口或论文数异常");
 if (papers.total !== 486 || !papers.hasFilter || papers.visible < 20 || !papers.hasCoder || !papers.hasEngram) failures.push("论文库 DeepSeek 聚光异常");
-if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4 || mobile.artifactTabs !== 5 || mobile.artifactHeatCells !== 64) failures.push("移动端导航或实验异常");
+if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4 || mobile.artifactTabs !== 6 || mobile.artifactHeatCells !== 64) failures.push("移动端导航或实验异常");
 if (mobile.offenders.length) failures.push(`移动端越界元素：${JSON.stringify(mobile.offenders)}`);
 if (exceptions.length) failures.push(`浏览器异常：${exceptions.join(" | ")}`);
@@ -3,12 +3,17 @@ import rawTrace from "@/data/deepseek-v2-lite-trace.json";
 import rawRepro from "@/data/deepseek-v2-lite-trace-repro.json";
 import rawAbsorb from "@/data/deepseek-v2-lite-absorb.json";
 import rawAbsorbRepro from "@/data/deepseek-v2-lite-absorb-repro.json";
 import rawCorpus from "@/data/deepseek-v2-lite-routing-corpus.json";
 import rawCorpusRepro from "@/data/deepseek-v2-lite-routing-corpus-repro.json";
 const trace = rawTrace as any;
 const repro = rawRepro as any;
 const absorb = rawAbsorb as any;
 const absorbRepro = rawAbsorbRepro as any;
 const corpus = rawCorpus as any;
 const corpusRepro = rawCorpusRepro as any;
 const absorbExact = JSON.stringify(absorb) === JSON.stringify(absorbRepro);
 const corpusExact = JSON.stringify(corpus) === JSON.stringify(corpusRepro);
 const bytes = (value: number) => value >= 1024
  ? `${(value / 1024).toFixed(2)} KiB`
  : `${value.toLocaleString()} B`;
@@ -44,6 +49,19 @@ const compact = {
  },
 };
 const compactJson = JSON.stringify(compact).replaceAll("<", "\\u003c");
 const corpusCompact = {
  domains: corpus.corpus_contract.domains,
  labels: corpus.corpus_contract.domain_labels,
  counts: corpus.corpus_contract.counts,
  inference: corpus.inference_contract,
  statistics: corpus.statistical_contract,
  layers: corpus.layers.slice(1).map((layer: any) => ({
    layer: layer.layer,
    routes: layer.routes,
    modes: layer.statistics,
  })),
 };
 const corpusCompactJson = JSON.stringify(corpusCompact).replaceAll("<", "\\u003c");
 const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoint_tensor_bytes;
 ---
@@ -55,7 +73,8 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    </div>
    <p>
      固定官方 revision、tokenizer、模型代码和 BF16 第一分片；RTX 5090 连续执行 layer 0–6，
-      捕获 3,240 次真实路由，并让 layer-1 权重继续走入官方吸收式 cache。所有结论都带证据身份与停止线。
+      从 3,240 次 token 显微轨迹扩到 304,560 次公开语料路由，并让 layer-1 权重继续走入官方吸收式 cache。
      所有结论都带证据身份与停止线。
    </p>
  </header>
@@ -79,8 +98,11 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    <button type="button" role="tab" data-artifact-tab="absorb" aria-selected="false" tabindex="-1">
      <span>04</span><b>吸收式执行</b><small>real cache · SM120</small>
    </button>
    <button type="button" role="tab" data-artifact-tab="corpus" aria-selected="false" tabindex="-1">
      <span>05</span><b>128 样本区间</b><small>4 domains · bootstrap</small>
    </button>
    <button type="button" role="tab" data-artifact-tab="evidence" aria-selected="false" tabindex="-1">
-      <span>05</span><b>证据断面</b><small>revision · shards · rerun</small>
+      <span>06</span><b>证据断面</b><small>revision · shards · rerun</small>
    </button>
  </div>
@@ -366,6 +388,94 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    </div>
  </section>
  <section class="artifact-panel" data-artifact-panel="corpus" hidden>
    <div class="panel-lead">
      <div><span>X + S / FIXED PUBLIC CORPUS</span><h4>从四条示例，走到 128 条可重建样本与区间</h4></div>
      <p>
        WikiText-2、TNEWS、HumanEval、GSM8K 各取 32 条；固定哈希选样、最多 96 tokens。
        每个区间都重采样 prompt，而不是把同一 prompt 里的 token 假装成独立样本。
      </p>
    </div>
    <div class="corpus-ledger">
      <article><span>PROMPTS</span><b>{corpus.inference_contract.total_prompts}</b><p>4 domains × 32</p></article>
      <article><span>VALID TOKENS</span><b>{corpus.inference_contract.valid_tokens.toLocaleString()}</b><p>答案未使用，代码未执行</p></article>
      <article><span>REAL ROUTES</span><b>{(corpus.inference_contract.routes_per_moe_layer * 6).toLocaleString()}</b><p>50,760 / layer × 6 MoE layers</p></article>
      <article><span>BOOTSTRAP</span><b>{corpus.statistical_contract.replicates.toLocaleString()}</b><p>prompt-level / domain-stratified</p></article>
      <article class="exact"><span>INDEPENDENT RERUN</span><b>{corpusExact ? "BYTE-EXACT" : "MISMATCH"}</b><p>SHA-256 4678a1d1…a09e4</p></article>
    </div>
    <div class="corpus-controls">
      <div>
        <span>MOE LAYER</span>
        <div class="layer-switch corpus-layer-switch" role="group" aria-label="选择公开语料路由层">
          {[1, 2, 3, 4, 5, 6].map((layer) => (
            <button type="button" data-corpus-layer={layer} class={layer === 1 ? "active" : ""}>L{layer}</button>
          ))}
        </div>
      </div>
      <div>
        <span>AGGREGATION</span>
        <div class="corpus-mode-switch" role="group" aria-label="选择公开语料聚合口径">
          <button type="button" data-corpus-mode="prompt_balanced" aria-pressed="true">每条 prompt 等权</button>
          <button type="button" data-corpus-mode="token_weighted" aria-pressed="false">按 token 加权</button>
        </div>
      </div>
      <p data-corpus-mode-note>
        先把每条 prompt 的 64 维路由分布归一，再平均；短中文标题与长代码 prompt 各有一票。
      </p>
    </div>
    <div class="corpus-domain-table" role="table" aria-label="四域专家负载与 bootstrap 区间">
      <div class="head" role="row">
        <b role="columnheader">DOMAIN / TOKENS</b>
        <b role="columnheader">CV · 95% CI</b>
        <b role="columnheader">EFFECTIVE · 95% CI</b>
        <b role="columnheader">TOP EXPERT · SHARE</b>
      </div>
      <div data-corpus-domain-rows></div>
    </div>
    <div class="corpus-heat-head">
      <div><span>64 EXPERT SHARES</span><b data-corpus-heat-title>layer 1 · 每条 prompt 等权</b></div>
      <p>每一行独立着色；悬停查看 expert ID、份额与 95% 区间。同号 expert 只在当前层内有意义。</p>
    </div>
    <div class="corpus-heat-scroll">
      <div class="corpus-heatmap" data-corpus-heatmap aria-label="四个语料域的 64 专家份额热图"></div>
    </div>
    <div class="corpus-comparison">
      <div>
        <span>PAIRWISE DISTANCE / JENSEN–SHANNON · NATS</span>
        <h5>同层四域分布距离</h5>
        <p>0 表示两条分布相同，理论上界 ln(2)≈0.693；这里展示点估计与 prompt bootstrap 区间。</p>
      </div>
      <div class="corpus-jsd" data-corpus-jsd role="table" aria-label="四域 Jensen-Shannon divergence 矩阵"></div>
    </div>
    <div class="corpus-findings">
      <article><span>CURRENT HIGHEST CV</span><b data-corpus-highest-cv></b><p data-corpus-highest-cv-ci></p></article>
      <article><span>CURRENT LARGEST JSD</span><b data-corpus-largest-jsd></b><p data-corpus-largest-jsd-ci></p></article>
      <article><span>WHAT CHANGED</span><b>区间替代单点印象</b><p>看到“不同”之后，继续问 prompt 换一批时波动多大。</p></article>
    </div>
    <div class="evidence-links">
      <a href="https://huggingface.co/datasets/Salesforce/wikitext" rel="noreferrer">WikiText-2 数据卡 ↗</a>
      <a href="https://github.com/CLUEbenchmark/CLUE" rel="noreferrer">CLUE / TNEWS ↗</a>
      <a href="https://github.com/openai/human-eval" rel="noreferrer">OpenAI HumanEval ↗</a>
      <a href="https://github.com/openai/grade-school-math" rel="noreferrer">OpenAI GSM8K ↗</a>
    </div>
    <div class="artifact-boundary">
      <b>DESCRIPTIVE, NOT SEMANTICS</b>
      <p>
        这是固定 128 条公开 prompt 上的前六个 MoE 层，不是训练分布或线上流量。
        bootstrap 区间描述本探针换 prompt 的稳定性，不是零差异假设检验；没有多重比较校正，
        也不能把 E29、E48 等参数索引命名成“中文专家”或“代码专家”。
      </p>
    </div>
  </section>
  <section class="artifact-panel" data-artifact-panel="evidence" hidden>
    <div class="panel-lead">
      <div><span>O + X / EVIDENCE SLICE</span><h4>为什么执行到 layer 6 就停，而不是把“部分下载”写成“完整复现”</h4></div>
@@ -455,11 +565,12 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    <span>可复现入口</span>
    <code>experiments/deepseek/v2_lite_trace.py</code> ·
    <code>experiments/deepseek/v2_lite_absorb_probe.py</code> ·
-    <code>src/data/deepseek-v2-lite-trace.json</code> ·
+    <code>experiments/deepseek/v2_lite_routing_corpus.py</code> ·
-    <code>research/DEEPSEEK_MLA_ABSORB_AUDIT.md</code>
+    <code>research/DEEPSEEK_ROUTING_CORPUS_AUDIT.md</code>
  </figcaption>
  <script is:inline type="application/json" data-dsv2-trace set:html={compactJson}></script>
  <script is:inline type="application/json" data-dsv2-corpus set:html={corpusCompactJson}></script>
 </figure>
 <script>
@@ -471,8 +582,10 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
      if (node) node.textContent = value;
    };
    const payloadNode = one<HTMLScriptElement>("[data-dsv2-trace]");
-    if (!payloadNode?.textContent) return;
+    const corpusNode = one<HTMLScriptElement>("[data-dsv2-corpus]");
    if (!payloadNode?.textContent || !corpusNode?.textContent) return;
    const data = JSON.parse(payloadNode.textContent);
    const corpusData = JSON.parse(corpusNode.textContent);
    const tabs = all<HTMLButtonElement>("[data-artifact-tab]");
    const panels = all<HTMLElement>("[data-artifact-panel]");
@@ -659,6 +772,173 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
        control.addEventListener("change", renderCache);
      });
    renderCache();
    const corpusLabels: Record<string, string> = {
      english: "英文百科",
      chinese: "中文新闻",
      code: "Python 代码",
      math: "小学数学",
    };
    const corpusLayer = (layer: number) => corpusData.layers.find((item: any) => item.layer === layer);
    let corpusLayerNumber = 1;
    let corpusMode = "prompt_balanced";
    const formatCi = (ci: number[], digits = 3) => `[${ci[0].toFixed(digits)}, ${ci[1].toFixed(digits)}]`;
    const renderCorpus = () => {
      const layer = corpusLayer(corpusLayerNumber);
      const mode = layer.modes[corpusMode];
      all<HTMLButtonElement>("[data-corpus-layer]").forEach((button) => {
        button.classList.toggle("active", Number(button.dataset.corpusLayer) === corpusLayerNumber);
      });
      all<HTMLButtonElement>("[data-corpus-mode]").forEach((button) => {
        button.setAttribute("aria-pressed", String(button.dataset.corpusMode === corpusMode));
      });
      set(
        "[data-corpus-mode-note]",
        corpusMode === "prompt_balanced"
          ? "先把每条 prompt 的 64 维路由分布归一，再平均；短中文标题与长代码 prompt 各有一票。"
          : "直接汇总全部 token 的路由计数；长 prompt 权重更高，适合回答“本批 token 实际流向哪里”。",
      );
      set(
        "[data-corpus-heat-title]",
        `layer ${corpusLayerNumber} · ${corpusMode === "prompt_balanced" ? "每条 prompt 等权" : "按 token 加权"}`,
      );
      const rows = one<HTMLElement>("[data-corpus-domain-rows]");
      if (rows) {
        rows.replaceChildren(...corpusData.domains.map((domain: string) => {
          const result = mode.domains[domain];
          const metrics = result.metrics;
          const top = result.top_experts[0];
          const row = document.createElement("div");
          row.setAttribute("role", "row");
          const identity = document.createElement("span");
          const identityLabel = document.createElement("b");
          const identityMeta = document.createElement("small");
          identityLabel.textContent = corpusLabels[domain];
          identityMeta.textContent = `32 prompts · ${corpusData.counts[domain].valid_tokens.toLocaleString()} tokens`;
          identity.append(identityLabel, identityMeta);
          const cv = document.createElement("span");
          const cvPoint = document.createElement("b");
          const cvCi = document.createElement("small");
          cvPoint.textContent = metrics.cv.point.toFixed(3);
          cvCi.textContent = formatCi(metrics.cv.ci95);
          cv.append(cvPoint, cvCi);
          const effective = document.createElement("span");
          const effectivePoint = document.createElement("b");
          const effectiveCi = document.createElement("small");
          effectivePoint.textContent = metrics.effective_experts.point.toFixed(1);
          effectiveCi.textContent = formatCi(metrics.effective_experts.ci95, 1);
          effective.append(effectivePoint, effectiveCi);
          const topExpert = document.createElement("span");
          const topLabel = document.createElement("b");
          const topCi = document.createElement("small");
          topLabel.textContent = `E${top.expert} · ${(top.share * 100).toFixed(2)}%`;
          topCi.textContent = `${(top.ci95[0] * 100).toFixed(2)}–${(top.ci95[1] * 100).toFixed(2)}%`;
          topExpert.append(topLabel, topCi);
          row.append(identity, cv, effective, topExpert);
          return row;
        }));
      }
      const heatmap = one<HTMLElement>("[data-corpus-heatmap]");
      if (heatmap) {
        heatmap.replaceChildren(...corpusData.domains.map((domain: string) => {
          const result = mode.domains[domain];
          const max = Math.max(...result.distribution);
          const row = document.createElement("div");
          const label = document.createElement("b");
          label.textContent = corpusLabels[domain];
          row.append(label);
          result.distribution.forEach((share: number, expert: number) => {
            const cell = document.createElement("span");
            const ci = result.expert_share_ci95[expert];
            cell.style.setProperty("--share", String(share / max));
            cell.title = `${corpusLabels[domain]} · L${corpusLayerNumber} · E${expert} · ${(share * 100).toFixed(2)}% · 95% CI ${(ci[0] * 100).toFixed(2)}–${(ci[1] * 100).toFixed(2)}%`;
            cell.setAttribute("aria-label", cell.title);
            row.append(cell);
          });
          return row;
        }));
      }
      const jsd = one<HTMLElement>("[data-corpus-jsd]");
      const pairFor = (left: string, right: string) => mode.pairs.find((pair: any) =>
        (pair.left === left && pair.right === right) || (pair.left === right && pair.right === left)
      );
      if (jsd) {
        const cells: HTMLElement[] = [];
        const corner = document.createElement("b");
        corner.textContent = "DOMAIN";
        cells.push(corner);
        corpusData.domains.forEach((domain: string) => {
          const header = document.createElement("b");
          header.textContent = corpusLabels[domain];
          cells.push(header);
        });
        corpusData.domains.forEach((left: string) => {
          const header = document.createElement("b");
          header.textContent = corpusLabels[left];
          cells.push(header);
          corpusData.domains.forEach((right: string) => {
            const cell = document.createElement("span");
            if (left === right) {
              cell.className = "diagonal";
              cell.textContent = "0";
            } else {
              const result = pairFor(left, right).js_divergence;
              const point = document.createElement("b");
              const ci = document.createElement("small");
              point.textContent = result.point.toFixed(3);
              ci.textContent = formatCi(result.ci95);
              cell.append(point, ci);
            }
            cells.push(cell);
          });
        });
        jsd.replaceChildren(...cells);
      }
      const highest = corpusData.domains
        .map((domain: string) => ({ domain, value: mode.domains[domain].metrics.cv }))
        .sort((left: any, right: any) => right.value.point - left.value.point)[0];
      const largest = [...mode.pairs]
        .sort((left: any, right: any) =>
          right.js_divergence.point - left.js_divergence.point
        )[0];
      set(
        "[data-corpus-highest-cv]",
        `${corpusLabels[highest.domain]} · ${highest.value.point.toFixed(3)}`,
      );
      set(
        "[data-corpus-highest-cv-ci]",
        `95% CI ${formatCi(highest.value.ci95)} · L${corpusLayerNumber}`,
      );
      set(
        "[data-corpus-largest-jsd]",
        `${corpusLabels[largest.left]} ↔ ${corpusLabels[largest.right]} · ${largest.js_divergence.point.toFixed(3)}`,
      );
      set(
        "[data-corpus-largest-jsd-ci]",
        `95% CI ${formatCi(largest.js_divergence.ci95)} · nats`,
      );
    };
    all<HTMLButtonElement>("[data-corpus-layer]").forEach((button) => {
      button.addEventListener("click", () => {
        corpusLayerNumber = Number(button.dataset.corpusLayer);
        renderCorpus();
      });
    });
    all<HTMLButtonElement>("[data-corpus-mode]").forEach((button) => {
      button.addEventListener("click", () => {
        corpusMode = button.dataset.corpusMode ?? "prompt_balanced";
        renderCorpus();
      });
    });
    renderCorpus();
  });
 </script>
@@ -702,7 +982,12 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
  .execution-split span,
  .repro-gate span,
  .dependency-split span,
-  .checksum-grid span {
+  .checksum-grid span,
  .corpus-ledger span,
  .corpus-controls > div > span,
  .corpus-heat-head span,
  .corpus-comparison span,
  .corpus-findings span {
    margin: 0;
    color: var(--blue);
    font: 700 .69rem/1.3 var(--font-mono);
@@ -756,7 +1041,7 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
  .artifact-status b { color: var(--ink); font-size: .72rem; }
  .artifact-tabs {
    display: grid;
-    grid-template-columns: repeat(5, 1fr);
+    grid-template-columns: repeat(6, 1fr);
    background: var(--ink);
  }
  .artifact-tabs button {
@@ -1052,6 +1337,212 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
  .jaccard-block :global([data-load-jaccard] article b) { font: 700 .62rem/1 var(--font-mono); text-align: right; }
  .load-lessons { grid-template-columns: repeat(3, 1fr); }
  .load-lessons b { display: block; margin-top: .35rem; font-size: .77rem; }
  .corpus-ledger {
    display: grid;
    grid-template-columns: repeat(5, 1fr);
    border: 1px solid rgba(32,32,39,.14);
  }
  .corpus-ledger article {
    padding: .85rem;
    border-right: 1px solid rgba(32,32,39,.12);
  }
  .corpus-ledger article:last-child { border-right: 0; }
  .corpus-ledger article.exact { background: rgba(57,120,110,.1); }
  .corpus-ledger b {
    display: block;
    margin-top: .4rem;
    font: 750 1.2rem/1 var(--font-display);
  }
  .corpus-ledger p {
    margin: .3rem 0 0;
    color: rgba(32,32,39,.56);
    font-size: .63rem;
    line-height: 1.4;
  }
  .corpus-controls {
    display: grid;
    grid-template-columns: auto 1.1fr minmax(15rem, 1.3fr);
    gap: 1rem;
    align-items: end;
    margin-top: .8rem;
    padding: .85rem;
    border: 1px solid rgba(32,32,39,.14);
    background: #e8e2d7;
  }
  .corpus-controls > div { display: grid; gap: .45rem; }
  .corpus-controls .layer-switch { margin: 0; }
  .corpus-mode-switch { display: flex; }
  .corpus-mode-switch button {
    padding: .58rem .75rem;
    border: 1px solid rgba(32,32,39,.22);
    background: #fffdf8;
    color: var(--ink);
    font: 650 .67rem/1 var(--font-mono);
    cursor: pointer;
  }
  .corpus-mode-switch button + button { border-left: 0; }
  .corpus-mode-switch button[aria-pressed="true"] {
    border-color: var(--blue);
    background: var(--blue);
    color: white;
  }
  .corpus-controls > p {
    margin: 0;
    color: rgba(32,32,39,.62);
    font-size: .69rem;
    line-height: 1.5;
  }
  .corpus-domain-table {
    margin-top: .8rem;
    border: 1px solid rgba(32,32,39,.15);
    overflow-x: auto;
    background: #fffdf8;
  }
  .corpus-domain-table .head,
  .corpus-domain-table :global([data-corpus-domain-rows] > div) {
    display: grid;
    grid-template-columns: 1.25fr repeat(3, 1fr);
    min-width: 680px;
  }
  .corpus-domain-table .head { background: var(--ink); color: white; }
  .corpus-domain-table .head b,
  .corpus-domain-table :global([data-corpus-domain-rows] > div > span) {
    display: grid;
    gap: .25rem;
    padding: .7rem;
    border-right: 1px solid rgba(32,32,39,.1);
    border-bottom: 1px solid rgba(32,32,39,.1);
  }
  .corpus-domain-table .head b {
    font: 650 .62rem/1.2 var(--font-mono);
    border-color: rgba(255,255,255,.12);
  }
  .corpus-domain-table :global([data-corpus-domain-rows] b) {
    font: 720 .76rem/1.2 var(--font-mono);
  }
  .corpus-domain-table :global([data-corpus-domain-rows] small) {
    color: rgba(32,32,39,.52);
    font: .61rem/1.25 var(--font-mono);
  }
  .corpus-heat-head {
    display: grid;
    grid-template-columns: 1fr 1.4fr;
    gap: 1rem;
    align-items: end;
    margin: 1rem 0 .55rem;
  }
  .corpus-heat-head > div { display: grid; gap: .25rem; }
  .corpus-heat-head b { font-size: .78rem; }
  .corpus-heat-head p {
    margin: 0;
    color: rgba(32,32,39,.58);
    font-size: .67rem;
    line-height: 1.45;
    text-align: right;
  }
  .corpus-heat-scroll {
    overflow-x: auto;
    border: 1px solid rgba(32,32,39,.14);
    background: #ddd6ca;
  }
  .corpus-heatmap { min-width: 880px; }
  .corpus-heatmap > :global(div) {
    display: grid;
    grid-template-columns: 6.5rem repeat(64, minmax(8px, 1fr));
    gap: 2px;
    padding: 3px;
    border-bottom: 1px solid rgba(32,32,39,.12);
  }
  .corpus-heatmap > :global(div:last-child) { border-bottom: 0; }
  .corpus-heatmap > :global(div > b) {
    position: sticky;
    left: 0;
    z-index: 1;
    display: grid;
    align-items: center;
    padding: .45rem;
    background: #eee8de;
    font: 700 .61rem/1.1 var(--font-mono);
  }
  .corpus-heatmap > :global(div > span) {
    min-height: 1.55rem;
    background: color-mix(in srgb, var(--blue) calc(var(--share) * 88%), #f2ede4);
    cursor: help;
  }
  .corpus-heatmap > :global(div > span:hover) {
    outline: 2px solid var(--amber);
    z-index: 2;
  }
  .corpus-comparison {
    display: grid;
    grid-template-columns: .65fr 1.35fr;
    gap: 1rem;
    margin-top: .8rem;
    padding: 1rem;
    border: 1px solid rgba(32,32,39,.14);
    background: #e8e2d7;
  }
  .corpus-comparison h5 {
    margin: .4rem 0;
    font: 720 1rem/1.15 var(--font-display);
  }
  .corpus-comparison p {
    margin: 0;
    color: rgba(32,32,39,.58);
    font-size: .68rem;
    line-height: 1.5;
  }
  .corpus-jsd {
    display: grid;
    grid-template-columns: 6rem repeat(4, minmax(5.5rem, 1fr));
    overflow-x: auto;
    border: 1px solid rgba(32,32,39,.13);
    background: #fffdf8;
  }
  .corpus-jsd > :global(b),
  .corpus-jsd > :global(span) {
    display: grid;
    align-content: center;
    gap: .2rem;
    min-height: 3.1rem;
    padding: .45rem;
    border-right: 1px solid rgba(32,32,39,.1);
    border-bottom: 1px solid rgba(32,32,39,.1);
    font: 650 .59rem/1.2 var(--font-mono);
  }
  .corpus-jsd > :global(b) { background: var(--ink); color: white; }
  .corpus-jsd > :global(span > b) { font-size: .72rem; }
  .corpus-jsd > :global(span > small) {
    color: rgba(32,32,39,.48);
    font-size: .52rem;
  }
  .corpus-jsd > :global(span.diagonal) {
    place-items: center;
    background: rgba(57,120,110,.08);
    color: var(--teal);
  }
  .corpus-findings {
    display: grid;
    grid-template-columns: repeat(3, 1fr);
    margin-top: .8rem;
    border: 1px solid rgba(32,32,39,.14);
  }
  .corpus-findings article {
    padding: .85rem;
    border-right: 1px solid rgba(32,32,39,.12);
  }
  .corpus-findings article:last-child { border-right: 0; }
  .corpus-findings b {
    display: block;
    margin-top: .4rem;
    font-size: .78rem;
  }
  .corpus-findings p {
    margin: .3rem 0 0;
    color: rgba(32,32,39,.56);
    font-size: .65rem;
    line-height: 1.45;
  }
  .observed-cache {
    display: grid;
    grid-template-columns: 1fr auto 1.25fr;
@@ -1283,7 +1774,10 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    .heat-head,
    .aggregate-card,
    .jaccard-block,
-    .kernel-contract { grid-template-columns: 1fr; }
+    .kernel-contract,
    .corpus-controls,
    .corpus-heat-head,
    .corpus-comparison { grid-template-columns: 1fr; }
    .artifact-status { grid-template-columns: 1fr 1fr; }
    .artifact-tabs { grid-template-columns: 1fr 1fr; }
    .route-controls { grid-template-columns: 1fr 1fr; }
@@ -1294,6 +1788,8 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    .route-metrics,
    .checksum-grid,
    .absorb-metrics { grid-template-columns: 1fr 1fr; }
    .corpus-ledger { grid-template-columns: repeat(3, 1fr); }
    .corpus-heat-head p { text-align: left; }
    .layer-evidence { grid-template-columns: repeat(9, 1fr); }
    .repro-gate { grid-template-columns: 1fr 1fr; }
    .repro-gate > p { grid-column: 1 / -1; padding: .8rem 0 0; border-left: 0; border-top: 1px solid rgba(255,255,255,.18); }
@@ -1319,13 +1815,19 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
    .absorb-metrics,
    .artifact-identity,
    .checksum-grid,
-    .cache-ledger { grid-template-columns: 1fr; }
+    .cache-ledger,
    .corpus-ledger,
    .corpus-findings { grid-template-columns: 1fr; }
    .route-metrics article,
    .cache-ratio article,
    .load-lessons article,
    .absorb-metrics article,
    .artifact-identity article,
    .checksum-grid article { border-right: 0; border-bottom: 1px solid rgba(32,32,39,.12); }
    .corpus-ledger article,
    .corpus-findings article { border-right: 0; border-bottom: 1px solid rgba(32,32,39,.12); }
    .corpus-mode-switch { display: grid; grid-template-columns: 1fr; }
    .corpus-mode-switch button + button { border-left: 1px solid rgba(32,32,39,.22); border-top: 0; }
    .artifact-boundary { grid-template-columns: 1fr; }
    .load-dials { grid-template-columns: 1fr; }
    .observed-cache,
@@ -37,7 +37,7 @@ const toc = [
 <BaseLayout
  title="DeepSeek 技术谱系与真实权重深读：从 Dense、MoE、MLA 到 R1 与 V4"
-  description="用二十四张问题账、十次技术转向、九个交互实验、真实 V2-Lite 权重与吸收式缓存 trace 和六十个一手节点，完整理解 DeepSeek 的 MoE、MLA、FP8、DualPipe、GRPO、R1、V3.2 与 V4。"
+  description="用二十四张问题账、十次技术转向、十个交互实验、真实 V2-Lite 权重、公开语料路由区间与吸收式缓存 trace 和六十个一手节点，完整理解 DeepSeek 的 MoE、MLA、FP8、DualPipe、GRPO、R1、V3.2 与 V4。"
  section="deepseek"
 >
  <header class="page-hero deepseek-hero">
@@ -55,7 +55,7 @@ const toc = [
        <div><dt>SPAN</dt><dd>2024.01 → 2026.06</dd></div>
        <div><dt>LEDGERS</dt><dd>24 张问题账</dd></div>
        <div><dt>LINEAGE</dt><dd>10 次技术转向</dd></div>
-        <div><dt>LABS</dt><dd>9 个可操作实验</dd></div>
+        <div><dt>LABS</dt><dd>10 个可操作实验</dd></div>
        <div><dt>EVIDENCE</dt><dd>60 个一手 / 官方节点</dd></div>
        <div><dt>STATUS</dt><dd>三轮 · 真实权重执行</dd></div>
      </dl>
@@ -768,15 +768,15 @@ const toc = [
        <p class="eyebrow"><span>22</span> OFFICIAL WEIGHTS / EXECUTED</p>
        <h2>从“MLA 与 MoE 的概念”再往前一步：让官方 V2-Lite 权重真的跑起来</h2>
        <p class="lede">
-          前面的四联实验负责建立公式与角色合同；下面的五联工件实验固定官方 revision、tokenizer、
+          前面的四联实验负责建立公式与角色合同；下面的六联工件实验固定官方 revision、tokenizer、
          模型代码和 checkpoint 第一分片，在 RTX 5090 上连续执行 layer 0–6。它把真实观测、shape 推导、
-          吸收式 latent cache、实现差距和未覆盖范围放在同一张证据图里。
+          吸收式 latent cache、128 条公开语料的路由区间、实现差距和未覆盖范围放在同一张证据图里。
        </p>
        <div class="artifact-callout">
          <article><span>X / FORWARD</span><b>7 / 27 layers</b><p>1 个 dense 层 + 6 个 MoE 层；layer 7 因跨分片停止。</p></article>
-          <article><span>X / ROUTES</span><b>3,240</b><p>90 个有效 token × 6 层 × top-6 routed experts。</p></article>
+          <article><span>X / ROUTES</span><b>304,560</b><p>128 条公开 prompt、8,460 token、6 个 MoE 层的真实 top-6 选择。</p></article>
          <article><span>X / ABSORB CACHE</span><b>266,240 → 29,952 B</b><p>同一真实 layer-1 权重的 naive / absorb active buffers。</p></article>
-          <article><span>X / RERUN</span><b>31 / 31 exact</b><p>hidden hashes、MLA shapes、loads 与全部 token routes。</p></article>
+          <article><span>X / RERUN</span><b>BYTE-EXACT</b><p>固定选样、逐 prompt loads 与 2,000 次 bootstrap 摘要完整复跑。</p></article>
        </div>
        <DeepSeekArtifactLab />
      </section>
@@ -15,7 +15,7 @@ const workstreams = [
  { label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
  { label: "Scaling Laws", value: 74, next: "加入真实拟合复现、置信区间与更多模型族对照" },
  { label: "数据工程与预训练配方", value: 73, next: "逐图精读 FineWeb / DCLM，加入真实去重与 mixture traces" },
-  { label: "DeepSeek 专题", value: 89, next: "SM90 FlashMLA 优化 kernel、更大样本负载、FP8/pipeline 与 R1-like RL 复现" },
+  { label: "DeepSeek 专题", value: 91, next: "SM90 FlashMLA kernel、完整 27 层、长度匹配对照、FP8/pipeline 与 R1-like RL 复现" },
  { label: "指令微调与人类偏好", value: 75, next: "加入真实偏好分歧样本、RM 长度偏置与 PPO/DPO 小模型复现" },
  { label: "推理与测试时扩展", value: 76, next: "真实模型采样曲线、PRM 案例与逐篇图表精读" },
  { label: "工具使用与长程 Agent", value: 74, next: "补真实环境 traces、cross-harness 对照、Agent RL 训练曲线与安全案例" },
@@ -50,7 +50,7 @@ const workstreams = [
        <div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
        <div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
        <div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
-        <div><dt>UPDATED</dt><dd>2026-07-29 14:15 CST</dd></div>
+        <div><dt>UPDATED</dt><dd>2026-07-29 15:25 CST</dd></div>
        <div><dt>MODE</dt><dd>持续迭代，不锁死版本</dd></div>
      </dl>
    </div>
@@ -97,12 +97,12 @@ const workstreams = [
      <article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
      <article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全，包含先修依赖和三条贯穿案例。</p></article>
      <article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
-      <article><span>✓</span><h3>七十六个原创交互视图</h3><p>K3 三轴图、八联报告实验与四联开放工件实验，DeepSeek 四联公式实验与五联真实权重实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
+      <article><span>✓</span><h3>七十七个原创交互视图</h3><p>K3 三轴图、八联报告实验与四联开放工件实验，DeepSeek 四联公式实验与六联真实权重实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
      <article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
      <article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
      <article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
      <article><span>✓</span><h3>表示、位置与残差高速公路深度专题</h3><p>二十张问题账、66 个一手节点、DeepSeek/Kimi 双谱系，以及 Token—位置—Norm—Residual/FFN 四联实验。</p></article>
-      <article><span>✓</span><h3>DeepSeek 三轮真实权重里程碑</h3><p>在二十四张问题账、十次转向与四联公式实验上，新增 V2-Lite 7/27 层连续 forward、3,240 次真实专家选择、MLA/eager cache 实现账、31/31 exact 复跑，以及官方 V3 absorb 的 576 元素真实缓存五联实验。</p></article>
+      <article><span>✓</span><h3>DeepSeek 三轮真实权重里程碑</h3><p>在二十四张问题账、十次转向与四联公式实验上，新增 V2-Lite 7/27 层连续 forward、官方 V3 absorb 的 576 元素真实缓存，以及四域 128 条固定公开 prompt、304,560 次真实路由、2,000 次 prompt bootstrap 与 byte-exact 独立重跑的六联实验。</p></article>
      <article><span>✓</span><h3>Kimi K3 技术报告二轮深读</h3><p>三十二张问题账、Figure 1–16 / Table 1–5 审计、100 节点阅读链，以及 Delta—Decay—AttnRes—LatentMoE—SiTU—QB—MOPD—Cache 八联实验。</p></article>
      <article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions，审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes；四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
      <article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel；6/6 官方参考逐元素相等，并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
@@ -134,7 +134,7 @@ const workstreams = [
    <div class="queue-table">
      <div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
      <div><span>P0</span><strong>K3 三轮</strong><p>开放权重 traces → FlashKDA / AttnRes / MoE 真实行为 → Figure 1–16 数值重绘与独立复现</p><em>运行证据 + 逐图复现</em></div>
-      <div><span>P0</span><strong>DeepSeek 三轮</strong><p>SM90 FlashMLA 优化 kernel / 更大负载样本 → FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
+      <div><span>P0</span><strong>DeepSeek 三轮</strong><p>SM90 FlashMLA kernel / 完整 27 层 / 长度匹配对照 → FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
      <div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
      <div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
      <div><span>P0</span><strong>语言模型前史二轮</strong><p>Kneser–Ney / LSTM / Bahdanau 逐图 → 真实小语料复现 → tokenizer 公平性</p><em>可复现实验 + 逐图笔记</em></div>