feat: add DeepSeek message-history factorial probe
This commit is contained in:
wuyang
+12
-3
@@ -14,7 +14,7 @@
|
||||
| 表示、位置与残差高速公路 | 完成首版 | 81% | 真实 hidden-state / norm traces、长上下文位置外推与深层稳定性消融 |
|
||||
| Scaling Laws | 完成首版 | 74% | 真实拟合复现、置信区间与更多模型族对照 |
|
||||
| 数据工程与预训练配方 | 完成首版 | 73% | FineWeb / DCLM 逐图精读、真实去重误伤与 mixture traces |
|
||||
| DeepSeek 专题 | 三轮实证进行中 | 93% | SM90 FlashMLA kernel、完整 27 层、词元边界 / system / few-shot 正交扰动、FP8/pipeline 与 R1-like RL 复现 |
|
||||
| DeepSeek 专题 | 三轮实证进行中 | 94% | SM90 FlashMLA kernel、完整 27 层、词元边界 / 距离 / EOS / 角色正交控制、FP8/pipeline 与 R1-like RL 复现 |
|
||||
| 指令微调与人类偏好 | 完成首版 | 75% | 真实偏好分歧、RM 长度偏置与 PPO/DPO 小模型复现 |
|
||||
| 推理与测试时扩展 | 完成首版 | 76% | 真实模型采样曲线、PRM 案例与逐篇图表精读 |
|
||||
| 工具使用与长程 Agent | 完成首版 | 74% | 真实环境 traces、cross-harness 对照、Agent RL 曲线与安全案例 |
|
||||
@@ -41,7 +41,7 @@
|
||||
- [x] 完成 486 篇关键论文索引，覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。
|
||||
- [x] 完成可检索、可按专题筛选的论文库页面。
|
||||
- [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。
|
||||
- [x] 完成 K3 三轴架构、八联报告实验与四联开放工件实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 十一联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等七十八个原创交互视图。
|
||||
- [x] 完成 K3 三轴架构、八联报告实验与四联开放工件实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 十二联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等七十九个原创交互视图。
|
||||
- [x] 完成长上下文首版：五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
|
||||
- [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文，并建立长上下文研究账本。
|
||||
- [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文，并建立 MoE 研究账本。
|
||||
@@ -196,11 +196,17 @@
|
||||
- [x] RAW→USER 的模板敏感性不写成单向规律：L1 英文/中文与 L2 中文/代码更平，L3 中文、L5 中文/代码与 L6 四域更集中；网站第七个真实工件页签联动 layer、scope、aggregation，并展示 paired 95% 区间与逐 token top-6 稳定性。
|
||||
- [x] 模板扰动里程碑本地闸门通过：69 个 Astro 文件零诊断、21 个页面、1,151 个站内引用、12 个跨页锚点零失败，十六套真实 Chrome 回归全部通过，桌面与 390px 移动端无文档级横向溢出。
|
||||
- [x] DeepSeek 官方模板敏感性版本以源提交 `b0acc8b`、不可变镜像 `20260729T092426Z-b0acc8b` 发布；OCI digest `sha256:f1bf2d09…e1463`，NAS / VPS / NPM / DNS / TLS / HTTP2 / gzip / 门户与十六套生产 Chrome 回归全通过；保留 `20260729T081849Z-9ca0850` 回滚。
|
||||
- [x] 消息历史实验复用上一轮完全相同的 128-source / 23-content-token cohort，把 system 与 one-shot 构成 2×2：system 在两个 one-shot 水平都固定每条 +16 tokens，one-shot 在两个 system 水平都固定每条 +17 tokens，四格同一 padded batch。
|
||||
- [x] RTX 5090 执行 512 个消息历史变体、24,040 个输入 token，新增 865,440 次真实 top-6 路由，使公开语料累计达到 1,736,352 次；四格各有相同的 2,874 个精确对齐目标 token。
|
||||
- [x] 2×2 source-paired bootstrap 同报 system main、few-shot main 与 difference-in-differences interaction；CV 区间完全正 / 负 / 跨零分别为 system `8/13/3`、few-shot `7/7/10`、interaction `8/8/8`，不压成单向规律。
|
||||
- [x] 固定消息中的历史缓冲模式闭环：system-at-F1 相对 system-at-F0 的目标 TV 在 24 / 24 个 layer×domain 全部下降，均值 `.073→.019`；绝对 CV 变化 21 / 24 下降，逐 token top-6 set exact 在六层均明显提高。
|
||||
- [x] 消息历史正式运行与独立复跑 SHA-256 均为 `5765fbf8…c1fb`，约 28 MiB JSON byte-exact；网站第八个真实工件页签联动 layer、scope、aggregation 与 system/few-shot/interaction depth map。
|
||||
- [x] 消息历史里程碑本地闸门通过：69 个 Astro 文件零诊断、21 个页面、1,151 个站内引用、12 个跨页锚点零失败，十六套真实 Chrome 回归全部通过，桌面与 390px 移动端无文档级横向溢出。
|
||||
|
||||
## 正在进行
|
||||
|
||||
- [ ] K3 三轮下一闸门：获得真实 token hidden states、expert load 与 cache traces，解释或修订 `A_log [128]` 工件冲突，再做 Figure 3/4/5 数值重绘和独立小模型复现。
|
||||
- [ ] DeepSeek 三轮下一闸门：在官方支持的 SM90 环境执行 FlashMLA 优化 kernel；扩到完整 27 层并补词元边界 / system / few-shot 正交扰动，再推进 FP8 / pipeline traces 与 R1-like RL 小模型复现。
|
||||
- [ ] DeepSeek 三轮下一闸门：在官方支持的 SM90 环境执行 FlashMLA 优化 kernel；扩到完整 27 层并继续拆分词元边界、历史距离、EOS、角色与示例内容，再推进 FP8 / pipeline traces 与 R1-like RL 小模型复现。
|
||||
- [ ] 表示、位置与残差二轮：真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。
|
||||
- [ ] 评测安全二轮：真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。
|
||||
- [ ] 推理服务二轮：真实 GPU kernel / workload traces、功耗与成本、跨 vLLM / SGLang / TensorRT-LLM 复现。
|
||||
@@ -335,6 +341,9 @@
|
||||
| 2026-07-29 | `Assistant:` 追加条件承担 causal-mask 负对照 | 21,852 个共享前缀 ordered top-6 全部 exact；证明未来 suffix 不改写过去，但不推出 suffix 自身没有路由作用 |
|
||||
| 2026-07-29 | 模板效应永久分 scope、layer 与 domain 报告 | 对齐内容回答上下文条件化，完整输入回答真实协议流量；RAW→USER 的 CV 方向跨层翻转，不压成“角色模板更均衡/更集中” |
|
||||
| 2026-07-29 | DeepSeek 官方模板敏感性里程碑以 `20260729T092426Z-b0acc8b` 发布 | OCI digest `sha256:f1bf2d09…e1463`；复用 NAS 12010→8080、NPM 31 / cert 41、门户 order 180；十六套生产 Chrome 回归通过，保留上一不可变镜像回滚 |
|
||||
| 2026-07-29 | system × one-shot 使用固定 16 / 17-token 正交增量 | 四格同 cohort、同 target、同 batch；主效应与交互共用 source-prompt bootstrap indices，避免把两条独立两组实验误拼成因子结论 |
|
||||
| 2026-07-29 | one-shot 后的 system-edge TV 下降只命名为历史缓冲模式 | 24 / 24 格下降与逐 token set/Jaccard 同向；示例语义、距离、EOS、角色转换和固定文本尚未拆开，因此不写成 few-shot 因果语义 |
|
||||
| 2026-07-29 | 因子分布 effect 使用 half-L1 magnitude 而非普通 TV | 主效应和 interaction 是带符号 expert-share 向量；只有四条实际条件边继续使用标准 TV / JSD |
|
||||
| 2026-07-29 | K3 二轮按 32 张对象账与完整报告顺序重建 | total/active、2.5×、KDA state、深度来源、专家路由、视觉目标、轨迹、缓存与评测协议不再压成一页组件摘要 |
|
||||
| 2026-07-29 | K3 原生视觉事实回到 §2.4 / §3.3 核验 | 删除“先冻结语言模型再解冻”旧表述；明确 MoonViT-V2 从头训练，视觉/文本从开始共同 NTP |
|
||||
| 2026-07-29 | K3 Figure 1–16 / Table 1–5 全部建立课程视觉契约 | 每张图同时写支持范围与不可外推项；作者报告、论文、推导与 toy model 使用 R/P/D/T 标签 |
|
||||
|
||||
@@ -19,7 +19,7 @@
|
||||
|
||||
当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读，
|
||||
语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化，以及评测与安全深度专题，
|
||||
以及 78 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
|
||||
以及 79 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
|
||||
8 个交互实验和 100 个一手/官方节点，完整覆盖架构、预训练、后训练、系统、评测、案例与附录。
|
||||
第三轮已完成开放工件与首个真实 kernel 里程碑：固定官方模型与 FlashKDA revisions，审计 96 个 checkpoint shards、
|
||||
497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计，并用 4 个新视图
|
||||
@@ -28,7 +28,7 @@
|
||||
[K3_ARTIFACT_AUDIT.md](./research/K3_ARTIFACT_AUDIT.md) 与
|
||||
[checkpoint_probe.py](./experiments/k3/checkpoint_probe.py)、[FlashKDA probe](./experiments/k3/flashkda/)。
|
||||
DeepSeek 三轮专题以 24 张问题账、10 次技术转向、
|
||||
11 个交互实验和 60 个一手/官方节点，串起 Dense、MoE、MLA、V3 协同、R1 与 V4；
|
||||
12 个交互实验和 60 个一手/官方节点，串起 Dense、MoE、MLA、V3 协同、R1 与 V4；
|
||||
并固定官方 V2-Lite revision，在 RTX 5090 上连续执行 7/27 层，记录 3,240 次真实专家选择、
|
||||
MLA/HF eager cache shapes 与 `31/31` exact 独立复跑；进一步用真实 layer-1 权重执行官方 V3
|
||||
naive/absorb 路径，实际写入 576 元素 latent cache，并以 FP32 将两种结合顺序的最大误差压到
|
||||
@@ -40,13 +40,17 @@ naive/absorb 路径，实际写入 576 元素 latent cache，并以 FP32 将两
|
||||
最新一轮再用官方 chat template 对同一段内容构造 raw / user / generation 三个条件，
|
||||
新增 382,032 次真实路由：精确对齐 2,874 个内容 token 后，RAW→USER 的方向随层与域改变，
|
||||
而 USER→GENERATION 的 21,852 个共享前缀 ordered top-6 全部 exact，验证未来 suffix
|
||||
不能改写过去路由。当前累计 870,912 次公开语料路由。FlashMLA 的 SM90/SM100 官方支持矩阵
|
||||
与本机 SM120 边界单独记账。详见
|
||||
不能改写过去路由。随后又把 system 与 one-shot 组成固定 16 / 17-token 正交增量的
|
||||
2×2 实验，新增 865,440 次真实路由；在目标内容上，加入 one-shot 历史后 system-edge
|
||||
TV 在 24 / 24 个 layer×domain 中都下降，均值从 `0.073` 降至 `0.019`，但这一模式不被
|
||||
越界解释为示例语义或能力提升。当前累计 1,736,352 次公开语料路由。FlashMLA 的
|
||||
SM90/SM100 官方支持矩阵与本机 SM120 边界单独记账。详见
|
||||
[DEEPSEEK_V2_LITE_TRACE.md](./research/DEEPSEEK_V2_LITE_TRACE.md) 与
|
||||
[DEEPSEEK_MLA_ABSORB_AUDIT.md](./research/DEEPSEEK_MLA_ABSORB_AUDIT.md)、
|
||||
[DEEPSEEK_ROUTING_CORPUS_AUDIT.md](./research/DEEPSEEK_ROUTING_CORPUS_AUDIT.md)、
|
||||
[DEEPSEEK_ROUTING_LENGTH_CONTROL_AUDIT.md](./research/DEEPSEEK_ROUTING_LENGTH_CONTROL_AUDIT.md) 与
|
||||
[DEEPSEEK_ROUTING_TEMPLATE_AUDIT.md](./research/DEEPSEEK_ROUTING_TEMPLATE_AUDIT.md)。
|
||||
[DEEPSEEK_ROUTING_TEMPLATE_AUDIT.md](./research/DEEPSEEK_ROUTING_TEMPLATE_AUDIT.md)、
|
||||
[DEEPSEEK_ROUTING_HISTORY_FACTORIAL_AUDIT.md](./research/DEEPSEEK_ROUTING_HISTORY_FACTORIAL_AUDIT.md)。
|
||||
其余专题按进度账本持续扩建。
|
||||
|
||||
## 本地开发
|
||||
|
||||
@@ -205,3 +205,62 @@ da1f10333b2fa269e64f9716a0ca6c1a656d23d3e70b8a25d6e59f8a5b3bc1b9
|
||||
|
||||
See `research/DEEPSEEK_ROUTING_TEMPLATE_AUDIT.md` for the aligned-content
|
||||
contract, paired intervals, per-token route stability, and claim boundaries.
|
||||
|
||||
## System × one-shot message-history factorial
|
||||
|
||||
`v2_lite_routing_history_factorial_probe.py` reuses the exact 128-source,
|
||||
23-content-token cohort from the official-template probe and renders four
|
||||
official chat histories:
|
||||
|
||||
```text
|
||||
S0F0 target user
|
||||
S1F0 fixed system + target user
|
||||
S0F1 fixed demo user/assistant + target user
|
||||
S1F1 fixed system + fixed demo user/assistant + target user
|
||||
```
|
||||
|
||||
Every condition uses `add_generation_prompt=True`. The fixed system treatment
|
||||
adds 16 tokens per source in both `F0` and `F1`; the fixed one-shot treatment
|
||||
adds 17 tokens per source in both `S0` and `S1`. All four variants of one
|
||||
source execute in the same padded batch.
|
||||
|
||||
The output contains:
|
||||
|
||||
- full-input and exact target-content scopes;
|
||||
- token-weighted and prompt-balanced aggregation;
|
||||
- four paired factor edges;
|
||||
- system and one-shot main effects;
|
||||
- difference-in-differences interaction;
|
||||
- per-target-token ordered/set top-6 stability and Jaccard;
|
||||
- 2,000 source-prompt bootstrap replicates shared by all four cells.
|
||||
|
||||
```bash
|
||||
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
|
||||
python -B experiments/deepseek/v2_lite_routing_history_factorial_probe.py \
|
||||
--artifact-dir /path/to/deepseek-v2-lite \
|
||||
--human-eval /path/to/HumanEval.jsonl.gz \
|
||||
--gsm8k /path/to/gsm8k/test.jsonl \
|
||||
--tnews /path/to/tnews/test.json \
|
||||
--tnews-archive /path/to/tnews_public.zip \
|
||||
--wikitext /path/to/wikitext-validation.parquet \
|
||||
--output src/data/deepseek-v2-lite-routing-history-factorial.json \
|
||||
--per-domain 32 \
|
||||
--content-tokens 23 \
|
||||
--batch-prompts 8 \
|
||||
--layers 7 \
|
||||
--bootstrap 2000 \
|
||||
--seed 20260729 \
|
||||
--captured-at 2026-07-29T09:36:00+00:00
|
||||
```
|
||||
|
||||
The four cells add 865,440 real top-6 route selections. The committed run and
|
||||
independent rerun are byte-exact:
|
||||
|
||||
```text
|
||||
5765fbf85fa6fd948cca90f11e2237254fd58c32f373670f44530cff0a70c1fb
|
||||
```
|
||||
|
||||
See `research/DEEPSEEK_ROUTING_HISTORY_FACTORIAL_AUDIT.md` for the factorial
|
||||
definition, all CV intervals, the 24/24 system-edge TV attenuation pattern,
|
||||
per-token stability, and the strict boundary between a message-history effect
|
||||
and a role-semantic claim.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,439 @@
|
||||
# DeepSeek-V2-Lite 路由的消息历史 2×2 因子审计
|
||||
|
||||
> 状态：真实官方 BF16 权重、本机 RTX 5090、layer 0–6 连续 forward。
|
||||
> 固定模型 revision：`604d5664dddd88a0433dbae533b7fe9472482de0`。
|
||||
> 研究单位：上一轮同一批 128 条公开 source prompt 的四种消息历史。
|
||||
> 结论身份：**消息历史组成敏感性探针**，不是能力评测、角色语义、训练分布或线上流量。
|
||||
|
||||
## 1. 为什么上一轮之后还要做这一轮
|
||||
|
||||
上一轮已经把相同内容放入三种输入：
|
||||
|
||||
```text
|
||||
RAW → OFFICIAL USER → OFFICIAL USER + Assistant:
|
||||
```
|
||||
|
||||
它回答了两个问题：
|
||||
|
||||
1. `User:` 前缀会不会改变后续内容 token 的路由？会，而且方向随层与域改变。
|
||||
2. 只在未来追加 `Assistant:` 会不会反向改写过去？不会，21,852 / 21,852 个共享前缀
|
||||
ordered top-6 完全一致。
|
||||
|
||||
但真实对话通常不只有一个 user message。目标请求之前还可能有：
|
||||
|
||||
- system instruction；
|
||||
- 一轮或多轮示例；
|
||||
- assistant 历史；
|
||||
- EOS 与角色边界。
|
||||
|
||||
如果只做“有 system / 无 system”两组，system 的结果还可能取决于它是否直接贴近目标。
|
||||
如果只做“有示例 / 无示例”两组，也无法知道示例的作用是否依赖 system。
|
||||
|
||||
所以本轮使用最小完整设计：
|
||||
|
||||
> **system × one-shot 的 2×2 因子实验。**
|
||||
|
||||
它能同时计算：
|
||||
|
||||
- system 主效应；
|
||||
- one-shot 主效应；
|
||||
- 两者是否可以简单相加；
|
||||
- system 在“直接贴近目标”和“先经过一轮示例”时是否表现相同。
|
||||
|
||||
## 2. 四个条件
|
||||
|
||||
固定文本：
|
||||
|
||||
```text
|
||||
SYSTEM
|
||||
Answer accurately and concisely. 请准确、简洁地回答。
|
||||
|
||||
DEMO USER
|
||||
Reply with OK. 只回复 OK。
|
||||
|
||||
DEMO ASSISTANT
|
||||
OK
|
||||
```
|
||||
|
||||
目标 user 内容来自四个公开域，每条固定为 23 个 regular-tokenizer tokens。
|
||||
四个条件全部使用官方 `apply_chat_template(..., add_generation_prompt=True)`：
|
||||
|
||||
| 条件 | System | One-shot | 官方消息序列 |
|
||||
| --- | ---: | ---: | --- |
|
||||
| S0F0 | 0 | 0 | target user → `Assistant:` |
|
||||
| S1F0 | 1 | 0 | system → target user → `Assistant:` |
|
||||
| S0F1 | 0 | 1 | demo user → demo assistant + EOS → target user → `Assistant:` |
|
||||
| S1F1 | 1 | 1 | system → demo user → demo assistant + EOS → target user → `Assistant:` |
|
||||
|
||||
这里的 `S` 表示 system factor，`F` 表示 few-shot factor。
|
||||
|
||||
### 2.1 为什么称为正交长度增量
|
||||
|
||||
四格的输入 token 总数：
|
||||
|
||||
| 条件 | 输入 tokens | 相对 S0F0 | 每条 source 的固定增量 |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| S0F0 | 3,898 | — | — |
|
||||
| S1F0 | 5,946 | +2,048 | +16 system tokens |
|
||||
| S0F1 | 6,074 | +2,176 | +17 one-shot tokens |
|
||||
| S1F1 | 8,122 | +4,224 | +16 + 17 tokens |
|
||||
|
||||
因此：
|
||||
|
||||
```text
|
||||
S1F0 − S0F0 = S1F1 − S0F1 = 每条 +16 tokens
|
||||
S0F1 − S0F0 = S1F1 − S1F0 = 每条 +17 tokens
|
||||
```
|
||||
|
||||
这并没有把“角色、文本、距离和长度”彼此拆开，但它保证 system 与 one-shot
|
||||
两个处理的 token 增量互不偷换。
|
||||
|
||||
## 3. Cohort 完全复用，而不是重新抽样
|
||||
|
||||
选样 salt 继续固定为：
|
||||
|
||||
```text
|
||||
llm-atlas-deepseek-routing-template-control-v1
|
||||
```
|
||||
|
||||
四域仍为：
|
||||
|
||||
| 域 | 来源 | 字段 | 数量 |
|
||||
| --- | --- | --- | ---: |
|
||||
| 英文百科 | WikiText-2 raw validation | `text` | 32 |
|
||||
| 中文新闻 | CLUE TNEWS public test | `sentence` | 32 |
|
||||
| Python 代码 | OpenAI HumanEval | `prompt` | 32 |
|
||||
| 小学数学 | OpenAI GSM8K test | `question` | 32 |
|
||||
|
||||
本轮 128 个 `(source ID, content SHA-256)` 与上一轮模板探针逐项完全相同。
|
||||
答案不输入，HumanEval 代码不执行。
|
||||
|
||||
这让两个实验可以连续阅读：
|
||||
|
||||
```text
|
||||
上一轮：RAW / USER / GENERATION
|
||||
本轮：USER 条件内部继续展开 system × one-shot 历史
|
||||
```
|
||||
|
||||
## 4. 目标内容如何精确对齐
|
||||
|
||||
四个条件的目标 user 前都有同一个字面前缀 `User: `，因此没有重新引入 RAW 条件的
|
||||
首 token BPE 边界差异。
|
||||
|
||||
脚本仍不依赖这一经验假设，而是为每个目标 token 记录：
|
||||
|
||||
```text
|
||||
(相对目标内容字符起点, 相对终点, token ID)
|
||||
```
|
||||
|
||||
只有四个条件都存在的三元组才进入 `target_content`。
|
||||
|
||||
| 条件 | 目标范围 tokens | 四条件精确交集 |
|
||||
| --- | ---: | ---: |
|
||||
| S0F0 | 2,874 | 2,874 |
|
||||
| S1F0 | 2,874 | 2,874 |
|
||||
| S0F1 | 2,874 | 2,874 |
|
||||
| S1F1 | 2,874 | 2,874 |
|
||||
|
||||
因此本轮目标内容对照没有丢弃任何一个条件特有的目标 token。
|
||||
|
||||
## 5. 执行账
|
||||
|
||||
| 对象 | 数量 |
|
||||
| --- | ---: |
|
||||
| source prompts | 128 |
|
||||
| prompt variants | 512 |
|
||||
| 输入 tokens | 24,040 |
|
||||
| 执行层 | layer 0–6 |
|
||||
| 测量 MoE 层 | layer 1–6 |
|
||||
| 每 token routed experts | top-6 |
|
||||
| S0F0 routes | 140,328 |
|
||||
| S1F0 routes | 214,056 |
|
||||
| S0F1 routes | 218,664 |
|
||||
| S1F1 routes | 292,392 |
|
||||
| 本轮真实 routes | **865,440** |
|
||||
|
||||
与前四个真实语料实验合计：
|
||||
|
||||
```text
|
||||
488,880 长度三 cohort
|
||||
+382,032 RAW / USER / GENERATION
|
||||
+865,440 system × one-shot
|
||||
=1,736,352 次真实 top-6 路由
|
||||
```
|
||||
|
||||
四个条件的同一 source prompt 在同一个 padded batch 中执行；层、checkpoint、dtype、
|
||||
attention implementation 和 bootstrap source indices 都保持一致。
|
||||
|
||||
## 6. 因子统计到底怎么算
|
||||
|
||||
每个 layer × scope × aggregation × domain，先得到四格统计：
|
||||
|
||||
```text
|
||||
m00 = metric(S0F0)
|
||||
m10 = metric(S1F0)
|
||||
m01 = metric(S0F1)
|
||||
m11 = metric(S1F1)
|
||||
```
|
||||
|
||||
然后定义：
|
||||
|
||||
```text
|
||||
system main effect
|
||||
= 0.5 × [(m10 − m00) + (m11 − m01)]
|
||||
|
||||
few-shot main effect
|
||||
= 0.5 × [(m01 − m00) + (m11 − m10)]
|
||||
|
||||
interaction
|
||||
= (m11 − m01) − (m10 − m00)
|
||||
```
|
||||
|
||||
若 interaction 为 0，system 在 F0 和 F1 下的增量相同；若不为 0，两个处理不能简单相加。
|
||||
|
||||
### 6.1 Bootstrap
|
||||
|
||||
- 每域 32 条 source prompt；
|
||||
- 有放回重采样 2,000 次；
|
||||
- 四格使用完全相同的 source indices；
|
||||
- 固定 seed `20260729`；
|
||||
- 主要结果使用 prompt-balanced 聚合；
|
||||
- 同时保留 token-weighted；
|
||||
- 报告 percentile 95% 区间；
|
||||
- 不做假设检验；
|
||||
- 不做多重比较校正。
|
||||
|
||||
## 7. 目标内容 CV：四格、主效应与交互
|
||||
|
||||
下表均为 `target_content / prompt_balanced`。
|
||||
四格顺序是 `S0F0 / S1F0 / S0F1 / S1F1`。
|
||||
|
||||
| 层 | 域 | 四格 CV | System main [95%] | Few-shot main [95%] | Interaction [95%] |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| L1 | 英文 | .596 / .810 / .779 / .737 | +.086 [.057,.114] | +.055 [.023,.088] | -.256 [-.300,-.208] |
|
||||
| L1 | 中文 | .621 / .426 / .621 / .575 | -.121 [-.144,-.090] | +.074 [.046,.099] | +.150 [.100,.192] |
|
||||
| L1 | 代码 | .882 / .949 / .972 / .972 | +.034 [.017,.049] | +.057 [.034,.079] | -.068 [-.101,-.036] |
|
||||
| L1 | 数学 | .793 / .937 / .912 / .867 | +.050 [.023,.075] | +.025 [-.009,.057] | -.188 [-.234,-.137] |
|
||||
| L2 | 英文 | .403 / .339 / .346 / .337 | -.037 [-.049,-.018] | -.029 [-.041,-.012] | +.056 [.023,.080] |
|
||||
| L2 | 中文 | .353 / .316 / .323 / .319 | -.021 [-.027,-.010] | -.014 [-.023,-.001] | +.034 [.011,.050] |
|
||||
| L2 | 代码 | .617 / .652 / .658 / .667 | +.022 [.011,.031] | +.028 [.012,.043] | -.027 [-.045,-.010] |
|
||||
| L2 | 数学 | .455 / .414 / .433 / .412 | -.031 [-.043,-.016] | -.012 [-.028,.005] | +.021 [-.007,.048] |
|
||||
| L3 | 英文 | .423 / .399 / .393 / .379 | -.019 [-.033,-.002] | -.025 [-.040,-.008] | +.010 [-.013,.034] |
|
||||
| L3 | 中文 | .469 / .452 / .455 / .446 | -.013 [-.021,-.002] | -.010 [-.024,.005] | +.007 [-.013,.028] |
|
||||
| L3 | 代码 | .813 / .827 / .835 / .843 | +.011 [.001,.020] | +.019 [.007,.032] | -.006 [-.022,.010] |
|
||||
| L3 | 数学 | .548 / .536 / .540 / .532 | -.010 [-.020,.000] | -.006 [-.019,.007] | +.004 [-.018,.025] |
|
||||
| L4 | 英文 | .558 / .423 / .441 / .413 | -.082 [-.099,-.054] | -.064 [-.073,-.046] | +.107 [.068,.133] |
|
||||
| L4 | 中文 | .851 / .611 / .719 / .683 | -.138 [-.153,-.117] | -.030 [-.051,-.013] | +.204 [.171,.228] |
|
||||
| L4 | 代码 | 1.037 / 1.034 / 1.045 / 1.019 | -.015 [-.023,-.006] | -.003 [-.015,.011] | -.024 [-.038,-.007] |
|
||||
| L4 | 数学 | .647 / .550 / .589 / .580 | -.053 [-.074,-.031] | -.014 [-.031,.006] | +.088 [.052,.118] |
|
||||
| L5 | 英文 | .453 / .437 / .412 / .414 | -.007 [-.021,.008] | -.032 [-.044,-.015] | +.018 [-.008,.037] |
|
||||
| L5 | 中文 | .503 / .469 / .483 / .488 | -.014 [-.025,-.001] | -.001 [-.012,.012] | +.040 [.017,.057] |
|
||||
| L5 | 代码 | .852 / .839 / .851 / .834 | -.015 [-.026,-.004] | -.003 [-.014,.008] | -.004 [-.017,.010] |
|
||||
| L5 | 数学 | .781 / .825 / .828 / .831 | +.023 [.005,.038] | +.026 [.006,.046] | -.041 [-.065,-.013] |
|
||||
| L6 | 英文 | .566 / .489 / .514 / .485 | -.053 [-.070,-.032] | -.028 [-.042,-.010] | +.048 [.014,.076] |
|
||||
| L6 | 中文 | .543 / .539 / .549 / .540 | -.006 [-.017,.006] | +.003 [-.009,.014] | -.005 [-.027,.014] |
|
||||
| L6 | 代码 | .907 / .940 / .921 / .912 | +.012 [.002,.024] | -.007 [-.017,.003] | -.041 [-.061,-.023] |
|
||||
| L6 | 数学 | .775 / .845 / .839 / .836 | +.033 [.013,.051] | +.028 [.010,.045] | -.073 [-.097,-.047] |
|
||||
|
||||
### 7.1 不能写成单向规律
|
||||
|
||||
24 个 layer × domain 中：
|
||||
|
||||
| 效应 | 区间完全正 | 区间完全负 | 跨 0 |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| system main | 8 | 13 | 3 |
|
||||
| few-shot main | 7 | 7 | 10 |
|
||||
| interaction | 8 | 8 | 8 |
|
||||
|
||||
所以不能写：
|
||||
|
||||
- “system 总会让路由更集中”；
|
||||
- “few-shot 总会让路由更均衡”；
|
||||
- “system 与 few-shot 总是互相增强”。
|
||||
|
||||
所有三个方向都随层和域改变。
|
||||
|
||||
## 8. 最清晰的模式：one-shot 历史缓冲了 system 扰动
|
||||
|
||||
比较两条 system 边：
|
||||
|
||||
```text
|
||||
system at F0: S0F0 → S1F0
|
||||
system at F1: S0F1 → S1F1
|
||||
```
|
||||
|
||||
两条边都给目标绝对位置增加相同的 16 tokens。区别是：
|
||||
|
||||
- F0：system 后直接进入目标 user；
|
||||
- F1：system 后先经过 demo user、demo assistant、EOS，再进入目标 user。
|
||||
|
||||
### 8.1 目标路由分布 TV
|
||||
|
||||
各层四域平均：
|
||||
|
||||
| Layer | System TV at F0 | System TV at F1 |
|
||||
| --- | ---: | ---: |
|
||||
| L1 | .107 | .022 |
|
||||
| L2 | .059 | .017 |
|
||||
| L3 | .051 | .015 |
|
||||
| L4 | .085 | .019 |
|
||||
| L5 | .061 | .020 |
|
||||
| L6 | .079 | .022 |
|
||||
| 24 格平均 | **.073** | **.019** |
|
||||
|
||||
结果：
|
||||
|
||||
- 24 / 24 个 layer × domain 的 TV 都变小；
|
||||
- 平均 TV 下降约 74%；
|
||||
- system 边的绝对 CV 变化在 21 / 24 格变小；
|
||||
- 绝对 CV 变化均值从 .068 降至 .016。
|
||||
|
||||
### 8.2 这能否叫“few-shot 稳定路由”
|
||||
|
||||
不能直接这样写。
|
||||
|
||||
本实验固定的是一整段处理：
|
||||
|
||||
```text
|
||||
demo 文本 + user/assistant 角色 + assistant EOS + 更长历史 + 更远的 system
|
||||
```
|
||||
|
||||
因此当前最强且不越界的表述是：
|
||||
|
||||
> 在这组固定消息中，让 system 的影响先穿过一轮 one-shot 历史后，
|
||||
> 目标内容的聚合路由分布对 system toggle 更不敏感。
|
||||
|
||||
要拆开“示例语义、距离、EOS、角色转换和长度”，仍需新的等长正交控制。
|
||||
|
||||
## 9. 逐 token top-6 也看到相同缓冲模式
|
||||
|
||||
下表把四域 2,874 个目标 token 合并；每格为：
|
||||
|
||||
```text
|
||||
top-6 set exact rate / mean Jaccard
|
||||
```
|
||||
|
||||
| Layer | System at F0 | System at F1 | Few-shot at S0 | Few-shot at S1 |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| L1 | .317 / .756 | .789 / .939 | .318 / .751 | .454 / .820 |
|
||||
| L2 | .458 / .820 | .815 / .946 | .473 / .825 | .536 / .857 |
|
||||
| L3 | .477 / .830 | .828 / .951 | .517 / .845 | .589 / .875 |
|
||||
| L4 | .388 / .783 | .793 / .940 | .480 / .825 | .535 / .857 |
|
||||
| L5 | .435 / .798 | .788 / .938 | .500 / .827 | .537 / .857 |
|
||||
| L6 | .389 / .771 | .755 / .928 | .473 / .814 | .530 / .851 |
|
||||
|
||||
system-at-F1 的 top-6 set exact 在六层均大幅高于 system-at-F0。
|
||||
这说明聚合 TV 的下降不是只有少数专家份额抵消后的表象。
|
||||
|
||||
但 set exact 不等于 hidden state exact，也不等于输出答案相同。
|
||||
|
||||
## 10. 完整输入与目标内容是两个问题
|
||||
|
||||
`target_content` 只问：
|
||||
|
||||
> 相同目标 token 在不同左侧历史下如何路由？
|
||||
|
||||
`full_input` 还把 system、demo、EOS、角色前缀与 `Assistant:` 自己都计入：
|
||||
|
||||
> 这四种实际协议流量整体会如何路由？
|
||||
|
||||
完整输入的 CV 主效应点估计如下，顺序为：
|
||||
|
||||
```text
|
||||
system main / few-shot main / interaction
|
||||
```
|
||||
|
||||
| Layer | 英文 | 中文 | 代码 | 数学 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| L1 | -.015 / +.186 / -.243 | -.129 / +.182 / +.020 | -.062 / +.084 / -.092 | -.053 / +.139 / -.212 |
|
||||
| L2 | -.003 / +.128 / -.060 | +.022 / +.125 / -.071 | -.009 / +.094 / -.088 | +.004 / +.124 / -.057 |
|
||||
| L3 | +.035 / +.092 / -.035 | +.027 / +.089 / -.037 | -.058 / -.012 / +.048 | +.012 / +.065 / +.011 |
|
||||
| L4 | -.057 / +.110 / -.027 | -.125 / +.003 / +.041 | -.095 / +.022 / -.056 | -.074 / +.111 / -.004 |
|
||||
| L5 | -.003 / +.114 / +.011 | +.038 / +.139 / -.037 | -.014 / +.067 / -.057 | -.030 / +.063 / +.049 |
|
||||
| L6 | +.012 / +.138 / +.009 | +.061 / +.184 / -.071 | +.010 / +.066 / -.037 | +.017 / +.117 / -.021 |
|
||||
|
||||
完整输入中，one-shot main 大多使 CV 上升；但这包含新增示例 token 本身，不能拿来替代
|
||||
目标内容的上下文条件化结论。
|
||||
|
||||
## 11. 分布效应不是普通 TV
|
||||
|
||||
对 system / few-shot 主效应和 interaction，脚本同时保存 64 维 expert-share effect vector。
|
||||
|
||||
其强度定义为：
|
||||
|
||||
```text
|
||||
0.5 × L1 norm(effect vector)
|
||||
```
|
||||
|
||||
对两组普通概率分布，这个形式等于 TV；但主效应与 interaction vector 可能含正负抵消，
|
||||
所以账本明确称为 `half_l1_magnitude`，不冒充两分布之间的标准 TV。
|
||||
|
||||
普通四条边仍单独报告：
|
||||
|
||||
- total variation；
|
||||
- JSD；
|
||||
- expert-share delta；
|
||||
- paired 95% 区间。
|
||||
|
||||
## 12. 独立复跑
|
||||
|
||||
正式运行与第二次完整运行：
|
||||
|
||||
```text
|
||||
SHA-256
|
||||
5765fbf85fa6fd948cca90f11e2237254fd58c32f373670f44530cff0a70c1fb
|
||||
```
|
||||
|
||||
两份约 28 MiB JSON：
|
||||
|
||||
```text
|
||||
byte-for-byte identical
|
||||
```
|
||||
|
||||
一致内容包含：
|
||||
|
||||
- corpus identity；
|
||||
- 四种官方模板渲染的 token IDs 与 hashes；
|
||||
- layer 0–6 权重与环境身份；
|
||||
- 每 prompt × condition 的 64-expert loads；
|
||||
- 每个目标 token 的 top-6 route hashes；
|
||||
- 四条因子边的 route alignment；
|
||||
- 两种 scope；
|
||||
- 两种 aggregation；
|
||||
- 2,000 次 paired bootstrap 后的全部区间；
|
||||
- system / few-shot / interaction vectors。
|
||||
|
||||
## 13. 解释边界
|
||||
|
||||
本轮可以说：
|
||||
|
||||
- 官方模板下，system 与 one-shot 都会改变目标内容路由；
|
||||
- 变化随层和域不同；
|
||||
- system 与 one-shot 存在明显非加性交互；
|
||||
- 在这组固定示例中，system 先经过 one-shot 历史后，目标路由对 system toggle 更稳定；
|
||||
- 这个模式同时出现在聚合 TV 与逐 token top-6 set/Jaccard。
|
||||
|
||||
本轮不能说：
|
||||
|
||||
- “system 专家”或“few-shot 专家”被识别出来；
|
||||
- one-shot 的语义本身因果地稳定了路由；
|
||||
- 结果能代表完整 27 层；
|
||||
- 结果能代表线上多轮对话；
|
||||
- 路由更稳定就意味着答案更好；
|
||||
- CV 更低就意味着模型能力更强；
|
||||
- 2,000 次 bootstrap 是 2,000 次模型运行；
|
||||
- 区间是经过多重比较校正的显著性结论。
|
||||
|
||||
## 14. 下一步
|
||||
|
||||
这轮已经把 system 与 one-shot 的 token 增量做成正交 2×2。
|
||||
最自然的后续控制是继续拆解“历史缓冲”：
|
||||
|
||||
1. **距离控制**：用等长无语义 filler 把 system 推到相同距离；
|
||||
2. **EOS 控制**：保留相同文本，只切换 assistant EOS / role boundary；
|
||||
3. **角色控制**：相同 token 文本放入 system / user / assistant 不同角色；
|
||||
4. **示例内容控制**：固定长度，替换正确示例、无关示例与冲突示例；
|
||||
5. **完整模型**：获得其余 shards 后扩到全部 27 层；
|
||||
6. **能力联结**：生成答案并用独立任务指标检查路由变化是否与行为相关。
|
||||
|
||||
在这些控制完成之前，“历史缓冲”保持为这组固定官方协议下的描述性实证模式。
|
||||
@@ -398,6 +398,69 @@ await evaluate(`(() => {
|
||||
await pause(120);
|
||||
await screenshot("/tmp/llm-atlas-deepseek-template-results-desktop.png");
|
||||
|
||||
const artifactHistory = await evaluate(`(() => {
|
||||
const root = document.querySelector("[data-dsv2-lab]");
|
||||
root.querySelector('[data-artifact-tab="history"]').click();
|
||||
const read = () => ({
|
||||
panel: root.querySelector("[data-artifact-panel]:not([hidden])").dataset.artifactPanel,
|
||||
matrixRows: root.querySelectorAll(".history-row").length,
|
||||
matrixCells: root.querySelectorAll(".history-row > article").length,
|
||||
domainCards: root.querySelectorAll("[data-history-domain-grid] > article").length,
|
||||
domains: [...root.querySelectorAll("[data-history-domain-grid] > article")].map((node) => ({
|
||||
label: node.querySelector("span").textContent.trim(),
|
||||
cells: [...node.querySelectorAll(".history-cell-values i")].map((cell) => ({
|
||||
name: cell.querySelector("small").textContent.trim(),
|
||||
value: cell.querySelector("b").textContent.trim(),
|
||||
})),
|
||||
effect: node.querySelector("strong").textContent.trim(),
|
||||
className: node.querySelector("strong").className,
|
||||
ci: node.querySelector("p").textContent.trim(),
|
||||
allEffects: node.querySelector(":scope > small").textContent.trim(),
|
||||
edges: node.querySelector("em").textContent.trim(),
|
||||
})),
|
||||
bufferSummary: [...root.querySelectorAll(".history-buffer-summary article b")].map((node) => node.textContent.trim()),
|
||||
bufferCards: [...root.querySelectorAll("[data-history-buffer-grid] > article")].map((node) => ({
|
||||
label: node.querySelector("span").textContent.trim(),
|
||||
tv: node.querySelector("b").textContent.trim(),
|
||||
reduction: node.querySelector("strong").textContent.trim(),
|
||||
stability: node.querySelector("p").textContent.trim(),
|
||||
})),
|
||||
depthRows: root.querySelectorAll("[data-history-depth-map] > div").length,
|
||||
depthCells: root.querySelectorAll("[data-history-depth-map] > div > span").length,
|
||||
depthTitle: root.querySelector("[data-history-depth-title]").textContent.trim(),
|
||||
exact: root.querySelector(".history-ledger .exact b").textContent.trim(),
|
||||
note: root.querySelector("[data-history-note]").textContent.trim(),
|
||||
activeLayer: root.querySelector("[data-history-layer].active").textContent.trim(),
|
||||
activeScope: root.querySelector('[data-history-scope][aria-pressed="true"]').dataset.historyScope,
|
||||
activeMode: root.querySelector('[data-history-mode][aria-pressed="true"]').dataset.historyMode,
|
||||
activeEffect: root.querySelector('[data-history-effect][aria-pressed="true"]').dataset.historyEffect,
|
||||
});
|
||||
const layer1Interaction = read();
|
||||
root.querySelector('[data-history-layer="4"]').click();
|
||||
const layer4Interaction = read();
|
||||
root.querySelector('[data-history-effect="system_main"]').click();
|
||||
const layer4System = read();
|
||||
root.querySelector('[data-history-scope="full_input"]').click();
|
||||
const layer4FullSystem = read();
|
||||
root.querySelector('[data-history-mode="token_weighted"]').click();
|
||||
const layer4FullToken = read();
|
||||
root.querySelector('[data-history-scope="target_content"]').click();
|
||||
root.querySelector('[data-history-mode="prompt_balanced"]').click();
|
||||
root.querySelector('[data-history-effect="interaction"]').click();
|
||||
return { layer1Interaction, layer4Interaction, layer4System, layer4FullSystem, layer4FullToken, restored: read() };
|
||||
})()`);
|
||||
await evaluate(`(() => {
|
||||
document.querySelector("[data-dsv2-lab]").scrollIntoView({ block: "start", behavior: "instant" });
|
||||
window.scrollBy(0, -82);
|
||||
})()`);
|
||||
await pause(180);
|
||||
await screenshot("/tmp/llm-atlas-deepseek-history-desktop.png");
|
||||
await evaluate(`(() => {
|
||||
document.querySelector(".history-domain-grid").scrollIntoView({ block: "center", behavior: "instant" });
|
||||
})()`);
|
||||
await pause(120);
|
||||
await screenshot("/tmp/llm-atlas-deepseek-history-results-desktop.png");
|
||||
|
||||
const artifactEvidence = await evaluate(`(() => {
|
||||
const root = document.querySelector("[data-dsv2-lab]");
|
||||
root.querySelector('[data-artifact-tab="evidence"]').click();
|
||||
@@ -476,6 +539,12 @@ const mobile = await evaluate(`(() => {
|
||||
templateModes: artifact.querySelectorAll("[data-template-mode]").length,
|
||||
templateDomainCards: artifact.querySelectorAll("[data-template-domain-grid] > article").length,
|
||||
templateDepthCells: artifact.querySelectorAll("[data-template-depth-map] > div > span").length,
|
||||
historyLayers: artifact.querySelectorAll("[data-history-layer]").length,
|
||||
historyScopes: artifact.querySelectorAll("[data-history-scope]").length,
|
||||
historyModes: artifact.querySelectorAll("[data-history-mode]").length,
|
||||
historyEffects: artifact.querySelectorAll("[data-history-effect]").length,
|
||||
historyDomainCards: artifact.querySelectorAll("[data-history-domain-grid] > article").length,
|
||||
historyDepthCells: artifact.querySelectorAll("[data-history-depth-map] > div > span").length,
|
||||
offenders: [...document.querySelectorAll("body *")]
|
||||
.filter((node) => !node.closest(".paper-chain, .advantage-table, .precision-table, .mapping-table, [data-deepseek-lab], [data-dsv2-lab]"))
|
||||
.filter((node) => node.getBoundingClientRect().right > document.documentElement.clientWidth + 1)
|
||||
@@ -517,8 +586,22 @@ await evaluate(`(() => {
|
||||
})()`);
|
||||
await pause(120);
|
||||
await screenshot("/tmp/llm-atlas-deepseek-template-results-mobile.png");
|
||||
await evaluate(`(() => {
|
||||
const artifact = document.querySelector("[data-dsv2-lab]");
|
||||
artifact.querySelector('[data-artifact-tab="history"]').click();
|
||||
artifact.scrollIntoView({ block: "start", behavior: "instant" });
|
||||
window.scrollBy(0, -70);
|
||||
})()`);
|
||||
await pause(180);
|
||||
await screenshot("/tmp/llm-atlas-deepseek-history-mobile.png");
|
||||
await evaluate(`(() => {
|
||||
document.querySelector(".history-domain-grid").scrollIntoView({ block: "start", behavior: "instant" });
|
||||
window.scrollBy(0, -72);
|
||||
})()`);
|
||||
await pause(120);
|
||||
await screenshot("/tmp/llm-atlas-deepseek-history-results-mobile.png");
|
||||
|
||||
const report = { overview, capacity, cache, codesign, rl, artifactRoute, artifactLoad, artifactCache, artifactAbsorb, artifactCorpus, artifactTemplate, artifactEvidence, home, papers, mobile, exceptions };
|
||||
const report = { overview, capacity, cache, codesign, rl, artifactRoute, artifactLoad, artifactCache, artifactAbsorb, artifactCorpus, artifactTemplate, artifactHistory, artifactEvidence, home, papers, mobile, exceptions };
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
|
||||
const numeric = (text) => Number.parseFloat(text.replaceAll(",", ""));
|
||||
@@ -528,8 +611,8 @@ if (overview.sections !== 26 || overview.tocLinks !== 26) failures.push("二十
|
||||
if (overview.ledgers !== 24 || overview.waves !== 10) failures.push("二十四张问题账或十次转向结构异常");
|
||||
if (overview.paperLinks !== 60 || overview.branches !== 5 || overview.followups !== 1) failures.push("论文链、旁支或公开后续标记异常");
|
||||
if (overview.labTabs !== 4 || overview.labPanels !== 4) failures.push("四联实验结构异常");
|
||||
if (overview.artifactTabs !== 7 || overview.artifactPanels !== 7 || overview.artifactLayers !== 27) failures.push("真实权重七联实验结构异常");
|
||||
if (overview.heroLabs !== "11 个可操作实验") failures.push("DeepSeek 实验总数账异常");
|
||||
if (overview.artifactTabs !== 8 || overview.artifactPanels !== 8 || overview.artifactLayers !== 27) failures.push("真实权重八联实验结构异常");
|
||||
if (overview.heroLabs !== "12 个可操作实验") failures.push("DeepSeek 实验总数账异常");
|
||||
if (overview.navLinks !== 20 || home.navLinks !== 20 || mobile.mobileLinks !== 20 || overview.activeNav !== "DeepSeek") failures.push("全站导航未同步 DeepSeek");
|
||||
if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出");
|
||||
if (capacity.initial.panel !== "capacity" || capacity.initial.total !== "32.1× FFN" || capacity.initial.active !== "1.13× FFN") failures.push("V3 稀疏容量初始账异常");
|
||||
@@ -569,12 +652,19 @@ if (artifactTemplate.layer1Content.prefixExact !== "3,642 / 3,642 EXACT · L1" |
|
||||
if (artifactTemplate.layer6Content.domains.some((domain) => domain.className !== "up") || artifactTemplate.layer6Content.domains[0].delta !== "Δ +0.060" || artifactTemplate.layer6Content.domains[2].delta !== "Δ +0.058") failures.push("L6 对齐内容跨域方向异常");
|
||||
if (artifactTemplate.layer6Full.domains[0].delta !== "Δ +0.134" || !artifactTemplate.layer6Full.note.includes("完整输入") || artifactTemplate.layer6Full.activeScope !== "full_input") failures.push("模板完整输入 scope 切换异常");
|
||||
if (artifactTemplate.layer6FullToken.activeMode !== "token_weighted" || artifactTemplate.layer2Content.activeLayer !== "L2" || artifactTemplate.layer2Content.domains[2].delta !== "Δ -0.067") failures.push("模板层或聚合口径切换异常");
|
||||
if (artifactHistory.layer1Interaction.panel !== "history" || artifactHistory.layer1Interaction.matrixRows !== 2 || artifactHistory.layer1Interaction.matrixCells !== 4 || artifactHistory.layer1Interaction.domainCards !== 4 || artifactHistory.layer1Interaction.depthRows !== 4 || artifactHistory.layer1Interaction.depthCells !== 24 || artifactHistory.layer1Interaction.exact !== "BYTE-EXACT") failures.push("消息历史 2×2 结构或独立复跑闸门异常");
|
||||
if (artifactHistory.layer1Interaction.domains[0].cells.map((cell) => cell.value).join("/") !== "0.596/0.810/0.779/0.737" || artifactHistory.layer1Interaction.domains[0].effect !== "INTERACTION · Δ -0.256" || artifactHistory.layer1Interaction.domains[1].effect !== "INTERACTION · Δ +0.150") failures.push("L1 消息历史四格或 interaction 统计异常");
|
||||
if (artifactHistory.layer1Interaction.bufferSummary.join("|") !== "24 / 24 ↓|.073 → .019|21 / 24 ↓|HISTORY BUFFER" || artifactHistory.layer1Interaction.bufferCards.length !== 4 || !artifactHistory.layer1Interaction.bufferCards[0].tv.startsWith("TV ") || !artifactHistory.layer1Interaction.bufferCards[0].stability.includes("top-6 set exact")) failures.push("消息历史缓冲总账或逐 token 稳定性异常");
|
||||
if (artifactHistory.layer4Interaction.domains[1].cells.map((cell) => cell.value).join("/") !== "0.851/0.611/0.719/0.683" || artifactHistory.layer4Interaction.domains[1].effect !== "INTERACTION · Δ +0.204") failures.push("L4 中文消息历史 interaction 异常");
|
||||
if (artifactHistory.layer4System.domains[0].effect !== "SYSTEM MAIN · Δ -0.082" || artifactHistory.layer4System.activeEffect !== "system_main" || !artifactHistory.layer4System.depthTitle.includes("System main")) failures.push("消息历史 effect 切换异常");
|
||||
if (artifactHistory.layer4FullSystem.domains[0].effect !== "SYSTEM MAIN · Δ -0.057" || artifactHistory.layer4FullSystem.activeScope !== "full_input" || !artifactHistory.layer4FullSystem.note.includes("完整输入")) failures.push("消息历史完整输入 scope 异常");
|
||||
if (artifactHistory.layer4FullToken.activeMode !== "token_weighted" || artifactHistory.restored.activeScope !== "target_content" || artifactHistory.restored.activeMode !== "prompt_balanced" || artifactHistory.restored.activeEffect !== "interaction") failures.push("消息历史聚合口径或恢复状态异常");
|
||||
if (artifactEvidence.panel !== "evidence" || artifactEvidence.layers !== 27 || artifactEvidence.executed !== 7 || artifactEvidence.split !== 1 || artifactEvidence.unloaded !== 19 || artifactEvidence.exact !== "31 / 31") failures.push("真实工件执行边界或复跑闸门异常");
|
||||
if (!artifactEvidence.dependency.includes("Transformers 5.5") || !artifactEvidence.dependency.includes("4.41.2") || !artifactEvidence.boundary.includes("完整 27 层生成")) failures.push("依赖版本或未覆盖边界异常");
|
||||
if (artifactEvidence.keyboardSelected !== "load" || artifactEvidence.keyboardVisible !== "load") failures.push("真实工件实验键盘 tab 导航异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/" || home.paperCount !== "486") failures.push("首页 DeepSeek 首发入口或论文数异常");
|
||||
if (papers.total !== 486 || !papers.hasFilter || papers.visible < 20 || !papers.hasCoder || !papers.hasEngram) failures.push("论文库 DeepSeek 聚光异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4 || mobile.artifactTabs !== 7 || mobile.artifactHeatCells !== 64 || mobile.corpusCohorts !== 3 || mobile.lengthDeltaCards !== 4 || mobile.templateLayers !== 6 || mobile.templateScopes !== 2 || mobile.templateModes !== 2 || mobile.templateDomainCards !== 4 || mobile.templateDepthCells !== 24) failures.push("移动端导航或实验异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4 || mobile.artifactTabs !== 8 || mobile.artifactHeatCells !== 64 || mobile.corpusCohorts !== 3 || mobile.lengthDeltaCards !== 4 || mobile.templateLayers !== 6 || mobile.templateScopes !== 2 || mobile.templateModes !== 2 || mobile.templateDomainCards !== 4 || mobile.templateDepthCells !== 24 || mobile.historyLayers !== 6 || mobile.historyScopes !== 2 || mobile.historyModes !== 2 || mobile.historyEffects !== 3 || mobile.historyDomainCards !== 4 || mobile.historyDepthCells !== 24) failures.push("移动端导航或实验异常");
|
||||
if (mobile.offenders.length) failures.push(`移动端越界元素：${JSON.stringify(mobile.offenders)}`);
|
||||
if (exceptions.length) failures.push(`浏览器异常：${exceptions.join(" | ")}`);
|
||||
|
||||
|
||||
@@ -12,6 +12,8 @@ import rawMatched24Repro from "@/data/deepseek-v2-lite-routing-matched24-repro.j
|
||||
import rawLengthSensitivity from "@/data/deepseek-v2-lite-routing-length-sensitivity.json";
|
||||
import rawTemplate from "@/data/deepseek-v2-lite-routing-template.json";
|
||||
import rawTemplateRepro from "@/data/deepseek-v2-lite-routing-template-repro.json";
|
||||
import rawHistory from "@/data/deepseek-v2-lite-routing-history-factorial.json";
|
||||
import rawHistoryRepro from "@/data/deepseek-v2-lite-routing-history-factorial-repro.json";
|
||||
|
||||
const trace = rawTrace as any;
|
||||
const repro = rawRepro as any;
|
||||
@@ -26,11 +28,14 @@ const matched24Repro = rawMatched24Repro as any;
|
||||
const lengthSensitivity = rawLengthSensitivity as any;
|
||||
const template = rawTemplate as any;
|
||||
const templateRepro = rawTemplateRepro as any;
|
||||
const history = rawHistory as any;
|
||||
const historyRepro = rawHistoryRepro as any;
|
||||
const absorbExact = JSON.stringify(absorb) === JSON.stringify(absorbRepro);
|
||||
const corpusExact = JSON.stringify(corpus) === JSON.stringify(corpusRepro);
|
||||
const matched16Exact = JSON.stringify(matched16) === JSON.stringify(matched16Repro);
|
||||
const matched24Exact = JSON.stringify(matched24) === JSON.stringify(matched24Repro);
|
||||
const templateExact = JSON.stringify(template) === JSON.stringify(templateRepro);
|
||||
const historyExact = JSON.stringify(history) === JSON.stringify(historyRepro);
|
||||
const bytes = (value: number) => value >= 1024
|
||||
? `${(value / 1024).toFixed(2)} KiB`
|
||||
: `${value.toLocaleString()} B`;
|
||||
@@ -144,6 +149,101 @@ const templateCompact = {
|
||||
})),
|
||||
};
|
||||
const templateCompactJson = JSON.stringify(templateCompact).replaceAll("<", "\\u003c");
|
||||
const historyConditions = ["s0f0", "s1f0", "s0f1", "s1f1"];
|
||||
const historyEdges = ["system_at_f0", "system_at_f1", "fewshot_at_s0", "fewshot_at_s1"];
|
||||
const aggregateHistoryAlignment = (layer: any, domain: string, edge: string) => {
|
||||
const rows = layer.prompts
|
||||
.filter((prompt: any) => prompt.domain === domain)
|
||||
.map((prompt: any) => prompt.alignments[edge]);
|
||||
const aligned = rows.reduce((sum: number, row: any) => sum + row.aligned_tokens, 0);
|
||||
const setExact = rows.reduce((sum: number, row: any) => sum + row.set_topk_exact, 0);
|
||||
const orderedExact = rows.reduce((sum: number, row: any) => sum + row.ordered_topk_exact, 0);
|
||||
const weightedJaccard = rows.reduce(
|
||||
(sum: number, row: any) => sum + row.mean_jaccard * row.aligned_tokens,
|
||||
0,
|
||||
);
|
||||
return {
|
||||
aligned,
|
||||
setExactRate: setExact / aligned,
|
||||
orderedExactRate: orderedExact / aligned,
|
||||
meanJaccard: weightedJaccard / aligned,
|
||||
};
|
||||
};
|
||||
const historyCompact = {
|
||||
domains: history.corpus_contract.domains,
|
||||
labels: history.corpus_contract.domain_labels,
|
||||
inference: history.inference_contract,
|
||||
messages: {
|
||||
system: history.message_history_contract.system_message,
|
||||
demoUser: history.message_history_contract.demo_user,
|
||||
demoAssistant: history.message_history_contract.demo_assistant,
|
||||
},
|
||||
exact: historyExact,
|
||||
layers: history.layers.slice(1).map((layer: any) => ({
|
||||
layer: layer.layer,
|
||||
alignment: Object.fromEntries(
|
||||
history.corpus_contract.domains.map((domain: string) => [
|
||||
domain,
|
||||
Object.fromEntries(
|
||||
historyEdges.map((edge) => [
|
||||
edge,
|
||||
aggregateHistoryAlignment(layer, domain, edge),
|
||||
]),
|
||||
),
|
||||
]),
|
||||
),
|
||||
scopes: Object.fromEntries(
|
||||
["target_content", "full_input"].map((scope) => [
|
||||
scope,
|
||||
{
|
||||
modes: Object.fromEntries(
|
||||
["prompt_balanced", "token_weighted"].map((mode) => {
|
||||
const statistics = layer.statistics[scope].modes[mode];
|
||||
return [
|
||||
mode,
|
||||
{
|
||||
conditions: Object.fromEntries(
|
||||
historyConditions.map((condition) => [
|
||||
condition,
|
||||
Object.fromEntries(
|
||||
history.corpus_contract.domains.map((domain: string) => [
|
||||
domain,
|
||||
statistics.conditions[condition][domain].metrics.cv.point,
|
||||
]),
|
||||
),
|
||||
]),
|
||||
),
|
||||
factorial: Object.fromEntries(
|
||||
history.corpus_contract.domains.map((domain: string) => [
|
||||
domain,
|
||||
statistics.factorial[domain].metric_effects.cv,
|
||||
]),
|
||||
),
|
||||
comparisons: Object.fromEntries(
|
||||
historyEdges.map((edge) => [
|
||||
edge,
|
||||
Object.fromEntries(
|
||||
history.corpus_contract.domains.map((domain: string) => [
|
||||
domain,
|
||||
{
|
||||
cv: statistics.comparisons[edge][domain].metrics.cv,
|
||||
tv: statistics.comparisons[edge][domain].total_variation,
|
||||
jsd: statistics.comparisons[edge][domain].js_divergence,
|
||||
},
|
||||
]),
|
||||
),
|
||||
]),
|
||||
),
|
||||
},
|
||||
];
|
||||
}),
|
||||
),
|
||||
},
|
||||
]),
|
||||
),
|
||||
})),
|
||||
};
|
||||
const historyCompactJson = JSON.stringify(historyCompact).replaceAll("<", "\\u003c");
|
||||
const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoint_tensor_bytes;
|
||||
---
|
||||
|
||||
@@ -155,7 +255,7 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
</div>
|
||||
<p>
|
||||
固定官方 revision、tokenizer、模型代码和 BF16 第一分片；RTX 5090 连续执行 layer 0–6，
|
||||
从 3,240 次 token 显微轨迹扩到 870,912 次公开语料路由，并让 layer-1 权重继续走入官方吸收式 cache。
|
||||
从 3,240 次 token 显微轨迹扩到 1,736,352 次公开语料路由，并让 layer-1 权重继续走入官方吸收式 cache。
|
||||
所有结论都带证据身份与停止线。
|
||||
</p>
|
||||
</header>
|
||||
@@ -186,8 +286,11 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
<button type="button" role="tab" data-artifact-tab="template" aria-selected="false" tabindex="-1">
|
||||
<span>06</span><b>官方模板扰动</b><small>raw → user → assistant</small>
|
||||
</button>
|
||||
<button type="button" role="tab" data-artifact-tab="history" aria-selected="false" tabindex="-1">
|
||||
<span>07</span><b>消息历史 2×2</b><small>system × one-shot</small>
|
||||
</button>
|
||||
<button type="button" role="tab" data-artifact-tab="evidence" aria-selected="false" tabindex="-1">
|
||||
<span>07</span><b>证据断面</b><small>revision · shards · rerun</small>
|
||||
<span>08</span><b>证据断面</b><small>revision · shards · rerun</small>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
@@ -717,6 +820,136 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="artifact-panel" data-artifact-panel="history" hidden>
|
||||
<div class="panel-lead">
|
||||
<div><span>X / MESSAGE-HISTORY FACTORIAL</span><h4>把 system 与 one-shot 拆成四格，而不是只做两组对比</h4></div>
|
||||
<p>
|
||||
同一批 128 条目标内容，在官方模板中切换两个固定处理。system 每条恒增 16 tokens，
|
||||
one-shot 每条恒增 17 tokens；四格同 batch，目标内容精确对齐。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="history-ledger">
|
||||
<article><span>SOURCE PROMPTS</span><b>128</b><p>与上一模板探针逐项同 cohort</p></article>
|
||||
<article><span>2×2 VARIANTS</span><b>512</b><p>S0F0 · S1F0 · S0F1 · S1F1</p></article>
|
||||
<article><span>INPUT TOKENS</span><b>24,040</b><p>四格完整协议输入</p></article>
|
||||
<article><span>REAL ROUTES</span><b>865,440</b><p>四格 × 前六个 MoE 层</p></article>
|
||||
<article><span>ALIGNED TARGET</span><b>2,874 × 4</b><p>相同字符跨度与 token ID</p></article>
|
||||
<article class="exact"><span>INDEPENDENT RERUN</span><b>{historyExact ? "BYTE-EXACT" : "MISMATCH"}</b><p>完整 JSON SHA-256 5765fbf8…c1fb</p></article>
|
||||
</div>
|
||||
|
||||
<div class="history-factorial" aria-label="System 与 one-shot 的二乘二实验矩阵">
|
||||
<div class="history-axis">
|
||||
<span>ONE-SHOT FACTOR →</span>
|
||||
<b>F0 · 无示例</b>
|
||||
<b>F1 · 固定 user / assistant 示例</b>
|
||||
</div>
|
||||
<div class="history-row">
|
||||
<strong>S0<br /><small>无 SYSTEM</small></strong>
|
||||
<article>
|
||||
<span>S0F0 / BASE</span>
|
||||
<div><i>USER TARGET</i><em>Assistant:</em></div>
|
||||
<p>每条约 30 tokens；只含目标单轮。</p>
|
||||
</article>
|
||||
<article>
|
||||
<span>S0F1 / ONE-SHOT</span>
|
||||
<div><b>USER DEMO</b><b>ASSISTANT · EOS</b><i>USER TARGET</i><em>Assistant:</em></div>
|
||||
<p>相对 base 每条固定 +17 tokens。</p>
|
||||
</article>
|
||||
</div>
|
||||
<div class="history-row">
|
||||
<strong>S1<br /><small>固定 SYSTEM</small></strong>
|
||||
<article>
|
||||
<span>S1F0 / SYSTEM</span>
|
||||
<div><u>SYSTEM</u><i>USER TARGET</i><em>Assistant:</em></div>
|
||||
<p>相对 base 每条固定 +16 tokens。</p>
|
||||
</article>
|
||||
<article>
|
||||
<span>S1F1 / COMBINED</span>
|
||||
<div><u>SYSTEM</u><b>USER DEMO</b><b>ASSISTANT · EOS</b><i>USER TARGET</i><em>Assistant:</em></div>
|
||||
<p>两种增量严格相加：+16 +17 tokens。</p>
|
||||
</article>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="history-controls">
|
||||
<div>
|
||||
<span>MOE LAYER</span>
|
||||
<div class="layer-switch history-layer-switch" role="group" aria-label="选择消息历史层">
|
||||
{[1, 2, 3, 4, 5, 6].map((layer) => (
|
||||
<button type="button" data-history-layer={layer} class={layer === 1 ? "active" : ""}>L{layer}</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<span>MEASUREMENT SCOPE</span>
|
||||
<div class="history-scope-switch" role="group" aria-label="选择消息历史统计范围">
|
||||
<button type="button" data-history-scope="target_content" aria-pressed="true">目标内容</button>
|
||||
<button type="button" data-history-scope="full_input" aria-pressed="false">完整输入</button>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<span>AGGREGATION</span>
|
||||
<div class="history-mode-switch" role="group" aria-label="选择消息历史聚合口径">
|
||||
<button type="button" data-history-mode="prompt_balanced" aria-pressed="true">prompt 等权</button>
|
||||
<button type="button" data-history-mode="token_weighted" aria-pressed="false">token 加权</button>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<span>DEPTH MAP EFFECT</span>
|
||||
<div class="history-effect-switch" role="group" aria-label="选择消息历史因子效应">
|
||||
<button type="button" data-history-effect="system_main" aria-pressed="false">System</button>
|
||||
<button type="button" data-history-effect="fewshot_main" aria-pressed="false">One-shot</button>
|
||||
<button type="button" data-history-effect="interaction" aria-pressed="true">Interaction</button>
|
||||
</div>
|
||||
</div>
|
||||
<p data-history-note>
|
||||
目标内容：四格只保留相同相对字符跨度与相同 token ID；下方 Δ 均为 CV 的 source-paired 2×2 效应。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="history-domain-grid" data-history-domain-grid></div>
|
||||
|
||||
<div class="history-buffer-summary">
|
||||
<article><span>SYSTEM EDGE TV</span><b>24 / 24 ↓</b><p>有 one-shot 时，六层四域的 system-edge TV 全部下降。</p></article>
|
||||
<article><span>MEAN TARGET TV</span><b>.073 → .019</b><p>system at F0 → system at F1，平均下降约 74%。</p></article>
|
||||
<article><span>|Δ CV|</span><b>21 / 24 ↓</b><p>绝对 CV system effect 从均值 .068 降至 .016。</p></article>
|
||||
<article><span>BOUNDARY</span><b>HISTORY BUFFER</b><p>不能单独归因给示例语义；距离、EOS、角色与文本共同变化。</p></article>
|
||||
</div>
|
||||
|
||||
<div class="history-buffer">
|
||||
<div>
|
||||
<span>CURRENT LAYER / SYSTEM EDGE</span>
|
||||
<h5>同样增加 16 tokens，先经过一轮历史后变化更小</h5>
|
||||
<p>每域同时显示目标路由 TV 与逐 token top-6 set exact；左为 F0，右为 F1。</p>
|
||||
</div>
|
||||
<div data-history-buffer-grid></div>
|
||||
</div>
|
||||
|
||||
<div class="history-depth">
|
||||
<div>
|
||||
<span>DEPTH MAP / Δ CV</span>
|
||||
<h5 data-history-depth-title>Interaction：两个处理是否可以简单相加</h5>
|
||||
<p>绿色为 CV 下降，红色为 CV 上升；颜色身份只表示方向，不表示能力好坏。</p>
|
||||
</div>
|
||||
<div data-history-depth-map></div>
|
||||
</div>
|
||||
|
||||
<div class="evidence-links">
|
||||
<a href="https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite/blob/main/tokenizer_config.json" rel="noreferrer">官方 tokenizer_config ↗</a>
|
||||
<a href="https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite" rel="noreferrer">官方 V2-Lite 模型卡 ↗</a>
|
||||
<a href="https://arxiv.org/abs/2405.04434" rel="noreferrer">DeepSeek-V2 技术报告 ↗</a>
|
||||
</div>
|
||||
|
||||
<div class="artifact-boundary">
|
||||
<b>HISTORY COMPOSITION, NOT ROLE SEMANTICS</b>
|
||||
<p>
|
||||
这组 2×2 同时改变固定文本、角色边界、EOS、距离与长度；“24 / 24 TV 下降”是本探针中的
|
||||
历史缓冲模式，不证明 one-shot 语义本身稳定了路由，更不证明答案质量提升。
|
||||
</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="artifact-panel" data-artifact-panel="evidence" hidden>
|
||||
<div class="panel-lead">
|
||||
<div><span>O + X / EVIDENCE SLICE</span><h4>为什么执行到 layer 6 就停，而不是把“部分下载”写成“完整复现”</h4></div>
|
||||
@@ -810,12 +1043,15 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
<code>experiments/deepseek/compare_routing_length_control.py</code> ·
|
||||
<code>research/DEEPSEEK_ROUTING_LENGTH_CONTROL_AUDIT.md</code> ·
|
||||
<code>experiments/deepseek/v2_lite_routing_template_probe.py</code> ·
|
||||
<code>research/DEEPSEEK_ROUTING_TEMPLATE_AUDIT.md</code>
|
||||
<code>research/DEEPSEEK_ROUTING_TEMPLATE_AUDIT.md</code> ·
|
||||
<code>experiments/deepseek/v2_lite_routing_history_factorial_probe.py</code> ·
|
||||
<code>research/DEEPSEEK_ROUTING_HISTORY_FACTORIAL_AUDIT.md</code>
|
||||
</figcaption>
|
||||
|
||||
<script is:inline type="application/json" data-dsv2-trace set:html={compactJson}></script>
|
||||
<script is:inline type="application/json" data-dsv2-corpus set:html={corpusCompactJson}></script>
|
||||
<script is:inline type="application/json" data-dsv2-template set:html={templateCompactJson}></script>
|
||||
<script is:inline type="application/json" data-dsv2-history set:html={historyCompactJson}></script>
|
||||
</figure>
|
||||
|
||||
<script>
|
||||
@@ -829,10 +1065,17 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
const payloadNode = one<HTMLScriptElement>("[data-dsv2-trace]");
|
||||
const corpusNode = one<HTMLScriptElement>("[data-dsv2-corpus]");
|
||||
const templateNode = one<HTMLScriptElement>("[data-dsv2-template]");
|
||||
if (!payloadNode?.textContent || !corpusNode?.textContent || !templateNode?.textContent) return;
|
||||
const historyNode = one<HTMLScriptElement>("[data-dsv2-history]");
|
||||
if (
|
||||
!payloadNode?.textContent
|
||||
|| !corpusNode?.textContent
|
||||
|| !templateNode?.textContent
|
||||
|| !historyNode?.textContent
|
||||
) return;
|
||||
const data = JSON.parse(payloadNode.textContent);
|
||||
const corpusData = JSON.parse(corpusNode.textContent);
|
||||
const templateData = JSON.parse(templateNode.textContent);
|
||||
const historyData = JSON.parse(historyNode.textContent);
|
||||
|
||||
const tabs = all<HTMLButtonElement>("[data-artifact-tab]");
|
||||
const panels = all<HTMLElement>("[data-artifact-panel]");
|
||||
@@ -1405,6 +1648,175 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
});
|
||||
});
|
||||
renderTemplate();
|
||||
|
||||
let historyLayerNumber = 1;
|
||||
let historyScope = "target_content";
|
||||
let historyMode = "prompt_balanced";
|
||||
let historyEffect = "interaction";
|
||||
const historyEffectLabels: Record<string, string> = {
|
||||
system_main: "SYSTEM MAIN",
|
||||
fewshot_main: "ONE-SHOT MAIN",
|
||||
interaction: "INTERACTION",
|
||||
};
|
||||
const historyConditionLabels: Record<string, string> = {
|
||||
s0f0: "S0F0",
|
||||
s1f0: "S1F0",
|
||||
s0f1: "S0F1",
|
||||
s1f1: "S1F1",
|
||||
};
|
||||
const renderHistory = () => {
|
||||
all<HTMLButtonElement>("[data-history-layer]").forEach((button) => {
|
||||
button.classList.toggle(
|
||||
"active",
|
||||
Number(button.dataset.historyLayer) === historyLayerNumber,
|
||||
);
|
||||
});
|
||||
all<HTMLButtonElement>("[data-history-scope]").forEach((button) => {
|
||||
button.setAttribute(
|
||||
"aria-pressed",
|
||||
String(button.dataset.historyScope === historyScope),
|
||||
);
|
||||
});
|
||||
all<HTMLButtonElement>("[data-history-mode]").forEach((button) => {
|
||||
button.setAttribute(
|
||||
"aria-pressed",
|
||||
String(button.dataset.historyMode === historyMode),
|
||||
);
|
||||
});
|
||||
all<HTMLButtonElement>("[data-history-effect]").forEach((button) => {
|
||||
button.setAttribute(
|
||||
"aria-pressed",
|
||||
String(button.dataset.historyEffect === historyEffect),
|
||||
);
|
||||
});
|
||||
set(
|
||||
"[data-history-note]",
|
||||
historyScope === "target_content"
|
||||
? "目标内容：四格只保留相同相对字符跨度与相同 token ID；下方 Δ 均为 CV 的 source-paired 2×2 效应。"
|
||||
: "完整输入：system、demo、EOS、目标与 Assistant: 全部进入统计；它回答协议流量，不等同于目标内容本身。",
|
||||
);
|
||||
|
||||
const currentLayer = historyData.layers.find(
|
||||
(item: any) => item.layer === historyLayerNumber,
|
||||
);
|
||||
const view = currentLayer.scopes[historyScope].modes[historyMode];
|
||||
const grid = one<HTMLElement>("[data-history-domain-grid]");
|
||||
if (grid) {
|
||||
grid.replaceChildren(...historyData.domains.map((domain: string) => {
|
||||
const card = document.createElement("article");
|
||||
const label = document.createElement("span");
|
||||
const cells = document.createElement("div");
|
||||
const effect = view.factorial[domain][historyEffect];
|
||||
const primary = document.createElement("strong");
|
||||
const ci = document.createElement("p");
|
||||
const allEffects = document.createElement("small");
|
||||
const edges = document.createElement("em");
|
||||
label.textContent = corpusLabels[domain];
|
||||
cells.className = "history-cell-values";
|
||||
["s0f0", "s1f0", "s0f1", "s1f1"].forEach((condition) => {
|
||||
const cell = document.createElement("i");
|
||||
const name = document.createElement("small");
|
||||
const value = document.createElement("b");
|
||||
name.textContent = historyConditionLabels[condition];
|
||||
value.textContent = view.conditions[condition][domain].toFixed(3);
|
||||
cell.append(name, value);
|
||||
cells.append(cell);
|
||||
});
|
||||
primary.textContent = `${historyEffectLabels[historyEffect]} · Δ ${signed(effect.point)}`;
|
||||
primary.className = deltaClass(effect.ci95);
|
||||
ci.textContent = `source-paired 95% ${formatSignedCi(effect.ci95)}`;
|
||||
allEffects.textContent = [
|
||||
`S ${signed(view.factorial[domain].system_main.point)}`,
|
||||
`F ${signed(view.factorial[domain].fewshot_main.point)}`,
|
||||
`I ${signed(view.factorial[domain].interaction.point)}`,
|
||||
].join(" · ");
|
||||
edges.textContent = [
|
||||
`system TV at F0 ${view.comparisons.system_at_f0[domain].tv.point.toFixed(3)}`,
|
||||
`at F1 ${view.comparisons.system_at_f1[domain].tv.point.toFixed(3)}`,
|
||||
].join(" → ");
|
||||
card.append(label, cells, primary, ci, allEffects, edges);
|
||||
return card;
|
||||
}));
|
||||
}
|
||||
|
||||
const bufferView = currentLayer.scopes.target_content
|
||||
.modes.prompt_balanced;
|
||||
const bufferGrid = one<HTMLElement>("[data-history-buffer-grid]");
|
||||
if (bufferGrid) {
|
||||
bufferGrid.replaceChildren(...historyData.domains.map((domain: string) => {
|
||||
const card = document.createElement("article");
|
||||
const label = document.createElement("span");
|
||||
const tv = document.createElement("b");
|
||||
const reduction = document.createElement("strong");
|
||||
const stability = document.createElement("p");
|
||||
const f0 = bufferView.comparisons.system_at_f0[domain].tv.point;
|
||||
const f1 = bufferView.comparisons.system_at_f1[domain].tv.point;
|
||||
const alignF0 = currentLayer.alignment[domain].system_at_f0;
|
||||
const alignF1 = currentLayer.alignment[domain].system_at_f1;
|
||||
label.textContent = corpusLabels[domain];
|
||||
tv.textContent = `TV ${f0.toFixed(3)} → ${f1.toFixed(3)}`;
|
||||
reduction.textContent = `↓ ${((1 - f1 / f0) * 100).toFixed(0)}%`;
|
||||
stability.textContent = `top-6 set exact ${(alignF0.setExactRate * 100).toFixed(1)}% → ${(alignF1.setExactRate * 100).toFixed(1)}% · J ${alignF0.meanJaccard.toFixed(3)} → ${alignF1.meanJaccard.toFixed(3)}`;
|
||||
card.append(label, tv, reduction, stability);
|
||||
return card;
|
||||
}));
|
||||
}
|
||||
|
||||
const effectTitles: Record<string, string> = {
|
||||
system_main: "System main：平均两个 one-shot 水平后的 system 增量",
|
||||
fewshot_main: "One-shot main：平均两个 system 水平后的示例增量",
|
||||
interaction: "Interaction：两个处理是否可以简单相加",
|
||||
};
|
||||
set("[data-history-depth-title]", effectTitles[historyEffect]);
|
||||
const depth = one<HTMLElement>("[data-history-depth-map]");
|
||||
if (depth) {
|
||||
depth.replaceChildren(...historyData.domains.map((domain: string) => {
|
||||
const row = document.createElement("div");
|
||||
const label = document.createElement("b");
|
||||
label.textContent = corpusLabels[domain];
|
||||
row.append(label);
|
||||
historyData.layers.forEach((layer: any) => {
|
||||
const effect = layer.scopes[historyScope].modes[historyMode]
|
||||
.factorial[domain][historyEffect];
|
||||
const cell = document.createElement("span");
|
||||
cell.className = deltaClass(effect.ci95);
|
||||
cell.style.setProperty(
|
||||
"--strength",
|
||||
String(Math.min(1, Math.abs(effect.point) / 0.22)),
|
||||
);
|
||||
cell.textContent = `L${layer.layer} ${signed(effect.point)}`;
|
||||
cell.title = `${corpusLabels[domain]} · L${layer.layer} · ${historyEffectLabels[historyEffect]} Δ CV ${signed(effect.point)} · paired 95% ${formatSignedCi(effect.ci95)}`;
|
||||
row.append(cell);
|
||||
});
|
||||
return row;
|
||||
}));
|
||||
}
|
||||
};
|
||||
all<HTMLButtonElement>("[data-history-layer]").forEach((button) => {
|
||||
button.addEventListener("click", () => {
|
||||
historyLayerNumber = Number(button.dataset.historyLayer);
|
||||
renderHistory();
|
||||
});
|
||||
});
|
||||
all<HTMLButtonElement>("[data-history-scope]").forEach((button) => {
|
||||
button.addEventListener("click", () => {
|
||||
historyScope = button.dataset.historyScope ?? "target_content";
|
||||
renderHistory();
|
||||
});
|
||||
});
|
||||
all<HTMLButtonElement>("[data-history-mode]").forEach((button) => {
|
||||
button.addEventListener("click", () => {
|
||||
historyMode = button.dataset.historyMode ?? "prompt_balanced";
|
||||
renderHistory();
|
||||
});
|
||||
});
|
||||
all<HTMLButtonElement>("[data-history-effect]").forEach((button) => {
|
||||
button.addEventListener("click", () => {
|
||||
historyEffect = button.dataset.historyEffect ?? "interaction";
|
||||
renderHistory();
|
||||
});
|
||||
});
|
||||
renderHistory();
|
||||
});
|
||||
</script>
|
||||
|
||||
@@ -1460,7 +1872,14 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
.template-controls > div > span,
|
||||
.template-domain-grid > :global(article > span),
|
||||
.template-negative-control span,
|
||||
.template-depth span {
|
||||
.template-depth span,
|
||||
.history-ledger span,
|
||||
.history-factorial span,
|
||||
.history-controls > div > span,
|
||||
.history-domain-grid > :global(article > span),
|
||||
.history-buffer-summary span,
|
||||
.history-buffer span,
|
||||
.history-depth span {
|
||||
margin: 0;
|
||||
color: var(--blue);
|
||||
font: 700 .69rem/1.3 var(--font-mono);
|
||||
@@ -1514,7 +1933,7 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
.artifact-status b { color: var(--ink); font-size: .72rem; }
|
||||
.artifact-tabs {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(7, 1fr);
|
||||
grid-template-columns: repeat(8, 1fr);
|
||||
background: var(--ink);
|
||||
}
|
||||
.artifact-tabs button {
|
||||
@@ -2341,6 +2760,311 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
background: rgba(186,118,44,.1);
|
||||
color: var(--ink);
|
||||
}
|
||||
.history-ledger {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(6, 1fr);
|
||||
border: 1px solid rgba(32,32,39,.14);
|
||||
}
|
||||
.history-ledger article {
|
||||
padding: .85rem;
|
||||
border-right: 1px solid rgba(32,32,39,.12);
|
||||
}
|
||||
.history-ledger article:last-child { border-right: 0; }
|
||||
.history-ledger article.exact { background: rgba(57,120,110,.1); }
|
||||
.history-ledger b {
|
||||
display: block;
|
||||
margin-top: .4rem;
|
||||
font: 750 1.05rem/1.05 var(--font-display);
|
||||
}
|
||||
.history-ledger p {
|
||||
margin: .3rem 0 0;
|
||||
color: rgba(32,32,39,.56);
|
||||
font-size: .61rem;
|
||||
line-height: 1.4;
|
||||
}
|
||||
.history-factorial {
|
||||
margin-top: .8rem;
|
||||
border: 1px solid rgba(32,32,39,.15);
|
||||
overflow-x: auto;
|
||||
background: #fffdf8;
|
||||
}
|
||||
.history-axis,
|
||||
.history-row {
|
||||
display: grid;
|
||||
grid-template-columns: 7rem 1fr 1fr;
|
||||
min-width: 720px;
|
||||
}
|
||||
.history-axis {
|
||||
background: var(--ink);
|
||||
color: white;
|
||||
}
|
||||
.history-axis > * {
|
||||
padding: .65rem .8rem;
|
||||
border-right: 1px solid rgba(255,255,255,.13);
|
||||
}
|
||||
.history-axis > b {
|
||||
font: 650 .64rem/1.3 var(--font-mono);
|
||||
}
|
||||
.history-row > strong {
|
||||
display: grid;
|
||||
align-content: center;
|
||||
padding: .8rem;
|
||||
border-right: 1px solid rgba(32,32,39,.12);
|
||||
border-bottom: 1px solid rgba(32,32,39,.12);
|
||||
background: #e5ded2;
|
||||
color: var(--blue);
|
||||
font: 750 .8rem/1.4 var(--font-mono);
|
||||
}
|
||||
.history-row > strong small {
|
||||
color: rgba(32,32,39,.55);
|
||||
font: .56rem/1.35 var(--font-mono);
|
||||
}
|
||||
.history-row article {
|
||||
padding: .8rem;
|
||||
border-right: 1px solid rgba(32,32,39,.12);
|
||||
border-bottom: 1px solid rgba(32,32,39,.12);
|
||||
}
|
||||
.history-row article > div {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: .2rem;
|
||||
margin-top: .55rem;
|
||||
}
|
||||
.history-row article > div > * {
|
||||
padding: .34rem .4rem;
|
||||
border: 1px solid rgba(32,32,39,.13);
|
||||
font: 650 .56rem/1 var(--font-mono);
|
||||
font-style: normal;
|
||||
text-decoration: none;
|
||||
}
|
||||
.history-row article u { background: rgba(161,77,77,.1); color: var(--red); }
|
||||
.history-row article b { background: rgba(186,118,44,.1); color: var(--amber); }
|
||||
.history-row article i { background: rgba(57,120,110,.11); color: var(--teal); }
|
||||
.history-row article em { background: rgba(98,105,155,.13); color: var(--blue); }
|
||||
.history-row article p {
|
||||
margin: .5rem 0 0;
|
||||
color: rgba(32,32,39,.56);
|
||||
font-size: .61rem;
|
||||
line-height: 1.4;
|
||||
}
|
||||
.history-controls {
|
||||
display: grid;
|
||||
grid-template-columns: auto .9fr .9fr 1.35fr;
|
||||
gap: .8rem;
|
||||
align-items: end;
|
||||
margin-top: .8rem;
|
||||
padding: .85rem;
|
||||
border: 1px solid rgba(32,32,39,.14);
|
||||
background: #e8e2d7;
|
||||
}
|
||||
.history-controls > div { display: grid; gap: .45rem; }
|
||||
.history-controls .layer-switch { margin: 0; }
|
||||
.history-scope-switch,
|
||||
.history-mode-switch,
|
||||
.history-effect-switch { display: flex; }
|
||||
.history-scope-switch button,
|
||||
.history-mode-switch button,
|
||||
.history-effect-switch button {
|
||||
padding: .58rem .66rem;
|
||||
border: 1px solid rgba(32,32,39,.22);
|
||||
background: #fffdf8;
|
||||
color: var(--ink);
|
||||
font: 650 .61rem/1 var(--font-mono);
|
||||
cursor: pointer;
|
||||
}
|
||||
.history-scope-switch button + button,
|
||||
.history-mode-switch button + button,
|
||||
.history-effect-switch button + button { border-left: 0; }
|
||||
.history-scope-switch button[aria-pressed="true"],
|
||||
.history-mode-switch button[aria-pressed="true"],
|
||||
.history-effect-switch button[aria-pressed="true"] {
|
||||
border-color: var(--blue);
|
||||
background: var(--blue);
|
||||
color: white;
|
||||
}
|
||||
.history-controls > p {
|
||||
grid-column: 1 / -1;
|
||||
margin: 0;
|
||||
padding-top: .75rem;
|
||||
border-top: 1px solid rgba(32,32,39,.12);
|
||||
color: rgba(32,32,39,.62);
|
||||
font-size: .69rem;
|
||||
line-height: 1.5;
|
||||
}
|
||||
.history-domain-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(4, 1fr);
|
||||
margin-top: .8rem;
|
||||
border: 1px solid rgba(32,32,39,.14);
|
||||
background: #fffdf8;
|
||||
}
|
||||
.history-domain-grid > :global(article) {
|
||||
padding: .85rem;
|
||||
border-right: 1px solid rgba(32,32,39,.12);
|
||||
}
|
||||
.history-domain-grid > :global(article:last-child) { border-right: 0; }
|
||||
.history-domain-grid :global(.history-cell-values) {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: .25rem;
|
||||
margin-top: .55rem;
|
||||
}
|
||||
.history-domain-grid :global(.history-cell-values > i) {
|
||||
display: grid;
|
||||
gap: .18rem;
|
||||
padding: .4rem;
|
||||
background: #e8e2d7;
|
||||
font-style: normal;
|
||||
}
|
||||
.history-domain-grid :global(.history-cell-values small) {
|
||||
color: rgba(32,32,39,.5);
|
||||
font: 650 .53rem/1 var(--font-mono);
|
||||
}
|
||||
.history-domain-grid :global(.history-cell-values b) {
|
||||
font: 720 .7rem/1 var(--font-mono);
|
||||
}
|
||||
.history-domain-grid > :global(article > strong) {
|
||||
display: inline-block;
|
||||
margin-top: .48rem;
|
||||
padding: .26rem .38rem;
|
||||
font: 750 .65rem/1 var(--font-mono);
|
||||
}
|
||||
.history-domain-grid > :global(article > strong.down),
|
||||
.history-depth :global(span.down) {
|
||||
background: rgba(57,120,110,.13);
|
||||
color: var(--teal);
|
||||
}
|
||||
.history-domain-grid > :global(article > strong.up),
|
||||
.history-depth :global(span.up) {
|
||||
background: rgba(161,77,77,.12);
|
||||
color: var(--red);
|
||||
}
|
||||
.history-domain-grid > :global(article > strong.neutral),
|
||||
.history-depth :global(span.neutral) {
|
||||
background: rgba(186,118,44,.12);
|
||||
color: var(--amber);
|
||||
}
|
||||
.history-domain-grid > :global(article > p) {
|
||||
margin: .4rem 0 0;
|
||||
color: rgba(32,32,39,.56);
|
||||
font: .58rem/1.4 var(--font-mono);
|
||||
}
|
||||
.history-domain-grid > :global(article > small),
|
||||
.history-domain-grid > :global(article > em) {
|
||||
display: block;
|
||||
margin-top: .35rem;
|
||||
color: rgba(32,32,39,.57);
|
||||
font: .57rem/1.4 var(--font-mono);
|
||||
font-style: normal;
|
||||
}
|
||||
.history-domain-grid > :global(article > em) {
|
||||
padding-top: .35rem;
|
||||
border-top: 1px solid rgba(32,32,39,.1);
|
||||
}
|
||||
.history-buffer-summary {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(4, 1fr);
|
||||
margin-top: .8rem;
|
||||
border: 1px solid rgba(32,32,39,.14);
|
||||
background:
|
||||
linear-gradient(115deg, rgba(57,120,110,.11), transparent 48%),
|
||||
#e8e2d7;
|
||||
}
|
||||
.history-buffer-summary article {
|
||||
padding: .9rem;
|
||||
border-right: 1px solid rgba(32,32,39,.12);
|
||||
}
|
||||
.history-buffer-summary article:last-child { border-right: 0; }
|
||||
.history-buffer-summary b {
|
||||
display: block;
|
||||
margin-top: .4rem;
|
||||
font: 750 .94rem/1.05 var(--font-display);
|
||||
}
|
||||
.history-buffer-summary p {
|
||||
margin: .4rem 0 0;
|
||||
color: rgba(32,32,39,.58);
|
||||
font-size: .62rem;
|
||||
line-height: 1.45;
|
||||
}
|
||||
.history-buffer,
|
||||
.history-depth {
|
||||
display: grid;
|
||||
grid-template-columns: .52fr 1.48fr;
|
||||
gap: 1rem;
|
||||
margin-top: .8rem;
|
||||
padding: 1rem;
|
||||
border: 1px solid rgba(32,32,39,.14);
|
||||
}
|
||||
.history-buffer h5,
|
||||
.history-depth h5 {
|
||||
margin: .4rem 0;
|
||||
font: 720 1rem/1.15 var(--font-display);
|
||||
}
|
||||
.history-buffer p,
|
||||
.history-depth p {
|
||||
margin: 0;
|
||||
color: rgba(32,32,39,.58);
|
||||
font-size: .66rem;
|
||||
line-height: 1.5;
|
||||
}
|
||||
.history-buffer > :global([data-history-buffer-grid]) {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: .35rem;
|
||||
}
|
||||
.history-buffer :global([data-history-buffer-grid] article) {
|
||||
padding: .65rem;
|
||||
background: #fffdf8;
|
||||
border: 1px solid rgba(32,32,39,.12);
|
||||
}
|
||||
.history-buffer :global([data-history-buffer-grid] article > *) {
|
||||
display: block;
|
||||
}
|
||||
.history-buffer :global([data-history-buffer-grid] b) {
|
||||
margin-top: .35rem;
|
||||
font: 720 .74rem/1.2 var(--font-mono);
|
||||
}
|
||||
.history-buffer :global([data-history-buffer-grid] strong) {
|
||||
margin-top: .3rem;
|
||||
color: var(--teal);
|
||||
font: 750 .68rem/1 var(--font-mono);
|
||||
}
|
||||
.history-buffer :global([data-history-buffer-grid] p) {
|
||||
margin-top: .35rem;
|
||||
font: .55rem/1.4 var(--font-mono);
|
||||
}
|
||||
.history-depth > :global([data-history-depth-map]) {
|
||||
display: grid;
|
||||
gap: .35rem;
|
||||
}
|
||||
.history-depth :global([data-history-depth-map] > div) {
|
||||
display: grid;
|
||||
grid-template-columns: 5.5rem repeat(6, 1fr);
|
||||
gap: .25rem;
|
||||
}
|
||||
.history-depth :global([data-history-depth-map] > div > b),
|
||||
.history-depth :global([data-history-depth-map] > div > span) {
|
||||
display: grid;
|
||||
align-items: center;
|
||||
min-height: 2.2rem;
|
||||
padding: .35rem;
|
||||
font: 650 .57rem/1.2 var(--font-mono);
|
||||
}
|
||||
.history-depth :global([data-history-depth-map] > div > b) {
|
||||
color: var(--blue);
|
||||
}
|
||||
.history-depth :global([data-history-depth-map] > div > span.down) {
|
||||
background: color-mix(in srgb, var(--teal) calc(var(--strength) * 55%), #eef0e9);
|
||||
color: var(--ink);
|
||||
}
|
||||
.history-depth :global([data-history-depth-map] > div > span.up) {
|
||||
background: color-mix(in srgb, var(--red) calc(var(--strength) * 48%), #f3ebe6);
|
||||
color: var(--ink);
|
||||
}
|
||||
.history-depth :global([data-history-depth-map] > div > span.neutral) {
|
||||
background: rgba(186,118,44,.1);
|
||||
color: var(--ink);
|
||||
}
|
||||
.observed-cache {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr auto 1.25fr;
|
||||
@@ -2579,7 +3303,10 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
.length-sensitivity-head,
|
||||
.template-protocol,
|
||||
.template-controls,
|
||||
.template-depth { grid-template-columns: 1fr; }
|
||||
.template-depth,
|
||||
.history-controls,
|
||||
.history-buffer,
|
||||
.history-depth { grid-template-columns: 1fr; }
|
||||
.artifact-status { grid-template-columns: 1fr 1fr; }
|
||||
.artifact-tabs { grid-template-columns: 1fr 1fr; }
|
||||
.route-controls { grid-template-columns: 1fr 1fr; }
|
||||
@@ -2593,6 +3320,9 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
.corpus-ledger { grid-template-columns: repeat(3, 1fr); }
|
||||
.template-ledger { grid-template-columns: repeat(3, 1fr); }
|
||||
.template-domain-grid { grid-template-columns: 1fr 1fr; }
|
||||
.history-ledger { grid-template-columns: repeat(3, 1fr); }
|
||||
.history-domain-grid,
|
||||
.history-buffer-summary { grid-template-columns: 1fr 1fr; }
|
||||
.template-protocol > i { transform: rotate(90deg); justify-self: center; }
|
||||
.length-delta-grid { grid-template-columns: 1fr 1fr; }
|
||||
.corpus-heat-head p { text-align: left; }
|
||||
@@ -2628,7 +3358,10 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
.length-pair-summary,
|
||||
.template-ledger,
|
||||
.template-domain-grid,
|
||||
.template-negative-control { grid-template-columns: 1fr; }
|
||||
.template-negative-control,
|
||||
.history-ledger,
|
||||
.history-domain-grid,
|
||||
.history-buffer-summary { grid-template-columns: 1fr; }
|
||||
.route-metrics article,
|
||||
.cache-ratio article,
|
||||
.load-lessons article,
|
||||
@@ -2639,15 +3372,24 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
.corpus-findings article,
|
||||
.template-ledger article,
|
||||
.template-domain-grid > :global(article),
|
||||
.template-negative-control article { border-right: 0; border-bottom: 1px solid rgba(32,32,39,.12); }
|
||||
.template-negative-control article,
|
||||
.history-ledger article,
|
||||
.history-domain-grid > :global(article),
|
||||
.history-buffer-summary article { border-right: 0; border-bottom: 1px solid rgba(32,32,39,.12); }
|
||||
.corpus-mode-switch,
|
||||
.corpus-cohort-switch,
|
||||
.template-scope-switch,
|
||||
.template-mode-switch { display: grid; grid-template-columns: 1fr; }
|
||||
.template-mode-switch,
|
||||
.history-scope-switch,
|
||||
.history-mode-switch,
|
||||
.history-effect-switch { display: grid; grid-template-columns: 1fr; }
|
||||
.corpus-mode-switch button + button,
|
||||
.corpus-cohort-switch button + button,
|
||||
.template-scope-switch button + button,
|
||||
.template-mode-switch button + button { border-left: 1px solid rgba(32,32,39,.22); border-top: 0; }
|
||||
.template-mode-switch button + button,
|
||||
.history-scope-switch button + button,
|
||||
.history-mode-switch button + button,
|
||||
.history-effect-switch button + button { border-left: 1px solid rgba(32,32,39,.22); border-top: 0; }
|
||||
.length-delta-grid > :global(article),
|
||||
.length-pair-summary article { border-right: 0; border-bottom: 1px solid rgba(32,32,39,.11); }
|
||||
.artifact-boundary { grid-template-columns: 1fr; }
|
||||
@@ -2664,6 +3406,9 @@ const shardFraction = trace.provenance.shard_1_bytes / trace.provenance.checkpoi
|
||||
.precision-lens b { text-align: left; }
|
||||
.template-depth { overflow-x: auto; }
|
||||
.template-depth > :global([data-template-depth-map]) { min-width: 620px; }
|
||||
.history-buffer > :global([data-history-buffer-grid]) { grid-template-columns: 1fr; }
|
||||
.history-depth { overflow-x: auto; }
|
||||
.history-depth > :global([data-history-depth-map]) { min-width: 620px; }
|
||||
.layer-evidence { grid-template-columns: repeat(7, 1fr); }
|
||||
.repro-gate { grid-template-columns: 1fr; }
|
||||
.repro-gate > p { grid-column: auto; }
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -37,7 +37,7 @@ const toc = [
|
||||
|
||||
<BaseLayout
|
||||
title="DeepSeek 技术谱系与真实权重深读：从 Dense、MoE、MLA 到 R1 与 V4"
|
||||
description="用二十四张问题账、十次技术转向、十一个交互实验、真实 V2-Lite 权重、公开语料路由区间、官方模板扰动与吸收式缓存 trace 和六十个一手节点，完整理解 DeepSeek 的 MoE、MLA、FP8、DualPipe、GRPO、R1、V3.2 与 V4。"
|
||||
description="用二十四张问题账、十次技术转向、十二个交互实验、真实 V2-Lite 权重、公开语料路由区间、官方模板与消息历史因子实验、吸收式缓存 trace 和六十个一手节点，完整理解 DeepSeek 的 MoE、MLA、FP8、DualPipe、GRPO、R1、V3.2 与 V4。"
|
||||
section="deepseek"
|
||||
>
|
||||
<header class="page-hero deepseek-hero">
|
||||
@@ -55,7 +55,7 @@ const toc = [
|
||||
<div><dt>SPAN</dt><dd>2024.01 → 2026.06</dd></div>
|
||||
<div><dt>LEDGERS</dt><dd>24 张问题账</dd></div>
|
||||
<div><dt>LINEAGE</dt><dd>10 次技术转向</dd></div>
|
||||
<div><dt>LABS</dt><dd>11 个可操作实验</dd></div>
|
||||
<div><dt>LABS</dt><dd>12 个可操作实验</dd></div>
|
||||
<div><dt>EVIDENCE</dt><dd>60 个一手 / 官方节点</dd></div>
|
||||
<div><dt>STATUS</dt><dd>三轮 · 真实权重执行</dd></div>
|
||||
</dl>
|
||||
@@ -768,15 +768,15 @@ const toc = [
|
||||
<p class="eyebrow"><span>22</span> OFFICIAL WEIGHTS / EXECUTED</p>
|
||||
<h2>从“MLA 与 MoE 的概念”再往前一步：让官方 V2-Lite 权重真的跑起来</h2>
|
||||
<p class="lede">
|
||||
前面的四联实验负责建立公式与角色合同；下面的七联工件实验固定官方 revision、tokenizer、
|
||||
前面的四联实验负责建立公式与角色合同；下面的八联工件实验固定官方 revision、tokenizer、
|
||||
模型代码和 checkpoint 第一分片，在 RTX 5090 上连续执行 layer 0–6。它把真实观测、shape 推导、
|
||||
吸收式 latent cache、长度对照、官方 chat-template 扰动、实现差距和未覆盖范围放在同一张证据图里。
|
||||
</p>
|
||||
<div class="artifact-callout">
|
||||
<article><span>X / FORWARD</span><b>7 / 27 layers</b><p>1 个 dense 层 + 6 个 MoE 层；layer 7 因跨分片停止。</p></article>
|
||||
<article><span>X / ROUTES</span><b>870,912</b><p>三档长度 cohort 加 raw / user / generation 模板探针的真实 top-6 选择。</p></article>
|
||||
<article><span>X / ROUTES</span><b>1,736,352</b><p>三档长度、raw/user/generation 与 system × one-shot 四格的真实 top-6 选择。</p></article>
|
||||
<article><span>X / ABSORB CACHE</span><b>266,240 → 29,952 B</b><p>同一真实 layer-1 权重的 naive / absorb active buffers。</p></article>
|
||||
<article><span>X / RERUN</span><b>4 / 4 EXACT</b><p>三档长度 trace 与官方模板探针均 byte-exact；比较使用 paired prompt bootstrap。</p></article>
|
||||
<article><span>X / RERUN</span><b>5 / 5 EXACT</b><p>三档长度、官方模板与消息历史因子 trace 均 byte-exact；比较使用 paired prompt bootstrap。</p></article>
|
||||
</div>
|
||||
<DeepSeekArtifactLab />
|
||||
</section>
|
||||
|
||||
@@ -15,7 +15,7 @@ const workstreams = [
|
||||
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
|
||||
{ label: "Scaling Laws", value: 74, next: "加入真实拟合复现、置信区间与更多模型族对照" },
|
||||
{ label: "数据工程与预训练配方", value: 73, next: "逐图精读 FineWeb / DCLM，加入真实去重与 mixture traces" },
|
||||
{ label: "DeepSeek 专题", value: 93, next: "SM90 FlashMLA kernel、完整 27 层、词元边界 / system / few-shot 正交扰动、FP8/pipeline 与 R1-like RL 复现" },
|
||||
{ label: "DeepSeek 专题", value: 94, next: "SM90 FlashMLA kernel、完整 27 层、词元边界 / 距离 / EOS / 角色正交控制、FP8/pipeline 与 R1-like RL 复现" },
|
||||
{ label: "指令微调与人类偏好", value: 75, next: "加入真实偏好分歧样本、RM 长度偏置与 PPO/DPO 小模型复现" },
|
||||
{ label: "推理与测试时扩展", value: 76, next: "真实模型采样曲线、PRM 案例与逐篇图表精读" },
|
||||
{ label: "工具使用与长程 Agent", value: 74, next: "补真实环境 traces、cross-harness 对照、Agent RL 训练曲线与安全案例" },
|
||||
@@ -50,7 +50,7 @@ const workstreams = [
|
||||
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
|
||||
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
|
||||
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
|
||||
<div><dt>UPDATED</dt><dd>2026-07-29 17:17 CST</dd></div>
|
||||
<div><dt>UPDATED</dt><dd>2026-07-29 18:02 CST</dd></div>
|
||||
<div><dt>MODE</dt><dd>持续迭代，不锁死版本</dd></div>
|
||||
</dl>
|
||||
</div>
|
||||
@@ -97,12 +97,12 @@ const workstreams = [
|
||||
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
|
||||
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全，包含先修依赖和三条贯穿案例。</p></article>
|
||||
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
|
||||
<article><span>✓</span><h3>七十八个原创交互视图</h3><p>K3 三轴图、八联报告实验与四联开放工件实验，DeepSeek 四联公式实验与七联真实权重实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
|
||||
<article><span>✓</span><h3>七十九个原创交互视图</h3><p>K3 三轴图、八联报告实验与四联开放工件实验，DeepSeek 四联公式实验与八联真实权重实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
|
||||
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
|
||||
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
|
||||
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
|
||||
<article><span>✓</span><h3>表示、位置与残差高速公路深度专题</h3><p>二十张问题账、66 个一手节点、DeepSeek/Kimi 双谱系，以及 Token—位置—Norm—Residual/FFN 四联实验。</p></article>
|
||||
<article><span>✓</span><h3>DeepSeek 三轮真实权重里程碑</h3><p>在二十四张问题账、十次转向与四联公式实验上，新增 V2-Lite 7/27 层连续 forward、官方 V3 absorb 的 576 元素真实缓存、自然长度 / 同源长度对照与官方模板三条件实验；累计 870,912 次真实路由，模板共享前缀 21,852 / 21,852 ordered top-6 exact，四份运行结果均 byte-exact 独立复跑。</p></article>
|
||||
<article><span>✓</span><h3>DeepSeek 三轮真实权重里程碑</h3><p>在二十四张问题账、十次转向与四联公式实验上，新增 V2-Lite 7/27 层连续 forward、官方 V3 absorb、长度/模板对照与 system × one-shot 2×2；累计 1,736,352 次真实路由，模板共享前缀 21,852 / 21,852 exact，历史缓冲 TV 在 24 / 24 格下降，五份运行结果均 byte-exact 独立复跑。</p></article>
|
||||
<article><span>✓</span><h3>Kimi K3 技术报告二轮深读</h3><p>三十二张问题账、Figure 1–16 / Table 1–5 审计、100 节点阅读链，以及 Delta—Decay—AttnRes—LatentMoE—SiTU—QB—MOPD—Cache 八联实验。</p></article>
|
||||
<article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions，审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes；四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
|
||||
<article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel；6/6 官方参考逐元素相等，并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
|
||||
@@ -134,7 +134,7 @@ const workstreams = [
|
||||
<div class="queue-table">
|
||||
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
|
||||
<div><span>P0</span><strong>K3 三轮</strong><p>开放权重 traces → FlashKDA / AttnRes / MoE 真实行为 → Figure 1–16 数值重绘与独立复现</p><em>运行证据 + 逐图复现</em></div>
|
||||
<div><span>P0</span><strong>DeepSeek 三轮</strong><p>SM90 FlashMLA kernel / 完整 27 层 / 词元边界与 system / few-shot 正交扰动 → FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
|
||||
<div><span>P0</span><strong>DeepSeek 三轮</strong><p>SM90 FlashMLA kernel / 完整 27 层 / 词元边界、距离、EOS、角色与示例内容正交控制 → FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
|
||||
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
|
||||
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
|
||||
<div><span>P0</span><strong>语言模型前史二轮</strong><p>Kneser–Ney / LSTM / Bahdanau 逐图 → 真实小语料复现 → tokenizer 公平性</p><em>可复现实验 + 逐图笔记</em></div>
|
||||
|
||||
Reference in New Issue
Block a user