feat: deepen Kimi K3 report guide
This commit is contained in:
wuyang
+13
-2
@@ -8,7 +8,7 @@
|
||||
|---|---:|---:|---|
|
||||
| 研究框架与规范 | 进行中 | 83% | Scaling Laws 二轮拟合复现与逐图精读 |
|
||||
| 网站设计系统 | 进行中 | 89% | 打印样式与更多通用可视化组件 |
|
||||
| Kimi K3 深读 | 进行中 | 66% | 扩写 pre-training / infra 逐图笔记 |
|
||||
| Kimi K3 深读 | 完成二轮 | 88% | 第三轮加入官方权重 traces、独立复现与逐图数值重绘 |
|
||||
| 语言模型前史 | 完成首版 | 78% | Kneser–Ney、LSTM、Bahdanau 逐图精读与真实小语料复现 |
|
||||
| Transformer 基础 | 完成首版 | 79% | 多头电路、归一化 traces 与真实 kernel / KV 配置 |
|
||||
| 表示、位置与残差高速公路 | 完成首版 | 81% | 真实 hidden-state / norm traces、长上下文位置外推与深层稳定性消融 |
|
||||
@@ -41,7 +41,7 @@
|
||||
- [x] 完成 486 篇关键论文索引，覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。
|
||||
- [x] 完成可检索、可按专题筛选的论文库页面。
|
||||
- [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。
|
||||
- [x] 完成 K3 三轴架构、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 四联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等五十九个原创交互视图。
|
||||
- [x] 完成 K3 三轴架构与八联报告实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 四联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等六十七个原创交互视图。
|
||||
- [x] 完成长上下文首版：五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
|
||||
- [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文，并建立长上下文研究账本。
|
||||
- [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文，并建立 MoE 研究账本。
|
||||
@@ -149,9 +149,16 @@
|
||||
- [x] DeepSeek 专属 Chrome 断言通过：24 张账、10 次转向、60 节点、MLA 576 元素、FP8 角色、DualPipe、MTP、GRPO/DAPO/Dr.GRPO、R1 身份、键盘 tabs 与 390px 移动端均正确响应。
|
||||
- [x] Astro 类型检查、生产构建、21 个页面、1145 个站内引用和 14 个跨页锚点通过；DeepSeek 与既有十四专题共十五套本地真实 Chrome 回归全部通过。
|
||||
- [x] DeepSeek 二轮以源提交 `cd96dab`、不可变镜像 `20260729T031901Z-cd96dab` 发布；NAS、VPS/Tailscale、NPM、DNS、HTTPS、证书、门户、公开 Forgejo 与十五套生产 Chrome 回归全链路通过。
|
||||
- [x] 启动 K3 二轮深读：Grok Headless 只负责逐节查漏，候选问题、实验和红线保存在 `K3_GROK_LEADS.md`，正式结论逐项回到 47 页官方报告。
|
||||
- [x] 建立 K3 正式研究账本：32 张问题账、Figure 1–16 / Table 1–5 视觉契约、8 个实验合同、完整数值红线与 100 节点阅读链。
|
||||
- [x] 纠正 K3 原生多模态训练的旧表述：MoonViT-V2 从头训练，视觉与文本从训练开始在同一个 NTP objective 中联合优化，不采用“冻结语言模型再解冻”的 post-hoc 路线。
|
||||
- [x] 完成 K3 二轮正文：30 个编号章节逐节覆盖架构、预训练、后训练、环境、系统、评测、案例与 XTML 附录，并与 DeepSeek / 17 个课程专题交叉链接。
|
||||
- [x] 完成 K3 八联交互实验：Delta Rule、bounded decay、Block AttnRes、LatentMoE payload、SiTU-GLU、Quantile Balancing、MOPD/partial rollout、hybrid prefix cache。
|
||||
- [x] K3 专属 Chrome 断言通过：32/21/100 内容计数、八个实验计算、键盘 tabs、事实纠错、桌面与 390px 移动端均无异常。
|
||||
|
||||
## 正在进行
|
||||
|
||||
- [ ] K3 三轮：使用开放权重与官方实现加入 KDA/AttnRes/MoE 真实 traces、FlashKDA kernel 对照、逐图数值重绘与独立复现。
|
||||
- [ ] DeepSeek 三轮：真实专家负载、MLA kernel、FP8 / pipeline 与 R1-like RL traces，外加独立小模型复现。
|
||||
- [ ] 表示、位置与残差二轮：真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。
|
||||
- [ ] 评测安全二轮：真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。
|
||||
@@ -265,6 +272,10 @@
|
||||
| 2026-07-29 | 论文库扩充到 486 篇 | 新增 DeepSeek-Coder/Coder-V2、ESFT、Prover-V1.5/V2 与 Engram 6 个旁支节点 |
|
||||
| 2026-07-29 | DeepSeek 二轮用四个独立实验闭环 | 稀疏容量、MLA 缓存、V3 协同与 RL 偏差分别操作；精确公式、作者报告和 teaching model 永久分级 |
|
||||
| 2026-07-29 | DeepSeek 二轮用不可变镜像 `20260729T031901Z-cd96dab` 发布 | OCI digest `sha256:4bfb3faf…fa3968`；复用 `12010→8080`、NPM host 31 / cert 41、门户 order 180 与公开 Forgejo；保留 `20260729T023329Z-a2c9298` 回滚 |
|
||||
| 2026-07-29 | K3 二轮按 32 张对象账与完整报告顺序重建 | total/active、2.5×、KDA state、深度来源、专家路由、视觉目标、轨迹、缓存与评测协议不再压成一页组件摘要 |
|
||||
| 2026-07-29 | K3 原生视觉事实回到 §2.4 / §3.3 核验 | 删除“先冻结语言模型再解冻”旧表述；明确 MoonViT-V2 从头训练，视觉/文本从开始共同 NTP |
|
||||
| 2026-07-29 | K3 Figure 1–16 / Table 1–5 全部建立课程视觉契约 | 每张图同时写支持范围与不可外推项；作者报告、论文、推导与 toy model 使用 R/P/D/T 标签 |
|
||||
| 2026-07-29 | K3 二轮用八个独立实验闭环 | Delta memory、BF16 decay、AttnRes、LatentMoE、SiTU、QB、MOPD/RL 与 prefix cache 分开操作，不合成伪“架构总分” |
|
||||
|
||||
## 未决问题
|
||||
|
||||
|
||||
@@ -19,7 +19,9 @@
|
||||
|
||||
当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读，
|
||||
语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化，以及评测与安全深度专题，
|
||||
以及 59 个覆盖核心机制的原创交互视图。DeepSeek 二轮专题以 24 张问题账、10 次技术转向、
|
||||
以及 67 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
|
||||
8 个交互实验和 100 个一手/官方节点，完整覆盖架构、预训练、后训练、系统、评测、案例与附录。
|
||||
DeepSeek 二轮专题以 24 张问题账、10 次技术转向、
|
||||
4 个交互实验和 60 个一手/官方节点，串起 Dense、MoE、MLA、V3 协同、R1 与 V4。
|
||||
其余专题按进度账本持续扩建。
|
||||
|
||||
|
||||
+2
-1
@@ -153,7 +153,8 @@ pass^k、校准、动态基准、代码 Verifier、LLM Judge、Arena、Agent 最
|
||||
|
||||
## 三条贯穿式案例
|
||||
|
||||
1. **Kimi K3 解剖**：把上述全部专题重新汇总到一张架构与训练系统图。
|
||||
1. **Kimi K3 解剖**：二轮已完成 32 张问题账、Figure 1–16 / Table 1–5 审计、
|
||||
8 个交互实验与 100 节点阅读链，把全部专题重新汇入架构—预训练—后训练—系统—评测因果链。
|
||||
2. **DeepSeek 技术谱系**：DeepSeek LLM → DeepSeekMoE → V2/MLA → V3/FP8/MTP/DualPipe → Math/GRPO → R1 → V3.2/DSA → V4 长上下文。
|
||||
3. **“一个 Token 的旅行”**：从文本分词，经注意力、MoE、GPU 集群、后训练，再到线上推理与工具调用。
|
||||
|
||||
|
||||
+2
-1
@@ -24,7 +24,8 @@
|
||||
"check:inference-serving-browser": "node scripts/check-inference-serving-browser.mjs",
|
||||
"check:evaluation-browser": "node scripts/check-evaluation-browser.mjs",
|
||||
"check:representation-browser": "node scripts/check-representation-browser.mjs",
|
||||
"check:deepseek-browser": "node scripts/check-deepseek-browser.mjs"
|
||||
"check:deepseek-browser": "node scripts/check-deepseek-browser.mjs",
|
||||
"check:k3-browser": "node scripts/check-k3-browser.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@astrojs/sitemap": "3.7.3",
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
# Kimi K3 Grok 候选线索台账
|
||||
|
||||
> 状态：**UNVERIFIED / 仅用于发现问题，不可作为课程证据引用。**
|
||||
>
|
||||
> 生成方式：2026-07-29 使用本机 Grok CLI Headless，限定只读取
|
||||
> `research/sources/kimi-k3/k3_tech_report.txt` 与当时的
|
||||
> `src/pages/k3/index.astro`，要求逐节对照并提出遗漏、疑点、实验和阅读节点。
|
||||
> 所有候选结论必须回到 K3 官方报告、论文原文或官方代码核验后，才能进入
|
||||
> `K3_RESEARCH.md` 与正式页面。
|
||||
|
||||
## 1. 候选审计结论
|
||||
|
||||
现有 K3 页面有一条可用的“序列—深度—宽度—系统”骨架，但把 47 页报告压缩得过度：
|
||||
|
||||
- 只覆盖 13 个正文入口、9 个一手来源和一个通用架构浏览器；
|
||||
- Figure 1–16、Table 1–5 大多没有逐图解释；
|
||||
- §3 预训练、§4 后训练、§5 系统、§6 评测、§7 案例与附录只保留了摘要；
|
||||
- 关键数字、机制的适用边界、报告未披露事项尚未形成统一红线；
|
||||
- 视觉训练段落疑似沿用了“后接视觉塔”的常见范式，需优先核验。
|
||||
|
||||
## 2. 候选问题清单
|
||||
|
||||
以下 32 个问题是第二轮深读的索引，不代表其答案已经核验：
|
||||
|
||||
1. 2.78T total 与 104.2B active 分别在计算和容量上意味着什么？
|
||||
2. 报告所说约 2.5× scaling efficiency 的对照对象和口径是什么？
|
||||
3. 3 KDA + 1 MLA 的比例在 93 层里怎样落地？
|
||||
4. KDA 的状态更新与普通线性注意力有何不同？
|
||||
5. delta rule 为什么是“纠错写入”，而不是简单累加？
|
||||
6. `g_min=-5` 解决的数值与内核问题是什么？
|
||||
7. chunkwise KDA 怎样同时实现训练并行与解码递归？
|
||||
8. KDA/MLA 的全秩输入相关输出门做什么？
|
||||
9. MLA 为什么采用 NoPE；位置从哪里来？
|
||||
10. attention output 保持 FP32 在修复什么舍入问题？
|
||||
11. Full AttnRes 与 Block AttnRes 的准确关系是什么？
|
||||
12. LatentMoE 的 3584 维路由空间为何能降低通信？
|
||||
13. shared experts 与 routed experts 怎样分工？
|
||||
14. SiTU-GLU 的有界激活和具体参数是什么？
|
||||
15. Quantile Balancing 如何由 Top-(k+1) cutoff 更新 bias？
|
||||
16. 直方图近似如何让近千专家的分位统计可通信？
|
||||
17. MoonViT-V2 是否从头训练；与 SigLIP 初始化对照是什么？
|
||||
18. “原生多模态”在数据目标和共享主干上准确指什么？
|
||||
19. Per-Head Muon 为什么按 attention head 分组？
|
||||
20. 8K→64K→256K→1M 的 context curriculum 如何安排？
|
||||
21. 九个专家如何由三领域 × 三 effort 形成？
|
||||
22. MOPD 的 on-policy 稠密 token reward 如何定义？
|
||||
23. reasoning effort 的 per-problem budget 怎样退火？
|
||||
24. partial rollout 怎样处理 straggler 与 stale trajectory？
|
||||
25. Agentic GRM 如何控制冗长 reward hacking？
|
||||
26. KDA Context Parallelism 如何划分 1M 序列？
|
||||
27. MoonEP 如何把路由不均转成静态执行形状？
|
||||
28. prefix cache 为什么必须联合保存 KDA state 与 MLA KV？
|
||||
29. 512-token hash boundary 与 6144-token physical block 为什么解耦？
|
||||
30. 评测中的 effort、工具、harness、fallback 如何影响可比性？
|
||||
31. 报告自己承认的 research reasoning / cyber 等弱项是什么？
|
||||
32. 第三方评测分数、推理成本和公开权重应怎样并排展示？
|
||||
|
||||
## 3. 候选逐图任务
|
||||
|
||||
| 报告对象 | 候选解释任务 |
|
||||
|---|---|
|
||||
| Figure 1 | 把总体结果与“仍落后最强闭源模型”放在同一视图 |
|
||||
| Figure 2 | 标出 3:1 KDA–MLA、Block AttnRes、Stable LatentMoE、MoonViT |
|
||||
| Figure 3 | 交互展示无界与有界 log-decay 在 BF16 中的差异 |
|
||||
| Figure 4 | 并排比较 GLU、SwiGLU、SiTU-GLU 的大输入行为 |
|
||||
| Figure 5 | 动画解释 Top-(k+1) cutoff 与 Quantile Balancing |
|
||||
| Figure 6 | 只在报告消融范围内解释 from-scratch ViT 稳定性 |
|
||||
| Figure 7 | 区分训练 loss scaling、吞吐与下游能力 |
|
||||
| Figure 8 | 并排画 RL FLOPs、平均 steps 与分数 |
|
||||
| Figure 9 | 展示 knowledge graph→材料→任务→验证的合成链 |
|
||||
| Figure 10 | 解释 AET curriculum 与独立 verifier |
|
||||
| Figure 11 | 解释 pipeline / offload 中状态的移动 |
|
||||
| Figure 12 | 动画解释物理块、hash block、KDA checkpoint |
|
||||
| Figure 13 | 将第三方总分与 token/cost 放在同一坐标系 |
|
||||
| Figure 14 | 明确 kernel agent 的 24h 个案边界 |
|
||||
| Figure 15 | 明确 MiniTriton 的 L20 roofline 和个案边界 |
|
||||
| Figure 16 | 可视化 XTML 选项、channel 与 message boundary |
|
||||
| Table 1 | 做 K2→K3 的逐字段差异表 |
|
||||
| Tables 2–5 | 按能力轴重排，并给每个数字附评测协议脚注 |
|
||||
|
||||
## 4. 候选交互实验
|
||||
|
||||
1. **Delta Rule 工作记忆**：反复写入相同 key，对比累加与纠错写入。
|
||||
2. **有界 log-decay / BF16**：拖动下界、chunk 长度，观察累计倒数与溢出风险。
|
||||
3. **Block AttnRes**：调层数和 block size，比较深度来源、缓存与通信。
|
||||
4. **LatentMoE 通信账本**：调 full width、latent width、激活专家数，比较 token payload。
|
||||
5. **SiTU-GLU 曲线**：调参数并对比 SwiGLU 的大正输入。
|
||||
6. **Quantile Balancing**：模拟长尾 router score 和 bias 更新。
|
||||
7. **MOPD / effort**：调 domain、budget multiplier、partial rollout threshold。
|
||||
8. **BrowseComp 上下文管理**：比较大窗口直塞、压缩、prefix reuse 的代价。
|
||||
|
||||
## 5. 候选阅读节点
|
||||
|
||||
Grok 建议把报告参考文献扩成约 100 个可导航节点，按以下簇核验：
|
||||
|
||||
- Kimi 谱系：MoonViT、Kimi Linear、Kimi k1.5、K2、K2.5、K3；
|
||||
- 注意力与状态：Transformer、FlashAttention、linear attention、DeltaNet、
|
||||
Gated DeltaNet、Mamba、RWKV、RetNet、NoPE、长上下文扩展；
|
||||
- 深度与宽度：residual、Highway/DenseNet、AttnRes、MoE、Switch、
|
||||
DeepSeekMoE、MLA、LatentMoE；
|
||||
- 优化与数值：Muon、Muon scaling、weight clipping、BF16、MXFP4/8、QAT；
|
||||
- 后训练：SFT、RLHF、PPO/GRPO 类方法、partial rollout、OPD、MOPD、
|
||||
speculative decoding、EAGLE-3、LK loss；
|
||||
- 系统：FlashKDA、KCP、MoonEP、expert parallelism、KV/prefix cache、
|
||||
Firecracker、AgentENV、WarpDecode；
|
||||
- Agent 与评测：white-box harness、AET、Agentic GRM、BrowseComp、
|
||||
SWE、TerminalBench、OSWorld、HLE、视觉基准；
|
||||
- 开放实现：Kimi-K3、AgentENV、MiniTriton、nano-kpu 及报告明确列出的代码仓库。
|
||||
|
||||
这些节点只能在逐项核对标题、年份、主张与 URL 后进入正式论文链。
|
||||
|
||||
## 6. 候选红线
|
||||
|
||||
- 不把 2.78T total 写成每 token 计算量；
|
||||
- 不把约 2.5× 写成推理加速或 KDA 单项收益；
|
||||
- 不把 KDA 写成能够无损精确回忆全部历史；
|
||||
- 不把 NoPE 写成“没有任何位置信息”；
|
||||
- 不忽略 `g_min`、SiTU 参数、QB inference bias 冻结等实现条件；
|
||||
- 不把路由负载平衡与系统执行平衡混成一个机制；
|
||||
- 不把 MoonViT-V2 写成先接到已经训练好的语言模型；
|
||||
- 不把 1M window 写成 1M 内容都能可靠利用；
|
||||
- 不把 MOPD 写成九个模型在推理时投票；
|
||||
- 不把 effort 写成一个固定全局 token 上限；
|
||||
- 不把模型分数和 harness / tool 的系统分数混为一谈；
|
||||
- 不省略评测日期、effort、工具、fallback、guard；
|
||||
- 不把 kernel、MiniTriton、芯片案例推广为普遍外部结论；
|
||||
- 不补写报告没有披露的总训练 token、精确数据比例、集群和成本。
|
||||
|
||||
## 7. 处置状态
|
||||
|
||||
- [x] Grok 候选审计已保存，与正式证据隔离。
|
||||
- [x] “冻结语言模型再解冻”疑点已回到 K3 §2.4 / §3.3 核验，确认现页错误。
|
||||
- [ ] 32 个问题逐项建立正式证据台账。
|
||||
- [ ] 16 图、5 表逐项建立视觉契约。
|
||||
- [ ] 100 个候选阅读节点逐项核验。
|
||||
- [ ] 8 个实验建立交互、公式与边界测试。
|
||||
|
||||
@@ -0,0 +1,324 @@
|
||||
# Kimi K3 第二轮深读：正式研究与实现台账
|
||||
|
||||
> 锚点：Kimi Team, **Kimi K3: Open Frontier Intelligence**, Technical Report,
|
||||
> 2026-07，47 页。正式引用以仓库保存的官方 PDF
|
||||
> `research/sources/kimi-k3/k3_tech_report.pdf` 为准；文本抽取文件只用于检索。
|
||||
>
|
||||
> 证据等级：`R` = K3 官方报告直接陈述；`P` = 被报告引用的论文/官方实现；
|
||||
> `D` = 由公开公式作出的确定性推导；`T` = 本站教学模型，不是实测。
|
||||
|
||||
## 0. 阅读目标
|
||||
|
||||
K3 页面第二轮不再做“组件摘要”，而要回答四层问题：
|
||||
|
||||
1. **对象是什么**：参数、状态、激活、通信、轨迹、环境或评测协议；
|
||||
2. **为什么需要它**：先说明失败模式，再引入机制；
|
||||
3. **怎样工作的**：公式、数据流、训练流、系统状态流；
|
||||
4. **证据能走多远**：作者报告、独立论文、确定性计算与 toy model 分标签。
|
||||
|
||||
验收目标：
|
||||
|
||||
- 32 张问题账全部有答案与边界；
|
||||
- Figure 1–16、Table 1–5 全部有课程对应物；
|
||||
- 至少 8 个可操作实验，不将 toy output 冒充 checkpoint / 集群实测；
|
||||
- 100 个经过标题、年份、URL 和作用核对的阅读节点；
|
||||
- 明确纠正原生多模态、2.5×、1M、MoE、MOPD、评测等常见误读；
|
||||
- K3、DeepSeek 与全站专题互相链接，而不是形成孤岛。
|
||||
|
||||
## 1. 32 张问题账
|
||||
|
||||
| ID | 对象 | 正式答案 | 证据 | 页面验收 |
|
||||
|---|---|---|---|---|
|
||||
| Q01 | total / active | Table 1 给出 2.78T total、104.2B active。total 是总容量；active 是一条 Token 路径经过的参数规模，仍不等于端到端 FLOPs。 | R §2.6 Table 1 | 精确规格表；hero 可四舍五入 |
|
||||
| Q02 | 2.5× | 这是团队在调 batch、LR、TPP、shape、架构和 recipe 后，相对 K2 得到的**整体 scaling efficiency**，不是推理速度，也不是 KDA 单项收益。 | R Abstract, §3.2, Fig. 7 | 三种错误解释并排 |
|
||||
| Q03 | 3:1 hybrid | 93 层中 69 KDA + 24 Gated MLA，模式为 3 KDA 后 1 MLA，最后一层为 MLA；另有 1 dense layer。 | R §2.1, §2.6 Table 1 | 可展开 93 层条带 |
|
||||
| Q04 | KDA state | KDA 维护固定形状 recurrent state，用 channel-wise decay 与 delta-rule 写入；不保存所有 token pair。 | R §2.1 Eq. 1 | 状态读写动画 |
|
||||
| Q05 | delta rule | 新 value 写入前减掉 state 对当前 key 的已有预测，减少相同 key 的重复累加；不等于无损键值数据库。 | R §2.1; P Kimi Linear / Gated DeltaNet | 累加 vs 纠错实验 |
|
||||
| Q06 | bounded decay | K3 将 log-decay 下界设为 `g_min=-5`，限制 chunk 内累计衰减倒数，令对角 tile 也可用 BF16 Tensor Core 稠密矩阵乘。 | R §2.1.1, Fig. 3 | BF16 风险实验 |
|
||||
| Q07 | chunkwise KDA | chunk 之间递归传状态，chunk 内转换成并行矩阵计算；训练与 prefill 并行、decode 保持 O(1) state update。 | R §2.1.1; P FlashKDA | chunk 数据流图 |
|
||||
| Q08 | output gates | KDA 和 MLA 都使用输入相关、全秩输出门；其作用是动态控制读出通道，不是 router。 | R §2.1–2.1.2 | 与 MoE gate 对照 |
|
||||
| Q09 | NoPE | MLA 无显式位置编码；KDA 的递归门控/衰减隐式携带顺序信息。NoPE 不等于没有位置或顺序信号。 | R §2.1.2, §3.4 | 位置来源图 |
|
||||
| Q10 | FP32 attention output | 报告称 attention output 保持 FP32 以纠正 FlashAttention 有偏舍入误差，并重新安排 kernel tile buffer。 | R §2.1.2; P ref. 99 | 数值合同卡 |
|
||||
| Q11 | AttnRes | Full AttnRes 让每层以可学习 pseudo-query 聚合 embedding 与所有早层输出；Block 版先在块内累加、跨块注意，以降低保存和跨 stage 通信。 | R §2.2 Eq. 8–9 | 深度路由实验 |
|
||||
| Q12 | block size | 报告经验称约 8 个 block 保留大部分收益；K3 每 12 层一 block，加 embedding 共 9 个来源组。芯片 nano case 的 block size=2 不是主模型。 | R §2.2, §7.3 | 主模型/个案红线 |
|
||||
| Q13 | LatentMoE | shared experts 走 full width，routed experts 在 3584 latent width 中执行，之后上投影回 7168；降低多专家激活的 token payload 与权重读取。 | R §2.3, Table 1 | 通信账实验 |
|
||||
| Q14 | experts | K3 每层 896 routed、激活 16，另有 2 shared；“稀疏”不表示消费级硬件轻松运行。 | R Table 1 | 896→16→2 规格图 |
|
||||
| Q15 | SiTU-GLU | 为 routed path 的连续矩阵乘限制大正输入，使用有界 tanh 分支近似 SwiGLU 原点附近行为。页面必须展示报告参数而非随意拟合。 | R §2.3.2, Fig. 4 | 函数曲线实验 |
|
||||
| Q16 | QB | 以每个 expert 的 Top-(k+1) score cutoff 分位量更新 bias；bias 影响选择，训练结束后冻结用于 inference。 | R §2.3.3, Fig. 5 | router 分位实验 |
|
||||
| Q17 | QB 通信 | 用 score histogram 的 all-reduce 近似全局 quantile，精度受 bin width 限制；这与 MoonEP 的执行平衡是两个层级。 | R §2.3.3 | 模型侧/系统侧分层 |
|
||||
| Q18 | native vision | MoonViT-V2 从头训练，语言和视觉从训练开始就在同一 NTP objective 联合优化；不是把视觉塔后接到预训练 LLM。 | R §2.4, §3.3 | 删除冻结/解冻错误；双路线图 |
|
||||
| Q19 | vision encoder | 约 401M、27 层、patch 14、12 heads；图像/视频共享参数，空间/时间 factorized attention、temporal pooling、2×2 pixel shuffle，最高 3584² 输入。 | R §2.4, Table 1 | 视觉 token 流 |
|
||||
| Q20 | Per-Head Muon | Q/K/V 等多头投影按 head 分组应用 Muon；训练同时用 K2 weight clipping、QB、cosine、1% warmup、WD=0.1。 | R §2.5, §3.3 | 优化器角色卡 |
|
||||
| Q21 | data | 四类文本为 Web Text、Code、Mathematics、Knowledge，加视觉语料；规则、质量分类与去重后，借小模型消融调 domain sampling。未公开完整配比。 | R §3.1 | 数据管线与“不披露” |
|
||||
| Q22 | context curriculum | pretrain 8K→64K，cooldown 256K→1M；长数据清洗、上采样，并合成需要跨全局证据的多模态任务。 | R §3.3–3.4 | 四阶段长度图 |
|
||||
| Q23 | SFT | 前代 Kimi 专家合成长 agent trajectory，经多阶段验证与 HITL 标注，以 XTML 序列化；SFT 起即进入 QAT。 | R §4.1.1 | 轨迹来源与协议 |
|
||||
| Q24 | nine experts | general tasks、general agents、coding agents × low/high/max effort = 9 个专家策略；不是 inference ensemble。 | R §4.1.2 | 3×3 矩阵 |
|
||||
| Q25 | effort budget | 每题由 cold-start 估计 `b0(x)`，超过 `τ·b0(x)` 的轨迹 reward 改为 −1；agent task 统计思考与 tool-call argument 等累计输出，τ 按 domain 由人工指导退火。 | R §4.1.2 | budget 实验 |
|
||||
| Q26 | partial rollout | N×K 轨迹中 λ 比例完成即暂停生成并更新；未完轨迹下一轮优先恢复，per-token regularization 处理极端 stale/off-policy。 | R §4.1.2 | λ/straggler 实验 |
|
||||
| Q27 | MOPD | 学生 on-policy 生成前缀；相应 domain/effort teacher 给下一 token 的 clipped log-ratio dense reward。九个老师只用于训练指导。 | R §4.1.3 Eq. 15 | reward 公式与流程 |
|
||||
| Q28 | deployment-aware PT | experts 权重 MXFP4、激活 MXFP8；非 expert 高精度。SFT 与 RL 都 QAT，rollout/training 同配方。MTP 层再调为 EAGLE-3 draft，优化 LK acceptance loss。 | R §4.1.4 Eq. 16 | 精度角色和 draft 生命周期 |
|
||||
| Q29 | agent environment | harness 被拆成 tool、prompt、context、skills、memory、subagent 等模块，训练时动态组合；AET 由目标状态和 verifier 给奖励。 | R §4.2 | harness 组合器图 |
|
||||
| Q30 | system state | pretrain 管参数/optimizer/KDA state；RL 还管 KV 与 sandbox；serving 联合管 KDA state、MLA KV 与 fleet budget。 | R §5 | 三类状态寿命图 |
|
||||
| Q31 | hybrid prefix cache | MLA KV 随 token 增长，KDA state 固定但 checkpoint 大；统一页池，512-token hash boundary 可落在 6144-token physical block 内，命中必须两类 state 同时存在。 | R §5.4.1 Fig. 12 | 可交互命中边界 |
|
||||
| Q32 | evaluation | 分数必须带 effort、tool、harness、fallback/guard、date；报告结论是总体领先其比较的开放/部分闭源模型，但仍落后 Claude Fable 5 与 GPT-5.6 Sol。 | R Abstract, §6, Tables 2–5 | 协议审计器 |
|
||||
|
||||
## 2. 关键规格台账
|
||||
|
||||
K2 → K3（Table 1；小数与单位保持报告口径）：
|
||||
|
||||
| 项 | Kimi K2 | Kimi K3 |
|
||||
|---|---:|---:|
|
||||
| Transformer layers | 61 | 93 |
|
||||
| Total parameters | 1.04T | 2.78T |
|
||||
| Activated parameters | 32.6B | 104.2B |
|
||||
| Hidden dimension | 7168 | 7168 |
|
||||
| Routed latent dimension | – | 3584 |
|
||||
| MoE expert hidden | 2048 | 3072 |
|
||||
| Routed experts | 384 | 896 |
|
||||
| Activated routed experts | 8 | 16 |
|
||||
| Shared experts | 1 | 2 |
|
||||
| Attention heads | 64 | 96 |
|
||||
| Dense layers | 1 | 1 |
|
||||
| Vocabulary | 160K | 160K |
|
||||
| Context | 128K | 1M |
|
||||
| Attention | MLA | Hybrid KDA–MLA |
|
||||
| Activation | SwiGLU | SiTU-GLU |
|
||||
| Attention composition | 61 MLA | 69 KDA + 24 MLA |
|
||||
| MTP layers | 1 | 1 |
|
||||
| Vision encoder | – | 401M / 27 layers / patch 14 / 12 heads |
|
||||
|
||||
## 3. Figure 1–16 视觉契约
|
||||
|
||||
| 图 | 报告主张 | 课程视觉 | 不允许的扩张 |
|
||||
|---|---|---|---|
|
||||
| 1 | 主评测总体位置 | 四能力轴 + 协议脚注 | 不以单榜替代总体结论 |
|
||||
| 2 | K3 architecture overview | 视觉 token→hybrid stack→LatentMoE→head；深度路由单画 | 不把 93 层画成一层 |
|
||||
| 3 | bounded decay 的稳定/内核动机 | log-decay、累计倒数、BF16 风险三联图 | 不写成精度绝对保证 |
|
||||
| 4 | GLU/SwiGLU/SiTU 形状 | 同坐标曲线 + 大输入放大镜 | 不凭曲线声称所有模型更好 |
|
||||
| 5 | Quantile Balancing | score cutoff→quantile→bias→下一 step | 不与 auxiliary loss-free 等同 |
|
||||
| 6 | from-scratch MoonViT-V2 消融 | gradient norm/spike + baseline eval 边界 | 只限作者配置，不普遍否定初始化 |
|
||||
| 7 | K3 相对 K2 的 scaling | loss/compute 概念曲线 + 2.5× 口径 | 不画成 inference 2.5× |
|
||||
| 8 | RL FLOPs、steps、能力同升 | 三轴小 multiples | 相关性不自动证明通用因果 |
|
||||
| 9 | KG-guided synthesis | concept→material→task type→verification | 不假装公开完整图谱 |
|
||||
| 10 | AET curriculum | initial state→actions→verifier→harder task | reward 来自结果，不是模型自评 |
|
||||
| 11 | pretraining pipeline/offload | 参数/activation/encoder 在设备间的时间线 | 不补写未公开集群规模 |
|
||||
| 12 | hybrid prefix cache | 6144 physical / 512 hash / sparse checkpoint | 数字是报告示例，不是唯一配置 |
|
||||
| 13 | score vs output token/cost | 二维/三维比较并列 protocol | 不把价格点当架构结论 |
|
||||
| 14 | kernel agent case | 283.6→114.4ms 等逐项轨迹 | 24h/task 个案，不推广 |
|
||||
| 15 | MiniTriton L20 roofline | 实测点相对 roofline | 只限报告硬件与任务 |
|
||||
| 16 | XTML | global / one-shot / input options + channels | 不把 template 等同能力 |
|
||||
|
||||
## 4. Table 2–5 协议台账
|
||||
|
||||
页面不追求把 PDF 表格逐格搬运，而采用“结果 + 协议 + 风险”三层：
|
||||
|
||||
- Table 2：Reasoning & Knowledge / Coding；
|
||||
- Table 3：Agentic；
|
||||
- Table 4：Vision；
|
||||
- Table 5：第三方评测与 cost / output token 关系；
|
||||
- 所有分数要说明日期、reasoning effort、工具、harness、fallback 与 guard；
|
||||
- 对 BrowseComp 同时保留 300K context management 的 91.2 与完整 1M 无管理的 90.4，
|
||||
用于说明“窗口容量 ≠ 上下文策略”；
|
||||
- 报告对 research reasoning、cyber 等弱项的陈述必须与强项同页。
|
||||
|
||||
## 5. 八个实验合同
|
||||
|
||||
### Lab A — Delta Rule 工作记忆
|
||||
|
||||
- 输入：key 重复率、写入强度 β、旧状态预测、目标 value；
|
||||
- 输出：additive update 与 delta update 的 state/value；
|
||||
- 精确部分：给定 toy 向量后的矩阵运算；
|
||||
- 边界：不代表真实 head 维度或模型回忆率。
|
||||
|
||||
### Lab B — Bounded decay / BF16
|
||||
|
||||
- 输入：log-decay 下界、chunk length、累计 decay；
|
||||
- 输出：倒数幅度与风险等级；
|
||||
- 精确部分：数学值；
|
||||
- 边界：风险标签不替代 FlashKDA kernel 实测。
|
||||
|
||||
### Lab C — Block AttnRes
|
||||
|
||||
- 输入：层数、block size；
|
||||
- 输出：来源组数、Full/Block 缓存单位、深度路径；
|
||||
- 精确部分：计数；
|
||||
- 边界：吞吐仅方向展示。
|
||||
|
||||
### Lab D — LatentMoE 通信
|
||||
|
||||
- 输入：full width、latent width、top-k、token 数、dtype；
|
||||
- 输出：routed payload 的确定性元素/字节比例；
|
||||
- 边界：不包含 all-to-all 拓扑、权重流与 kernel utilization。
|
||||
|
||||
### Lab E — SiTU-GLU
|
||||
|
||||
- 输入：x 与报告参数；
|
||||
- 输出：GLU/SwiGLU/SiTU 曲线；
|
||||
- 精确部分：函数值；
|
||||
- 边界：不从曲线推导训练 loss。
|
||||
|
||||
### Lab F — Quantile Balancing
|
||||
|
||||
- 输入：expert score samples、top-k、quantile target、bias step；
|
||||
- 输出：Top-(k+1) cutoff、近似 quantile、下一 step bias；
|
||||
- 边界：toy histogram，不代表 K3 router trace。
|
||||
|
||||
### Lab G — MOPD / partial rollout
|
||||
|
||||
- 输入：domain、effort、τ、λ、teacher/student token probability；
|
||||
- 输出：budget pass/fail、暂停轨迹数、clipped dense reward；
|
||||
- 精确部分：报告公式的标量例子；
|
||||
- 边界：不模拟完整 policy optimization。
|
||||
|
||||
### Lab H — Prefix cache / context management
|
||||
|
||||
- 输入：matched prefix、physical block、hash block、checkpoint boundary；
|
||||
- 输出：可命中最长边界、需重算 token；
|
||||
- 精确部分：边界算术；
|
||||
- 边界：不代表线上 cache hit rate。
|
||||
|
||||
## 6. §3 数据、Scaling 与训练配方
|
||||
|
||||
### 6.1 数据
|
||||
|
||||
- 文本四域：Web Text、Code、Mathematics、Knowledge；
|
||||
- 视觉包含 captions、interleaved documents、OCR、perception、video、visual coding；
|
||||
- 坐标同时使用绝对坐标与 `[0,1]` 归一坐标；
|
||||
- programmatic visual data 包括 SVG、3D、Webpage、Game、CAD；
|
||||
- K2 参与知识与数学 rephrasing：多样风格/视角、chunk-wise AR generation、fidelity verification；
|
||||
- 不能披露/推断：完整来源清单、精确配比、总 token、版权构成。
|
||||
|
||||
### 6.2 Scaling
|
||||
|
||||
- 小模型实验重调 batch size、learning rate、tokens per parameter、model shape；
|
||||
- 使用 held-out OOD data；
|
||||
- cosine 与 WSD 各自搜索最优超参，作者设置中 cosine 最终 loss 更低；
|
||||
- 约 2.5× 是架构、recipe 与数据共同结果。
|
||||
|
||||
### 6.3 长上下文
|
||||
|
||||
- NoPE 省去 RoPE rescale / interpolation，但不解决全部训练与利用问题；
|
||||
- long document/video 用 exact+fuzzy dedup、frame perceptual hash、heuristic/classifier filter、
|
||||
structural validation；
|
||||
- 稀缺长数据被上采样；
|
||||
- synthetic concatenation/permutation 要求答案依赖跨整段分散证据；
|
||||
- curriculum：8K→64K（pretrain），256K→1M（cooldown）。
|
||||
|
||||
## 7. §4 后训练链
|
||||
|
||||
```text
|
||||
verified/HITL trajectory synthesis
|
||||
↓
|
||||
SFT cold start + XTML + MXFP4/8 QAT
|
||||
↓
|
||||
3 domains × 3 effort = 9 RL experts
|
||||
↓ ↘
|
||||
partial rollout + stale tolerance agentic GRM / verifier
|
||||
↓
|
||||
MOPD: student on-policy prefix + matched teacher dense reward
|
||||
↓
|
||||
one unified model with selectable effort
|
||||
↓
|
||||
MTP → EAGLE-3 draft + LK acceptance loss
|
||||
```
|
||||
|
||||
关键边界：
|
||||
|
||||
- “九专家”是训练中的策略老师，不是服务时并发 9 模型；
|
||||
- effort budget 每题、每域变化，不是固定全局长度；
|
||||
- agentic task 的 token 账包含 tool-call arguments 等输出；
|
||||
- partial rollout 缓解长尾，但引入 stale/off-policy；
|
||||
- on-policy distillation 用学生访问的 prefix，仍依赖教师质量与领域映射；
|
||||
- non-verifiable reward 的 rubric/judge 仍可能有偏，报告用强制协议和 verbosity budget 缓解。
|
||||
|
||||
## 8. §5 系统状态图
|
||||
|
||||
| 层 | 主要状态 | 生命周期 | 主要机制 |
|
||||
|---|---|---|---|
|
||||
| pretraining | weights、optimizer、activation、KDA state、vision tokens | step / run | FlashKDA、KCP、MoonEP、pipeline/offload |
|
||||
| agentic RL | policy/ref、KV、trajectory、sandbox、files/apps | 多 iteration | partial rollout、adaptive throttle、AgentENV pause/resume/fork/snapshot |
|
||||
| serving engine | KDA recurrent state、MLA KV、prefix hash | request/session | unified page pool、sparse checkpoints、COW |
|
||||
| device | KDA replay input、AttnRes blocks、latent/routed weights | token / batch | fused decode、SP、side stream、WarpDecode-like token-centric kernel |
|
||||
| fleet | prefix affinity、request class budget | session / traffic | primary+secondary consistent hash、budget admission |
|
||||
|
||||
正式数字：
|
||||
|
||||
- AgentENV：报告称训练/评测期间创建 51,219,741 sandboxes，涉及 1,505,678 images；
|
||||
- pause 时 sandbox 不消耗 CPU/内存；报告称等待 inference 可达其生命周期 98%；
|
||||
- incremental checkpoint/resume 最低延迟分别 133 ms / 49 ms；
|
||||
- real workload memory overcommit 最高 6.5×；
|
||||
- serving 示例：typical coding input 400K prefix、4K increment；请求成本跨约三数量级；
|
||||
- 这些均是作者系统报告数字，尚非本站复跑。
|
||||
|
||||
## 9. §7 案例与附录
|
||||
|
||||
### 9.1 Kernel optimization
|
||||
|
||||
- AttnRes 个案 283.6 ms → 114.4 ms；
|
||||
- DSA / KDA reductions 为 55.1% / 73.6%；
|
||||
- MLA 达到报告所称超过一半 peak；
|
||||
- budget 为 24h / task；
|
||||
- 只能表述为作者案例，不能推广为平均收益。
|
||||
|
||||
### 9.2 MiniTriton
|
||||
|
||||
- 模型构建 compiler、tensor library、autograd、distributed stack；
|
||||
- 报告结果基于 L20 与其任务/roofline；
|
||||
- 展示“Agent 可改整个栈”，不证明自动编译普遍优于人工系统。
|
||||
|
||||
### 9.3 Chip
|
||||
|
||||
- nano model 与主架构同型，但 Block AttnRes block size=2；
|
||||
- 48h agent run；nano-kpu；4mm²、100MHz、RTL simulation >8700 tok/s 等均为报告个案；
|
||||
- 不得把 nano 参数写回 K3 主模型规格。
|
||||
|
||||
### 9.4 XTML
|
||||
|
||||
- message markers：`[open]`、`[sep]`、`[close]`、`[end_of_msg]`；
|
||||
- global options 在 history 前；one-shot options 在 history 后以保留 KV cache；
|
||||
- input options 可在会话中动态追加工具；
|
||||
- channels：think / response / tools；
|
||||
- thinking / instruct 由 prefix 选择。
|
||||
|
||||
## 10. 100 节点阅读链的选择规则
|
||||
|
||||
正式页面的 100 个节点按 `src/data/k3.ts` 维护，遵守：
|
||||
|
||||
1. 优先 K3 报告的 151 条参考文献与作者官方仓库；
|
||||
2. 论文链接到 arXiv、PMLR、OpenReview、NeurIPS/USENIX 等原始页面；
|
||||
3. 软件链接到作者/组织官方仓库；
|
||||
4. benchmark 链接到论文或官方站；
|
||||
5. 2026 年厂商比较只作为评测上下文，不进入基础技术因果链；
|
||||
6. 每个节点只写一个“为什么此处要读它”，不伪造继承关系；
|
||||
7. DeepSeekMoE、V2/MLA、V3、V4 与 R1 单列交叉入口，满足重点讲解偏好；
|
||||
8. K3 报告未引用但为历史必需的节点，必须标成课程补充，不伪装成报告引用。
|
||||
|
||||
## 11. 不可越过的红线
|
||||
|
||||
- `2.8T` 是四舍五入 hero 数；精确表用 `2.78T / 104.2B`；
|
||||
- `2.5×` 不写成推理加速、吞吐或 KDA 单项收益；
|
||||
- KDA 是压缩状态，不承诺逐 token 无损 recall；
|
||||
- NoPE 没有**显式** positional embedding，不是无顺序；
|
||||
- QB、MoonEP、fleet scheduling 分属模型、执行、服务三个层次；
|
||||
- MoonViT-V2 从头训练，视觉/文本从开始联合 NTP；
|
||||
- 1M 是窗口与训练 curriculum，不保证所有任务能有效利用完整窗口；
|
||||
- MOPD 是训练整合，不是 inference ensemble；
|
||||
- benchmark 是 model × effort × tool × harness × protocol；
|
||||
- case study 数字始终带硬件、时间、任务或仿真边界；
|
||||
- 不补写总训练 token、完整数据配比、总 GPU 数、训练成本；
|
||||
- 2026 新组件必须明确第三方复现仍有限。
|
||||
|
||||
## 12. 实现检查表
|
||||
|
||||
- [x] 保存本地官方 PDF 与可检索文本；
|
||||
- [x] Grok 候选线索与正式证据隔离；
|
||||
- [x] 32 张问题账完成正式核验；
|
||||
- [x] Figure 1–16、Table 1–5 建立视觉契约；
|
||||
- [x] 八个实验建立输入/输出/边界合同；
|
||||
- [x] 实现 `src/data/k3.ts` 的 100 节点、问题账与图表数据；
|
||||
- [x] 实现 `K3ReportLab.astro`；
|
||||
- [x] 重写 K3 页面并修正原生多模态错误；
|
||||
- [x] 同步 papers、roadmap、progress、首页发布记录；
|
||||
- [x] 添加 K3 专属 browser regression；
|
||||
- [x] 本地 K3 browser regression、Astro check 与 build 通过；
|
||||
- [ ] 生产全量验收与不可变发布记录。
|
||||
@@ -228,7 +228,7 @@ if (numeric(reliability.initial.passAt) <= numeric(reliability.k2.passAt) || num
|
||||
if (reliability.nonIdempotent.sideRisk === "LOW") failures.push("非幂等写操作风险没有提升");
|
||||
if (numeric(rl.wait.utilization) >= numeric(rl.full.utilization) || numeric(rl.wait.lostWork) <= numeric(rl.full.lostWork)) failures.push("wait-all 长尾/重算方向异常");
|
||||
if (!rl.wait.takeaway.includes("wait-all") || rl.keyboardSelected !== "rl" || rl.keyboardVisible !== "rl") failures.push("长程 RL 解释或键盘导航异常");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") failures.push("首页评测首发入口异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常");
|
||||
if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasAgentFilter || papers.agentVisible < 52) failures.push("论文库 Agent 标签或论文总数异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
|
||||
if (exceptions.length) failures.push(`浏览器异常：${exceptions.join(" | ")}`);
|
||||
|
||||
@@ -226,7 +226,7 @@ if (!update.steps[0].includes("Fixed preference")) failures.push("DPO 更新流
|
||||
if (!recipe.family.includes("Multi-effort") || !recipe.regime.includes("9 RL experts") || !recipe.constraints.includes("verbosity")) failures.push("K3 配方合同异常");
|
||||
if (!recipe.path.some((step) => step.includes("3 domains × 3 efforts")) || !recipe.path.some((step) => step.includes("MOPD"))) failures.push("K3 配方路径异常");
|
||||
if (recipe.keyboardSelected !== "recipe" || recipe.keyboardVisible !== "recipe") failures.push("实验 tab 键盘导航异常");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") failures.push("首页评测首发入口异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常");
|
||||
if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasAlignmentFilter || papers.alignmentVisible < 35) failures.push("论文库后训练标签或论文总数异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
|
||||
if (exceptions.length) failures.push(`浏览器异常：${exceptions.join(" | ")}`);
|
||||
|
||||
@@ -234,7 +234,7 @@ if (layout.navLinks !== 20 || mobile.mobileLinks !== 20 || home.navLinks !== 20)
|
||||
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
|
||||
if (layout.navGap < 0) failures.push(`桌面导航碰撞：${layout.navGap}px`);
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") {
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") {
|
||||
failures.push("首页 Transformer 新章入口异常");
|
||||
}
|
||||
if (home.paperCount !== "486") failures.push(`首页论文总数异常：${home.paperCount}`);
|
||||
|
||||
@@ -287,7 +287,7 @@ if (rl.initial.signal !== "GROUP-RELATIVE SIGNAL" || rl.same.signal !== "ZERO GR
|
||||
if (!rl.dapo.provenance.includes("2503.14476") || !rl.dapo.algorithm.includes("FOLLOW-UP") || !rl.dr.provenance.includes("2503.20783")) failures.push("DAPO / Dr.GRPO 来源边界异常");
|
||||
if (!rl.r1.includes("cold start") || !rl.distill.includes("没有重演")) failures.push("R1 / distill 身份切换异常");
|
||||
if (rl.keyboardSelected !== "cache" || rl.keyboardVisible !== "cache") failures.push("实验键盘 tab 导航异常");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/" || home.paperCount !== "486") failures.push("首页 DeepSeek 首发入口或论文数异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/" || home.paperCount !== "486") failures.push("首页 DeepSeek 首发入口或论文数异常");
|
||||
if (papers.total !== 486 || !papers.hasFilter || papers.visible < 20 || !papers.hasCoder || !papers.hasEngram) failures.push("论文库 DeepSeek 聚光异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
|
||||
if (mobile.offenders.length) failures.push(`移动端越界元素：${JSON.stringify(mobile.offenders)}`);
|
||||
|
||||
@@ -277,7 +277,7 @@ if (numeric(system.initial.success) <= numeric(system.initial.model) || numeric(
|
||||
if (numeric(system.cheap.success) >= numeric(system.initial.success) || numeric(system.cheap.cost) !== 4) failures.push("低预算没有降低成功率 / 成本");
|
||||
if (numeric(system.locked.unsafe) !== 0 || numeric(system.locked.overrefusal) <= numeric(system.initial.overrefusal)) failures.push("安全壳没有展现危险服从 / 过拒权衡");
|
||||
if (system.keyboardSelected !== "judge" || system.keyboardVisible !== "judge") failures.push("实验键盘 tab 导航异常");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") failures.push("首页评测首发入口异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常");
|
||||
if (home.paperCount !== "486" || home.topicCount !== "17" || papers.total !== 486 || !papers.hasFilter || papers.visible < 80) failures.push("首页 / 论文库评测索引异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
|
||||
if (mobile.offenders.length) failures.push(`移动端越界元素：${JSON.stringify(mobile.offenders)}`);
|
||||
|
||||
@@ -264,7 +264,7 @@ if (!fleet.k3.avoided.includes("320K") || fleet.k3.shortSlo !== "PROTECTED") fai
|
||||
if (!fleet.failed.state.includes("SECONDARY RE-PREFILL") || !fleet.failed.recompute.includes("FAILED PRIMARY")) failures.push("缓存故障没有触发原子失效后的重算");
|
||||
if (fleet.bursty.shortSlo !== "VIOLATED") failures.push("平均并发阈值没有暴露长请求突发");
|
||||
if (fleet.keyboardSelected !== "phase" || fleet.keyboardVisible !== "phase") failures.push("实验键盘 tab 导航异常");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") failures.push("首页评测首发入口异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常");
|
||||
if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasFilter || papers.visible !== 46) failures.push("论文库推理服务标签或总数异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
|
||||
if (mobile.offenders.length) failures.push(`移动端越界元素：${JSON.stringify(mobile.offenders)}`);
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
import { writeFileSync } from "node:fs";
|
||||
|
||||
const cdpPort = process.env.CDP_PORT ?? "9227";
|
||||
const baseUrl = process.env.SITE_URL ?? "http://127.0.0.1:4327";
|
||||
const pages = await fetch(`http://127.0.0.1:${cdpPort}/json/list`).then((response) => response.json());
|
||||
const page = pages.find((entry) => entry.type === "page");
|
||||
if (!page) throw new Error(`CDP ${cdpPort} 没有可用页面`);
|
||||
|
||||
const socket = new WebSocket(page.webSocketDebuggerUrl);
|
||||
await new Promise((resolve, reject) => {
|
||||
socket.addEventListener("open", resolve, { once: true });
|
||||
socket.addEventListener("error", reject, { once: true });
|
||||
});
|
||||
|
||||
let nextId = 0;
|
||||
const pending = new Map();
|
||||
const exceptions = [];
|
||||
socket.addEventListener("message", (event) => {
|
||||
const message = JSON.parse(event.data);
|
||||
if (message.id && pending.has(message.id)) {
|
||||
const { resolve, reject } = pending.get(message.id);
|
||||
pending.delete(message.id);
|
||||
if (message.error) reject(new Error(message.error.message));
|
||||
else resolve(message.result);
|
||||
}
|
||||
if (message.method === "Runtime.exceptionThrown") {
|
||||
exceptions.push(message.params.exceptionDetails.exception?.description ?? message.params.exceptionDetails.text);
|
||||
}
|
||||
});
|
||||
|
||||
const command = (method, params = {}) => new Promise((resolve, reject) => {
|
||||
const id = ++nextId;
|
||||
pending.set(id, { resolve, reject });
|
||||
socket.send(JSON.stringify({ id, method, params }));
|
||||
});
|
||||
const pause = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
|
||||
const evaluate = async (expression) => {
|
||||
const result = await command("Runtime.evaluate", { expression, returnByValue: true, awaitPromise: true });
|
||||
if (result.exceptionDetails) throw new Error(result.exceptionDetails.exception?.description ?? result.exceptionDetails.text);
|
||||
return result.result.value;
|
||||
};
|
||||
const navigate = async (path) => {
|
||||
await command("Page.navigate", { url: `${baseUrl}${path}` });
|
||||
for (let attempt = 0; attempt < 80; attempt += 1) {
|
||||
await pause(100);
|
||||
if (await evaluate("document.readyState === 'complete'")) return;
|
||||
}
|
||||
throw new Error(`${path} 加载超时`);
|
||||
};
|
||||
const screenshot = async (path, beyond = false) => {
|
||||
const result = await command("Page.captureScreenshot", { format: "png", captureBeyondViewport: beyond });
|
||||
writeFileSync(path, Buffer.from(result.data, "base64"));
|
||||
};
|
||||
|
||||
await command("Page.enable");
|
||||
await command("Runtime.enable");
|
||||
await command("Emulation.setDeviceMetricsOverride", {
|
||||
width: 1440,
|
||||
height: 1100,
|
||||
deviceScaleFactor: 1,
|
||||
mobile: false,
|
||||
});
|
||||
|
||||
await navigate("/k3/");
|
||||
await screenshot("/tmp/llm-atlas-k3-desktop.png");
|
||||
|
||||
const overview = await evaluate(`(() => ({
|
||||
title: document.querySelector("h1")?.textContent.trim(),
|
||||
sections: document.querySelectorAll(".article-section").length,
|
||||
tocLinks: document.querySelectorAll(".side-rail a").length,
|
||||
ledgers: document.querySelectorAll(".ledger-card").length,
|
||||
reportMap: document.querySelectorAll(".report-map > article").length,
|
||||
figureAtlas: document.querySelectorAll(".figure-atlas > article").length,
|
||||
paperLinks: document.querySelectorAll("#papers .paper-row").length,
|
||||
paperGroups: document.querySelectorAll("#papers .paper-group").length,
|
||||
labTabs: document.querySelectorAll("[data-k3-tab]").length,
|
||||
labPanels: document.querySelectorAll("[data-k3-panel]").length,
|
||||
nativeVisionCorrected: document.body.textContent.includes("MoonViT‑V2 从头训练") &&
|
||||
document.body.textContent.includes("同一个 next-token prediction objective"),
|
||||
staleVisionClaim: document.body.textContent.includes("先固定语言模型训练视觉组件"),
|
||||
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
|
||||
}))()`);
|
||||
|
||||
const labs = await evaluate(`(() => {
|
||||
const root = document.querySelector("[data-k3-lab]");
|
||||
const panel = () => root.querySelector("[data-k3-panel]:not([hidden])").dataset.k3Panel;
|
||||
const text = (selector) => root.querySelector(selector).textContent.trim();
|
||||
const input = (selector, value) => {
|
||||
const node = root.querySelector(selector);
|
||||
node.value = value;
|
||||
node.dispatchEvent(new Event("input", { bubbles: true }));
|
||||
};
|
||||
|
||||
const memoryInitial = {
|
||||
panel: panel(),
|
||||
additive: text("[data-additive-result]"),
|
||||
delta: text("[data-delta-result]"),
|
||||
additiveError: text("[data-additive-error]"),
|
||||
deltaError: text("[data-delta-error]"),
|
||||
};
|
||||
input("[data-memory-writes]", "12");
|
||||
const memoryChanged = { additive: text("[data-additive-result]"), delta: text("[data-delta-result]") };
|
||||
|
||||
root.querySelector('[data-k3-tab="decay"]').click();
|
||||
const decayInitial = { panel: panel(), verdict: text("[data-decay-verdict] b"), log: text("[data-decay-log]"), recip: text("[data-decay-recip]") };
|
||||
input("[data-decay-g]", "-120");
|
||||
input("[data-decay-tile]", "32");
|
||||
const decayRisk = { verdict: text("[data-decay-verdict] b"), log: text("[data-decay-log]") };
|
||||
|
||||
root.querySelector('[data-k3-tab="depth"]').click();
|
||||
const depthInitial = { sources: text("[data-block-sources]"), contract: text("[data-depth-reduction]") };
|
||||
input("[data-depth-block]", "6");
|
||||
const depthChanged = { sources: text("[data-block-sources]"), contract: text("[data-depth-reduction]") };
|
||||
|
||||
root.querySelector('[data-k3-tab="width"]').click();
|
||||
const widthInitial = { conventional: text("[data-width-conventional]"), latent: text("[data-width-latent]"), reduction: text("[data-width-reduction]") };
|
||||
input("[data-width-latent]", "7168");
|
||||
const widthFull = { reduction: text("[data-width-reduction]") };
|
||||
|
||||
root.querySelector('[data-k3-tab="situ"]').click();
|
||||
const situInitial = { bound: text("[data-situ-bound]"), value: text("[data-situ-value]"), swiglu: text("[data-situ-swiglu]") };
|
||||
input("[data-situ-x]", "1000");
|
||||
const situLarge = { bound: text("[data-situ-bound]"), value: text("[data-situ-value]"), swiglu: text("[data-situ-swiglu]") };
|
||||
|
||||
root.querySelector('[data-k3-tab="qb"]').click();
|
||||
const qbInitial = { before: text("[data-qb-before-copy]"), after: text("[data-qb-after-copy]"), bias: text("[data-qb-bias]"), gap: text("[data-qb-gap]") };
|
||||
input("[data-qb-strength]", "0");
|
||||
const qbOff = { after: text("[data-qb-after-copy]"), gap: text("[data-qb-gap]") };
|
||||
|
||||
root.querySelector('[data-k3-tab="rl"]').click();
|
||||
const rlInitial = {
|
||||
complete: text("[data-rl-complete]"),
|
||||
paused: text("[data-rl-paused]"),
|
||||
budget: text("[data-rl-budget-copy]"),
|
||||
reward: text("[data-rl-reward]"),
|
||||
};
|
||||
input("[data-rl-tokens]", "40");
|
||||
const rlOver = { budget: text("[data-rl-budget-copy]") };
|
||||
|
||||
root.querySelector('[data-k3-tab="cache"]').click();
|
||||
const cacheInitial = {
|
||||
mla: text("[data-cache-mla]"),
|
||||
kda: text("[data-cache-kda]"),
|
||||
hit: text("[data-cache-hit]"),
|
||||
recompute: text("[data-cache-recompute]"),
|
||||
};
|
||||
input("[data-cache-checkpoint]", "3");
|
||||
const cacheSparse = { kda: text("[data-cache-kda]"), hit: text("[data-cache-hit]"), recompute: text("[data-cache-recompute]") };
|
||||
|
||||
const first = root.querySelector('[data-k3-tab="memory"]');
|
||||
first.focus();
|
||||
first.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowRight", bubbles: true }));
|
||||
return {
|
||||
memoryInitial, memoryChanged, decayInitial, decayRisk, depthInitial, depthChanged,
|
||||
widthInitial, widthFull, situInitial, situLarge, qbInitial, qbOff, rlInitial, rlOver,
|
||||
cacheInitial, cacheSparse,
|
||||
keyboardSelected: root.querySelector('[data-k3-tab][aria-selected="true"]').dataset.k3Tab,
|
||||
keyboardVisible: panel(),
|
||||
};
|
||||
})()`);
|
||||
|
||||
await evaluate(`(() => {
|
||||
document.querySelector("[data-k3-lab]").scrollIntoView({ block: "start", behavior: "instant" });
|
||||
window.scrollBy(0, -82);
|
||||
})()`);
|
||||
await pause(180);
|
||||
await screenshot("/tmp/llm-atlas-k3-lab-desktop.png");
|
||||
|
||||
await command("Emulation.setDeviceMetricsOverride", {
|
||||
width: 390,
|
||||
height: 844,
|
||||
deviceScaleFactor: 1,
|
||||
mobile: true,
|
||||
});
|
||||
await navigate("/k3/");
|
||||
const mobile = await evaluate(`(() => {
|
||||
const root = document.querySelector("[data-k3-lab]");
|
||||
root.scrollIntoView({ block: "start", behavior: "instant" });
|
||||
const toggle = document.querySelector("#menu-toggle");
|
||||
toggle?.click();
|
||||
return {
|
||||
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
|
||||
menuVisible: getComputedStyle(toggle).display !== "none",
|
||||
menuOpen: toggle.getAttribute("aria-expanded"),
|
||||
tabs: root.querySelectorAll("[data-k3-tab]").length,
|
||||
offenders: [...document.querySelectorAll("body *")]
|
||||
.filter((node) => !node.closest(".paper-chain, .spec-table-wrap, .cache-strip, .architecture-explorer, [data-k3-lab]"))
|
||||
.filter((node) => node.getBoundingClientRect().right > document.documentElement.clientWidth + 1)
|
||||
.slice(0, 15)
|
||||
.map((node) => ({
|
||||
tag: node.tagName,
|
||||
className: typeof node.className === "string" ? node.className : "",
|
||||
right: Math.round(node.getBoundingClientRect().right),
|
||||
width: Math.round(node.getBoundingClientRect().width),
|
||||
})),
|
||||
};
|
||||
})()`);
|
||||
await evaluate(`(() => {
|
||||
document.querySelector("#menu-toggle")?.click();
|
||||
window.scrollBy(0, -82);
|
||||
})()`);
|
||||
await pause(180);
|
||||
await screenshot("/tmp/llm-atlas-k3-mobile.png");
|
||||
|
||||
const report = { overview, labs, mobile, exceptions };
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
|
||||
const numeric = (text) => Number.parseFloat(text.replaceAll(",", "").replace("−", "-"));
|
||||
const failures = [];
|
||||
if (!overview.title.includes("因果环节")) failures.push("K3 二轮标题异常");
|
||||
if (overview.sections !== 31 || overview.tocLinks !== 31) failures.push("30 个编号专题加阅读链的目录结构异常");
|
||||
if (overview.ledgers !== 32 || overview.reportMap !== 9) failures.push("32 张问题账或报告地图异常");
|
||||
if (overview.figureAtlas !== 21 || overview.paperLinks !== 100 || overview.paperGroups < 12) failures.push("图表审计或 100 节点阅读链异常");
|
||||
if (overview.labTabs !== 8 || overview.labPanels !== 8) failures.push("八联实验结构异常");
|
||||
if (!overview.nativeVisionCorrected || overview.staleVisionClaim) failures.push("原生多模态纠错未生效或旧错误残留");
|
||||
if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出");
|
||||
if (labs.memoryInitial.panel !== "memory" || numeric(labs.memoryInitial.additiveError) <= numeric(labs.memoryInitial.deltaError)) failures.push("Delta memory 初始递推异常");
|
||||
if (numeric(labs.memoryChanged.additive) <= numeric(labs.memoryInitial.additive) || numeric(labs.memoryChanged.delta) <= numeric(labs.memoryInitial.delta)) failures.push("Delta memory 控件未更新");
|
||||
if (labs.decayInitial.panel !== "decay" || labs.decayInitial.verdict !== "WITHIN RANGE" || numeric(labs.decayInitial.log) !== -80) failures.push("K3 默认 bounded decay 范围异常");
|
||||
if (labs.decayRisk.verdict !== "OVERFLOW RISK" || numeric(labs.decayRisk.log) !== -384) failures.push("bounded decay 风险探针异常");
|
||||
if (numeric(labs.depthInitial.sources) !== 9 || numeric(labs.depthChanged.sources) !== 17) failures.push("Block AttnRes 来源计数异常");
|
||||
if (numeric(labs.widthInitial.reduction) !== 50 || numeric(labs.widthFull.reduction) !== 0) failures.push("LatentMoE payload 账异常");
|
||||
if (numeric(labs.situInitial.bound) !== 100 || numeric(labs.situLarge.value) > 100.01 || numeric(labs.situLarge.swiglu) <= 1000) failures.push("SiTU formal bound 异常");
|
||||
if (!labs.qbInitial.before.includes("4, 3, 1, 0") || numeric(labs.qbInitial.gap) > 1 || numeric(labs.qbOff.gap) < 3) failures.push("Quantile Balancing toy route 异常");
|
||||
if (!labs.rlInitial.complete.includes("192") || !labs.rlInitial.paused.includes("64") || numeric(labs.rlInitial.reward) !== 0.693) failures.push("MOPD / partial rollout 初始账异常");
|
||||
if (!labs.rlOver.budget.includes("reward 改为 −1")) failures.push("Reasoning effort 超预算未触发");
|
||||
if (numeric(labs.cacheInitial.hit) !== 2560 || !labs.cacheInitial.recompute.includes("256")) failures.push("Hybrid prefix cache 默认命中异常");
|
||||
if (numeric(labs.cacheSparse.hit) >= numeric(labs.cacheInitial.hit) || numeric(labs.cacheSparse.recompute) <= numeric(labs.cacheInitial.recompute)) failures.push("稀疏 KDA checkpoint 未降低 joint hit");
|
||||
if (labs.keyboardSelected !== "decay" || labs.keyboardVisible !== "decay") failures.push("实验键盘 tab 导航异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 8) failures.push("移动端导航或实验异常");
|
||||
if (mobile.offenders.length) failures.push(`移动端越界元素：${JSON.stringify(mobile.offenders)}`);
|
||||
if (exceptions.length) failures.push(`浏览器异常：${exceptions.join(" | ")}`);
|
||||
|
||||
if (failures.length) {
|
||||
console.error(`\nFAIL\n- ${failures.join("\n- ")}`);
|
||||
process.exitCode = 1;
|
||||
} else {
|
||||
console.log("\nPASS K3 browser regression");
|
||||
}
|
||||
|
||||
socket.close();
|
||||
@@ -177,7 +177,7 @@ if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentO
|
||||
}
|
||||
if (layout.navGap < 0) failures.push(`桌面导航碰撞：${layout.navGap}px`);
|
||||
if (!mobile.menuVisible) failures.push("移动端菜单按钮未显示");
|
||||
if (home.releaseCards !== 16) failures.push(`首页新章卡数量异常：${home.releaseCards}`);
|
||||
if (home.releaseCards !== 17) failures.push(`首页新章卡数量异常：${home.releaseCards}`);
|
||||
if (exceptions.length) failures.push(`浏览器脚本异常：${exceptions.join("; ")}`);
|
||||
|
||||
socket.close();
|
||||
|
||||
@@ -246,7 +246,7 @@ if (ocr.unreported.status !== "OUT OF EVIDENCE" || ocr.unreported.accuracy !== "
|
||||
if (loop.toolsStart.state !== "OPEN" || loop.toolsEnd.state !== "VERIFIED" || loop.toolsEnd.evidence !== "97%" || loop.toolsEnd.tools !== "3") failures.push("vision-in-the-loop 终局异常");
|
||||
if (loop.cotEnd.state !== "FAILED" || !loop.cotEnd.takeaway.includes("不能凭空增加")) failures.push("文字 CoT 与新观察没有分开");
|
||||
if (loop.keyboardSelected !== "connector" || loop.keyboardVisible !== "connector") failures.push("实验键盘 tab 导航异常");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") failures.push("首页评测首发入口异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常");
|
||||
if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasFilter || papers.multimodalVisible < 59) failures.push("论文库多模态标签或总数异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
|
||||
if (mobile.offenders.length) failures.push(`移动端越界元素：${JSON.stringify(mobile.offenders)}`);
|
||||
|
||||
@@ -277,7 +277,7 @@ if (layout.navLinks !== 20 || mobile.mobileLinks !== 20 || home.navLinks !== 20)
|
||||
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
|
||||
if (layout.navGap < 0) failures.push(`桌面导航碰撞：${layout.navGap}px`);
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") {
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") {
|
||||
failures.push("首页 Transformer 新章入口异常");
|
||||
}
|
||||
if (home.paperCount !== "486") failures.push(`首页论文总数异常：${home.paperCount}`);
|
||||
|
||||
@@ -289,7 +289,7 @@ if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentO
|
||||
}
|
||||
if (layout.navGap < 0) failures.push(`桌面导航碰撞：${layout.navGap}px`);
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文")) failures.push("首页评测新章入口异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要")) failures.push("首页评测新章入口异常");
|
||||
if (exceptions.length) failures.push(`浏览器脚本异常：${exceptions.join("; ")}`);
|
||||
|
||||
socket.close();
|
||||
|
||||
@@ -288,7 +288,7 @@ if (numeric(residual.attnres.states) !== 9 || !residual.attnres.routeExplain.inc
|
||||
if (!residual.clamp.activation.includes("V4") || !residual.clamp.bound.includes("100")) failures.push("DeepSeek-V4 clamp 展示异常");
|
||||
if (!residual.situ.activation.includes("KIMI") || !residual.situ.bound.includes("100")) failures.push("K3 SiTU 上界展示异常");
|
||||
if (residual.keyboardSelected !== "position" || residual.keyboardVisible !== "position") failures.push("实验键盘 tab 导航异常");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") failures.push("首页表示新章入口异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页表示新章入口异常");
|
||||
if (home.paperCount !== "486" || home.topicCount !== "17" || papers.total !== 486 || !papers.hasFilter || papers.visible < 30) failures.push("首页 / 论文库表示索引异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
|
||||
if (mobile.offenders.length) failures.push(`移动端越界元素：${JSON.stringify(mobile.offenders)}`);
|
||||
|
||||
@@ -273,7 +273,7 @@ if (layout.navLinks !== 20 || mobile.mobileLinks !== 20 || home.navLinks !== 20)
|
||||
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
|
||||
if (layout.navGap < 0) failures.push(`桌面导航碰撞：${layout.navGap}px`);
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") {
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") {
|
||||
failures.push("首页 Transformer 新章入口异常");
|
||||
}
|
||||
if (home.paperCount !== "486") failures.push(`首页论文总数异常：${home.paperCount}`);
|
||||
|
||||
@@ -233,7 +233,7 @@ if (layout.articleSections !== 16 || layout.paperLinks !== 37 || layout.labTabs
|
||||
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
|
||||
if (layout.navGap < 0) failures.push(`桌面导航碰撞：${layout.navGap}px`);
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文")) failures.push("首页评测新章入口异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要")) failures.push("首页评测新章入口异常");
|
||||
if (exceptions.length) failures.push(`浏览器脚本异常：${exceptions.join("; ")}`);
|
||||
|
||||
socket.close();
|
||||
|
||||
@@ -236,7 +236,7 @@ if (block.family.trim() !== "Hybrid MoE" || !block.kv.includes("3 KDA : 1 Gated
|
||||
if (!block.path.some((step) => step.includes("KDA × 3")) || !block.note.includes("AttnRes")) failures.push("K3 Block 路径异常");
|
||||
if (block.context.trim() !== "128K" || numeric(block.mha) !== 400 || numeric(block.kda) !== 1) failures.push("KV 成本缩放异常");
|
||||
if (block.keyboardSelected !== "block" || block.keyboardVisible !== "block") failures.push("实验 tab 键盘导航异常");
|
||||
if (home.releaseCards !== 16 || !home.firstRelease.includes("从 Dense 到百万上下文") || home.firstHref !== "/deepseek/") failures.push("首页评测首发入口异常");
|
||||
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常");
|
||||
if (home.paperCount !== "486" || papers.total !== 486 || papers.transformerVisible < 30) failures.push("论文库或首页论文数量异常");
|
||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
|
||||
if (exceptions.length) failures.push(`浏览器异常：${exceptions.join(" | ")}`);
|
||||
|
||||
@@ -0,0 +1,725 @@
|
||||
<figure class="k3-lab" data-k3-lab>
|
||||
<div class="lab-head">
|
||||
<div>
|
||||
<p>INTERACTIVE / EIGHT REPORT WORKBENCHES</p>
|
||||
<h3>把八个容易误读的机制，拆成可以动手验算的对象</h3>
|
||||
</div>
|
||||
<p>
|
||||
数学输出按页面公式实时计算；风险、通信与吞吐标签只是显式 toy model。
|
||||
本实验没有复跑 K3 checkpoint、FlashKDA kernel、MoonEP 集群或线上 cache。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="lab-tabs" role="tablist" aria-label="选择 K3 技术实验">
|
||||
{[
|
||||
["memory", "01", "DELTA MEMORY", "纠错写入"],
|
||||
["decay", "02", "BOUNDED DECAY", "BF16 范围"],
|
||||
["depth", "03", "ATTNRES", "深度来源"],
|
||||
["width", "04", "LATENTMOE", "通信 payload"],
|
||||
["situ", "05", "SITU-GLU", "有界激活"],
|
||||
["qb", "06", "QB ROUTER", "分位均衡"],
|
||||
["rl", "07", "MOPD / RL", "预算与长尾"],
|
||||
["cache", "08", "PREFIX CACHE", "混合状态命中"],
|
||||
].map(([id, number, title, subtitle], index) => (
|
||||
<button
|
||||
type="button"
|
||||
role="tab"
|
||||
data-k3-tab={id}
|
||||
aria-selected={index === 0 ? "true" : "false"}
|
||||
tabindex={index === 0 ? "0" : "-1"}
|
||||
>
|
||||
<span>{number}</span><b>{title}</b><small>{subtitle}</small>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<section class="lab-panel" data-k3-panel="memory">
|
||||
<div class="panel-intro">
|
||||
<div><span>WORKBENCH 01 / DELTA RULE</span><h4>同一个 key 反复出现时，累加和纠错写入会走向完全不同的 state</h4></div>
|
||||
<p>为便于看懂，这里把矩阵 state 压成“当前 key 对应的标量预测”。真实 KDA 是每个 head 的矩阵状态。</p>
|
||||
</div>
|
||||
<div class="controls four">
|
||||
<label><span>重复写入次数 <output data-memory-writes-label>6</output></span><input data-memory-writes type="range" min="1" max="20" value="6" /></label>
|
||||
<label><span>写入强度 β <output data-memory-beta-label>0.50</output></span><input data-memory-beta type="range" min="5" max="100" value="50" /></label>
|
||||
<label><span>旧预测 <output data-memory-old-label>0.00</output></span><input data-memory-old type="range" min="-100" max="100" value="0" /></label>
|
||||
<label><span>目标 value <output data-memory-target-label>1.00</output></span><input data-memory-target type="range" min="-100" max="200" value="100" /></label>
|
||||
</div>
|
||||
<div class="memory-stage">
|
||||
<article>
|
||||
<span>NAIVE ADDITIVE</span>
|
||||
<code>s ← s + βv</code>
|
||||
<div class="trace" data-additive-trace></div>
|
||||
<b data-additive-result>3.00</b>
|
||||
<p>每次都继续追加；重复 key 会把同一 value 越写越大。</p>
|
||||
</article>
|
||||
<article class="accent">
|
||||
<span>DELTA UPDATE</span>
|
||||
<code>s ← s + β(v − s)</code>
|
||||
<div class="trace" data-delta-trace></div>
|
||||
<b data-delta-result>0.98</b>
|
||||
<p>只写“目标与旧预测的差”；在这个 toy key 上逐步逼近目标。</p>
|
||||
</article>
|
||||
</div>
|
||||
<div class="metrics three">
|
||||
<article><span>ADDITIVE ERROR</span><b data-additive-error>2.00</b><p>|state − target|</p></article>
|
||||
<article><span>DELTA ERROR</span><b data-delta-error>0.02</b><p>|state − target|</p></article>
|
||||
<article class="dark"><span>REAL KDA</span><b>matrix state</b><p>S ∈ R<sup>dₖ×dᵥ</sup>，另有 α 与 key geometry</p></article>
|
||||
</div>
|
||||
<div class="boundary"><b>精确标量递推 / 教学降维</b><p>这个实验只解释“纠错写入”直觉，不证明固定状态能无损回忆任意历史。</p></div>
|
||||
</section>
|
||||
|
||||
<section class="lab-panel" data-k3-panel="decay" hidden>
|
||||
<div class="panel-intro">
|
||||
<div><span>WORKBENCH 02 / LOWER-BOUNDED DECAY</span><h4>关键不是“衰减得慢”，而是让 16-token tile 的最坏 rescale 有有限上界</h4></div>
|
||||
<p>K3 报告固定 g<sub>min</sub>=−5；此时 α>e⁻⁵，16 步累计 log-decay>−80，倒数小于 e⁸⁰。</p>
|
||||
</div>
|
||||
<div class="controls three">
|
||||
<label><span>log-decay 下界 gmin <output data-decay-g-label>−5.0</output></span><input data-decay-g type="range" min="-120" max="-10" value="-50" /></label>
|
||||
<label><span>secondary tile <output data-decay-tile-label>16</output></span><input data-decay-tile type="range" min="1" max="32" value="16" /></label>
|
||||
<label><span>BF16 最大量级</span><input value="≈ 3.39 × 10³⁸" disabled /></label>
|
||||
</div>
|
||||
<div class="decay-plot">
|
||||
<article><span>MIN RETENTION / STEP</span><b data-decay-alpha>6.74 × 10⁻³</b><div><i data-decay-alpha-bar></i></div><p>αmin = exp(gmin)</p></article>
|
||||
<article><span>WORST TILE LOG-DECAY</span><b data-decay-log>−80.0</b><div><i data-decay-log-bar></i></div><p>tile × gmin</p></article>
|
||||
<article class="accent"><span>RECIPROCAL RESCALE</span><b data-decay-recip>≈ 5.54 × 10³⁴</b><div><i data-decay-recip-bar></i></div><p>exp(−tile × gmin)</p></article>
|
||||
</div>
|
||||
<div class="verdict" data-decay-verdict><span>BF16 RANGE</span><b>WITHIN RANGE</b><p>报告默认 −5 × 16 = −80，理论倒数仍低于 BF16 最大有限值。</p></div>
|
||||
<div class="boundary"><b>确定性范围计算 / 非 kernel benchmark</b><p>能落进动态范围不等于精度误差为零；报告的系统收益还来自把 diagonal tile 改成 Tensor Core 稠密矩阵乘。</p></div>
|
||||
</section>
|
||||
|
||||
<section class="lab-panel" data-k3-panel="depth" hidden>
|
||||
<div class="panel-intro">
|
||||
<div><span>WORKBENCH 03 / BLOCK ATTENTION RESIDUALS</span><h4>把“保存每一层”改成“保存每一块”，再看深度来源数量怎样变化</h4></div>
|
||||
<p>计数是确定性的；内存只用“每 Token 每 hidden element 的来源份数”表示，不冒充真实运行时 GiB。</p>
|
||||
</div>
|
||||
<div class="controls three">
|
||||
<label><span>backbone layers L <output data-depth-layers-label>93</output></span><input data-depth-layers type="range" min="24" max="120" value="93" /></label>
|
||||
<label><span>block size S <output data-depth-block-label>12</output></span><input data-depth-block type="range" min="2" max="24" value="12" /></label>
|
||||
<label><span>hidden d <output data-depth-hidden-label>7168</output></span><input data-depth-hidden type="range" min="1024" max="8192" step="512" value="7168" /></label>
|
||||
</div>
|
||||
<div class="depth-track" data-depth-track aria-label="Block AttnRes 教学层条带"></div>
|
||||
<div class="metrics four">
|
||||
<article><span>FULL SOURCES</span><b data-full-sources>94</b><p>L 层输出 + embedding</p></article>
|
||||
<article><span>BLOCK SOURCES</span><b data-block-sources>9</b><p>ceil(L/S) + embedding</p></article>
|
||||
<article><span>SOURCE REDUCTION</span><b data-depth-reduction>90.4%</b><p>来源份数的确定性比较</p></article>
|
||||
<article class="dark"><span>K3 CONTRACT</span><b>12 / 8 / 9</b><p>12 层/块 · 8 个 layer blocks · 含 embedding 共 9 来源</p></article>
|
||||
</div>
|
||||
<div class="boundary"><b>精确计数 / 非吞吐预测</b><p>报告称 Full 的算术因 L<100 尚可，真正代价在存活 activation 和 pipeline 跨 stage 通信。</p></div>
|
||||
</section>
|
||||
|
||||
<section class="lab-panel" data-k3-panel="width" hidden>
|
||||
<div class="panel-intro">
|
||||
<div><span>WORKBENCH 04 / LATENTMOE PAYLOAD</span><h4>激活更多专家时，先缩窄 routed representation 能省下多少 dispatch payload？</h4></div>
|
||||
<p>元素数与字节数按选择精确计算；真实成本还含 router、all-to-all、权重读取、shared experts 与拓扑。</p>
|
||||
</div>
|
||||
<div class="controls five">
|
||||
<label><span>full width d <output data-width-full-label>7168</output></span><input data-width-full type="range" min="1024" max="8192" step="512" value="7168" /></label>
|
||||
<label><span>latent width ℓ <output data-width-latent-label>3584</output></span><input data-width-latent type="range" min="512" max="7168" step="256" value="3584" /></label>
|
||||
<label><span>active experts k <output data-width-topk-label>16</output></span><input data-width-topk type="range" min="1" max="32" value="16" /></label>
|
||||
<label><span>tokens <output data-width-tokens-label>4096</output></span><input data-width-tokens type="range" min="256" max="8192" step="256" value="4096" /></label>
|
||||
<label><span>payload dtype</span><select data-width-bytes><option value="1">1 byte</option><option value="2" selected>2 bytes</option><option value="4">4 bytes</option></select></label>
|
||||
</div>
|
||||
<div class="payload-compare">
|
||||
<article><span>CONVENTIONAL FULL-WIDTH ROUTE</span><b data-width-conventional>896.0 MiB</b><div><i></i></div><p>tokens × k × d × bytes</p></article>
|
||||
<article class="accent"><span>LATENT ROUTE</span><b data-width-latent>448.0 MiB</b><div><i data-width-latent-bar></i></div><p>tokens × k × ℓ × bytes</p></article>
|
||||
</div>
|
||||
<div class="metrics three">
|
||||
<article><span>PAYLOAD REDUCTION</span><b data-width-reduction>50.0%</b><p>只比较 routed token elements</p></article>
|
||||
<article><span>SPARSITY</span><b>896 / 16 = 56×</b><p>K3 expert pool / active routed</p></article>
|
||||
<article class="dark"><span>SHARED PATH</span><b>2 × full width</b><p>公共变换仍要付激活计算</p></article>
|
||||
</div>
|
||||
<div class="boundary"><b>精确 payload 算术 / 非网络流量实测</b><p>实际系统由 MoonEP 静态 shape、zero-copy communication 与专家执行共同决定。</p></div>
|
||||
</section>
|
||||
|
||||
<section class="lab-panel" data-k3-panel="situ" hidden>
|
||||
<div class="panel-intro">
|
||||
<div><span>WORKBENCH 05 / SITU-GLU</span><h4>在原点附近保留 SwiGLU 的形状，在大正输入处给乘积明确上界</h4></div>
|
||||
<p>报告配置 β₁=4、β₂=25，因此标量切片的 |f(x)|≤100。曲线是函数，不是 validation loss。</p>
|
||||
</div>
|
||||
<div class="controls three">
|
||||
<label><span>输入 x <output data-situ-x-label>20.0</output></span><input data-situ-x type="range" min="-100" max="1000" value="200" /></label>
|
||||
<label><span>gate cap β₁ <output data-situ-b1-label>4</output></span><input data-situ-b1 type="range" min="1" max="12" value="4" /></label>
|
||||
<label><span>up cap β₂ <output data-situ-b2-label>25</output></span><input data-situ-b2 type="range" min="5" max="50" value="25" /></label>
|
||||
</div>
|
||||
<div class="curve-stage">
|
||||
<svg viewBox="0 0 720 260" role="img" aria-label="GLU、SwiGLU 与 SiTU-GLU 教学曲线">
|
||||
<line x1="36" y1="224" x2="700" y2="224"></line>
|
||||
<line x1="94" y1="20" x2="94" y2="240"></line>
|
||||
<path data-curve-glu class="curve glu"></path>
|
||||
<path data-curve-swiglu class="curve swiglu"></path>
|
||||
<path data-curve-situ class="curve situ"></path>
|
||||
</svg>
|
||||
<div class="curve-legend"><span class="glu">GLU</span><span class="swiglu">SwiGLU</span><span class="situ">SiTU-GLU</span></div>
|
||||
</div>
|
||||
<div class="metrics four">
|
||||
<article><span>GLU(x)</span><b data-situ-glu>20.00</b><p>σ(x) · x</p></article>
|
||||
<article><span>SWIGLU(x)</span><b data-situ-swiglu>400.00</b><p>[xσ(x)] · x</p></article>
|
||||
<article><span>SITU-GLU(x)</span><b data-situ-value>65.63</b><p>[β₁tanh(x/β₁)σ(x)]·β₂tanh(x/β₂)</p></article>
|
||||
<article class="dark"><span>FORMAL BOUND</span><b data-situ-bound>100</b><p>β₁ × β₂</p></article>
|
||||
</div>
|
||||
<div class="boundary"><b>精确标量函数 / 非完整 FFN</b><p>真实模型的 Wg x 与 Wu x 并不相同；本图复现报告 Figure 4 的共同标量切片。</p></div>
|
||||
</section>
|
||||
|
||||
<section class="lab-panel" data-k3-panel="qb" hidden>
|
||||
<div class="panel-intro">
|
||||
<div><span>WORKBENCH 06 / QUANTILE BALANCING</span><h4>一次 forward 里先照常路由，再用“进入 Top-k 还差多少”决定下一步 bias</h4></div>
|
||||
<p>固定使用报告 Figure 5 的 m=8、n=4、k=1 规模；score 是本站构造，不是 K3 trace。</p>
|
||||
</div>
|
||||
<div class="controls two">
|
||||
<label><span>bias 更新强度 <output data-qb-strength-label>1.00</output></span><input data-qb-strength type="range" min="0" max="150" value="100" /></label>
|
||||
<label><span>histogram bins <output data-qb-bins-label>64</output></span><input data-qb-bins type="range" min="4" max="128" step="4" value="64" /></label>
|
||||
</div>
|
||||
<div class="qb-stage">
|
||||
<article><span>CURRENT TOP-1</span><div class="load-bars" data-qb-before></div><p data-qb-before-copy>loads = (4, 3, 1, 0)</p></article>
|
||||
<i>→</i>
|
||||
<article class="quantile-box"><span>TOP-2 CUTOFF</span><b>margin = score − α</b><p>每个 expert 取目标 q=mk/n=2 对应的分位阈值，再移除公共 offset。</p></article>
|
||||
<i>→</i>
|
||||
<article class="accent"><span>NEXT-STEP TOP-1</span><div class="load-bars" data-qb-after></div><p data-qb-after-copy>loads = (2, 2, 2, 2)</p></article>
|
||||
</div>
|
||||
<div class="metrics four">
|
||||
<article><span>TARGET / EXPERT</span><b>q = 2</b><p>8 tokens × top-1 / 4 experts</p></article>
|
||||
<article><span>BIAS VECTOR</span><b data-qb-bias>—</b><p>居中后用于 next step</p></article>
|
||||
<article><span>MAX LOAD GAP</span><b data-qb-gap>0</b><p>after max − min</p></article>
|
||||
<article class="dark"><span>HIST ERROR</span><b data-qb-error>≤ 1/64 range</b><p>通信近似只保证 bin-width 级</p></article>
|
||||
</div>
|
||||
<div class="boundary"><b>公式级 toy router / 非训练轨迹</b><p>bias 只影响 Top-k selection，不进入最终 mixture weight；当前 batch 不能使用由自身算出的新 bias。</p></div>
|
||||
</section>
|
||||
|
||||
<section class="lab-panel" data-k3-panel="rl" hidden>
|
||||
<div class="panel-intro">
|
||||
<div><span>WORKBENCH 07 / EFFORT × PARTIAL ROLLOUT × MOPD</span><h4>长程 RL 同时要记三本账：每题预算、完成阈值、教师密集信号</h4></div>
|
||||
<p>三块标量计算互不替代：budget 管过度思考，λ 管 straggler，MOPD reward 管学生当前 prefix。</p>
|
||||
</div>
|
||||
<div class="controls five">
|
||||
<label><span>prompts N <output data-rl-n-label>32</output></span><input data-rl-n type="range" min="4" max="64" step="4" value="32" /></label>
|
||||
<label><span>rollouts K <output data-rl-k-label>8</output></span><input data-rl-k type="range" min="1" max="16" value="8" /></label>
|
||||
<label><span>pause fraction λ <output data-rl-lambda-label>0.75</output></span><input data-rl-lambda type="range" min="10" max="100" value="75" /></label>
|
||||
<label><span>base budget b₀ <output data-rl-base-label>8K</output></span><input data-rl-base type="range" min="2" max="32" value="8" /></label>
|
||||
<label><span>effort multiplier τ <output data-rl-tau-label>2.0</output></span><input data-rl-tau type="range" min="5" max="50" value="20" /></label>
|
||||
</div>
|
||||
<div class="controls four">
|
||||
<label><span>trajectory tokens T(y) <output data-rl-tokens-label>12K</output></span><input data-rl-tokens type="range" min="1" max="96" value="12" /></label>
|
||||
<label><span>teacher p(yt) <output data-rl-teacher-label>0.60</output></span><input data-rl-teacher type="range" min="1" max="99" value="60" /></label>
|
||||
<label><span>student p(yt) <output data-rl-student-label>0.30</output></span><input data-rl-student type="range" min="1" max="99" value="30" /></label>
|
||||
<label><span>clip Rmax <output data-rl-clip-label>1.0</output></span><input data-rl-clip type="range" min="1" max="30" value="10" /></label>
|
||||
</div>
|
||||
<div class="rl-flow">
|
||||
<article><span>EFFORT BUDGET</span><b data-rl-budget>16K</b><p data-rl-budget-copy>12K ≤ 2.0 × 8K：保留任务 reward</p></article>
|
||||
<i>→</i>
|
||||
<article><span>PARTIAL ROLLOUT</span><b data-rl-complete>192 complete</b><p data-rl-paused>64 paused / resume next iteration</p></article>
|
||||
<i>→</i>
|
||||
<article class="accent"><span>MOPD TOKEN REWARD</span><b data-rl-reward>+0.693</b><p>clip(log pteacher/pstudent, ±Rmax)</p></article>
|
||||
</div>
|
||||
<div class="boundary"><b>报告公式的标量例子 / 非 policy optimizer</b><p>实际系统还有 per-token regularization、domain/effort teacher selection、sandbox state 与 group completion dispatch。</p></div>
|
||||
</section>
|
||||
|
||||
<section class="lab-panel" data-k3-panel="cache" hidden>
|
||||
<div class="panel-intro">
|
||||
<div><span>WORKBENCH 08 / HYBRID PREFIX CACHE</span><h4>MLA prefix 对上了还不够：同一边界必须有每个 KDA group 的 state checkpoint</h4></div>
|
||||
<p>默认复现报告 Figure 12 的 6144-token physical block 与 512-token hash block；checkpoint 周期是教学变量。</p>
|
||||
</div>
|
||||
<div class="controls four">
|
||||
<label><span>matched prefix <output data-cache-matched-label>2800</output></span><input data-cache-matched type="range" min="0" max="12288" step="128" value="2816" /></label>
|
||||
<label><span>physical block <output data-cache-physical-label>6144</output></span><input data-cache-physical type="range" min="2048" max="8192" step="512" value="6144" /></label>
|
||||
<label><span>hash block <output data-cache-hash-label>512</output></span><input data-cache-hash type="range" min="128" max="1024" step="128" value="512" /></label>
|
||||
<label><span>KDA checkpoint / hash blocks <output data-cache-checkpoint-label>1</output></span><input data-cache-checkpoint type="range" min="1" max="12" value="1" /></label>
|
||||
</div>
|
||||
<div class="cache-strip" data-cache-strip aria-label="混合 KDA MLA prefix cache 条带"></div>
|
||||
<div class="metrics four">
|
||||
<article><span>MLA HASH MATCH</span><b data-cache-mla>2560</b><p>floor(match/hash) × hash</p></article>
|
||||
<article><span>KDA CHECKPOINT</span><b data-cache-kda>2560</b><p>最长同时存在的 state 边界</p></article>
|
||||
<article><span>JOINT HIT</span><b data-cache-hit>2560</b><p>两类 cache 的共同最长前缀</p></article>
|
||||
<article class="dark"><span>RECOMPUTE</span><b data-cache-recompute>256 tokens</b><p>matched − joint hit</p></article>
|
||||
</div>
|
||||
<div class="boundary"><b>精确边界算术 / 非线上 hit rate</b><p>真实系统还需 pin、copy-on-write、跨 KDA group 原子失效、调度 step 可见性和 prefill/decode TP re-layout。</p></div>
|
||||
</section>
|
||||
</figure>
|
||||
|
||||
<script>
|
||||
const roots = document.querySelectorAll<HTMLElement>("[data-k3-lab]");
|
||||
const fmt = new Intl.NumberFormat("en-US", { maximumFractionDigits: 2 });
|
||||
|
||||
roots.forEach((root) => {
|
||||
const $ = <T extends HTMLElement = HTMLElement>(selector: string) => root.querySelector<T>(selector)!;
|
||||
const $$ = <T extends HTMLElement = HTMLElement>(selector: string) => [...root.querySelectorAll<T>(selector)];
|
||||
const value = (selector: string) => Number(root.querySelector<HTMLInputElement | HTMLSelectElement>(selector)?.value ?? 0);
|
||||
const put = (selector: string, content: string | number) => {
|
||||
const node = $(selector);
|
||||
if (node) node.textContent = String(content);
|
||||
};
|
||||
|
||||
const tabs = $$<HTMLButtonElement>("[data-k3-tab]");
|
||||
const panels = $$<HTMLElement>("[data-k3-panel]");
|
||||
const selectTab = (id: string) => {
|
||||
tabs.forEach((tab) => {
|
||||
const active = tab.dataset.k3Tab === id;
|
||||
tab.setAttribute("aria-selected", String(active));
|
||||
tab.tabIndex = active ? 0 : -1;
|
||||
});
|
||||
panels.forEach((panel) => {
|
||||
panel.hidden = panel.dataset.k3Panel !== id;
|
||||
});
|
||||
};
|
||||
tabs.forEach((tab, index) => {
|
||||
tab.addEventListener("click", () => selectTab(tab.dataset.k3Tab ?? "memory"));
|
||||
tab.addEventListener("keydown", (event: KeyboardEvent) => {
|
||||
if (!["ArrowLeft", "ArrowRight", "Home", "End"].includes(event.key)) return;
|
||||
event.preventDefault();
|
||||
let next = index;
|
||||
if (event.key === "ArrowLeft") next = (index - 1 + tabs.length) % tabs.length;
|
||||
if (event.key === "ArrowRight") next = (index + 1) % tabs.length;
|
||||
if (event.key === "Home") next = 0;
|
||||
if (event.key === "End") next = tabs.length - 1;
|
||||
selectTab(tabs[next].dataset.k3Tab ?? "memory");
|
||||
tabs[next].focus();
|
||||
});
|
||||
});
|
||||
|
||||
const formatSigned = (n: number, digits = 2) => `${n >= 0 ? "+" : ""}${n.toFixed(digits)}`;
|
||||
const formatBytes = (bytes: number) => {
|
||||
if (bytes >= 2 ** 30) return `${(bytes / 2 ** 30).toFixed(2)} GiB`;
|
||||
if (bytes >= 2 ** 20) return `${(bytes / 2 ** 20).toFixed(1)} MiB`;
|
||||
if (bytes >= 2 ** 10) return `${(bytes / 2 ** 10).toFixed(1)} KiB`;
|
||||
return `${fmt.format(bytes)} B`;
|
||||
};
|
||||
const sigmoid = (x: number) => 1 / (1 + Math.exp(-x));
|
||||
|
||||
const renderMemory = () => {
|
||||
const writes = value("[data-memory-writes]");
|
||||
const beta = value("[data-memory-beta]") / 100;
|
||||
const initial = value("[data-memory-old]") / 100;
|
||||
const target = value("[data-memory-target]") / 100;
|
||||
let additive = initial;
|
||||
let delta = initial;
|
||||
const additiveTrace = [additive];
|
||||
const deltaTrace = [delta];
|
||||
for (let i = 0; i < writes; i += 1) {
|
||||
additive += beta * target;
|
||||
delta += beta * (target - delta);
|
||||
additiveTrace.push(additive);
|
||||
deltaTrace.push(delta);
|
||||
}
|
||||
put("[data-memory-writes-label]", writes);
|
||||
put("[data-memory-beta-label]", beta.toFixed(2));
|
||||
put("[data-memory-old-label]", initial.toFixed(2));
|
||||
put("[data-memory-target-label]", target.toFixed(2));
|
||||
put("[data-additive-result]", additive.toFixed(3));
|
||||
put("[data-delta-result]", delta.toFixed(3));
|
||||
put("[data-additive-error]", Math.abs(additive - target).toFixed(3));
|
||||
put("[data-delta-error]", Math.abs(delta - target).toFixed(3));
|
||||
const draw = (selector: string, trace: number[], colorClass: string) => {
|
||||
const maxAbs = Math.max(1, ...trace.map((v) => Math.abs(v)));
|
||||
$(selector).innerHTML = trace.map((v, i) =>
|
||||
`<i class="${colorClass}" style="height:${Math.max(4, Math.abs(v) / maxAbs * 62)}px" title="step ${i}: ${v.toFixed(3)}"></i>`
|
||||
).join("");
|
||||
};
|
||||
draw("[data-additive-trace]", additiveTrace, "plain");
|
||||
draw("[data-delta-trace]", deltaTrace, "accent");
|
||||
};
|
||||
|
||||
const renderDecay = () => {
|
||||
const g = value("[data-decay-g]") / 10;
|
||||
const tile = value("[data-decay-tile]");
|
||||
const alpha = Math.exp(g);
|
||||
const logDecay = g * tile;
|
||||
const log10Recip = -logDecay / Math.LN10;
|
||||
const bf16Log10 = Math.log10(3.3895313892515355e38);
|
||||
const within = log10Recip <= bf16Log10;
|
||||
put("[data-decay-g-label]", g.toFixed(1).replace("-", "−"));
|
||||
put("[data-decay-tile-label]", tile);
|
||||
put("[data-decay-alpha]", alpha < .001 ? alpha.toExponential(2) : alpha.toPrecision(3));
|
||||
put("[data-decay-log]", logDecay.toFixed(1).replace("-", "−"));
|
||||
put("[data-decay-recip]", `≈ 10^${log10Recip.toFixed(2)}`);
|
||||
$("[data-decay-alpha-bar]").style.width = `${Math.max(2, Math.min(100, alpha * 100))}%`;
|
||||
$("[data-decay-log-bar]").style.width = `${Math.min(100, Math.abs(logDecay) / 120 * 100)}%`;
|
||||
$("[data-decay-recip-bar]").style.width = `${Math.min(100, log10Recip / bf16Log10 * 100)}%`;
|
||||
const verdict = $("[data-decay-verdict]");
|
||||
verdict.classList.toggle("danger", !within);
|
||||
verdict.querySelector<HTMLElement>("b")!.textContent = within ? "WITHIN RANGE" : "OVERFLOW RISK";
|
||||
verdict.querySelector<HTMLElement>("p")!.textContent = within
|
||||
? `最坏倒数约 10^${log10Recip.toFixed(2)}，低于 BF16 最大有限量级。`
|
||||
: `最坏倒数约 10^${log10Recip.toFixed(2)}，超过 BF16 最大有限量级。`;
|
||||
};
|
||||
|
||||
const renderDepth = () => {
|
||||
const layers = value("[data-depth-layers]");
|
||||
const size = value("[data-depth-block]");
|
||||
const hidden = value("[data-depth-hidden]");
|
||||
const blocks = Math.ceil(layers / size);
|
||||
const full = layers + 1;
|
||||
const grouped = blocks + 1;
|
||||
put("[data-depth-layers-label]", layers);
|
||||
put("[data-depth-block-label]", size);
|
||||
put("[data-depth-hidden-label]", hidden);
|
||||
put("[data-full-sources]", full);
|
||||
put("[data-block-sources]", grouped);
|
||||
put("[data-depth-reduction]", `${((1 - grouped / full) * 100).toFixed(1)}%`);
|
||||
const track = $("[data-depth-track]");
|
||||
track.innerHTML = Array.from({ length: blocks }, (_, index) => {
|
||||
const count = Math.min(size, layers - index * size);
|
||||
return `<div style="flex:${count}" title="block ${index + 1}: ${count} layers"><span>B${index + 1}</span><b>${count}</b></div>`;
|
||||
}).join("");
|
||||
};
|
||||
|
||||
const renderWidth = () => {
|
||||
const full = value("[data-width-full]");
|
||||
const latent = Math.min(value("[data-width-latent]"), full);
|
||||
const topk = value("[data-width-topk]");
|
||||
const tokens = value("[data-width-tokens]");
|
||||
const bytes = value("[data-width-bytes]");
|
||||
const conventional = tokens * topk * full * bytes;
|
||||
const compressed = tokens * topk * latent * bytes;
|
||||
put("[data-width-full-label]", full);
|
||||
put("[data-width-latent-label]", latent);
|
||||
put("[data-width-topk-label]", topk);
|
||||
put("[data-width-tokens-label]", tokens);
|
||||
put("[data-width-conventional]", formatBytes(conventional));
|
||||
put("[data-width-latent]", formatBytes(compressed));
|
||||
put("[data-width-reduction]", `${((1 - compressed / conventional) * 100).toFixed(1)}%`);
|
||||
$("[data-width-latent-bar]").style.width = `${latent / full * 100}%`;
|
||||
};
|
||||
|
||||
const situ = (x: number, b1: number, b2: number) => b1 * Math.tanh(x / b1) * sigmoid(x) * b2 * Math.tanh(x / b2);
|
||||
const renderSitu = () => {
|
||||
const x = value("[data-situ-x]") / 10;
|
||||
const b1 = value("[data-situ-b1]");
|
||||
const b2 = value("[data-situ-b2]");
|
||||
const glu = sigmoid(x) * x;
|
||||
const swiglu = x * sigmoid(x) * x;
|
||||
put("[data-situ-x-label]", x.toFixed(1));
|
||||
put("[data-situ-b1-label]", b1);
|
||||
put("[data-situ-b2-label]", b2);
|
||||
put("[data-situ-glu]", glu.toFixed(2));
|
||||
put("[data-situ-swiglu]", swiglu.toFixed(2));
|
||||
put("[data-situ-value]", situ(x, b1, b2).toFixed(2));
|
||||
put("[data-situ-bound]", b1 * b2);
|
||||
const ranges = Array.from({ length: 121 }, (_, i) => -10 + i * (110 / 120));
|
||||
const mapX = (v: number) => 36 + (v + 10) / 110 * 664;
|
||||
const mapY = (v: number) => 224 - Math.max(-20, Math.min(120, v)) / 140 * 190;
|
||||
const path = (fn: (input: number) => number) => ranges.map((v, i) => `${i ? "L" : "M"}${mapX(v).toFixed(1)},${mapY(fn(v)).toFixed(1)}`).join(" ");
|
||||
$("[data-curve-glu]").setAttribute("d", path((v) => sigmoid(v) * v));
|
||||
$("[data-curve-swiglu]").setAttribute("d", path((v) => v * sigmoid(v) * v));
|
||||
$("[data-curve-situ]").setAttribute("d", path((v) => situ(v, b1, b2)));
|
||||
};
|
||||
|
||||
const scores = [
|
||||
[.58, .46, .77, .21],
|
||||
[.61, .91, .71, .90],
|
||||
[.81, .59, .13, .43],
|
||||
[.71, .30, .64, .30],
|
||||
[.83, .23, .52, .70],
|
||||
[.44, .74, .64, .33],
|
||||
[.81, .69, .42, .15],
|
||||
[.75, .84, .56, .64],
|
||||
];
|
||||
const topIndex = (row: number[]) => row.reduce((best, score, index) => score > row[best] ? index : best, 0);
|
||||
const loadsFor = (bias: number[]) => {
|
||||
const loads = [0, 0, 0, 0];
|
||||
scores.forEach((row) => {
|
||||
const adjusted = row.map((score, index) => score + bias[index]);
|
||||
loads[topIndex(adjusted)] += 1;
|
||||
});
|
||||
return loads;
|
||||
};
|
||||
const renderLoadBars = (selector: string, loads: number[]) => {
|
||||
const max = Math.max(1, ...loads);
|
||||
$(selector).innerHTML = loads.map((load, index) =>
|
||||
`<div><i style="height:${load / max * 80}px"></i><b>E${index + 1}</b><span>${load}</span></div>`
|
||||
).join("");
|
||||
};
|
||||
const renderQB = () => {
|
||||
const strength = value("[data-qb-strength]") / 100;
|
||||
const bins = value("[data-qb-bins]");
|
||||
const before = loadsFor([0, 0, 0, 0]);
|
||||
const margins: number[][] = [[], [], [], []];
|
||||
scores.forEach((row) => {
|
||||
const sorted = [...row].sort((a, b) => b - a);
|
||||
const cutoff = sorted[1];
|
||||
row.forEach((score, expert) => margins[expert].push(score - cutoff));
|
||||
});
|
||||
const rawBias = margins.map((values) => {
|
||||
const ordered = [...values].sort((a, b) => b - a);
|
||||
return -ordered[2];
|
||||
});
|
||||
const mean = rawBias.reduce((sum, n) => sum + n, 0) / rawBias.length;
|
||||
const centered = rawBias.map((n) => (n - mean) * strength);
|
||||
const after = loadsFor(centered);
|
||||
put("[data-qb-strength-label]", strength.toFixed(2));
|
||||
put("[data-qb-bins-label]", bins);
|
||||
put("[data-qb-before-copy]", `loads = (${before.join(", ")})`);
|
||||
put("[data-qb-after-copy]", `loads = (${after.join(", ")})`);
|
||||
put("[data-qb-bias]", centered.map((n) => formatSigned(n, 2)).join(" · "));
|
||||
put("[data-qb-gap]", Math.max(...after) - Math.min(...after));
|
||||
put("[data-qb-error]", `≤ 1/${bins} range`);
|
||||
renderLoadBars("[data-qb-before]", before);
|
||||
renderLoadBars("[data-qb-after]", after);
|
||||
};
|
||||
|
||||
const renderRL = () => {
|
||||
const n = value("[data-rl-n]");
|
||||
const k = value("[data-rl-k]");
|
||||
const lambda = value("[data-rl-lambda]") / 100;
|
||||
const base = value("[data-rl-base]");
|
||||
const tau = value("[data-rl-tau]") / 10;
|
||||
const tokens = value("[data-rl-tokens]");
|
||||
const teacher = value("[data-rl-teacher]") / 100;
|
||||
const student = value("[data-rl-student]") / 100;
|
||||
const clip = value("[data-rl-clip]") / 10;
|
||||
const total = n * k;
|
||||
const complete = Math.ceil(total * lambda);
|
||||
const budget = base * tau;
|
||||
const reward = Math.max(-clip, Math.min(clip, Math.log(teacher / student)));
|
||||
const pass = tokens <= budget;
|
||||
put("[data-rl-n-label]", n);
|
||||
put("[data-rl-k-label]", k);
|
||||
put("[data-rl-lambda-label]", lambda.toFixed(2));
|
||||
put("[data-rl-base-label]", `${base}K`);
|
||||
put("[data-rl-tau-label]", tau.toFixed(1));
|
||||
put("[data-rl-tokens-label]", `${tokens}K`);
|
||||
put("[data-rl-teacher-label]", teacher.toFixed(2));
|
||||
put("[data-rl-student-label]", student.toFixed(2));
|
||||
put("[data-rl-clip-label]", clip.toFixed(1));
|
||||
put("[data-rl-budget]", `${budget.toFixed(1).replace(".0", "")}K`);
|
||||
put("[data-rl-budget-copy]", `${tokens}K ${pass ? "≤" : ">"} ${tau.toFixed(1)} × ${base}K：${pass ? "保留任务 reward" : "reward 改为 −1"}`);
|
||||
put("[data-rl-complete]", `${complete} complete`);
|
||||
put("[data-rl-paused]", `${total - complete} paused / resume next iteration`);
|
||||
put("[data-rl-reward]", formatSigned(reward, 3));
|
||||
};
|
||||
|
||||
const renderCache = () => {
|
||||
const matched = value("[data-cache-matched]");
|
||||
const physical = value("[data-cache-physical]");
|
||||
const hash = Math.min(value("[data-cache-hash]"), physical);
|
||||
const checkpointEvery = value("[data-cache-checkpoint]");
|
||||
const mla = Math.floor(matched / hash) * hash;
|
||||
const checkpointSpan = hash * checkpointEvery;
|
||||
const kda = Math.floor(mla / checkpointSpan) * checkpointSpan;
|
||||
const hit = Math.min(mla, kda);
|
||||
put("[data-cache-matched-label]", matched);
|
||||
put("[data-cache-physical-label]", physical);
|
||||
put("[data-cache-hash-label]", hash);
|
||||
put("[data-cache-checkpoint-label]", checkpointEvery);
|
||||
put("[data-cache-mla]", mla);
|
||||
put("[data-cache-kda]", kda);
|
||||
put("[data-cache-hit]", hit);
|
||||
put("[data-cache-recompute]", `${matched - hit} tokens`);
|
||||
const slots = Math.max(4, Math.ceil(Math.min(physical * 2, Math.max(matched, physical)) / hash));
|
||||
$("[data-cache-strip]").innerHTML = Array.from({ length: slots }, (_, index) => {
|
||||
const end = (index + 1) * hash;
|
||||
const matchedBlock = end <= mla;
|
||||
const checkpoint = end % checkpointSpan === 0;
|
||||
const joint = end <= hit;
|
||||
return `<div class="${joint ? "joint" : matchedBlock ? "mla" : ""}">
|
||||
<span>${index + 1}</span><i></i><b>${end}</b>${checkpoint ? "<em title=\"KDA checkpoint\">●</em>" : "<em>○</em>"}
|
||||
</div>`;
|
||||
}).join("");
|
||||
};
|
||||
|
||||
const renders = [renderMemory, renderDecay, renderDepth, renderWidth, renderSitu, renderQB, renderRL, renderCache];
|
||||
$$<HTMLInputElement | HTMLSelectElement>("input, select").forEach((control) => control.addEventListener("input", () => renders.forEach((render) => render())));
|
||||
renders.forEach((render) => render());
|
||||
});
|
||||
</script>
|
||||
|
||||
<style>
|
||||
.k3-lab {
|
||||
max-width: 1080px;
|
||||
margin: 42px 0;
|
||||
border: 1px solid var(--ink);
|
||||
background: var(--paper-raised);
|
||||
}
|
||||
.lab-head, .panel-intro {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(0, 1.1fr) minmax(260px, .9fr);
|
||||
gap: 30px;
|
||||
padding: 28px;
|
||||
border-bottom: 1px solid var(--line);
|
||||
}
|
||||
.lab-head { color: var(--paper); background: var(--ink); }
|
||||
.lab-head p, .panel-intro p { margin: 0; font-size: .72rem; line-height: 1.75; }
|
||||
.lab-head > div > p, .panel-intro span {
|
||||
color: var(--copper);
|
||||
font: .6rem/1.2 var(--mono);
|
||||
letter-spacing: .12em;
|
||||
}
|
||||
.lab-head h3, .panel-intro h4 { margin: 10px 0 0; font-size: 1.25rem; line-height: 1.35; }
|
||||
.lab-head h3 { color: var(--paper); }
|
||||
.lab-head > p { color: rgba(255,255,255,.65); }
|
||||
.lab-tabs {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(4, 1fr);
|
||||
border-bottom: 1px solid var(--line);
|
||||
}
|
||||
.lab-tabs button {
|
||||
min-height: 78px;
|
||||
padding: 13px 15px;
|
||||
border: 0;
|
||||
border-right: 1px solid var(--line);
|
||||
border-bottom: 1px solid var(--line);
|
||||
color: var(--ink);
|
||||
background: transparent;
|
||||
text-align: left;
|
||||
cursor: pointer;
|
||||
}
|
||||
.lab-tabs button:nth-child(4n) { border-right: 0; }
|
||||
.lab-tabs button:nth-child(n+5) { border-bottom: 0; }
|
||||
.lab-tabs button[aria-selected="true"] { color: var(--paper); background: var(--copper); }
|
||||
.lab-tabs span, .lab-tabs b, .lab-tabs small { display: block; }
|
||||
.lab-tabs span { opacity: .7; font: .55rem/1 var(--mono); }
|
||||
.lab-tabs b { margin-top: 8px; font: 700 .67rem/1 var(--mono); letter-spacing: .06em; }
|
||||
.lab-tabs small { margin-top: 5px; opacity: .72; font-size: .61rem; }
|
||||
.lab-panel { padding-bottom: 26px; }
|
||||
.panel-intro { padding: 24px 28px; }
|
||||
.panel-intro > p { color: var(--muted); }
|
||||
.controls {
|
||||
display: grid;
|
||||
gap: 0;
|
||||
margin: 0 28px 26px;
|
||||
border: 1px solid var(--line);
|
||||
}
|
||||
.controls.two { grid-template-columns: repeat(2, 1fr); }
|
||||
.controls.three { grid-template-columns: repeat(3, 1fr); }
|
||||
.controls.four { grid-template-columns: repeat(4, 1fr); }
|
||||
.controls.five { grid-template-columns: repeat(5, 1fr); }
|
||||
.controls label {
|
||||
min-width: 0;
|
||||
padding: 14px;
|
||||
border-right: 1px solid var(--line);
|
||||
color: var(--muted);
|
||||
font: .62rem/1.35 var(--mono);
|
||||
}
|
||||
.controls label:last-child { border-right: 0; }
|
||||
.controls span { display: flex; justify-content: space-between; gap: 8px; min-height: 28px; }
|
||||
.controls output { color: var(--copper); }
|
||||
.controls input, .controls select { width: 100%; margin-top: 11px; accent-color: var(--copper); }
|
||||
.controls input[disabled] { padding: 5px; border: 1px solid var(--line); color: var(--ink); background: var(--paper); font: .65rem var(--mono); }
|
||||
.memory-stage, .decay-plot, .payload-compare, .qb-stage, .rl-flow {
|
||||
display: grid;
|
||||
gap: 14px;
|
||||
margin: 0 28px 26px;
|
||||
}
|
||||
.memory-stage, .payload-compare { grid-template-columns: repeat(2, 1fr); }
|
||||
.decay-plot { grid-template-columns: repeat(3, 1fr); }
|
||||
.qb-stage, .rl-flow { grid-template-columns: 1fr 24px 1fr 24px 1fr; align-items: center; }
|
||||
.memory-stage article, .decay-plot article, .payload-compare article, .qb-stage article, .rl-flow article {
|
||||
min-width: 0;
|
||||
padding: 20px;
|
||||
border: 1px solid var(--line);
|
||||
background: var(--paper);
|
||||
}
|
||||
.accent { border-color: rgba(173,100,69,.55) !important; background: var(--copper-pale) !important; }
|
||||
.memory-stage span, .decay-plot span, .payload-compare span, .qb-stage span, .rl-flow span {
|
||||
color: var(--muted);
|
||||
font: .58rem/1 var(--mono);
|
||||
letter-spacing: .08em;
|
||||
}
|
||||
.memory-stage code { display: block; margin: 13px 0; font-size: .72rem; }
|
||||
.memory-stage b, .decay-plot b, .payload-compare b, .rl-flow b { display: block; margin: 14px 0 7px; font: 700 1.05rem/1 var(--mono); }
|
||||
.memory-stage p, .decay-plot p, .payload-compare p, .qb-stage p, .rl-flow p { margin: 7px 0 0; color: var(--muted); font-size: .68rem; line-height: 1.55; }
|
||||
.trace { display: flex; height: 68px; gap: 3px; align-items: flex-end; border-bottom: 1px solid var(--line); }
|
||||
.trace i { flex: 1; min-width: 2px; background: var(--sage); }
|
||||
.trace i.accent { background: var(--copper) !important; }
|
||||
.metrics {
|
||||
display: grid;
|
||||
margin: 0 28px 24px;
|
||||
border: 1px solid var(--line);
|
||||
}
|
||||
.metrics.three { grid-template-columns: repeat(3, 1fr); }
|
||||
.metrics.four { grid-template-columns: repeat(4, 1fr); }
|
||||
.metrics article { min-width: 0; min-height: 112px; padding: 17px; border-right: 1px solid var(--line); }
|
||||
.metrics article:last-child { border-right: 0; }
|
||||
.metrics span { color: var(--muted); font: .55rem/1 var(--mono); letter-spacing: .08em; }
|
||||
.metrics b { display: block; margin-top: 18px; overflow-wrap: anywhere; color: var(--copper); font: 700 .92rem/1.2 var(--mono); }
|
||||
.metrics p { margin: 8px 0 0; color: var(--muted); font-size: .62rem; line-height: 1.5; }
|
||||
.metrics .dark { color: var(--paper); background: var(--ink); }
|
||||
.metrics .dark span, .metrics .dark p { color: rgba(255,255,255,.55); }
|
||||
.metrics .dark b { color: var(--paper); }
|
||||
.boundary {
|
||||
margin: 0 28px;
|
||||
padding: 15px 17px;
|
||||
border-left: 3px solid var(--sage);
|
||||
background: var(--sage-pale);
|
||||
}
|
||||
.boundary b { font: 700 .62rem/1 var(--mono); letter-spacing: .06em; }
|
||||
.boundary p { margin: 7px 0 0; color: var(--muted); font-size: .67rem; line-height: 1.55; }
|
||||
.decay-plot article > div, .payload-compare article > div { height: 7px; margin-top: 18px; overflow: hidden; background: var(--line); }
|
||||
.decay-plot i, .payload-compare i { display: block; width: 100%; height: 100%; background: var(--sage); }
|
||||
.decay-plot article.accent i, .payload-compare article.accent i { background: var(--copper); }
|
||||
.verdict {
|
||||
display: grid;
|
||||
grid-template-columns: .6fr .8fr 1.6fr;
|
||||
gap: 20px;
|
||||
align-items: center;
|
||||
margin: 0 28px 24px;
|
||||
padding: 18px;
|
||||
color: var(--paper);
|
||||
background: var(--sage);
|
||||
}
|
||||
.verdict.danger { background: #8d3f35; }
|
||||
.verdict span { font: .58rem var(--mono); opacity: .7; }
|
||||
.verdict b { font: 700 .9rem var(--mono); }
|
||||
.verdict p { margin: 0; font-size: .67rem; line-height: 1.55; }
|
||||
.depth-track {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
min-height: 88px;
|
||||
margin: 0 28px 26px;
|
||||
align-items: stretch;
|
||||
}
|
||||
.depth-track div { display: flex; min-width: 24px; padding: 10px 4px; flex-direction: column; justify-content: space-between; border: 1px solid var(--line); background: var(--sage-pale); text-align: center; }
|
||||
.depth-track span { color: var(--muted); font: .51rem var(--mono); }
|
||||
.depth-track b { color: var(--sage); font: 700 .75rem var(--mono); }
|
||||
.curve-stage { position: relative; margin: 0 28px 24px; border: 1px solid var(--line); background: var(--paper); }
|
||||
.curve-stage svg { display: block; width: 100%; }
|
||||
.curve-stage line { stroke: var(--line); stroke-width: 1; }
|
||||
.curve { fill: none; stroke-width: 2.4; }
|
||||
.curve.glu { stroke: var(--sage); }
|
||||
.curve.swiglu { stroke: var(--ink); }
|
||||
.curve.situ { stroke: var(--copper); stroke-width: 3.2; }
|
||||
.curve-legend { position: absolute; top: 13px; right: 14px; display: flex; gap: 12px; padding: 7px 9px; background: rgba(255,255,255,.84); }
|
||||
.curve-legend span { font: .55rem var(--mono); }
|
||||
.curve-legend .glu { color: var(--sage); }
|
||||
.curve-legend .situ { color: var(--copper); }
|
||||
.qb-stage > i, .rl-flow > i { color: var(--muted-light); font-style: normal; text-align: center; }
|
||||
.quantile-box b { display: block; margin: 14px 0; font: .7rem var(--mono); }
|
||||
.load-bars { display: flex; height: 104px; gap: 10px; align-items: flex-end; margin-top: 13px; }
|
||||
.load-bars div { display: flex; flex: 1; min-width: 0; height: 100%; flex-direction: column; align-items: center; justify-content: flex-end; }
|
||||
.load-bars i { width: 70%; min-height: 3px; background: var(--sage); }
|
||||
.accent .load-bars i { background: var(--copper); }
|
||||
.load-bars b, .load-bars span { margin-top: 5px; font: .55rem var(--mono); }
|
||||
.cache-strip {
|
||||
display: flex;
|
||||
gap: 3px;
|
||||
margin: 0 28px 26px;
|
||||
overflow-x: auto;
|
||||
padding-bottom: 8px;
|
||||
}
|
||||
.cache-strip div { position: relative; flex: 1 0 48px; min-width: 48px; padding: 9px 4px 7px; border: 1px solid var(--line); text-align: center; background: var(--paper); }
|
||||
.cache-strip div.mla { background: #e8efed; }
|
||||
.cache-strip div.joint { border-color: var(--copper); background: var(--copper-pale); }
|
||||
.cache-strip span, .cache-strip b { display: block; color: var(--muted); font: .49rem var(--mono); }
|
||||
.cache-strip i { display: block; height: 20px; margin: 6px 0; background: var(--line); }
|
||||
.cache-strip .mla i { background: var(--sage); }
|
||||
.cache-strip .joint i { background: var(--copper); }
|
||||
.cache-strip em { position: absolute; right: 3px; bottom: 2px; color: var(--copper); font-style: normal; font-size: .55rem; }
|
||||
@media (max-width: 900px) {
|
||||
.lab-head, .panel-intro { grid-template-columns: 1fr; }
|
||||
.controls.five { grid-template-columns: repeat(3, 1fr); }
|
||||
.metrics.four { grid-template-columns: repeat(2, 1fr); }
|
||||
.qb-stage, .rl-flow { grid-template-columns: 1fr; }
|
||||
.qb-stage > i, .rl-flow > i { transform: rotate(90deg); }
|
||||
}
|
||||
@media (max-width: 640px) {
|
||||
.lab-tabs { grid-template-columns: repeat(2, 1fr); }
|
||||
.lab-tabs button:nth-child(2n) { border-right: 0; }
|
||||
.lab-tabs button:nth-child(n+5) { border-bottom: 1px solid var(--line); }
|
||||
.lab-tabs button:nth-child(n+7) { border-bottom: 0; }
|
||||
.controls.two, .controls.three, .controls.four, .controls.five,
|
||||
.memory-stage, .decay-plot, .payload-compare, .metrics.three, .metrics.four {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
.controls label, .metrics article { border-right: 0; border-bottom: 1px solid var(--line); }
|
||||
.controls label:last-child, .metrics article:last-child { border-bottom: 0; }
|
||||
.verdict { grid-template-columns: 1fr; }
|
||||
}
|
||||
</style>
|
||||
+173
@@ -0,0 +1,173 @@
|
||||
export const k3Ledgers = [
|
||||
["Q01", "参数角色", "2.78T / 104.2B 各表示什么？", "total 是总容量，active 是一条 Token 路径经过的参数规模；二者都不等于端到端 FLOPs。"],
|
||||
["Q02", "效率口径", "约 2.5× 究竟是什么？", "它是架构、数据与 recipe 共同形成的相对 K2 scaling efficiency，不是推理速度或 KDA 单项收益。"],
|
||||
["Q03", "混合节奏", "3 KDA + 1 MLA 如何落到 93 层？", "Table 1 给出 69 KDA + 24 MLA，最后一层也是 MLA；另有一层 dense。"],
|
||||
["Q04", "序列状态", "KDA 保存了什么？", "它维护固定形状 recurrent state，而不是所有历史 Token 的两两配对。"],
|
||||
["Q05", "纠错写入", "Delta Rule 为什么不是简单累加？", "先减掉 state 对当前 key 的已有预测，再写真实 value 与预测的差。"],
|
||||
["Q06", "数值下界", "gmin = −5 在解决什么？", "它限制 chunk 内累计衰减倒数，令关键 tile 可落在 BF16 Tensor Core 范围。"],
|
||||
["Q07", "并行算法", "递归状态如何训练并行？", "chunk 间传状态，chunk 内改写成并行矩阵乘；decode 仍逐步更新固定状态。"],
|
||||
["Q08", "读出门", "KDA / MLA 的 output gate 是 router 吗？", "不是；它按输入控制 attention 读出通道，MoE router 则选择专家。"],
|
||||
["Q09", "位置来源", "NoPE 是否意味着没有顺序？", "没有显式位置 embedding；KDA 的门控与衰减隐式传递顺序，MLA 负责全局内容匹配。"],
|
||||
["Q10", "舍入合同", "为什么 attention output 保留 FP32？", "报告用它修正 FlashAttention 有偏舍入，并为片上 tile buffer 重新排布。"],
|
||||
["Q11", "深度路由", "AttnRes 怎样改变 residual？", "每层用 pseudo-query 选择早层来源；Block 版用块内累加换取更低内存与跨 stage 通信。"],
|
||||
["Q12", "主模型边界", "12 层一块还是 2 层一块？", "K3 主模型为 12；§7 芯片 nano case 的 2 不能写回主规格。"],
|
||||
["Q13", "路由宽度", "LatentMoE 为什么先压到 3584？", "shared path 保留 7168，routed path 在较窄空间执行，以降低多专家激活的 payload 与权重流。"],
|
||||
["Q14", "专家角色", "896、16、2 怎样读？", "896 routed 中每 Token 选 16，另有 2 shared；稀疏不等于消费级硬件轻松运行。"],
|
||||
["Q15", "有界激活", "SiTU-GLU 在稳定什么？", "它限制 routed path 大正输入的幅值，同时近似 SwiGLU 在原点附近的形状。"],
|
||||
["Q16", "路由分位", "Quantile Balancing 怎样更新 bias？", "用 Top-(k+1) score cutoff 的全局分位量做下一步更新；最终 bias 冻结用于推理。"],
|
||||
["Q17", "平衡层次", "QB 与 MoonEP 是同一件事吗？", "QB 调模型路由偏置；MoonEP 调系统执行与通信形状；二者互补但不能合并。"],
|
||||
["Q18", "原生视觉", "视觉塔是否后接到预训练 LLM？", "不是。MoonViT-V2 从头训练，视觉与文本从训练开始就在同一 NTP objective 联合优化。"],
|
||||
["Q19", "视觉流", "图像和视频怎样进入主干？", "401M / 27 层编码器共享图像视频参数，经时空分解、pooling、pixel shuffle 与 projector 进入主干。"],
|
||||
["Q20", "优化器", "Per-Head Muon 为什么按 head 分组？", "多头投影按相对独立的 head 组织矩阵更新；它只是完整训练 recipe 的一部分。"],
|
||||
["Q21", "数据披露", "我们知道和不知道哪些语料事实？", "知道四类文本、视觉种类与清洗/重写流程；不知道完整来源、精确配比和总训练 Token。"],
|
||||
["Q22", "长度课程", "1M 是怎样训练出来的？", "pretrain 8K→64K，cooldown 256K→1M，并配长数据清洗、上采样和跨全局证据的合成任务。"],
|
||||
["Q23", "SFT 冷启动", "Agent 轨迹从哪里来？", "前代 Kimi 专家合成，经多阶段验证与 HITL 标注，再用 XTML 序列化；SFT 起即做 QAT。"],
|
||||
["Q24", "九位老师", "3 × 3 专家是否在线投票？", "不是。它们是三领域 × 三 effort 的训练策略，最终经 MOPD 整合进一个学生。"],
|
||||
["Q25", "思考预算", "low / high / max 如何定义？", "每题从 cold-start 估计 b₀(x)，按 domain 退火 τ；超过 τb₀ 的轨迹 reward 改为 −1。"],
|
||||
["Q26", "长尾轨迹", "Partial rollout 暂停了什么？", "N×K 轨迹中 λ 比例完成就更新，未完轨迹下轮优先恢复；代价是 stale/off-policy。"],
|
||||
["Q27", "蒸馏信号", "MOPD 为什么是 on-policy？", "学生生成自己会访问的 prefix，匹配的 teacher 在该 prefix 上给 clipped token log-ratio reward。"],
|
||||
["Q28", "部署约束", "量化与 speculative draft 何时进入？", "SFT/RL 全程 experts MXFP4/activations MXFP8 QAT；MTP 后调为 EAGLE-3 draft 并优化 LK loss。"],
|
||||
["Q29", "Harness", "Agent 能力属于模型还是脚手架？", "系统结果来自模型与 tool、prompt、context、skills、memory、subagent、verifier 的组合。"],
|
||||
["Q30", "状态寿命", "训练、RL、服务分别搬运什么？", "参数/优化器、KDA/MLA 序列状态、KV/轨迹、sandbox 与 prefix 的寿命完全不同。"],
|
||||
["Q31", "混合缓存", "KDA state 与 MLA KV 怎样共同命中？", "统一页池；细 hash boundary 与粗 physical block 解耦，命中点必须两类 cache 同时有效。"],
|
||||
["Q32", "评测协议", "一个分数至少需要哪些脚注？", "effort、tool、harness、fallback/guard、日期；报告也明确总体仍落后最强闭源模型。"],
|
||||
] as const;
|
||||
|
||||
export const k3ReportMap = [
|
||||
["§1", "Introduction", "四条扩展轴与总体结论", "Figure 1"],
|
||||
["§2", "Architecture", "KDA、AttnRes、Stable LatentMoE、MoonViT、Muon", "Figures 2–6 · Table 1"],
|
||||
["§3", "Pre-Training", "数据、scaling law、联合训练、1M curriculum", "Figure 7"],
|
||||
["§4", "Post-Training", "SFT、9 experts、partial rollout、MOPD、QAT、环境", "Figures 8–10"],
|
||||
["§5", "Infrastructure", "FlashKDA/KCP、MoonEP、RL 系统、AgentENV、serving", "Figures 11–12"],
|
||||
["§6", "Evaluations", "四能力轴、第三方结果、成本与协议", "Figure 13 · Tables 2–5"],
|
||||
["§7", "Case Studies", "kernel、MiniTriton、chip、research、knowledge、video", "Figures 14–15"],
|
||||
["§8", "Conclusion", "开放权重与剩余差距", "—"],
|
||||
["App.", "A–F", "KDA/AttnRes 细节、评测补充、XTML", "Figure 16"],
|
||||
] as const;
|
||||
|
||||
export const k3FigureAtlas = [
|
||||
["F01", "Figure 1", "主结果", "四能力轴必须与总体仍落后 Claude Fable 5 / GPT-5.6 Sol 的结论同屏。"],
|
||||
["F02", "Figure 2", "完整架构", "把视觉入口、3:1 hybrid stack、Block AttnRes 与 Stable LatentMoE 分成不同信息流。"],
|
||||
["F03", "Figure 3", "Bounded decay", "展示下界、累计倒数与 BF16 风险，不承诺任意配置绝对稳定。"],
|
||||
["F04", "Figure 4", "SiTU-GLU", "对比 GLU / SwiGLU / SiTU 的函数形状，不从曲线直接推训练 loss。"],
|
||||
["F05", "Figure 5", "Quantile Balancing", "Top-(k+1) cutoff → global quantile → expert bias → next step。"],
|
||||
["F06", "Figure 6", "原生视觉", "只在作者消融配置内说明 from-scratch MoonViT-V2 梯度更平稳。"],
|
||||
["F07", "Figure 7", "Scaling law", "约 2.5× 是相对 K2 的整体 scaling efficiency，不是 inference speedup。"],
|
||||
["F08", "Figure 8", "RL scaling", "RL FLOPs、平均 steps 与能力共同增长；相关性不自动成为跨域因果律。"],
|
||||
["F09", "Figure 9", "任务合成", "知识图谱 → 材料检索 → knowledge/coding/vision task → verification。"],
|
||||
["F10", "Figure 10", "AET", "初始状态、动作、独立 verifier 与 curriculum，奖励落在环境结果。"],
|
||||
["F11", "Figure 11", "训练系统", "画状态的放置与移动，不补写报告未公开的总集群规模。"],
|
||||
["F12", "Figure 12", "Prefix cache", "6144 physical / 512 hash / sparse KDA checkpoint 是报告示例。"],
|
||||
["F13", "Figure 13", "分数与成本", "score 与 output token / cost 同图，并保留比较协议。"],
|
||||
["F14", "Figure 14", "Kernel agent", "283.6→114.4 ms 等数字限定在作者 24h/task 个案。"],
|
||||
["F15", "Figure 15", "MiniTriton", "L20 实测点对 roofline；不推广到其他硬件和工作负载。"],
|
||||
["F16", "Figure 16", "XTML", "global / one-shot / input options、channel 和 message boundary 分层。"],
|
||||
["T01", "Table 1", "K2 → K3", "精确逐字段规格，hero 的 2.8T / 104B 只作易读四舍五入。"],
|
||||
["T02", "Table 2", "Reasoning / Coding", "把 effort、harness、tool 与 fallback 变成表头的一部分。"],
|
||||
["T03", "Table 3", "Agentic", "分数是 model × harness × environment × verifier 的系统结果。"],
|
||||
["T04", "Table 4", "Vision", "有/无 Python tool 必须分列。"],
|
||||
["T05", "Table 5", "第三方 / 成本", "价格点只描述当时服务条件，不作为架构因果证据。"],
|
||||
] as const;
|
||||
|
||||
export const k3PaperChain = [
|
||||
["001", "2014", "Neural Machine Translation by Jointly Learning to Align and Translate", "https://arxiv.org/abs/1409.0473", "内容寻址注意力前史", "FOUNDATION"],
|
||||
["002", "2015", "Deep Residual Learning for Image Recognition", "https://arxiv.org/abs/1512.03385", "标准 residual 起点", "DEPTH"],
|
||||
["003", "2017", "Attention Is All You Need", "https://arxiv.org/abs/1706.03762", "全局 self-attention 坐标", "FOUNDATION"],
|
||||
["004", "2017", "Language Modeling with Gated Convolutional Networks", "https://proceedings.mlr.press/v70/dauphin17a.html", "GLU 前史", "NUMERICS"],
|
||||
["005", "2018", "Parallelizing Linear Recurrent Neural Nets Over Sequence Length", "https://openreview.net/forum?id=HyUNwulC-", "递归的并行扫描", "STATE"],
|
||||
["006", "2018", "Quantization and Training of Neural Networks", "https://arxiv.org/abs/1712.05877", "QAT 前史", "DEPLOY"],
|
||||
["007", "2018", "Online normalizer calculation for softmax", "https://arxiv.org/abs/1805.02867", "online-softmax merge", "SYSTEM"],
|
||||
["008", "2019", "RMSNorm", "https://arxiv.org/abs/1910.07467", "尺度控制", "NUMERICS"],
|
||||
["009", "2019", "GPipe", "https://arxiv.org/abs/1811.06965", "pipeline schedule 前史", "SYSTEM"],
|
||||
["010", "2019", "Triton", "https://dl.acm.org/doi/10.1145/3315508.3329973", "tile compiler 前史", "SYSTEM"],
|
||||
["011", "2020", "Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention", "https://proceedings.mlr.press/v119/katharopoulos20a.html", "线性状态视角", "STATE"],
|
||||
["012", "2020", "Scaling Laws for Neural Language Models", "https://arxiv.org/abs/2001.08361", "经验缩放坐标", "SCALING"],
|
||||
["013", "2020", "GLU Variants Improve Transformer", "https://arxiv.org/abs/2002.05202", "SwiGLU 坐标", "NUMERICS"],
|
||||
["014", "2020", "GShard", "https://arxiv.org/abs/2006.16668", "大规模 expert parallel", "MOE"],
|
||||
["015", "2020", "ZeRO", "https://arxiv.org/abs/1910.02054", "训练状态分片", "SYSTEM"],
|
||||
["016", "2020", "Firecracker", "https://www.usenix.org/conference/nsdi20/presentation/agache", "AgentENV 隔离前史", "AGENT"],
|
||||
["017", "2021", "Linear Transformers Are Secretly Fast Weight Programmers", "https://proceedings.mlr.press/v139/schlag21a.html", "Delta memory 直觉", "STATE"],
|
||||
["018", "2021", "Switch Transformers", "https://arxiv.org/abs/2101.03961", "稀疏容量坐标", "MOE"],
|
||||
["019", "2021", "BASE Layers", "https://proceedings.mlr.press/v139/lewis21a.html", "专家负载分配", "MOE"],
|
||||
["020", "2021", "MLIR", "https://ieeexplore.ieee.org/document/9370308", "MiniTriton 编译基础", "SYSTEM"],
|
||||
["021", "2022", "Training Compute-Optimal Large Language Models", "https://arxiv.org/abs/2203.15556", "参数/数据预算", "SCALING"],
|
||||
["022", "2022", "FlashAttention", "https://arxiv.org/abs/2205.14135", "IO-aware exact attention", "SYSTEM"],
|
||||
["023", "2023", "Ring Attention", "https://arxiv.org/abs/2310.01889", "长序列块式并行", "CONTEXT"],
|
||||
["024", "2023", "DeepSpeed Ulysses", "https://arxiv.org/abs/2309.14509", "sequence parallel 对照", "CONTEXT"],
|
||||
["025", "2023", "YaRN", "https://arxiv.org/abs/2309.00071", "RoPE 外推对照", "CONTEXT"],
|
||||
["026", "2023", "Microscaling Data Formats for Deep Learning", "https://arxiv.org/abs/2310.10537", "MX formats", "DEPLOY"],
|
||||
["027", "2024", "DeepSeekMoE", "https://arxiv.org/abs/2401.06066", "shared + fine-grained experts", "DEEPSEEK"],
|
||||
["028", "2024", "DeepSeek-V2", "https://arxiv.org/abs/2405.04434", "MLA 与 MoE 直接祖先", "DEEPSEEK"],
|
||||
["029", "2024", "DeepSeek-V3", "https://arxiv.org/abs/2412.19437", "aux-loss-free、FP8、MTP 对照", "DEEPSEEK"],
|
||||
["030", "2024", "Griffin", "https://arxiv.org/abs/2402.19427", "recurrent + attention hybrid", "STATE"],
|
||||
["031", "2024", "Mamba-2 / Transformers are SSMs", "https://arxiv.org/abs/2405.21060", "状态空间二元性", "STATE"],
|
||||
["032", "2024", "HGRN2", "https://arxiv.org/abs/2404.07904", "gated recurrence 与 state expansion", "STATE"],
|
||||
["033", "2024", "Linear Attention Sequence Parallelism", "https://arxiv.org/abs/2404.02882", "linear attention CP", "CONTEXT"],
|
||||
["034", "2024", "Gated Linear Attention Transformers", "https://proceedings.mlr.press/v235/yang24ab.html", "硬件友好 GLA", "STATE"],
|
||||
["035", "2024", "Parallelizing Linear Transformers with the Delta Rule", "https://arxiv.org/abs/2406.06484", "chunkwise delta rule", "STATE"],
|
||||
["036", "2024", "Mooncake", "https://arxiv.org/abs/2407.00079", "KV-centric serving", "SYSTEM"],
|
||||
["037", "2025", "Kimi k1.5", "https://arxiv.org/abs/2501.12599", "长 CoT 与 RL 前代", "KIMI"],
|
||||
["038", "2025", "Gated Delta Networks", "https://openreview.net/forum?id=r8H7xhYPwz", "KDA 机制近邻", "STATE"],
|
||||
["039", "2025", "Kimi-VL", "https://arxiv.org/abs/2504.07491", "Kimi 视觉前代", "KIMI"],
|
||||
["040", "2025", "Kimi K2", "https://arxiv.org/abs/2507.20534", "主干、Muon、Agent 前代", "KIMI"],
|
||||
["041", "2025", "Kimi Linear", "https://arxiv.org/abs/2510.26692", "KDA 与 hybrid 直接前身", "KIMI"],
|
||||
["042", "2025", "Muon", "https://kellerjordan.github.io/posts/muon/", "矩阵正交化更新", "NUMERICS"],
|
||||
["043", "2025", "Muon is Scalable for LLM Training", "https://arxiv.org/abs/2502.16982", "Muon 扩展证据", "NUMERICS"],
|
||||
["044", "2025", "LASP-2", "https://arxiv.org/abs/2502.07563", "hybrid linear attention SP", "CONTEXT"],
|
||||
["045", "2025", "EAGLE-3", "https://arxiv.org/abs/2503.01840", "draft model 直接来源", "DEPLOY"],
|
||||
["046", "2025", "Gated Attention for Large Language Models", "https://arxiv.org/abs/2505.06708", "attention output gating", "STATE"],
|
||||
["047", "2025", "Optimus", "https://www.usenix.org/conference/atc25/presentation/feng", "多模态训练 bubble", "SYSTEM"],
|
||||
["048", "2025", "On-policy distillation", "https://thinkingmachines.ai/blog/on-policy-distillation/", "学生分布上的蒸馏", "RL"],
|
||||
["049", "2025", "DeepEP", "https://github.com/deepseek-ai/DeepEP", "专家并行对照实现", "DEEPSEEK"],
|
||||
["050", "2025", "Flash Linear Attention", "https://github.com/fla-org/flash-linear-attention", "线性注意力实现生态", "SYSTEM"],
|
||||
["051", "2025", "Video-MME", "https://arxiv.org/abs/2405.21075", "视频评测", "EVAL"],
|
||||
["052", "2025", "MMMU-Pro", "https://arxiv.org/abs/2409.02813", "稳健多学科视觉评测", "EVAL"],
|
||||
["053", "2025", "CharXiv", "https://arxiv.org/abs/2406.18521", "图表理解评测", "EVAL"],
|
||||
["054", "2025", "Math-Vision", "https://arxiv.org/abs/2402.14804", "视觉数学评测", "EVAL"],
|
||||
["055", "2025", "OmniDocBench", "https://arxiv.org/abs/2412.07626", "文档解析评测", "EVAL"],
|
||||
["056", "2025", "Humanity's Last Exam", "https://arxiv.org/abs/2501.14249", "高难知识推理评测", "EVAL"],
|
||||
["057", "2025", "BrowseComp", "https://arxiv.org/abs/2504.12516", "浏览与 context management", "EVAL"],
|
||||
["058", "2025", "SciCode", "https://arxiv.org/abs/2407.13168", "科研编码评测", "EVAL"],
|
||||
["059", "2025", "Tool Decathlon", "https://arxiv.org/abs/2510.25726", "长程工具使用评测", "EVAL"],
|
||||
["060", "2025", "DeepSearchQA", "https://storage.googleapis.com/deepmind-media/DeepSearchQA/DeepSearchQA_benchmark_paper.pdf", "深度研究覆盖度", "EVAL"],
|
||||
["061", "2025", "OSWorld-Verified", "https://xlang.ai/blog/osworld-verified", "电脑使用验证", "EVAL"],
|
||||
["062", "2025", "Terminal-Bench", "https://arxiv.org/abs/2601.11868", "终端 Agent 评测", "EVAL"],
|
||||
["063", "2025", "GDPval", "https://arxiv.org/abs/2510.04374", "经济价值任务", "EVAL"],
|
||||
["064", "2025", "ZeroBench", "https://arxiv.org/abs/2502.09696", "高难视觉基准", "EVAL"],
|
||||
["065", "2025", "SonicMoE", "https://arxiv.org/abs/2512.14080", "IO / tile-aware MoE", "SYSTEM"],
|
||||
["066", "2025", "TileLang", "https://arxiv.org/abs/2504.17577", "tile programming 对照", "SYSTEM"],
|
||||
["067", "2025", "ThunderKittens", "https://openreview.net/forum?id=0fJfVOSUra", "GPU kernel DSL 对照", "SYSTEM"],
|
||||
["068", "2026", "Kimi K2.5", "https://arxiv.org/abs/2602.02276", "视觉 Agent 与 RL 直接前代", "KIMI"],
|
||||
["069", "2026", "LatentMoE", "https://arxiv.org/abs/2601.18089", "routed latent width", "MOE"],
|
||||
["070", "2026", "Attention Residuals", "https://arxiv.org/abs/2603.15031", "跨深度选择", "DEPTH"],
|
||||
["071", "2026", "Kimi K3", "https://arxiv.org/abs/2607.24653", "本课程锚点报告", "KIMI"],
|
||||
["072", "2026", "Kimi K3 official repository", "https://github.com/MoonshotAI/Kimi-K3", "权重、报告与开放边界", "KIMI"],
|
||||
["073", "2026", "FlashKDA", "https://github.com/MoonshotAI/FlashKDA", "KDA 官方 kernel", "SYSTEM"],
|
||||
["074", "2026", "AgentENV", "https://github.com/kvcache-ai/AgentENV", "可恢复 microVM 环境", "AGENT"],
|
||||
["075", "2026", "ReplaySSM", "https://tridao.me/blog/2026/replayssm/", "缓存输入而非每步 state", "DEPLOY"],
|
||||
["076", "2026", "LK Losses", "https://arxiv.org/abs/2602.23881", "直接优化 speculative acceptance", "DEPLOY"],
|
||||
["077", "2026", "PowLU", "https://arxiv.org/abs/2605.25704", "有界/稳定激活对照", "NUMERICS"],
|
||||
["078", "2026", "Why Low-Precision Transformer Training Fails", "https://arxiv.org/abs/2510.04212", "FlashAttention 舍入风险", "NUMERICS"],
|
||||
["079", "2026", "Expert Threshold Routing", "https://arxiv.org/abs/2603.11535", "动态专家与均衡对照", "MOE"],
|
||||
["080", "2026", "BIP Expert Load Balancing", "https://arxiv.org/abs/2502.15451", "专家分配对照", "MOE"],
|
||||
["081", "2026", "UltraEP", "https://arxiv.org/abs/2606.04101", "rack-scale expert parallel", "SYSTEM"],
|
||||
["082", "2026", "Megatron Core MoE", "https://arxiv.org/abs/2603.07685", "MoE 系统复现坐标", "SYSTEM"],
|
||||
["083", "2026", "Warp Decode", "https://cursor.com/blog/warp-decode", "token-centric MoE decode", "SYSTEM"],
|
||||
["084", "2026", "DeepSeek-V4", "https://arxiv.org/abs/2606.19348", "百万上下文同期对照", "DEEPSEEK"],
|
||||
["085", "2025", "DeepSeek-R1", "https://arxiv.org/abs/2501.12948", "reasoning RL 重点对照", "DEEPSEEK"],
|
||||
["086", "2026", "MCP-Atlas", "https://arxiv.org/abs/2602.00933", "MCP 工具能力评测", "EVAL"],
|
||||
["087", "2026", "MCPMark", "https://arxiv.org/abs/2509.24002", "真实 MCP 压力测试", "EVAL"],
|
||||
["088", "2026", "ResearchRubrics", "https://openreview.net/forum?id=ErnvfmSX0P", "深度研究 rubric", "EVAL"],
|
||||
["089", "2026", "AutomationBench", "https://arxiv.org/abs/2604.18934", "自动化 Agent 评测", "EVAL"],
|
||||
["090", "2026", "SaaS-Bench", "https://arxiv.org/abs/2605.15777", "专业 SaaS workflow", "EVAL"],
|
||||
["091", "2026", "Agents' Last Exam", "https://arxiv.org/abs/2606.05405", "通用 Agent 压力测试", "EVAL"],
|
||||
["092", "2026", "APEX-Agents", "https://arxiv.org/abs/2601.14242", "跨域 Agent 评测", "EVAL"],
|
||||
["093", "2026", "OSWorld 2.0", "https://arxiv.org/abs/2606.29537", "长程电脑使用", "EVAL"],
|
||||
["094", "2026", "OfficeQA Pro", "https://arxiv.org/abs/2603.08655", "企业知识工作", "EVAL"],
|
||||
["095", "2026", "SpreadsheetBench 2", "https://arxiv.org/abs/2606.29955", "端到端表格工作", "EVAL"],
|
||||
["096", "2026", "BabyVision", "https://arxiv.org/abs/2601.06521", "超越语言先验的视觉推理", "EVAL"],
|
||||
["097", "2026", "MMVU", "https://openaccess.thecvf.com/content/CVPR2025/html/Zhao_MMVU_Measuring_Expert-Level_Multi-Discipline_Video_Understanding_CVPR_2025_paper.html", "专家级视频理解", "EVAL"],
|
||||
["098", "2026", "WorldVQA", "https://arxiv.org/abs/2602.02537", "原子视觉世界知识", "EVAL"],
|
||||
["099", "2026", "PerceptionBench", "https://www.kimi.com/blog/perception-bench", "Kimi 原子视觉感知", "KIMI"],
|
||||
["100", "2026", "Preliminary Assessment of Kimi K3's Cyber Capabilities", "https://www.aisi.gov.uk/blog/preliminary-assessment-of-kimi-k3s-cyber-capabilities", "独立安全评估边界", "EVAL"],
|
||||
] as const;
|
||||
+28
-2
@@ -40,8 +40,8 @@ const paths = [
|
||||
label: "K3 反向拆解",
|
||||
title: "先看全貌，再沿组件回到每条技术祖先",
|
||||
text: "适合已经用过大模型、想迅速读懂 K3 报告的人。每个组件都能跳回其历史专题。",
|
||||
steps: ["K3 三维信息流", "KDA 与 MLA", "Stable LatentMoE", "1M Agentic RL 与系统"],
|
||||
time: "6–10 小时",
|
||||
steps: ["32 张问题账", "KDA / MLA / AttnRes / LatentMoE", "预训练与 MOPD", "1M RL / 服务 / 评测"],
|
||||
time: "10–16 小时",
|
||||
target: "能逐节解释 K3 技术报告",
|
||||
},
|
||||
{
|
||||
@@ -126,6 +126,22 @@ const paths = [
|
||||
|
||||
<section class="section compact release-section" id="new-chapters">
|
||||
<div class="release-grid">
|
||||
<a class="release-card k3-release" href="/k3/">
|
||||
<div>
|
||||
<p class="eyebrow"><span>NEW / K3 ROUND 02</span> REPORT · FORMULA · SYSTEM · EVIDENCE</p>
|
||||
<h2>47 页不再压成摘要：把 K3 的每个因果环节重新展开</h2>
|
||||
<p>
|
||||
用三十二张问题账逐节读完 KDA、Gated MLA、AttnRes、Stable LatentMoE、原生视觉、
|
||||
预训练、九专家 MOPD、Agent 环境、FlashKDA / MoonEP、混合 prefix cache、评测与案例边界。
|
||||
</p>
|
||||
</div>
|
||||
<dl>
|
||||
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div>
|
||||
<div><dt>NODES</dt><dd>100 个一手 / 官方节点</dd></div>
|
||||
<div><dt>LAB</dt><dd>Delta · Decay · AttnRes · MoE · QB · RL · Cache</dd></div>
|
||||
</dl>
|
||||
<span class="release-arrow" aria-hidden="true">从报告目录进入完整因果链 →</span>
|
||||
</a>
|
||||
<a class="release-card deepseek-release" href="/deepseek/">
|
||||
<div>
|
||||
<p class="eyebrow"><span>NEW / DEEPSEEK ROUND 02</span> CAPACITY · STATE · SYSTEM · REASONING</p>
|
||||
@@ -615,6 +631,7 @@ const paths = [
|
||||
transition: transform 180ms ease, border-color 180ms ease;
|
||||
}
|
||||
|
||||
.k3-release,
|
||||
.deepseek-release,
|
||||
.representation-release,
|
||||
.inference-release,
|
||||
@@ -639,6 +656,14 @@ const paths = [
|
||||
var(--paper-raised);
|
||||
}
|
||||
|
||||
.k3-release {
|
||||
background:
|
||||
radial-gradient(circle at 82% 16%, rgba(159, 91, 52, .28), transparent 31%),
|
||||
radial-gradient(circle at 58% 74%, rgba(76, 118, 112, .22), transparent 30%),
|
||||
repeating-linear-gradient(135deg, transparent 0 62px, rgba(159, 91, 52, .045) 62px 63px),
|
||||
var(--paper-raised);
|
||||
}
|
||||
|
||||
.representation-release {
|
||||
background:
|
||||
radial-gradient(circle at 82% 18%, rgba(159, 91, 52, 0.22), transparent 31%),
|
||||
@@ -785,6 +810,7 @@ const paths = [
|
||||
padding-bottom: 76px;
|
||||
}
|
||||
|
||||
.k3-release,
|
||||
.deepseek-release,
|
||||
.representation-release,
|
||||
.inference-release,
|
||||
|
||||
+1045
-527
File diff suppressed because it is too large
Load Diff
@@ -9,7 +9,7 @@ const researching = chapters.filter((chapter) => ["researching", "drafting"].inc
|
||||
const workstreams = [
|
||||
{ label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" },
|
||||
{ label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" },
|
||||
{ label: "Kimi K3 深读", value: 66, next: "扩写 pre-training / infra 逐图笔记" },
|
||||
{ label: "Kimi K3 深读", value: 88, next: "第三轮加入官方权重 traces、独立复现与逐图数值重绘" },
|
||||
{ label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau，并加入真实小语料复现" },
|
||||
{ label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" },
|
||||
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
|
||||
@@ -50,7 +50,7 @@ const workstreams = [
|
||||
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
|
||||
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
|
||||
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
|
||||
<div><dt>UPDATED</dt><dd>2026-07-29 11:10 CST</dd></div>
|
||||
<div><dt>UPDATED</dt><dd>2026-07-29 11:54 CST</dd></div>
|
||||
<div><dt>MODE</dt><dd>持续迭代，不锁死版本</dd></div>
|
||||
</dl>
|
||||
</div>
|
||||
@@ -97,12 +97,13 @@ const workstreams = [
|
||||
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
|
||||
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全，包含先修依赖和三条贯穿案例。</p></article>
|
||||
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
|
||||
<article><span>✓</span><h3>五十九个原创交互视图</h3><p>K3、语言模型前史、Transformer、表示深度、DeepSeek 四联实验、长上下文、MoE、推理、Agent、多模态，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
|
||||
<article><span>✓</span><h3>六十七个原创交互视图</h3><p>K3 三轴图与八联实验、语言模型前史、Transformer、表示深度、DeepSeek 四联实验、长上下文、MoE、推理、Agent、多模态，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
|
||||
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
|
||||
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
|
||||
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
|
||||
<article><span>✓</span><h3>表示、位置与残差高速公路深度专题</h3><p>二十张问题账、66 个一手节点、DeepSeek/Kimi 双谱系，以及 Token—位置—Norm—Residual/FFN 四联实验。</p></article>
|
||||
<article><span>✓</span><h3>DeepSeek 技术谱系二轮深读</h3><p>二十四张问题账、十次技术转向、60 个一手/官方节点，以及稀疏容量—MLA 缓存—V3 协同—RL 偏差四联实验。</p></article>
|
||||
<article><span>✓</span><h3>Kimi K3 技术报告二轮深读</h3><p>三十二张问题账、Figure 1–16 / Table 1–5 审计、100 节点阅读链，以及 Delta—Decay—AttnRes—LatentMoE—SiTU—QB—MOPD—Cache 八联实验。</p></article>
|
||||
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
|
||||
<article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article>
|
||||
<article><span>✓</span><h3>长上下文深度专题</h3><p>五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。</p></article>
|
||||
@@ -130,6 +131,7 @@ const workstreams = [
|
||||
</div>
|
||||
<div class="queue-table">
|
||||
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
|
||||
<div><span>P0</span><strong>K3 三轮</strong><p>开放权重 traces → FlashKDA / AttnRes / MoE 真实行为 → Figure 1–16 数值重绘与独立复现</p><em>运行证据 + 逐图复现</em></div>
|
||||
<div><span>P0</span><strong>DeepSeek 三轮</strong><p>真实 expert load / MLA kernel → FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
|
||||
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
|
||||
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
|
||||
@@ -204,6 +206,9 @@ const workstreams = [
|
||||
<div><time>2026-07-29</time><b>DeepSeek 与 Kimi 服务谱系按状态对象重建</b><p>MLA→V4 异构状态与 Mooncake→KDA→K3 混合缓存分开说明；作者报告、精确公式和教学估算使用不同标签。</p></div>
|
||||
<div><time>2026-07-29</time><b>表示与深度按二十张账组织</b><p>计算单位、词表接口、上下文化、位置、外推、Norm 对象、拓扑、残差路由与非线性极值不再混成一个 hidden-state 名词。</p></div>
|
||||
<div><time>2026-07-29</time><b>K3 Block AttnRes 来源数按原报告重算</b><p>93 层按 12 层形成 8 个 layer blocks（7 个完整块加 1 个尾块）；再加 embedding，共 9 个 block-level 来源，废弃早期错误的“2 层一块”读法。</p></div>
|
||||
<div><time>2026-07-29</time><b>K3 二轮按三十二张账重建</b><p>从架构组件摘要升级为覆盖预训练、后训练、环境、系统、评测、案例与附录的完整报告因果链。</p></div>
|
||||
<div><time>2026-07-29</time><b>K3 原生视觉事实纠错</b><p>MoonViT-V2 从头训练；视觉与文本从训练开始在同一个 NTP objective 中联合优化，不再沿用冻结/解冻式 post-hoc 叙述。</p></div>
|
||||
<div><time>2026-07-29</time><b>K3 图表与实验永久分级</b><p>Figure 1–16 / Table 1–5 建立视觉契约；报告事实、原论文、确定性推导与教学模型使用 R/P/D/T 四种身份。</p></div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
|
||||
@@ -101,13 +101,13 @@ const stages = [
|
||||
</article>
|
||||
<article>
|
||||
<span>ROUTE B · K3 REVERSE</span>
|
||||
<h3>从 K3 反向拆组件</h3>
|
||||
<p>K3 → 07 → 06 → 03 → 11 → 08/14</p>
|
||||
<h3>从 47 页 K3 报告反向拆组件</h3>
|
||||
<p>K3 架构 → 07/06/03 → 05/04 → 11/12 → 08/14/15</p>
|
||||
<ol>
|
||||
<li>解释三维信息流</li>
|
||||
<li>比较 KDA 与 MLA</li>
|
||||
<li>解释 2.8T / 104B</li>
|
||||
<li>读懂 1M Agentic RL 系统</li>
|
||||
<li>沿 32 张问题账解释三维信息流</li>
|
||||
<li>用 8 个实验比较 KDA、MLA、AttnRes 与 LatentMoE</li>
|
||||
<li>分清 2.78T / 104.2B、2.5× 与 1M 的证据口径</li>
|
||||
<li>读懂九专家 MOPD、AgentENV、混合缓存与评测协议</li>
|
||||
</ol>
|
||||
</article>
|
||||
<article>
|
||||
|
||||
Reference in New Issue
Block a user