feat: add AttnRes gradient scale lab

This commit is contained in:
wuyang
2026-07-30 10:24:42 +08:00
parent 26fc824409
commit f7670efcdd
9 changed files with 1191 additions and 17 deletions
+14 -2
View File
@@ -41,7 +41,7 @@
- [x] 完成 486 篇关键论文索引,覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。 - [x] 完成 486 篇关键论文索引,覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。
- [x] 完成可检索、可按专题筛选的论文库页面。 - [x] 完成可检索、可按专题筛选的论文库页面。
- [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。 - [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。
- [x] 完成 K3 三轴架构、八联报告实验、四联开放工件实验与五联 AttnRes 独立实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 二十二联实验、长上下文、MoE 路由、推理三页签,以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等九十四个原创交互视图。 - [x] 完成 K3 三轴架构、八联报告实验、四联开放工件实验、Round 04 / 05 各五联 AttnRes 独立实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 二十二联实验、长上下文、MoE 路由、推理三页签,以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等九十九个原创交互视图。
- [x] 完成长上下文首版:五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。 - [x] 完成长上下文首版:五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
- [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文,并建立长上下文研究账本。 - [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文,并建立长上下文研究账本。
- [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文,并建立 MoE 研究账本。 - [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文,并建立 MoE 研究账本。
@@ -279,10 +279,18 @@
- [x] K3 Round 04 五视图实验室完成:三 seed BPC 曲线、Residual RMS / Block 锯齿、Full / Block depth-weight heatmap、梯度反证、成本/哈希/claim boundary 分开展示;完整 9-run JSON、compact 数据、复现清单、协议、审计、训练与聚合代码进入公开仓库。 - [x] K3 Round 04 五视图实验室完成:三 seed BPC 曲线、Residual RMS / Block 锯齿、Full / Block depth-weight heatmap、梯度反证、成本/哈希/claim boundary 分开展示;完整 9-run JSON、compact 数据、复现清单、协议、审计、训练与聚合代码进入公开仓库。
- [x] Round 04 本地闸门通过:91 个受检文件零诊断/提示,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;冻结数据、AttnRes 专项与 K3 全量真实 Chrome 回归通过,桌面/390px 移动端零文档级溢出、零运行时异常。 - [x] Round 04 本地闸门通过:91 个受检文件零诊断/提示,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;冻结数据、AttnRes 专项与 K3 全量真实 Chrome 回归通过,桌面/390px 移动端零文档级溢出、零运行时异常。
- [x] K3 Round 04 以功能源提交 `4ce780d`、不可变镜像 `20260729T233142Z-4ce780d` 发布;OCI index digest `sha256:6e89f802…25f582`,复用 NAS `12010→8080`、NPM host 31 / cert 41 与门户 `LLM ATLAS / projects / 180`。容器 healthy、0 次重启,21/21 公网页面、HTTPS/2、gzip / immutable assets、AttnRes 专项与 K3 全量生产 Chrome 回归通过;保留 `20260729T221654Z-975ed3d` 回滚。 - [x] K3 Round 04 以功能源提交 `4ce780d`、不可变镜像 `20260729T233142Z-4ce780d` 发布;OCI index digest `sha256:6e89f802…25f582`,复用 NAS `12010→8080`、NPM host 31 / cert 41 与门户 `LLM ATLAS / projects / 180`。容器 healthy、0 次重启,21/21 公网页面、HTTPS/2、gzip / immutable assets、AttnRes 专项与 K3 全量生产 Chrome 回归通过;保留 `20260729T221654Z-975ed3d` 回滚。
- [x] K3 Round 05 一手定义审计确认 Figure 5(c) 未公开 gradient tensor、norm、reduction、diagnostic batch、AMP / clipping 时点或统计代码;Round 04 参数梯度与 Round 05 post-MLP output activation gradient 永久分对象记账,不把本站 operationalization 冒充作者实现。
- [x] 在任何 formal 输出前冻结 16 / 32 blocks、Baseline / Block、三个 seed、8,000 steps、六个诊断点、CV + 首尾四分位 imbalance 联合判据、FP32 residual accumulator、20-step 双 smoke 与 depth-32 Block 完整 replay;768,000-window schedule SHA-256 为 `5041e09b…f4e`。
- [x] 12 个 formal 格全部完成,共 786,432,000 target bytes;公共主干与三个 gate input tensor hashes 在 paired 架构间 exact。Block 的验证 BPC 在 6 / 6 配对中更低,depth-16 / 32 mean delta 为 `−.008938 / −.009872`,但不追加事后 BPC support 阈值。
- [x] activation-gradient 结果分裂:首/末四分位 imbalance 在 6 / 6 配对改善,depth-16 / 32 均值为 `+61.0% / +72.0%`;全层 CV 却在 6 / 6 配对恶化,均值相对 reduction 为 `−10.3% / −60.0%`。两个 depth 都按预注册规则判为 `mixed / inconclusive`,总判定 `depth-dependent or inconclusive`。
- [x] 绝对 gradient mean 仅为 Baseline 的 `57.4% / 54.4%`;参数 gradient CV 从 `0.416→0.683 / 0.397→0.772`,继续保留反结果。Output RMS 最后/第一层比则由 Baseline `4.59× / 6.08×` 降至 Block `1.17× / 1.89×`。
- [x] 指定 depth-32 / Block / seed-2026073001 从初始化完整重训 8,000 steps;排除 run-kind / timing 后冻结字段 compare SHA-256 同为 `46300a45…4817`,model / optimizer state hashes exact。正式/compact/reproduction 物理 SHA-256 为 `ad461cbe…a8d / 5377a5e7…68e3 / aedcde6a…dea6`。
- [x] K3 Round 05 五视图实验室完成:论文定义已知/未定义、绝对/归一化深度谱、六 checkpoint 时间轨迹、Output RMS 组节律、activation/parameter/BPC/成本/重放联合账全部可切换;21 个 raw JSON、完整 aggregate、compact、runner、analyzer、协议与审计进入公开树。
- [x] Round 05 本地闸门通过:94 个 Astro 文件零诊断/提示,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;冻结数据、新专项、Round 04 与 K3 全量真实 Chrome 回归通过,桌面/390px 移动端零文档级溢出、零运行时异常。
## 正在进行 ## 正在进行
- [ ] K3 四轮下一闸门:对齐 AttnRes 论文的 activation / residual-output gradient 定义,增加模型深度与训练预算,检验本轮梯度反结果是否随尺度翻转;真实 K3 forward 继续等待 `A_log [128]→[96]` 官方转换或权重修订。 - [ ] K3 五轮下一闸门:对齐 layer 21–25 的 activation-gradient 尖峰、pre-attention / pre-MLP 位置与 mixer source weights,并做公开 reduction sensitivity;真实 K3 forward 继续等待 `A_log [128]→[96]` 官方转换或权重修订。
- [ ] DeepSeek 八轮下一闸门:推进干预式 mediation、SM90 FlashMLA、FP8 / pipeline traces 与 R1-like RL 小模型复现。 - [ ] DeepSeek 八轮下一闸门:推进干预式 mediation、SM90 FlashMLA、FP8 / pipeline traces 与 R1-like RL 小模型复现。
- [ ] 表示、位置与残差二轮:真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。 - [ ] 表示、位置与残差二轮:真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。
- [ ] 评测安全二轮:真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。 - [ ] 评测安全二轮:真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。
@@ -476,6 +484,10 @@
| 2026-07-30 | 主结果与机制反结果同时发布 | Full / Block BPC 方向支持;核心参数 gradient RMS CV 却高于 Baseline,明确写成未复现论文梯度叙述 | | 2026-07-30 | 主结果与机制反结果同时发布 | Full / Block BPC 方向支持;核心参数 gradient RMS CV 却高于 Baseline,明确写成未复现论文梯度叙述 |
| 2026-07-30 | 独立重放按数值合同而非计时合同验收 | Block / seed-1 的八组冻结字段 2,000 steps exact;wall time 受调度影响,不要求或声称 bit-exact | | 2026-07-30 | 独立重放按数值合同而非计时合同验收 | Block / seed-1 的八组冻结字段 2,000 steps exact;wall time 受调度影响,不要求或声称 bit-exact |
| 2026-07-30 | K3 Round 04 缩小 AttnRes 里程碑发布 | 功能源 `4ce780d`、镜像 `20260729T233142Z-4ce780d`、OCI `sha256:6e89f802…25f582`;21/21 公网页面与生产专项/全量 Chrome 通过,保留 Round 08 回滚点 | | 2026-07-30 | K3 Round 04 缩小 AttnRes 里程碑发布 | 功能源 `4ce780d`、镜像 `20260729T233142Z-4ce780d`、OCI `sha256:6e89f802…25f582`;21/21 公网页面与生产专项/全量 Chrome 通过,保留 Round 08 回滚点 |
| 2026-07-30 | AttnRes 的“梯度”先按公开证据拆对象 | 论文 Figure 5 没有公开唯一 telemetry 合同;参数梯度与 post-MLP activation gradient 不再互相代称 |
| 2026-07-30 | “更均匀”拆成首尾失衡与全层 CV | Block 6/6 改善 first/last,却 6/6 恶化 CV;局部尖峰与系统性早层隆起必须分开解释 |
| 2026-07-30 | 绝对尺度与归一化形状永久同报 | Block activation-gradient mean 约为 Baseline 54%–57%;不能把更接近 1 的首尾比自动解释为各层信号更强 |
| 2026-07-30 | Round 05 完整重放过闸 | depth-32 Block seed-1 从零重训 8,000 steps;全部冻结字段与 model/optimizer state hashes exact,timing 仍单独报告 |
## 未决问题 ## 未决问题
+14 -1
View File
@@ -19,7 +19,7 @@
当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读, 当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读,
语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化,以及评测与安全深度专题, 语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化,以及评测与安全深度专题,
以及 94 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、 以及 99 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
8 个交互实验和 100 个一手/官方节点,完整覆盖架构、预训练、后训练、系统、评测、案例与附录。 8 个交互实验和 100 个一手/官方节点,完整覆盖架构、预训练、后训练、系统、评测、案例与附录。
第三轮已完成开放工件与首个真实 kernel 里程碑:固定官方模型与 FlashKDA revisions,审计 96 个 checkpoint shards、 第三轮已完成开放工件与首个真实 kernel 里程碑:固定官方模型与 FlashKDA revisions,审计 96 个 checkpoint shards、
497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计,并用 4 个新视图 497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计,并用 4 个新视图
@@ -39,6 +39,19 @@
[K3_ATTNRES_REDUCED_PROTOCOL.md](./research/K3_ATTNRES_REDUCED_PROTOCOL.md)、 [K3_ATTNRES_REDUCED_PROTOCOL.md](./research/K3_ATTNRES_REDUCED_PROTOCOL.md)、
[K3_ATTNRES_REDUCED_AUDIT.md](./research/K3_ATTNRES_REDUCED_AUDIT.md) 与 [K3_ATTNRES_REDUCED_AUDIT.md](./research/K3_ATTNRES_REDUCED_AUDIT.md) 与
[AttnRes experiment](./experiments/k3/attnres/)。 [AttnRes experiment](./experiments/k3/attnres/)。
第五轮先审计 Attention Residuals Figure 5 的公开定义边界,再冻结
`llm-atlas-k3-attnres-gradient-scale-v1`:以 post-MLP block output activation gradient
为公开 operationalization,把深度扩为 16 / 32 blocks、预算扩为每格 8,000 steps,
完成 Baseline / Block × 三 seed 共 12 格、786,432,000 formal target bytes。结果把
“更均匀”拆成两个相反方向:Block 在 6 / 6 配对中把首/末四分位失衡改善 56%–81%,
却因中后段局部尖峰让全层 CV 在 6 / 6 配对中恶化;两个深度都按预注册联合规则判为
mixed / inconclusive。验证 BPC 仍在 6 / 6 配对中更低,但实际 step time 约 2.6×、
peak allocated memory 约 2.2×,不冒充同算力优势。指定 depth-32 / Block / seed-1
从零重训完整 8,000 steps,全部冻结字段以及 model / optimizer state hashes exact。
详见
[K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md](./research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md)、
[K3_ATTNRES_GRADIENT_SCALE_AUDIT.md](./research/K3_ATTNRES_GRADIENT_SCALE_AUDIT.md) 与
[gradient experiment](./experiments/k3/attnres_gradient/)。
DeepSeek 八轮专题以 24 张问题账、10 次技术转向、 DeepSeek 八轮专题以 24 张问题账、10 次技术转向、
22 个交互实验和 60 个一手/官方节点,串起 Dense、MoE、MLA、V3 协同、R1 与 V4; 22 个交互实验和 60 个一手/官方节点,串起 Dense、MoE、MLA、V3 协同、R1 与 V4;
并固定官方 V2-Lite revision,在 RTX 5090 上连续执行 7/27 层,记录 3,240 次真实专家选择、 并固定官方 V2-Lite revision,在 RTX 5090 上连续执行 7/27 层,记录 3,240 次真实专家选择、
+2
View File
@@ -20,6 +20,7 @@
"build:data:deepseek-chat-task-bootstrap": "node scripts/build-deepseek-chat-task-bootstrap-crn-compact.mjs", "build:data:deepseek-chat-task-bootstrap": "node scripts/build-deepseek-chat-task-bootstrap-crn-compact.mjs",
"check:data:deepseek-chat-task-bootstrap": "node scripts/check-deepseek-chat-task-bootstrap-crn-data.mjs", "check:data:deepseek-chat-task-bootstrap": "node scripts/check-deepseek-chat-task-bootstrap-crn-data.mjs",
"check:data:k3-attnres": "node scripts/check-k3-attnres-data.mjs", "check:data:k3-attnres": "node scripts/check-k3-attnres-data.mjs",
"check:data:k3-attnres-gradient": "node scripts/check-k3-attnres-gradient-data.mjs",
"check:site": "node scripts/check-site.mjs", "check:site": "node scripts/check-site.mjs",
"check:moe-browser": "node scripts/check-moe-browser.mjs", "check:moe-browser": "node scripts/check-moe-browser.mjs",
"check:reasoning-browser": "node scripts/check-reasoning-browser.mjs", "check:reasoning-browser": "node scripts/check-reasoning-browser.mjs",
@@ -40,6 +41,7 @@
"check:deepseek-cross-source-sampling-browser": "node scripts/check-deepseek-cross-source-sampling-browser.mjs", "check:deepseek-cross-source-sampling-browser": "node scripts/check-deepseek-cross-source-sampling-browser.mjs",
"check:deepseek-task-bootstrap-browser": "node scripts/check-deepseek-task-bootstrap-browser.mjs", "check:deepseek-task-bootstrap-browser": "node scripts/check-deepseek-task-bootstrap-browser.mjs",
"check:k3-attnres-browser": "node scripts/check-k3-attnres-browser.mjs", "check:k3-attnres-browser": "node scripts/check-k3-attnres-browser.mjs",
"check:k3-attnres-gradient-browser": "node scripts/check-k3-attnres-gradient-browser.mjs",
"check:k3-browser": "node scripts/check-k3-browser.mjs" "check:k3-browser": "node scripts/check-k3-browser.mjs"
}, },
"dependencies": { "dependencies": {
@@ -0,0 +1,231 @@
import { writeFileSync } from "node:fs";
const cdpPort = process.env.CDP_PORT ?? "9228";
const baseUrl = process.env.SITE_URL ?? "http://127.0.0.1:4328";
const pages = await fetch(`http://127.0.0.1:${cdpPort}/json/list`).then((response) => response.json());
const page = pages.find((entry) => entry.type === "page");
if (!page) throw new Error(`CDP ${cdpPort} 没有可用页面`);
const socket = new WebSocket(page.webSocketDebuggerUrl);
await new Promise((resolve, reject) => {
socket.addEventListener("open", resolve, { once: true });
socket.addEventListener("error", reject, { once: true });
});
let nextId = 0;
const pending = new Map();
const exceptions = [];
socket.addEventListener("message", (event) => {
const message = JSON.parse(event.data);
if (message.id && pending.has(message.id)) {
const { resolve, reject } = pending.get(message.id);
pending.delete(message.id);
if (message.error) reject(new Error(message.error.message));
else resolve(message.result);
}
if (message.method === "Runtime.exceptionThrown") {
exceptions.push(message.params.exceptionDetails.exception?.description ?? message.params.exceptionDetails.text);
}
});
const command = (method, params = {}) => new Promise((resolve, reject) => {
const id = ++nextId;
pending.set(id, { resolve, reject });
socket.send(JSON.stringify({ id, method, params }));
});
const pause = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
const evaluate = async (expression) => {
const result = await command("Runtime.evaluate", { expression, returnByValue: true, awaitPromise: true });
if (result.exceptionDetails) throw new Error(result.exceptionDetails.exception?.description ?? result.exceptionDetails.text);
return result.result.value;
};
const navigate = async (path) => {
await command("Page.navigate", { url: `${baseUrl}${path}` });
for (let attempt = 0; attempt < 100; attempt += 1) {
await pause(100);
if (await evaluate("document.readyState === 'complete'")) return;
}
throw new Error(`${path} 加载超时`);
};
const screenshot = async (path) => {
const result = await command("Page.captureScreenshot", { format: "png", captureBeyondViewport: false });
writeFileSync(path, Buffer.from(result.data, "base64"));
};
await command("Page.enable");
await command("Runtime.enable");
await command("Emulation.setDeviceMetricsOverride", {
width: 1440,
height: 1100,
deviceScaleFactor: 1,
mobile: false,
});
await navigate("/k3/");
const desktop = await evaluate(`(() => {
const root = document.querySelector("[data-gradient-lab]");
root.scrollIntoView({ block: "start", behavior: "instant" });
window.scrollBy(0, -78);
const text = (selector) => root.querySelector(selector)?.textContent.trim();
const panel = () => root.querySelector("[data-gradient-panel]:not([hidden])")?.dataset.gradientPanel;
const points = (selector) => root.querySelector(selector)?.getAttribute("points");
const setSelect = (selector, value) => {
const node = root.querySelector(selector);
node.value = value;
node.dispatchEvent(new Event("change", { bubbles: true }));
};
const initial = {
panel: panel(),
tabs: root.querySelectorAll("[data-gradient-tab]").length,
panels: root.querySelectorAll("[data-gradient-panel]").length,
ledger: root.querySelectorAll(".gradient-ledger article").length,
definition: root.textContent.includes("论文作者就是这样算的") &&
root.textContent.includes("activation gradient") &&
root.textContent.includes("parameter gradient"),
};
root.querySelector('[data-gradient-tab="spectrum"]').click();
const spectrumInitial = {
panel: panel(),
baseCv: text("[data-spectrum-base-cv]"),
blockCv: text("[data-spectrum-block-cv]"),
baseRatio: text("[data-spectrum-base-ratio]"),
blockRatio: text("[data-spectrum-block-ratio]"),
baseLine: points('[data-chart-line="baseline"]'),
blockLine: points('[data-chart-line="block"]'),
boundaries: root.querySelectorAll("[data-chart-groups] .group-boundary").length,
};
root.querySelector('[data-spectrum-scale="normalized"]').click();
const normalized = {
title: text("[data-spectrum-title]"),
baseLine: points('[data-chart-line="baseline"]'),
};
root.querySelector('[data-spectrum-depth="16"]').click();
const depth16 = {
baseCv: text("[data-spectrum-base-cv]"),
blockCv: text("[data-spectrum-block-cv]"),
pointCount: points('[data-chart-line="baseline"]').split(" ").length,
boundaries: root.querySelectorAll("[data-chart-groups] .group-boundary").length,
};
setSelect("[data-spectrum-seed]", "2026073001");
setSelect("[data-spectrum-step]", "2000");
const seedStep = {
state: text("[data-spectrum-state]"),
baseCv: text("[data-spectrum-base-cv]"),
blockCv: text("[data-spectrum-block-cv]"),
};
root.querySelector('[data-gradient-tab="timeline"]').click();
const timelineInitial = {
panel: panel(),
title: text("[data-time-title]"),
line: points('[data-time-line="block"]'),
pointCount: root.querySelectorAll('[data-time-points="block"] circle').length,
};
root.querySelector('[data-time-metric="imbalance"]').click();
const timelineChanged = {
title: text("[data-time-title]"),
line: points('[data-time-line="block"]'),
};
root.querySelector('[data-gradient-tab="output"]').click();
const outputInitial = {
panel: panel(),
pointCount: points('[data-output-line="block"]').split(" ").length,
bars: root.querySelectorAll("[data-output-bars] i").length,
boundaries: root.querySelectorAll("[data-output-groups] .group-boundary").length,
copy: text("[data-output-copy]"),
};
root.querySelector('[data-output-depth="16"]').click();
const outputDepth16 = {
pointCount: points('[data-output-line="block"]').split(" ").length,
bars: root.querySelectorAll("[data-output-bars] i").length,
copy: text("[data-output-copy]"),
};
root.querySelector('[data-gradient-tab="verdict"]').click();
const verdict = {
panel: panel(),
rows: root.querySelectorAll(".verdict-table tbody tr").length,
metrics: root.querySelectorAll(".metric-pairs article").length,
costs: root.querySelectorAll(".cost-compare article").length,
hashes: root.querySelectorAll(".hash-ledger code").length,
exact: root.textContent.includes("model + optimizer exact") &&
root.textContent.includes("all frozen fields exact"),
mixed: root.textContent.includes("depth-dependent or inconclusive"),
};
const first = root.querySelector('[data-gradient-tab="definition"]');
first.focus();
first.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowRight", bubbles: true }));
const keyboard = {
selected: root.querySelector('[data-gradient-tab][aria-selected="true"]').dataset.gradientTab,
panel: panel(),
};
return {
initial, spectrumInitial, normalized, depth16, seedStep,
timelineInitial, timelineChanged, outputInitial, outputDepth16,
verdict, keyboard,
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
rootOverflow: root.scrollWidth - root.clientWidth,
};
})()`);
await pause(180);
await screenshot("/tmp/llm-atlas-k3-attnres-gradient-desktop.png");
await command("Emulation.setDeviceMetricsOverride", {
width: 390,
height: 844,
deviceScaleFactor: 1,
mobile: true,
});
await navigate("/k3/");
const mobile = await evaluate(`(() => {
const root = document.querySelector("[data-gradient-lab]");
root.scrollIntoView({ block: "start", behavior: "instant" });
window.scrollBy(0, -64);
root.querySelector('[data-gradient-tab="spectrum"]').click();
root.querySelector('[data-spectrum-depth="16"]').click();
root.querySelector('[data-gradient-tab="output"]').click();
root.querySelector('[data-output-depth="16"]').click();
return {
tabs: root.querySelectorAll("[data-gradient-tab]").length,
ledger: root.querySelectorAll(".gradient-ledger article").length,
outputBars: root.querySelectorAll("[data-output-bars] i").length,
visiblePanel: root.querySelector("[data-gradient-panel]:not([hidden])")?.dataset.gradientPanel,
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
rootOverflow: root.scrollWidth - root.clientWidth,
};
})()`);
await pause(180);
await screenshot("/tmp/llm-atlas-k3-attnres-gradient-mobile.png");
const report = { desktop, mobile, exceptions };
console.log(JSON.stringify(report, null, 2));
const numeric = (value) => Number.parseFloat(value.replace("−", "-"));
const failures = [];
if (desktop.initial.panel !== "definition" || desktop.initial.tabs !== 5 || desktop.initial.panels !== 5 || desktop.initial.ledger !== 6) failures.push("五视图初始结构异常");
if (!desktop.initial.definition) failures.push("定义与对象边界缺失");
if (desktop.spectrumInitial.panel !== "spectrum" || Math.abs(numeric(desktop.spectrumInitial.baseCv) - 0.3786) > 1e-4 || Math.abs(numeric(desktop.spectrumInitial.blockCv) - 0.5996) > 1e-4) failures.push("depth-32 final spectrum 读数异常");
if (desktop.spectrumInitial.boundaries !== 7 || desktop.spectrumInitial.baseLine === desktop.normalized.baseLine || !desktop.normalized.title.includes("NORMALIZED")) failures.push("绝对/归一化谱或组边界异常");
if (desktop.depth16.pointCount !== 16 || desktop.depth16.boundaries !== 7 || Math.abs(numeric(desktop.depth16.baseCv) - 0.4246) > 1e-4 || Math.abs(numeric(desktop.depth16.blockCv) - 0.4678) > 1e-4) failures.push("depth-16 spectrum 切换异常");
if (!desktop.seedStep.state.includes("2026073001") || !desktop.seedStep.state.includes("2,000") || Math.abs(numeric(desktop.seedStep.baseCv) - 0.4101) > 1e-4 || Math.abs(numeric(desktop.seedStep.blockCv) - 0.3656) > 1e-4) failures.push("seed/checkpoint spectrum 切换异常");
if (desktop.timelineInitial.panel !== "timeline" || desktop.timelineInitial.pointCount !== 6 || desktop.timelineInitial.line === desktop.timelineChanged.line || !desktop.timelineChanged.title.includes("FIRST / LAST")) failures.push("六时点轨迹指标切换异常");
if (desktop.outputInitial.panel !== "output" || desktop.outputInitial.pointCount !== 32 || desktop.outputInitial.bars !== 32 || desktop.outputInitial.boundaries !== 7 || desktop.outputDepth16.pointCount !== 16 || desktop.outputDepth16.bars !== 16 || !desktop.outputDepth16.copy.includes("DEPTH 16")) failures.push("Output RMS 深度/组节律切换异常");
if (desktop.verdict.panel !== "verdict" || desktop.verdict.rows !== 2 || desktop.verdict.metrics !== 4 || desktop.verdict.costs !== 3 || desktop.verdict.hashes !== 4 || !desktop.verdict.exact || !desktop.verdict.mixed) failures.push("联合判定、成本或重放视图异常");
if (desktop.keyboard.selected !== "spectrum" || desktop.keyboard.panel !== "spectrum") failures.push("键盘 tab 导航异常");
if (desktop.documentOverflow > 1 || desktop.rootOverflow > 1 || mobile.documentOverflow > 1 || mobile.rootOverflow > 1) failures.push("桌面或移动端出现文档级横向溢出");
if (mobile.tabs !== 5 || mobile.ledger !== 6 || mobile.outputBars !== 16 || mobile.visiblePanel !== "output") failures.push("移动端交互结构异常");
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
if (failures.length) {
console.error(`\nFAIL\n- ${failures.join("\n- ")}`);
process.exitCode = 1;
} else {
console.log("\nPASS K3 AttnRes gradient browser regression");
}
socket.close();
@@ -0,0 +1,68 @@
import { createHash } from "node:crypto";
import { readdirSync, readFileSync } from "node:fs";
const read = (path) => {
const bytes = readFileSync(new URL(path, import.meta.url));
return {
bytes,
json: JSON.parse(bytes),
sha256: createHash("sha256").update(bytes).digest("hex"),
};
};
const raw = read("../src/data/k3-attnres-gradient.json");
const compact = read("../src/data/k3-attnres-gradient-compact.json");
const reproduction = read("../experiments/k3/attnres_gradient/reproduction.json");
const rawDirectory = new URL("../experiments/k3/attnres_gradient/results/raw/", import.meta.url);
const failures = [];
const expect = (condition, message) => {
if (!condition) failures.push(message);
};
expect(raw.sha256 === "ad461cbe74fc671f356288d37c9b628618814915116e4fbe03aaaa48367e6a8d", "raw aggregate SHA-256 changed");
expect(compact.sha256 === "5377a5e731db2fdc85a0327f05d43f1cd067d3fc34633619d3004f3fc31968e3", "compact payload SHA-256 changed");
expect(reproduction.sha256 === "aedcde6accc6eb1c24b122ffb55706ef620062e848d9fe4c5f622b084a4fdea6", "reproduction payload SHA-256 changed");
expect(compact.json.protocol_id === "llm-atlas-k3-attnres-gradient-scale-v1", "protocol identity mismatch");
expect(compact.json.study.formal_runs === 12, "formal grid is not 12 cells");
expect(compact.json.study.formal_target_bytes === 786_432_000, "formal target-byte budget mismatch");
expect(compact.json.study.replay_target_bytes === 65_536_000, "replay byte budget mismatch");
expect(compact.json.cells.length === 12, "compact cell count mismatch");
expect(Object.keys(raw.json.runs).length === 12, "raw formal cell count mismatch");
expect(readdirSync(rawDirectory).filter((name) => name.endsWith(".json")).length === 21, "public raw output count is not 21");
expect(reproduction.json.replay_exact.exact, "full formal replay is not exact");
expect(Object.values(reproduction.json.smoke_exact).every((row) => row.exact && row.gradient_gate.passed), "paired smoke or loss-scale gate failed");
for (const depth of ["16", "32"]) {
const summary = compact.json.depth_summaries[depth];
expect(summary.verdict.label === "mixed / inconclusive at this depth", `depth ${depth} verdict changed`);
expect(summary.by_seed.length === 3, `depth ${depth} seed count changed`);
expect(summary.by_seed.every((row) => row.block_minus_baseline_bpc < 0), `depth ${depth} BPC pairing changed`);
expect(summary.by_seed.every((row) => row.relative_cv_reduction < 0), `depth ${depth} CV counterevidence changed`);
expect(summary.by_seed.every((row) => row.relative_imbalance_reduction > 0), `depth ${depth} first/last improvement changed`);
}
expect(Math.abs(compact.json.depth_summaries["16"].means.relative_cv_reduction - (-0.10264135379685868)) < 1e-15, "depth-16 CV contrast changed");
expect(Math.abs(compact.json.depth_summaries["32"].means.relative_cv_reduction - (-0.600330100169428)) < 1e-15, "depth-32 CV contrast changed");
expect(Math.abs(compact.json.depth_summaries["16"].means.relative_imbalance_reduction - 0.6099652484299795) < 1e-15, "depth-16 imbalance contrast changed");
expect(Math.abs(compact.json.depth_summaries["32"].means.relative_imbalance_reduction - 0.7200517407719272) < 1e-15, "depth-32 imbalance contrast changed");
expect(compact.json.overall_verdict === "depth-dependent or inconclusive", "overall preregistered verdict changed");
if (failures.length) {
console.error(`FAIL K3 AttnRes gradient data\n- ${failures.join("\n- ")}`);
process.exit(1);
}
console.log(JSON.stringify({
protocol: compact.json.protocol_id,
formalRuns: compact.json.study.formal_runs,
formalTargetBytes: compact.json.study.formal_target_bytes,
depth16: compact.json.depth_summaries["16"].verdict.label,
depth32: compact.json.depth_summaries["32"].verdict.label,
replayExact: reproduction.json.replay_exact.exact,
hashes: {
raw: raw.sha256,
compact: compact.sha256,
reproduction: reproduction.sha256,
},
}, null, 2));
console.log("PASS K3 AttnRes gradient frozen data");
+7 -3
View File
@@ -81,6 +81,8 @@ const overview = await evaluate(`(() => ({
artifactMismatch: document.querySelector("#artifacts")?.textContent.includes("A_log [128] ≠ expected [96]"), artifactMismatch: document.querySelector("#artifacts")?.textContent.includes("A_log [128] ≠ expected [96]"),
attnresTabs: document.querySelectorAll("[data-attnres-tab]").length, attnresTabs: document.querySelectorAll("[data-attnres-tab]").length,
attnresPanels: document.querySelectorAll("[data-attnres-panel]").length, attnresPanels: document.querySelectorAll("[data-attnres-panel]").length,
gradientTabs: document.querySelectorAll("[data-gradient-tab]").length,
gradientPanels: document.querySelectorAll("[data-gradient-panel]").length,
nativeVisionCorrected: document.body.textContent.includes("MoonViT‑V2 从头训练") && nativeVisionCorrected: document.body.textContent.includes("MoonViT‑V2 从头训练") &&
document.body.textContent.includes("同一个 next-token prediction objective"), document.body.textContent.includes("同一个 next-token prediction objective"),
staleVisionClaim: document.body.textContent.includes("先固定语言模型训练视觉组件"), staleVisionClaim: document.body.textContent.includes("先固定语言模型训练视觉组件"),
@@ -290,8 +292,9 @@ const mobile = await evaluate(`(() => {
artifactTabs: document.querySelectorAll("[data-artifact-tab]").length, artifactTabs: document.querySelectorAll("[data-artifact-tab]").length,
artifactLayers: document.querySelectorAll("[data-layer-cell]").length, artifactLayers: document.querySelectorAll("[data-layer-cell]").length,
attnresTabs: document.querySelectorAll("[data-attnres-tab]").length, attnresTabs: document.querySelectorAll("[data-attnres-tab]").length,
gradientTabs: document.querySelectorAll("[data-gradient-tab]").length,
offenders: [...document.querySelectorAll("body *")] offenders: [...document.querySelectorAll("body *")]
.filter((node) => !node.closest(".paper-chain, .spec-table-wrap, .cache-strip, .architecture-explorer, [data-k3-lab], [data-k3-artifact-lab], [data-attnres-lab]")) .filter((node) => !node.closest(".paper-chain, .spec-table-wrap, .cache-strip, .architecture-explorer, [data-k3-lab], [data-k3-artifact-lab], [data-attnres-lab], [data-gradient-lab]"))
.filter((node) => node.getBoundingClientRect().right > document.documentElement.clientWidth + 1) .filter((node) => node.getBoundingClientRect().right > document.documentElement.clientWidth + 1)
.slice(0, 15) .slice(0, 15)
.map((node) => ({ .map((node) => ({
@@ -322,12 +325,13 @@ console.log(JSON.stringify(report, null, 2));
const numeric = (text) => Number.parseFloat(text.replaceAll(",", "").replace("−", "-")); const numeric = (text) => Number.parseFloat(text.replaceAll(",", "").replace("−", "-"));
const failures = []; const failures = [];
if (!overview.title.includes("因果环节")) failures.push("K3 二轮标题异常"); if (!overview.title.includes("因果环节")) failures.push("K3 二轮标题异常");
if (overview.sections !== 33 || overview.tocLinks !== 33) failures.push("32 个编号专题加阅读链的目录结构异常"); if (overview.sections !== 34 || overview.tocLinks !== 34) failures.push("33 个编号专题加阅读链的目录结构异常");
if (overview.ledgers !== 32 || overview.reportMap !== 9) failures.push("32 张问题账或报告地图异常"); if (overview.ledgers !== 32 || overview.reportMap !== 9) failures.push("32 张问题账或报告地图异常");
if (overview.figureAtlas !== 21 || overview.paperLinks !== 100 || overview.paperGroups < 12) failures.push("图表审计或 100 节点阅读链异常"); if (overview.figureAtlas !== 21 || overview.paperLinks !== 100 || overview.paperGroups < 12) failures.push("图表审计或 100 节点阅读链异常");
if (overview.labTabs !== 8 || overview.labPanels !== 8) failures.push("八联实验结构异常"); if (overview.labTabs !== 8 || overview.labPanels !== 8) failures.push("八联实验结构异常");
if (overview.artifactTabs !== 4 || overview.artifactPanels !== 4 || overview.artifactLayers !== 93 || !overview.artifactMismatch) failures.push("开放工件四视图、93 层条带或形状冲突异常"); if (overview.artifactTabs !== 4 || overview.artifactPanels !== 4 || overview.artifactLayers !== 93 || !overview.artifactMismatch) failures.push("开放工件四视图、93 层条带或形状冲突异常");
if (overview.attnresTabs !== 5 || overview.attnresPanels !== 5) failures.push("AttnRes 独立实验五视图异常"); if (overview.attnresTabs !== 5 || overview.attnresPanels !== 5) failures.push("AttnRes 独立实验五视图异常");
if (overview.gradientTabs !== 5 || overview.gradientPanels !== 5) failures.push("AttnRes 梯度定义扩展五视图异常");
if (!overview.nativeVisionCorrected || overview.staleVisionClaim) failures.push("原生多模态纠错未生效或旧错误残留"); if (!overview.nativeVisionCorrected || overview.staleVisionClaim) failures.push("原生多模态纠错未生效或旧错误残留");
if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出"); if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出");
if (labs.memoryInitial.panel !== "memory" || numeric(labs.memoryInitial.additiveError) <= numeric(labs.memoryInitial.deltaError)) failures.push("Delta memory 初始递推异常"); if (labs.memoryInitial.panel !== "memory" || numeric(labs.memoryInitial.additiveError) <= numeric(labs.memoryInitial.deltaError)) failures.push("Delta memory 初始递推异常");
@@ -352,7 +356,7 @@ if (artifacts.parameterChanged.shape !== "[96,128] F32" || !artifacts.parameterC
if (artifacts.reproductionInitial.panel !== "reproduction" || numeric(artifacts.reproductionInitial.speedup) !== 1.85 || numeric(artifacts.reproductionInitial.localMean) < 2.6 || !artifacts.reproductionInitial.exactSuite || numeric(artifacts.reproductionInitial.cv) < 2) failures.push("FlashKDA H20、本机 exact suite 或 router 初始探针异常"); if (artifacts.reproductionInitial.panel !== "reproduction" || numeric(artifacts.reproductionInitial.speedup) !== 1.85 || numeric(artifacts.reproductionInitial.localMean) < 2.6 || !artifacts.reproductionInitial.exactSuite || numeric(artifacts.reproductionInitial.cv) < 2) failures.push("FlashKDA H20、本机 exact suite 或 router 初始探针异常");
if (numeric(artifacts.reproductionChanged.speedup) !== 3.27 || numeric(artifacts.reproductionChanged.flash) !== 0.7064 || numeric(artifacts.reproductionChanged.localMean) >= numeric(artifacts.reproductionInitial.localMean) || !artifacts.reproductionChanged.localMode.includes("FP32 state") || numeric(artifacts.reproductionChanged.cv) <= numeric(artifacts.reproductionInitial.cv) || numeric(artifacts.reproductionChanged.zero) <= numeric(artifacts.reproductionInitial.zero)) failures.push("GB200 benchmark、本机 varlen/state 或 synthetic router counterexample 未更新"); if (numeric(artifacts.reproductionChanged.speedup) !== 3.27 || numeric(artifacts.reproductionChanged.flash) !== 0.7064 || numeric(artifacts.reproductionChanged.localMean) >= numeric(artifacts.reproductionInitial.localMean) || !artifacts.reproductionChanged.localMode.includes("FP32 state") || numeric(artifacts.reproductionChanged.cv) <= numeric(artifacts.reproductionInitial.cv) || numeric(artifacts.reproductionChanged.zero) <= numeric(artifacts.reproductionInitial.zero)) failures.push("GB200 benchmark、本机 varlen/state 或 synthetic router counterexample 未更新");
if (artifacts.keyboardSelected !== "tensors" || artifacts.keyboardVisible !== "tensors") failures.push("开放工件键盘 tab 导航异常"); if (artifacts.keyboardSelected !== "tensors" || artifacts.keyboardVisible !== "tensors") failures.push("开放工件键盘 tab 导航异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 8 || mobile.artifactTabs !== 4 || mobile.artifactLayers !== 93 || mobile.attnresTabs !== 5) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 8 || mobile.artifactTabs !== 4 || mobile.artifactLayers !== 93 || mobile.attnresTabs !== 5 || mobile.gradientTabs !== 5) failures.push("移动端导航或实验异常");
if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`); if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`);
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`); if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
+813
View File
@@ -0,0 +1,813 @@
---
import rawLab from "@/data/k3-attnres-gradient-compact.json";
const lab = rawLab as any;
const json = JSON.stringify(lab).replaceAll("<", "\\u003c");
const depth16 = lab.depth_summaries["16"];
const depth32 = lab.depth_summaries["32"];
const formatSigned = (value: number, digits = 4) =>
`${value < 0 ? "−" : value > 0 ? "+" : ""}${Math.abs(value).toFixed(digits)}`;
const pct = (value: number, digits = 1) => `${(value * 100).toFixed(digits)}%`;
const shortHash = (value: string) => `${value.slice(0, 10)}…${value.slice(-8)}`;
---
<figure class="gradient-lab" data-gradient-lab>
<header class="gradient-head">
<div>
<p>ROUND 05 / GRADIENT DEFINITION × DEPTH SCALE</p>
<h3>“早层不再过大”与“整条谱更均匀”不是同一件事</h3>
</div>
<p>
16 / 32 Transformer blocks · Baseline / Block · 3 seeds · 8,000 steps。
被测对象是 post-MLP output activation gradient,不冒充论文未公开的 Figure 5 实现。
</p>
</header>
<div class="gradient-ledger">
<article><span>FORMAL GRID</span><b>2 × 2 × 3</b><p>12 个独立训练格</p></article>
<article><span>TARGET BYTES</span><b>786.432M</b><p>每格 65,536,000</p></article>
<article><span>DEPTH</span><b>16 → 32</b><p>宽度固定 192</p></article>
<article class="split"><span>CV</span><b>6 / 6 更差</b><p>中后段局部尖峰</p></article>
<article class="pass"><span>FIRST ↔ LAST</span><b>6 / 6 更近</b><p>失衡改善 56%–81%</p></article>
<article class="pass"><span>FULL REPLAY</span><b>exact</b><p>model + optimizer state</p></article>
</div>
<div class="gradient-tabs" role="tablist" aria-label="选择 AttnRes 梯度定义与深度实验视图">
<button type="button" role="tab" data-gradient-tab="definition" aria-selected="true">
<span>01</span><b>论文到底定义了什么</b><small>known · underdefined · operationalization</small>
</button>
<button type="button" role="tab" data-gradient-tab="spectrum" aria-selected="false">
<span>02</span><b>绝对谱与归一化谱</b><small>depth · seed · checkpoint</small>
</button>
<button type="button" role="tab" data-gradient-tab="timeline" aria-selected="false">
<span>03</span><b>六个时点怎样演化</b><small>CV · imbalance · mean scale</small>
</button>
<button type="button" role="tab" data-gradient-tab="output" aria-selected="false">
<span>04</span><b>Output RMS 与块节律</b><small>growth · reset · group boundary</small>
</button>
<button type="button" role="tab" data-gradient-tab="verdict" aria-selected="false">
<span>05</span><b>联合判定、成本与重放</b><small>activation ≠ parameter</small>
</button>
</div>
<section class="gradient-panel" data-gradient-panel="definition">
<div class="panel-lead">
<div><span>I / DEFINITION AUDIT</span><h4>论文画出一条“梯度曲线”,却没有给出足以唯一重算的测量合同</h4></div>
<p>
Figure 5(c) 只写 “Each transformer block’s gradient magnitude”。
官方仓库没有训练代码、checkpoint、统计脚本或原始数组。
</p>
</div>
<div class="known-grid">
<article class="known">
<span>OFFICIAL / KNOWN</span>
<h5>图与文字能确认</h5>
<ul>
<li>横轴是 Transformer block index。</li>
<li>Figure 5(b) 同时画 block output magnitude。</li>
<li>正文说 Baseline 早层梯度过大,Block 更均匀。</li>
<li>最终模型约 27 blocks / 54 residual layers。</li>
</ul>
</article>
<article class="unknown">
<span>OFFICIAL / UNDERDEFINED</span>
<h5>无法从公开工件唯一恢复</h5>
<ul>
<li>activation、branch 还是 parameter gradient?</li>
<li>L2、RMS、mean absolute 还是别的 norm?</li>
<li>batch / token / channel 怎样 reduction?</li>
<li>哪个 checkpoint、AMP / clip 前还是后?</li>
</ul>
</article>
</div>
<div class="object-chain" aria-label="Round 05 activation gradient 测量对象">
<div><span>FIXED INPUT</span><b>16 × 256 bytes</b><p>同一 diagnostic tensor</p></div>
<i>→</i>
<div><span>POST-MLP OUTPUT</span><b>h₁ … h<sub>L</sub></b><p>每个 Transformer block 一个</p></div>
<i>→</i>
<div><span>TOKEN-MEAN CE</span><b>ℒ</b><p>FP32 cross entropy</p></div>
<i>→</i>
<div class="accent"><span>MEASURE</span><b>RMS(∂ℒ/∂h<sub>l</sub>)</b><p>B × T × C 联合 RMS</p></div>
</div>
<div class="object-compare">
<article><span>ROUND 04</span><b>∇<sub>θl</sub>ℒ</b><p>核心参数梯度:权重收到多大更新信号。</p></article>
<i>≠</i>
<article><span>ROUND 05</span><b>∂ℒ/∂h<sub>l</sub></b><p>activation gradient:损失对这一深度表示有多敏感。</p></article>
<p>两种对象都公开;新指标不会覆盖上一轮反结果。</p>
</div>
<div class="definition-boundary">
<b>能说</b><p>“这是与 Figure 5 叙述对齐的一种公开 operationalization。”</p>
<b>不能说</b><p>“论文作者就是这样算的,或本站复画了 Figure 5(c)。”</p>
</div>
</section>
<section class="gradient-panel" data-gradient-panel="spectrum" hidden>
<div class="panel-lead">
<div><span>II / ALIGNED DEPTH SPECTRUM</span><h4>同一条梯度谱,绝对值与归一化形状要一起看</h4></div>
<p>竖线是 8 个 AttnRes aggregation groups 的边界;Baseline 也画同位置,方便逐层配对。</p>
</div>
<div class="lab-controls">
<div role="group" aria-label="选择深度">
<button type="button" data-spectrum-depth="16" aria-pressed="false">DEPTH 16</button>
<button type="button" data-spectrum-depth="32" aria-pressed="true">DEPTH 32</button>
</div>
<label>SEED
<select data-spectrum-seed>
<option value="mean">3-SEED MEAN</option>
{lab.study.seeds.map((seed: number) => <option value={String(seed)}>{seed}</option>)}
</select>
</label>
<label>CHECKPOINT
<select data-spectrum-step>
{lab.study.diagnostic_steps.map((step: number) => <option value={String(step)} selected={step === 8000}>STEP {step.toLocaleString("en-US")}</option>)}
</select>
</label>
<div role="group" aria-label="选择绝对或归一化梯度谱">
<button type="button" data-spectrum-scale="absolute" aria-pressed="true">ABSOLUTE RMS</button>
<button type="button" data-spectrum-scale="normalized" aria-pressed="false">÷ LAYER MEAN</button>
</div>
</div>
<div class="spectrum-layout">
<div class="chart-shell">
<header><span data-spectrum-title>ACTIVATION GRADIENT RMS</span><b>POST-MLP BLOCK OUTPUT</b></header>
<svg data-spectrum-chart viewBox="0 0 920 360" role="img" aria-label="Baseline 与 Block 的逐层 activation gradient 谱">
<g data-chart-groups></g>
<g data-chart-grid></g>
<polyline data-chart-line="baseline" class="series baseline"></polyline>
<polyline data-chart-line="block" class="series block"></polyline>
<g data-chart-points="baseline"></g>
<g data-chart-points="block"></g>
<text x="460" y="350" class="axis-title">TRANSFORMER BLOCK INDEX</text>
</svg>
<div class="chart-legend">
<span><i class="baseline"></i>Baseline</span>
<span><i class="block"></i>Block AttnRes</span>
<span><i class="boundary"></i>aggregation boundary</span>
</div>
</div>
<div class="spectrum-readout">
<span data-spectrum-state>DEPTH 32 · 3-SEED MEAN · STEP 8,000</span>
<article><b>POPULATION CV</b><div><span>BASE</span><strong data-spectrum-base-cv>0.3786</strong></div><div><span>BLOCK</span><strong data-spectrum-block-cv>0.5996</strong></div></article>
<article><b>FIRST / LAST QUARTILE</b><div><span>BASE</span><strong data-spectrum-base-ratio>3.21×</strong></div><div><span>BLOCK</span><strong data-spectrum-block-ratio>0.95×</strong></div></article>
<article><b>MEAN ABSOLUTE SCALE</b><div><span>BASE</span><strong data-spectrum-base-mean>1.44e−4</strong></div><div><span>BLOCK</span><strong data-spectrum-block-mean>0.78e−4</strong></div></article>
</div>
</div>
<div class="split-result">
<article><span>FIRST ↔ LAST</span><b>更接近</b><p>Baseline 的早层整体隆起被削弱。</p></article>
<i>但</i>
<article><span>ALL-LAYER CV</span><b>反而更高</b><p>中后段少数位置形成更尖的峰。</p></article>
<i>所以</i>
<article class="accent"><span>PRE-REGISTERED</span><b>mixed</b><p>“更均匀”必须拆成至少两个指标。</p></article>
</div>
</section>
<section class="gradient-panel" data-gradient-panel="timeline" hidden>
<div class="panel-lead">
<div><span>III / SIX FROZEN CHECKPOINTS</span><h4>Block 的中期优势会反转;不能挑一个 checkpoint 讲故事</h4></div>
<p>横轴按六个预注册诊断时点等距排列;标签保留真实 step,不暗示实际时间等距。</p>
</div>
<div class="lab-controls">
<div role="group" aria-label="选择时间轨迹深度">
<button type="button" data-time-depth="16" aria-pressed="false">DEPTH 16</button>
<button type="button" data-time-depth="32" aria-pressed="true">DEPTH 32</button>
</div>
<label>SEED
<select data-time-seed>
<option value="mean">3-SEED MEAN</option>
{lab.study.seeds.map((seed: number) => <option value={String(seed)}>{seed}</option>)}
</select>
</label>
<div role="group" aria-label="选择时间轨迹指标">
<button type="button" data-time-metric="cv" aria-pressed="true">CV</button>
<button type="button" data-time-metric="imbalance" aria-pressed="false">FIRST/LAST IMBALANCE</button>
<button type="button" data-time-metric="mean" aria-pressed="false">ABSOLUTE MEAN</button>
</div>
</div>
<div class="chart-shell timeline-chart">
<header><span data-time-title>POPULATION CV</span><b data-time-copy>DEPTH 32 · 3-SEED MEAN</b></header>
<svg data-time-chart viewBox="0 0 920 360" role="img" aria-label="六个固定训练时点的 activation gradient 指标轨迹">
<g data-time-grid></g>
<polyline data-time-line="baseline" class="series baseline"></polyline>
<polyline data-time-line="block" class="series block"></polyline>
<g data-time-points="baseline"></g>
<g data-time-points="block"></g>
<text x="460" y="350" class="axis-title">PREREGISTERED DIAGNOSTIC STEP</text>
</svg>
<div class="chart-legend"><span><i class="baseline"></i>Baseline</span><span><i class="block"></i>Block AttnRes</span></div>
</div>
<div class="timeline-notes">
<article><span>SEED 01 / DEPTH 32</span><b>0.344 → 0.538</b><p>step 2,000:Block 已显著更尖。</p></article>
<article><span>SEED 02 / DEPTH 32</span><b>0.352 → 0.618</b><p>同一时点复现恶化方向。</p></article>
<article class="counter"><span>SEED 03 / DEPTH 32</span><b>0.400 → 0.349</b><p>中期反例:Block 此时反而更平。</p></article>
<article><span>SEED 03 / FINAL</span><b>0.403 → 0.427</b><p>到 8,000 step 才轻微反转。</p></article>
</div>
</section>
<section class="gradient-panel" data-gradient-panel="output" hidden>
<div class="panel-lead">
<div><span>IV / OUTPUT MAGNITUDE</span><h4>梯度结论 mixed,不代表论文所有训练动力学叙述都没有出现</h4></div>
<p>同一个 post-MLP 位置计算 output RMS;这里不做 backward,也不改变主判定。</p>
</div>
<div class="lab-controls">
<div role="group" aria-label="选择 output RMS 深度">
<button type="button" data-output-depth="16" aria-pressed="false">DEPTH 16</button>
<button type="button" data-output-depth="32" aria-pressed="true">DEPTH 32</button>
</div>
<label>SEED
<select data-output-seed>
<option value="mean">3-SEED MEAN</option>
{lab.study.seeds.map((seed: number) => <option value={String(seed)}>{seed}</option>)}
</select>
</label>
<label>CHECKPOINT
<select data-output-step>
{lab.study.diagnostic_steps.map((step: number) => <option value={String(step)} selected={step === 8000}>STEP {step.toLocaleString("en-US")}</option>)}
</select>
</label>
</div>
<div class="chart-shell">
<header><span>POST-MLP OUTPUT RMS</span><b data-output-copy>DEPTH 32 · 3-SEED MEAN · STEP 8,000</b></header>
<svg data-output-chart viewBox="0 0 920 360" role="img" aria-label="Baseline 与 Block 的逐层 output RMS">
<g data-output-groups></g>
<g data-output-grid></g>
<polyline data-output-line="baseline" class="series baseline"></polyline>
<polyline data-output-line="block" class="series block"></polyline>
<g data-output-points="baseline"></g>
<g data-output-points="block"></g>
<text x="460" y="350" class="axis-title">TRANSFORMER BLOCK INDEX</text>
</svg>
<div class="chart-legend"><span><i class="baseline"></i>Baseline</span><span><i class="block"></i>Block AttnRes</span><span><i class="boundary"></i>aggregation boundary</span></div>
</div>
<div class="output-ratios">
<article><span>DEPTH 16 / LAST ÷ FIRST</span><div><b>Baseline</b><strong>4.59×</strong></div><div><b>Block</b><strong>1.17×</strong></div></article>
<article><span>DEPTH 32 / LAST ÷ FIRST</span><div><b>Baseline</b><strong>6.08×</strong></div><div><b>Block</b><strong>1.89×</strong></div></article>
<article class="accent"><span>WHAT THE CURVE SAYS</span><b>全局累积 → 组内锯齿</b><p>Block 限制 output magnitude 持续跨深度增长;这一方向与 Figure 5(b) 叙述一致。</p></article>
</div>
<div class="group-rhythm">
<header><span data-rhythm-title>BLOCK ATTNRES · DEPTH 32</span><b>8 AGGREGATION GROUPS</b></header>
<div data-output-bars aria-label="Block AttnRes output RMS 组内节律"></div>
<p>粗分隔线是 group boundary;柱高来自当前选择的 checkpoint / seed,不是示意动画。</p>
</div>
</section>
<section class="gradient-panel" data-gradient-panel="verdict" hidden>
<div class="panel-lead">
<div><span>V / JOINT VERDICT</span><h4>一个正结果、两个反结果和一张成本账,要同时摆在桌面上</h4></div>
<p>Round 05 的主判定只读 activation CV + imbalance;BPC、参数梯度与成本是必须公开的次要结果。</p>
</div>
<div class="verdict-table-wrap">
<table class="verdict-table">
<thead><tr><th>Depth</th><th>Δ BPC</th><th>Activation CV</th><th>First/last imbalance</th><th>Mean grad scale</th><th>Parameter CV</th><th>Verdict</th></tr></thead>
<tbody>
<tr>
<th>16</th>
<td>{formatSigned(depth16.means.block_minus_baseline_bpc, 5)}</td>
<td class="bad">{pct(depth16.means.relative_cv_reduction)} reduction</td>
<td class="good">+{pct(depth16.means.relative_imbalance_reduction)}</td>
<td>{pct(depth16.means.block_to_baseline_activation_grad_mean)} of Base</td>
<td>{depth16.means.baseline_parameter_grad_cv.toFixed(3)} → {depth16.means.block_parameter_grad_cv.toFixed(3)}</td>
<td>mixed</td>
</tr>
<tr>
<th>32</th>
<td>{formatSigned(depth32.means.block_minus_baseline_bpc, 5)}</td>
<td class="bad">{pct(depth32.means.relative_cv_reduction)} reduction</td>
<td class="good">+{pct(depth32.means.relative_imbalance_reduction)}</td>
<td>{pct(depth32.means.block_to_baseline_activation_grad_mean)} of Base</td>
<td>{depth32.means.baseline_parameter_grad_cv.toFixed(3)} → {depth32.means.block_parameter_grad_cv.toFixed(3)}</td>
<td>mixed</td>
</tr>
</tbody>
</table>
</div>
<div class="metric-pairs">
<article><span>ACTIVATION / FIRST-LAST</span><b>Block 改善</b><p>6 / 6 配对更接近 1。</p></article>
<article class="warn"><span>ACTIVATION / ALL-LAYER CV</span><b>Block 恶化</b><p>6 / 6 配对 CV 更高。</p></article>
<article class="warn"><span>PARAMETER / ALL-LAYER CV</span><b>Block 恶化</b><p>0.416→0.683;0.397→0.772。</p></article>
<article class="pass"><span>VALIDATION BPC</span><b>Block 更低</b><p>6 / 6 配对为负;非同算力。</p></article>
</div>
<div class="cost-compare">
<article><span>DEPTH 16</span><div><b>STEP TIME</b><strong>21.47 → 54.71 ms</strong></div><div><b>PEAK ALLOC</b><strong>3.04 → 6.54 GB</strong></div><p>约 2.55× time · 2.15× memory</p></article>
<article><span>DEPTH 32</span><div><b>STEP TIME</b><strong>42.11 → 109.38 ms</strong></div><div><b>PEAK ALLOC</b><strong>5.94 → 12.82 GB</strong></div><p>约 2.60× time · 2.16× memory</p></article>
<article class="boundary"><span>CLAIM BOUNDARY</span><b>同 token / step</b><p>不能写成同 FLOPs、同 wall time,或 K3 生产成本。</p></article>
</div>
<div class="replay-ledger">
<article><span>FORMAL</span><b>depth-32 · Block · seed-01</b><p>8,000 steps from initialization</p></article>
<i>≡</i>
<article><span>FRESH REPLAY</span><b>all frozen fields exact</b><p>额外 65,536,000 target bytes</p></article>
<i>→</i>
<article class="pass"><span>STATE HASH</span><b>model + optimizer exact</b><p>{shortHash(lab.cells.find((cell: any) => cell.depth === 32 && cell.architecture === "block" && cell.seed === 2026073001).hashes.final_model_state)}</p></article>
</div>
<div class="hash-ledger">
<article><span>MANIFEST</span><code>{lab.manifest_summary.file_sha256}</code></article>
<article><span>SCHEDULE</span><code>{lab.manifest_summary.formal_schedule_sha256}</code></article>
<article><span>COMPACT CANONICAL</span><code>{lab.canonical_sha256_without_self}</code></article>
<article><span>OVERALL</span><code>{lab.overall_verdict}</code></article>
</div>
<div class="claim-grid">
<article class="yes"><span>THIS STUDY SUPPORTS</span><ul><li>在公开定义下,Block 改写了梯度失衡的形态。</li><li>早/晚深度更接近,但中后段局部峰更尖。</li><li>Output RMS 的跨深度增长显著受限。</li></ul></article>
<article class="no"><span>THIS STUDY DOES NOT SUPPORT</span><ul><li>复现论文 Figure 5(c) 的数值或隐藏实现。</li><li>测到 K3 checkpoint 的真实梯度。</li><li>证明 AttnRes 一般更稳定或更省算力。</li></ul></article>
</div>
</section>
<script is:inline type="application/json" data-gradient-payload set:html={json}></script>
</figure>
<script>
const initializeGradientLab = (root: HTMLElement) => {
if (root.dataset.gradientReady === "true") return;
root.dataset.gradientReady = "true";
const payload = root.querySelector<HTMLScriptElement>("[data-gradient-payload]");
if (!payload?.textContent) return;
const data = JSON.parse(payload.textContent);
const architectures = ["baseline", "block"];
const steps = data.study.diagnostic_steps;
const seeds = data.study.seeds;
const ns = "http://www.w3.org/2000/svg";
const colors: Record<string, string> = { baseline: "#77746b", block: "#ba603b" };
const tabs = [...root.querySelectorAll<HTMLButtonElement>("[data-gradient-tab]")];
const panels = [...root.querySelectorAll<HTMLElement>("[data-gradient-panel]")];
const selectTab = (id: string) => {
tabs.forEach((tab) => tab.setAttribute("aria-selected", String(tab.dataset.gradientTab === id)));
panels.forEach((panel) => { panel.hidden = panel.dataset.gradientPanel !== id; });
};
tabs.forEach((tab, index) => {
tab.addEventListener("click", () => selectTab(tab.dataset.gradientTab || "definition"));
tab.addEventListener("keydown", (event) => {
if (!["ArrowLeft", "ArrowRight", "Home", "End"].includes(event.key)) return;
event.preventDefault();
const nextIndex = event.key === "Home" ? 0
: event.key === "End" ? tabs.length - 1
: (index + (event.key === "ArrowRight" ? 1 : -1) + tabs.length) % tabs.length;
tabs[nextIndex].focus();
selectTab(tabs[nextIndex].dataset.gradientTab || "definition");
});
});
const findCell = (depth: number, architecture: string, seed: number) =>
data.cells.find((cell: any) => cell.depth === depth && cell.architecture === architecture && cell.seed === seed);
const findDiagnostic = (cell: any, step: number) =>
cell.diagnostics.find((row: any) => row.step === step);
const seedList = (seedKey: string) => seedKey === "mean" ? seeds : [Number(seedKey)];
const average = (values: number[]) => values.reduce((sum, value) => sum + value, 0) / values.length;
const arrayFor = (depth: number, architecture: string, seedKey: string, step: number, field: string) => {
const rows = seedList(seedKey).map((seed: number) => findDiagnostic(findCell(depth, architecture, seed), step)[field]);
return rows[0].map((_: number, index: number) => average(rows.map((row: number[]) => row[index])));
};
const statFor = (depth: number, architecture: string, seedKey: string, step: number, group: string, field: string) =>
average(seedList(seedKey).map((seed: number) => findDiagnostic(findCell(depth, architecture, seed), step)[group][field]));
const formatStep = (value: number) => value.toLocaleString("en-US");
const formatScientific = (value: number) => value.toExponential(2).replace("e-", "e−");
const svgNode = (name: string, attrs: Record<string, string | number>) => {
const node = document.createElementNS(ns, name);
Object.entries(attrs).forEach(([key, value]) => node.setAttribute(key, String(value)));
return node;
};
const drawGroups = (group: SVGGElement, depth: number) => {
group.replaceChildren();
for (let index = 1; index < 8; index += 1) {
const block = depth / 8 * index;
const x = 58 + block / (depth - 1) * 822;
group.append(svgNode("line", { x1: x, x2: x, y1: 24, y2: 304, class: "group-boundary" }));
}
};
const drawChart = (
svg: SVGSVGElement,
groups: SVGGElement | null,
grid: SVGGElement,
lines: Record<string, SVGPolylineElement | null>,
pointGroups: Record<string, SVGGElement | null>,
series: Record<string, number[]>,
labels: string[],
options: { depth?: number; formatter?: (value: number) => string; zero?: boolean } = {},
) => {
grid.replaceChildren();
const values = Object.values(series).flat();
const rawMin = options.zero === false ? Math.min(...values) : 0;
const rawMax = Math.max(...values);
const padding = rawMax === rawMin ? 1 : (rawMax - rawMin) * 0.08;
const minimum = Math.max(0, rawMin - padding);
const maximum = rawMax + padding;
const xAt = (index: number) => 58 + index / Math.max(1, labels.length - 1) * 822;
const yAt = (value: number) => 24 + (maximum - value) / Math.max(1e-30, maximum - minimum) * 280;
const formatter = options.formatter || ((value: number) => value.toFixed(2));
for (let index = 0; index < 5; index += 1) {
const value = minimum + (maximum - minimum) * (4 - index) / 4;
const y = 24 + index / 4 * 280;
grid.append(svgNode("line", { x1: 58, x2: 880, y1: y, y2: y, class: "grid-line" }));
const text = svgNode("text", { x: 48, y: y + 4, class: "axis-label y" });
text.textContent = formatter(value);
grid.append(text);
}
labels.forEach((label, index) => {
const text = svgNode("text", { x: xAt(index), y: 326, class: "axis-label x" });
text.textContent = label;
grid.append(text);
});
if (groups && options.depth) drawGroups(groups, options.depth);
architectures.forEach((architecture) => {
const points = series[architecture].map((value, index) => `${xAt(index)},${yAt(value)}`).join(" ");
lines[architecture]?.setAttribute("points", points);
pointGroups[architecture]?.replaceChildren();
series[architecture].forEach((value, index) => {
const point = svgNode("circle", {
cx: xAt(index), cy: yAt(value), r: 3.6, fill: colors[architecture],
});
const title = svgNode("title", {});
title.textContent = `${architecture === "baseline" ? "Baseline" : "Block"} · ${labels[index]} · ${formatter(value)}`;
point.append(title);
pointGroups[architecture]?.append(point);
});
});
svg.dataset.maximum = String(maximum);
};
let spectrumDepth = 32;
let spectrumScale = "absolute";
const spectrumSeed = root.querySelector<HTMLSelectElement>("[data-spectrum-seed]")!;
const spectrumStep = root.querySelector<HTMLSelectElement>("[data-spectrum-step]")!;
const spectrumChart = root.querySelector<SVGSVGElement>("[data-spectrum-chart]")!;
const updateSpectrum = () => {
const seedKey = spectrumSeed.value;
const step = Number(spectrumStep.value);
const absolute = Object.fromEntries(architectures.map((architecture) => [
architecture,
arrayFor(spectrumDepth, architecture, seedKey, step, "activation_grad_rms_by_block"),
]));
const series = spectrumScale === "normalized"
? Object.fromEntries(architectures.map((architecture) => {
const mean = average(absolute[architecture]);
return [architecture, absolute[architecture].map((value: number) => value / mean)];
}))
: absolute;
drawChart(
spectrumChart,
spectrumChart.querySelector("[data-chart-groups]"),
spectrumChart.querySelector("[data-chart-grid]")!,
Object.fromEntries(architectures.map((architecture) => [architecture, spectrumChart.querySelector(`[data-chart-line="${architecture}"]`)])),
Object.fromEntries(architectures.map((architecture) => [architecture, spectrumChart.querySelector(`[data-chart-points="${architecture}"]`)])),
series,
Array.from({ length: spectrumDepth }, (_, index) => String(index + 1)),
{
depth: spectrumDepth,
formatter: spectrumScale === "absolute" ? formatScientific : (value) => `${value.toFixed(1)}×`,
},
);
const title = root.querySelector<HTMLElement>("[data-spectrum-title]");
if (title) title.textContent = spectrumScale === "absolute" ? "ACTIVATION GRADIENT RMS" : "NORMALIZED BY LAYER MEAN";
const state = root.querySelector<HTMLElement>("[data-spectrum-state]");
if (state) state.textContent = `DEPTH ${spectrumDepth} · ${seedKey === "mean" ? "3-SEED MEAN" : seedKey} · STEP ${formatStep(step)}`;
architectures.forEach((architecture) => {
const cv = statFor(spectrumDepth, architecture, seedKey, step, "activation_grad_statistics", "population_cv");
const ratio = statFor(spectrumDepth, architecture, seedKey, step, "activation_grad_statistics", "first_to_last_ratio");
const mean = statFor(spectrumDepth, architecture, seedKey, step, "activation_grad_statistics", "mean");
const prefix = architecture === "baseline" ? "base" : "block";
root.querySelector<HTMLElement>(`[data-spectrum-${prefix}-cv]`)!.textContent = cv.toFixed(4);
root.querySelector<HTMLElement>(`[data-spectrum-${prefix}-ratio]`)!.textContent = `${ratio.toFixed(2)}×`;
root.querySelector<HTMLElement>(`[data-spectrum-${prefix}-mean]`)!.textContent = formatScientific(mean);
});
};
root.querySelectorAll<HTMLButtonElement>("[data-spectrum-depth]").forEach((button) => button.addEventListener("click", () => {
spectrumDepth = Number(button.dataset.spectrumDepth);
root.querySelectorAll<HTMLButtonElement>("[data-spectrum-depth]").forEach((peer) => peer.setAttribute("aria-pressed", String(peer === button)));
updateSpectrum();
}));
root.querySelectorAll<HTMLButtonElement>("[data-spectrum-scale]").forEach((button) => button.addEventListener("click", () => {
spectrumScale = button.dataset.spectrumScale || "absolute";
root.querySelectorAll<HTMLButtonElement>("[data-spectrum-scale]").forEach((peer) => peer.setAttribute("aria-pressed", String(peer === button)));
updateSpectrum();
}));
spectrumSeed.addEventListener("change", updateSpectrum);
spectrumStep.addEventListener("change", updateSpectrum);
updateSpectrum();
let timeDepth = 32;
let timeMetric = "cv";
const timeSeed = root.querySelector<HTMLSelectElement>("[data-time-seed]")!;
const timeChart = root.querySelector<SVGSVGElement>("[data-time-chart]")!;
const timeMetricMap: Record<string, [string, string, (value: number) => string]> = {
cv: ["activation_grad_statistics", "population_cv", (value) => value.toFixed(2)],
imbalance: ["activation_grad_statistics", "imbalance_abs_log_ratio", (value) => value.toFixed(2)],
mean: ["activation_grad_statistics", "mean", formatScientific],
};
const updateTimeline = () => {
const seedKey = timeSeed.value;
const [group, field, formatter] = timeMetricMap[timeMetric];
const series = Object.fromEntries(architectures.map((architecture) => [
architecture,
steps.map((step: number) => statFor(timeDepth, architecture, seedKey, step, group, field)),
]));
drawChart(
timeChart,
null,
timeChart.querySelector("[data-time-grid]")!,
Object.fromEntries(architectures.map((architecture) => [architecture, timeChart.querySelector(`[data-time-line="${architecture}"]`)])),
Object.fromEntries(architectures.map((architecture) => [architecture, timeChart.querySelector(`[data-time-points="${architecture}"]`)])),
series,
steps.map((step: number) => step >= 1000 ? `${step / 1000}K` : String(step)),
{ formatter },
);
const titles: Record<string, string> = {
cv: "POPULATION CV",
imbalance: "ABS(LOG(FIRST / LAST QUARTILE))",
mean: "MEAN ACTIVATION GRADIENT RMS",
};
root.querySelector<HTMLElement>("[data-time-title]")!.textContent = titles[timeMetric];
root.querySelector<HTMLElement>("[data-time-copy]")!.textContent = `DEPTH ${timeDepth} · ${seedKey === "mean" ? "3-SEED MEAN" : seedKey}`;
};
root.querySelectorAll<HTMLButtonElement>("[data-time-depth]").forEach((button) => button.addEventListener("click", () => {
timeDepth = Number(button.dataset.timeDepth);
root.querySelectorAll<HTMLButtonElement>("[data-time-depth]").forEach((peer) => peer.setAttribute("aria-pressed", String(peer === button)));
updateTimeline();
}));
root.querySelectorAll<HTMLButtonElement>("[data-time-metric]").forEach((button) => button.addEventListener("click", () => {
timeMetric = button.dataset.timeMetric || "cv";
root.querySelectorAll<HTMLButtonElement>("[data-time-metric]").forEach((peer) => peer.setAttribute("aria-pressed", String(peer === button)));
updateTimeline();
}));
timeSeed.addEventListener("change", updateTimeline);
updateTimeline();
let outputDepth = 32;
const outputSeed = root.querySelector<HTMLSelectElement>("[data-output-seed]")!;
const outputStep = root.querySelector<HTMLSelectElement>("[data-output-step]")!;
const outputChart = root.querySelector<SVGSVGElement>("[data-output-chart]")!;
const updateOutput = () => {
const seedKey = outputSeed.value;
const step = Number(outputStep.value);
const series = Object.fromEntries(architectures.map((architecture) => [
architecture,
arrayFor(outputDepth, architecture, seedKey, step, "activation_output_rms_by_block"),
]));
drawChart(
outputChart,
outputChart.querySelector("[data-output-groups]"),
outputChart.querySelector("[data-output-grid]")!,
Object.fromEntries(architectures.map((architecture) => [architecture, outputChart.querySelector(`[data-output-line="${architecture}"]`)])),
Object.fromEntries(architectures.map((architecture) => [architecture, outputChart.querySelector(`[data-output-points="${architecture}"]`)])),
series,
Array.from({ length: outputDepth }, (_, index) => String(index + 1)),
{ depth: outputDepth, formatter: (value) => value.toFixed(2) },
);
root.querySelector<HTMLElement>("[data-output-copy]")!.textContent = `DEPTH ${outputDepth} · ${seedKey === "mean" ? "3-SEED MEAN" : seedKey} · STEP ${formatStep(step)}`;
root.querySelector<HTMLElement>("[data-rhythm-title]")!.textContent = `BLOCK ATTNRES · DEPTH ${outputDepth}`;
const bars = root.querySelector<HTMLElement>("[data-output-bars]")!;
const values: number[] = series.block;
const maximum = Math.max(...values);
const blocksPerGroup = outputDepth / 8;
bars.replaceChildren();
values.forEach((value, index) => {
const bar = document.createElement("i");
bar.style.setProperty("--bar", `${value / maximum * 100}%`);
if ((index + 1) % blocksPerGroup === 0) bar.classList.add("boundary");
const label = document.createElement("span");
label.textContent = String(index + 1);
bar.append(label);
bars.append(bar);
});
};
root.querySelectorAll<HTMLButtonElement>("[data-output-depth]").forEach((button) => button.addEventListener("click", () => {
outputDepth = Number(button.dataset.outputDepth);
root.querySelectorAll<HTMLButtonElement>("[data-output-depth]").forEach((peer) => peer.setAttribute("aria-pressed", String(peer === button)));
updateOutput();
}));
outputSeed.addEventListener("change", updateOutput);
outputStep.addEventListener("change", updateOutput);
updateOutput();
};
document.querySelectorAll<HTMLElement>("[data-gradient-lab]").forEach(initializeGradientLab);
document.addEventListener("astro:page-load", () => {
document.querySelectorAll<HTMLElement>("[data-gradient-lab]").forEach(initializeGradientLab);
});
</script>
<style>
.gradient-lab {
--g-ink: #1c201e;
--g-muted: #77746b;
--g-line: rgba(28, 32, 30, .16);
--g-paper: #f4f0e7;
--g-raised: #faf7ef;
--g-copper: #ba603b;
--g-green: #163f3b;
width: min(1120px, 100%);
margin: 42px 0;
color: var(--g-ink);
border: 1px solid var(--g-line);
background: var(--g-paper);
box-shadow: 0 30px 80px rgba(28, 32, 30, .09);
}
.gradient-head {
display: grid;
grid-template-columns: minmax(0, 1.45fr) minmax(260px, .7fr);
gap: 44px;
padding: 30px;
color: #f5efe4;
background: var(--g-green);
}
.gradient-head p { margin: 0; color: rgba(245,239,228,.7); font: .65rem/1.7 var(--mono); }
.gradient-head div > p { color: #d58a68; letter-spacing: .08em; }
.gradient-head h3 { max-width: 720px; margin: 14px 0 0; color: inherit; font-size: clamp(1.15rem, 2.2vw, 1.75rem); line-height: 1.35; }
.gradient-ledger { display: grid; grid-template-columns: repeat(6, 1fr); border-bottom: 1px solid var(--g-line); }
.gradient-ledger article { min-height: 126px; padding: 18px 15px; border-right: 1px solid var(--g-line); }
.gradient-ledger article:last-child { border-right: 0; }
.gradient-ledger span, .panel-lead span { color: var(--g-muted); font: .56rem/1.2 var(--mono); letter-spacing: .08em; }
.gradient-ledger b { display: block; margin-top: 23px; font: 700 .95rem/1 var(--mono); }
.gradient-ledger p { margin: 8px 0 0; color: var(--g-muted); font-size: .6rem; line-height: 1.45; }
.gradient-ledger .pass { color: #f7f0e6; background: var(--g-green); }
.gradient-ledger .split { color: #f7f0e6; background: var(--g-copper); }
.gradient-ledger .pass span, .gradient-ledger .pass p, .gradient-ledger .split span, .gradient-ledger .split p { color: rgba(247,240,230,.72); }
.gradient-tabs { display: grid; grid-template-columns: repeat(5, 1fr); border-bottom: 1px solid var(--g-line); background: #e9e4da; }
.gradient-tabs button { min-height: 116px; padding: 16px; text-align: left; color: inherit; border: 0; border-right: 1px solid var(--g-line); background: transparent; cursor: pointer; }
.gradient-tabs button:last-child { border-right: 0; }
.gradient-tabs button[aria-selected="true"] { color: #f7f0e6; background: var(--g-copper); }
.gradient-tabs span, .gradient-tabs small { display: block; color: var(--g-muted); font: .54rem/1.25 var(--mono); }
.gradient-tabs b { display: block; margin: 15px 0 8px; font-size: .69rem; line-height: 1.35; }
.gradient-tabs button[aria-selected="true"] span, .gradient-tabs button[aria-selected="true"] small { color: rgba(247,240,230,.72); }
.gradient-panel { padding: 30px; }
.panel-lead { display: grid; grid-template-columns: 1.05fr .95fr; gap: 48px; align-items: end; margin-bottom: 28px; }
.panel-lead h4 { max-width: 680px; margin: 10px 0 0; font-size: 1.2rem; line-height: 1.4; }
.panel-lead p { margin: 0; color: var(--g-muted); font-size: .7rem; line-height: 1.7; }
.known-grid { display: grid; grid-template-columns: repeat(2, 1fr); border: 1px solid var(--g-line); }
.known-grid article { min-height: 270px; padding: 24px; }
.known-grid article + article { border-left: 1px solid var(--g-line); background: #ece2d6; }
.known-grid span, .object-chain span, .object-compare span, .split-result span, .timeline-notes span, .output-ratios span, .metric-pairs span, .cost-compare > article > span, .replay-ledger span, .hash-ledger span, .claim-grid span {
color: var(--g-copper); font: .56rem/1 var(--mono); letter-spacing: .06em;
}
.known-grid h5 { margin: 26px 0 16px; font-size: .95rem; }
.known-grid ul { margin: 0; padding-left: 18px; }
.known-grid li { margin-top: 11px; color: var(--g-muted); font-size: .68rem; line-height: 1.55; }
.object-chain { display: grid; grid-template-columns: 1fr 30px 1fr 30px .8fr 30px 1.2fr; gap: 6px; align-items: center; margin-top: 20px; }
.object-chain div { min-height: 145px; padding: 18px; border: 1px solid var(--g-line); background: var(--g-raised); }
.object-chain .accent { color: #f7f0e6; background: var(--g-green); }
.object-chain .accent span, .object-chain .accent p { color: rgba(247,240,230,.68); }
.object-chain b { display: block; margin-top: 25px; font: 700 .77rem/1.35 var(--mono); }
.object-chain p { color: var(--g-muted); font-size: .61rem; line-height: 1.45; }
.object-chain > i, .object-compare > i, .split-result > i, .replay-ledger > i { color: var(--g-copper); font-style: normal; text-align: center; }
.object-compare { display: grid; grid-template-columns: 1fr 50px 1fr; gap: 12px; align-items: center; margin-top: 20px; padding: 20px; background: #e9e4da; }
.object-compare article { padding: 12px; }
.object-compare b { display: block; margin-top: 18px; font: 700 1.1rem/1 var(--mono); }
.object-compare p { color: var(--g-muted); font-size: .65rem; line-height: 1.55; }
.object-compare > p { grid-column: 1/-1; margin: 0; padding-top: 16px; border-top: 1px solid var(--g-line); }
.definition-boundary { display: grid; grid-template-columns: 80px 1fr; margin-top: 20px; border-top: 1px solid var(--g-line); }
.definition-boundary > * { margin: 0; padding: 15px; border-right: 1px solid var(--g-line); border-bottom: 1px solid var(--g-line); }
.definition-boundary b { color: var(--g-copper); font: .6rem/1.4 var(--mono); }
.definition-boundary p { color: var(--g-muted); font-size: .66rem; line-height: 1.55; }
.lab-controls { display: flex; flex-wrap: wrap; gap: 10px 18px; align-items: end; margin-bottom: 20px; }
.lab-controls > div { display: flex; }
.lab-controls button, .lab-controls select { min-height: 38px; padding: 10px 12px; color: var(--g-muted); font: 700 .56rem/1 var(--mono); border: 1px solid var(--g-line); background: var(--g-raised); }
.lab-controls button { cursor: pointer; }
.lab-controls button + button { border-left: 0; }
.lab-controls button[aria-pressed="true"] { color: #fff9ef; background: var(--g-green); }
.lab-controls label { display: grid; gap: 6px; color: var(--g-muted); font: .52rem/1 var(--mono); }
.spectrum-layout { display: grid; grid-template-columns: minmax(0, 1fr) 235px; border: 1px solid var(--g-line); background: var(--g-raised); }
.chart-shell { min-width: 0; padding: 18px; border: 1px solid var(--g-line); background: var(--g-raised); }
.spectrum-layout .chart-shell { border: 0; border-right: 1px solid var(--g-line); }
.chart-shell header { display: flex; justify-content: space-between; gap: 12px; color: var(--g-muted); font: .55rem/1 var(--mono); }
.chart-shell svg { display: block; width: 100%; height: auto; margin-top: 10px; overflow: visible; }
.series { fill: none; stroke-width: 3; stroke-linejoin: round; stroke-linecap: round; }
.series.baseline { stroke: #77746b; }
.series.block { stroke: #ba603b; }
.grid-line { stroke: rgba(28,32,30,.1); stroke-width: 1; }
.group-boundary { stroke: rgba(22,63,59,.22); stroke-width: 1.5; stroke-dasharray: 4 4; }
.axis-label { fill: #8b867c; font: 11px var(--mono); }
.axis-label.y { text-anchor: end; }
.axis-label.x { text-anchor: middle; }
.axis-title { fill: #8b867c; font: 11px var(--mono); text-anchor: middle; }
.chart-legend { display: flex; flex-wrap: wrap; gap: 18px; margin-top: 4px; color: var(--g-muted); font: .56rem/1 var(--mono); }
.chart-legend span { display: inline-flex; gap: 7px; align-items: center; }
.chart-legend i { width: 22px; height: 3px; }
.chart-legend i.baseline { background: #77746b; }
.chart-legend i.block { background: #ba603b; }
.chart-legend i.boundary { height: 0; border-top: 2px dashed var(--g-green); background: transparent; }
.spectrum-readout { padding: 20px 18px; }
.spectrum-readout > span { color: var(--g-copper); font: .54rem/1.4 var(--mono); }
.spectrum-readout article { padding: 18px 0; border-bottom: 1px solid var(--g-line); }
.spectrum-readout article b { color: var(--g-muted); font: .52rem/1 var(--mono); }
.spectrum-readout article div { display: flex; justify-content: space-between; margin-top: 13px; }
.spectrum-readout article span { color: var(--g-muted); font: .5rem/1 var(--mono); }
.spectrum-readout article strong { font: 700 .7rem/1 var(--mono); }
.split-result { display: grid; grid-template-columns: 1fr 45px 1fr 45px 1fr; gap: 8px; align-items: center; margin-top: 20px; }
.split-result article { min-height: 145px; padding: 20px; border: 1px solid var(--g-line); }
.split-result .accent { color: #f7f0e6; background: var(--g-green); }
.split-result .accent span, .split-result .accent p { color: rgba(247,240,230,.68); }
.split-result b { display: block; margin-top: 25px; font-size: .83rem; }
.split-result p { color: var(--g-muted); font-size: .62rem; line-height: 1.5; }
.timeline-chart { margin-top: 0; }
.timeline-notes { display: grid; grid-template-columns: repeat(4, 1fr); margin-top: 20px; border-top: 1px solid var(--g-line); border-left: 1px solid var(--g-line); }
.timeline-notes article { min-height: 145px; padding: 18px; border-right: 1px solid var(--g-line); border-bottom: 1px solid var(--g-line); background: var(--g-raised); }
.timeline-notes .counter { background: #e9e4da; }
.timeline-notes b { display: block; margin-top: 25px; font: 700 .78rem/1 var(--mono); }
.timeline-notes p { color: var(--g-muted); font-size: .61rem; line-height: 1.5; }
.output-ratios { display: grid; grid-template-columns: 1fr 1fr 1.25fr; margin-top: 20px; border-top: 1px solid var(--g-line); border-left: 1px solid var(--g-line); }
.output-ratios article { min-height: 170px; padding: 20px; border-right: 1px solid var(--g-line); border-bottom: 1px solid var(--g-line); }
.output-ratios article > div { display: flex; justify-content: space-between; margin-top: 24px; }
.output-ratios article > div b { color: var(--g-muted); font: .55rem/1 var(--mono); }
.output-ratios article > div strong { font: 700 .75rem/1 var(--mono); }
.output-ratios .accent { color: #f7f0e6; background: var(--g-green); }
.output-ratios .accent span, .output-ratios .accent p { color: rgba(247,240,230,.68); }
.output-ratios .accent b { display: block; margin-top: 25px; font-size: .8rem; }
.output-ratios p { color: var(--g-muted); font-size: .62rem; line-height: 1.5; }
.group-rhythm { margin-top: 20px; padding: 20px; color: #f7f0e6; background: var(--g-green); overflow: hidden; }
.group-rhythm header { display: flex; justify-content: space-between; color: rgba(247,240,230,.68); font: .55rem/1 var(--mono); }
.group-rhythm > div { display: grid; grid-template-columns: repeat(auto-fit, minmax(8px, 1fr)); align-items: end; height: 170px; margin-top: 18px; border-bottom: 1px solid rgba(255,255,255,.3); }
.group-rhythm i { position: relative; display: block; height: max(5px, var(--bar)); margin-right: 2px; background: #d4835d; }
.group-rhythm i.boundary { margin-right: 8px; border-right: 2px solid rgba(255,255,255,.75); }
.group-rhythm i span { position: absolute; bottom: -18px; left: 50%; color: rgba(255,255,255,.55); font: .43rem/1 var(--mono); transform: translateX(-50%); }
.group-rhythm p { margin: 35px 0 0; color: rgba(247,240,230,.72); font-size: .64rem; }
.verdict-table-wrap { overflow-x: auto; }
.verdict-table { width: 100%; min-width: 850px; border-collapse: collapse; font-size: .62rem; }
.verdict-table th, .verdict-table td { padding: 15px 12px; border-bottom: 1px solid var(--g-line); text-align: left; }
.verdict-table thead th { color: var(--g-muted); font: .53rem/1.3 var(--mono); }
.verdict-table tbody th, .verdict-table tbody td:last-child { font: 700 .66rem/1 var(--mono); }
.verdict-table .good { color: var(--g-green); }
.verdict-table .bad { color: var(--g-copper); }
.metric-pairs { display: grid; grid-template-columns: repeat(4, 1fr); margin-top: 20px; border-top: 1px solid var(--g-line); border-left: 1px solid var(--g-line); }
.metric-pairs article { min-height: 150px; padding: 18px; border-right: 1px solid var(--g-line); border-bottom: 1px solid var(--g-line); background: var(--g-raised); }
.metric-pairs article.warn { border-top: 4px solid var(--g-copper); }
.metric-pairs article.pass { border-top: 4px solid var(--g-green); }
.metric-pairs b { display: block; margin-top: 25px; font-size: .77rem; }
.metric-pairs p { color: var(--g-muted); font-size: .61rem; line-height: 1.5; }
.cost-compare { display: grid; grid-template-columns: repeat(3, 1fr); margin-top: 20px; border-top: 1px solid var(--g-line); border-left: 1px solid var(--g-line); }
.cost-compare article { min-height: 205px; padding: 20px; border-right: 1px solid var(--g-line); border-bottom: 1px solid var(--g-line); }
.cost-compare article > div { display: flex; justify-content: space-between; gap: 12px; margin-top: 25px; }
.cost-compare article > div b { color: var(--g-muted); font: .52rem/1 var(--mono); }
.cost-compare article > div strong { font: 700 .62rem/1 var(--mono); text-align: right; }
.cost-compare p { color: var(--g-copper); font: .56rem/1.5 var(--mono); }
.cost-compare .boundary { color: #f7f0e6; background: var(--g-green); }
.cost-compare .boundary span, .cost-compare .boundary p { color: rgba(247,240,230,.68); }
.cost-compare .boundary b { display: block; margin-top: 30px; font-size: .85rem; }
.replay-ledger { display: grid; grid-template-columns: 1fr 40px 1fr 40px 1fr; gap: 8px; align-items: center; margin-top: 20px; }
.replay-ledger article { min-height: 145px; padding: 18px; border: 1px solid var(--g-line); background: var(--g-raised); }
.replay-ledger article.pass { color: #f7f0e6; background: var(--g-green); }
.replay-ledger article.pass span, .replay-ledger article.pass p { color: rgba(247,240,230,.68); }
.replay-ledger b { display: block; margin-top: 25px; font-size: .7rem; }
.replay-ledger p { color: var(--g-muted); font-size: .58rem; line-height: 1.5; overflow-wrap: anywhere; }
.hash-ledger { display: grid; grid-template-columns: repeat(2, 1fr); margin-top: 20px; border-top: 1px solid var(--g-line); border-left: 1px solid var(--g-line); }
.hash-ledger article { min-width: 0; padding: 16px; border-right: 1px solid var(--g-line); border-bottom: 1px solid var(--g-line); background: #e9e4da; }
.hash-ledger code { display: block; margin-top: 12px; overflow: hidden; color: var(--g-green); font: .54rem/1.3 var(--mono); text-overflow: ellipsis; }
.claim-grid { display: grid; grid-template-columns: repeat(2, 1fr); margin-top: 20px; }
.claim-grid article { padding: 22px; }
.claim-grid .yes { color: #f7f0e6; background: var(--g-green); }
.claim-grid .no { background: #e7d8ca; }
.claim-grid ul { margin: 18px 0 0; padding-left: 17px; }
.claim-grid li { margin-top: 10px; font-size: .65rem; line-height: 1.55; }
.claim-grid .yes span, .claim-grid .yes li { color: rgba(247,240,230,.78); }
@media (max-width: 920px) {
.gradient-ledger { grid-template-columns: repeat(3, 1fr); }
.gradient-ledger article:nth-child(3) { border-right: 0; }
.gradient-tabs { grid-template-columns: repeat(3, 1fr); }
.spectrum-layout { grid-template-columns: 1fr; }
.spectrum-layout .chart-shell { border-right: 0; border-bottom: 1px solid var(--g-line); }
.timeline-notes, .metric-pairs { grid-template-columns: repeat(2, 1fr); }
.object-chain { grid-template-columns: 1fr 24px 1fr; }
.object-chain > i:nth-of-type(n+3) { display: none; }
}
@media (max-width: 680px) {
.gradient-head, .panel-lead, .known-grid { grid-template-columns: 1fr; gap: 20px; }
.gradient-head, .gradient-panel { padding: 20px; }
.known-grid article + article { border-left: 0; border-top: 1px solid var(--g-line); }
.gradient-ledger { grid-template-columns: repeat(2, 1fr); }
.gradient-ledger article:nth-child(3) { border-right: 1px solid var(--g-line); }
.gradient-ledger article:nth-child(even) { border-right: 0; }
.gradient-tabs { display: flex; overflow-x: auto; }
.gradient-tabs button { flex: 0 0 190px; }
.lab-controls { align-items: stretch; }
.lab-controls > div, .lab-controls label { flex: 1 0 100%; }
.lab-controls button { flex: 1; }
.lab-controls select { width: 100%; }
.object-chain, .object-compare, .split-result, .replay-ledger { grid-template-columns: 1fr; }
.object-chain > i, .object-compare > i, .split-result > i, .replay-ledger > i { display: block !important; transform: rotate(90deg); }
.object-compare > p { grid-column: 1; }
.definition-boundary { grid-template-columns: 70px 1fr; }
.timeline-notes, .output-ratios, .metric-pairs, .cost-compare, .hash-ledger, .claim-grid { grid-template-columns: 1fr; }
.chart-shell { padding: 12px 8px; }
.chart-shell header { padding: 0 8px; }
.axis-label { font-size: 9px; }
.group-rhythm > div { min-width: 620px; }
.group-rhythm { overflow-x: auto; }
}
</style>
+33 -7
View File
@@ -2,6 +2,7 @@
import BaseLayout from "@/layouts/BaseLayout.astro"; import BaseLayout from "@/layouts/BaseLayout.astro";
import ArchitectureExplorer from "@/components/ArchitectureExplorer.astro"; import ArchitectureExplorer from "@/components/ArchitectureExplorer.astro";
import K3ArtifactLab from "@/components/K3ArtifactLab.astro"; import K3ArtifactLab from "@/components/K3ArtifactLab.astro";
import K3AttnResGradientLab from "@/components/K3AttnResGradientLab.astro";
import K3AttnResTraceLab from "@/components/K3AttnResTraceLab.astro"; import K3AttnResTraceLab from "@/components/K3AttnResTraceLab.astro";
import K3ReportLab from "@/components/K3ReportLab.astro"; import K3ReportLab from "@/components/K3ReportLab.astro";
import { k3FigureAtlas, k3Ledgers, k3PaperChain, k3ReportMap } from "@/data/k3"; import { k3FigureAtlas, k3Ledgers, k3PaperChain, k3ReportMap } from "@/data/k3";
@@ -38,7 +39,8 @@ const toc = [
["28", "lab", "八联交互实验"], ["28", "lab", "八联交互实验"],
["29", "artifacts", "开放权重工件审计"], ["29", "artifacts", "开放权重工件审计"],
["30", "attnres-reduced", "AttnRes 缩小机制实验"], ["30", "attnres-reduced", "AttnRes 缩小机制实验"],
["31", "audit", "21 张图表审计"], ["31", "attnres-gradient", "梯度定义与深度扩展"],
["32", "audit", "21 张图表审计"],
["↳", "papers", "100 节点阅读链"], ["↳", "papers", "100 节点阅读链"],
]; ];
@@ -107,13 +109,13 @@ const paperGroups = [
<BaseLayout <BaseLayout
title="Kimi K3 技术报告完整深读:架构、训练、RL、系统与评测" title="Kimi K3 技术报告完整深读:架构、训练、RL、系统与评测"
description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图、五个 AttnRes 独立实验视图与一百个一手阅读节点,逐节读懂 Kimi K3。" description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图、两轮十个 AttnRes 独立实验视图与一百个一手阅读节点,逐节读懂 Kimi K3。"
section="k3" section="k3"
> >
<header class="page-hero k3-hero"> <header class="page-hero k3-hero">
<div class="page-hero-inner"> <div class="page-hero-inner">
<div> <div>
<p class="eyebrow"><span>ANCHOR REPORT / ROUND 04</span> KIMI K3 · REPORT → ARTIFACTS → INDEPENDENT PROBE</p> <p class="eyebrow"><span>ANCHOR REPORT / ROUND 05</span> KIMI K3 · REPORT → ARTIFACTS → INDEPENDENT PROBE</p>
<h1>不把报告压成摘要<br />把每个因果环节<br />重新展开</h1> <h1>不把报告压成摘要<br />把每个因果环节<br />重新展开</h1>
<p class="lead"> <p class="lead">
K3 同时扩展序列、深度、宽度、视觉与 Agent 轨迹。真正值得读的不是 2.8T 这个最大数字, K3 同时扩展序列、深度、宽度、视觉与 Agent 轨迹。真正值得读的不是 2.8T 这个最大数字,
@@ -123,11 +125,11 @@ const paperGroups = [
<dl class="page-facts"> <dl class="page-facts">
<div><dt>QUESTIONS</dt><dd>32 张问题账</dd></div> <div><dt>QUESTIONS</dt><dd>32 张问题账</dd></div>
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div> <div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div>
<div><dt>LABS</dt><dd>8 + 4 + 5 个交互视图</dd></div> <div><dt>LABS</dt><dd>8 + 4 + 5 + 5 个交互视图</dd></div>
<div><dt>READING</dt><dd>100 个一手 / 官方节点</dd></div> <div><dt>READING</dt><dd>100 个一手 / 官方节点</dd></div>
<div><dt>MODEL</dt><dd>2.78T total / 104.2B active</dd></div> <div><dt>MODEL</dt><dd>2.78T total / 104.2B active</dd></div>
<div><dt>ARTIFACTS</dt><dd>96 shards · 497,220 tensors</dd></div> <div><dt>ARTIFACTS</dt><dd>96 shards · 497,220 tensors</dd></div>
<div><dt>STATUS</dt><dd>K3 四轮 · AttnRes 实验</dd></div> <div><dt>STATUS</dt><dd>K3 五轮 · 梯度定义闭环</dd></div>
</dl> </dl>
</div> </div>
</header> </header>
@@ -892,12 +894,36 @@ const paperGroups = [
<a class="button primary" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_REDUCED_AUDIT.md">阅读完整研究审计</a> <a class="button primary" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_REDUCED_AUDIT.md">阅读完整研究审计</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/attnres">复跑公开实验代码</a> <a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/attnres">复跑公开实验代码</a>
<a class="button" href="https://arxiv.org/abs/2603.15031">Attention Residuals 原论文</a> <a class="button" href="https://arxiv.org/abs/2603.15031">Attention Residuals 原论文</a>
<a class="button" href="https://github.com/MoonshotAI/Attention-Residuals">官方实现</a> <a class="button" href="https://github.com/MoonshotAI/Attention-Residuals">官方论文工件</a>
</div>
</section>
<section class="article-section" id="attnres-gradient">
<p class="eyebrow"><span>31</span> GRADIENT DEFINITION × DEPTH SCALE</p>
<h2>“论文说梯度更均匀”,和上一轮参数梯度反结果,测的是同一件事吗?</h2>
<p class="lede">
第五轮先审计 Attention Residuals 官方论文与仓库:Figure 5(c) 没有公开 gradient tensor、
norm、reduction、diagnostic batch、AMP / clipping 时点或统计代码。本站因此冻结一个可复现的
post-MLP output activation-gradient 定义,把深度扩到 16 / 32 blocks、预算扩到 8,000 steps,
再用三 seed 检查“首尾平衡”和“全层离散度”是否真的同方向。
</p>
<div class="artifact-callout">
<article><span>F / FROZEN</span><b>12 × 8,000 steps</b><p>786,432,000 formal target bytes;两深度、两结构、三 seed。</p></article>
<article><span>X / OBSERVED</span><b>first/last 6 / 6 改善</b><p>depth-16 平均 61.0%;depth-32 平均 72.0%。</p></article>
<article class="warning"><span>X / COUNTEREVIDENCE</span><b>CV 6 / 6 恶化</b><p>局部尖峰让 depth-16 / 32 平均相对恶化 10.3% / 60.0%。</p></article>
<article><span>R / REPLAY</span><b>model + optimizer exact</b><p>指定 32-layer Block 格从零重训 8,000 steps,冻结字段逐项一致。</p></article>
</div>
<K3AttnResGradientLab />
<div class="hero-actions">
<a class="button primary" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_GRADIENT_SCALE_AUDIT.md">阅读完整结果审计</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md">核对论文定义边界</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/attnres_gradient">复跑 12 格实验</a>
<a class="button" href="https://github.com/MoonshotAI/Attention-Residuals">官方一手工件</a>
</div> </div>
</section> </section>
<section class="article-section" id="audit"> <section class="article-section" id="audit">
<p class="eyebrow"><span>31</span> FIGURE & TABLE AUDIT</p> <p class="eyebrow"><span>32</span> FIGURE & TABLE AUDIT</p>
<h2>Figure 1–16、Table 1–5:每张图究竟支持什么,不能支持什么</h2> <h2>Figure 1–16、Table 1–5:每张图究竟支持什么,不能支持什么</h2>
<div class="figure-atlas"> <div class="figure-atlas">
{k3FigureAtlas.map(([id, report, title, contract]) => ( {k3FigureAtlas.map(([id, report, title, contract]) => (
+9 -4
View File
@@ -9,7 +9,7 @@ const researching = chapters.filter((chapter) => ["researching", "drafting"].inc
const workstreams = [ const workstreams = [
{ label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" }, { label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" },
{ label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" }, { label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" },
{ label: "Kimi K3 深读", value: 96, next: "对齐 AttnRes 梯度定义并扩展深度/预算;等待 A_log 官方转换合同" }, { label: "Kimi K3 深读", value: 98, next: "对齐 layer 21–25 梯度尖峰与 mixer weights;等待 A_log 官方转换合同" },
{ label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau,并加入真实小语料复现" }, { label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau,并加入真实小语料复现" },
{ label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" }, { label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" },
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" }, { label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
@@ -50,7 +50,7 @@ const workstreams = [
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div> <div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div> <div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div> <div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-30 07:30 CST</dd></div> <div><dt>UPDATED</dt><dd>2026-07-30 10:05 CST</dd></div>
<div><dt>MODE</dt><dd>持续迭代,不锁死版本</dd></div> <div><dt>MODE</dt><dd>持续迭代,不锁死版本</dd></div>
</dl> </dl>
</div> </div>
@@ -97,7 +97,7 @@ const workstreams = [
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article> <article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全,包含先修依赖和三条贯穿案例。</p></article> <article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全,包含先修依赖和三条贯穿案例。</p></article>
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article> <article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
<article><span>✓</span><h3>九十四个原创交互视图</h3><p>K3 三轴图、八联报告实验、四联开放工件实验与五联 AttnRes 独立实验,DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验,以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article> <article><span>✓</span><h3>九十九个原创交互视图</h3><p>K3 三轴图、八联报告实验、四联开放工件实验与两轮十联 AttnRes 独立实验,DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验,以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article> <article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article> <article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article> <article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
@@ -106,6 +106,7 @@ const workstreams = [
<article><span>✓</span><h3>Kimi K3 技术报告二轮深读</h3><p>三十二张问题账、Figure 1–16 / Table 1–5 审计、100 节点阅读链,以及 Delta—Decay—AttnRes—LatentMoE—SiTU—QB—MOPD—Cache 八联实验。</p></article> <article><span>✓</span><h3>Kimi K3 技术报告二轮深读</h3><p>三十二张问题账、Figure 1–16 / Table 1–5 审计、100 节点阅读链,以及 Delta—Decay—AttnRes—LatentMoE—SiTU—QB—MOPD—Cache 八联实验。</p></article>
<article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions,审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes;四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article> <article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions,审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes;四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
<article><span>✓</span><h3>Kimi K3 四轮 AttnRes 独立实验</h3><p>冻结三结构 × 三 seed 的 9 个 2,000-step 格;Full / Block 相对 Baseline 的平均 paired delta 为 −0.01457 / −0.04247 BPC,但核心参数梯度 CV 没有复现论文叙述。指定正式格全新进程八字段 exact,五视图同时展示结果、反证、成本与 claim boundary。</p></article> <article><span>✓</span><h3>Kimi K3 四轮 AttnRes 独立实验</h3><p>冻结三结构 × 三 seed 的 9 个 2,000-step 格;Full / Block 相对 Baseline 的平均 paired delta 为 −0.01457 / −0.04247 BPC,但核心参数梯度 CV 没有复现论文叙述。指定正式格全新进程八字段 exact,五视图同时展示结果、反证、成本与 claim boundary。</p></article>
<article><span>✓</span><h3>Kimi K3 五轮梯度定义与深度扩展</h3><p>先确认 Figure 5 没有公开唯一 gradient telemetry 合同,再冻结 16/32 blocks × Baseline/Block × 3 seeds 的 12 个 8,000-step 格。Block 的首尾失衡 6/6 改善但全层 CV 6/6 恶化,两个深度都判为 mixed;指定 32 层格完整重训的模型、优化器与全部冻结字段 exact。</p></article>
<article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel;6/6 官方参考逐元素相等,并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article> <article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel;6/6 官方参考逐元素相等,并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article> <article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
<article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article> <article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article>
@@ -134,7 +135,7 @@ const workstreams = [
</div> </div>
<div class="queue-table"> <div class="queue-table">
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div> <div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
<div><span>P0</span><strong>K3 四轮后续</strong><p>对齐论文梯度定义 → 增加 depth / budget → 等待 A_log 官方合同后进入真实 checkpoint forward</p><em>尺度复查 + 工件边界</em></div> <div><span>P0</span><strong>K3 五轮后续</strong><p>对齐 layer 21–25 尖峰、pre-attention / pre-MLP 与 mixer source weights → 等待 A_log 官方合同后进入真实 checkpoint forward</p><em>局部机制 + 工件边界</em></div>
<div><span>P0</span><strong>DeepSeek 八轮后续</strong><p>干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div> <div><span>P0</span><strong>DeepSeek 八轮后续</strong><p>干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div> <div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div> <div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
@@ -220,6 +221,10 @@ const workstreams = [
<div><time>2026-07-30</time><b>AttnRes 缩小实验先冻结、后运行</b><p>三结构共享公共主干、初始化、窗口与优化器;只按三个 paired seed 和预注册 −0.010 BPC 阈值给出本协议内方向判断。</p></div> <div><time>2026-07-30</time><b>AttnRes 缩小实验先冻结、后运行</b><p>三结构共享公共主干、初始化、窗口与优化器;只按三个 paired seed 和预注册 −0.010 BPC 阈值给出本协议内方向判断。</p></div>
<div><time>2026-07-30</time><b>支持结果与梯度反结果同时进入主视区</b><p>Full / Block 的最终 BPC 同向改善;核心参数 gradient RMS CV 却高于 Baseline,不换指标掩盖。</p></div> <div><time>2026-07-30</time><b>支持结果与梯度反结果同时进入主视区</b><p>Full / Block 的最终 BPC 同向改善;核心参数 gradient RMS CV 却高于 Baseline,不换指标掩盖。</p></div>
<div><time>2026-07-30</time><b>正式重放不把 wall time 纳入 exact</b><p>Block / seed-1 的模型、优化器、曲线、历史、诊断和环境八字段 exact;计时受调度影响,单独报告。</p></div> <div><time>2026-07-30</time><b>正式重放不把 wall time 纳入 exact</b><p>Block / seed-1 的模型、优化器、曲线、历史、诊断和环境八字段 exact;计时受调度影响,单独报告。</p></div>
<div><time>2026-07-30</time><b>Figure 5 的“梯度”不再靠猜测补合同</b><p>官方未公开 gradient tensor、norm、reduction 与统计代码;本站 activation-gradient 定义只叫 operationalization,不叫论文复画。</p></div>
<div><time>2026-07-30</time><b>首尾平衡与全层 CV 永久分账</b><p>Block 在 6/6 配对中改善 first/last,却因中后段局部尖峰让 CV 在 6/6 配对中恶化;联合判定保持 mixed。</p></div>
<div><time>2026-07-30</time><b>绝对梯度尺度必须与归一化谱同屏</b><p>Block mean gradient 约为 Baseline 的 54%–57%;更接近 1 的首尾比不能偷换成各层信号更强。</p></div>
<div><time>2026-07-30</time><b>32 层完整重放扩到状态哈希</b><p>8,000-step fresh replay 的全部冻结字段以及 model / optimizer state hashes exact;额外 replay bytes 单列,不混入 formal 预算。</p></div>
<div><time>2026-07-29</time><b>32-token 对照改为同源 16→24</b><p>TNEWS 只有 105/10,000 条达到 32 tokens,强行统一会落入约 1% 极端长尾;24-token eligibility 仍保留 1,609 条中文候选。</p></div> <div><time>2026-07-29</time><b>32-token 对照改为同源 16→24</b><p>TNEWS 只有 105/10,000 条达到 32 tokens,强行统一会落入约 1% 极端长尾;24-token eligibility 仍保留 1,609 条中文候选。</p></div>
<div><time>2026-07-29</time><b>长度敏感性必须成对重采样</b><p>16-token 输入严格是 24-token 输入前缀,2,000 次 bootstrap 共用 prompt indices;结果只描述固定 cohort 的长度敏感性。</p></div> <div><time>2026-07-29</time><b>长度敏感性必须成对重采样</b><p>16-token 输入严格是 24-token 输入前缀,2,000 次 bootstrap 共用 prompt indices;结果只描述固定 cohort 的长度敏感性。</p></div>
<div><time>2026-07-29</time><b>三类 cohort 永久分身份</b><p>自然长度回答本批样本如何路由;matched-16 / 24 回答同一 prompt 多看 8 tokens 后如何变化,不把二者混成内容因果。</p></div> <div><time>2026-07-29</time><b>三类 cohort 永久分身份</b><p>自然长度回答本批样本如何路由;matched-16 / 24 回答同一 prompt 多看 8 tokens 后如何变化,不把二者混成内容因果。</p></div>