site: explain AttnRes spike path study
This commit is contained in:
+17
-3
@@ -8,7 +8,7 @@
|
|||||||
|---|---:|---:|---|
|
|---|---:|---:|---|
|
||||||
| 研究框架与规范 | 进行中 | 83% | Scaling Laws 二轮拟合复现与逐图精读 |
|
| 研究框架与规范 | 进行中 | 83% | Scaling Laws 二轮拟合复现与逐图精读 |
|
||||||
| 网站设计系统 | 进行中 | 89% | 打印样式与更多通用可视化组件 |
|
| 网站设计系统 | 进行中 | 89% | 打印样式与更多通用可视化组件 |
|
||||||
| Kimi K3 深读 | 五轮实证进行中 | 98% | 定位 Block layer 21–25 尖峰、联查 mixer weights / reduction 敏感性,并等待 `A_log` 官方转换合同 |
|
| Kimi K3 深读 | 六轮实证进行中 | 99% | 对 group 6 / 7 做局部 mixer backward 干预,并等待 `A_log` 官方转换合同 |
|
||||||
| 语言模型前史 | 完成首版 | 78% | Kneser–Ney、LSTM、Bahdanau 逐图精读与真实小语料复现 |
|
| 语言模型前史 | 完成首版 | 78% | Kneser–Ney、LSTM、Bahdanau 逐图精读与真实小语料复现 |
|
||||||
| Transformer 基础 | 完成首版 | 79% | 多头电路、归一化 traces 与真实 kernel / KV 配置 |
|
| Transformer 基础 | 完成首版 | 79% | 多头电路、归一化 traces 与真实 kernel / KV 配置 |
|
||||||
| 表示、位置与残差高速公路 | 完成首版 | 81% | 真实 hidden-state / norm traces、长上下文位置外推与深层稳定性消融 |
|
| 表示、位置与残差高速公路 | 完成首版 | 81% | 真实 hidden-state / norm traces、长上下文位置外推与深层稳定性消融 |
|
||||||
@@ -41,7 +41,7 @@
|
|||||||
- [x] 完成 486 篇关键论文索引,覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。
|
- [x] 完成 486 篇关键论文索引,覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。
|
||||||
- [x] 完成可检索、可按专题筛选的论文库页面。
|
- [x] 完成可检索、可按专题筛选的论文库页面。
|
||||||
- [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。
|
- [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。
|
||||||
- [x] 完成 K3 三轴架构、八联报告实验、四联开放工件实验、Round 04 / 05 各五联 AttnRes 独立实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 二十二联实验、长上下文、MoE 路由、推理三页签,以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等九十九个原创交互视图。
|
- [x] 完成 K3 三轴架构、八联报告实验、四联开放工件实验、Round 04 / 05 / 06 各五联 AttnRes 独立实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 二十二联实验、长上下文、MoE 路由、推理三页签,以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等一百零四个原创交互视图。
|
||||||
- [x] 完成长上下文首版:五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
|
- [x] 完成长上下文首版:五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
|
||||||
- [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文,并建立长上下文研究账本。
|
- [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文,并建立长上下文研究账本。
|
||||||
- [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文,并建立 MoE 研究账本。
|
- [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文,并建立 MoE 研究账本。
|
||||||
@@ -288,10 +288,19 @@
|
|||||||
- [x] K3 Round 05 五视图实验室完成:论文定义已知/未定义、绝对/归一化深度谱、六 checkpoint 时间轨迹、Output RMS 组节律、activation/parameter/BPC/成本/重放联合账全部可切换;21 个 raw JSON、完整 aggregate、compact、runner、analyzer、协议与审计进入公开树。
|
- [x] K3 Round 05 五视图实验室完成:论文定义已知/未定义、绝对/归一化深度谱、六 checkpoint 时间轨迹、Output RMS 组节律、activation/parameter/BPC/成本/重放联合账全部可切换;21 个 raw JSON、完整 aggregate、compact、runner、analyzer、协议与审计进入公开树。
|
||||||
- [x] Round 05 本地闸门通过:94 个 Astro 文件零诊断/提示,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;冻结数据、新专项、Round 04 与 K3 全量真实 Chrome 回归通过,桌面/390px 移动端零文档级溢出、零运行时异常。
|
- [x] Round 05 本地闸门通过:94 个 Astro 文件零诊断/提示,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;冻结数据、新专项、Round 04 与 K3 全量真实 Chrome 回归通过,桌面/390px 移动端零文档级溢出、零运行时异常。
|
||||||
- [x] K3 Round 05 以功能提交 `f7670ef`、不可变镜像 `20260730T022610Z-f7670ef` 发布;OCI digest `sha256:02f902c1a64f976c6ed5d8614a826fca487b483f9addf2392637ea3f021cc09b`,NAS healthy / 零重启、21/21 公网路由、HTTPS/2、gzip/immutable 与三套 K3 生产 Chrome 回归全部通过;保留 `20260729T233142Z-4ce780d` 回滚。
|
- [x] K3 Round 05 以功能提交 `f7670ef`、不可变镜像 `20260730T022610Z-f7670ef` 发布;OCI digest `sha256:02f902c1a64f976c6ed5d8614a826fca487b483f9addf2392637ea3f021cc09b`,NAS healthy / 零重启、21/21 公网路由、HTTPS/2、gzip/immutable 与三套 K3 生产 Chrome 回归全部通过;保留 `20260729T233142Z-4ce780d` 回滚。
|
||||||
|
- [x] K3 Round 06 在正式训练前冻结 `llm-atlas-k3-attnres-spike-path-v1`:固定 Round 05 depth-32 Block、三个 seed、8,000 steps、六个 checkpoint、layer 21–25 目标集合、六个张量位置、四种 confirmatory reduction、三条 same-forward backward rule、20% 材料级阈值与完整 seed-1 replay;研究身份明确为受 Round 05 启发的 targeted follow-up,不冒充 blind discovery。
|
||||||
|
- [x] 三个正式格共消费 196,608,000 target bytes;最终模型、优化器、六个 BPC、全部训练历史与 Round 05 post-MLP 数组逐 seed exact。完整 replay 再消费 65,536,000 bytes,16 组冻结字段全部 exact,compare SHA-256 为 `05396eaf…cbb87`。
|
||||||
|
- [x] 尖峰发展轨迹显示 layer 21–25 不是初始化时固有:三 seed mean contrast 在 step 0 / 100 / 500 为 `0.757 / 0.563 / 0.524×`,到 step 2,000 / 4,000 / 8,000 升为 `2.233 / 2.594 / 2.754×`;seed 3 的最终 `1.881×` 同向但更弱。
|
||||||
|
- [x] 六个张量位置的最终 mean contrast 均超过 1.5,且每个位置 3 / 3 seed 可见;最早采样点 `pre_attention_input` 已可见,但观测顺序不升级为物理起源或注入位置。
|
||||||
|
- [x] 预注册 reduction 家族 12 / 12 格同时满足 contrast ≥ 1.5、top-5 与目标集合至少重叠 3 层、相对 element RMS 的 Spearman ≥ .8;正式结论严格写作 `robust within the preregistered reduction family`。
|
||||||
|
- [x] 全局切断 64 个 depth mixer 与 output mixer 的 softmax/query/key source-gradient paths 后,contrast / peak 平均相对“降幅”为 `−2.00% / −0.84%`,三 seed 均略微上升,不支持材料级降低。将 learned value-backward coefficients 换成 `1/N` 后,contrast / peak 平均下降 `70.20% / 37.03%`,三 seed 都超过 20% 阈值,peak layer 全部移到 layer 2。
|
||||||
|
- [x] mixer 观察关联与反向干预分层:layers 19–28 的 MLP latest weight 对 normalized gradient Pearson / Spearman 为 `.690 / .693`,MLP entropy 为 `−.636 / −.640`;相关性只提供候选机制,全局干预只支持 value-route sensitivity,不冒充局部 mixer attribution。
|
||||||
|
- [x] K3 Round 06 五视图实验室完成:训练轨迹、六位置谱、12 格 reduction 稳健性、same-forward 三规则干预与 mixer 散点/证据阶梯可交互;protocol、scoping、audit、runner、analyzer、raw/aggregate/compact/reproduction 全部进入公开树。
|
||||||
|
- [x] Round 06 本地闸门通过:97 个 Astro 文件零诊断,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;Round 04/05/06 三套冻结数据、三套专项与 K3 全量真实 Chrome 回归通过,桌面/390px 移动端零文档级溢出、零 offender、零运行时异常。
|
||||||
|
|
||||||
## 正在进行
|
## 正在进行
|
||||||
|
|
||||||
- [ ] K3 五轮下一闸门:对齐 layer 21–25 的 activation-gradient 尖峰、pre-attention / pre-MLP 位置与 mixer source weights,并做公开 reduction sensitivity;真实 K3 forward 继续等待 `A_log [128]→[96]` 官方转换或权重修订。
|
- [ ] K3 六轮下一闸门:把全局 value-route sensitivity 收缩成 group 6 / group 7 的局部 mixer intervention matrix,并设计前向训练变体;真实 K3 forward 继续等待 `A_log [128]→[96]` 官方转换或权重修订。
|
||||||
- [ ] DeepSeek 八轮下一闸门:推进干预式 mediation、SM90 FlashMLA、FP8 / pipeline traces 与 R1-like RL 小模型复现。
|
- [ ] DeepSeek 八轮下一闸门:推进干预式 mediation、SM90 FlashMLA、FP8 / pipeline traces 与 R1-like RL 小模型复现。
|
||||||
- [ ] 表示、位置与残差二轮:真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。
|
- [ ] 表示、位置与残差二轮:真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。
|
||||||
- [ ] 评测安全二轮:真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。
|
- [ ] 评测安全二轮:真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。
|
||||||
@@ -490,6 +499,11 @@
|
|||||||
| 2026-07-30 | 绝对尺度与归一化形状永久同报 | Block activation-gradient mean 约为 Baseline 54%–57%;不能把更接近 1 的首尾比自动解释为各层信号更强 |
|
| 2026-07-30 | 绝对尺度与归一化形状永久同报 | Block activation-gradient mean 约为 Baseline 54%–57%;不能把更接近 1 的首尾比自动解释为各层信号更强 |
|
||||||
| 2026-07-30 | Round 05 完整重放过闸 | depth-32 Block seed-1 从零重训 8,000 steps;全部冻结字段与 model/optimizer state hashes exact,timing 仍单独报告 |
|
| 2026-07-30 | Round 05 完整重放过闸 | depth-32 Block seed-1 从零重训 8,000 steps;全部冻结字段与 model/optimizer state hashes exact,timing 仍单独报告 |
|
||||||
| 2026-07-30 | K3 Round 05 梯度尺度里程碑发布 | 功能源 `f7670ef`、镜像 `20260730T022610Z-f7670ef`、OCI `sha256:02f902c1…cc09b`;复用 NAS `12010→8080`、NPM 31 / cert 41、门户 order 180,保留 Round 04 回滚点 |
|
| 2026-07-30 | K3 Round 05 梯度尺度里程碑发布 | 功能源 `f7670ef`、镜像 `20260730T022610Z-f7670ef`、OCI `sha256:02f902c1…cc09b`;复用 NAS `12010→8080`、NPM 31 / cert 41、门户 order 180,保留 Round 04 回滚点 |
|
||||||
|
| 2026-07-30 | 尖峰集合是 targeted follow-up,不是盲发现 | layer 21–25 来自 Round 05;Round 06 固定集合后才检查时间、位置、reduction 与路径,避免用同一结果发现并确认 |
|
||||||
|
| 2026-07-30 | “最早可见”与“物理起源”永久分账 | pre-attention input 是六个采样点中最早可见张量;更早 mixer 与反向传播已经作用,不能写成尖峰从这里注入 |
|
||||||
|
| 2026-07-30 | 同一前向只识别 backward-rule sensitivity | learned / detached / uniform 三模式的 logits、loss、activations 与 mixer summaries exact;结果不升级为重训模型或性能结论 |
|
||||||
|
| 2026-07-30 | value-route 降幅不写成因果贡献百分比 | 全局 uniform value-backward 让 contrast 平均下降 70.2%,只支持预注册阈值下的材料级敏感性;不声称 value 路径“解释了 70.2%” |
|
||||||
|
| 2026-07-30 | reduction 稳健性限定在预注册家族 | element RMS、token RMS mean/median/P95 的 12/12 格通过;探索性 reduction 和其他 batch 不被纳入确认性外推 |
|
||||||
|
|
||||||
## 未决问题
|
## 未决问题
|
||||||
|
|
||||||
|
|||||||
@@ -19,7 +19,7 @@
|
|||||||
|
|
||||||
当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读,
|
当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读,
|
||||||
语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化,以及评测与安全深度专题,
|
语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化,以及评测与安全深度专题,
|
||||||
以及 99 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
|
以及 104 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
|
||||||
8 个交互实验和 100 个一手/官方节点,完整覆盖架构、预训练、后训练、系统、评测、案例与附录。
|
8 个交互实验和 100 个一手/官方节点,完整覆盖架构、预训练、后训练、系统、评测、案例与附录。
|
||||||
第三轮已完成开放工件与首个真实 kernel 里程碑:固定官方模型与 FlashKDA revisions,审计 96 个 checkpoint shards、
|
第三轮已完成开放工件与首个真实 kernel 里程碑:固定官方模型与 FlashKDA revisions,审计 96 个 checkpoint shards、
|
||||||
497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计,并用 4 个新视图
|
497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计,并用 4 个新视图
|
||||||
@@ -52,6 +52,20 @@ peak allocated memory 约 2.2×,不冒充同算力优势。指定 depth-32 / B
|
|||||||
[K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md](./research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md)、
|
[K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md](./research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md)、
|
||||||
[K3_ATTNRES_GRADIENT_SCALE_AUDIT.md](./research/K3_ATTNRES_GRADIENT_SCALE_AUDIT.md) 与
|
[K3_ATTNRES_GRADIENT_SCALE_AUDIT.md](./research/K3_ATTNRES_GRADIENT_SCALE_AUDIT.md) 与
|
||||||
[gradient experiment](./experiments/k3/attnres_gradient/)。
|
[gradient experiment](./experiments/k3/attnres_gradient/)。
|
||||||
|
第六轮冻结 `llm-atlas-k3-attnres-spike-path-v1`,严格复用 Round 05 的 depth-32 Block
|
||||||
|
训练格,在 optimizer 之外追踪六个 checkpoint、六个 block 内张量位置与四种预注册
|
||||||
|
reduction,并在同一前向值下比较 learned、detached-learned 与 uniform-value-backward
|
||||||
|
三条反向规则。layer 21–25 的 post-MLP contrast 在 step 500 仍仅为三 seed 平均 `0.524×`,
|
||||||
|
到 step 2,000 升至 `2.233×`,说明尖峰在训练中形成;六个位置全部在 3/3 seed 可见,
|
||||||
|
四种 reduction 的 12/12 格同时通过 contrast、top-5 overlap 与 Spearman 闸门。
|
||||||
|
切断全局 softmax/query/key 源梯度后 contrast 反而平均增加约 2.0%;把 learned
|
||||||
|
value-backward coefficients 换成均匀系数后,contrast 平均下降 `70.2%`、peak/mean
|
||||||
|
下降 `37.0%`,三 seed 都达到预注册材料级敏感性阈值。该结果只支持全局反向路径敏感性,
|
||||||
|
不冒充训练变体、因果贡献百分比或局部 mixer 归因。正式三格与完整 replay 共
|
||||||
|
262,144,000 target bytes,Round 05 等价字段与 16 组 replay 字段全部 exact。详见
|
||||||
|
[K3_ATTNRES_SPIKE_PROTOCOL.md](./research/K3_ATTNRES_SPIKE_PROTOCOL.md)、
|
||||||
|
[K3_ATTNRES_SPIKE_AUDIT.md](./research/K3_ATTNRES_SPIKE_AUDIT.md) 与
|
||||||
|
[spike-path experiment](./experiments/k3/attnres_spike/)。
|
||||||
DeepSeek 八轮专题以 24 张问题账、10 次技术转向、
|
DeepSeek 八轮专题以 24 张问题账、10 次技术转向、
|
||||||
22 个交互实验和 60 个一手/官方节点,串起 Dense、MoE、MLA、V3 协同、R1 与 V4;
|
22 个交互实验和 60 个一手/官方节点,串起 Dense、MoE、MLA、V3 协同、R1 与 V4;
|
||||||
并固定官方 V2-Lite revision,在 RTX 5090 上连续执行 7/27 层,记录 3,240 次真实专家选择、
|
并固定官方 V2-Lite revision,在 RTX 5090 上连续执行 7/27 层,记录 3,240 次真实专家选择、
|
||||||
|
|||||||
@@ -43,6 +43,7 @@
|
|||||||
"check:deepseek-task-bootstrap-browser": "node scripts/check-deepseek-task-bootstrap-browser.mjs",
|
"check:deepseek-task-bootstrap-browser": "node scripts/check-deepseek-task-bootstrap-browser.mjs",
|
||||||
"check:k3-attnres-browser": "node scripts/check-k3-attnres-browser.mjs",
|
"check:k3-attnres-browser": "node scripts/check-k3-attnres-browser.mjs",
|
||||||
"check:k3-attnres-gradient-browser": "node scripts/check-k3-attnres-gradient-browser.mjs",
|
"check:k3-attnres-gradient-browser": "node scripts/check-k3-attnres-gradient-browser.mjs",
|
||||||
|
"check:k3-attnres-spike-browser": "node scripts/check-k3-attnres-spike-browser.mjs",
|
||||||
"check:k3-browser": "node scripts/check-k3-browser.mjs"
|
"check:k3-browser": "node scripts/check-k3-browser.mjs"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
|||||||
@@ -0,0 +1,216 @@
|
|||||||
|
import { writeFileSync } from "node:fs";
|
||||||
|
|
||||||
|
const cdpPort = process.env.CDP_PORT ?? "9229";
|
||||||
|
const baseUrl = process.env.SITE_URL ?? "http://127.0.0.1:4329";
|
||||||
|
const pages = await fetch(`http://127.0.0.1:${cdpPort}/json/list`).then((response) => response.json());
|
||||||
|
const page = pages.find((entry) => entry.type === "page");
|
||||||
|
if (!page) throw new Error(`CDP ${cdpPort} 没有可用页面`);
|
||||||
|
|
||||||
|
const socket = new WebSocket(page.webSocketDebuggerUrl);
|
||||||
|
await new Promise((resolve, reject) => {
|
||||||
|
socket.addEventListener("open", resolve, { once: true });
|
||||||
|
socket.addEventListener("error", reject, { once: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
let nextId = 0;
|
||||||
|
const pending = new Map();
|
||||||
|
const exceptions = [];
|
||||||
|
socket.addEventListener("message", (event) => {
|
||||||
|
const message = JSON.parse(event.data);
|
||||||
|
if (message.id && pending.has(message.id)) {
|
||||||
|
const { resolve, reject } = pending.get(message.id);
|
||||||
|
pending.delete(message.id);
|
||||||
|
if (message.error) reject(new Error(message.error.message));
|
||||||
|
else resolve(message.result);
|
||||||
|
}
|
||||||
|
if (message.method === "Runtime.exceptionThrown") {
|
||||||
|
exceptions.push(message.params.exceptionDetails.exception?.description ?? message.params.exceptionDetails.text);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
const command = (method, params = {}) => new Promise((resolve, reject) => {
|
||||||
|
const id = ++nextId;
|
||||||
|
pending.set(id, { resolve, reject });
|
||||||
|
socket.send(JSON.stringify({ id, method, params }));
|
||||||
|
});
|
||||||
|
const pause = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
|
||||||
|
const evaluate = async (expression) => {
|
||||||
|
const result = await command("Runtime.evaluate", { expression, returnByValue: true, awaitPromise: true });
|
||||||
|
if (result.exceptionDetails) throw new Error(result.exceptionDetails.exception?.description ?? result.exceptionDetails.text);
|
||||||
|
return result.result.value;
|
||||||
|
};
|
||||||
|
const navigate = async (path) => {
|
||||||
|
await command("Page.navigate", { url: `${baseUrl}${path}` });
|
||||||
|
for (let attempt = 0; attempt < 100; attempt += 1) {
|
||||||
|
await pause(100);
|
||||||
|
if (await evaluate("document.readyState === 'complete'")) return;
|
||||||
|
}
|
||||||
|
throw new Error(`${path} 加载超时`);
|
||||||
|
};
|
||||||
|
const screenshot = async (path) => {
|
||||||
|
const result = await command("Page.captureScreenshot", { format: "png", captureBeyondViewport: false });
|
||||||
|
writeFileSync(path, Buffer.from(result.data, "base64"));
|
||||||
|
};
|
||||||
|
|
||||||
|
await command("Page.enable");
|
||||||
|
await command("Runtime.enable");
|
||||||
|
await command("Emulation.setDeviceMetricsOverride", {
|
||||||
|
width: 1440,
|
||||||
|
height: 1100,
|
||||||
|
deviceScaleFactor: 1,
|
||||||
|
mobile: false,
|
||||||
|
});
|
||||||
|
await navigate("/k3/");
|
||||||
|
|
||||||
|
const desktop = await evaluate(`(() => {
|
||||||
|
const root = document.querySelector("[data-spike-lab]");
|
||||||
|
root.scrollIntoView({ block: "start", behavior: "instant" });
|
||||||
|
window.scrollBy(0, -78);
|
||||||
|
const text = (selector) => root.querySelector(selector)?.textContent.trim();
|
||||||
|
const panel = () => root.querySelector("[data-spike-panel]:not([hidden])")?.dataset.spikePanel;
|
||||||
|
const points = (selector) => root.querySelector(selector)?.getAttribute("points");
|
||||||
|
const setSelect = (selector, value) => {
|
||||||
|
const node = root.querySelector(selector);
|
||||||
|
node.value = value;
|
||||||
|
node.dispatchEvent(new Event("change", { bubbles: true }));
|
||||||
|
};
|
||||||
|
|
||||||
|
const initial = {
|
||||||
|
panel: panel(),
|
||||||
|
tabs: root.querySelectorAll("[data-spike-tab]").length,
|
||||||
|
panels: root.querySelectorAll("[data-spike-panel]").length,
|
||||||
|
ledger: root.querySelectorAll(".spike-ledger article").length,
|
||||||
|
trajectoryPoints: root.querySelectorAll("[data-spike-time-points] circle").length,
|
||||||
|
trajectoryLine: points("[data-spike-time-line]"),
|
||||||
|
boundary: root.textContent.includes("不是训练出的 uniform 模型") &&
|
||||||
|
root.textContent.includes("不是 70.2% 因果贡献"),
|
||||||
|
};
|
||||||
|
root.querySelector('[data-spike-time-metric="population_cv"]').click();
|
||||||
|
setSelect("[data-spike-time-seed]", "2026073003");
|
||||||
|
const trajectoryChanged = {
|
||||||
|
title: text("[data-spike-time-title]"),
|
||||||
|
state: text("[data-spike-time-state]"),
|
||||||
|
line: points("[data-spike-time-line]"),
|
||||||
|
thresholdHidden: getComputedStyle(root.querySelector("[data-spike-time-threshold]")).display === "none",
|
||||||
|
};
|
||||||
|
|
||||||
|
root.querySelector('[data-spike-tab="positions"]').click();
|
||||||
|
root.querySelector('[data-spike-position="pre_mlp_input"]').click();
|
||||||
|
setSelect("[data-spike-position-seed]", "2026073002");
|
||||||
|
const positions = {
|
||||||
|
panel: panel(),
|
||||||
|
buttons: root.querySelectorAll("[data-spike-position]").length,
|
||||||
|
state: text("[data-spike-position-state]"),
|
||||||
|
contrast: text("[data-spike-position-contrast]"),
|
||||||
|
peak: text("[data-spike-position-peak]"),
|
||||||
|
peakValue: text("[data-spike-position-peak-value]"),
|
||||||
|
pointCount: root.querySelectorAll("[data-spike-position-points] circle").length,
|
||||||
|
};
|
||||||
|
|
||||||
|
root.querySelector('[data-spike-tab="reductions"]').click();
|
||||||
|
setSelect("[data-spike-reduction-seed]", "2026073003");
|
||||||
|
setSelect("[data-spike-reduction]", "token_rms_median");
|
||||||
|
const reductions = {
|
||||||
|
panel: panel(),
|
||||||
|
rows: root.querySelectorAll(".reduction-table tbody tr").length,
|
||||||
|
state: text("[data-spike-reduction-state]"),
|
||||||
|
contrast: text("[data-spike-reduction-contrast]"),
|
||||||
|
overlap: text("[data-spike-reduction-overlap]"),
|
||||||
|
rho: text("[data-spike-reduction-rho]"),
|
||||||
|
pointCount: root.querySelectorAll("[data-spike-reduction-points] circle").length,
|
||||||
|
};
|
||||||
|
|
||||||
|
root.querySelector('[data-spike-tab="intervention"]').click();
|
||||||
|
setSelect("[data-spike-intervention-seed]", "2026073002");
|
||||||
|
const intervention = {
|
||||||
|
panel: panel(),
|
||||||
|
lines: root.querySelectorAll("[data-spike-intervention-line]").length,
|
||||||
|
keyDrop: text("[data-spike-key-drop]"),
|
||||||
|
valueDrop: text("[data-spike-value-drop]"),
|
||||||
|
peakShift: text("[data-spike-peak-shift]"),
|
||||||
|
linePoints: [...root.querySelectorAll("[data-spike-intervention-line]")].map((node) => node.getAttribute("points").split(" ").length),
|
||||||
|
forwardIdentity: root.textContent.includes("same logits · same loss"),
|
||||||
|
};
|
||||||
|
|
||||||
|
root.querySelector('[data-spike-tab="mixer"]').click();
|
||||||
|
const mixerInitial = {
|
||||||
|
panel: panel(),
|
||||||
|
points: root.querySelectorAll("[data-spike-mixer-points] circle").length,
|
||||||
|
pearson: text("[data-spike-mixer-pearson]"),
|
||||||
|
spearman: text("[data-spike-mixer-spearman]"),
|
||||||
|
title: text("[data-spike-mixer-title]"),
|
||||||
|
};
|
||||||
|
root.querySelector('[data-spike-mixer-metric="mlp_entropy"]').click();
|
||||||
|
const mixerChanged = {
|
||||||
|
points: root.querySelectorAll("[data-spike-mixer-points] circle").length,
|
||||||
|
pearson: text("[data-spike-mixer-pearson]"),
|
||||||
|
spearman: text("[data-spike-mixer-spearman]"),
|
||||||
|
title: text("[data-spike-mixer-title]"),
|
||||||
|
};
|
||||||
|
|
||||||
|
const first = root.querySelector('[data-spike-tab="trajectory"]');
|
||||||
|
first.focus();
|
||||||
|
first.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowRight", bubbles: true }));
|
||||||
|
const keyboard = {
|
||||||
|
selected: root.querySelector('[data-spike-tab][aria-selected="true"]').dataset.spikeTab,
|
||||||
|
panel: panel(),
|
||||||
|
};
|
||||||
|
|
||||||
|
return {
|
||||||
|
initial, trajectoryChanged, positions, reductions, intervention, mixerInitial, mixerChanged, keyboard,
|
||||||
|
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
|
||||||
|
rootOverflow: root.scrollWidth - root.clientWidth,
|
||||||
|
};
|
||||||
|
})()`);
|
||||||
|
await pause(180);
|
||||||
|
await screenshot("/tmp/llm-atlas-k3-attnres-spike-desktop.png");
|
||||||
|
|
||||||
|
await command("Emulation.setDeviceMetricsOverride", {
|
||||||
|
width: 390,
|
||||||
|
height: 844,
|
||||||
|
deviceScaleFactor: 1,
|
||||||
|
mobile: true,
|
||||||
|
});
|
||||||
|
await navigate("/k3/");
|
||||||
|
const mobile = await evaluate(`(() => {
|
||||||
|
const root = document.querySelector("[data-spike-lab]");
|
||||||
|
root.scrollIntoView({ block: "start", behavior: "instant" });
|
||||||
|
window.scrollBy(0, -64);
|
||||||
|
root.querySelector('[data-spike-tab="intervention"]').click();
|
||||||
|
return {
|
||||||
|
tabs: root.querySelectorAll("[data-spike-tab]").length,
|
||||||
|
ledger: root.querySelectorAll(".spike-ledger article").length,
|
||||||
|
visiblePanel: root.querySelector("[data-spike-panel]:not([hidden])")?.dataset.spikePanel,
|
||||||
|
lines: root.querySelectorAll("[data-spike-intervention-line]").length,
|
||||||
|
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
|
||||||
|
rootOverflow: root.scrollWidth - root.clientWidth,
|
||||||
|
};
|
||||||
|
})()`);
|
||||||
|
await pause(180);
|
||||||
|
await screenshot("/tmp/llm-atlas-k3-attnres-spike-mobile.png");
|
||||||
|
|
||||||
|
const report = { desktop, mobile, exceptions };
|
||||||
|
console.log(JSON.stringify(report, null, 2));
|
||||||
|
|
||||||
|
const numeric = (value) => Number.parseFloat(value.replace("−", "-").replace("%", ""));
|
||||||
|
const failures = [];
|
||||||
|
if (desktop.initial.panel !== "trajectory" || desktop.initial.tabs !== 5 || desktop.initial.panels !== 5 || desktop.initial.ledger !== 6 || desktop.initial.trajectoryPoints !== 6) failures.push("五视图或六点轨迹初始结构异常");
|
||||||
|
if (!desktop.initial.boundary) failures.push("因果与训练变体边界缺失");
|
||||||
|
if (!desktop.trajectoryChanged.title.includes("POPULATION CV") || !desktop.trajectoryChanged.state.includes("2026073003") || desktop.trajectoryChanged.line === desktop.initial.trajectoryLine || !desktop.trajectoryChanged.thresholdHidden) failures.push("轨迹 seed / metric 切换异常");
|
||||||
|
if (desktop.positions.panel !== "positions" || desktop.positions.buttons !== 6 || !desktop.positions.state.includes("MLP 前输入") || Math.abs(numeric(desktop.positions.contrast) - 4.527) > .001 || numeric(desktop.positions.peak) !== 21 || desktop.positions.pointCount !== 32) failures.push("六位置谱或 seed 切换异常");
|
||||||
|
if (desktop.reductions.panel !== "reductions" || desktop.reductions.rows !== 12 || !desktop.reductions.state.includes("Token RMS 中位数") || Math.abs(numeric(desktop.reductions.contrast) - 1.666) > .001 || desktop.reductions.overlap !== "3 / 5" || Math.abs(numeric(desktop.reductions.rho) - .934) > .001 || desktop.reductions.pointCount !== 32) failures.push("reduction robustness 视图异常");
|
||||||
|
if (desktop.intervention.panel !== "intervention" || desktop.intervention.lines !== 3 || Math.abs(numeric(desktop.intervention.keyDrop) + 1.93) > .01 || Math.abs(numeric(desktop.intervention.valueDrop) - 76.87) > .01 || desktop.intervention.peakShift !== "21 → 2" || desktop.intervention.linePoints.some((count) => count !== 32) || !desktop.intervention.forwardIdentity) failures.push("same-forward backward 干预视图异常");
|
||||||
|
if (desktop.mixerInitial.panel !== "mixer" || desktop.mixerInitial.points !== 30 || Math.abs(numeric(desktop.mixerInitial.pearson) - .690) > .001 || Math.abs(numeric(desktop.mixerInitial.spearman) - .693) > .001 || desktop.mixerChanged.points !== 30 || Math.abs(numeric(desktop.mixerChanged.pearson) + .636) > .001 || Math.abs(numeric(desktop.mixerChanged.spearman) + .640) > .001 || desktop.mixerInitial.title === desktop.mixerChanged.title) failures.push("mixer association 散点或指标切换异常");
|
||||||
|
if (desktop.keyboard.selected !== "positions" || desktop.keyboard.panel !== "positions") failures.push("键盘 tab 导航异常");
|
||||||
|
if (desktop.documentOverflow > 1 || desktop.rootOverflow > 1 || mobile.documentOverflow > 1 || mobile.rootOverflow > 1) failures.push("桌面或移动端出现文档级横向溢出");
|
||||||
|
if (mobile.tabs !== 5 || mobile.ledger !== 6 || mobile.visiblePanel !== "intervention" || mobile.lines !== 3) failures.push("移动端交互结构异常");
|
||||||
|
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
|
||||||
|
|
||||||
|
if (failures.length) {
|
||||||
|
console.error(`\nFAIL\n- ${failures.join("\n- ")}`);
|
||||||
|
process.exitCode = 1;
|
||||||
|
} else {
|
||||||
|
console.log("\nPASS K3 AttnRes spike browser regression");
|
||||||
|
}
|
||||||
|
|
||||||
|
socket.close();
|
||||||
@@ -83,6 +83,8 @@ const overview = await evaluate(`(() => ({
|
|||||||
attnresPanels: document.querySelectorAll("[data-attnres-panel]").length,
|
attnresPanels: document.querySelectorAll("[data-attnres-panel]").length,
|
||||||
gradientTabs: document.querySelectorAll("[data-gradient-tab]").length,
|
gradientTabs: document.querySelectorAll("[data-gradient-tab]").length,
|
||||||
gradientPanels: document.querySelectorAll("[data-gradient-panel]").length,
|
gradientPanels: document.querySelectorAll("[data-gradient-panel]").length,
|
||||||
|
spikeTabs: document.querySelectorAll("[data-spike-tab]").length,
|
||||||
|
spikePanels: document.querySelectorAll("[data-spike-panel]").length,
|
||||||
nativeVisionCorrected: document.body.textContent.includes("MoonViT‑V2 从头训练") &&
|
nativeVisionCorrected: document.body.textContent.includes("MoonViT‑V2 从头训练") &&
|
||||||
document.body.textContent.includes("同一个 next-token prediction objective"),
|
document.body.textContent.includes("同一个 next-token prediction objective"),
|
||||||
staleVisionClaim: document.body.textContent.includes("先固定语言模型训练视觉组件"),
|
staleVisionClaim: document.body.textContent.includes("先固定语言模型训练视觉组件"),
|
||||||
@@ -293,8 +295,9 @@ const mobile = await evaluate(`(() => {
|
|||||||
artifactLayers: document.querySelectorAll("[data-layer-cell]").length,
|
artifactLayers: document.querySelectorAll("[data-layer-cell]").length,
|
||||||
attnresTabs: document.querySelectorAll("[data-attnres-tab]").length,
|
attnresTabs: document.querySelectorAll("[data-attnres-tab]").length,
|
||||||
gradientTabs: document.querySelectorAll("[data-gradient-tab]").length,
|
gradientTabs: document.querySelectorAll("[data-gradient-tab]").length,
|
||||||
|
spikeTabs: document.querySelectorAll("[data-spike-tab]").length,
|
||||||
offenders: [...document.querySelectorAll("body *")]
|
offenders: [...document.querySelectorAll("body *")]
|
||||||
.filter((node) => !node.closest(".paper-chain, .spec-table-wrap, .cache-strip, .architecture-explorer, [data-k3-lab], [data-k3-artifact-lab], [data-attnres-lab], [data-gradient-lab]"))
|
.filter((node) => !node.closest(".paper-chain, .spec-table-wrap, .cache-strip, .architecture-explorer, [data-k3-lab], [data-k3-artifact-lab], [data-attnres-lab], [data-gradient-lab], [data-spike-lab]"))
|
||||||
.filter((node) => node.getBoundingClientRect().right > document.documentElement.clientWidth + 1)
|
.filter((node) => node.getBoundingClientRect().right > document.documentElement.clientWidth + 1)
|
||||||
.slice(0, 15)
|
.slice(0, 15)
|
||||||
.map((node) => ({
|
.map((node) => ({
|
||||||
@@ -325,13 +328,14 @@ console.log(JSON.stringify(report, null, 2));
|
|||||||
const numeric = (text) => Number.parseFloat(text.replaceAll(",", "").replace("−", "-"));
|
const numeric = (text) => Number.parseFloat(text.replaceAll(",", "").replace("−", "-"));
|
||||||
const failures = [];
|
const failures = [];
|
||||||
if (!overview.title.includes("因果环节")) failures.push("K3 二轮标题异常");
|
if (!overview.title.includes("因果环节")) failures.push("K3 二轮标题异常");
|
||||||
if (overview.sections !== 34 || overview.tocLinks !== 34) failures.push("33 个编号专题加阅读链的目录结构异常");
|
if (overview.sections !== 35 || overview.tocLinks !== 35) failures.push("34 个编号专题加阅读链的目录结构异常");
|
||||||
if (overview.ledgers !== 32 || overview.reportMap !== 9) failures.push("32 张问题账或报告地图异常");
|
if (overview.ledgers !== 32 || overview.reportMap !== 9) failures.push("32 张问题账或报告地图异常");
|
||||||
if (overview.figureAtlas !== 21 || overview.paperLinks !== 100 || overview.paperGroups < 12) failures.push("图表审计或 100 节点阅读链异常");
|
if (overview.figureAtlas !== 21 || overview.paperLinks !== 100 || overview.paperGroups < 12) failures.push("图表审计或 100 节点阅读链异常");
|
||||||
if (overview.labTabs !== 8 || overview.labPanels !== 8) failures.push("八联实验结构异常");
|
if (overview.labTabs !== 8 || overview.labPanels !== 8) failures.push("八联实验结构异常");
|
||||||
if (overview.artifactTabs !== 4 || overview.artifactPanels !== 4 || overview.artifactLayers !== 93 || !overview.artifactMismatch) failures.push("开放工件四视图、93 层条带或形状冲突异常");
|
if (overview.artifactTabs !== 4 || overview.artifactPanels !== 4 || overview.artifactLayers !== 93 || !overview.artifactMismatch) failures.push("开放工件四视图、93 层条带或形状冲突异常");
|
||||||
if (overview.attnresTabs !== 5 || overview.attnresPanels !== 5) failures.push("AttnRes 独立实验五视图异常");
|
if (overview.attnresTabs !== 5 || overview.attnresPanels !== 5) failures.push("AttnRes 独立实验五视图异常");
|
||||||
if (overview.gradientTabs !== 5 || overview.gradientPanels !== 5) failures.push("AttnRes 梯度定义扩展五视图异常");
|
if (overview.gradientTabs !== 5 || overview.gradientPanels !== 5) failures.push("AttnRes 梯度定义扩展五视图异常");
|
||||||
|
if (overview.spikeTabs !== 5 || overview.spikePanels !== 5) failures.push("AttnRes 尖峰路径五视图异常");
|
||||||
if (!overview.nativeVisionCorrected || overview.staleVisionClaim) failures.push("原生多模态纠错未生效或旧错误残留");
|
if (!overview.nativeVisionCorrected || overview.staleVisionClaim) failures.push("原生多模态纠错未生效或旧错误残留");
|
||||||
if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出");
|
if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出");
|
||||||
if (labs.memoryInitial.panel !== "memory" || numeric(labs.memoryInitial.additiveError) <= numeric(labs.memoryInitial.deltaError)) failures.push("Delta memory 初始递推异常");
|
if (labs.memoryInitial.panel !== "memory" || numeric(labs.memoryInitial.additiveError) <= numeric(labs.memoryInitial.deltaError)) failures.push("Delta memory 初始递推异常");
|
||||||
@@ -356,7 +360,7 @@ if (artifacts.parameterChanged.shape !== "[96,128] F32" || !artifacts.parameterC
|
|||||||
if (artifacts.reproductionInitial.panel !== "reproduction" || numeric(artifacts.reproductionInitial.speedup) !== 1.85 || numeric(artifacts.reproductionInitial.localMean) < 2.6 || !artifacts.reproductionInitial.exactSuite || numeric(artifacts.reproductionInitial.cv) < 2) failures.push("FlashKDA H20、本机 exact suite 或 router 初始探针异常");
|
if (artifacts.reproductionInitial.panel !== "reproduction" || numeric(artifacts.reproductionInitial.speedup) !== 1.85 || numeric(artifacts.reproductionInitial.localMean) < 2.6 || !artifacts.reproductionInitial.exactSuite || numeric(artifacts.reproductionInitial.cv) < 2) failures.push("FlashKDA H20、本机 exact suite 或 router 初始探针异常");
|
||||||
if (numeric(artifacts.reproductionChanged.speedup) !== 3.27 || numeric(artifacts.reproductionChanged.flash) !== 0.7064 || numeric(artifacts.reproductionChanged.localMean) >= numeric(artifacts.reproductionInitial.localMean) || !artifacts.reproductionChanged.localMode.includes("FP32 state") || numeric(artifacts.reproductionChanged.cv) <= numeric(artifacts.reproductionInitial.cv) || numeric(artifacts.reproductionChanged.zero) <= numeric(artifacts.reproductionInitial.zero)) failures.push("GB200 benchmark、本机 varlen/state 或 synthetic router counterexample 未更新");
|
if (numeric(artifacts.reproductionChanged.speedup) !== 3.27 || numeric(artifacts.reproductionChanged.flash) !== 0.7064 || numeric(artifacts.reproductionChanged.localMean) >= numeric(artifacts.reproductionInitial.localMean) || !artifacts.reproductionChanged.localMode.includes("FP32 state") || numeric(artifacts.reproductionChanged.cv) <= numeric(artifacts.reproductionInitial.cv) || numeric(artifacts.reproductionChanged.zero) <= numeric(artifacts.reproductionInitial.zero)) failures.push("GB200 benchmark、本机 varlen/state 或 synthetic router counterexample 未更新");
|
||||||
if (artifacts.keyboardSelected !== "tensors" || artifacts.keyboardVisible !== "tensors") failures.push("开放工件键盘 tab 导航异常");
|
if (artifacts.keyboardSelected !== "tensors" || artifacts.keyboardVisible !== "tensors") failures.push("开放工件键盘 tab 导航异常");
|
||||||
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 8 || mobile.artifactTabs !== 4 || mobile.artifactLayers !== 93 || mobile.attnresTabs !== 5 || mobile.gradientTabs !== 5) failures.push("移动端导航或实验异常");
|
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 8 || mobile.artifactTabs !== 4 || mobile.artifactLayers !== 93 || mobile.attnresTabs !== 5 || mobile.gradientTabs !== 5 || mobile.spikeTabs !== 5) failures.push("移动端导航或实验异常");
|
||||||
if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`);
|
if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`);
|
||||||
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
|
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,892 @@
|
|||||||
|
---
|
||||||
|
import rawLab from "@/data/k3-attnres-spike-compact.json";
|
||||||
|
|
||||||
|
const lab = rawLab as any;
|
||||||
|
const json = JSON.stringify(lab).replaceAll("<", "\\u003c");
|
||||||
|
const reductions = lab.verdicts.reduction_robustness.cells;
|
||||||
|
const replayHash = lab.gates.replay.frozen_compare_sha256;
|
||||||
|
const shortHash = (value: string) => `${value.slice(0, 10)}…${value.slice(-8)}`;
|
||||||
|
const reductionLabels: Record<string, string> = {
|
||||||
|
element_rms: "全元素 RMS",
|
||||||
|
token_rms_mean: "Token RMS 均值",
|
||||||
|
token_rms_median: "Token RMS 中位数",
|
||||||
|
token_rms_p95: "Token RMS P95",
|
||||||
|
};
|
||||||
|
---
|
||||||
|
|
||||||
|
<figure class="spike-lab" data-spike-lab>
|
||||||
|
<header class="spike-head">
|
||||||
|
<div>
|
||||||
|
<p>ROUND 06 / SPIKE TRAJECTORY × BACKWARD PATH</p>
|
||||||
|
<h3>尖峰不是出生时就有;它在训练中形成,并对 value 路径敏感</h3>
|
||||||
|
</div>
|
||||||
|
<p>
|
||||||
|
固定 Round 05 的 depth-32 Block 训练格,只在 optimizer 之外增加诊断。
|
||||||
|
3 个正式 seed、1 个完整重放;所有前向值完全相同,只改变同一次反向传播的局部导数规则。
|
||||||
|
</p>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<div class="spike-ledger">
|
||||||
|
<article><span>FORMAL TARGET BYTES</span><b>196.608M</b><p>3 × 8,000 steps</p></article>
|
||||||
|
<article><span>FULL REPLAY</span><b>exact</b><p>16 组冻结字段</p></article>
|
||||||
|
<article><span>VISIBLE POSITIONS</span><b>6 / 6</b><p>每个位置 3 / 3 seed</p></article>
|
||||||
|
<article class="pass"><span>REDUCTIONS</span><b>12 / 12</b><p>预注册家族内稳健</p></article>
|
||||||
|
<article class="counter"><span>KEY / SOFTMAX PATH</span><b>−2.0%</b><p>移除后反而略升</p></article>
|
||||||
|
<article class="pass"><span>VALUE COEFFICIENTS</span><b>−70.2%</b><p>平均 contrast 降幅</p></article>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="spike-tabs" role="tablist" aria-label="选择 AttnRes 尖峰路径实验视图">
|
||||||
|
<button type="button" role="tab" data-spike-tab="trajectory" aria-selected="true">
|
||||||
|
<span>01</span><b>尖峰何时形成</b><small>step 0 → 8,000</small>
|
||||||
|
</button>
|
||||||
|
<button type="button" role="tab" data-spike-tab="positions" aria-selected="false">
|
||||||
|
<span>02</span><b>六个位置都看见什么</b><small>visible ≠ origin</small>
|
||||||
|
</button>
|
||||||
|
<button type="button" role="tab" data-spike-tab="reductions" aria-selected="false">
|
||||||
|
<span>03</span><b>换一种算法还在吗</b><small>12 / 12 cells</small>
|
||||||
|
</button>
|
||||||
|
<button type="button" role="tab" data-spike-tab="intervention" aria-selected="false">
|
||||||
|
<span>04</span><b>哪条反向路径敏感</b><small>same forward · 3 rules</small>
|
||||||
|
</button>
|
||||||
|
<button type="button" role="tab" data-spike-tab="mixer" aria-selected="false">
|
||||||
|
<span>05</span><b>Mixer 线索与证据边界</b><small>association ≠ attribution</small>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<section class="spike-panel" data-spike-panel="trajectory">
|
||||||
|
<div class="panel-lead">
|
||||||
|
<div><span>I / DEVELOPMENT</span><h4>先问“训练到什么时候出现”,再问“由哪条路径放大”</h4></div>
|
||||||
|
<p>
|
||||||
|
尖峰集合 S 固定为 layer 21–25。纵轴 contrast = S 内均值 ÷ 其余层均值:
|
||||||
|
1 表示没有集中,越大表示梯度越集中在这五层。
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div class="lab-controls">
|
||||||
|
<label>SEED
|
||||||
|
<select data-spike-time-seed>
|
||||||
|
<option value="mean">3-SEED MEAN</option>
|
||||||
|
<option value="2026073001">2026073001</option>
|
||||||
|
<option value="2026073002">2026073002</option>
|
||||||
|
<option value="2026073003">2026073003</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<div>
|
||||||
|
<button type="button" data-spike-time-metric="spike_contrast" aria-pressed="true">SPIKE CONTRAST</button>
|
||||||
|
<button type="button" data-spike-time-metric="population_cv" aria-pressed="false">LAYER CV</button>
|
||||||
|
<button type="button" data-spike-time-metric="peak_normalized" aria-pressed="false">PEAK / MEAN</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="chart-shell">
|
||||||
|
<header><b data-spike-time-title>SPIKE CONTRAST · S / REST</b><span data-spike-time-state>3-SEED MEAN</span></header>
|
||||||
|
<svg viewBox="0 0 940 350" role="img" aria-label="六个训练检查点上的尖峰轨迹" data-spike-time-chart>
|
||||||
|
<g data-spike-time-grid></g>
|
||||||
|
<line class="threshold" x1="58" x2="900" y1="0" y2="0" data-spike-time-threshold></line>
|
||||||
|
<polyline class="series copper" points="" data-spike-time-line></polyline>
|
||||||
|
<g data-spike-time-points></g>
|
||||||
|
</svg>
|
||||||
|
<div class="chart-legend"><span><i class="copper"></i>post-MLP activation-gradient</span><span><i class="threshold-key"></i>预注册可见阈值 1.5</span></div>
|
||||||
|
</div>
|
||||||
|
<div class="trajectory-story">
|
||||||
|
<article><span>STEP 0</span><b>0.757×</b><p>三 seed 均低于 1;layer 21–25 并不特殊。</p></article>
|
||||||
|
<i>→</i>
|
||||||
|
<article><span>STEP 500</span><b>0.524×</b><p>学习已经发生,但尖峰尚未出现。</p></article>
|
||||||
|
<i>→</i>
|
||||||
|
<article class="accent"><span>STEP 2,000</span><b>2.233×</b><p>峰值层同时迁移到 layer 21,结构开始显形。</p></article>
|
||||||
|
<i>→</i>
|
||||||
|
<article><span>STEP 8,000</span><b>2.754×</b><p>seed 3 只有 1.881×:方向一致,强度并不整齐。</p></article>
|
||||||
|
</div>
|
||||||
|
<div class="plain-rule">
|
||||||
|
<b>通俗读法</b>
|
||||||
|
<p>这不像一个由初始化直接写死的“坏层”;它更像训练动力学与 Block mixer 共同塑出的深度模式。</p>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="spike-panel" data-spike-panel="positions" hidden>
|
||||||
|
<div class="panel-lead">
|
||||||
|
<div><span>II / LOCALIZATION</span><h4>从一个 Block 向前追:尖峰在六个候选张量上都已经可见</h4></div>
|
||||||
|
<p>
|
||||||
|
“最早观测到”是沿本站采样点的相对顺序;两个采样点之间仍有 mixer、归一化与残差运算,
|
||||||
|
所以它不是尖峰的物理出生点。
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div class="tensor-path" aria-label="一个 Transformer block 的六个梯度采样位置">
|
||||||
|
<button type="button" data-spike-position="pre_attention_input" aria-pressed="true"><span>01</span><b>Attn 前输入</b><small>3.724 / 3.189 / 2.040</small></button>
|
||||||
|
<i>→</i>
|
||||||
|
<button type="button" data-spike-position="attention_branch_output" aria-pressed="false"><span>02</span><b>Attn 分支输出</b><small>3.200 / 3.662 / 2.005</small></button>
|
||||||
|
<i>→</i>
|
||||||
|
<button type="button" data-spike-position="post_attention_state" aria-pressed="false"><span>03</span><b>Attn 后状态</b><small>3.200 / 3.662 / 2.005</small></button>
|
||||||
|
<i>→</i>
|
||||||
|
<button type="button" data-spike-position="pre_mlp_input" aria-pressed="false"><span>04</span><b>MLP 前输入</b><small>3.829 / 4.527 / 2.978</small></button>
|
||||||
|
<i>→</i>
|
||||||
|
<button type="button" data-spike-position="mlp_branch_output" aria-pressed="false"><span>05</span><b>MLP 分支输出</b><small>3.046 / 3.333 / 1.881</small></button>
|
||||||
|
<i>→</i>
|
||||||
|
<button type="button" data-spike-position="post_mlp_state" aria-pressed="false"><span>06</span><b>MLP 后状态</b><small>3.046 / 3.333 / 1.881</small></button>
|
||||||
|
</div>
|
||||||
|
<div class="lab-controls position-controls">
|
||||||
|
<label>SEED
|
||||||
|
<select data-spike-position-seed>
|
||||||
|
<option value="2026073001">2026073001</option>
|
||||||
|
<option value="2026073002">2026073002</option>
|
||||||
|
<option value="2026073003">2026073003</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<span data-spike-position-state>ATTENTION 前输入 · SEED 2026073001</span>
|
||||||
|
</div>
|
||||||
|
<div class="position-layout">
|
||||||
|
<div class="chart-shell">
|
||||||
|
<header><b>NORMALIZED GRADIENT BY LAYER</b><span>layer mean = 1</span></header>
|
||||||
|
<svg viewBox="0 0 940 350" role="img" aria-label="所选张量位置的 32 层归一化梯度谱" data-spike-position-chart>
|
||||||
|
<rect class="spike-zone" x="0" y="24" width="0" height="280" data-spike-position-zone></rect>
|
||||||
|
<g data-spike-position-grid></g>
|
||||||
|
<polyline class="series green" points="" data-spike-position-line></polyline>
|
||||||
|
<g data-spike-position-points></g>
|
||||||
|
</svg>
|
||||||
|
<div class="chart-legend"><span><i class="green"></i>所选位置</span><span><i class="zone-key"></i>S = layer 21–25</span></div>
|
||||||
|
</div>
|
||||||
|
<aside>
|
||||||
|
<span>SELECTED READOUT</span>
|
||||||
|
<b data-spike-position-label>Attention 前输入</b>
|
||||||
|
<dl>
|
||||||
|
<div><dt>SPIKE CONTRAST</dt><dd data-spike-position-contrast>3.724×</dd></div>
|
||||||
|
<div><dt>PEAK LAYER</dt><dd data-spike-position-peak>22</dd></div>
|
||||||
|
<div><dt>PEAK / MEAN</dt><dd data-spike-position-peak-value>4.233×</dd></div>
|
||||||
|
</dl>
|
||||||
|
<p>六个位置都满足 contrast ≥ 1.5 的 3 / 3 seed 规则。</p>
|
||||||
|
</aside>
|
||||||
|
</div>
|
||||||
|
<div class="boundary-pair">
|
||||||
|
<article class="yes"><span>可以说</span><b>最早采样点已经看见尖峰</b><p><code>pre_attention_input</code> 是本站六点链中最早的可见位置。</p></article>
|
||||||
|
<article class="no"><span>不能说</span><b>尖峰起源于 Attention 输入</b><p>观测链不是干预链;更早 mixer 与跨层回传已经作用于该张量。</p></article>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="spike-panel" data-spike-panel="reductions" hidden>
|
||||||
|
<div class="panel-lead">
|
||||||
|
<div><span>III / ROBUSTNESS</span><h4>把 “RMS” 拆成四种合理算法,layer 21–25 仍然突出</h4></div>
|
||||||
|
<p>
|
||||||
|
预注册主家族只包含四种 reduction。每个 seed × reduction 必须同时满足:
|
||||||
|
contrast ≥ 1.5、top-5 与 S 至少重叠 3 层、相对 element RMS 的 Spearman ≥ .8。
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div class="lab-controls">
|
||||||
|
<label>SEED
|
||||||
|
<select data-spike-reduction-seed>
|
||||||
|
<option value="2026073001">2026073001</option>
|
||||||
|
<option value="2026073002">2026073002</option>
|
||||||
|
<option value="2026073003">2026073003</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<label>REDUCTION
|
||||||
|
<select data-spike-reduction>
|
||||||
|
<option value="element_rms">全元素 RMS</option>
|
||||||
|
<option value="token_rms_mean">Token RMS 均值</option>
|
||||||
|
<option value="token_rms_median">Token RMS 中位数</option>
|
||||||
|
<option value="token_rms_p95">Token RMS P95</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<span data-spike-reduction-state>SEED 2026073001 · 全元素 RMS</span>
|
||||||
|
</div>
|
||||||
|
<div class="position-layout reduction-layout">
|
||||||
|
<div class="chart-shell">
|
||||||
|
<header><b>POST-MLP · NORMALIZED BY LAYER MEAN</b><span>32 layers</span></header>
|
||||||
|
<svg viewBox="0 0 940 350" role="img" aria-label="所选 reduction 下的归一化层梯度谱" data-spike-reduction-chart>
|
||||||
|
<rect class="spike-zone" x="0" y="24" width="0" height="280" data-spike-reduction-zone></rect>
|
||||||
|
<g data-spike-reduction-grid></g>
|
||||||
|
<polyline class="series copper" points="" data-spike-reduction-line></polyline>
|
||||||
|
<g data-spike-reduction-points></g>
|
||||||
|
</svg>
|
||||||
|
</div>
|
||||||
|
<aside>
|
||||||
|
<span>THREE GATES</span>
|
||||||
|
<dl>
|
||||||
|
<div><dt>CONTRAST ≥ 1.5</dt><dd data-spike-reduction-contrast>3.046×</dd></div>
|
||||||
|
<div><dt>TOP-5 OVERLAP ≥ 3</dt><dd data-spike-reduction-overlap>4 / 5</dd></div>
|
||||||
|
<div><dt>SPEARMAN ≥ .8</dt><dd data-spike-reduction-rho>1.000</dd></div>
|
||||||
|
</dl>
|
||||||
|
<p class="pass-copy">PASS · 三个条件必须同时成立</p>
|
||||||
|
</aside>
|
||||||
|
</div>
|
||||||
|
<div class="reduction-table-wrap">
|
||||||
|
<table class="reduction-table">
|
||||||
|
<thead><tr><th>SEED</th><th>REDUCTION</th><th>CONTRAST</th><th>TOP-5 ∩ S</th><th>SPEARMAN</th><th>VERDICT</th></tr></thead>
|
||||||
|
<tbody>
|
||||||
|
{reductions.map((cell: any) => (
|
||||||
|
<tr>
|
||||||
|
<th>{cell.seed}</th>
|
||||||
|
<td>{reductionLabels[cell.reduction]}</td>
|
||||||
|
<td>{cell.spike_contrast.toFixed(3)}×</td>
|
||||||
|
<td>{cell.top_five_spike_overlap} / 5</td>
|
||||||
|
<td>{cell.spearman_vs_element_rms.toFixed(3)}</td>
|
||||||
|
<td class="good">PASS</td>
|
||||||
|
</tr>
|
||||||
|
))}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
<div class="plain-rule">
|
||||||
|
<b>判定原文</b><p><code>robust within the preregistered reduction family</code>——只对这四种、这三个 seed、这个诊断 batch 成立。</p>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="spike-panel" data-spike-panel="intervention" hidden>
|
||||||
|
<div class="panel-lead">
|
||||||
|
<div><span>IV / SAME-FORWARD INTERVENTION</span><h4>前向数值一字不动,只问梯度经过 mixer 时沿哪条边回去</h4></div>
|
||||||
|
<p>
|
||||||
|
干预覆盖全部 64 个 depth mixer 与 output mixer。它能定位“反向路径敏感性”,
|
||||||
|
不能替代重新训练,更不能把降幅解释成因果贡献百分比。
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div class="backward-diagram">
|
||||||
|
<div class="forward"><span>FORWARD / ALL MODES EXACT</span><b>α = softmax(q·k)</b><i>→</i><b>y = Σ αᵢvᵢ</b><i>→</i><b>same logits · same loss</b></div>
|
||||||
|
<div class="paths">
|
||||||
|
<article><span>LEARNED</span><b>∂q, ∂k, ∂v</b><p>正常回传:权重路径和值路径都保留。</p></article>
|
||||||
|
<article><span>DETACHED LEARNED</span><b><s>∂q, ∂k</s> · ∂v</b><p>α 数值不变,但切断 softmax / query / key 源梯度。</p></article>
|
||||||
|
<article class="accent"><span>UNIFORM VALUE BACKWARD</span><b><s>α</s> → 1/N</b><p>前向仍用 learned α;仅 value-backward 系数改成均匀。</p></article>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="lab-controls">
|
||||||
|
<label>SEED
|
||||||
|
<select data-spike-intervention-seed>
|
||||||
|
<option value="2026073001">2026073001</option>
|
||||||
|
<option value="2026073002">2026073002</option>
|
||||||
|
<option value="2026073003">2026073003</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<span data-spike-intervention-state>SEED 2026073001 · THREE BACKWARD RULES</span>
|
||||||
|
</div>
|
||||||
|
<div class="chart-shell">
|
||||||
|
<header><b>POST-MLP NORMALIZED GRADIENT</b><span>same forward · different backward</span></header>
|
||||||
|
<svg viewBox="0 0 940 350" role="img" aria-label="三种反向规则下的 32 层梯度谱" data-spike-intervention-chart>
|
||||||
|
<rect class="spike-zone" x="0" y="24" width="0" height="280" data-spike-intervention-zone></rect>
|
||||||
|
<g data-spike-intervention-grid></g>
|
||||||
|
<polyline class="series gray" points="" data-spike-intervention-line="learned"></polyline>
|
||||||
|
<polyline class="series copper dash" points="" data-spike-intervention-line="detached_learned"></polyline>
|
||||||
|
<polyline class="series green" points="" data-spike-intervention-line="uniform_value_backward"></polyline>
|
||||||
|
</svg>
|
||||||
|
<div class="chart-legend">
|
||||||
|
<span><i class="gray"></i>learned</span>
|
||||||
|
<span><i class="copper dashed"></i>detached learned</span>
|
||||||
|
<span><i class="green"></i>uniform value backward</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="intervention-results">
|
||||||
|
<article class="counter">
|
||||||
|
<span>LEARNED → DETACHED</span>
|
||||||
|
<b data-spike-key-drop>−1.54%</b>
|
||||||
|
<p>contrast “降幅”为负;移除 key / softmax 路径后,峰值在 3 / 3 seed 略微上升。</p>
|
||||||
|
</article>
|
||||||
|
<article class="pass">
|
||||||
|
<span>DETACHED → UNIFORM VALUE</span>
|
||||||
|
<b data-spike-value-drop>74.79%</b>
|
||||||
|
<p>contrast 在 3 / 3 seed 至少下降 20%,达到预注册 material sensitivity 阈值。</p>
|
||||||
|
</article>
|
||||||
|
<article>
|
||||||
|
<span>PEAK LAYER</span>
|
||||||
|
<b data-spike-peak-shift>21 → 2</b>
|
||||||
|
<p>均匀 value-backward 后,三个 seed 的最高层都从中后段移到 layer 2。</p>
|
||||||
|
</article>
|
||||||
|
</div>
|
||||||
|
<div class="causal-warning">
|
||||||
|
<b>最强允许结论</b>
|
||||||
|
<p>该尖峰对“全局 learned value-backward coefficients”具有材料级敏感性;softmax / query / key 源梯度路径没有显示同方向材料级作用。</p>
|
||||||
|
<span>不是训练出的 uniform 模型 · 不是 70.2% 因果贡献 · 不是某个局部 mixer 的归因</span>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="spike-panel" data-spike-panel="mixer" hidden>
|
||||||
|
<div class="panel-lead">
|
||||||
|
<div><span>V / ASSOCIATION × AUDIT</span><h4>mixer 权重给出一条可追的线索;干预才把它推进到“路径敏感”</h4></div>
|
||||||
|
<p>
|
||||||
|
每个点是 layer 19–28 的一个 layer × seed。横轴是 MLP mixer 的统计量,
|
||||||
|
纵轴是归一化梯度;相关性仍然只是观察关系。
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div class="lab-controls">
|
||||||
|
<div>
|
||||||
|
<button type="button" data-spike-mixer-metric="mlp_latest" aria-pressed="true">LATEST-SOURCE WEIGHT</button>
|
||||||
|
<button type="button" data-spike-mixer-metric="mlp_entropy" aria-pressed="false">NORMALIZED ENTROPY</button>
|
||||||
|
</div>
|
||||||
|
<span data-spike-mixer-state>MLP LATEST · LAYERS 19–28 · N=30</span>
|
||||||
|
</div>
|
||||||
|
<div class="mixer-layout">
|
||||||
|
<div class="chart-shell">
|
||||||
|
<header><b data-spike-mixer-title>MLP LATEST-SOURCE WEIGHT × NORMALIZED GRADIENT</b><span>3 seeds</span></header>
|
||||||
|
<svg viewBox="0 0 940 390" role="img" aria-label="Mixer 统计量和层梯度的散点关系" data-spike-mixer-chart>
|
||||||
|
<g data-spike-mixer-grid></g>
|
||||||
|
<line class="fit" x1="0" x2="0" y1="0" y2="0" data-spike-mixer-fit></line>
|
||||||
|
<g data-spike-mixer-points></g>
|
||||||
|
</svg>
|
||||||
|
</div>
|
||||||
|
<aside>
|
||||||
|
<span>LAYERS 19–28</span>
|
||||||
|
<dl>
|
||||||
|
<div><dt>PEARSON</dt><dd data-spike-mixer-pearson>+.690</dd></div>
|
||||||
|
<div><dt>SPEARMAN</dt><dd data-spike-mixer-spearman>+.693</dd></div>
|
||||||
|
<div><dt>POINTS</dt><dd>30</dd></div>
|
||||||
|
</dl>
|
||||||
|
<p data-spike-mixer-copy>越依赖最新 residual source 的 MLP mixer,局部梯度往往越高。</p>
|
||||||
|
</aside>
|
||||||
|
</div>
|
||||||
|
<div class="evidence-ladder">
|
||||||
|
<article><span>1 / OBSERVE</span><b>layer 21–25 集中</b><p>六位置、四 reduction、三 seed 都能看见。</p></article>
|
||||||
|
<i>→</i>
|
||||||
|
<article><span>2 / ASSOCIATE</span><b>MLP latest r≈.69</b><p>相关性提供候选机制,不承担因果结论。</p></article>
|
||||||
|
<i>→</i>
|
||||||
|
<article class="accent"><span>3 / INTERVENE</span><b>value route −70.2%</b><p>全局反向规则干预支持路径敏感性。</p></article>
|
||||||
|
<i>→</i>
|
||||||
|
<article><span>4 / NEXT GATE</span><b>局部 group 6 / 7</b><p>下一轮冻结局部 mixer 的干预矩阵。</p></article>
|
||||||
|
</div>
|
||||||
|
<div class="audit-grid">
|
||||||
|
<article><span>ROUND 05 EQUIVALENCE</span><b>3 / 3 exact</b><p>model、optimizer、history、六个 BPC 与全部 post-MLP 数组一致。</p></article>
|
||||||
|
<article><span>FULL REPLAY</span><b>16 / 16 exact</b><p>compare hash <code>{shortHash(replayHash)}</code></p></article>
|
||||||
|
<article><span>NEGATIVE CONTROL</span><b>step 0 exact</b><p>query-zero 与 loss×2 gate 在三 seed 全部通过。</p></article>
|
||||||
|
<article class="boundary"><span>EVIDENCE IDENTITY</span><b>targeted follow-up</b><p>尖峰集合来自 Round 05;本轮不是盲发现研究。</p></article>
|
||||||
|
</div>
|
||||||
|
<div class="hash-strip">
|
||||||
|
<span>PROTOCOL <code>{lab.protocol_id}</code></span>
|
||||||
|
<span>REPLAY <code>{shortHash(replayHash)}</code></span>
|
||||||
|
<span>COMPACT <code>{shortHash(lab.canonical_sha256_without_self)}</code></span>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<script is:inline type="application/json" data-spike-payload set:html={json}></script>
|
||||||
|
</figure>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
const initializeSpikeLab = (root: HTMLElement) => {
|
||||||
|
if (root.dataset.ready === "true") return;
|
||||||
|
root.dataset.ready = "true";
|
||||||
|
const payload = root.querySelector<HTMLScriptElement>("[data-spike-payload]");
|
||||||
|
if (!payload) return;
|
||||||
|
const data = JSON.parse(payload.textContent || "{}");
|
||||||
|
const ns = "http://www.w3.org/2000/svg";
|
||||||
|
const seeds = data.study.seeds;
|
||||||
|
const steps = data.trajectory[0].points.map((point: any) => point.step);
|
||||||
|
const labels: Record<string, string> = {
|
||||||
|
pre_attention_input: "Attention 前输入",
|
||||||
|
attention_branch_output: "Attention 分支输出",
|
||||||
|
post_attention_state: "Attention 后状态",
|
||||||
|
pre_mlp_input: "MLP 前输入",
|
||||||
|
mlp_branch_output: "MLP 分支输出",
|
||||||
|
post_mlp_state: "MLP 后状态",
|
||||||
|
element_rms: "全元素 RMS",
|
||||||
|
token_rms_mean: "Token RMS 均值",
|
||||||
|
token_rms_median: "Token RMS 中位数",
|
||||||
|
token_rms_p95: "Token RMS P95",
|
||||||
|
};
|
||||||
|
const colors = ["#ba603b", "#163f3b", "#77746b"];
|
||||||
|
const average = (values: number[]) => values.reduce((sum, value) => sum + value, 0) / values.length;
|
||||||
|
const svgNode = (name: string, attrs: Record<string, string | number>) => {
|
||||||
|
const node = document.createElementNS(ns, name);
|
||||||
|
Object.entries(attrs).forEach(([key, value]) => node.setAttribute(key, String(value)));
|
||||||
|
return node;
|
||||||
|
};
|
||||||
|
const seedFinal = (seed: number) => data.final_arrays.find((row: any) => row.seed === seed);
|
||||||
|
const formatSignedPct = (value: number) => `${value < 0 ? "−" : "+"}${Math.abs(value * 100).toFixed(2)}%`;
|
||||||
|
|
||||||
|
const tabs = Array.from(root.querySelectorAll<HTMLButtonElement>("[data-spike-tab]"));
|
||||||
|
const selectTab = (name: string) => {
|
||||||
|
tabs.forEach((tab) => tab.setAttribute("aria-selected", String(tab.dataset.spikeTab === name)));
|
||||||
|
root.querySelectorAll<HTMLElement>("[data-spike-panel]").forEach((panel) => {
|
||||||
|
panel.hidden = panel.dataset.spikePanel !== name;
|
||||||
|
});
|
||||||
|
};
|
||||||
|
tabs.forEach((tab, index) => {
|
||||||
|
tab.addEventListener("click", () => selectTab(tab.dataset.spikeTab || "trajectory"));
|
||||||
|
tab.addEventListener("keydown", (event) => {
|
||||||
|
if (!["ArrowLeft", "ArrowRight", "Home", "End"].includes(event.key)) return;
|
||||||
|
event.preventDefault();
|
||||||
|
const next = event.key === "Home" ? 0
|
||||||
|
: event.key === "End" ? tabs.length - 1
|
||||||
|
: (index + (event.key === "ArrowRight" ? 1 : -1) + tabs.length) % tabs.length;
|
||||||
|
tabs[next].focus();
|
||||||
|
selectTab(tabs[next].dataset.spikeTab || "trajectory");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
const drawLine = (
|
||||||
|
svg: SVGSVGElement,
|
||||||
|
grid: SVGGElement,
|
||||||
|
series: Record<string, number[]>,
|
||||||
|
xLabels: string[],
|
||||||
|
options: {
|
||||||
|
formatter?: (value: number) => string;
|
||||||
|
zero?: boolean;
|
||||||
|
pointGroup?: SVGGElement | null;
|
||||||
|
threshold?: number;
|
||||||
|
thresholdLine?: SVGLineElement | null;
|
||||||
|
zone?: SVGRectElement | null;
|
||||||
|
} = {},
|
||||||
|
) => {
|
||||||
|
grid.replaceChildren();
|
||||||
|
const allValues = Object.values(series).flat();
|
||||||
|
const rawMin = options.zero === false ? Math.min(...allValues) : 0;
|
||||||
|
const rawMax = Math.max(...allValues, options.threshold || 0);
|
||||||
|
const padding = Math.max(1e-8, (rawMax - rawMin) * .08);
|
||||||
|
const min = options.zero === false ? rawMin - padding : 0;
|
||||||
|
const max = rawMax + padding;
|
||||||
|
const left = 58;
|
||||||
|
const right = 900;
|
||||||
|
const top = 24;
|
||||||
|
const bottom = 304;
|
||||||
|
const xAt = (index: number) => left + index / Math.max(1, xLabels.length - 1) * (right - left);
|
||||||
|
const yAt = (value: number) => top + (max - value) / Math.max(1e-30, max - min) * (bottom - top);
|
||||||
|
const formatter = options.formatter || ((value: number) => value.toFixed(2));
|
||||||
|
for (let index = 0; index < 5; index += 1) {
|
||||||
|
const value = max - index / 4 * (max - min);
|
||||||
|
const y = top + index / 4 * (bottom - top);
|
||||||
|
grid.append(svgNode("line", { x1: left, x2: right, y1: y, y2: y, class: "grid-line" }));
|
||||||
|
const text = svgNode("text", { x: left - 10, y: y + 4, class: "axis-label y" });
|
||||||
|
text.textContent = formatter(value);
|
||||||
|
grid.append(text);
|
||||||
|
}
|
||||||
|
xLabels.forEach((label, index) => {
|
||||||
|
if (xLabels.length > 12 && ![0, 3, 7, 11, 15, 19, 20, 24, 27, 31].includes(index)) return;
|
||||||
|
const text = svgNode("text", { x: xAt(index), y: 328, class: "axis-label x" });
|
||||||
|
text.textContent = label;
|
||||||
|
grid.append(text);
|
||||||
|
});
|
||||||
|
if (options.thresholdLine && options.threshold !== undefined) {
|
||||||
|
const y = yAt(options.threshold);
|
||||||
|
options.thresholdLine.setAttribute("y1", String(y));
|
||||||
|
options.thresholdLine.setAttribute("y2", String(y));
|
||||||
|
}
|
||||||
|
if (options.zone && xLabels.length === 32) {
|
||||||
|
const start = xAt(20) - (right - left) / 62;
|
||||||
|
const end = xAt(24) + (right - left) / 62;
|
||||||
|
options.zone.setAttribute("x", String(start));
|
||||||
|
options.zone.setAttribute("width", String(end - start));
|
||||||
|
}
|
||||||
|
Object.entries(series).forEach(([key, values], seriesIndex) => {
|
||||||
|
const line = svg.querySelector<SVGPolylineElement>(`[data-line-key="${key}"]`)
|
||||||
|
|| svg.querySelectorAll<SVGPolylineElement>("polyline")[seriesIndex];
|
||||||
|
const points = values.map((value, index) => `${xAt(index)},${yAt(value)}`).join(" ");
|
||||||
|
line?.setAttribute("points", points);
|
||||||
|
if (seriesIndex === 0 && options.pointGroup) {
|
||||||
|
options.pointGroup.replaceChildren();
|
||||||
|
values.forEach((value, index) => {
|
||||||
|
const circle = svgNode("circle", { cx: xAt(index), cy: yAt(value), r: 4, fill: colors[seriesIndex] });
|
||||||
|
const title = svgNode("title", {});
|
||||||
|
title.textContent = `${xLabels[index]} · ${formatter(value)}`;
|
||||||
|
circle.append(title);
|
||||||
|
options.pointGroup?.append(circle);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
return { min, max, xAt, yAt };
|
||||||
|
};
|
||||||
|
|
||||||
|
let timeMetric = "spike_contrast";
|
||||||
|
const timeSeed = root.querySelector<HTMLSelectElement>("[data-spike-time-seed]")!;
|
||||||
|
const timeChart = root.querySelector<SVGSVGElement>("[data-spike-time-chart]")!;
|
||||||
|
const updateTime = () => {
|
||||||
|
const selected = timeSeed.value === "mean"
|
||||||
|
? data.trajectory
|
||||||
|
: data.trajectory.filter((row: any) => String(row.seed) === timeSeed.value);
|
||||||
|
const values = steps.map((_: number, index: number) =>
|
||||||
|
average(selected.map((row: any) => row.points[index][timeMetric])));
|
||||||
|
const names: Record<string, string> = {
|
||||||
|
spike_contrast: "SPIKE CONTRAST · S / REST",
|
||||||
|
population_cv: "POPULATION CV ACROSS 32 LAYERS",
|
||||||
|
peak_normalized: "PEAK / LAYER MEAN",
|
||||||
|
};
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-time-title]")!.textContent = names[timeMetric];
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-time-state]")!.textContent =
|
||||||
|
timeSeed.value === "mean" ? "3-SEED MEAN" : `SEED ${timeSeed.value}`;
|
||||||
|
drawLine(
|
||||||
|
timeChart,
|
||||||
|
timeChart.querySelector("[data-spike-time-grid]")!,
|
||||||
|
{ trajectory: values },
|
||||||
|
steps.map((step: number) => step >= 1000 ? `${step / 1000}K` : String(step)),
|
||||||
|
{
|
||||||
|
formatter: (value) => value.toFixed(2),
|
||||||
|
pointGroup: timeChart.querySelector<SVGGElement>("[data-spike-time-points]"),
|
||||||
|
threshold: timeMetric === "spike_contrast" ? 1.5 : undefined,
|
||||||
|
thresholdLine: timeChart.querySelector<SVGLineElement>("[data-spike-time-threshold]"),
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const threshold = timeChart.querySelector<SVGLineElement>("[data-spike-time-threshold]");
|
||||||
|
if (threshold) threshold.style.display = timeMetric === "spike_contrast" ? "" : "none";
|
||||||
|
};
|
||||||
|
root.querySelectorAll<HTMLButtonElement>("[data-spike-time-metric]").forEach((button) => button.addEventListener("click", () => {
|
||||||
|
timeMetric = button.dataset.spikeTimeMetric || "spike_contrast";
|
||||||
|
root.querySelectorAll<HTMLButtonElement>("[data-spike-time-metric]").forEach((peer) =>
|
||||||
|
peer.setAttribute("aria-pressed", String(peer === button)));
|
||||||
|
updateTime();
|
||||||
|
}));
|
||||||
|
timeSeed.addEventListener("change", updateTime);
|
||||||
|
updateTime();
|
||||||
|
|
||||||
|
let positionName = "pre_attention_input";
|
||||||
|
const positionSeed = root.querySelector<HTMLSelectElement>("[data-spike-position-seed]")!;
|
||||||
|
const positionChart = root.querySelector<SVGSVGElement>("[data-spike-position-chart]")!;
|
||||||
|
const updatePosition = () => {
|
||||||
|
const seed = Number(positionSeed.value);
|
||||||
|
const cell = seedFinal(seed).position_element_rms[positionName];
|
||||||
|
drawLine(
|
||||||
|
positionChart,
|
||||||
|
positionChart.querySelector("[data-spike-position-grid]")!,
|
||||||
|
{ position: cell.statistics.normalized },
|
||||||
|
Array.from({ length: 32 }, (_, index) => String(index + 1)),
|
||||||
|
{
|
||||||
|
formatter: (value) => `${value.toFixed(1)}×`,
|
||||||
|
pointGroup: positionChart.querySelector<SVGGElement>("[data-spike-position-points]"),
|
||||||
|
zone: positionChart.querySelector<SVGRectElement>("[data-spike-position-zone]"),
|
||||||
|
},
|
||||||
|
);
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-position-state]")!.textContent = `${labels[positionName].toUpperCase()} · SEED ${seed}`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-position-label]")!.textContent = labels[positionName];
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-position-contrast]")!.textContent = `${cell.statistics.spike_contrast.toFixed(3)}×`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-position-peak]")!.textContent = String(cell.statistics.peak_layer);
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-position-peak-value]")!.textContent = `${cell.statistics.peak_normalized.toFixed(3)}×`;
|
||||||
|
};
|
||||||
|
root.querySelectorAll<HTMLButtonElement>("[data-spike-position]").forEach((button) => button.addEventListener("click", () => {
|
||||||
|
positionName = button.dataset.spikePosition || "pre_attention_input";
|
||||||
|
root.querySelectorAll<HTMLButtonElement>("[data-spike-position]").forEach((peer) =>
|
||||||
|
peer.setAttribute("aria-pressed", String(peer === button)));
|
||||||
|
updatePosition();
|
||||||
|
}));
|
||||||
|
positionSeed.addEventListener("change", updatePosition);
|
||||||
|
updatePosition();
|
||||||
|
|
||||||
|
const reductionSeed = root.querySelector<HTMLSelectElement>("[data-spike-reduction-seed]")!;
|
||||||
|
const reductionSelect = root.querySelector<HTMLSelectElement>("[data-spike-reduction]")!;
|
||||||
|
const reductionChart = root.querySelector<SVGSVGElement>("[data-spike-reduction-chart]")!;
|
||||||
|
const updateReduction = () => {
|
||||||
|
const seed = Number(reductionSeed.value);
|
||||||
|
const name = reductionSelect.value;
|
||||||
|
const cell = seedFinal(seed).post_mlp_reductions[name];
|
||||||
|
const verdict = data.verdicts.reduction_robustness.cells.find((row: any) => row.seed === seed && row.reduction === name);
|
||||||
|
drawLine(
|
||||||
|
reductionChart,
|
||||||
|
reductionChart.querySelector("[data-spike-reduction-grid]")!,
|
||||||
|
{ reduction: cell.statistics.normalized },
|
||||||
|
Array.from({ length: 32 }, (_, index) => String(index + 1)),
|
||||||
|
{
|
||||||
|
formatter: (value) => `${value.toFixed(1)}×`,
|
||||||
|
pointGroup: reductionChart.querySelector<SVGGElement>("[data-spike-reduction-points]"),
|
||||||
|
zone: reductionChart.querySelector<SVGRectElement>("[data-spike-reduction-zone]"),
|
||||||
|
},
|
||||||
|
);
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-reduction-state]")!.textContent = `SEED ${seed} · ${labels[name]}`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-reduction-contrast]")!.textContent = `${verdict.spike_contrast.toFixed(3)}×`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-reduction-overlap]")!.textContent = `${verdict.top_five_spike_overlap} / 5`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-reduction-rho]")!.textContent = verdict.spearman_vs_element_rms.toFixed(3);
|
||||||
|
};
|
||||||
|
reductionSeed.addEventListener("change", updateReduction);
|
||||||
|
reductionSelect.addEventListener("change", updateReduction);
|
||||||
|
updateReduction();
|
||||||
|
|
||||||
|
const interventionSeed = root.querySelector<HTMLSelectElement>("[data-spike-intervention-seed]")!;
|
||||||
|
const interventionChart = root.querySelector<SVGSVGElement>("[data-spike-intervention-chart]")!;
|
||||||
|
const updateIntervention = () => {
|
||||||
|
const seed = Number(interventionSeed.value);
|
||||||
|
const cell = seedFinal(seed);
|
||||||
|
const modes = ["learned", "detached_learned", "uniform_value_backward"];
|
||||||
|
const series = Object.fromEntries(modes.map((mode) => [mode, cell.interventions[mode].statistics.normalized]));
|
||||||
|
modes.forEach((mode) => interventionChart.querySelector(`[data-spike-intervention-line="${mode}"]`)?.setAttribute("data-line-key", mode));
|
||||||
|
drawLine(
|
||||||
|
interventionChart,
|
||||||
|
interventionChart.querySelector("[data-spike-intervention-grid]")!,
|
||||||
|
series,
|
||||||
|
Array.from({ length: 32 }, (_, index) => String(index + 1)),
|
||||||
|
{
|
||||||
|
formatter: (value) => `${value.toFixed(1)}×`,
|
||||||
|
zone: interventionChart.querySelector<SVGRectElement>("[data-spike-intervention-zone]"),
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const keyRow = data.verdicts.interventions.softmax_key_path.per_seed.find((row: any) => row.seed === seed);
|
||||||
|
const valueRow = data.verdicts.interventions.value_coefficients.per_seed.find((row: any) => row.seed === seed);
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-intervention-state]")!.textContent = `SEED ${seed} · THREE BACKWARD RULES`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-key-drop]")!.textContent = formatSignedPct(keyRow.relative_drop_contrast);
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-value-drop]")!.textContent = `${(valueRow.relative_drop_contrast * 100).toFixed(2)}%`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-peak-shift]")!.textContent =
|
||||||
|
`${cell.interventions.detached_learned.statistics.peak_layer} → ${cell.interventions.uniform_value_backward.statistics.peak_layer}`;
|
||||||
|
};
|
||||||
|
interventionSeed.addEventListener("change", updateIntervention);
|
||||||
|
updateIntervention();
|
||||||
|
|
||||||
|
let mixerMetric = "mlp_latest";
|
||||||
|
const mixerChart = root.querySelector<SVGSVGElement>("[data-spike-mixer-chart]")!;
|
||||||
|
const mixerPoints = data.mixer_associations.target_layers.flatMap((row: any) =>
|
||||||
|
row.per_seed_normalized_gradient.map((gradient: number, index: number) => ({
|
||||||
|
x: row[mixerMetric],
|
||||||
|
y: gradient,
|
||||||
|
seed: seeds[index],
|
||||||
|
layer: row.layer,
|
||||||
|
})));
|
||||||
|
const updateMixer = () => {
|
||||||
|
const points = data.mixer_associations.target_layers.flatMap((row: any) =>
|
||||||
|
row.per_seed_normalized_gradient.map((gradient: number, index: number) => ({
|
||||||
|
x: row[mixerMetric],
|
||||||
|
y: gradient,
|
||||||
|
seed: seeds[index],
|
||||||
|
layer: row.layer,
|
||||||
|
})));
|
||||||
|
const group = mixerChart.querySelector<SVGGElement>("[data-spike-mixer-points]")!;
|
||||||
|
const grid = mixerChart.querySelector<SVGGElement>("[data-spike-mixer-grid]")!;
|
||||||
|
const fit = mixerChart.querySelector<SVGLineElement>("[data-spike-mixer-fit]")!;
|
||||||
|
group.replaceChildren();
|
||||||
|
grid.replaceChildren();
|
||||||
|
const left = 72, right = 900, top = 24, bottom = 330;
|
||||||
|
const xs = points.map((point: any) => point.x);
|
||||||
|
const ys = points.map((point: any) => point.y);
|
||||||
|
const minX = Math.min(...xs) * .94;
|
||||||
|
const maxX = Math.max(...xs) * 1.04;
|
||||||
|
const minY = Math.max(0, Math.min(...ys) * .88);
|
||||||
|
const maxY = Math.max(...ys) * 1.08;
|
||||||
|
const xAt = (value: number) => left + (value - minX) / (maxX - minX) * (right - left);
|
||||||
|
const yAt = (value: number) => top + (maxY - value) / (maxY - minY) * (bottom - top);
|
||||||
|
for (let index = 0; index < 5; index += 1) {
|
||||||
|
const yValue = maxY - index / 4 * (maxY - minY);
|
||||||
|
const y = top + index / 4 * (bottom - top);
|
||||||
|
grid.append(svgNode("line", { x1: left, x2: right, y1: y, y2: y, class: "grid-line" }));
|
||||||
|
const yText = svgNode("text", { x: left - 10, y: y + 4, class: "axis-label y" });
|
||||||
|
yText.textContent = `${yValue.toFixed(1)}×`;
|
||||||
|
grid.append(yText);
|
||||||
|
const xValue = minX + index / 4 * (maxX - minX);
|
||||||
|
const x = left + index / 4 * (right - left);
|
||||||
|
const xText = svgNode("text", { x, y: 355, class: "axis-label x" });
|
||||||
|
xText.textContent = xValue.toFixed(2);
|
||||||
|
grid.append(xText);
|
||||||
|
}
|
||||||
|
points.forEach((point: any, index: number) => {
|
||||||
|
const circle = svgNode("circle", {
|
||||||
|
cx: xAt(point.x), cy: yAt(point.y), r: point.layer >= 21 && point.layer <= 25 ? 6 : 4,
|
||||||
|
fill: colors[index % 3], opacity: point.layer >= 21 && point.layer <= 25 ? .95 : .55,
|
||||||
|
});
|
||||||
|
const title = svgNode("title", {});
|
||||||
|
title.textContent = `L${point.layer} · ${point.seed} · x=${point.x.toFixed(3)} · gradient=${point.y.toFixed(3)}×`;
|
||||||
|
circle.append(title);
|
||||||
|
group.append(circle);
|
||||||
|
});
|
||||||
|
const meanX = average(xs);
|
||||||
|
const meanY = average(ys);
|
||||||
|
const slope = xs.reduce((sum: number, x: number, index: number) => sum + (x - meanX) * (ys[index] - meanY), 0)
|
||||||
|
/ xs.reduce((sum: number, x: number) => sum + (x - meanX) ** 2, 0);
|
||||||
|
const intercept = meanY - slope * meanX;
|
||||||
|
fit.setAttribute("x1", String(xAt(minX)));
|
||||||
|
fit.setAttribute("x2", String(xAt(maxX)));
|
||||||
|
fit.setAttribute("y1", String(yAt(intercept + slope * minX)));
|
||||||
|
fit.setAttribute("y2", String(yAt(intercept + slope * maxX)));
|
||||||
|
const stats = data.mixer_associations.layers_19_28[mixerMetric];
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-mixer-pearson]")!.textContent = `${stats.pearson >= 0 ? "+" : "−"}${Math.abs(stats.pearson).toFixed(3)}`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-mixer-spearman]")!.textContent = `${stats.spearman >= 0 ? "+" : "−"}${Math.abs(stats.spearman).toFixed(3)}`;
|
||||||
|
const latest = mixerMetric === "mlp_latest";
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-mixer-state]")!.textContent =
|
||||||
|
`${latest ? "MLP LATEST" : "MLP ENTROPY"} · LAYERS 19–28 · N=30`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-mixer-title]")!.textContent =
|
||||||
|
`${latest ? "MLP LATEST-SOURCE WEIGHT" : "MLP NORMALIZED ENTROPY"} × NORMALIZED GRADIENT`;
|
||||||
|
root.querySelector<HTMLElement>("[data-spike-mixer-copy]")!.textContent = latest
|
||||||
|
? "越依赖最新 residual source 的 MLP mixer,局部梯度往往越高。"
|
||||||
|
: "Mixer 越分散、熵越高,局部梯度往往越低;这仍不是局部因果归因。";
|
||||||
|
};
|
||||||
|
root.querySelectorAll<HTMLButtonElement>("[data-spike-mixer-metric]").forEach((button) => button.addEventListener("click", () => {
|
||||||
|
mixerMetric = button.dataset.spikeMixerMetric || "mlp_latest";
|
||||||
|
root.querySelectorAll<HTMLButtonElement>("[data-spike-mixer-metric]").forEach((peer) =>
|
||||||
|
peer.setAttribute("aria-pressed", String(peer === button)));
|
||||||
|
updateMixer();
|
||||||
|
}));
|
||||||
|
void mixerPoints;
|
||||||
|
updateMixer();
|
||||||
|
};
|
||||||
|
|
||||||
|
document.querySelectorAll<HTMLElement>("[data-spike-lab]").forEach(initializeSpikeLab);
|
||||||
|
document.addEventListener("astro:page-load", () => {
|
||||||
|
document.querySelectorAll<HTMLElement>("[data-spike-lab]").forEach(initializeSpikeLab);
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<style>
|
||||||
|
.spike-lab {
|
||||||
|
--s-ink: #1c201e;
|
||||||
|
--s-muted: #747168;
|
||||||
|
--s-line: rgba(28, 32, 30, .16);
|
||||||
|
--s-paper: #f4f0e7;
|
||||||
|
--s-raised: #faf7ef;
|
||||||
|
--s-copper: #ba603b;
|
||||||
|
--s-green: #163f3b;
|
||||||
|
width: min(1120px, 100%);
|
||||||
|
margin: 42px 0;
|
||||||
|
color: var(--s-ink);
|
||||||
|
border: 1px solid var(--s-line);
|
||||||
|
background: var(--s-paper);
|
||||||
|
box-shadow: 0 30px 80px rgba(28, 32, 30, .09);
|
||||||
|
}
|
||||||
|
.spike-head {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: minmax(0, 1.45fr) minmax(260px, .7fr);
|
||||||
|
gap: 44px;
|
||||||
|
padding: 30px;
|
||||||
|
color: #f5efe4;
|
||||||
|
background: var(--s-green);
|
||||||
|
}
|
||||||
|
.spike-head p { margin: 0; color: rgba(245,239,228,.7); font: .65rem/1.7 var(--mono); }
|
||||||
|
.spike-head div > p { color: #d58a68; letter-spacing: .08em; }
|
||||||
|
.spike-head h3 { max-width: 760px; margin: 14px 0 0; color: inherit; font-size: clamp(1.15rem, 2.2vw, 1.75rem); line-height: 1.35; }
|
||||||
|
.spike-ledger { display: grid; grid-template-columns: repeat(6, 1fr); border-bottom: 1px solid var(--s-line); }
|
||||||
|
.spike-ledger article { min-height: 126px; padding: 18px 15px; border-right: 1px solid var(--s-line); }
|
||||||
|
.spike-ledger article:last-child { border-right: 0; }
|
||||||
|
.spike-ledger span, .panel-lead span { color: var(--s-muted); font: .56rem/1.2 var(--mono); letter-spacing: .08em; }
|
||||||
|
.spike-ledger b { display: block; margin-top: 23px; font: 700 .95rem/1 var(--mono); }
|
||||||
|
.spike-ledger p { margin: 8px 0 0; color: var(--s-muted); font-size: .6rem; line-height: 1.45; }
|
||||||
|
.spike-ledger .pass { color: #f7f0e6; background: var(--s-green); }
|
||||||
|
.spike-ledger .counter { color: #f7f0e6; background: var(--s-copper); }
|
||||||
|
.spike-ledger .pass span, .spike-ledger .pass p, .spike-ledger .counter span, .spike-ledger .counter p { color: rgba(247,240,230,.72); }
|
||||||
|
.spike-tabs { display: grid; grid-template-columns: repeat(5, 1fr); border-bottom: 1px solid var(--s-line); background: #e9e4da; }
|
||||||
|
.spike-tabs button { min-height: 116px; padding: 16px; text-align: left; color: inherit; border: 0; border-right: 1px solid var(--s-line); background: transparent; cursor: pointer; }
|
||||||
|
.spike-tabs button:last-child { border-right: 0; }
|
||||||
|
.spike-tabs button[aria-selected="true"] { color: #f7f0e6; background: var(--s-copper); }
|
||||||
|
.spike-tabs span, .spike-tabs small { display: block; color: var(--s-muted); font: .54rem/1.25 var(--mono); }
|
||||||
|
.spike-tabs b { display: block; margin: 15px 0 8px; font-size: .69rem; line-height: 1.35; }
|
||||||
|
.spike-tabs button[aria-selected="true"] span, .spike-tabs button[aria-selected="true"] small { color: rgba(247,240,230,.72); }
|
||||||
|
.spike-panel { padding: 30px; }
|
||||||
|
.panel-lead { display: grid; grid-template-columns: 1.05fr .95fr; gap: 48px; align-items: end; margin-bottom: 28px; }
|
||||||
|
.panel-lead h4 { max-width: 680px; margin: 10px 0 0; font-size: 1.2rem; line-height: 1.4; }
|
||||||
|
.panel-lead p { margin: 0; color: var(--s-muted); font-size: .7rem; line-height: 1.7; }
|
||||||
|
.lab-controls { display: flex; flex-wrap: wrap; gap: 10px 18px; align-items: end; margin-bottom: 20px; }
|
||||||
|
.lab-controls > div { display: flex; }
|
||||||
|
.lab-controls button, .lab-controls select { min-height: 38px; padding: 10px 12px; color: var(--s-muted); font: 700 .56rem/1 var(--mono); border: 1px solid var(--s-line); background: var(--s-raised); }
|
||||||
|
.lab-controls button { cursor: pointer; }
|
||||||
|
.lab-controls button + button { border-left: 0; }
|
||||||
|
.lab-controls button[aria-pressed="true"] { color: #fff9ef; background: var(--s-green); }
|
||||||
|
.lab-controls label { display: grid; gap: 6px; color: var(--s-muted); font: .52rem/1 var(--mono); }
|
||||||
|
.lab-controls > span { margin: 0 0 11px auto; color: var(--s-copper); font: .55rem/1 var(--mono); }
|
||||||
|
.chart-shell { min-width: 0; padding: 18px; border: 1px solid var(--s-line); background: var(--s-raised); }
|
||||||
|
.chart-shell header { display: flex; justify-content: space-between; gap: 12px; color: var(--s-muted); font: .55rem/1 var(--mono); }
|
||||||
|
.chart-shell svg { display: block; width: 100%; height: auto; margin-top: 10px; overflow: visible; }
|
||||||
|
.series { fill: none; stroke-width: 3; stroke-linejoin: round; stroke-linecap: round; }
|
||||||
|
.series.copper { stroke: var(--s-copper); }
|
||||||
|
.series.green { stroke: var(--s-green); }
|
||||||
|
.series.gray { stroke: #77746b; }
|
||||||
|
.series.dash { stroke-dasharray: 8 7; }
|
||||||
|
.grid-line { stroke: rgba(28,32,30,.1); stroke-width: 1; }
|
||||||
|
.threshold { stroke: var(--s-copper); stroke-width: 1.5; stroke-dasharray: 6 5; }
|
||||||
|
.spike-zone { fill: rgba(186,96,59,.11); }
|
||||||
|
.axis-label { fill: #8b867c; font: 11px var(--mono); }
|
||||||
|
.axis-label.y { text-anchor: end; }
|
||||||
|
.axis-label.x { text-anchor: middle; }
|
||||||
|
.chart-legend { display: flex; flex-wrap: wrap; gap: 18px; margin-top: 4px; color: var(--s-muted); font: .56rem/1 var(--mono); }
|
||||||
|
.chart-legend span { display: inline-flex; gap: 7px; align-items: center; }
|
||||||
|
.chart-legend i { width: 22px; height: 3px; }
|
||||||
|
.chart-legend i.copper { background: var(--s-copper); }
|
||||||
|
.chart-legend i.green { background: var(--s-green); }
|
||||||
|
.chart-legend i.gray { background: #77746b; }
|
||||||
|
.chart-legend i.dashed { height: 0; border-top: 3px dashed var(--s-copper); background: transparent; }
|
||||||
|
.chart-legend i.threshold-key { height: 0; border-top: 2px dashed var(--s-copper); }
|
||||||
|
.chart-legend i.zone-key { width: 18px; height: 12px; background: rgba(186,96,59,.16); }
|
||||||
|
.trajectory-story, .evidence-ladder { display: grid; grid-template-columns: 1fr 26px 1fr 26px 1fr 26px 1fr; gap: 7px; align-items: center; margin-top: 20px; }
|
||||||
|
.trajectory-story article, .evidence-ladder article { min-height: 150px; padding: 18px; border: 1px solid var(--s-line); background: var(--s-raised); }
|
||||||
|
.trajectory-story article.accent, .evidence-ladder article.accent { color: #f7f0e6; background: var(--s-green); }
|
||||||
|
.trajectory-story span, .evidence-ladder span, .position-layout aside > span, .backward-diagram span, .intervention-results span, .causal-warning span, .audit-grid span {
|
||||||
|
color: var(--s-copper); font: .56rem/1 var(--mono); letter-spacing: .06em;
|
||||||
|
}
|
||||||
|
.trajectory-story .accent span, .trajectory-story .accent p, .evidence-ladder .accent span, .evidence-ladder .accent p { color: rgba(247,240,230,.7); }
|
||||||
|
.trajectory-story b, .evidence-ladder b { display: block; margin-top: 25px; font-size: .78rem; }
|
||||||
|
.trajectory-story p, .evidence-ladder p { color: var(--s-muted); font-size: .61rem; line-height: 1.5; }
|
||||||
|
.trajectory-story > i, .evidence-ladder > i, .tensor-path > i { color: var(--s-copper); font-style: normal; text-align: center; }
|
||||||
|
.plain-rule { display: grid; grid-template-columns: 120px 1fr; margin-top: 20px; color: #f7f0e6; background: var(--s-green); }
|
||||||
|
.plain-rule > * { margin: 0; padding: 17px; }
|
||||||
|
.plain-rule b { color: #d58a68; font: .6rem/1.6 var(--mono); }
|
||||||
|
.plain-rule p { font-size: .67rem; line-height: 1.6; }
|
||||||
|
.tensor-path { display: grid; grid-template-columns: repeat(11, auto); align-items: stretch; margin-bottom: 22px; overflow-x: auto; }
|
||||||
|
.tensor-path button { min-width: 130px; min-height: 145px; padding: 16px; text-align: left; color: inherit; border: 1px solid var(--s-line); background: var(--s-raised); cursor: pointer; }
|
||||||
|
.tensor-path button[aria-pressed="true"] { color: #f7f0e6; background: var(--s-green); }
|
||||||
|
.tensor-path span, .tensor-path small { display: block; color: var(--s-muted); font: .5rem/1.3 var(--mono); }
|
||||||
|
.tensor-path b { display: block; margin: 24px 0 10px; font-size: .68rem; }
|
||||||
|
.tensor-path button[aria-pressed="true"] span, .tensor-path button[aria-pressed="true"] small { color: rgba(247,240,230,.68); }
|
||||||
|
.tensor-path > i { align-self: center; padding: 0 6px; }
|
||||||
|
.position-layout, .mixer-layout { display: grid; grid-template-columns: minmax(0, 1fr) 235px; border: 1px solid var(--s-line); background: var(--s-raised); }
|
||||||
|
.position-layout .chart-shell, .mixer-layout .chart-shell { border: 0; border-right: 1px solid var(--s-line); }
|
||||||
|
.position-layout aside, .mixer-layout aside { padding: 20px 18px; }
|
||||||
|
.position-layout aside > b { display: block; margin-top: 18px; font-size: .86rem; }
|
||||||
|
.position-layout dl, .mixer-layout dl { margin: 24px 0 0; }
|
||||||
|
.position-layout dl div, .mixer-layout dl div { display: flex; justify-content: space-between; gap: 12px; padding: 15px 0; border-bottom: 1px solid var(--s-line); }
|
||||||
|
.position-layout dt, .mixer-layout dt { color: var(--s-muted); font: .5rem/1.2 var(--mono); }
|
||||||
|
.position-layout dd, .mixer-layout dd { margin: 0; font: 700 .68rem/1 var(--mono); }
|
||||||
|
.position-layout aside p, .mixer-layout aside p { color: var(--s-muted); font-size: .61rem; line-height: 1.55; }
|
||||||
|
.pass-copy { color: var(--s-green) !important; font: 700 .55rem/1.5 var(--mono) !important; }
|
||||||
|
.boundary-pair { display: grid; grid-template-columns: repeat(2, 1fr); margin-top: 20px; }
|
||||||
|
.boundary-pair article { min-height: 170px; padding: 22px; }
|
||||||
|
.boundary-pair .yes { color: #f7f0e6; background: var(--s-green); }
|
||||||
|
.boundary-pair .no { background: #e7d8ca; }
|
||||||
|
.boundary-pair span { color: var(--s-copper); font: .56rem/1 var(--mono); }
|
||||||
|
.boundary-pair .yes span, .boundary-pair .yes p { color: rgba(247,240,230,.72); }
|
||||||
|
.boundary-pair b { display: block; margin-top: 25px; font-size: .83rem; }
|
||||||
|
.boundary-pair p { color: var(--s-muted); font-size: .63rem; line-height: 1.55; }
|
||||||
|
.reduction-table-wrap { margin-top: 20px; overflow-x: auto; }
|
||||||
|
.reduction-table { width: 100%; min-width: 790px; border-collapse: collapse; font-size: .62rem; }
|
||||||
|
.reduction-table th, .reduction-table td { padding: 13px 11px; border-bottom: 1px solid var(--s-line); text-align: left; }
|
||||||
|
.reduction-table thead th { color: var(--s-muted); font: .52rem/1.3 var(--mono); }
|
||||||
|
.reduction-table tbody th, .reduction-table .good { font: 700 .61rem/1 var(--mono); }
|
||||||
|
.reduction-table .good { color: var(--s-green); }
|
||||||
|
.backward-diagram { border: 1px solid var(--s-line); background: var(--s-raised); }
|
||||||
|
.backward-diagram .forward { display: flex; flex-wrap: wrap; gap: 14px; align-items: center; padding: 18px; color: #f7f0e6; background: var(--s-green); }
|
||||||
|
.backward-diagram .forward span { margin-right: auto; color: #d58a68; }
|
||||||
|
.backward-diagram .forward b { font: 700 .65rem/1 var(--mono); }
|
||||||
|
.backward-diagram .forward i { color: #d58a68; font-style: normal; }
|
||||||
|
.backward-diagram .paths { display: grid; grid-template-columns: repeat(3, 1fr); }
|
||||||
|
.backward-diagram article { min-height: 180px; padding: 20px; border-right: 1px solid var(--s-line); }
|
||||||
|
.backward-diagram article:last-child { border-right: 0; }
|
||||||
|
.backward-diagram article.accent { background: #e7d8ca; }
|
||||||
|
.backward-diagram article b { display: block; margin-top: 30px; font-size: .85rem; }
|
||||||
|
.backward-diagram article p { color: var(--s-muted); font-size: .62rem; line-height: 1.55; }
|
||||||
|
.intervention-results { display: grid; grid-template-columns: repeat(3, 1fr); margin-top: 20px; }
|
||||||
|
.intervention-results article { min-height: 185px; padding: 20px; border: 1px solid var(--s-line); background: var(--s-raised); }
|
||||||
|
.intervention-results .counter { color: #f7f0e6; background: var(--s-copper); }
|
||||||
|
.intervention-results .pass { color: #f7f0e6; background: var(--s-green); }
|
||||||
|
.intervention-results .counter span, .intervention-results .counter p, .intervention-results .pass span, .intervention-results .pass p { color: rgba(247,240,230,.72); }
|
||||||
|
.intervention-results b { display: block; margin-top: 28px; font: 700 1.1rem/1 var(--mono); }
|
||||||
|
.intervention-results p { color: var(--s-muted); font-size: .62rem; line-height: 1.55; }
|
||||||
|
.causal-warning { display: grid; grid-template-columns: 145px 1fr; margin-top: 20px; padding: 20px; border-left: 5px solid var(--s-copper); background: #e7d8ca; }
|
||||||
|
.causal-warning b { font-size: .78rem; }
|
||||||
|
.causal-warning p { margin: 0; font-size: .67rem; line-height: 1.6; }
|
||||||
|
.causal-warning span { grid-column: 2; margin-top: 13px; line-height: 1.5; }
|
||||||
|
.fit { stroke: var(--s-copper); stroke-width: 2; stroke-dasharray: 8 6; }
|
||||||
|
.audit-grid { display: grid; grid-template-columns: repeat(4, 1fr); margin-top: 20px; }
|
||||||
|
.audit-grid article { min-height: 175px; padding: 18px; border: 1px solid var(--s-line); background: var(--s-raised); }
|
||||||
|
.audit-grid .boundary { color: #f7f0e6; background: var(--s-green); }
|
||||||
|
.audit-grid .boundary span, .audit-grid .boundary p { color: rgba(247,240,230,.7); }
|
||||||
|
.audit-grid b { display: block; margin-top: 25px; font-size: .75rem; }
|
||||||
|
.audit-grid p { color: var(--s-muted); font-size: .59rem; line-height: 1.55; overflow-wrap: anywhere; }
|
||||||
|
.audit-grid code { font-size: .52rem; }
|
||||||
|
.hash-strip { display: flex; flex-wrap: wrap; gap: 12px 28px; margin-top: 20px; padding: 16px; background: #e9e4da; }
|
||||||
|
.hash-strip span { color: var(--s-muted); font: .5rem/1.5 var(--mono); }
|
||||||
|
.hash-strip code { color: var(--s-green); }
|
||||||
|
@media (max-width: 920px) {
|
||||||
|
.spike-ledger { grid-template-columns: repeat(3, 1fr); }
|
||||||
|
.spike-ledger article:nth-child(3) { border-right: 0; }
|
||||||
|
.spike-tabs { grid-template-columns: repeat(3, 1fr); }
|
||||||
|
.trajectory-story, .evidence-ladder { grid-template-columns: 1fr 24px 1fr; }
|
||||||
|
.trajectory-story > i:nth-of-type(n+3), .evidence-ladder > i:nth-of-type(n+3) { display: none; }
|
||||||
|
.audit-grid { grid-template-columns: repeat(2, 1fr); }
|
||||||
|
}
|
||||||
|
@media (max-width: 680px) {
|
||||||
|
.spike-head, .panel-lead { grid-template-columns: 1fr; gap: 20px; }
|
||||||
|
.spike-head, .spike-panel { padding: 20px; }
|
||||||
|
.spike-ledger { grid-template-columns: repeat(2, 1fr); }
|
||||||
|
.spike-ledger article:nth-child(3) { border-right: 1px solid var(--s-line); }
|
||||||
|
.spike-ledger article:nth-child(even) { border-right: 0; }
|
||||||
|
.spike-tabs { display: flex; overflow-x: auto; }
|
||||||
|
.spike-tabs button { flex: 0 0 190px; }
|
||||||
|
.lab-controls { align-items: stretch; }
|
||||||
|
.lab-controls > div, .lab-controls label { flex: 1 0 100%; }
|
||||||
|
.lab-controls button { flex: 1; }
|
||||||
|
.lab-controls select { width: 100%; }
|
||||||
|
.lab-controls > span { margin: 5px 0 0; line-height: 1.4; }
|
||||||
|
.chart-shell { padding: 12px 8px; }
|
||||||
|
.chart-shell header { padding: 0 8px; }
|
||||||
|
.axis-label { font-size: 9px; }
|
||||||
|
.trajectory-story, .evidence-ladder { grid-template-columns: 1fr; }
|
||||||
|
.trajectory-story > i, .evidence-ladder > i { display: block !important; transform: rotate(90deg); }
|
||||||
|
.plain-rule, .causal-warning { grid-template-columns: 1fr; }
|
||||||
|
.plain-rule p, .causal-warning span { grid-column: 1; }
|
||||||
|
.position-layout, .mixer-layout { grid-template-columns: 1fr; }
|
||||||
|
.position-layout .chart-shell, .mixer-layout .chart-shell { border-right: 0; border-bottom: 1px solid var(--s-line); }
|
||||||
|
.boundary-pair, .backward-diagram .paths, .intervention-results, .audit-grid { grid-template-columns: 1fr; }
|
||||||
|
.backward-diagram article { border-right: 0; border-bottom: 1px solid var(--s-line); }
|
||||||
|
.tensor-path { padding-bottom: 8px; }
|
||||||
|
.chart-shell svg { min-width: 620px; }
|
||||||
|
.chart-shell { overflow-x: auto; }
|
||||||
|
}
|
||||||
|
</style>
|
||||||
@@ -3,6 +3,7 @@ import BaseLayout from "@/layouts/BaseLayout.astro";
|
|||||||
import ArchitectureExplorer from "@/components/ArchitectureExplorer.astro";
|
import ArchitectureExplorer from "@/components/ArchitectureExplorer.astro";
|
||||||
import K3ArtifactLab from "@/components/K3ArtifactLab.astro";
|
import K3ArtifactLab from "@/components/K3ArtifactLab.astro";
|
||||||
import K3AttnResGradientLab from "@/components/K3AttnResGradientLab.astro";
|
import K3AttnResGradientLab from "@/components/K3AttnResGradientLab.astro";
|
||||||
|
import K3AttnResSpikeLab from "@/components/K3AttnResSpikeLab.astro";
|
||||||
import K3AttnResTraceLab from "@/components/K3AttnResTraceLab.astro";
|
import K3AttnResTraceLab from "@/components/K3AttnResTraceLab.astro";
|
||||||
import K3ReportLab from "@/components/K3ReportLab.astro";
|
import K3ReportLab from "@/components/K3ReportLab.astro";
|
||||||
import { k3FigureAtlas, k3Ledgers, k3PaperChain, k3ReportMap } from "@/data/k3";
|
import { k3FigureAtlas, k3Ledgers, k3PaperChain, k3ReportMap } from "@/data/k3";
|
||||||
@@ -40,7 +41,8 @@ const toc = [
|
|||||||
["29", "artifacts", "开放权重工件审计"],
|
["29", "artifacts", "开放权重工件审计"],
|
||||||
["30", "attnres-reduced", "AttnRes 缩小机制实验"],
|
["30", "attnres-reduced", "AttnRes 缩小机制实验"],
|
||||||
["31", "attnres-gradient", "梯度定义与深度扩展"],
|
["31", "attnres-gradient", "梯度定义与深度扩展"],
|
||||||
["32", "audit", "21 张图表审计"],
|
["32", "attnres-spike", "尖峰轨迹与反向路径"],
|
||||||
|
["33", "audit", "21 张图表审计"],
|
||||||
["↳", "papers", "100 节点阅读链"],
|
["↳", "papers", "100 节点阅读链"],
|
||||||
];
|
];
|
||||||
|
|
||||||
@@ -109,13 +111,13 @@ const paperGroups = [
|
|||||||
|
|
||||||
<BaseLayout
|
<BaseLayout
|
||||||
title="Kimi K3 技术报告完整深读:架构、训练、RL、系统与评测"
|
title="Kimi K3 技术报告完整深读:架构、训练、RL、系统与评测"
|
||||||
description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图、两轮十个 AttnRes 独立实验视图与一百个一手阅读节点,逐节读懂 Kimi K3。"
|
description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图、三轮十五个 AttnRes 独立实验视图与一百个一手阅读节点,逐节读懂 Kimi K3。"
|
||||||
section="k3"
|
section="k3"
|
||||||
>
|
>
|
||||||
<header class="page-hero k3-hero">
|
<header class="page-hero k3-hero">
|
||||||
<div class="page-hero-inner">
|
<div class="page-hero-inner">
|
||||||
<div>
|
<div>
|
||||||
<p class="eyebrow"><span>ANCHOR REPORT / ROUND 05</span> KIMI K3 · REPORT → ARTIFACTS → INDEPENDENT PROBE</p>
|
<p class="eyebrow"><span>ANCHOR REPORT / ROUND 06</span> KIMI K3 · REPORT → ARTIFACTS → INDEPENDENT PROBE</p>
|
||||||
<h1>不把报告压成摘要<br />把每个因果环节<br />重新展开</h1>
|
<h1>不把报告压成摘要<br />把每个因果环节<br />重新展开</h1>
|
||||||
<p class="lead">
|
<p class="lead">
|
||||||
K3 同时扩展序列、深度、宽度、视觉与 Agent 轨迹。真正值得读的不是 2.8T 这个最大数字,
|
K3 同时扩展序列、深度、宽度、视觉与 Agent 轨迹。真正值得读的不是 2.8T 这个最大数字,
|
||||||
@@ -125,11 +127,11 @@ const paperGroups = [
|
|||||||
<dl class="page-facts">
|
<dl class="page-facts">
|
||||||
<div><dt>QUESTIONS</dt><dd>32 张问题账</dd></div>
|
<div><dt>QUESTIONS</dt><dd>32 张问题账</dd></div>
|
||||||
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div>
|
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div>
|
||||||
<div><dt>LABS</dt><dd>8 + 4 + 5 + 5 个交互视图</dd></div>
|
<div><dt>LABS</dt><dd>8 + 4 + 5 + 5 + 5 个交互视图</dd></div>
|
||||||
<div><dt>READING</dt><dd>100 个一手 / 官方节点</dd></div>
|
<div><dt>READING</dt><dd>100 个一手 / 官方节点</dd></div>
|
||||||
<div><dt>MODEL</dt><dd>2.78T total / 104.2B active</dd></div>
|
<div><dt>MODEL</dt><dd>2.78T total / 104.2B active</dd></div>
|
||||||
<div><dt>ARTIFACTS</dt><dd>96 shards · 497,220 tensors</dd></div>
|
<div><dt>ARTIFACTS</dt><dd>96 shards · 497,220 tensors</dd></div>
|
||||||
<div><dt>STATUS</dt><dd>K3 五轮 · 梯度定义闭环</dd></div>
|
<div><dt>STATUS</dt><dd>K3 六轮 · 尖峰路径审计</dd></div>
|
||||||
</dl>
|
</dl>
|
||||||
</div>
|
</div>
|
||||||
</header>
|
</header>
|
||||||
@@ -922,8 +924,33 @@ const paperGroups = [
|
|||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
|
<section class="article-section" id="attnres-spike">
|
||||||
|
<p class="eyebrow"><span>32</span> SPIKE TRAJECTORY × BACKWARD PATH</p>
|
||||||
|
<h2>layer 21–25 的尖峰从什么时候出现,又对 mixer 的哪条反向路径敏感?</h2>
|
||||||
|
<p class="lede">
|
||||||
|
第六轮不把上一轮的局部尖峰直接解释成机制。本站预注册六个训练时点、六个张量位置、
|
||||||
|
四种 confirmatory reduction 与三种 same-forward backward rule;正式训练严格复用
|
||||||
|
Round 05 depth-32 Block 的模型、数据、optimizer 与 schedule,诊断全部位于 optimizer 之外。
|
||||||
|
结果显示尖峰在 step 500 后才形成、六个位置均可见;切断 softmax / query / key 源梯度没有降低尖峰,
|
||||||
|
而把全局 learned value-backward coefficients 改成均匀后,contrast 平均下降 70.2%。
|
||||||
|
</p>
|
||||||
|
<div class="artifact-callout">
|
||||||
|
<article><span>F / FROZEN</span><b>3 formal + 1 replay</b><p>196,608,000 formal target bytes;完整重放再加 65,536,000。</p></article>
|
||||||
|
<article><span>X / DEVELOPMENT</span><b>step 500 → 2,000</b><p>三 seed 的尖峰 contrast 从 0.524× 跃到平均 2.233×。</p></article>
|
||||||
|
<article><span>X / ROBUSTNESS</span><b>12 / 12 PASS</b><p>四种预注册 reduction 都保留 layer 21–25 集中。</p></article>
|
||||||
|
<article class="warning"><span>B / BOUNDARY</span><b>global sensitivity</b><p>不是训练变体、因果贡献百分比或局部 mixer 归因。</p></article>
|
||||||
|
</div>
|
||||||
|
<K3AttnResSpikeLab />
|
||||||
|
<div class="hero-actions">
|
||||||
|
<a class="button primary" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_SPIKE_AUDIT.md">阅读完整结果审计</a>
|
||||||
|
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_SPIKE_PROTOCOL.md">核对预注册协议</a>
|
||||||
|
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_SPIKE_SCOPING.md">查看范围冻结</a>
|
||||||
|
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/attnres_spike">复跑实验与分析</a>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
<section class="article-section" id="audit">
|
<section class="article-section" id="audit">
|
||||||
<p class="eyebrow"><span>32</span> FIGURE & TABLE AUDIT</p>
|
<p class="eyebrow"><span>33</span> FIGURE & TABLE AUDIT</p>
|
||||||
<h2>Figure 1–16、Table 1–5:每张图究竟支持什么,不能支持什么</h2>
|
<h2>Figure 1–16、Table 1–5:每张图究竟支持什么,不能支持什么</h2>
|
||||||
<div class="figure-atlas">
|
<div class="figure-atlas">
|
||||||
{k3FigureAtlas.map(([id, report, title, contract]) => (
|
{k3FigureAtlas.map(([id, report, title, contract]) => (
|
||||||
|
|||||||
@@ -9,7 +9,7 @@ const researching = chapters.filter((chapter) => ["researching", "drafting"].inc
|
|||||||
const workstreams = [
|
const workstreams = [
|
||||||
{ label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" },
|
{ label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" },
|
||||||
{ label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" },
|
{ label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" },
|
||||||
{ label: "Kimi K3 深读", value: 98, next: "对齐 layer 21–25 梯度尖峰与 mixer weights;等待 A_log 官方转换合同" },
|
{ label: "Kimi K3 深读", value: 99, next: "对 group 6 / 7 做局部 mixer backward 干预;等待 A_log 官方转换合同" },
|
||||||
{ label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau,并加入真实小语料复现" },
|
{ label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau,并加入真实小语料复现" },
|
||||||
{ label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" },
|
{ label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" },
|
||||||
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
|
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
|
||||||
@@ -50,7 +50,7 @@ const workstreams = [
|
|||||||
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
|
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
|
||||||
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
|
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
|
||||||
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
|
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
|
||||||
<div><dt>UPDATED</dt><dd>2026-07-30 10:05 CST</dd></div>
|
<div><dt>UPDATED</dt><dd>2026-07-30 12:20 CST</dd></div>
|
||||||
<div><dt>MODE</dt><dd>持续迭代,不锁死版本</dd></div>
|
<div><dt>MODE</dt><dd>持续迭代,不锁死版本</dd></div>
|
||||||
</dl>
|
</dl>
|
||||||
</div>
|
</div>
|
||||||
@@ -97,7 +97,7 @@ const workstreams = [
|
|||||||
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
|
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
|
||||||
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全,包含先修依赖和三条贯穿案例。</p></article>
|
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全,包含先修依赖和三条贯穿案例。</p></article>
|
||||||
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
|
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
|
||||||
<article><span>✓</span><h3>九十九个原创交互视图</h3><p>K3 三轴图、八联报告实验、四联开放工件实验与两轮十联 AttnRes 独立实验,DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验,以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
|
<article><span>✓</span><h3>一百零四个原创交互视图</h3><p>K3 三轴图、八联报告实验、四联开放工件实验与三轮十五联 AttnRes 独立实验,DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验,以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
|
||||||
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
|
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
|
||||||
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
|
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
|
||||||
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
|
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
|
||||||
@@ -107,6 +107,7 @@ const workstreams = [
|
|||||||
<article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions,审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes;四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
|
<article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions,审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes;四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
|
||||||
<article><span>✓</span><h3>Kimi K3 四轮 AttnRes 独立实验</h3><p>冻结三结构 × 三 seed 的 9 个 2,000-step 格;Full / Block 相对 Baseline 的平均 paired delta 为 −0.01457 / −0.04247 BPC,但核心参数梯度 CV 没有复现论文叙述。指定正式格全新进程八字段 exact,五视图同时展示结果、反证、成本与 claim boundary。</p></article>
|
<article><span>✓</span><h3>Kimi K3 四轮 AttnRes 独立实验</h3><p>冻结三结构 × 三 seed 的 9 个 2,000-step 格;Full / Block 相对 Baseline 的平均 paired delta 为 −0.01457 / −0.04247 BPC,但核心参数梯度 CV 没有复现论文叙述。指定正式格全新进程八字段 exact,五视图同时展示结果、反证、成本与 claim boundary。</p></article>
|
||||||
<article><span>✓</span><h3>Kimi K3 五轮梯度定义与深度扩展</h3><p>先确认 Figure 5 没有公开唯一 gradient telemetry 合同,再冻结 16/32 blocks × Baseline/Block × 3 seeds 的 12 个 8,000-step 格。Block 的首尾失衡 6/6 改善但全层 CV 6/6 恶化,两个深度都判为 mixed;指定 32 层格完整重训的模型、优化器与全部冻结字段 exact。</p></article>
|
<article><span>✓</span><h3>Kimi K3 五轮梯度定义与深度扩展</h3><p>先确认 Figure 5 没有公开唯一 gradient telemetry 合同,再冻结 16/32 blocks × Baseline/Block × 3 seeds 的 12 个 8,000-step 格。Block 的首尾失衡 6/6 改善但全层 CV 6/6 恶化,两个深度都判为 mixed;指定 32 层格完整重训的模型、优化器与全部冻结字段 exact。</p></article>
|
||||||
|
<article><span>✓</span><h3>Kimi K3 六轮尖峰轨迹与反向路径</h3><p>严格复用 Round 05 depth-32 Block 的三个正式格:尖峰在 step 500 后形成,六个位置 3/3 seed 可见,四种 reduction 12/12 格稳健。切断 key/softmax 源梯度没有降低尖峰;uniform value-backward 让 contrast 平均下降 70.2%,只判为全局 backward-rule sensitivity。完整 replay 的 16 组冻结字段 exact。</p></article>
|
||||||
<article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel;6/6 官方参考逐元素相等,并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
|
<article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel;6/6 官方参考逐元素相等,并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
|
||||||
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
|
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
|
||||||
<article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article>
|
<article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article>
|
||||||
@@ -135,7 +136,7 @@ const workstreams = [
|
|||||||
</div>
|
</div>
|
||||||
<div class="queue-table">
|
<div class="queue-table">
|
||||||
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
|
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
|
||||||
<div><span>P0</span><strong>K3 五轮后续</strong><p>对齐 layer 21–25 尖峰、pre-attention / pre-MLP 与 mixer source weights → 等待 A_log 官方合同后进入真实 checkpoint forward</p><em>局部机制 + 工件边界</em></div>
|
<div><span>P0</span><strong>K3 六轮后续</strong><p>group 6 / 7 局部 mixer intervention matrix → 前向训练变体 → 等待 A_log 官方合同后进入真实 checkpoint forward</p><em>局部机制 + 工件边界</em></div>
|
||||||
<div><span>P0</span><strong>DeepSeek 八轮后续</strong><p>干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
|
<div><span>P0</span><strong>DeepSeek 八轮后续</strong><p>干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
|
||||||
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
|
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
|
||||||
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
|
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
|
||||||
@@ -225,6 +226,10 @@ const workstreams = [
|
|||||||
<div><time>2026-07-30</time><b>首尾平衡与全层 CV 永久分账</b><p>Block 在 6/6 配对中改善 first/last,却因中后段局部尖峰让 CV 在 6/6 配对中恶化;联合判定保持 mixed。</p></div>
|
<div><time>2026-07-30</time><b>首尾平衡与全层 CV 永久分账</b><p>Block 在 6/6 配对中改善 first/last,却因中后段局部尖峰让 CV 在 6/6 配对中恶化;联合判定保持 mixed。</p></div>
|
||||||
<div><time>2026-07-30</time><b>绝对梯度尺度必须与归一化谱同屏</b><p>Block mean gradient 约为 Baseline 的 54%–57%;更接近 1 的首尾比不能偷换成各层信号更强。</p></div>
|
<div><time>2026-07-30</time><b>绝对梯度尺度必须与归一化谱同屏</b><p>Block mean gradient 约为 Baseline 的 54%–57%;更接近 1 的首尾比不能偷换成各层信号更强。</p></div>
|
||||||
<div><time>2026-07-30</time><b>32 层完整重放扩到状态哈希</b><p>8,000-step fresh replay 的全部冻结字段以及 model / optimizer state hashes exact;额外 replay bytes 单列,不混入 formal 预算。</p></div>
|
<div><time>2026-07-30</time><b>32 层完整重放扩到状态哈希</b><p>8,000-step fresh replay 的全部冻结字段以及 model / optimizer state hashes exact;额外 replay bytes 单列,不混入 formal 预算。</p></div>
|
||||||
|
<div><time>2026-07-30</time><b>尖峰是定向复查,不是盲发现</b><p>layer 21–25 来自 Round 05;Round 06 先固定目标集合,再检查训练时点、张量位置、reduction 与反向路径。</p></div>
|
||||||
|
<div><time>2026-07-30</time><b>最早可见不等于物理起源</b><p>pre-attention input 是六个采样点中最早可见位置;更早 mixer 与跨层回传已经作用,不能写成尖峰从这里注入。</p></div>
|
||||||
|
<div><time>2026-07-30</time><b>同一前向只识别反向规则敏感性</b><p>三模式的 logits、loss、activations 与 mixer summaries exact;uniform value-backward 的 70.2% contrast 降幅不是训练变体或因果贡献百分比。</p></div>
|
||||||
|
<div><time>2026-07-30</time><b>相关性、全局干预与局部归因分三层</b><p>MLP latest weight 的局部 r≈.69 只提供候选;全局 value-route 干预支持路径敏感性,下一轮才做 group 6 / 7 局部归因矩阵。</p></div>
|
||||||
<div><time>2026-07-29</time><b>32-token 对照改为同源 16→24</b><p>TNEWS 只有 105/10,000 条达到 32 tokens,强行统一会落入约 1% 极端长尾;24-token eligibility 仍保留 1,609 条中文候选。</p></div>
|
<div><time>2026-07-29</time><b>32-token 对照改为同源 16→24</b><p>TNEWS 只有 105/10,000 条达到 32 tokens,强行统一会落入约 1% 极端长尾;24-token eligibility 仍保留 1,609 条中文候选。</p></div>
|
||||||
<div><time>2026-07-29</time><b>长度敏感性必须成对重采样</b><p>16-token 输入严格是 24-token 输入前缀,2,000 次 bootstrap 共用 prompt indices;结果只描述固定 cohort 的长度敏感性。</p></div>
|
<div><time>2026-07-29</time><b>长度敏感性必须成对重采样</b><p>16-token 输入严格是 24-token 输入前缀,2,000 次 bootstrap 共用 prompt indices;结果只描述固定 cohort 的长度敏感性。</p></div>
|
||||||
<div><time>2026-07-29</time><b>三类 cohort 永久分身份</b><p>自然长度回答本批样本如何路由;matched-16 / 24 回答同一 prompt 多看 8 tokens 后如何变化,不把二者混成内容因果。</p></div>
|
<div><time>2026-07-29</time><b>三类 cohort 永久分身份</b><p>自然长度回答本批样本如何路由;matched-16 / 24 回答同一 prompt 多看 8 tokens 后如何变化,不把二者混成内容因果。</p></div>
|
||||||
|
|||||||
Reference in New Issue
Block a user