Compare commits

...

17 Commits

Author SHA1 Message Date
wuyang f8f0712103 docs: record K3 Round 08 release 2026-07-30 18:46:12 +08:00
wuyang f177fa676d research: publish AttnRes forward training study 2026-07-30 18:41:18 +08:00
wuyang 7ea91caabb experiment: implement AttnRes forward training runner 2026-07-30 15:14:15 +08:00
wuyang d81aacfc68 research: preregister AttnRes forward training study 2026-07-30 15:03:31 +08:00
wuyang badf498598 docs: record K3 round seven release 2026-07-30 14:43:16 +08:00
wuyang dc8ec30f78 site: explain K3 round seven local path audit 2026-07-30 14:35:48 +08:00
wuyang 2dbfd7944a research: audit AttnRes local path matrix 2026-07-30 14:35:32 +08:00
wuyang 39a9ad6215 experiment: implement AttnRes local path runner 2026-07-30 12:53:22 +08:00
wuyang 6911efc6e7 research: preregister AttnRes local path study 2026-07-30 12:46:59 +08:00
wuyang 5e64990a7d docs: record AttnRes spike release 2026-07-30 12:29:51 +08:00
wuyang b39378006a site: surface K3 round six on homepage 2026-07-30 12:26:35 +08:00
wuyang a1a52d4280 site: explain AttnRes spike path study 2026-07-30 12:20:47 +08:00
wuyang 3cbb163715 research: audit AttnRes spike path study 2026-07-30 12:03:37 +08:00
wuyang e9ca86c21e research: lock AttnRes spike analyzer 2026-07-30 11:58:34 +08:00
wuyang 54c80fb115 research: lock AttnRes spike diagnostic runner 2026-07-30 10:54:08 +08:00
wuyang 204f599304 research: preregister AttnRes spike path study 2026-07-30 10:44:50 +08:00
wuyang 97ec50a642 docs: record AttnRes gradient release 2026-07-30 10:30:38 +08:00
93 changed files with 1802769 additions and 45 deletions
+44 -4
View File
@@ -8,7 +8,7 @@
|---|---:|---:|---| |---|---:|---:|---|
| 研究框架与规范 | 进行中 | 83% | Scaling Laws 二轮拟合复现与逐图精读 | | 研究框架与规范 | 进行中 | 83% | Scaling Laws 二轮拟合复现与逐图精读 |
| 网站设计系统 | 进行中 | 89% | 打印样式与更多通用可视化组件 | | 网站设计系统 | 进行中 | 89% | 打印样式与更多通用可视化组件 |
| Kimi K3 深读 | 四轮实证进行中 | 96% | 对齐论文梯度定义、扩展深度/预算,并等待 `A_log` 官方转换合同 | | Kimi K3 深读 | 八轮实证已收敛 | 100% | 稳定维护;真实 forward 等待 `A_log` 官方裁决 |
| 语言模型前史 | 完成首版 | 78% | Kneser–Ney、LSTM、Bahdanau 逐图精读与真实小语料复现 | | 语言模型前史 | 完成首版 | 78% | Kneser–Ney、LSTM、Bahdanau 逐图精读与真实小语料复现 |
| Transformer 基础 | 完成首版 | 79% | 多头电路、归一化 traces 与真实 kernel / KV 配置 | | Transformer 基础 | 完成首版 | 79% | 多头电路、归一化 traces 与真实 kernel / KV 配置 |
| 表示、位置与残差高速公路 | 完成首版 | 81% | 真实 hidden-state / norm traces、长上下文位置外推与深层稳定性消融 | | 表示、位置与残差高速公路 | 完成首版 | 81% | 真实 hidden-state / norm traces、长上下文位置外推与深层稳定性消融 |
@@ -41,7 +41,7 @@
- [x] 完成 486 篇关键论文索引,覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。 - [x] 完成 486 篇关键论文索引,覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。
- [x] 完成可检索、可按专题筛选的论文库页面。 - [x] 完成可检索、可按专题筛选的论文库页面。
- [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。 - [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。
- [x] 完成 K3 三轴架构、八联报告实验、四联开放工件实验、Round 04 / 05 各五联 AttnRes 独立实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 二十二联实验、长上下文、MoE 路由、推理三页签,以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等九十九个原创交互视图。 - [x] 完成 K3 三轴架构、八联报告实验、四联开放工件实验、Round 04 / 05 / 06 / 07 / 08 各五联 AttnRes 独立实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 二十二联实验、长上下文、MoE 路由、推理三页签,以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等一百一十四个原创交互视图。
- [x] 完成长上下文首版:五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。 - [x] 完成长上下文首版:五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
- [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文,并建立长上下文研究账本。 - [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文,并建立长上下文研究账本。
- [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文,并建立 MoE 研究账本。 - [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文,并建立 MoE 研究账本。
@@ -287,10 +287,34 @@
- [x] 指定 depth-32 / Block / seed-2026073001 从初始化完整重训 8,000 steps;排除 run-kind / timing 后冻结字段 compare SHA-256 同为 `46300a45…4817`,model / optimizer state hashes exact。正式/compact/reproduction 物理 SHA-256 为 `ad461cbe…a8d / 5377a5e7…68e3 / aedcde6a…dea6`。 - [x] 指定 depth-32 / Block / seed-2026073001 从初始化完整重训 8,000 steps;排除 run-kind / timing 后冻结字段 compare SHA-256 同为 `46300a45…4817`,model / optimizer state hashes exact。正式/compact/reproduction 物理 SHA-256 为 `ad461cbe…a8d / 5377a5e7…68e3 / aedcde6a…dea6`。
- [x] K3 Round 05 五视图实验室完成:论文定义已知/未定义、绝对/归一化深度谱、六 checkpoint 时间轨迹、Output RMS 组节律、activation/parameter/BPC/成本/重放联合账全部可切换;21 个 raw JSON、完整 aggregate、compact、runner、analyzer、协议与审计进入公开树。 - [x] K3 Round 05 五视图实验室完成:论文定义已知/未定义、绝对/归一化深度谱、六 checkpoint 时间轨迹、Output RMS 组节律、activation/parameter/BPC/成本/重放联合账全部可切换;21 个 raw JSON、完整 aggregate、compact、runner、analyzer、协议与审计进入公开树。
- [x] Round 05 本地闸门通过:94 个 Astro 文件零诊断/提示,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;冻结数据、新专项、Round 04 与 K3 全量真实 Chrome 回归通过,桌面/390px 移动端零文档级溢出、零运行时异常。 - [x] Round 05 本地闸门通过:94 个 Astro 文件零诊断/提示,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;冻结数据、新专项、Round 04 与 K3 全量真实 Chrome 回归通过,桌面/390px 移动端零文档级溢出、零运行时异常。
- [x] K3 Round 05 以功能提交 `f7670ef`、不可变镜像 `20260730T022610Z-f7670ef` 发布;OCI digest `sha256:02f902c1a64f976c6ed5d8614a826fca487b483f9addf2392637ea3f021cc09b`,NAS healthy / 零重启、21/21 公网路由、HTTPS/2、gzip/immutable 与三套 K3 生产 Chrome 回归全部通过;保留 `20260729T233142Z-4ce780d` 回滚。
- [x] K3 Round 06 在正式训练前冻结 `llm-atlas-k3-attnres-spike-path-v1`:固定 Round 05 depth-32 Block、三个 seed、8,000 steps、六个 checkpoint、layer 21–25 目标集合、六个张量位置、四种 confirmatory reduction、三条 same-forward backward rule、20% 材料级阈值与完整 seed-1 replay;研究身份明确为受 Round 05 启发的 targeted follow-up,不冒充 blind discovery。
- [x] 三个正式格共消费 196,608,000 target bytes;最终模型、优化器、六个 BPC、全部训练历史与 Round 05 post-MLP 数组逐 seed exact。完整 replay 再消费 65,536,000 bytes,16 组冻结字段全部 exact,compare SHA-256 为 `05396eaf…cbb87`。
- [x] 尖峰发展轨迹显示 layer 21–25 不是初始化时固有:三 seed mean contrast 在 step 0 / 100 / 500 为 `0.757 / 0.563 / 0.524×`,到 step 2,000 / 4,000 / 8,000 升为 `2.233 / 2.594 / 2.754×`;seed 3 的最终 `1.881×` 同向但更弱。
- [x] 六个张量位置的最终 mean contrast 均超过 1.5,且每个位置 3 / 3 seed 可见;最早采样点 `pre_attention_input` 已可见,但观测顺序不升级为物理起源或注入位置。
- [x] 预注册 reduction 家族 12 / 12 格同时满足 contrast ≥ 1.5、top-5 与目标集合至少重叠 3 层、相对 element RMS 的 Spearman ≥ .8;正式结论严格写作 `robust within the preregistered reduction family`。
- [x] 全局切断 64 个 depth mixer 与 output mixer 的 softmax/query/key source-gradient paths 后,contrast / peak 平均相对“降幅”为 `−2.00% / −0.84%`,三 seed 均略微上升,不支持材料级降低。将 learned value-backward coefficients 换成 `1/N` 后,contrast / peak 平均下降 `70.20% / 37.03%`,三 seed 都超过 20% 阈值,peak layer 全部移到 layer 2。
- [x] mixer 观察关联与反向干预分层:layers 19–28 的 MLP latest weight 对 normalized gradient Pearson / Spearman 为 `.690 / .693`,MLP entropy 为 `−.636 / −.640`;相关性只提供候选机制,全局干预只支持 value-route sensitivity,不冒充局部 mixer attribution。
- [x] K3 Round 06 五视图实验室完成:训练轨迹、六位置谱、12 格 reduction 稳健性、same-forward 三规则干预与 mixer 散点/证据阶梯可交互;protocol、scoping、audit、runner、analyzer、raw/aggregate/compact/reproduction 全部进入公开树。
- [x] Round 06 本地闸门通过:97 个 Astro 文件零诊断,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;Round 04/05/06 三套冻结数据、三套专项、K3 全量与全站 22 套真实 Chrome 回归通过,桌面/390px 移动端零文档级溢出、零 offender、零运行时异常;首页首发卡与陈旧跨页断言同步到 Round 06。
- [x] K3 Round 06 以运行源提交 `b393780`、不可变镜像 `20260730T042651Z-b393780` 发布;OCI index digest `sha256:4e4cb2e065526f50c10cf759bab80a5a871177812ca9fa5c9da77c662c81a63a`,NAS healthy / 0 次重启、Compose Manager 标签、VPS→NAS、NPM host 31 / cert 41、DNS、HTTPS/2、首页/K3 公网内容、门户 `LLM ATLAS / projects / 180` 与全站 22 套生产 Chrome 回归全部通过;保留 Round 05 `20260730T022610Z-f7670ef` 与中间 `20260730T042110Z-a1a52d4` 回滚点。
- [x] K3 Round 07 在正式输出前冻结 `llm-atlas-k3-attnres-local-path-v1`:固定 Round 06 三个 depth-32 Block 正式格、layer 21–25 spike set、groups 6+7、14 个 exact selector masks、sufficiency / restoration 双向 log-gap score、主门 50%、单组/分支/输出控制与完整 seed-1 replay;Grok 只读对抗审阅提出的六个 blocking protocol 问题全部在冻结前修正。
- [x] 三个正式格与 replay 共消费 262,144,000 target bytes;Round 06 parent equivalence、forward identity、65-node census、selector identity / order / uniqueness、负对照与 loss-scale 闸门全部通过。正式最终 BPC 为 `1.7123525941 / 1.7093240656 / 1.7030966813`。
- [x] global gap 在两个指标 × 三 seed 的 6 / 6 格通过。groups 6+7 sufficiency 的 mean score 为 contrast `.677`、peak `1.700`,6 / 6 ≥ `.50`;restoration mean 为 contrast `.650`、peak `.380`,contrast 3 / 3 通过而 peak 0 / 3 通过,正式状态固定为 `one_sided_evidence_localization_not_established`。
- [x] 次级控制显示 group 7 MLP-only branch 在 sufficiency 方向 6 / 6 通过 material + margin 门;group 6 MLP-only 为 5 / 6,不能宣布 dominance。output-only mean sufficiency 仅 `.131 / .187`、0 / 6 过 50%;all-depth 为 `.949 / 1.145`。所有 score 都是非加性 log-gap 诊断,不写成贡献率。
- [x] K3 Round 07 五视图实验室完成:65-node 路径图、14-mask 全矩阵、双向主门、branch/output controls 与 32 层原始谱/replay 审计可交互;protocol、scoping、Grok 结果前审阅、audit、runner、analyzer、packager、四个 raw JSON、aggregate、compact 与 reproduction 全部进入公开树。
- [x] Round 07 本地闸门通过:100 个 Astro 文件零诊断,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;Round 04/05/06/07 四套冻结数据、四套 K3 专项、K3 全量与全站 23 套真实 Chrome 回归通过。动态生成矩阵的 scoped CSS 退化由截图复查发现并修复;桌面/390px 移动端零文档级溢出、零 offender、零运行时异常。
- [x] K3 Round 07 以功能源提交 `dc8ec30`、不可变镜像 `20260730T063811Z-dc8ec30` 发布;OCI index digest `sha256:98d441412408774628195ce23f431d25034bb1590210c11230d884513898e71d`。NAS `12010→8080` healthy / 0 次重启、Compose Manager 标签、VPS→NAS、NPM host 31 / cert 41、DNS、HTTPS/2、gzip、immutable asset、首页/K3 公网内容、门户 `LLM ATLAS / projects / 180` 与全站 23 套生产 Chrome 回归全部通过;保留 Round 06 `20260730T042651Z-b393780` 回滚点。
- [x] K3 Round 08 在正式输出前冻结 `llm-atlas-k3-attnres-forward-training-v1`:四个 train-time forward variants × 三 seed × 8,000 steps、同 seed Round 05 historical pairing、step 8,000 的 layers 21–25 contrast / peak 双指标 20% 主门、BPC 每 seed / mean 护栏、描述性 `I67` 与完整 primary replay;不把架构消融写成 pure-forward 因果实验。
- [x] 12 formal + 1 replay 全部一次完成,每格 65,536,000 target bytes;新处理总量 851,968,000,历史 references 196,608,000 单列。联合 groups 6+7 的 contrast / peak 六格降幅为 62.1%–77.3% / 32.3%–62.0%,6 / 6 通过;BPC 三 seed 最大 `+.009598`、均值 `+.006578`,质量门 4 / 4 通过。
- [x] primary seed-1 replay scientific payload exact,SHA-256 为 `b85563ca…c051`;冻结主状态为 `forward_training_attenuation_established_within_reduced_protocol`。`I67` 的 step-8,000 mean 为 contrast `−.3670`、peak `−.1704`,只保留为跨独立训练的描述性 log residual。
- [x] 结果前 Grok 实现审阅指出 smoke-only empty selector 的空 census;矩阵结束后删除 early return、加入 `forward_calls > 0`,修补后的 learned wrapper 实际执行 39 次 forward,父/包装器 15 组科学字段仍 exact。结果后 Grok 只读复算报告 `blocking_errors=0`、status / replay 均确认。
- [x] K3 Round 08 五视图实验室完成:forward contract、六 checkpoint 训练轨迹、attenuation+BPC 主门、non-additivity map 与 32 层谱/replay audit 可交互;protocol、scoping、两阶段 Grok 审阅、runner、analyzer、13 raw、aggregate、compact、reproduction 与结果审计全部进入公开树。
- [x] Round 08 本地闸门通过:101 个 Astro 文件零诊断,21 个页面、1,151 个站内引用、12 个跨页锚点零失败;Round 04–08 五套冻结数据、五套 K3 专项、K3 全量与全站 24 套真实 Chrome 回归通过。截图复查修复黑色实验室标题对比度;桌面/390px 移动端零文档级溢出、零 offender、零运行时异常。
- [x] K3 Round 08 以功能源提交 `f177fa6`、不可变镜像 `20260730T104228Z-f177fa6` 发布;OCI index digest `sha256:0b5b9d9cbf1b4538a37b01727ba1d16be13f991408d3d60e20bc76023b9008bb`。NAS `12010→8080` healthy / 0 次重启、Compose Manager、VPS→NAS、NPM host 31 / cert 41、DNS、HTTPS/2、gzip、首页/K3/进度页、门户 `LLM ATLAS / projects / 180` 与全站 24 套生产 Chrome 回归全部通过;保留 Round 07 `20260730T064331Z-badf498` 回滚点。
## 正在进行 ## 正在进行
- [ ] K3 五轮下一闸门:对齐 layer 21–25 的 activation-gradient 尖峰、pre-attention / pre-MLP 位置与 mixer source weights,并做公开 reduction sensitivity;真实 K3 forward 继续等待 `A_log [128]→[96]` 官方转换或权重修订。
- [ ] DeepSeek 八轮下一闸门:推进干预式 mediation、SM90 FlashMLA、FP8 / pipeline traces 与 R1-like RL 小模型复现。 - [ ] DeepSeek 八轮下一闸门:推进干预式 mediation、SM90 FlashMLA、FP8 / pipeline traces 与 R1-like RL 小模型复现。
- [ ] 表示、位置与残差二轮:真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。 - [ ] 表示、位置与残差二轮:真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。
- [ ] 评测安全二轮:真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。 - [ ] 评测安全二轮:真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。
@@ -479,7 +503,7 @@
| 2026-07-30 | 输出长度揭示强任务域交互 | system-at-period 在 Code 为负、Math 为正,两个选定任务带都不跨零;不从长度外推能力 | | 2026-07-30 | 输出长度揭示强任务域交互 | system-at-period 在 Code 为负、Math 为正,两个选定任务带都不跨零;不从长度外推能力 |
| 2026-07-30 | Round 08 十二字段重放过闸 | 64/64 exact;uniform hash、完整 token IDs、文本、stop 与 RNG 一并进入复现合同,评分 gold 提前加载的流程偏差公开保留 | | 2026-07-30 | Round 08 十二字段重放过闸 | 64/64 exact;uniform hash、完整 token IDs、文本、stop 与 RNG 一并进入复现合同,评分 gold 提前加载的流程偏差公开保留 |
| 2026-07-30 | DeepSeek Round 08 任务 bootstrap / CRN 里程碑发布 | 源提交 `975ed3d`、镜像 `20260729T221654Z-975ed3d`、OCI `sha256:759a8446…21452b`;21/21 公网路由与生产专项/全量 Chrome 通过,保留 Round 07 回滚点 | | 2026-07-30 | DeepSeek Round 08 任务 bootstrap / CRN 里程碑发布 | 源提交 `975ed3d`、镜像 `20260729T221654Z-975ed3d`、OCI `sha256:759a8446…21452b`;21/21 公网路由与生产专项/全量 Chrome 通过,保留 Round 07 回滚点 |
| 2026-07-30 | K3 `A_log [128]` 冲突继续阻断真实 checkpoint forward | 当前 HF / GitHub / FlashKDA / vLLM / SGLang 均未公开 128→96 转换;不裁剪、不 reshape、不把假设输出冒充 K3 | | 2026-07-30 | K3 `A_log [128]` 冲突仍没有官方裁决 | 官方 main 仍期望 96;社区 #144 改为 128,#150 验证 tail zero 后裁为 96,两案都未合并。不把候选 patch 冒充官方 K3 forward |
| 2026-07-30 | AttnRes 缩小实验先冻结再训练 | 三结构共享公共主干、初始化、窗口与优化器;三 seed paired BPC 只按预注册 `3/3 same direction + mean≤−.010` 判为本协议内方向支持 | | 2026-07-30 | AttnRes 缩小实验先冻结再训练 | 三结构共享公共主干、初始化、窗口与优化器;三 seed paired BPC 只按预注册 `3/3 same direction + mean≤−.010` 判为本协议内方向支持 |
| 2026-07-30 | 主结果与机制反结果同时发布 | Full / Block BPC 方向支持;核心参数 gradient RMS CV 却高于 Baseline,明确写成未复现论文梯度叙述 | | 2026-07-30 | 主结果与机制反结果同时发布 | Full / Block BPC 方向支持;核心参数 gradient RMS CV 却高于 Baseline,明确写成未复现论文梯度叙述 |
| 2026-07-30 | 独立重放按数值合同而非计时合同验收 | Block / seed-1 的八组冻结字段 2,000 steps exact;wall time 受调度影响,不要求或声称 bit-exact | | 2026-07-30 | 独立重放按数值合同而非计时合同验收 | Block / seed-1 的八组冻结字段 2,000 steps exact;wall time 受调度影响,不要求或声称 bit-exact |
@@ -488,6 +512,22 @@
| 2026-07-30 | “更均匀”拆成首尾失衡与全层 CV | Block 6/6 改善 first/last,却 6/6 恶化 CV;局部尖峰与系统性早层隆起必须分开解释 | | 2026-07-30 | “更均匀”拆成首尾失衡与全层 CV | Block 6/6 改善 first/last,却 6/6 恶化 CV;局部尖峰与系统性早层隆起必须分开解释 |
| 2026-07-30 | 绝对尺度与归一化形状永久同报 | Block activation-gradient mean 约为 Baseline 54%–57%;不能把更接近 1 的首尾比自动解释为各层信号更强 | | 2026-07-30 | 绝对尺度与归一化形状永久同报 | Block activation-gradient mean 约为 Baseline 54%–57%;不能把更接近 1 的首尾比自动解释为各层信号更强 |
| 2026-07-30 | Round 05 完整重放过闸 | depth-32 Block seed-1 从零重训 8,000 steps;全部冻结字段与 model/optimizer state hashes exact,timing 仍单独报告 | | 2026-07-30 | Round 05 完整重放过闸 | depth-32 Block seed-1 从零重训 8,000 steps;全部冻结字段与 model/optimizer state hashes exact,timing 仍单独报告 |
| 2026-07-30 | K3 Round 05 梯度尺度里程碑发布 | 功能源 `f7670ef`、镜像 `20260730T022610Z-f7670ef`、OCI `sha256:02f902c1…cc09b`;复用 NAS `12010→8080`、NPM 31 / cert 41、门户 order 180,保留 Round 04 回滚点 |
| 2026-07-30 | 尖峰集合是 targeted follow-up,不是盲发现 | layer 21–25 来自 Round 05;Round 06 固定集合后才检查时间、位置、reduction 与路径,避免用同一结果发现并确认 |
| 2026-07-30 | “最早可见”与“物理起源”永久分账 | pre-attention input 是六个采样点中最早可见张量;更早 mixer 与反向传播已经作用,不能写成尖峰从这里注入 |
| 2026-07-30 | 同一前向只识别 backward-rule sensitivity | learned / detached / uniform 三模式的 logits、loss、activations 与 mixer summaries exact;结果不升级为重训模型或性能结论 |
| 2026-07-30 | value-route 降幅不写成因果贡献百分比 | 全局 uniform value-backward 让 contrast 平均下降 70.2%,只支持预注册阈值下的材料级敏感性;不声称 value 路径“解释了 70.2%” |
| 2026-07-30 | reduction 稳健性限定在预注册家族 | element RMS、token RMS mean/median/P95 的 12/12 格通过;探索性 reduction 和其他 batch 不被纳入确认性外推 |
| 2026-07-30 | K3 Round 06 尖峰路径里程碑发布 | 运行源 `b393780`、镜像 `20260730T042651Z-b393780`、OCI `sha256:4e4cb2e0…1a63a`;21/21 公网页面链路与全站 22 套生产 Chrome 通过,保留 Round 05 与中间 Round 06 回滚点 |
| 2026-07-30 | 局部路径必须同时通过 sufficiency 与 restoration | groups 6+7 的 sufficiency 6/6 过 `.50`,restoration 仅 contrast 3/3 通过、peak 0/3 通过;强单侧证据不升级为 localization |
| 2026-07-30 | 局部 score 不写成可加贡献率 | 同一 scope 在 learned 与 uniform 背景的响应不同,`S_peak > 1` 与负 interaction residual 都是非加性诊断,不是 170% 贡献或方差分解 |
| 2026-07-30 | branch 与 output 控制保持次级证据身份 | group 7 MLP-only 的 6/6 只属于 sufficiency branch gate;group 6 为 5/6,output-only 为 0/6,都不能补救失败的双向主门 |
| 2026-07-30 | K3 Round 07 局部路径里程碑发布 | 功能源 `dc8ec30`、镜像 `20260730T063811Z-dc8ec30`、OCI `sha256:98d44141…e71d`;21/21 公网页面链路与全站 23 套生产 Chrome 通过,保留 Round 06 回滚点 |
| 2026-07-30 | 训练期前向干预属于架构消融 | selected uniform mixer 同时改变 train/eval forward、natural backward 与后续 updates;不能写成只改 forward 的路径因果 |
| 2026-07-30 | Round 08 主门与质量门同时成立 | groups 6+7 六个 attenuation 格 6/6 ≥20%;三 seed ΔBPC mean `+.006578`,4/4 过闸;结论只限固定缩小协议 |
| 2026-07-30 | non-additivity 永久保留描述身份 | `I67` 来自三套独立训练,只是 cross-run log residual,不是因果 interaction、Shapley 或贡献率 |
| 2026-07-30 | K3 研究线在 Round 08 主动收敛 | 不启动 Round 09;公开 `A_log [128]↔[96]` 冲突继续等待官方裁决,现有五轮实证停在可复现、可回滚的稳定边界 |
| 2026-07-30 | K3 Round 08 训练期前向里程碑发布 | 功能源 `f177fa6`、镜像 `20260730T104228Z-f177fa6`、OCI `sha256:0b5b9d9c…008bb`;21/21 公网页面链路与全站 24 套生产 Chrome 通过,保留 Round 07 回滚点 |
## 未决问题 ## 未决问题
+27 -1
View File
@@ -19,7 +19,7 @@
当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读, 当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读,
语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化,以及评测与安全深度专题, 语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化,以及评测与安全深度专题,
以及 99 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、 以及 109 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
8 个交互实验和 100 个一手/官方节点,完整覆盖架构、预训练、后训练、系统、评测、案例与附录。 8 个交互实验和 100 个一手/官方节点,完整覆盖架构、预训练、后训练、系统、评测、案例与附录。
第三轮已完成开放工件与首个真实 kernel 里程碑:固定官方模型与 FlashKDA revisions,审计 96 个 checkpoint shards、 第三轮已完成开放工件与首个真实 kernel 里程碑:固定官方模型与 FlashKDA revisions,审计 96 个 checkpoint shards、
497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计,并用 4 个新视图 497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计,并用 4 个新视图
@@ -52,6 +52,32 @@ peak allocated memory 约 2.2×,不冒充同算力优势。指定 depth-32 / B
[K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md](./research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md)、 [K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md](./research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md)、
[K3_ATTNRES_GRADIENT_SCALE_AUDIT.md](./research/K3_ATTNRES_GRADIENT_SCALE_AUDIT.md) 与 [K3_ATTNRES_GRADIENT_SCALE_AUDIT.md](./research/K3_ATTNRES_GRADIENT_SCALE_AUDIT.md) 与
[gradient experiment](./experiments/k3/attnres_gradient/)。 [gradient experiment](./experiments/k3/attnres_gradient/)。
第六轮冻结 `llm-atlas-k3-attnres-spike-path-v1`,严格复用 Round 05 的 depth-32 Block
训练格,在 optimizer 之外追踪六个 checkpoint、六个 block 内张量位置与四种预注册
reduction,并在同一前向值下比较 learned、detached-learned 与 uniform-value-backward
三条反向规则。layer 21–25 的 post-MLP contrast 在 step 500 仍仅为三 seed 平均 `0.524×`,
到 step 2,000 升至 `2.233×`,说明尖峰在训练中形成;六个位置全部在 3/3 seed 可见,
四种 reduction 的 12/12 格同时通过 contrast、top-5 overlap 与 Spearman 闸门。
切断全局 softmax/query/key 源梯度后 contrast 反而平均增加约 2.0%;把 learned
value-backward coefficients 换成均匀系数后,contrast 平均下降 `70.2%`、peak/mean
下降 `37.0%`,三 seed 都达到预注册材料级敏感性阈值。该结果只支持全局反向路径敏感性,
不冒充训练变体、因果贡献百分比或局部 mixer 归因。正式三格与完整 replay 共
262,144,000 target bytes,Round 05 等价字段与 16 组 replay 字段全部 exact。详见
[K3_ATTNRES_SPIKE_PROTOCOL.md](./research/K3_ATTNRES_SPIKE_PROTOCOL.md)、
[K3_ATTNRES_SPIKE_AUDIT.md](./research/K3_ATTNRES_SPIKE_AUDIT.md) 与
[spike-path experiment](./experiments/k3/attnres_spike/)。
第七轮冻结 `llm-atlas-k3-attnres-local-path-v1`,把 Round 06 的全局 value-route
sensitivity 收缩为 14 个 exact selector masks,并同时从 learned 背景测 sufficiency、
从 all-uniform 背景测 restoration。groups 6+7 的 16 个 depth mixers 在充分性方向
两个指标 × 三 seed 的 `6 / 6` 格全部超过 `.50` global log gap;恢复性却只有 contrast
`3 / 3` 通过,peak `0 / 3` 通过,mean peak restoration 仅 `.380`。因此冻结结论是
`one_sided_evidence_localization_not_established`,不是“定位成功”。group 7 的
MLP-only 次级门 6 / 6 通过,但只属于 sufficiency 探索;output-only 为 0 / 6。
三个正式格与完整 replay 共处理 262,144,000 target bytes,Round 06 equivalence、
14 / 14 forward identity、65-node census、selector 与 canonical replay 全部 exact。
详见 [K3_ATTNRES_LOCAL_PATH_PROTOCOL.md](./research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md)、
[K3_ATTNRES_LOCAL_PATH_AUDIT.md](./research/K3_ATTNRES_LOCAL_PATH_AUDIT.md) 与
[local-path experiment](./experiments/k3/attnres_local_path/)。
DeepSeek 八轮专题以 24 张问题账、10 次技术转向、 DeepSeek 八轮专题以 24 张问题账、10 次技术转向、
22 个交互实验和 60 个一手/官方节点,串起 Dense、MoE、MLA、V3 协同、R1 与 V4; 22 个交互实验和 60 个一手/官方节点,串起 Dense、MoE、MLA、V3 协同、R1 与 V4;
并固定官方 V2-Lite revision,在 RTX 5090 上连续执行 7/27 层,记录 3,240 次真实专家选择、 并固定官方 V2-Lite revision,在 RTX 5090 上连续执行 7/27 层,记录 3,240 次真实专家选择、
+70
View File
@@ -0,0 +1,70 @@
# Attention Residuals train-time forward intervention
This directory implements preregistered protocol
`llm-atlas-k3-attnres-forward-training-v1`:
- `research/K3_ATTNRES_FORWARD_TRAINING_SCOPING.md`
- `research/K3_ATTNRES_FORWARD_TRAINING_PROTOCOL.md`
- `research/K3_ATTNRES_FORWARD_TRAINING_GROK_REVIEW.md`
- `research/K3_ATTNRES_FORWARD_TRAINING_IMPLEMENTATION_REVIEW.md`
- `research/K3_ATTNRES_FORWARD_TRAINING_AUDIT.md`
It is a depth-32 reduced Block AttnRes architecture ablation. It is not a
Kimi-K3 checkpoint forward pass and does not claim to recover unpublished
Figure 5 telemetry.
## Frozen environment
```text
Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
PyTorch 2.11.0+cu128
GPU NVIDIA GeForce RTX 5090
CUBLAS_WORKSPACE_CONFIG=:4096:8
maximum concurrency 2
```
## Pre-result gates
The checked-in gate artifacts must pass before formal output:
```bash
CUBLAS_WORKSPACE_CONFIG=:4096:8 \
/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
experiments/k3/attnres_forward/verify.py step-zero \
--cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
--parent-manifest experiments/k3/attnres_gradient/manifest.json \
--study-manifest experiments/k3/attnres_forward/manifest.json \
--output experiments/k3/attnres_forward/results/gates/step-zero.json
```
`learned_reference` is smoke-only. Its 20-step result is compared with a
fresh parent Round 05 smoke using `verify.py smoke-compare`.
## Formal matrix
```bash
/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
experiments/k3/attnres_forward/run_matrix.py \
--python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
--cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
--parent-manifest experiments/k3/attnres_gradient/manifest.json \
--study-manifest experiments/k3/attnres_forward/manifest.json \
--output-dir experiments/k3/attnres_forward/results/raw \
--phase all \
--concurrency 2
```
This runs 12 formal cells and one full replay. The analyzer reads all cells,
the frozen historical paired references, and generates the only authoritative
status, interaction map, and website compact artifact.
The checked-in Round 08 release contains:
- 13 raw results under `results/raw/`;
- `reproduction.json` with the exact primary scientific-payload hash;
- aggregate / compact website data under `src/data/`;
- a frozen-data checker and real-Chrome five-view regression in `scripts/`.
The established status is deliberately scoped to this reduced protocol. It is
not a real Kimi-K3 checkpoint result or a reproduction of unpublished Figure
5(c) telemetry.
+699
View File
@@ -0,0 +1,699 @@
#!/usr/bin/env python3
"""Aggregate and gate preregistered Round 08 forward-training results."""
from __future__ import annotations
import argparse
import copy
import hashlib
import json
import math
import statistics
from pathlib import Path
from typing import Any, Iterable
PROTOCOL_ID = "llm-atlas-k3-attnres-forward-training-v1"
PARENT_PROTOCOL_ID = "llm-atlas-k3-attnres-gradient-scale-v1"
METRICS = ("spike_contrast", "peak_normalized")
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument("--manifest", type=Path, required=True)
parser.add_argument("--formal", type=Path, action="append", required=True)
parser.add_argument("--replay", type=Path, required=True)
parser.add_argument("--reference-dir", type=Path, required=True)
parser.add_argument("--aggregate-output", type=Path, required=True)
parser.add_argument("--compact-output", type=Path, required=True)
parser.add_argument("--reproduction-output", type=Path, required=True)
return parser.parse_args()
def canonical_sha256(value: Any) -> str:
return hashlib.sha256(
json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode()
).hexdigest()
def file_sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def mean(values: Iterable[float]) -> float:
return statistics.fmean(values)
def read_result(path: Path, expected_protocol: str) -> dict[str, Any]:
value = json.loads(path.read_text())
if value.get("protocol_id") != expected_protocol:
raise RuntimeError(f"protocol mismatch: {path}")
expected = value.get("canonical_sha256_without_self")
payload = {
key: item
for key, item in value.items()
if key != "canonical_sha256_without_self"
}
if not isinstance(expected, str) or canonical_sha256(payload) != expected:
raise RuntimeError(f"canonical self-hash mismatch: {path}")
return value
def exactly_one(values: list[dict[str, Any]], step: int) -> dict[str, Any]:
matches = [value for value in values if value["step"] == step]
if len(matches) != 1:
raise RuntimeError(f"step {step} missing or duplicated")
return matches[0]
def spectrum_metrics(
diagnostic: dict[str, Any],
spike_layers: tuple[int, ...],
epsilon: float,
) -> dict[str, Any]:
values = [
float(value)
for value in diagnostic["activation_grad_rms_by_block"]
]
if len(values) != 32:
raise RuntimeError("activation-gradient spectrum must have 32 layers")
if any(not math.isfinite(value) or value <= epsilon for value in values):
raise RuntimeError("activation-gradient spectrum is non-finite/non-positive")
spike_indices = {layer - 1 for layer in spike_layers}
spike_values = [
value for index, value in enumerate(values) if index in spike_indices
]
reference_values = [
value for index, value in enumerate(values) if index not in spike_indices
]
spike_mean = mean(spike_values)
reference_mean = mean(reference_values)
global_mean = mean(values)
contrast = spike_mean / reference_mean
peak = max(values) / global_mean
if any(
not math.isfinite(value) or value <= epsilon
for value in (spike_mean, reference_mean, contrast, peak)
):
raise RuntimeError("derived spike metric is non-finite/non-positive")
ordered = sorted(range(32), key=lambda index: (-values[index], index))
return {
"values": values,
"normalized": [value / global_mean for value in values],
"spike_mean": spike_mean,
"reference_mean": reference_mean,
"global_mean": global_mean,
"spike_contrast": contrast,
"peak_normalized": peak,
"peak_layer_1based": ordered[0] + 1,
"top_five_layers_1based": [index + 1 for index in ordered[:5]],
}
def final_bpc(value: dict[str, Any], step: int) -> float:
result = float(exactly_one(value["evaluations"], step)["bits_per_byte"])
if not math.isfinite(result):
raise RuntimeError("final validation BPC is non-finite")
return result
def stable_environment(value: dict[str, Any]) -> dict[str, Any]:
keys = (
"cublas_workspace_config",
"deterministic_algorithms",
"autocast",
"compile",
)
return {key: value["environment"][key] for key in keys}
def pairing_checks(
run: dict[str, Any], reference: dict[str, Any]
) -> dict[str, bool]:
manifest_fields = (
"formal_schedule_sha256",
"validation_tensor_sha256",
"diagnostic_tensor_sha256",
"input_gate_tensor_hashes",
)
checks = {
"seed": run["seed"] == reference["seed"],
"architecture": (
run["architecture"] == reference["architecture"] == "block"
),
"depth": run["depth"] == reference["depth"] == 32,
"steps": run["steps"] == reference["steps"] == 8000,
"batch_size": run["batch_size"] == reference["batch_size"] == 32,
"initial_public_parameters": (
run["hashes"]["initial_public_parameters"]
== reference["hashes"]["initial_public_parameters"]
),
"initial_mixer_parameters": (
run["hashes"]["initial_mixer_parameters"]
== reference["hashes"]["initial_mixer_parameters"]
),
"model_topology": run["model"] == reference["model"],
"optimizer_hyperparameters": (
run["optimizer"] == reference["optimizer"]
),
"scientific_environment": (
stable_environment(run) == stable_environment(reference)
),
}
for field in manifest_fields:
checks[f"manifest.{field}"] = (
run["manifest"][field] == reference["manifest"][field]
)
return checks
def scientific_replay_payload(value: dict[str, Any]) -> dict[str, Any]:
payload = copy.deepcopy(value)
for key in (
"run_kind",
"timing",
"canonical_sha256_without_self",
"parent_runner_canonical_sha256",
):
payload.pop(key, None)
payload["manifest"].pop("path", None)
payload["study_manifest"].pop("path", None)
payload["environment"] = stable_environment(value)
return payload
def quality_gate(
variant_runs: dict[int, dict[str, Any]],
references: dict[int, dict[str, Any]],
*,
step: int,
per_seed_maximum: float,
mean_maximum: float,
) -> dict[str, Any]:
per_seed = {}
for seed, run in sorted(variant_runs.items()):
variant_bpc = final_bpc(run, step)
reference_bpc = final_bpc(references[seed], step)
delta = variant_bpc - reference_bpc
per_seed[str(seed)] = {
"variant_bpc": variant_bpc,
"reference_bpc": reference_bpc,
"delta_bpc": delta,
"passed": delta <= per_seed_maximum,
}
mean_delta = mean(item["delta_bpc"] for item in per_seed.values())
per_seed_passed = all(item["passed"] for item in per_seed.values())
mean_passed = mean_delta <= mean_maximum
return {
"passed": per_seed_passed and mean_passed,
"passed_checks": (
sum(item["passed"] for item in per_seed.values())
+ int(mean_passed)
),
"required_checks": 4,
"per_seed_maximum": per_seed_maximum,
"mean_maximum": mean_maximum,
"mean_delta_bpc": mean_delta,
"mean_passed": mean_passed,
"per_seed": per_seed,
}
def variant_effect(
variant: str,
runs: dict[int, dict[str, Any]],
references: dict[int, dict[str, Any]],
metrics_by_cell: dict[tuple[str, int, int], dict[str, Any]],
*,
step: int,
threshold: float,
quality: dict[str, Any],
) -> dict[str, Any]:
cells = []
for seed in sorted(runs):
candidate = metrics_by_cell[(variant, seed, step)]
reference = metrics_by_cell[("learned_reference", seed, step)]
for metric in METRICS:
reference_value = reference[metric]
candidate_value = candidate[metric]
relative_drop = (
reference_value - candidate_value
) / reference_value
cells.append(
{
"seed": seed,
"metric": metric,
"reference": reference_value,
"variant": candidate_value,
"relative_drop": relative_drop,
"passed": relative_drop >= threshold,
}
)
attenuation_passed = all(cell["passed"] for cell in cells)
return {
"variant": variant,
"threshold": threshold,
"passed_cells": sum(cell["passed"] for cell in cells),
"required_cells": len(cells),
"attenuation_passed": attenuation_passed,
"quality": quality,
"material_response_passed": (
attenuation_passed and quality["passed"]
),
"cells": cells,
}
def interaction_map(
metrics_by_cell: dict[tuple[str, int, int], dict[str, Any]],
seeds: tuple[int, ...],
steps: tuple[int, ...],
) -> dict[str, Any]:
cells = []
for step in steps:
for seed in seeds:
reference = metrics_by_cell[
("learned_reference", seed, step)
]
group6 = metrics_by_cell[
("uniform_group_6_forward", seed, step)
]
group7 = metrics_by_cell[
("uniform_group_7_forward", seed, step)
]
joint = metrics_by_cell[
("uniform_groups_6_7_forward", seed, step)
]
for metric in METRICS:
ref = reference[metric]
effects = {
"group6": math.log(ref / group6[metric]),
"group7": math.log(ref / group7[metric]),
"groups6_7": math.log(ref / joint[metric]),
}
residual = (
effects["groups6_7"]
- effects["group6"]
- effects["group7"]
)
cells.append(
{
"step": step,
"seed": seed,
"metric": metric,
"log_effects": effects,
"interaction_residual": residual,
"relative_drops": {
"group6": (ref - group6[metric]) / ref,
"group7": (ref - group7[metric]) / ref,
"groups6_7": (ref - joint[metric]) / ref,
},
}
)
summaries = []
for step in steps:
for metric in METRICS:
selected = [
cell
for cell in cells
if cell["step"] == step and cell["metric"] == metric
]
residuals = [
cell["interaction_residual"] for cell in selected
]
summaries.append(
{
"step": step,
"metric": metric,
"mean_interaction_residual": mean(residuals),
"minimum": min(residuals),
"maximum": max(residuals),
}
)
return {
"definition": "I67=ln(Xref/X67)-ln(Xref/X6)-ln(Xref/X7)",
"interpretation": (
"descriptive cross-run log-attenuation residual from three "
"independently trained variants; not a causal interaction"
),
"cells": cells,
"summaries": summaries,
}
def environment_metadata(value: dict[str, Any]) -> dict[str, Any]:
return {
key: value["environment"].get(key)
for key in ("gpu", "torch", "cuda", "compute_capability")
}
def write_hashed(path: Path, value: dict[str, Any]) -> None:
value["canonical_sha256_without_self"] = canonical_sha256(value)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
)
def main() -> None:
args = parse_args()
manifest = json.loads(args.manifest.read_text())
if (
manifest["protocol_id"] != PROTOCOL_ID
or manifest["status"] != "frozen-before-model-output"
):
raise RuntimeError("manifest is not the frozen Round 08 contract")
variants = tuple(manifest["variants"].keys())
seeds = tuple(manifest["formal_seeds"])
steps = tuple(manifest["diagnostic_steps"])
primary_step = manifest["primary_step"]
epsilon = manifest["thresholds"]["positive_denominator_epsilon"]
spike_layers = tuple(manifest["spike_layers_1based"])
expected_cells = {(variant, seed) for variant in variants for seed in seeds}
if len(args.formal) != len(expected_cells):
raise RuntimeError("formal path count does not match the 4×3 matrix")
runs: dict[tuple[str, int], dict[str, Any]] = {}
run_paths: dict[tuple[str, int], Path] = {}
pairing: dict[str, Any] = {}
references: dict[int, dict[str, Any]] = {}
reference_paths: dict[int, Path] = {}
for seed in seeds:
path = args.reference_dir / (
f"formal-depth-32-block-seed-{seed}.json"
)
references[seed] = read_result(path, PARENT_PROTOCOL_ID)
reference_paths[seed] = path
for path in args.formal:
value = read_result(path, PROTOCOL_ID)
identity = (value["variant"], value["seed"])
if identity in runs:
raise RuntimeError(f"duplicate formal cell: {identity}")
if (
value["run_kind"] != "formal"
or value["steps"] != manifest["formal_steps"]
or not value["forward_intervention"]["passed"]
):
raise RuntimeError(f"invalid formal cell: {path}")
runs[identity] = value
run_paths[identity] = path
if set(runs) != expected_cells:
raise RuntimeError("formal matrix identities do not match manifest")
for (variant, seed), value in sorted(runs.items()):
checks = pairing_checks(value, references[seed])
if not all(checks.values()):
raise RuntimeError(
f"historical reference pairing failed: "
f"{variant}/{seed}: {checks}"
)
pairing[f"{variant}:{seed}"] = {
"passed": True,
"checks": checks,
"run_environment": environment_metadata(value),
"reference_environment": environment_metadata(references[seed]),
"metadata_equal": (
environment_metadata(value)
== environment_metadata(references[seed])
),
}
metadata_warnings = [
{
"cell": cell,
"message": (
"GPU/version metadata differs from the historical paired "
"reference; frozen scientific-environment fields still match"
),
"run_environment": item["run_environment"],
"reference_environment": item["reference_environment"],
}
for cell, item in pairing.items()
if not item["metadata_equal"]
]
replay = read_result(args.replay, PROTOCOL_ID)
replay_contract = manifest["replay"]
if (
replay["run_kind"] != "replay"
or replay["variant"] != replay_contract["variant"]
or replay["seed"] != replay_contract["seed"]
or replay["steps"] != manifest["formal_steps"]
or not replay["forward_intervention"]["passed"]
):
raise RuntimeError("invalid replay identity/audit")
formal_primary = runs[
(replay_contract["variant"], replay_contract["seed"])
]
formal_payload = scientific_replay_payload(formal_primary)
replay_payload = scientific_replay_payload(replay)
replay_exact = formal_payload == replay_payload
if not replay_exact:
raise RuntimeError("primary formal/replay scientific payload mismatch")
metrics_by_cell: dict[tuple[str, int, int], dict[str, Any]] = {}
for seed, reference in references.items():
for step in steps:
metrics_by_cell[("learned_reference", seed, step)] = (
spectrum_metrics(
exactly_one(reference["diagnostics"], step),
spike_layers,
epsilon,
)
)
for (variant, seed), value in runs.items():
if tuple(item["step"] for item in value["diagnostics"]) != steps:
raise RuntimeError(f"diagnostic schedule drift: {variant}/{seed}")
for step in steps:
metrics_by_cell[(variant, seed, step)] = spectrum_metrics(
exactly_one(value["diagnostics"], step),
spike_layers,
epsilon,
)
runs_by_variant = {
variant: {seed: runs[(variant, seed)] for seed in seeds}
for variant in variants
}
qualities = {
variant: quality_gate(
variant_runs,
references,
step=primary_step,
per_seed_maximum=manifest["thresholds"][
"final_bpc_delta_per_seed_maximum"
],
mean_maximum=manifest["thresholds"][
"final_bpc_delta_mean_maximum"
],
)
for variant, variant_runs in runs_by_variant.items()
}
effects = {
variant: variant_effect(
variant,
variant_runs,
references,
metrics_by_cell,
step=primary_step,
threshold=manifest["thresholds"]["material_relative_drop"],
quality=qualities[variant],
)
for variant, variant_runs in runs_by_variant.items()
}
primary = effects[manifest["primary_variant"]]
if primary["attenuation_passed"] and primary["quality"]["passed"]:
status = (
"forward_training_attenuation_established_within_reduced_protocol"
)
elif primary["attenuation_passed"]:
status = "quality_guard_failed"
elif primary["quality"]["passed"]:
status = "attenuation_not_established"
else:
status = "attenuation_and_quality_failed"
secondary = {
variant: (
"secondary_material_response"
if effect["material_response_passed"]
else "secondary_response_not_established"
)
for variant, effect in effects.items()
if variant != manifest["primary_variant"]
}
interaction = interaction_map(metrics_by_cell, seeds, steps)
trajectories = []
final_spectra = []
for variant in ("learned_reference",) + variants:
for seed in seeds:
for step in steps:
record = metrics_by_cell[(variant, seed, step)]
reference = metrics_by_cell[
("learned_reference", seed, step)
]
trajectories.append(
{
"variant": variant,
"seed": seed,
"step": step,
"spike_mean": record["spike_mean"],
"reference_mean": record["reference_mean"],
"spike_contrast": record["spike_contrast"],
"peak_normalized": record["peak_normalized"],
"relative_drop": {
metric: (
reference[metric] - record[metric]
)
/ reference[metric]
for metric in METRICS
},
}
)
final = metrics_by_cell[(variant, seed, primary_step)]
final_spectra.append(
{
"variant": variant,
"seed": seed,
**final,
}
)
input_files = {
"manifest": {
"path": str(args.manifest),
"sha256": file_sha256(args.manifest),
},
"formal": [
{
"variant": variant,
"seed": seed,
"path": str(run_paths[(variant, seed)]),
"sha256": file_sha256(run_paths[(variant, seed)]),
}
for variant, seed in sorted(runs)
],
"references": [
{
"seed": seed,
"path": str(reference_paths[seed]),
"sha256": file_sha256(reference_paths[seed]),
}
for seed in seeds
],
"replay": {
"path": str(args.replay),
"sha256": file_sha256(args.replay),
},
}
aggregate = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"status": status,
"scope": (
"depth-32 reduced Block AttnRes train-time architecture "
"ablation; not a real Kimi-K3 checkpoint result"
),
"primary_step": primary_step,
"spike_layers_1based": list(spike_layers),
"thresholds": manifest["thresholds"],
"input_files": input_files,
"historical_pairing": pairing,
"metadata_warnings": metadata_warnings,
"replay": {
"passed": replay_exact,
"scientific_payload_sha256": canonical_sha256(formal_payload),
"excluded": [
"run_kind",
"timing",
"self hashes",
"manifest path strings",
"GPU/version metadata",
],
},
"primary": primary,
"secondary_status": secondary,
"effects": effects,
"interaction": interaction,
"trajectories": trajectories,
"final_spectra": final_spectra,
"processed_target_bytes": manifest["new_target_bytes"],
"historical_reference_target_bytes": (
manifest["historical_reference_target_bytes"]
),
"reporting_boundary": (
"C can change through spike-window numerator and the 27-layer "
"reference denominator; layers 26-28 are intervened but belong "
"to the denominator."
),
}
write_hashed(args.aggregate_output, aggregate)
compact = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"status": status,
"primary_step": primary_step,
"spike_layers_1based": list(spike_layers),
"thresholds": manifest["thresholds"],
"primary": primary,
"secondary_status": secondary,
"effects": effects,
"interaction": interaction,
"trajectories": trajectories,
"final_spectra": final_spectra,
"replay": aggregate["replay"],
"metadata_warnings": metadata_warnings,
"processed_target_bytes": manifest["new_target_bytes"],
"reporting_boundary": aggregate["reporting_boundary"],
"aggregate_sha256": aggregate["canonical_sha256_without_self"],
}
write_hashed(args.compact_output, compact)
reproduction = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"passed": replay_exact,
"formal_variant": replay_contract["variant"],
"seed": replay_contract["seed"],
"formal_file_sha256": file_sha256(
run_paths[
(replay_contract["variant"], replay_contract["seed"])
]
),
"replay_file_sha256": file_sha256(args.replay),
"scientific_payload_sha256": canonical_sha256(formal_payload),
"excluded_fields": aggregate["replay"]["excluded"],
}
write_hashed(args.reproduction_output, reproduction)
print(
json.dumps(
{
"status": status,
"primary_attenuation": {
"passed_cells": primary["passed_cells"],
"required_cells": primary["required_cells"],
},
"primary_quality": {
"passed_checks": primary["quality"]["passed_checks"],
"required_checks": primary["quality"]["required_checks"],
},
"replay_exact": replay_exact,
"aggregate": str(args.aggregate_output),
"compact": str(args.compact_output),
},
ensure_ascii=False,
indent=2,
)
)
if __name__ == "__main__":
main()
@@ -0,0 +1,89 @@
{
"schema_version": 1,
"protocol_id": "llm-atlas-k3-attnres-forward-training-v1",
"status": "frozen-before-model-output",
"parent_protocol_id": "llm-atlas-k3-attnres-gradient-scale-v1",
"parent_manifest": "experiments/k3/attnres_gradient/manifest.json",
"architecture": "block",
"depth": 32,
"formal_steps": 8000,
"smoke_steps": 20,
"batch_size": 32,
"formal_seeds": [
2026073001,
2026073002,
2026073003
],
"diagnostic_steps": [
0,
100,
500,
2000,
4000,
8000
],
"primary_step": 8000,
"spike_layers_1based": [21, 22, 23, 24, 25],
"historical_reference": {
"directory": "experiments/k3/attnres_gradient/results/raw",
"filename_template": "formal-depth-32-block-seed-{seed}.json",
"identity": "historical-paired-reference-not-contemporaneous-randomized-control"
},
"variants": {
"uniform_group_6_forward": {
"selected_depth_indices": [40, 41, 42, 43, 44, 45, 46, 47]
},
"uniform_group_7_forward": {
"selected_depth_indices": [48, 49, 50, 51, 52, 53, 54, 55]
},
"uniform_groups_6_7_forward": {
"selected_depth_indices": [40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55]
},
"uniform_group_7_mlp_forward": {
"selected_depth_indices": [49, 51, 53, 55]
}
},
"smoke_only_variants": [
"learned_reference"
],
"primary_variant": "uniform_groups_6_7_forward",
"replay": {
"variant": "uniform_groups_6_7_forward",
"seed": 2026073001
},
"selected_source_counts": {
"40": 6,
"41": 7,
"42": 7,
"43": 7,
"44": 7,
"45": 7,
"46": 7,
"47": 7,
"48": 7,
"49": 8,
"50": 8,
"51": 8,
"52": 8,
"53": 8,
"54": 8,
"55": 8
},
"thresholds": {
"positive_denominator_epsilon": 1e-30,
"material_relative_drop": 0.2,
"final_bpc_delta_per_seed_maximum": 0.05,
"final_bpc_delta_mean_maximum": 0.03,
"uniform_weight_max_abs_error": 1e-12,
"loss_scale_ratio_abs_error": 1e-5,
"loss_scale_shape_abs_error": 1e-6
},
"new_target_bytes": {
"per_cell": 65536000,
"formal_12_cells": 786432000,
"primary_replay": 65536000,
"total": 851968000
},
"historical_reference_target_bytes": 196608000,
"concurrency_maximum": 2
}
@@ -0,0 +1,25 @@
{
"canonical_sha256_without_self": "57346df80c0d76bd1d306d5fa213feed74ac2094237c22ebcb49f16ea16437e0",
"excluded_fields": [
"run_kind",
"timing",
"self hashes",
"manifest path strings",
"GPU/version metadata"
],
"formal_file_sha256": "0962ebd1a00a11e61ac795282bfa99412166c8752f2a3731f137030d7f134dc1",
"formal_variant": "uniform_groups_6_7_forward",
"passed": true,
"post_result_grok_review": {
"blocking_errors": 0,
"claim_boundary_confirmed": true,
"replay_confirmed": true,
"session_id": "019fb28b-a9e1-7643-8e43-06f5e16a2077",
"status_confirmed": true
},
"protocol_id": "llm-atlas-k3-attnres-forward-training-v1",
"replay_file_sha256": "b85ac8062b2b0b8b3f2305d4b22a7c0212466fbfb28a9d63099cadd0a6917c8e",
"schema_version": 1,
"scientific_payload_sha256": "b85563ca5cb53e60b39c3801d372376206105b8a089a8633b3e81973a7f0c051",
"seed": 2026073001
}
@@ -0,0 +1,39 @@
{
"canonical_sha256_without_self": "99aeefc0ba35c199725ed7af377450ecbb5cc5f1aabdda9ac2345a30f03b444f",
"excluded_fields": [
"protocol wrapper fields",
"timing",
"self hash",
"parent runner self hash",
"study manifest"
],
"field_checks": {
"architecture": true,
"batch_size": true,
"depth": true,
"diagnostics": true,
"environment": true,
"evaluations": true,
"gradient_gate": true,
"hashes": true,
"manifest": true,
"model": true,
"optimizer": true,
"seed": true,
"steps": true,
"target_bytes_seen": true,
"training_history": true
},
"gate": "empty-selector-parent-equivalence",
"parent_file": "/home/wuyang/Code/K3/experiments/k3/attnres_forward/results/gates/parent-smoke.json",
"passed": true,
"protocol_id": "llm-atlas-k3-attnres-forward-training-v1",
"schema_version": 1,
"wrapper_file": "/home/wuyang/Code/K3/experiments/k3/attnres_forward/results/gates/wrapper-smoke.json",
"wrapper_identity": {
"forward_audit": true,
"parent_protocol": true,
"protocol": true,
"variant": true
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,220 @@
#!/usr/bin/env python3
"""Run the frozen Round 08 matrix with at most two isolated processes."""
from __future__ import annotations
import argparse
import json
import os
import subprocess
import time
from pathlib import Path
from typing import Any
VARIANTS = (
"uniform_group_6_forward",
"uniform_group_7_forward",
"uniform_groups_6_7_forward",
"uniform_group_7_mlp_forward",
)
SEEDS = (2026073001, 2026073002, 2026073003)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument("--python", type=Path, required=True)
parser.add_argument("--cache-dir", type=Path, required=True)
parser.add_argument("--parent-manifest", type=Path, required=True)
parser.add_argument("--study-manifest", type=Path, required=True)
parser.add_argument("--output-dir", type=Path, required=True)
parser.add_argument(
"--phase", choices=("formal", "replay", "all"), default="all"
)
parser.add_argument("--concurrency", type=int, default=2)
return parser.parse_args()
def cell_output(
output_dir: Path, variant: str, seed: int, run_kind: str
) -> Path:
return output_dir / (
f"{run_kind}-{variant}-seed-{seed}.json"
)
def command_for(
args: argparse.Namespace, variant: str, seed: int, run_kind: str
) -> list[str]:
runner = Path(__file__).resolve().parent / "train.py"
return [
str(args.python),
str(runner),
"--variant",
variant,
"--study-manifest",
str(args.study_manifest),
"--run-kind",
run_kind,
"--architecture",
"block",
"--depth",
"32",
"--seed",
str(seed),
"--cache-dir",
str(args.cache_dir),
"--manifest",
str(args.parent_manifest),
"--output",
str(cell_output(args.output_dir, variant, seed, run_kind)),
]
def validate_manifest(args: argparse.Namespace) -> None:
manifest = json.loads(args.study_manifest.read_text())
if (
manifest["status"] != "frozen-before-model-output"
or tuple(manifest["variants"]) != VARIANTS
or tuple(manifest["formal_seeds"]) != SEEDS
or manifest["concurrency_maximum"] != 2
):
raise RuntimeError("study manifest matrix/concurrency drift")
if args.concurrency < 1 or args.concurrency > 2:
raise ValueError("the frozen protocol permits one or two processes")
def stop_processes(items: list[dict[str, Any]]) -> None:
for item in items:
if item["process"].poll() is None:
item["process"].terminate()
for item in items:
process = item["process"]
if process.poll() is not None:
continue
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait()
def quarantine_failed_output(
output_dir: Path, variant: str, seed: int, run_kind: str
) -> str | None:
output = cell_output(output_dir, variant, seed, run_kind)
if not output.exists():
return None
failed = output.with_suffix(".failed.json")
if failed.exists():
failed = output.with_suffix(f".failed-{time.time_ns()}.json")
output.replace(failed)
return str(failed)
def run_cells(
args: argparse.Namespace,
cells: list[tuple[str, int, str]],
) -> None:
args.output_dir.mkdir(parents=True, exist_ok=True)
for variant, seed, run_kind in cells:
output = cell_output(args.output_dir, variant, seed, run_kind)
if output.exists():
raise FileExistsError(
f"refusing to overwrite existing result: {output}"
)
environment = dict(os.environ)
environment["CUBLAS_WORKSPACE_CONFIG"] = ":4096:8"
pending = list(cells)
running: list[dict[str, Any]] = []
completed = 0
while pending or running:
while pending and len(running) < args.concurrency:
variant, seed, run_kind = pending.pop(0)
command = command_for(args, variant, seed, run_kind)
process = subprocess.Popen(command, env=environment)
running.append(
{
"identity": (variant, seed, run_kind),
"process": process,
"started": time.monotonic(),
}
)
print(
json.dumps(
{
"event": "cell_started",
"variant": variant,
"seed": seed,
"run_kind": run_kind,
"pid": process.pid,
"active": len(running),
"remaining": len(pending),
},
sort_keys=True,
),
flush=True,
)
time.sleep(1)
survivors = []
for item in running:
return_code = item["process"].poll()
if return_code is None:
survivors.append(item)
continue
variant, seed, run_kind = item["identity"]
elapsed = time.monotonic() - item["started"]
if return_code != 0:
stop_processes(
[candidate for candidate in running if candidate is not item]
)
quarantined = quarantine_failed_output(
args.output_dir, variant, seed, run_kind
)
raise RuntimeError(
f"cell failed: {variant}/{seed}/{run_kind}: {return_code}; "
f"quarantined_output={quarantined}"
)
completed += 1
print(
json.dumps(
{
"event": "cell_completed",
"variant": variant,
"seed": seed,
"run_kind": run_kind,
"elapsed_seconds": elapsed,
"completed": completed,
"total": len(cells),
},
sort_keys=True,
),
flush=True,
)
running = survivors
def main() -> None:
args = parse_args()
validate_manifest(args)
formal = [
(variant, seed, "formal")
for variant in VARIANTS
for seed in SEEDS
]
replay = [
("uniform_groups_6_7_forward", 2026073001, "replay")
]
cells = (
formal
if args.phase == "formal"
else replay
if args.phase == "replay"
else formal + replay
)
run_cells(args, cells)
if __name__ == "__main__":
main()
+449
View File
@@ -0,0 +1,449 @@
#!/usr/bin/env python3
"""Run one preregistered Round 08 train-time uniform-forward cell."""
from __future__ import annotations
import importlib.util
import json
import math
import os
import sys
from pathlib import Path
from typing import Any
import torch
import torch.nn.functional as F
PROTOCOL_ID = "llm-atlas-k3-attnres-forward-training-v1"
PARENT_PROTOCOL_ID = "llm-atlas-k3-attnres-gradient-scale-v1"
VARIANTS = {
"learned_reference": (),
"uniform_group_6_forward": tuple(range(40, 48)),
"uniform_group_7_forward": tuple(range(48, 56)),
"uniform_groups_6_7_forward": tuple(range(40, 56)),
"uniform_group_7_mlp_forward": (49, 51, 53, 55),
}
FORMAL_VARIANTS = tuple(name for name in VARIANTS if name != "learned_reference")
EXPECTED_SOURCE_COUNTS = {
**{40: 6},
**{index: 7 for index in range(41, 49)},
**{index: 8 for index in range(49, 56)},
}
def load_parent_module() -> Any:
path = Path(__file__).resolve().parents[1] / "attnres_gradient" / "train.py"
spec = importlib.util.spec_from_file_location("k3_attnres_round05_train", path)
if spec is None or spec.loader is None:
raise RuntimeError(f"cannot import Round 05 runner from {path}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
parent = load_parent_module()
ACTIVE_VARIANT = "learned_reference"
LAST_MODEL: ForwardInterventionLanguageModel | None = None
LAST_OPTIMIZER: torch.optim.Optimizer | None = None
def extract_wrapper_argument(name: str) -> str:
try:
index = sys.argv.index(name)
except ValueError as error:
raise ValueError(f"missing required wrapper argument: {name}") from error
if index + 1 >= len(sys.argv):
raise ValueError(f"missing value for wrapper argument: {name}")
value = sys.argv[index + 1]
del sys.argv[index : index + 2]
return value
def argument_value(name: str, default: str | None = None) -> str | None:
try:
index = sys.argv.index(name)
except ValueError:
return default
if index + 1 >= len(sys.argv):
raise ValueError(f"missing value for argument: {name}")
return sys.argv[index + 1]
def parameter_names_for_indices(indices: tuple[int, ...]) -> tuple[str, ...]:
names = []
for index in indices:
names.extend(
(
f"mixers.{index}.query",
f"mixers.{index}.key_norm.weight",
)
)
return tuple(names)
class ForwardInterventionLanguageModel(parent.GradientLanguageModel):
"""Round 05 model with one frozen selector and parameter-free uniform mixers."""
def __init__(self, architecture: str):
super().__init__(architecture)
global LAST_MODEL
if architecture != "block":
raise ValueError("Round 08 only permits the block architecture")
if ACTIVE_VARIANT not in VARIANTS:
raise ValueError(f"unknown Round 08 variant: {ACTIVE_VARIANT}")
self.forward_variant = ACTIVE_VARIANT
self.selected_indices = tuple(VARIANTS[ACTIVE_VARIANT])
self.selected_set = frozenset(self.selected_indices)
self.forward_calls = 0
self.depth_visits = [0] * len(self.mixers)
self.output_visits = 0
self.source_counts: dict[int, set[int]] = {
index: set() for index in range(len(self.mixers))
}
self.uniform_weight_max_abs_error = 0.0
selected_names = parameter_names_for_indices(self.selected_indices)
named_parameters = dict(self.named_parameters())
self.selected_initial_tensors = {
name: named_parameters[name].detach().cpu().clone()
for name in selected_names
}
self.gradient_hook_calls = {
name: 0
for name in named_parameters
if name.startswith("mixers.") or name.startswith("output_mixer.")
}
self._gradient_hooks = []
for name, parameter in named_parameters.items():
if name not in self.gradient_hook_calls:
continue
def count_hook(
gradient: torch.Tensor, *, parameter_name: str = name
) -> torch.Tensor:
self.gradient_hook_calls[parameter_name] += 1
return gradient
self._gradient_hooks.append(parameter.register_hook(count_hook))
LAST_MODEL = self
def mix(
self,
mixer_index: int,
sources: list[torch.Tensor],
capture: bool,
) -> tuple[torch.Tensor, dict[str, Any] | None]:
self.depth_visits[mixer_index] += 1
self.source_counts[mixer_index].add(len(sources))
if mixer_index not in self.selected_set:
return self.mixers[mixer_index](sources, capture)
values = torch.stack(sources, dim=0)
logits = torch.zeros(
values.shape[0],
values.shape[1],
values.shape[2],
dtype=torch.float32,
device=values.device,
)
weights = torch.softmax(logits, dim=0)
expected = torch.tensor(
1.0 / len(sources), dtype=weights.dtype, device=weights.device
)
error = (weights - expected).abs().max().detach().cpu().item()
self.uniform_weight_max_abs_error = max(
self.uniform_weight_max_abs_error, error
)
output = torch.einsum(
"nbt,nbtd->btd", weights, values.float()
).to(values.dtype)
if not capture:
return output, None
entropy = -(weights * torch.log(weights.clamp_min(1e-30))).sum(dim=0)
return output, {
"mean_weights": weights.mean(dim=(1, 2)).detach().cpu().tolist(),
"entropy_mean": entropy.mean().detach().cpu().item(),
"sources": len(sources),
}
def forward(
self, input_ids: torch.Tensor, capture: bool = False
) -> tuple[torch.Tensor, parent.ActivationTrace | None]:
self.forward_calls += 1
embedded = self.embed(input_ids)
trace = parent.ActivationTrace([], [], [], [], []) if capture else None
completed = [embedded]
partial: torch.Tensor | None = None
mixer_index = 0
for block in self.blocks:
for branch_index in range(2):
sources = completed + ([] if partial is None else [partial])
branch_input, weights = self.mix(
mixer_index, sources, capture
)
mixer_index += 1
if branch_index == 0:
branch_output = block.attention(
block.attention_norm(branch_input)
)
else:
branch_output = block.mlp(block.mlp_norm(branch_input))
branch_for_residual = branch_output.float()
partial = (
branch_for_residual
if partial is None
else partial + branch_for_residual
)
if trace is not None:
trace.layer_input_rms.append(parent.rms(branch_input))
trace.branch_output_rms.append(parent.rms(branch_output))
trace.stream_state_rms.append(parent.rms(partial))
trace.depth_weights.append(weights or {})
if branch_index == 1:
partial.retain_grad()
trace.block_outputs.append(partial)
if mixer_index % parent.round04.SUBLAYERS_PER_BLOCK == 0:
completed.append(partial)
partial = None
if partial is not None or len(completed) != parent.BLOCK_GROUPS + 1:
raise RuntimeError("Round 08 Block AttnRes aggregation failed")
if self.output_mixer is None:
raise RuntimeError("Round 08 output mixer missing")
self.output_visits += 1
hidden, output_weights = self.output_mixer(completed, capture)
if trace is not None:
trace.output_weights = output_weights
normalized = self.final_norm(hidden)
logits = F.linear(normalized, self.token_embedding.weight)
return logits, trace
def tensor_exact(left: torch.Tensor, right: torch.Tensor) -> bool:
return (
left.dtype == right.dtype
and tuple(left.shape) == tuple(right.shape)
and torch.equal(left.detach().cpu(), right.detach().cpu())
)
def build_intervention_audit(
model: ForwardInterventionLanguageModel,
optimizer: torch.optim.Optimizer,
study_manifest: dict[str, Any],
) -> dict[str, Any]:
selected = tuple(model.selected_indices)
selected_names = set(parameter_names_for_indices(selected))
mixer_parameters = {
name: parameter
for name, parameter in model.named_parameters()
if name.startswith("mixers.") or name.startswith("output_mixer.")
}
optimizer_parameters = {
parameter
for group in optimizer.param_groups
for parameter in group["params"]
}
selected_parameter_checks = {}
for name in sorted(selected_names):
parameter = mixer_parameters[name]
selected_parameter_checks[name] = {
"gradient_hook_calls": model.gradient_hook_calls[name],
"in_optimizer_param_group": parameter in optimizer_parameters,
"optimizer_state_present": parameter in optimizer.state,
"final_equals_initial": tensor_exact(
parameter, model.selected_initial_tensors[name]
),
}
unselected_parameter_checks = {}
for name, parameter in sorted(mixer_parameters.items()):
if name in selected_names:
continue
unselected_parameter_checks[name] = {
"gradient_hook_calls": model.gradient_hook_calls[name],
"in_optimizer_param_group": parameter in optimizer_parameters,
"optimizer_state_present": parameter in optimizer.state,
}
source_counts = {
str(index): sorted(values)
for index, values in model.source_counts.items()
}
selected_source_gate = {
str(index): (
source_counts[str(index)]
== [study_manifest["selected_source_counts"][str(index)]]
== [EXPECTED_SOURCE_COUNTS[index]]
)
for index in selected
}
visit_gate = (
model.forward_calls > 0
and all(value == model.forward_calls for value in model.depth_visits)
and model.output_visits == model.forward_calls
)
selected_parameter_gate = all(
check["gradient_hook_calls"] == 0
and check["in_optimizer_param_group"]
and not check["optimizer_state_present"]
and check["final_equals_initial"]
for check in selected_parameter_checks.values()
)
unselected_parameter_gate = all(
check["gradient_hook_calls"] > 0
and check["in_optimizer_param_group"]
and check["optimizer_state_present"]
for check in unselected_parameter_checks.values()
)
expected_selected = tuple(
study_manifest["variants"]
.get(model.forward_variant, {"selected_depth_indices": []})[
"selected_depth_indices"
]
)
selector_gate = (
selected == expected_selected
and 64 not in selected
and selected_source_gate == {
str(index): True for index in selected
}
)
threshold = study_manifest["thresholds"][
"uniform_weight_max_abs_error"
]
uniform_gate = model.uniform_weight_max_abs_error <= threshold
passed = (
visit_gate
and selector_gate
and selected_parameter_gate
and unselected_parameter_gate
and uniform_gate
)
return {
"passed": passed,
"variant": model.forward_variant,
"selected_depth_indices": list(selected),
"output_mixer_selected": False,
"forward_calls": model.forward_calls,
"depth_visit_counts": model.depth_visits,
"output_visit_count": model.output_visits,
"visit_gate": visit_gate,
"source_counts_by_depth_index": source_counts,
"selected_source_count_checks": selected_source_gate,
"selector_gate": selector_gate,
"uniform_weight_max_abs_error": model.uniform_weight_max_abs_error,
"uniform_weight_threshold": threshold,
"uniform_weight_gate": uniform_gate,
"selected_parameters": selected_parameter_checks,
"selected_parameter_reachability_gate": selected_parameter_gate,
"unselected_parameters": unselected_parameter_checks,
"unselected_parameter_reachability_gate": unselected_parameter_gate,
"semantics": (
"selected depth mixers use parameter-free constant-zero logits "
"with the parent softmax+einsum arithmetic kernel"
),
}
def rewrite_result(
output_path: Path,
study_manifest_path: Path,
study_manifest: dict[str, Any],
) -> None:
if LAST_MODEL is None or LAST_OPTIMIZER is None:
raise RuntimeError("runner capture state missing")
result = json.loads(output_path.read_text())
parent_self_hash = result.pop("canonical_sha256_without_self")
if result["protocol_id"] != PARENT_PROTOCOL_ID:
raise RuntimeError("parent runner protocol drift")
result["schema_version"] = 2
result["protocol_id"] = PROTOCOL_ID
result["parent_protocol_id"] = PARENT_PROTOCOL_ID
result["variant"] = ACTIVE_VARIANT
result["parent_runner_canonical_sha256"] = parent_self_hash
result["study_manifest"] = {
"path": str(study_manifest_path),
"file_sha256": parent.file_sha256(study_manifest_path),
"status": study_manifest["status"],
}
result["forward_intervention"] = build_intervention_audit(
LAST_MODEL, LAST_OPTIMIZER, study_manifest
)
result["canonical_sha256_without_self"] = parent.canonical_sha256(result)
temporary = output_path.with_suffix(output_path.suffix + ".round08.tmp")
temporary.write_text(
json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
)
os.replace(temporary, output_path)
if not result["forward_intervention"]["passed"]:
raise RuntimeError(
f"forward intervention audit failed: "
f"{result['forward_intervention']}"
)
def main() -> None:
global ACTIVE_VARIANT, LAST_OPTIMIZER
variant = extract_wrapper_argument("--variant")
study_manifest_path = Path(
extract_wrapper_argument("--study-manifest")
).resolve()
if variant not in VARIANTS:
raise ValueError(f"unknown variant: {variant}")
run_kind = argument_value("--run-kind", "formal")
if run_kind in ("formal", "replay") and variant not in FORMAL_VARIANTS:
raise ValueError("learned_reference is smoke-only")
if argument_value("--architecture") != "block":
raise ValueError("Round 08 requires --architecture block")
if argument_value("--depth") != "32":
raise ValueError("Round 08 requires --depth 32")
if run_kind == "replay" and variant != "uniform_groups_6_7_forward":
raise ValueError("the frozen replay uses the primary joint variant")
study_manifest = json.loads(study_manifest_path.read_text())
if (
study_manifest["protocol_id"] != PROTOCOL_ID
or study_manifest["status"] != "frozen-before-model-output"
):
raise ValueError("study manifest is not the frozen Round 08 contract")
expected = tuple(
study_manifest["variants"]
.get(variant, {"selected_depth_indices": []})[
"selected_depth_indices"
]
)
if expected != VARIANTS[variant]:
raise ValueError("study manifest selector drift")
for index, source_count in EXPECTED_SOURCE_COUNTS.items():
if (
study_manifest["selected_source_counts"].get(str(index))
!= source_count
):
raise ValueError(
f"study manifest source-count drift at depth index {index}"
)
output_value = argument_value("--output")
if output_value is None:
raise ValueError("--output is required")
output_path = Path(output_value).resolve()
ACTIVE_VARIANT = variant
parent.GradientLanguageModel = ForwardInterventionLanguageModel
original_adamw = torch.optim.AdamW
def capture_adamw(*args: Any, **kwargs: Any) -> torch.optim.Optimizer:
global LAST_OPTIMIZER
LAST_OPTIMIZER = original_adamw(*args, **kwargs)
return LAST_OPTIMIZER
torch.optim.AdamW = capture_adamw # type: ignore[assignment]
try:
parent.main()
finally:
torch.optim.AdamW = original_adamw # type: ignore[assignment]
rewrite_result(output_path, study_manifest_path, study_manifest)
if __name__ == "__main__":
main()
+294
View File
@@ -0,0 +1,294 @@
#!/usr/bin/env python3
"""Run pre-result Round 08 identity gates."""
from __future__ import annotations
import argparse
import hashlib
import importlib.util
import json
import os
import sys
from pathlib import Path
from typing import Any
import torch
def load_runner() -> Any:
path = Path(__file__).resolve().parent / "train.py"
spec = importlib.util.spec_from_file_location("k3_attnres_round08_train", path)
if spec is None or spec.loader is None:
raise RuntimeError(f"cannot import Round 08 runner from {path}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
runner = load_runner()
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest="command", required=True)
step_zero = subparsers.add_parser("step-zero")
step_zero.add_argument("--cache-dir", type=Path, required=True)
step_zero.add_argument("--parent-manifest", type=Path, required=True)
step_zero.add_argument("--study-manifest", type=Path, required=True)
step_zero.add_argument("--output", type=Path, required=True)
step_zero.add_argument("--seed", type=int, default=2026073001)
smoke = subparsers.add_parser("smoke-compare")
smoke.add_argument("--parent", type=Path, required=True)
smoke.add_argument("--wrapper", type=Path, required=True)
smoke.add_argument("--output", type=Path, required=True)
return parser.parse_args()
def canonical_sha256(value: Any) -> str:
return hashlib.sha256(
json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode()
).hexdigest()
def tensor_sha256(value: torch.Tensor) -> str:
return hashlib.sha256(runner.parent.tensor_bytes(value)).hexdigest()
def read_and_verify(path: Path) -> dict[str, Any]:
value = json.loads(path.read_text())
expected = value["canonical_sha256_without_self"]
payload = {
key: item
for key, item in value.items()
if key != "canonical_sha256_without_self"
}
if canonical_sha256(payload) != expected:
raise RuntimeError(f"canonical self-hash failed: {path}")
return value
def smoke_compare(args: argparse.Namespace) -> None:
parent_result = read_and_verify(args.parent)
wrapper_result = read_and_verify(args.wrapper)
fields = (
"architecture",
"depth",
"seed",
"steps",
"batch_size",
"target_bytes_seen",
"manifest",
"model",
"optimizer",
"hashes",
"evaluations",
"diagnostics",
"training_history",
"gradient_gate",
"environment",
)
checks = {}
for field in fields:
parent_value = exact_structure(parent_result[field])
wrapper_value = exact_structure(wrapper_result[field])
if field == "manifest":
parent_value.pop("path", None)
wrapper_value.pop("path", None)
checks[field] = parent_value == wrapper_value
wrapper_identity = {
"protocol": wrapper_result["protocol_id"] == runner.PROTOCOL_ID,
"parent_protocol": (
wrapper_result["parent_protocol_id"]
== runner.PARENT_PROTOCOL_ID
),
"variant": wrapper_result["variant"] == "learned_reference",
"forward_audit": wrapper_result["forward_intervention"]["passed"],
}
passed = all(checks.values()) and all(wrapper_identity.values())
result = {
"schema_version": 1,
"protocol_id": runner.PROTOCOL_ID,
"gate": "empty-selector-parent-equivalence",
"passed": passed,
"field_checks": checks,
"wrapper_identity": wrapper_identity,
"excluded_fields": [
"protocol wrapper fields",
"timing",
"self hash",
"parent runner self hash",
"study manifest",
],
"parent_file": str(args.parent.resolve()),
"wrapper_file": str(args.wrapper.resolve()),
}
result["canonical_sha256_without_self"] = canonical_sha256(result)
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(
json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
)
if not passed:
raise RuntimeError(f"empty-selector parent equivalence failed: {checks}")
def exact_structure(value: Any) -> Any:
return json.loads(
json.dumps(value, ensure_ascii=False, sort_keys=True)
)
def step_zero(args: argparse.Namespace) -> None:
if not torch.cuda.is_available():
raise RuntimeError("CUDA is required by the frozen step-zero gate")
study_manifest = json.loads(args.study_manifest.read_text())
if study_manifest["protocol_id"] != runner.PROTOCOL_ID:
raise RuntimeError("study manifest mismatch")
parent_manifest = json.loads(args.parent_manifest.read_text())
if parent_manifest["protocol_id"] != runner.PARENT_PROTOCOL_ID:
raise RuntimeError("parent manifest mismatch")
parent = runner.parent
parent.configure_round04_globals(32)
device = torch.device("cuda")
corpus = parent.round04.ByteCorpus(
args.cache_dir, parent_manifest, device
)
inputs, targets = corpus.fixed_batch(
corpus.diagnostic_starts, 0, 16
)
variants = ("learned_reference",) + tuple(
study_manifest["variants"].keys()
)
observations: dict[str, Any] = {}
reference_payload: dict[str, Any] | None = None
for variant in variants:
parent.configure_determinism(args.seed)
runner.ACTIVE_VARIANT = variant
model = runner.ForwardInterventionLanguageModel("block").to(device)
initial_public = parent.named_state_hash(
model, include_mixers=False
)
initial_mixer = parent.named_state_hash(
model, include_mixers=True
)
with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
logits, trace = model(inputs, capture=True)
loss = parent.cross_entropy(logits, targets)
if trace is None:
raise RuntimeError("step-zero trace missing")
evaluation = parent.evaluate(model, corpus, 64, 8)
diagnostic = parent.diagnostic(model, corpus, 16)
payload = {
"initial_public_hash": initial_public,
"initial_mixer_hash": initial_mixer,
"logits_sha256": tensor_sha256(logits),
"loss_nats": loss.detach().cpu().item(),
"loss_tensor_sha256": tensor_sha256(loss),
"evaluation": exact_structure(evaluation),
"diagnostic": exact_structure(diagnostic),
}
if reference_payload is None:
reference_payload = payload
exact_checks = {
key: payload[key] == reference_payload[key]
for key in payload
}
selected = tuple(runner.VARIANTS[variant])
capture_checks = {}
for index in selected:
summary = trace.depth_weights[index]
source_count = summary["sources"]
capture_checks[str(index)] = {
"source_count": source_count,
"expected_source_count": study_manifest[
"selected_source_counts"
][str(index)],
"capture_summary_exact_vs_learned": (
payload["diagnostic"]["depth_weights"][index]
== reference_payload["diagnostic"]["depth_weights"][index]
),
"passed": (
source_count
== study_manifest["selected_source_counts"][str(index)]
and payload["diagnostic"]["depth_weights"][index]
== reference_payload["diagnostic"]["depth_weights"][index]
),
}
runtime_uniform_gate = (
model.uniform_weight_max_abs_error
<= study_manifest["thresholds"][
"uniform_weight_max_abs_error"
]
)
observations[variant] = {
"payload": payload,
"exact_vs_learned_reference": exact_checks,
"selected_capture_checks": capture_checks,
"pre_reduction_uniform_weight_max_abs_error": (
model.uniform_weight_max_abs_error
),
"pre_reduction_uniform_weight_gate": runtime_uniform_gate,
"passed": (
all(exact_checks.values())
and all(
item["passed"] for item in capture_checks.values()
)
and runtime_uniform_gate
),
}
del model, logits, loss, trace
torch.cuda.empty_cache()
passed = all(item["passed"] for item in observations.values())
result = {
"schema_version": 1,
"protocol_id": runner.PROTOCOL_ID,
"gate": "step-zero-cross-variant-byte-exact",
"seed": args.seed,
"passed": passed,
"variants": observations,
"parent_manifest_sha256": runner.parent.file_sha256(
args.parent_manifest
),
"study_manifest_sha256": runner.parent.file_sha256(
args.study_manifest
),
"environment": {
"gpu": torch.cuda.get_device_name(0),
"torch": torch.__version__,
"cuda": torch.version.cuda,
"cublas_workspace_config": os.environ.get(
"CUBLAS_WORKSPACE_CONFIG"
),
},
}
result["canonical_sha256_without_self"] = canonical_sha256(result)
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(
json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
)
if not passed:
failed = [
name
for name, value in observations.items()
if not value["passed"]
]
raise RuntimeError(f"step-zero exactness failed: {failed}")
def main() -> None:
args = parse_args()
if args.command == "step-zero":
step_zero(args)
else:
smoke_compare(args)
if __name__ == "__main__":
main()
@@ -0,0 +1,50 @@
# Attention Residuals local mixer-path diagnostics
This directory implements preregistered protocol
`llm-atlas-k3-attnres-local-path-v1`.
It is a targeted follow-up to Round 06. It exact-replays the same depth-32
Block training and keeps the learned forward unchanged while switching source
value-gradient coefficients only at frozen mixer scopes. It is not a Kimi K3
checkpoint run, a trainable variant, an additive attribution, or a reproduction
of unpublished Figure 5 telemetry.
## Frozen environment
```text
Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
PyTorch 2.11.0+cu128
GPU NVIDIA GeForce RTX 5090
CUBLAS_WORKSPACE_CONFIG=:4096:8
```
The preregistration was committed as `6911efc` before the runner or any result
file existed.
## Step-0 smoke
```bash
CUBLAS_WORKSPACE_CONFIG=:4096:8 \
/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
experiments/k3/attnres_local_path/train.py \
--run-kind smoke \
--seed 2026073001 \
--cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
--data-manifest experiments/k3/attnres_gradient/manifest.json \
--parent-manifest experiments/k3/attnres_spike/manifest.json \
--manifest experiments/k3/attnres_local_path/manifest.json \
--output /home/wuyang/.cache/llm-atlas/k3-attnres-local-path-v1/smoke/seed-2026073001.json
```
The first smoke passed all 14-mode forward-identity, selector, Round 06 endpoint,
initialization-negative-control, parent-learned, and loss-scale gates. Its
canonical content hash is
`f708200f4fb122f61f30b97393839382a2094a71a8ea5d48cf47fb7fa094e69b`.
Formal cells use the same command with `--run-kind formal`, one of the three
manifest seeds, and a new output path. The independent seed-2026073001 run uses
`--run-kind replay`.
Only `analyze.py` may calculate the global log gap, sufficiency/restoration
scores, and preregistered gates. The site consumes its frozen aggregate rather
than reimplementing thresholds in TypeScript.
@@ -0,0 +1,498 @@
#!/usr/bin/env python3
"""Aggregate and gate preregistered Round 07 local-path results."""
from __future__ import annotations
import argparse
import copy
import hashlib
import json
import math
import statistics
from pathlib import Path
from typing import Any
PROTOCOL_ID = "llm-atlas-k3-attnres-local-path-v1"
METRICS = ("spike_contrast", "peak_normalized")
SUFFICIENCY_MODES = (
"uniform_group_6_only",
"uniform_group_7_only",
"uniform_groups_6_7_only",
"uniform_group_6_attention_only",
"uniform_group_6_mlp_only",
"uniform_group_7_attention_only",
"uniform_group_7_mlp_only",
"uniform_output_only",
"uniform_depth_all",
"uniform_all",
)
RESTORATION_MODES = (
"uniform_except_group_6",
"uniform_except_group_7",
"uniform_except_groups_6_7",
)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument("--manifest", type=Path, required=True)
parser.add_argument(
"--formal", type=Path, action="append", required=True
)
parser.add_argument("--replay", type=Path, required=True)
parser.add_argument("--output", type=Path, required=True)
return parser.parse_args()
def file_sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def canonical_sha256(value: Any) -> str:
payload = json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode()
return hashlib.sha256(payload).hexdigest()
def read_result(path: Path) -> dict[str, Any]:
value = json.loads(path.read_text())
if value["protocol_id"] != PROTOCOL_ID:
raise RuntimeError(f"protocol mismatch: {path}")
expected = value["canonical_sha256_without_self"]
without_self = {
key: item
for key, item in value.items()
if key != "canonical_sha256_without_self"
}
if canonical_sha256(without_self) != expected:
raise RuntimeError(f"canonical self-hash mismatch: {path}")
return value
def replay_payload(value: dict[str, Any]) -> dict[str, Any]:
cleaned = copy.deepcopy(value)
for key in ("run_kind", "timing", "canonical_sha256_without_self"):
cleaned.pop(key)
return cleaned
def primary_metrics(mode: dict[str, Any]) -> dict[str, float]:
stats = mode["positions"]["post_mlp_state"]["reductions"][
"element_rms"
]["statistics"]
return {name: float(stats[name]) for name in METRICS}
def all_true(values: list[bool]) -> bool:
return len(values) > 0 and all(values)
def mixed_status(
by_seed_metric: dict[str, dict[str, dict[str, Any]]],
threshold: float,
) -> dict[str, Any]:
passed = []
signs = []
metric_passes = {metric: [] for metric in METRICS}
for seed_values in by_seed_metric.values():
for metric in METRICS:
score = seed_values[metric]["score"]
cell_passed = score is not None and score >= threshold
passed.append(cell_passed)
metric_passes[metric].append(cell_passed)
if score is not None:
signs.append(1 if score >= 0 else -1)
reasons = []
if any(passed) and not all(passed):
reasons.append("seed_or_metric_pass_split")
if metric_passes[METRICS[0]] != metric_passes[METRICS[1]]:
reasons.append("metric_direction_split")
if len(set(signs)) > 1:
reasons.append("score_sign_split")
return {
"passed": all_true(passed),
"threshold": threshold,
"required_cells": len(passed),
"passed_cells": sum(passed),
"mixed": bool(reasons),
"mixed_reasons": reasons,
}
def main() -> None:
args = parse_args()
manifest = json.loads(args.manifest.read_text())
if manifest["protocol_id"] != PROTOCOL_ID:
raise RuntimeError("manifest protocol mismatch")
if len(args.formal) != 3:
raise RuntimeError("exactly three formal result paths are required")
formal_pairs = [(path, read_result(path)) for path in args.formal]
formal_pairs.sort(key=lambda item: item[1]["seed"])
expected_seeds = manifest["formal_seeds"]
if [value["seed"] for _, value in formal_pairs] != expected_seeds:
raise RuntimeError("formal seeds do not match manifest")
for path, value in formal_pairs:
if value["run_kind"] != "formal" or value["steps"] != 8000:
raise RuntimeError(f"invalid formal cell: {path}")
if not value["round06_equivalence"]["passed"]:
raise RuntimeError(f"parent equivalence failed: {path}")
for diagnostic in value["diagnostics"]:
if not diagnostic["parent_learned_round06_exact"]:
raise RuntimeError(f"parent diagnostic mismatch: {path}")
if diagnostic["local_matrix"] is not None:
for gate in ("forward_identity_gate", "endpoint_exactness"):
if not diagnostic[gate]["passed"]:
raise RuntimeError(f"{gate} failed: {path}")
for mode in manifest["matrix_modes"]:
if not diagnostic["local_matrix"][mode]["selector"][
"passed"
]:
raise RuntimeError(f"selector failed: {path}:{mode}")
step0 = value["diagnostics"][0]
if (
not step0["initialization_negative_control"]["passed"]
or not step0["loss_scale_gate"]["passed"]
):
raise RuntimeError(f"step-0 control failed: {path}")
replay = read_result(args.replay)
if (
replay["run_kind"] != "replay"
or replay["seed"] != expected_seeds[0]
or replay["steps"] != 8000
):
raise RuntimeError("invalid replay cell")
replay_exact = (
replay_payload(formal_pairs[0][1]) == replay_payload(replay)
)
if not replay_exact:
raise RuntimeError("formal seed1 and replay are not canonical exact")
thresholds = manifest["thresholds"]
cells = []
sufficiency_by_mode: dict[str, dict[str, dict[str, Any]]] = {
mode: {} for mode in SUFFICIENCY_MODES
}
restoration_by_mode: dict[str, dict[str, dict[str, Any]]] = {
mode: {} for mode in RESTORATION_MODES
}
for path, value in formal_pairs:
seed_key = str(value["seed"])
final = value["diagnostics"][-1]["local_matrix"]
mode_metrics = {
mode: primary_metrics(final[mode])
for mode in manifest["matrix_modes"]
}
global_metrics = {}
for metric in METRICS:
reference = mode_metrics["detached_learned"][metric]
uniform_all = mode_metrics["uniform_all"][metric]
log_gap = math.log(reference / uniform_all)
relative_drop = (reference - uniform_all) / reference
established = (
reference
> thresholds["positive_denominator_epsilon"]
and uniform_all
> thresholds["positive_denominator_epsilon"]
and log_gap > 0
and relative_drop
>= thresholds["global_relative_drop_minimum"]
)
global_metrics[metric] = {
"reference": reference,
"uniform_all": uniform_all,
"log_gap": log_gap,
"relative_drop": relative_drop,
"established": established,
}
for mode in SUFFICIENCY_MODES:
sufficiency_by_mode[mode][seed_key] = {}
for metric in METRICS:
established = global_metrics[metric]["established"]
score = (
math.log(
mode_metrics["detached_learned"][metric]
/ mode_metrics[mode][metric]
)
/ global_metrics[metric]["log_gap"]
if established
else None
)
sufficiency_by_mode[mode][seed_key][metric] = {
"score": score,
"metric_value": mode_metrics[mode][metric],
"global_gap_established": established,
}
for mode in RESTORATION_MODES:
restoration_by_mode[mode][seed_key] = {}
for metric in METRICS:
established = global_metrics[metric]["established"]
score = (
math.log(
mode_metrics[mode][metric]
/ mode_metrics["uniform_all"][metric]
)
/ global_metrics[metric]["log_gap"]
if established
else None
)
restoration_by_mode[mode][seed_key][metric] = {
"score": score,
"metric_value": mode_metrics[mode][metric],
"global_gap_established": established,
}
cells.append(
{
"seed": value["seed"],
"path": str(path),
"file_sha256": file_sha256(path),
"canonical_sha256": value[
"canonical_sha256_without_self"
],
"mode_metrics": mode_metrics,
"global": global_metrics,
"interaction_residual": {
metric: (
1.0
- sufficiency_by_mode["uniform_output_only"][
seed_key
][metric]["score"]
- sufficiency_by_mode["uniform_depth_all"][
seed_key
][metric]["score"]
)
for metric in METRICS
},
}
)
global_gap_passed = all_true(
[
cell["global"][metric]["established"]
for cell in cells
for metric in METRICS
]
)
sufficiency_gates = {
"groups_6_7": mixed_status(
sufficiency_by_mode["uniform_groups_6_7_only"],
thresholds["groups_6_7_sufficiency_minimum"],
),
"group_6": mixed_status(
sufficiency_by_mode["uniform_group_6_only"],
thresholds["single_group_material_minimum"],
),
"group_7": mixed_status(
sufficiency_by_mode["uniform_group_7_only"],
thresholds["single_group_material_minimum"],
),
"output_half_gap": mixed_status(
sufficiency_by_mode["uniform_output_only"],
thresholds["output_half_gap_minimum"],
),
}
restoration_gates = {
"groups_6_7": mixed_status(
restoration_by_mode["uniform_except_groups_6_7"],
thresholds["groups_6_7_restoration_minimum"],
),
"group_6": mixed_status(
restoration_by_mode["uniform_except_group_6"],
thresholds["single_group_material_minimum"],
),
"group_7": mixed_status(
restoration_by_mode["uniform_except_group_7"],
thresholds["single_group_material_minimum"],
),
}
branch_gates = {}
for group in (6, 7):
group_passed = sufficiency_gates[f"group_{group}"]["passed"]
candidates = {}
for branch, sibling in (("attention", "mlp"), ("mlp", "attention")):
branch_mode = f"uniform_group_{group}_{branch}_only"
sibling_mode = f"uniform_group_{group}_{sibling}_only"
checks = []
margins = []
for seed in expected_seeds:
seed_key = str(seed)
for metric in METRICS:
left = sufficiency_by_mode[branch_mode][seed_key][
metric
]["score"]
right = sufficiency_by_mode[sibling_mode][seed_key][
metric
]["score"]
margin = (
left - right
if left is not None and right is not None
else None
)
margins.append(margin)
checks.append(
left is not None
and left >= thresholds["branch_material_minimum"]
and margin is not None
and margin
>= thresholds["branch_dominance_margin"]
)
candidates[branch] = {
"passed": group_passed and all_true(checks),
"group_gate_passed": group_passed,
"passed_cells": sum(checks),
"required_cells": len(checks),
"margins": margins,
}
dominant = [
branch
for branch, gate in candidates.items()
if gate["passed"]
]
branch_gates[f"group_{group}"] = {
"passed": len(dominant) == 1,
"dominant_branch": dominant[0] if len(dominant) == 1 else None,
"exploratory_sufficiency_only": True,
"candidates": candidates,
}
localization_passed = (
global_gap_passed
and sufficiency_gates["groups_6_7"]["passed"]
and restoration_gates["groups_6_7"]["passed"]
)
localization_status = (
"established_at_preregistered_bidirectional_50pct_threshold"
if localization_passed
else (
"one_sided_evidence_localization_not_established"
if (
sufficiency_gates["groups_6_7"]["passed"]
!= restoration_gates["groups_6_7"]["passed"]
)
else "not_established_at_preregistered_threshold"
)
)
mode_means = {}
for mode in manifest["matrix_modes"]:
mode_means[mode] = {
metric: statistics.fmean(
cell["mode_metrics"][mode][metric] for cell in cells
)
for metric in METRICS
}
sufficiency_means = {
mode: {
metric: statistics.fmean(
sufficiency_by_mode[mode][str(seed)][metric]["score"]
for seed in expected_seeds
)
for metric in METRICS
}
for mode in SUFFICIENCY_MODES
}
restoration_means = {
mode: {
metric: statistics.fmean(
restoration_by_mode[mode][str(seed)][metric]["score"]
for seed in expected_seeds
)
for metric in METRICS
}
for mode in RESTORATION_MODES
}
result = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"study_identity": manifest["study_identity"],
"manifest": {
"path": str(args.manifest),
"file_sha256": file_sha256(args.manifest),
},
"formal_cells": cells,
"replay": {
"path": str(args.replay),
"file_sha256": file_sha256(args.replay),
"canonical_sha256": replay[
"canonical_sha256_without_self"
],
"formal_seed1_exact_excluding_run_kind_and_timing": replay_exact,
},
"scores": {
"sufficiency": sufficiency_by_mode,
"restoration": restoration_by_mode,
},
"means": {
"mode_metrics": mode_means,
"sufficiency": sufficiency_means,
"restoration": restoration_means,
},
"gates": {
"all_input_and_parent_gates_passed": True,
"global_gap": {
"passed": global_gap_passed,
"required_cells": 6,
"passed_cells": sum(
cell["global"][metric]["established"]
for cell in cells
for metric in METRICS
),
},
"sufficiency": sufficiency_gates,
"restoration": restoration_gates,
"localization": {
"passed": localization_passed,
"status": localization_status,
"requires": (
"groups 6+7 sufficiency and restoration both >=0.50 "
"for 3/3 seeds and both metrics"
),
},
"branch_dominance": branch_gates,
},
"limitations": [
"same-forward diagnostic backward-rule sensitivity only",
"reduced byte-level language model, not the Kimi K3 checkpoint",
"effects are non-additive and are not contribution percentages",
"group 7 includes layers 26-28 outside fixed spike set 21-25",
"three-seed threshold gates are not population inference",
],
}
result["canonical_sha256_without_self"] = canonical_sha256(result)
args.output.parent.mkdir(parents=True, exist_ok=True)
temporary = args.output.with_suffix(args.output.suffix + ".tmp")
temporary.write_text(
json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True)
+ "\n"
)
temporary.replace(args.output)
print(
json.dumps(
{
"output": str(args.output),
"formal_cells": len(cells),
"replay_exact": replay_exact,
"global_gap": global_gap_passed,
"localization": localization_status,
"canonical_sha256": result[
"canonical_sha256_without_self"
],
},
ensure_ascii=False,
indent=2,
)
)
if __name__ == "__main__":
main()
@@ -0,0 +1,540 @@
{
"schema_version": 1,
"protocol_id": "llm-atlas-k3-attnres-local-path-v1",
"parent_protocol_id": "llm-atlas-k3-attnres-spike-path-v1",
"study_identity": "targeted local-path follow-up informed by Round 05 and Round 06; not blind discovery",
"architecture": "block",
"depth": 32,
"aggregation_groups": 8,
"blocks_per_group": 4,
"depth_mixers": 64,
"output_mixers": 1,
"formal_seeds": [
2026073001,
2026073002,
2026073003
],
"replay": {
"architecture": "block",
"depth": 32,
"seed": 2026073001,
"environment_scope": "same host, GPU, Python, PyTorch, CUDA and CUBLAS_WORKSPACE_CONFIG"
},
"training": {
"steps": 8000,
"batch_size": 32,
"context": 256,
"target_bytes_per_cell": 65536000,
"parent_diagnostic_steps": [
0,
100,
500,
2000,
4000,
8000
],
"local_matrix_steps": [
0,
8000
],
"matrix_used_during_training": false
},
"primary_object": {
"position": "post_mlp_state",
"reduction": "element_rms",
"spike_layers_one_based": [
21,
22,
23,
24,
25
],
"metrics": [
"spike_contrast",
"peak_normalized"
]
},
"matrix_modes": [
"detached_learned",
"uniform_group_6_only",
"uniform_group_7_only",
"uniform_groups_6_7_only",
"uniform_group_6_attention_only",
"uniform_group_6_mlp_only",
"uniform_group_7_attention_only",
"uniform_group_7_mlp_only",
"uniform_output_only",
"uniform_depth_all",
"uniform_all",
"uniform_except_group_6",
"uniform_except_group_7",
"uniform_except_groups_6_7"
],
"selector": {
"coefficient_choices": [
"detached_learned",
"uniform"
],
"full_autograd_learned_in_matrix": false,
"depth_identity": "kind=depth,index=0..63; layer=floor(index/2)+1; attention iff index even",
"output_identity": "kind=output,index=64; parallel-schema alias only; parent mixer_index remains null",
"uniform_depth_indices": {
"detached_learned": [],
"uniform_group_6_only": [
40,
41,
42,
43,
44,
45,
46,
47
],
"uniform_group_7_only": [
48,
49,
50,
51,
52,
53,
54,
55
],
"uniform_groups_6_7_only": [
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55
],
"uniform_group_6_attention_only": [
40,
42,
44,
46
],
"uniform_group_6_mlp_only": [
41,
43,
45,
47
],
"uniform_group_7_attention_only": [
48,
50,
52,
54
],
"uniform_group_7_mlp_only": [
49,
51,
53,
55
],
"uniform_output_only": [],
"uniform_depth_all": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63
],
"uniform_all": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63
],
"uniform_except_group_6": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63
],
"uniform_except_group_7": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
56,
57,
58,
59,
60,
61,
62,
63
],
"uniform_except_groups_6_7": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
56,
57,
58,
59,
60,
61,
62,
63
]
},
"uniform_output": {
"detached_learned": false,
"uniform_group_6_only": false,
"uniform_group_7_only": false,
"uniform_groups_6_7_only": false,
"uniform_group_6_attention_only": false,
"uniform_group_6_mlp_only": false,
"uniform_group_7_attention_only": false,
"uniform_group_7_mlp_only": false,
"uniform_output_only": true,
"uniform_depth_all": false,
"uniform_all": true,
"uniform_except_group_6": true,
"uniform_except_group_7": true,
"uniform_except_groups_6_7": true
},
"expected_uniform_counts": {
"detached_learned": 0,
"uniform_group_6_only": 8,
"uniform_group_7_only": 8,
"uniform_groups_6_7_only": 16,
"uniform_group_6_attention_only": 4,
"uniform_group_6_mlp_only": 4,
"uniform_group_7_attention_only": 4,
"uniform_group_7_mlp_only": 4,
"uniform_output_only": 1,
"uniform_depth_all": 64,
"uniform_all": 65,
"uniform_except_group_6": 57,
"uniform_except_group_7": 57,
"uniform_except_groups_6_7": 49
}
},
"thresholds": {
"positive_denominator_epsilon": 1e-30,
"step0_spectrum_tolerance": 1e-6,
"loss_scale_tolerance": 1e-5,
"global_relative_drop_minimum": 0.2,
"groups_6_7_sufficiency_minimum": 0.5,
"groups_6_7_restoration_minimum": 0.5,
"single_group_material_minimum": 0.2,
"branch_material_minimum": 0.2,
"branch_dominance_margin": 0.15,
"output_half_gap_minimum": 0.5,
"formal_seed_gate": "3/3 independently for both metrics; means are display-only"
},
"formulas": {
"global_log_gap": "G_X = ln(X_ref / X_uniform_all)",
"global_relative_drop": "(X_ref - X_uniform_all) / X_ref",
"sufficiency": "S_X(m) = ln(X_ref / X_m) / G_X",
"restoration": "R_X(r) = ln(X_r / X_uniform_all) / G_X",
"score_clipping": false
},
"parent_artifacts": {
"manifest_path": "experiments/k3/attnres_spike/manifest.json",
"manifest_sha256": "d5302a249249a07d362819134763d14e7d32307f22cff416c665ed9606142fef",
"runner_path": "experiments/k3/attnres_spike/train.py",
"runner_sha256": "77298081d3c491d2e88e4705995174b9879ef377f520eb5fe5ea107e7a1da084",
"protocol_path": "research/K3_ATTNRES_SPIKE_PROTOCOL.md",
"protocol_sha256": "6cb101b8760d9f1c81caeb2f16880b16152da103867224a06761a75a12984a16",
"scoping_path": "research/K3_ATTNRES_SPIKE_SCOPING.md",
"scoping_sha256": "590166bd62580bb8238293823cfcc39bc0a465fec4c697025343f3f1138abd27",
"formal_schedule_sha256": "5041e09b167f229248d2462324e8c254b8f5938975f135dcd8192b00a54a4f4e",
"validation_tensor_sha256": "f459316f13078a163b47c133511bb7181e05170ab89516e196490113893ce338",
"diagnostic_tensor_sha256": "21117e31db302b10d67b63f035665dc8f220b879d216ccd12b7d2ba86e7b1716"
},
"current_artifacts": {
"protocol_path": "research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md",
"protocol_sha256": "5ecc7ca92314ddb50aecf0cb50e115814c8983aa8bffb30e3634f7b3ce6dca1d",
"scoping_path": "research/K3_ATTNRES_LOCAL_PATH_SCOPING.md",
"scoping_sha256": "670ca4edf31a4be1f54937d9c7a760dba7a96e1e820c38c6b10405e22b078fc8",
"grok_review_path": "research/K3_ATTNRES_LOCAL_PATH_GROK_REVIEW.md",
"grok_review_sha256": "2da1b6bf1f455c4121a7a2c5cfe40e102327dabafc7e24dccc23ed0d00ac6d71",
"grok_session": "019fb151-9627-76c1-b7d7-53012874f85c"
},
"round06_expected": {
"2026073001": {
"raw_file_sha256": "e39e93b7a7fce3c56f5f14f95cfdc04afdce53628affee1202fe62bd1bdb7f71",
"canonical_sha256": "76b0ccfb55c38baef50c395788ac4b351cbe0d58de70064b50702acb5c93f515",
"final_model_state": "3f0b97ece3a15571ba3d656f589f512ca0bb9e20083c9f58a42ccaee14892f59",
"final_optimizer_state": "ed03e6fbd4a12d8b063dcb22e0437754285f54d585374cd52fbd534f05d24637"
},
"2026073002": {
"raw_file_sha256": "1c6f6c731030ec0adb2a8e7a4d586e0c4005cc3319568a7ac83c08c2a4b8eaf8",
"canonical_sha256": "5352c74eca853b375c0e85933dafd7c5916c39fc59052e742ca14ffd6d68bc78",
"final_model_state": "bd2556388aeaa211b798c283c7cbd8ccd29edf166a2922fa13d172e8dfdc38d1",
"final_optimizer_state": "0b101eab3bc7d8d654be2ea335c86fc25563ce19912d721844ee4e639c569e77"
},
"2026073003": {
"raw_file_sha256": "115f8245577ece6dfaaa8ada68445c186e6523a7f3b26efcc3eb4c0c4ce82406",
"canonical_sha256": "7e764c07e90b78c4cd0acc2e99600225f16428cbb25d5188a0a5a8fe797f8766",
"final_model_state": "638568aede21890773b6932a19ec4e112f5ac0a4770ba3402fcd82980a9ecf76",
"final_optimizer_state": "83947fd743ec8e3e31ca7788fd201981846f0f1c7e9e38c88afcef95cfc6ec4e"
}
}
}
@@ -0,0 +1,274 @@
#!/usr/bin/env python3
"""Package frozen Round 07 outputs without recomputing any result gate."""
from __future__ import annotations
import argparse
import copy
import hashlib
import json
from pathlib import Path
from typing import Any
PROTOCOL_ID = "llm-atlas-k3-attnres-local-path-v1"
SEEDS = (2026073001, 2026073002, 2026073003)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument("--raw-dir", type=Path, required=True)
parser.add_argument("--manifest", type=Path, required=True)
parser.add_argument("--aggregate", type=Path, required=True)
parser.add_argument("--reproduction-output", type=Path, required=True)
parser.add_argument("--compact-output", type=Path, required=True)
return parser.parse_args()
def file_sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def canonical_sha256(value: Any) -> str:
payload = json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode()
return hashlib.sha256(payload).hexdigest()
def load_canonical(path: Path) -> dict[str, Any]:
value = json.loads(path.read_text())
expected = value["canonical_sha256_without_self"]
payload = {
key: item
for key, item in value.items()
if key != "canonical_sha256_without_self"
}
if canonical_sha256(payload) != expected:
raise RuntimeError(f"canonical hash mismatch: {path}")
return value
def write_canonical(path: Path, value: dict[str, Any]) -> None:
value["canonical_sha256_without_self"] = canonical_sha256(value)
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_suffix(path.suffix + ".tmp")
temporary.write_text(
json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True)
+ "\n"
)
temporary.replace(path)
def replay_payload(value: dict[str, Any]) -> dict[str, Any]:
cleaned = copy.deepcopy(value)
for key in ("run_kind", "timing", "canonical_sha256_without_self"):
cleaned.pop(key)
return cleaned
def artifact_hashes(repo_root: Path) -> dict[str, str]:
paths = {
"runner": "experiments/k3/attnres_local_path/train.py",
"analyzer": "experiments/k3/attnres_local_path/analyze.py",
"packager": "experiments/k3/attnres_local_path/package.py",
"manifest": "experiments/k3/attnres_local_path/manifest.json",
"protocol": "research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md",
"scoping": "research/K3_ATTNRES_LOCAL_PATH_SCOPING.md",
"preresult_grok_review": (
"research/K3_ATTNRES_LOCAL_PATH_GROK_REVIEW.md"
),
}
return {
name: file_sha256(repo_root / path) for name, path in paths.items()
}
def main() -> None:
args = parse_args()
repo_root = Path(__file__).resolve().parents[3]
manifest = json.loads(args.manifest.read_text())
aggregate = load_canonical(args.aggregate)
if (
manifest["protocol_id"] != PROTOCOL_ID
or aggregate["protocol_id"] != PROTOCOL_ID
):
raise RuntimeError("protocol mismatch")
formal = {}
raw_files = {}
for seed in SEEDS:
name = f"formal-seed-{seed}.json"
path = args.raw_dir / name
run = load_canonical(path)
if (
run["run_kind"] != "formal"
or run["seed"] != seed
or not run["round06_equivalence"]["passed"]
):
raise RuntimeError(f"invalid formal run: {name}")
formal[seed] = run
raw_files[name] = {
"file_sha256": file_sha256(path),
"canonical_sha256": run["canonical_sha256_without_self"],
}
replay_name = f"replay-seed-{SEEDS[0]}.json"
replay_path = args.raw_dir / replay_name
replay = load_canonical(replay_path)
if replay["run_kind"] != "replay" or replay["seed"] != SEEDS[0]:
raise RuntimeError("invalid replay")
raw_files[replay_name] = {
"file_sha256": file_sha256(replay_path),
"canonical_sha256": replay["canonical_sha256_without_self"],
}
compare_payload = replay_payload(formal[SEEDS[0]])
replay_exact = compare_payload == replay_payload(replay)
if not replay_exact or not aggregate["replay"][
"formal_seed1_exact_excluding_run_kind_and_timing"
]:
raise RuntimeError("replay exactness failed")
replay_gate = {
"passed": True,
"excluded_fields": [
"run_kind",
"timing",
"canonical_sha256_without_self",
],
"frozen_compare_sha256": canonical_sha256(compare_payload),
}
reproduction = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"raw_files": raw_files,
"replay_gate": replay_gate,
"artifacts": artifact_hashes(repo_root),
"aggregate": {
"file_sha256": file_sha256(args.aggregate),
"canonical_sha256": aggregate[
"canonical_sha256_without_self"
],
},
"post_result_grok_review": {
"session": "019fb19d-94a3-7231-9a63-3a1ef33a9892",
"role": "read-only adversarial implementation audit; not an evidence source",
"blocking_errors": 0,
"localization_status_confirmed": True,
},
}
write_canonical(args.reproduction_output, reproduction)
final_spectra = []
for seed in SEEDS:
final = formal[seed]["diagnostics"][-1]["local_matrix"]
final_spectra.append(
{
"seed": seed,
"modes": {
mode: {
"normalized": final[mode]["positions"][
"post_mlp_state"
]["reductions"]["element_rms"]["statistics"][
"normalized"
],
"spike_contrast": final[mode]["positions"][
"post_mlp_state"
]["reductions"]["element_rms"]["statistics"][
"spike_contrast"
],
"peak_normalized": final[mode]["positions"][
"post_mlp_state"
]["reductions"]["element_rms"]["statistics"][
"peak_normalized"
],
"peak_layer": final[mode]["positions"][
"post_mlp_state"
]["reductions"]["element_rms"]["statistics"][
"peak_layer"
],
"uniform_count": final[mode]["selector"][
"uniform_count"
],
}
for mode in manifest["matrix_modes"]
},
}
)
compact = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"study": {
"identity": manifest["study_identity"],
"seeds": list(SEEDS),
"steps": manifest["training"]["steps"],
"formal_target_bytes": (
len(SEEDS)
* manifest["training"]["target_bytes_per_cell"]
),
"total_target_bytes_with_replay": (
(len(SEEDS) + 1)
* manifest["training"]["target_bytes_per_cell"]
),
"modes": manifest["matrix_modes"],
"spike_layers": manifest["primary_object"][
"spike_layers_one_based"
],
"metrics": manifest["primary_object"]["metrics"],
},
"thresholds": manifest["thresholds"],
"formulas": manifest["formulas"],
"formal_cells": aggregate["formal_cells"],
"scores": aggregate["scores"],
"means": aggregate["means"],
"gates": aggregate["gates"],
"replay": {
**aggregate["replay"],
"frozen_compare_sha256": replay_gate[
"frozen_compare_sha256"
],
},
"final_spectra": final_spectra,
"limitations": aggregate["limitations"],
"hashes": {
"aggregate_file_sha256": file_sha256(args.aggregate),
"aggregate_canonical_sha256": aggregate[
"canonical_sha256_without_self"
],
"reproduction_file_sha256": file_sha256(
args.reproduction_output
),
"reproduction_canonical_sha256": reproduction[
"canonical_sha256_without_self"
],
"manifest_file_sha256": file_sha256(args.manifest),
},
}
write_canonical(args.compact_output, compact)
print(
json.dumps(
{
"reproduction": str(args.reproduction_output),
"compact": str(args.compact_output),
"raw_files": len(raw_files),
"replay_exact": replay_exact,
"localization": aggregate["gates"]["localization"][
"status"
],
"compact_canonical_sha256": compact[
"canonical_sha256_without_self"
],
},
ensure_ascii=False,
indent=2,
)
)
if __name__ == "__main__":
main()
@@ -0,0 +1,51 @@
{
"aggregate": {
"canonical_sha256": "b86d119cd2f106e2cbee8a35760ed3244336a2fcfeb9178ea1e7dab13fc6f215",
"file_sha256": "bb0ec9fce5b30d50ad5c50c4b95af7a892d614f205a2c20cfc2662125e10160e"
},
"artifacts": {
"analyzer": "e0921562463e43d1ba6d47e4d23015107eb2df23579921088550b69afd47d02b",
"manifest": "db01e92ef2cf0896212fcd529429bd94a344de0e1195db7f87b9a56dc3449139",
"packager": "6a9ada0bc35b40475f45d7aca82877a93aeca667ed117c8e4417d924416c9196",
"preresult_grok_review": "2da1b6bf1f455c4121a7a2c5cfe40e102327dabafc7e24dccc23ed0d00ac6d71",
"protocol": "5ecc7ca92314ddb50aecf0cb50e115814c8983aa8bffb30e3634f7b3ce6dca1d",
"runner": "b42879e242a2f2d54aa6a87a718aeac4cf4509eae42da2b14403656675a8b03d",
"scoping": "670ca4edf31a4be1f54937d9c7a760dba7a96e1e820c38c6b10405e22b078fc8"
},
"canonical_sha256_without_self": "6f5d98fce6446fecc966dd2675f272f2c4f0c9a39a5741fabc4ffad6852ca7f4",
"post_result_grok_review": {
"blocking_errors": 0,
"localization_status_confirmed": true,
"role": "read-only adversarial implementation audit; not an evidence source",
"session": "019fb19d-94a3-7231-9a63-3a1ef33a9892"
},
"protocol_id": "llm-atlas-k3-attnres-local-path-v1",
"raw_files": {
"formal-seed-2026073001.json": {
"canonical_sha256": "f0a44f119836ed632c15880c3c2bb225173c0a05c50ea07abbe0e464ff407592",
"file_sha256": "73d46ae443d3e5ae3fe839c1656cda758f5f41aaee5f220c971c3c39b8a8cc3f"
},
"formal-seed-2026073002.json": {
"canonical_sha256": "b4629672b7d3b88a6be5525d2839e63e34fc9e4603ba1e7b98e957558da6da05",
"file_sha256": "bca4674c746e35a035acde7d2094a9c3bd59a988b052cd3feb30eb66eb0ca60a"
},
"formal-seed-2026073003.json": {
"canonical_sha256": "190b3deb06ae06caba287fce047b55cee613af6f1ebeb1661fcb53dd245abab0",
"file_sha256": "712f349e7715fee71f8e4678dcde0619d01b8d6c3b5c34825c88b9af0bf2abbb"
},
"replay-seed-2026073001.json": {
"canonical_sha256": "378df53ed9c89b2a4e0f3045b4c1e72754a7108d87fa9436e1db7aefa442e5eb",
"file_sha256": "872aabd9285ac346b4016c23de10769ab83c4dc29e62d8bc8b3156af98228d4e"
}
},
"replay_gate": {
"excluded_fields": [
"run_kind",
"timing",
"canonical_sha256_without_self"
],
"frozen_compare_sha256": "7dbd15ad03fbd357c5d91e159706d63b24703722f76c492ed1dc733535d6b9cf",
"passed": true
},
"schema_version": 1
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+933
View File
@@ -0,0 +1,933 @@
#!/usr/bin/env python3
"""Exact-replay Round 06 with preregistered local mixer-path interventions."""
from __future__ import annotations
import argparse
import importlib.util
import json
import math
import os
import platform
import statistics
import sys
import time
from pathlib import Path
from typing import Any
import torch
import torch.nn as nn
PROTOCOL_ID = "llm-atlas-k3-attnres-local-path-v1"
PARENT_PROTOCOL_ID = "llm-atlas-k3-attnres-spike-path-v1"
DATA_PROTOCOL_ID = "llm-atlas-k3-attnres-gradient-scale-v1"
DEPTH = 32
SEEDS = (2026073001, 2026073002, 2026073003)
FORMAL_STEPS = 8000
PARENT_DIAGNOSTIC_STEPS = (0, 100, 500, 2000, 4000, 8000)
MATRIX_STEPS = (0, 8000)
MATRIX_MODES = (
"detached_learned",
"uniform_group_6_only",
"uniform_group_7_only",
"uniform_groups_6_7_only",
"uniform_group_6_attention_only",
"uniform_group_6_mlp_only",
"uniform_group_7_attention_only",
"uniform_group_7_mlp_only",
"uniform_output_only",
"uniform_depth_all",
"uniform_all",
"uniform_except_group_6",
"uniform_except_group_7",
"uniform_except_groups_6_7",
)
TRAIN_BATCH_SIZE = 32
VALIDATION_WINDOWS = 64
EVAL_BATCH_SIZE = 8
TIMING_WARMUP = 20
def load_parent() -> Any:
path = Path(__file__).resolve().parents[1] / "attnres_spike" / "train.py"
spec = importlib.util.spec_from_file_location(
"k3_attnres_spike_parent", path
)
if spec is None or spec.loader is None:
raise RuntimeError(f"cannot import parent runner from {path}")
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
parent = load_parent()
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument(
"--run-kind", choices=("smoke", "formal", "replay"), required=True
)
parser.add_argument("--seed", type=int, required=True)
parser.add_argument("--cache-dir", type=Path, required=True)
parser.add_argument("--data-manifest", type=Path, required=True)
parser.add_argument("--parent-manifest", type=Path, required=True)
parser.add_argument("--manifest", type=Path, required=True)
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
if args.seed not in SEEDS:
parser.error(f"seed must be one of {SEEDS}")
if args.run_kind == "replay" and args.seed != SEEDS[0]:
parser.error(f"replay seed must be {SEEDS[0]}")
return args
class LocalPathLanguageModel(parent.SpikeLanguageModel):
"""Parent model with a parallel, audited per-mixer coefficient selector."""
def __init__(self, architecture: str, manifest: dict[str, Any]):
super().__init__(architecture)
selector = manifest["selector"]
self.uniform_depth = {
mode: frozenset(indices)
for mode, indices in selector["uniform_depth_indices"].items()
}
self.uniform_output = selector["uniform_output"]
self.expected_uniform_counts = selector["expected_uniform_counts"]
self._active_selector_visits: list[dict[str, Any]] | None = None
self.last_selector_visits: list[dict[str, Any]] | None = None
def forward(
self,
input_ids: torch.Tensor,
capture: bool = False,
mixer_backward_mode: str = "learned",
) -> tuple[torch.Tensor, parent.SpikeTrace | None]:
if not capture:
if mixer_backward_mode != "learned":
raise RuntimeError("training/evaluation cannot use an intervention")
return super().forward(
input_ids, capture=False, mixer_backward_mode="learned"
)
if mixer_backward_mode == "learned":
self.last_selector_visits = None
return super().forward(
input_ids, capture=True, mixer_backward_mode="learned"
)
if mixer_backward_mode not in MATRIX_MODES:
raise ValueError(f"unknown local matrix mode: {mixer_backward_mode}")
self._active_selector_visits = []
logits, trace = self._diagnostic_forward(
input_ids, mixer_backward_mode
)
self.last_selector_visits = self._active_selector_visits
self._active_selector_visits = None
return logits, trace
def _mix(
self,
mixer: nn.Module,
sources: list[torch.Tensor],
labels: list[str],
*,
mode: str,
mixer_index: int | None,
layer: int | None,
branch: str,
group: int | None,
offset: int | None,
) -> tuple[torch.Tensor, dict[str, Any]]:
if mode == "learned":
return super()._mix(
mixer,
sources,
labels,
mode=mode,
mixer_index=mixer_index,
layer=layer,
branch=branch,
group=group,
offset=offset,
)
if self._active_selector_visits is None:
raise RuntimeError("local selector visit log is not active")
if mixer_index is None:
if (
layer is not None
or group is not None
or branch != "output"
or offset is not None
):
raise RuntimeError("invalid output mixer identity")
identity = {
"kind": "output",
"index": 64,
"layer": None,
"group": None,
"branch": "output",
"offset": None,
}
use_uniform = bool(self.uniform_output[mode])
else:
expected_layer = mixer_index // 2 + 1
expected_branch = "attention" if mixer_index % 2 == 0 else "mlp"
expected_group = (expected_layer - 1) // 4 + 1
expected_offset = (expected_layer - 1) % 4 + 1
if (
not 0 <= mixer_index < 64
or layer != expected_layer
or branch != expected_branch
or group != expected_group
or offset != expected_offset
):
raise RuntimeError("invalid depth mixer identity")
identity = {
"kind": "depth",
"index": mixer_index,
"layer": layer,
"group": group,
"branch": branch,
"offset": offset,
}
use_uniform = mixer_index in self.uniform_depth[mode]
parent_output, _ = mixer(sources, False)
weights = parent.recompute_weights(mixer, sources)
summary = parent.weight_summary(
weights,
labels,
mixer_index=mixer_index,
layer=layer,
branch=branch,
group=group,
offset=offset,
)
backward_weights = (
torch.full_like(weights, 1.0 / len(sources))
if use_uniform
else weights
)
values = torch.stack(sources, dim=0)
routed = parent.RoutedSourceBackward.apply(
values, parent_output, backward_weights
)
self._active_selector_visits.append(
{
**identity,
"sources": len(sources),
"use_uniform": use_uniform,
"coefficient": (
"uniform" if use_uniform else "detached_learned"
),
}
)
return routed, summary
def expected_visit_identities() -> list[dict[str, Any]]:
result = []
for index in range(64):
layer = index // 2 + 1
result.append(
{
"kind": "depth",
"index": index,
"layer": layer,
"group": (layer - 1) // 4 + 1,
"branch": "attention" if index % 2 == 0 else "mlp",
"offset": (layer - 1) % 4 + 1,
}
)
result.append(
{
"kind": "output",
"index": 64,
"layer": None,
"group": None,
"branch": "output",
"offset": None,
}
)
return result
def validate_selector_visits(
mode: str,
visits: list[dict[str, Any]] | None,
manifest: dict[str, Any],
) -> dict[str, Any]:
if visits is None or len(visits) != 65:
raise RuntimeError("selector visit count mismatch")
identity_keys = ("kind", "index", "layer", "group", "branch", "offset")
actual_identities = [
{key: visit[key] for key in identity_keys} for visit in visits
]
expected_identities = expected_visit_identities()
if actual_identities != expected_identities:
raise RuntimeError("selector identity order mismatch")
if len({(item["kind"], item["index"]) for item in visits}) != 65:
raise RuntimeError("selector identities are not unique")
expected_depth = set(
manifest["selector"]["uniform_depth_indices"][mode]
)
expected_output = manifest["selector"]["uniform_output"][mode]
actual_depth = {
item["index"]
for item in visits
if item["kind"] == "depth" and item["use_uniform"]
}
actual_output = visits[-1]["use_uniform"]
if actual_depth != expected_depth or actual_output != expected_output:
raise RuntimeError("selector exact-set mismatch")
uniform_count = sum(int(item["use_uniform"]) for item in visits)
expected_count = manifest["selector"]["expected_uniform_counts"][mode]
if uniform_count != expected_count:
raise RuntimeError("selector uniform census mismatch")
return {
"passed": True,
"visit_count": len(visits),
"identities_unique": True,
"identity_order_sha256": parent.canonical_sha256(
actual_identities
),
"uniform_indices": [
item["index"] for item in visits if item["use_uniform"]
],
"uniform_count": uniform_count,
"expected_uniform_count": expected_count,
"visits_sha256": parent.canonical_sha256(visits),
"visits": visits,
}
def run_local_diagnostic(
model: LocalPathLanguageModel,
corpus: Any,
optimizer: torch.optim.Optimizer,
manifest: dict[str, Any],
*,
mode: str,
loss_scale: float = 1.0,
) -> dict[str, Any]:
result = parent.run_diagnostic(
model, corpus, optimizer, mode=mode, loss_scale=loss_scale
)
result["selector"] = validate_selector_visits(
mode, model.last_selector_visits, manifest
)
return result
def forward_identity_gate(matrix: dict[str, Any]) -> dict[str, Any]:
reference = matrix["detached_learned"]["forward"]
comparisons = {}
for mode in MATRIX_MODES[1:]:
other = matrix[mode]["forward"]
comparisons[mode] = {
"logits_exact": (
other["logits_sha256"] == reference["logits_sha256"]
),
"loss_exact": other["loss_nats"] == reference["loss_nats"],
"activations_exact": (
other["activation_sha256"]
== reference["activation_sha256"]
),
"mixer_summaries_exact": (
other["mixer_summary_sha256"]
== reference["mixer_summary_sha256"]
),
}
if not all(all(checks.values()) for checks in comparisons.values()):
raise RuntimeError("local matrix forward identity failed")
return {"passed": True, "comparisons": comparisons}
def spectrum_agreement(
left: dict[str, Any], right: dict[str, Any]
) -> dict[str, Any]:
checks = {}
passed = True
for position in parent.POSITIONS:
left_metric = left["positions"][position]["reductions"][
"element_rms"
]
right_metric = right["positions"][position]["reductions"][
"element_rms"
]
raw_errors = [
abs(a - b) / a
for a, b in zip(
left_metric["values"], right_metric["values"]
)
]
normalized_errors = [
abs(a - b)
for a, b in zip(
left_metric["statistics"]["normalized"],
right_metric["statistics"]["normalized"],
)
]
item_passed = (
all(
math.isfinite(value) and value > 0
for value in left_metric["values"]
)
and max(raw_errors) <= parent.SPECTRUM_TOLERANCE
and max(normalized_errors) <= parent.SPECTRUM_TOLERANCE
)
passed = passed and item_passed
checks[position] = {
"passed": item_passed,
"max_raw_relative_error": max(raw_errors),
"max_normalized_absolute_error": max(normalized_errors),
}
return {"passed": passed, "checks": checks}
def initialization_negative_control(
parent_learned: dict[str, Any], matrix: dict[str, Any]
) -> dict[str, Any]:
reference = matrix["detached_learned"]
comparisons = {
"parent_learned_vs_detached": spectrum_agreement(
parent_learned, reference
)
}
for mode in MATRIX_MODES[1:]:
comparisons[mode] = spectrum_agreement(reference, matrix[mode])
passed = all(item["passed"] for item in comparisons.values())
if not passed:
raise RuntimeError("initialization negative control failed")
return {"passed": True, "comparisons": comparisons}
def without_selector(result: dict[str, Any], rename: str | None = None) -> dict[str, Any]:
cleaned = {key: value for key, value in result.items() if key != "selector"}
if rename is not None:
cleaned["mode"] = rename
return cleaned
def endpoint_exactness(
matrix: dict[str, Any], parent_diagnostic: dict[str, Any]
) -> dict[str, Any]:
reference_exact = (
without_selector(matrix["detached_learned"])
== parent_diagnostic["modes"]["detached_learned"]
)
uniform_exact = (
without_selector(
matrix["uniform_all"], rename="uniform_value_backward"
)
== parent_diagnostic["modes"]["uniform_value_backward"]
)
checks = {
"detached_learned_round06_exact": reference_exact,
"uniform_all_round06_exact": uniform_exact,
}
if not all(checks.values()):
raise RuntimeError(f"Round 06 endpoint exactness failed: {checks}")
return {"passed": True, "checks": checks}
def run_diagnostic_bundle(
model: LocalPathLanguageModel,
corpus: Any,
optimizer: torch.optim.Optimizer,
manifest: dict[str, Any],
parent_diagnostic: dict[str, Any],
step: int,
) -> dict[str, Any]:
parent_learned = parent.run_diagnostic(
model, corpus, optimizer, mode="learned"
)
if parent_learned != parent_diagnostic["modes"]["learned"]:
raise RuntimeError("parent learned diagnostic is not Round 06 exact")
result: dict[str, Any] = {
"step": step,
"parent_learned": parent_learned,
"parent_learned_round06_exact": True,
"local_matrix": None,
}
if step not in MATRIX_STEPS:
return result
matrix = {
mode: run_local_diagnostic(
model, corpus, optimizer, manifest, mode=mode
)
for mode in MATRIX_MODES
}
result["local_matrix"] = matrix
result["forward_identity_gate"] = forward_identity_gate(matrix)
result["endpoint_exactness"] = endpoint_exactness(
matrix, parent_diagnostic
)
if step == 0:
result["initialization_negative_control"] = (
initialization_negative_control(parent_learned, matrix)
)
doubled = run_local_diagnostic(
model,
corpus,
optimizer,
manifest,
mode="detached_learned",
loss_scale=2.0,
)
result["loss_scale_gate"] = parent.loss_scale_gate(
matrix["detached_learned"], doubled
)
model.zero_grad(set_to_none=True)
return result
def load_and_verify_inputs(
args: argparse.Namespace,
) -> tuple[dict[str, Any], dict[str, Any], dict[str, Any], Path]:
manifest = json.loads(args.manifest.read_text())
parent_manifest = json.loads(args.parent_manifest.read_text())
data_manifest = json.loads(args.data_manifest.read_text())
repo_root = Path(__file__).resolve().parents[3]
if manifest["protocol_id"] != PROTOCOL_ID:
raise RuntimeError("Round 07 manifest protocol mismatch")
if parent_manifest["protocol_id"] != PARENT_PROTOCOL_ID:
raise RuntimeError("Round 06 parent manifest protocol mismatch")
if data_manifest["protocol_id"] != DATA_PROTOCOL_ID:
raise RuntimeError("data manifest protocol mismatch")
if manifest["formal_seeds"] != list(SEEDS):
raise RuntimeError("formal seed mismatch")
if manifest["matrix_modes"] != list(MATRIX_MODES):
raise RuntimeError("local matrix mode mismatch")
if (
manifest["training"]["parent_diagnostic_steps"]
!= list(PARENT_DIAGNOSTIC_STEPS)
or manifest["training"]["local_matrix_steps"] != list(MATRIX_STEPS)
or manifest["training"]["steps"] != FORMAL_STEPS
):
raise RuntimeError("diagnostic/training schedule mismatch")
parent_artifacts = manifest["parent_artifacts"]
if parent.file_sha256(args.parent_manifest) != parent_artifacts[
"manifest_sha256"
]:
raise RuntimeError("Round 06 manifest physical hash mismatch")
if parent.file_sha256(Path(parent.__file__)) != parent_artifacts[
"runner_sha256"
]:
raise RuntimeError("Round 06 runner physical hash mismatch")
for name in ("protocol", "scoping"):
path = repo_root / parent_artifacts[f"{name}_path"]
if parent.file_sha256(path) != parent_artifacts[f"{name}_sha256"]:
raise RuntimeError(f"Round 06 {name} physical hash mismatch")
for name in ("protocol", "scoping", "grok_review"):
path = repo_root / manifest["current_artifacts"][f"{name}_path"]
if parent.file_sha256(path) != manifest["current_artifacts"][
f"{name}_sha256"
]:
raise RuntimeError(f"Round 07 {name} physical hash mismatch")
for key in (
"formal_schedule_sha256",
"validation_tensor_sha256",
"diagnostic_tensor_sha256",
):
if (
data_manifest["windows"][key]
!= parent_artifacts[key]
or parent_manifest["parent_artifacts"][key]
!= parent_artifacts[key]
):
raise RuntimeError(f"frozen data hash mismatch: {key}")
parent_raw_path = (
repo_root
/ "experiments"
/ "k3"
/ "attnres_spike"
/ "results"
/ "raw"
/ f"formal-seed-{args.seed}.json"
)
expected = manifest["round06_expected"][str(args.seed)]
if parent.file_sha256(parent_raw_path) != expected["raw_file_sha256"]:
raise RuntimeError("Round 06 raw physical hash mismatch")
parent_raw = json.loads(parent_raw_path.read_text())
if (
parent_raw["canonical_sha256_without_self"]
!= expected["canonical_sha256"]
or parent_raw["hashes"]["final_model_state"]
!= expected["final_model_state"]
or parent_raw["hashes"]["final_optimizer_state"]
!= expected["final_optimizer_state"]
):
raise RuntimeError("Round 06 raw expected-state mismatch")
return manifest, data_manifest, parent_raw, repo_root
def frozen_training_compare(
result: dict[str, Any], parent_raw: dict[str, Any]
) -> dict[str, Any]:
checks = {
"final_model_state": (
result["hashes"]["final_model_state"]
== parent_raw["hashes"]["final_model_state"]
),
"final_optimizer_state": (
result["hashes"]["final_optimizer_state"]
== parent_raw["hashes"]["final_optimizer_state"]
),
"evaluations": result["evaluations"] == parent_raw["evaluations"],
"training_history": (
result["training_history"] == parent_raw["training_history"]
),
}
parent_diagnostics_exact = []
endpoint_exact = []
for new, old in zip(result["diagnostics"], parent_raw["diagnostics"]):
parent_diagnostics_exact.append(
new["step"] == old["step"]
and new["parent_learned"] == old["modes"]["learned"]
)
if new["step"] in MATRIX_STEPS:
endpoint_exact.append(new["endpoint_exactness"]["passed"])
checks["parent_learned_diagnostics"] = all(parent_diagnostics_exact)
checks["round06_endpoints"] = len(endpoint_exact) == 2 and all(
endpoint_exact
)
if not all(checks.values()):
raise RuntimeError(f"Round 06 training equivalence failed: {checks}")
return {"passed": True, "checks": checks}
def main() -> None:
args = parse_args()
if not torch.cuda.is_available():
raise RuntimeError("CUDA is required")
if os.environ.get("CUBLAS_WORKSPACE_CONFIG") != ":4096:8":
raise RuntimeError("CUBLAS_WORKSPACE_CONFIG must be :4096:8")
manifest, data_manifest, parent_raw, repo_root = load_and_verify_inputs(args)
parent.parent.configure_round04_globals(DEPTH)
parent.parent.configure_determinism(args.seed)
corpus = parent.parent.round04.ByteCorpus(
args.cache_dir, data_manifest, torch.device("cuda")
)
model = LocalPathLanguageModel("block", manifest).to(
torch.device("cuda")
)
initial_public_hash = parent.parent.named_state_hash(
model, include_mixers=False
)
initial_mixer_hash = parent.parent.named_state_hash(
model, include_mixers=True
)
public_structure_hash, public_tensors, public_elements = (
parent.parent.state_structure_hash(model, include_mixers=False)
)
input_gate_hashes = parent.parent.model_input_gate_hashes(
corpus, data_manifest, args.seed, TRAIN_BATCH_SIZE
)
decay_parameters: list[nn.Parameter] = []
no_decay_parameters: list[nn.Parameter] = []
for parameter in model.parameters():
target = decay_parameters if parameter.ndim >= 2 else no_decay_parameters
target.append(parameter)
optimizer = torch.optim.AdamW(
[
{
"params": decay_parameters,
"weight_decay": parent.parent.WEIGHT_DECAY,
},
{"params": no_decay_parameters, "weight_decay": 0.0},
],
lr=parent.parent.PEAK_LR,
betas=parent.parent.BETAS,
eps=parent.parent.ADAM_EPS,
)
parent_by_step = {
item["step"]: item for item in parent_raw["diagnostics"]
}
evaluations = [
{
"step": 0,
**parent.parent.evaluate(
model, corpus, VALIDATION_WINDOWS, EVAL_BATCH_SIZE
),
}
]
diagnostics = [
run_diagnostic_bundle(
model,
corpus,
optimizer,
manifest,
parent_by_step[0],
0,
)
]
print(
json.dumps(
{
"event": "local_matrix",
"step": 0,
"seed": args.seed,
"modes": len(MATRIX_MODES),
"endpoint_exact": diagnostics[0][
"endpoint_exactness"
]["passed"],
},
sort_keys=True,
),
flush=True,
)
steps = 0 if args.run_kind == "smoke" else FORMAL_STEPS
training_history: list[dict[str, float | int]] = []
step_times: list[float] = []
if steps:
model.train()
for step in range(1, steps + 1):
lr = parent.parent.learning_rate(step, steps)
for group in optimizer.param_groups:
group["lr"] = lr
inputs, targets = corpus.training_batch(
args.seed, step, TRAIN_BATCH_SIZE
)
optimizer.zero_grad(set_to_none=True)
torch.cuda.synchronize()
started = time.perf_counter()
with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
logits, trace = model(inputs)
if trace is not None:
raise RuntimeError(
"training unexpectedly captured a trace"
)
loss = parent.parent.cross_entropy(logits, targets)
if not torch.isfinite(loss):
raise RuntimeError(f"non-finite loss at step {step}")
loss.backward()
unclipped_norm = torch.nn.utils.clip_grad_norm_(
model.parameters(), parent.parent.GRAD_CLIP
)
optimizer.step()
torch.cuda.synchronize()
elapsed_ms = (time.perf_counter() - started) * 1000
if step == TIMING_WARMUP:
torch.cuda.reset_peak_memory_stats()
elif step > TIMING_WARMUP:
step_times.append(elapsed_ms)
if step == 1 or step % 10 == 0 or step == steps:
training_history.append(
{
"step": step,
"loss_nats": loss.detach().cpu().item(),
"bits_per_byte": (
loss.detach().cpu().item() / math.log(2)
),
"learning_rate": lr,
"unclipped_grad_norm": float(
unclipped_norm.detach().cpu()
),
}
)
if step in PARENT_DIAGNOSTIC_STEPS:
evaluations.append(
{
"step": step,
**parent.parent.evaluate(
model,
corpus,
VALIDATION_WINDOWS,
EVAL_BATCH_SIZE,
),
}
)
diagnostic = run_diagnostic_bundle(
model,
corpus,
optimizer,
manifest,
parent_by_step[step],
step,
)
diagnostics.append(diagnostic)
event = {
"event": "diagnostic",
"step": step,
"seed": args.seed,
"validation_bpc": evaluations[-1]["bits_per_byte"],
"parent_exact": diagnostic[
"parent_learned_round06_exact"
],
}
if diagnostic["local_matrix"] is not None:
event["modes"] = len(MATRIX_MODES)
event["endpoint_exact"] = diagnostic[
"endpoint_exactness"
]["passed"]
print(json.dumps(event, sort_keys=True), flush=True)
model.train()
timing = {
"warmup_steps_excluded": TIMING_WARMUP,
"measured_steps": len(step_times),
"mean_ms": (
statistics.fmean(step_times) if step_times else None
),
"median_ms": (
statistics.median(step_times) if step_times else None
),
"p95_ms": (
parent.type7_quantile(torch.tensor(sorted(step_times)), 0.95)
if step_times
else None
),
"peak_allocated_bytes": torch.cuda.max_memory_allocated(),
"peak_reserved_bytes": torch.cuda.max_memory_reserved(),
}
result = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"parent_protocol_id": PARENT_PROTOCOL_ID,
"run_kind": args.run_kind,
"architecture": "block",
"depth": DEPTH,
"seed": args.seed,
"steps": steps,
"batch_size": TRAIN_BATCH_SIZE,
"target_bytes_seen": (
steps * TRAIN_BATCH_SIZE * parent.CONTEXT
),
"manifest": {
"path": str(args.manifest),
"file_sha256": parent.file_sha256(args.manifest),
"parent_path": str(args.parent_manifest),
"parent_file_sha256": parent.file_sha256(
args.parent_manifest
),
"data_path": str(args.data_manifest),
"data_file_sha256": parent.file_sha256(args.data_manifest),
"formal_schedule_sha256": data_manifest["windows"][
"formal_schedule_sha256"
],
"validation_tensor_sha256": data_manifest["windows"][
"validation_tensor_sha256"
],
"diagnostic_tensor_sha256": data_manifest["windows"][
"diagnostic_tensor_sha256"
],
"input_gate_tensor_hashes": input_gate_hashes,
"selector_contract_sha256": parent.canonical_sha256(
manifest["selector"]
),
},
"model": {
"layers": DEPTH,
"aggregation_groups": 8,
"blocks_per_group": 4,
"d_model": parent.parent.round04.D_MODEL,
"heads": parent.parent.round04.HEADS,
"d_ff": parent.parent.round04.D_FF,
"parameters": parent.parent.parameter_inventory(model),
},
"hashes": {
"initial_public_parameter_structure": public_structure_hash,
"initial_public_parameter_tensors": public_tensors,
"initial_public_parameter_elements": public_elements,
"initial_public_parameters": initial_public_hash,
"initial_mixer_parameters": initial_mixer_hash,
"final_public_parameters": parent.parent.named_state_hash(
model, include_mixers=False
),
"final_mixer_parameters": parent.parent.named_state_hash(
model, include_mixers=True
),
"final_model_state": parent.parent.named_state_hash(
model, include_mixers=None
),
"final_optimizer_state": parent.parent.recursive_state_hash(
optimizer.state_dict()
),
},
"evaluations": evaluations,
"diagnostics": diagnostics,
"training_history": training_history,
"timing": timing,
"environment": {
"python": platform.python_version(),
"torch": torch.__version__,
"cuda": torch.version.cuda,
"gpu": torch.cuda.get_device_name(0),
"compute_capability": list(
torch.cuda.get_device_capability(0)
),
"cublas_workspace_config": os.environ[
"CUBLAS_WORKSPACE_CONFIG"
],
"deterministic_algorithms": (
torch.are_deterministic_algorithms_enabled()
),
"autocast": "cuda-bfloat16-forward-fp32-cross-entropy",
"compile": False,
},
"artifacts": {
"runner_sha256": parent.file_sha256(Path(__file__)),
"protocol_sha256": parent.file_sha256(
repo_root
/ "research"
/ "K3_ATTNRES_LOCAL_PATH_PROTOCOL.md"
),
"scoping_sha256": parent.file_sha256(
repo_root
/ "research"
/ "K3_ATTNRES_LOCAL_PATH_SCOPING.md"
),
"grok_review_sha256": parent.file_sha256(
repo_root
/ "research"
/ "K3_ATTNRES_LOCAL_PATH_GROK_REVIEW.md"
),
},
}
result["round06_equivalence"] = (
frozen_training_compare(result, parent_raw) if steps else None
)
result["canonical_sha256_without_self"] = parent.canonical_sha256(
result
)
args.output.parent.mkdir(parents=True, exist_ok=True)
temporary = args.output.with_suffix(args.output.suffix + ".tmp")
temporary.write_text(
json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True)
+ "\n"
)
os.replace(temporary, args.output)
print(
json.dumps(
{
"output": str(args.output),
"run_kind": args.run_kind,
"seed": args.seed,
"steps": steps,
"final_bpc": evaluations[-1]["bits_per_byte"],
"canonical_sha256": result[
"canonical_sha256_without_self"
],
"timing": timing,
},
ensure_ascii=False,
indent=2,
),
flush=True,
)
if __name__ == "__main__":
main()
+40
View File
@@ -0,0 +1,40 @@
# Attention Residuals spike-path diagnostics
This directory implements preregistered protocol
`llm-atlas-k3-attnres-spike-path-v1`.
It is a targeted follow-up to Round 05. It replays the exact depth-32 Block
training contract and adds diagnostic-only activation positions, gradient
reductions, and same-forward backward-rule interventions. It is not a Kimi K3
checkpoint run and does not recover the paper's unpublished Figure 5 telemetry.
## Frozen environment
```text
Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
PyTorch 2.11.0+cu128
GPU NVIDIA GeForce RTX 5090
CUBLAS_WORKSPACE_CONFIG=:4096:8
```
## Step-0 smoke
```bash
CUBLAS_WORKSPACE_CONFIG=:4096:8 \
/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
experiments/k3/attnres_spike/train.py \
--run-kind smoke \
--seed 2026073001 \
--cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
--parent-manifest experiments/k3/attnres_gradient/manifest.json \
--manifest experiments/k3/attnres_spike/manifest.json \
--output /home/wuyang/.cache/llm-atlas/k3-attnres-spike-path-v1/smoke/seed-2026073001.json
```
Formal cells use `--run-kind formal` and all three preregistered seeds. The
independent replay uses `--run-kind replay --seed 2026073001`. Formal and replay
runs are fixed to 8,000 steps; smoke performs the complete step-0 diagnostic
gate without an optimizer step.
Raw outputs are copied into `results/raw/` only after training equivalence,
forward identity, loss-scale, reduction, and replay gates pass.
+751
View File
@@ -0,0 +1,751 @@
#!/usr/bin/env python3
"""Aggregate and gate the preregistered AttnRes spike-path study."""
from __future__ import annotations
import argparse
import hashlib
import json
import math
import statistics
from pathlib import Path
from typing import Any, Iterable
PROTOCOL_ID = "llm-atlas-k3-attnres-spike-path-v1"
SEEDS = (2026073001, 2026073002, 2026073003)
POSITIONS = (
"pre_attention_input",
"attention_branch_output",
"post_attention_state",
"pre_mlp_input",
"mlp_branch_output",
"post_mlp_state",
)
MAIN_REDUCTIONS = (
"element_rms",
"token_rms_mean",
"token_rms_median",
"token_rms_p95",
)
ALL_REDUCTIONS = MAIN_REDUCTIONS + (
"batch_mean_rms",
"token_mean_rms",
"global_l2",
)
MODES = (
"learned",
"detached_learned",
"uniform_value_backward",
)
SPIKE_LAYERS = (21, 22, 23, 24, 25)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument("--raw-dir", type=Path, required=True)
parser.add_argument("--manifest", type=Path, required=True)
parser.add_argument("--aggregate-output", type=Path, required=True)
parser.add_argument("--compact-output", type=Path, required=True)
parser.add_argument("--reproduction-output", type=Path, required=True)
return parser.parse_args()
def file_sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for block in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(block)
return digest.hexdigest()
def canonical_sha256(value: Any) -> str:
return hashlib.sha256(
json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode()
).hexdigest()
def mean(values: Iterable[float]) -> float:
return statistics.fmean(values)
def average_ranks(values: list[float]) -> list[float]:
ordered = sorted(range(len(values)), key=lambda index: (values[index], index))
ranks = [0.0] * len(values)
cursor = 0
while cursor < len(ordered):
end = cursor + 1
while end < len(ordered) and values[ordered[end]] == values[ordered[cursor]]:
end += 1
average_rank = (cursor + 1 + end) / 2
for offset in range(cursor, end):
ranks[ordered[offset]] = average_rank
cursor = end
return ranks
def pearson(left: list[float], right: list[float]) -> float:
left_mean = mean(left)
right_mean = mean(right)
numerator = sum(
(x - left_mean) * (y - right_mean)
for x, y in zip(left, right)
)
left_square = sum((value - left_mean) ** 2 for value in left)
right_square = sum((value - right_mean) ** 2 for value in right)
if left_square == 0 or right_square == 0:
raise RuntimeError("correlation is undefined for a constant vector")
return numerator / math.sqrt(left_square * right_square)
def spearman(left: list[float], right: list[float]) -> float:
return pearson(average_ranks(left), average_ranks(right))
def load_run(path: Path, *, expected_kind: str, expected_seed: int) -> dict[str, Any]:
value = json.loads(path.read_text())
if value["protocol_id"] != PROTOCOL_ID:
raise RuntimeError(f"protocol mismatch: {path}")
if value["run_kind"] != expected_kind or value["seed"] != expected_seed:
raise RuntimeError(f"run identity mismatch: {path}")
canonical = value.pop("canonical_sha256_without_self")
if canonical_sha256(value) != canonical:
raise RuntimeError(f"canonical hash mismatch: {path}")
value["canonical_sha256_without_self"] = canonical
if expected_kind != "smoke":
if not value["round05_equivalence"]["passed"]:
raise RuntimeError(f"Round 05 equivalence failed: {path}")
if value["steps"] != 8000:
raise RuntimeError(f"formal step count mismatch: {path}")
return value
def final_diagnostic(run: dict[str, Any]) -> dict[str, Any]:
matches = [item for item in run["diagnostics"] if item["step"] == 8000]
if len(matches) != 1:
raise RuntimeError("final diagnostic missing or duplicated")
return matches[0]
def metric(
run: dict[str, Any],
*,
position: str,
reduction: str,
mode: str = "learned",
) -> dict[str, Any]:
return final_diagnostic(run)["modes"][mode]["positions"][position][
"reductions"
][reduction]
def compare_replay(formal: dict[str, Any], replay: dict[str, Any]) -> dict[str, Any]:
fields = (
"architecture",
"depth",
"seed",
"steps",
"batch_size",
"target_bytes_seen",
"manifest",
"model",
"optimizer",
"hashes",
"evaluations",
"diagnostics",
"training_history",
"environment",
"artifacts",
"round05_equivalence",
)
checks = {field: formal[field] == replay[field] for field in fields}
passed = all(checks.values())
if not passed:
raise RuntimeError(f"Round 06 replay mismatch: {checks}")
compare_payload = {field: formal[field] for field in fields}
return {
"passed": True,
"checks": checks,
"frozen_compare_sha256": canonical_sha256(compare_payload),
}
def reduction_robustness(runs: list[dict[str, Any]], manifest: dict[str, Any]) -> dict[str, Any]:
threshold = manifest["thresholds"]
cells = []
for run in runs:
reference = metric(
run,
position="post_mlp_state",
reduction="element_rms",
)
for reduction in MAIN_REDUCTIONS:
candidate = metric(
run,
position="post_mlp_state",
reduction=reduction,
)
stats = candidate["statistics"]
rho = spearman(reference["values"], candidate["values"])
checks = {
"spike_contrast": (
stats["spike_contrast"]
>= threshold["spike_contrast"]
),
"top_five_overlap": (
stats["top_five_spike_overlap"]
>= threshold["top_five_min_overlap"]
),
"spearman": rho >= threshold["spearman_minimum"],
}
cells.append(
{
"seed": run["seed"],
"reduction": reduction,
"spike_contrast": stats["spike_contrast"],
"top_five_layers": stats["top_five_layers"],
"top_five_spike_overlap": stats[
"top_five_spike_overlap"
],
"spearman_vs_element_rms": rho,
"checks": checks,
"passed": all(checks.values()),
}
)
passed_count = sum(item["passed"] for item in cells)
if passed_count == len(cells):
verdict = "robust within the preregistered reduction family"
elif passed_count == 0:
verdict = "not robust at this threshold"
else:
verdict = "mixed"
return {
"verdict": verdict,
"passed_cells": passed_count,
"total_cells": len(cells),
"cells": cells,
}
def visible_positions(runs: list[dict[str, Any]], manifest: dict[str, Any]) -> dict[str, Any]:
threshold = manifest["thresholds"]["spike_contrast"]
positions = []
for position in POSITIONS:
per_seed = []
for run in runs:
stats = metric(
run, position=position, reduction="element_rms"
)["statistics"]
per_seed.append(
{
"seed": run["seed"],
"spike_contrast": stats["spike_contrast"],
"peak_layer": stats["peak_layer"],
"peak_normalized": stats["peak_normalized"],
"passed": stats["spike_contrast"] >= threshold,
}
)
positions.append(
{
"position": position,
"visible_3_of_3": all(item["passed"] for item in per_seed),
"per_seed": per_seed,
"mean_spike_contrast": mean(
item["spike_contrast"] for item in per_seed
),
}
)
visible = [item["position"] for item in positions if item["visible_3_of_3"]]
return {
"criterion": f"step 8000 element-RMS spike contrast >= {threshold} in 3/3 seeds",
"visible_positions": visible,
"earliest_observed_tensor": visible[0] if visible else None,
"verdict": "visible at one or more positions" if visible else "position-mixed",
"positions": positions,
}
def intervention_effect(
runs: list[dict[str, Any]],
manifest: dict[str, Any],
*,
source_mode: str,
target_mode: str,
label: str,
) -> dict[str, Any]:
threshold = manifest["thresholds"]["material_relative_drop"]
epsilon = manifest["thresholds"]["positive_denominator_epsilon"]
per_seed = []
for run in runs:
source = metric(
run,
position="post_mlp_state",
reduction="element_rms",
mode=source_mode,
)["statistics"]
target = metric(
run,
position="post_mlp_state",
reduction="element_rms",
mode=target_mode,
)["statistics"]
if (
not math.isfinite(source["spike_contrast"])
or not math.isfinite(source["peak_normalized"])
or source["spike_contrast"] <= epsilon
or source["peak_normalized"] <= epsilon
):
raise RuntimeError("invalid intervention denominator")
contrast_drop = (
source["spike_contrast"] - target["spike_contrast"]
) / source["spike_contrast"]
peak_drop = (
source["peak_normalized"] - target["peak_normalized"]
) / source["peak_normalized"]
per_seed.append(
{
"seed": run["seed"],
"source_spike_contrast": source["spike_contrast"],
"target_spike_contrast": target["spike_contrast"],
"relative_drop_contrast": contrast_drop,
"source_peak_normalized": source["peak_normalized"],
"target_peak_normalized": target["peak_normalized"],
"relative_drop_peak": peak_drop,
"passed": (
contrast_drop >= threshold and peak_drop >= threshold
),
}
)
passed = all(item["passed"] for item in per_seed)
same_direction = all(
item["relative_drop_contrast"] > 0
and item["relative_drop_peak"] > 0
for item in per_seed
)
opposite_direction = all(
item["relative_drop_contrast"] < 0
and item["relative_drop_peak"] < 0
for item in per_seed
)
if passed:
verdict = "material sensitivity at the preregistered threshold"
elif same_direction:
verdict = "same-direction but below the joint threshold"
elif opposite_direction:
verdict = "opposite direction in 3/3 seeds; no material reduction"
else:
verdict = "mixed"
return {
"label": label,
"source_mode": source_mode,
"target_mode": target_mode,
"threshold": threshold,
"passed_3_of_3": passed,
"same_direction_3_of_3": same_direction,
"opposite_direction_3_of_3": opposite_direction,
"verdict": verdict,
"per_seed": per_seed,
"mean_relative_drop_contrast": mean(
item["relative_drop_contrast"] for item in per_seed
),
"mean_relative_drop_peak": mean(
item["relative_drop_peak"] for item in per_seed
),
}
def intervention_summary(runs: list[dict[str, Any]], manifest: dict[str, Any]) -> dict[str, Any]:
forward_gates = []
for run in runs:
diagnostic = final_diagnostic(run)
forward_gates.append(
{
"seed": run["seed"],
"passed": diagnostic["forward_identity_gate"]["passed"],
}
)
if not all(item["passed"] for item in forward_gates):
raise RuntimeError("a final forward identity gate failed")
return {
"scope": manifest["interventions"]["scope"],
"forward_identity": forward_gates,
"softmax_key_path": intervention_effect(
runs,
manifest,
source_mode="learned",
target_mode="detached_learned",
label="global removal of all mixer softmax/query/key source-gradient paths",
),
"value_coefficients": intervention_effect(
runs,
manifest,
source_mode="detached_learned",
target_mode="uniform_value_backward",
label="global replacement of learned value-backward coefficients with 1/N",
),
}
def trajectory(runs: list[dict[str, Any]]) -> list[dict[str, Any]]:
result = []
for run in runs:
evaluations = {
item["step"]: item["bits_per_byte"]
for item in run["evaluations"]
}
points = []
for diagnostic in run["diagnostics"]:
stats = diagnostic["modes"]["learned"]["positions"][
"post_mlp_state"
]["reductions"]["element_rms"]["statistics"]
points.append(
{
"step": diagnostic["step"],
"bits_per_byte": evaluations[diagnostic["step"]],
"population_cv": stats["population_cv"],
"spike_contrast": stats["spike_contrast"],
"peak_layer": stats["peak_layer"],
"peak_normalized": stats["peak_normalized"],
}
)
result.append({"seed": run["seed"], "points": points})
return result
def mixer_associations(runs: list[dict[str, Any]]) -> dict[str, Any]:
rows = []
for run in runs:
diagnostic = final_diagnostic(run)
gradients = metric(
run,
position="post_mlp_state",
reduction="element_rms",
)["statistics"]["normalized"]
mixers = diagnostic["modes"]["learned"]["mixers"]
for layer in range(1, 33):
attention = mixers[2 * (layer - 1)]
mlp = mixers[2 * (layer - 1) + 1]
rows.append(
{
"seed": run["seed"],
"layer": layer,
"group": (layer - 1) // 4 + 1,
"offset": (layer - 1) % 4 + 1,
"normalized_gradient": gradients[layer - 1],
"attention_latest": attention["latest_source_mass"],
"attention_entropy": attention["normalized_entropy"],
"mlp_latest": mlp["latest_source_mass"],
"mlp_entropy": mlp["normalized_entropy"],
"attention_max": attention["max_source_mass"],
"mlp_max": mlp["max_source_mass"],
}
)
def correlations(selected: list[dict[str, Any]]) -> dict[str, Any]:
gradient = [row["normalized_gradient"] for row in selected]
result = {"points": len(selected)}
for key in (
"attention_latest",
"attention_entropy",
"mlp_latest",
"mlp_entropy",
"attention_max",
"mlp_max",
):
values = [row[key] for row in selected]
result[key] = {
"pearson": pearson(gradient, values),
"spearman": spearman(gradient, values),
}
return result
target_layers = []
for layer in range(19, 29):
selected = [row for row in rows if row["layer"] == layer]
target_layers.append(
{
"layer": layer,
"group": selected[0]["group"],
"offset": selected[0]["offset"],
**{
key: mean(row[key] for row in selected)
for key in (
"normalized_gradient",
"attention_latest",
"attention_entropy",
"mlp_latest",
"mlp_entropy",
"attention_max",
"mlp_max",
)
},
"per_seed_normalized_gradient": [
row["normalized_gradient"] for row in selected
],
}
)
return {
"identity": "observational association; not causal attribution",
"all_layers": correlations(rows),
"layers_19_28": correlations(
[row for row in rows if 19 <= row["layer"] <= 28]
),
"target_layers": target_layers,
}
def final_arrays(runs: list[dict[str, Any]]) -> list[dict[str, Any]]:
values = []
for run in runs:
positions = {}
for position in POSITIONS:
positions[position] = {}
for reduction in ALL_REDUCTIONS:
item = metric(
run, position=position, reduction=reduction
)
positions[position][reduction] = {
"values": item["values"],
"statistics": item["statistics"],
}
interventions = {}
for mode in MODES:
item = metric(
run,
position="post_mlp_state",
reduction="element_rms",
mode=mode,
)
interventions[mode] = {
"values": item["values"],
"statistics": item["statistics"],
}
values.append(
{
"seed": run["seed"],
"positions": positions,
"interventions": interventions,
"final_mixers": final_diagnostic(run)["modes"]["learned"][
"mixers"
],
}
)
return values
def compact_final_arrays(runs: list[dict[str, Any]]) -> list[dict[str, Any]]:
values = []
for run in runs:
position_element_rms = {
position: metric(
run,
position=position,
reduction="element_rms",
)
for position in POSITIONS
}
post_mlp_reductions = {
reduction: metric(
run,
position="post_mlp_state",
reduction=reduction,
)
for reduction in ALL_REDUCTIONS
}
interventions = {
mode: metric(
run,
position="post_mlp_state",
reduction="element_rms",
mode=mode,
)
for mode in MODES
}
values.append(
{
"seed": run["seed"],
"position_element_rms": position_element_rms,
"post_mlp_reductions": post_mlp_reductions,
"interventions": interventions,
}
)
return values
def write_json(path: Path, value: dict[str, Any]) -> None:
value["canonical_sha256_without_self"] = canonical_sha256(value)
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_suffix(path.suffix + ".tmp")
temporary.write_text(
json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
)
temporary.replace(path)
def main() -> None:
args = parse_args()
manifest = json.loads(args.manifest.read_text())
if manifest["protocol_id"] != PROTOCOL_ID:
raise RuntimeError("manifest protocol mismatch")
formal_paths = {
seed: args.raw_dir / f"formal-seed-{seed}.json"
for seed in SEEDS
}
replay_path = args.raw_dir / f"replay-seed-{SEEDS[0]}.json"
formal = [
load_run(formal_paths[seed], expected_kind="formal", expected_seed=seed)
for seed in SEEDS
]
replay = load_run(
replay_path, expected_kind="replay", expected_seed=SEEDS[0]
)
replay_gate = compare_replay(formal[0], replay)
reduction = reduction_robustness(formal, manifest)
positions = visible_positions(formal, manifest)
interventions = intervention_summary(formal, manifest)
trajectories = trajectory(formal)
mixers = mixer_associations(formal)
arrays = final_arrays(formal)
raw_files = {
path.name: {
"file_sha256": file_sha256(path),
"canonical_sha256": run["canonical_sha256_without_self"],
}
for path, run in [
*[(formal_paths[seed], run) for seed, run in zip(SEEDS, formal)],
(replay_path, replay),
]
}
reproduction = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"raw_files": raw_files,
"replay_gate": replay_gate,
"artifacts": {
"manifest": file_sha256(args.manifest),
"runner": formal[0]["artifacts"]["runner_sha256"],
"protocol": formal[0]["artifacts"]["protocol_sha256"],
"scoping": formal[0]["artifacts"]["scoping_sha256"],
"analyzer": file_sha256(Path(__file__)),
},
}
write_json(args.reproduction_output, reproduction)
aggregate = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"study": {
"identity": manifest["study_identity"],
"architecture": "block",
"depth": 32,
"seeds": list(SEEDS),
"steps": 8000,
"spike_layers": list(SPIKE_LAYERS),
},
"gates": {
"round05_equivalence": [
{
"seed": run["seed"],
**run["round05_equivalence"],
}
for run in formal
],
"replay": replay_gate,
"forward_identity": interventions["forward_identity"],
"initialization_negative_control": [
{
"seed": run["seed"],
"passed": run["diagnostics"][0][
"initialization_negative_control"
]["passed"],
}
for run in formal
],
"loss_scale": [
{
"seed": run["seed"],
"passed": run["diagnostics"][0][
"loss_scale_gate"
]["passed"],
}
for run in formal
],
},
"verdicts": {
"reduction_robustness": reduction,
"visible_positions": positions,
"interventions": interventions,
},
"trajectory": trajectories,
"mixer_associations": mixers,
"final_arrays": arrays,
"runs": [
{
"seed": run["seed"],
"canonical_sha256": run["canonical_sha256_without_self"],
"final_bpc": run["evaluations"][-1]["bits_per_byte"],
"final_model_state": run["hashes"]["final_model_state"],
"final_optimizer_state": run["hashes"][
"final_optimizer_state"
],
"timing": run["timing"],
}
for run in formal
],
"reproduction_canonical_sha256": reproduction[
"canonical_sha256_without_self"
],
}
write_json(args.aggregate_output, aggregate)
compact = {
"schema_version": 1,
"protocol_id": PROTOCOL_ID,
"study": aggregate["study"],
"gates": aggregate["gates"],
"verdicts": aggregate["verdicts"],
"trajectory": trajectories,
"mixer_associations": mixers,
"final_arrays": compact_final_arrays(formal),
"runs": aggregate["runs"],
"hashes": {
"aggregate_canonical_sha256": aggregate[
"canonical_sha256_without_self"
],
"reproduction_canonical_sha256": reproduction[
"canonical_sha256_without_self"
],
},
}
write_json(args.compact_output, compact)
print(
json.dumps(
{
"aggregate": str(args.aggregate_output),
"compact": str(args.compact_output),
"reproduction": str(args.reproduction_output),
"reduction_verdict": reduction["verdict"],
"earliest_observed_tensor": positions[
"earliest_observed_tensor"
],
"softmax_key_path": interventions["softmax_key_path"][
"verdict"
],
"value_coefficients": interventions["value_coefficients"][
"verdict"
],
"replay_exact": replay_gate["passed"],
},
ensure_ascii=False,
indent=2,
)
)
if __name__ == "__main__":
main()
+118
View File
@@ -0,0 +1,118 @@
{
"schema_version": 1,
"protocol_id": "llm-atlas-k3-attnres-spike-path-v1",
"parent_protocol_id": "llm-atlas-k3-attnres-gradient-scale-v1",
"study_identity": "targeted follow-up informed by Round 05; not blind discovery",
"architecture": "block",
"depth": 32,
"aggregation_groups": 8,
"formal_seeds": [
2026073001,
2026073002,
2026073003
],
"replay": {
"architecture": "block",
"depth": 32,
"seed": 2026073001
},
"training": {
"steps": 8000,
"batch_size": 32,
"context": 256,
"target_bytes_per_cell": 65536000,
"diagnostic_steps": [
0,
100,
500,
2000,
4000,
8000
]
},
"positions": [
"pre_attention_input",
"attention_branch_output",
"post_attention_state",
"pre_mlp_input",
"mlp_branch_output",
"post_mlp_state"
],
"reductions": {
"confirmatory": [
"element_rms",
"token_rms_mean",
"token_rms_median",
"token_rms_p95"
],
"exploratory": [
"batch_mean_rms",
"token_mean_rms"
],
"algebraic_control": [
"global_l2"
],
"quantile_definition": "Hyndman-Fan Type 7 linear interpolation on explicitly sorted values"
},
"interventions": {
"modes": [
"learned",
"detached_learned",
"uniform_value_backward"
],
"steps": [
0,
8000
],
"scope": "all 64 depth mixers plus the output mixer",
"training_uses_custom_autograd": false,
"confirmatory_reduction": "element_rms"
},
"spike_layers_one_based": [
21,
22,
23,
24,
25
],
"thresholds": {
"spike_contrast": 1.5,
"top_five_min_overlap": 3,
"spearman_minimum": 0.8,
"material_relative_drop": 0.2,
"spectrum_absolute_tolerance": 1e-06,
"raw_relative_tolerance": 1e-06,
"positive_denominator_epsilon": 1e-30
},
"parent_artifacts": {
"manifest_path": "experiments/k3/attnres_gradient/manifest.json",
"manifest_sha256": "080afb17d1e036c0bba0a799fdb8b98ee4ad652bd42dd1b3b67110dd2ede6371",
"runner_path": "experiments/k3/attnres_gradient/train.py",
"runner_sha256": "04ae69e10c58972c9193c2d31c7e09d924a0d0e834107afa4ba128c64ac5800f",
"protocol_path": "research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md",
"protocol_sha256": "f772629b3b82975b6756721c3a3bb57cc4171dfa26ba5b1e8043dc91c9dcce22",
"formal_schedule_sha256": "5041e09b167f229248d2462324e8c254b8f5938975f135dcd8192b00a54a4f4e",
"validation_tensor_sha256": "f459316f13078a163b47c133511bb7181e05170ab89516e196490113893ce338",
"diagnostic_tensor_sha256": "21117e31db302b10d67b63f035665dc8f220b879d216ccd12b7d2ba86e7b1716"
},
"round05_expected": {
"2026073001": {
"raw_file_sha256": "29e1d638b481619c7b32de402122523db8b17cc1fc67a8881fa1ba132a1d5c38",
"canonical_sha256": "c7554d9beea6fe9e63617aafd88403f7fc93b4b305a0c6d603a98a8572e5b5f1",
"final_model_state": "3f0b97ece3a15571ba3d656f589f512ca0bb9e20083c9f58a42ccaee14892f59",
"final_optimizer_state": "ed03e6fbd4a12d8b063dcb22e0437754285f54d585374cd52fbd534f05d24637"
},
"2026073002": {
"raw_file_sha256": "21199deb2199395061e51220fd8c7afd04a1135a6381e406da9b5795e3ad5032",
"canonical_sha256": "b4262e697e2269457fdebf31f75008383c6e8024ee1bbf96e5962fb2ba1dec18",
"final_model_state": "bd2556388aeaa211b798c283c7cbd8ccd29edf166a2922fa13d172e8dfdc38d1",
"final_optimizer_state": "0b101eab3bc7d8d654be2ea335c86fc25563ce19912d721844ee4e639c569e77"
},
"2026073003": {
"raw_file_sha256": "c0d7f1bcfa9fa7f8f3134ca4571bdf23a951182d03b7de9611a6b4b89667d4ac",
"canonical_sha256": "513b85666fcdbf75424596e968cd73733314ec188345e453998de08b18dcef65",
"final_model_state": "638568aede21890773b6932a19ec4e112f5ac0a4770ba3402fcd82980a9ecf76",
"final_optimizer_state": "83947fd743ec8e3e31ca7788fd201981846f0f1c7e9e38c88afcef95cfc6ec4e"
}
}
}
@@ -0,0 +1,52 @@
{
"artifacts": {
"analyzer": "b9dd7f723e8c5765ecffd9ae3050ff7a95643f95853adaeeaa2c2c323d0ca057",
"manifest": "d5302a249249a07d362819134763d14e7d32307f22cff416c665ed9606142fef",
"protocol": "6cb101b8760d9f1c81caeb2f16880b16152da103867224a06761a75a12984a16",
"runner": "77298081d3c491d2e88e4705995174b9879ef377f520eb5fe5ea107e7a1da084",
"scoping": "590166bd62580bb8238293823cfcc39bc0a465fec4c697025343f3f1138abd27"
},
"canonical_sha256_without_self": "7c522314742920961be84b750d823ce41814f10fd6129f0531208116e2fe3515",
"protocol_id": "llm-atlas-k3-attnres-spike-path-v1",
"raw_files": {
"formal-seed-2026073001.json": {
"canonical_sha256": "76b0ccfb55c38baef50c395788ac4b351cbe0d58de70064b50702acb5c93f515",
"file_sha256": "e39e93b7a7fce3c56f5f14f95cfdc04afdce53628affee1202fe62bd1bdb7f71"
},
"formal-seed-2026073002.json": {
"canonical_sha256": "5352c74eca853b375c0e85933dafd7c5916c39fc59052e742ca14ffd6d68bc78",
"file_sha256": "1c6f6c731030ec0adb2a8e7a4d586e0c4005cc3319568a7ac83c08c2a4b8eaf8"
},
"formal-seed-2026073003.json": {
"canonical_sha256": "7e764c07e90b78c4cd0acc2e99600225f16428cbb25d5188a0a5a8fe797f8766",
"file_sha256": "115f8245577ece6dfaaa8ada68445c186e6523a7f3b26efcc3eb4c0c4ce82406"
},
"replay-seed-2026073001.json": {
"canonical_sha256": "c9ebccba0acf10b05fea65beade50e588fd9a83b1807f7ebba8ec1ab836b26fb",
"file_sha256": "4f91c7a2698662ea69af9aa3425e144107834fd9b6b717fb82b84f153b867dc5"
}
},
"replay_gate": {
"checks": {
"architecture": true,
"artifacts": true,
"batch_size": true,
"depth": true,
"diagnostics": true,
"environment": true,
"evaluations": true,
"hashes": true,
"manifest": true,
"model": true,
"optimizer": true,
"round05_equivalence": true,
"seed": true,
"steps": true,
"target_bytes_seen": true,
"training_history": true
},
"frozen_compare_sha256": "05396eaf56429ca8d87815ecf00a5f78be5a9b2ec7cc96014856c09dcedcbb87",
"passed": true
},
"schema_version": 1
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+178
View File
@@ -0,0 +1,178 @@
#!/usr/bin/env python3
"""Rebuild the Round 06 scoping table from frozen Round 05 raw files."""
from __future__ import annotations
import argparse
import json
import math
import statistics
from pathlib import Path
from typing import Any, Callable
SEEDS = (2026073001, 2026073002, 2026073003)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument(
"--raw-dir",
type=Path,
default=Path(__file__).resolve().parents[1]
/ "attnres_gradient"
/ "results"
/ "raw",
)
return parser.parse_args()
def pearson(left: list[float], right: list[float]) -> float:
left_mean = statistics.fmean(left)
right_mean = statistics.fmean(right)
numerator = sum(
(x - left_mean) * (y - right_mean)
for x, y in zip(left, right)
)
left_square = sum((value - left_mean) ** 2 for value in left)
right_square = sum((value - right_mean) ** 2 for value in right)
return numerator / math.sqrt(left_square * right_square)
def mixer_metrics(value: dict[str, Any]) -> dict[str, float | int]:
weights = value["mean_weights"]
sources = value["sources"]
return {
"sources": sources,
"latest": weights[-1],
"maximum": max(weights),
"normalized_entropy": (
1.0
if sources == 1
else value["entropy_mean"] / math.log(sources)
),
}
def average(rows: list[dict[str, Any]], key: Callable[[dict[str, Any]], float]) -> float:
return statistics.fmean(key(row) for row in rows)
def main() -> None:
args = parse_args()
rows = []
source_hashes = {}
for seed in SEEDS:
path = (
args.raw_dir
/ f"formal-depth-32-block-seed-{seed}.json"
)
run = json.loads(path.read_text())
source_hashes[str(seed)] = run["canonical_sha256_without_self"]
diagnostic = next(
item for item in run["diagnostics"] if item["step"] == 8000
)
gradients = diagnostic["activation_grad_rms_by_block"]
gradient_mean = statistics.fmean(gradients)
for layer in range(32):
rows.append(
{
"seed": seed,
"layer": layer + 1,
"group": layer // 4 + 1,
"offset": layer % 4 + 1,
"normalized_gradient": gradients[layer] / gradient_mean,
"attention": mixer_metrics(
diagnostic["depth_weights"][2 * layer]
),
"mlp": mixer_metrics(
diagnostic["depth_weights"][2 * layer + 1]
),
}
)
layer_means = []
for layer in range(1, 33):
selected = [row for row in rows if row["layer"] == layer]
layer_means.append(
{
"layer": layer,
"group": selected[0]["group"],
"offset": selected[0]["offset"],
"normalized_gradient": average(
selected, lambda row: row["normalized_gradient"]
),
"attention_latest": average(
selected, lambda row: row["attention"]["latest"]
),
"mlp_latest": average(
selected, lambda row: row["mlp"]["latest"]
),
"attention_normalized_entropy": average(
selected,
lambda row: row["attention"]["normalized_entropy"],
),
"mlp_normalized_entropy": average(
selected, lambda row: row["mlp"]["normalized_entropy"]
),
"per_seed_normalized_gradient": [
row["normalized_gradient"] for row in selected
],
}
)
correlations = {}
for label, selected in (
("all_layers", rows),
("layers_19_28", [
row for row in rows if 19 <= row["layer"] <= 28
]),
):
gradients = [row["normalized_gradient"] for row in selected]
correlations[label] = {
"points": len(selected),
"attention_latest": pearson(
gradients,
[row["attention"]["latest"] for row in selected],
),
"mlp_latest": pearson(
gradients, [row["mlp"]["latest"] for row in selected]
),
"attention_normalized_entropy": pearson(
gradients,
[
row["attention"]["normalized_entropy"]
for row in selected
],
),
"mlp_normalized_entropy": pearson(
gradients,
[row["mlp"]["normalized_entropy"] for row in selected],
),
"attention_maximum": pearson(
gradients,
[row["attention"]["maximum"] for row in selected],
),
"mlp_maximum": pearson(
gradients, [row["mlp"]["maximum"] for row in selected]
),
}
print(
json.dumps(
{
"schema_version": 1,
"identity": "exploratory Round 05 scoping; not confirmatory Round 06",
"source_canonical_sha256": source_hashes,
"layers_19_28": layer_means[18:28],
"correlations": correlations,
},
ensure_ascii=False,
indent=2,
sort_keys=True,
)
)
if __name__ == "__main__":
main()
File diff suppressed because it is too large Load Diff
+6
View File
@@ -21,6 +21,9 @@
"check:data:deepseek-chat-task-bootstrap": "node scripts/check-deepseek-chat-task-bootstrap-crn-data.mjs", "check:data:deepseek-chat-task-bootstrap": "node scripts/check-deepseek-chat-task-bootstrap-crn-data.mjs",
"check:data:k3-attnres": "node scripts/check-k3-attnres-data.mjs", "check:data:k3-attnres": "node scripts/check-k3-attnres-data.mjs",
"check:data:k3-attnres-gradient": "node scripts/check-k3-attnres-gradient-data.mjs", "check:data:k3-attnres-gradient": "node scripts/check-k3-attnres-gradient-data.mjs",
"check:data:k3-attnres-spike": "node scripts/check-k3-attnres-spike-data.mjs",
"check:data:k3-attnres-local-path": "node scripts/check-k3-attnres-local-path-data.mjs",
"check:data:k3-attnres-forward": "node scripts/check-k3-attnres-forward-data.mjs",
"check:site": "node scripts/check-site.mjs", "check:site": "node scripts/check-site.mjs",
"check:moe-browser": "node scripts/check-moe-browser.mjs", "check:moe-browser": "node scripts/check-moe-browser.mjs",
"check:reasoning-browser": "node scripts/check-reasoning-browser.mjs", "check:reasoning-browser": "node scripts/check-reasoning-browser.mjs",
@@ -42,6 +45,9 @@
"check:deepseek-task-bootstrap-browser": "node scripts/check-deepseek-task-bootstrap-browser.mjs", "check:deepseek-task-bootstrap-browser": "node scripts/check-deepseek-task-bootstrap-browser.mjs",
"check:k3-attnres-browser": "node scripts/check-k3-attnres-browser.mjs", "check:k3-attnres-browser": "node scripts/check-k3-attnres-browser.mjs",
"check:k3-attnres-gradient-browser": "node scripts/check-k3-attnres-gradient-browser.mjs", "check:k3-attnres-gradient-browser": "node scripts/check-k3-attnres-gradient-browser.mjs",
"check:k3-attnres-spike-browser": "node scripts/check-k3-attnres-spike-browser.mjs",
"check:k3-attnres-local-path-browser": "node scripts/check-k3-attnres-local-path-browser.mjs",
"check:k3-attnres-forward-browser": "node scripts/check-k3-attnres-forward-browser.mjs",
"check:k3-browser": "node scripts/check-k3-browser.mjs" "check:k3-browser": "node scripts/check-k3-browser.mjs"
}, },
"dependencies": { "dependencies": {
@@ -0,0 +1,112 @@
# Round 08 AttnRes 训练期前向干预:结果审计
审计日期:2026-07-30
协议:`llm-atlas-k3-attnres-forward-training-v1`
结果后 Grok 会话:`019fb28b-a9e1-7643-8e43-06f5e16a2077`
## 1. 一句话结论
冻结 analyzer 的唯一主状态为:
```text
forward_training_attenuation_established_within_reduced_protocol
```
联合 `groups 6+7` 的训练期 uniform-forward 消融在三个预注册 seed 上同时通过
`spike contrast` 与 `peak / mean` 的 20% attenuation 门;逐 seed 与三 seed 平均
validation BPC 也全部通过质量护栏。结果后独立只读复算得到:
```text
blocking_errors = 0
status_confirmed = true
replay_confirmed = true
```
这只是在固定 depth-32 缩小 Block AttnRes、固定数据与 8,000-step 预算中的训练期
架构消融;**不是真实 Kimi-K3 / 2.8T checkpoint 结果,也不是 Figure 5(c) 未公开
telemetry 的复现。**
## 2. 主门复算
冻结定义:
```text
S = layers 21–25
R = other 27 layers
C = mean(g[S]) / mean(g[R])
P = max(g) / mean(g)
D = (X_reference - X_variant) / X_reference
```
主变体为 `uniform_groups_6_7_forward`,只读复算如下:
| seed | C reference | C variant | C drop | P reference | P variant | P drop | ΔBPC |
|---|---:|---:|---:|---:|---:|---:|---:|
| 2026073001 | 3.046269 | 0.806011 | 73.54% | 3.219067 | 1.435618 | 55.40% | +0.005696 |
| 2026073002 | 3.332848 | 0.758201 | 77.25% | 3.618924 | 1.375016 | 62.00% | +0.009598 |
| 2026073003 | 1.881492 | 0.712882 | 62.11% | 2.288267 | 1.548141 | 32.34% | +0.004441 |
因此 attenuation 为 `6/6`;最小 contrast drop 为 `62.11%`,最小 peak drop 为
`32.34%`,都高于冻结的 `20%` 门槛。BPC 三格都低于 `+0.05`,平均
`+0.006578` 低于 `+0.03`,质量门为 `4/4`。
这里的 contrast 下降不等价于“尖峰层被关闭”:它可能由 spike-window 分子下降、
27 层 reference 分母上升,或两者共同造成;被联合消融覆盖的 layers 26–28 仍属于
这个分母。
## 3. 完整性与复现
- 13 个新 raw 文件完整:12 formal + 1 primary replay;
- 每格 `65,536,000` target bytes,新处理总量 `851,968,000`;
- 三个历史 paired reference 合计 `196,608,000` bytes,未在 Round 08 重跑;
- 13/13 raw canonical self-hash、aggregate 与 reproduction self-hash 自洽;
- architecture / depth / steps / batch 固定为 Block / 32 / 8,000 / 32;
- initial public/mixer state、schedule、validation、diagnostic、input-gate、model、
optimizer 与 scientific environment 对同 seed historical reference 配对 exact;
- primary seed 1 的 formal / replay scientific payload exact:
`b85563ca5cb53e60b39c3801d372376206105b8a089a8633b3e81973a7f0c051`。
四个正式 selector 的 visit、source count、uniform arithmetic 与 reachability 均通过。
被选 mixer 的 query / key norm 留在 optimizer param groups,但 forward 不再调用它们:
gradient hook 为 0、Adam state 不存在、最终 tensor 与初始值 byte-exact。未选 mixer 的
optimizer-state 检查是比科学协议更强的实现审计,不参与主 status。
## 4. 描述性 non-additivity
冻结的 bookkeeping residual 为:
```text
I67 = ln(Xref / X67) - ln(Xref / X6) - ln(Xref / X7)
```
step 8,000 的三 seed 平均为:
- spike contrast:`-0.367038`
- peak / mean:`-0.170448`
它来自三套独立训练,只能描述 joint run 与两个 single runs 的 log-effect 残差;不能
写成因果 interaction、Shapley contribution 或“group 6/7 互相抑制”的机制结论。
## 5. 两阶段独立审阅
结果前 Grok 实现审阅指出 smoke-only empty selector 的 visit census 可空真。正式
4×3 路径全部是非空 selector,因此不影响 raw formal 数值;矩阵结束后已删除 early
return、加入 `forward_calls > 0`,并重跑 step-zero、parent smoke、wrapper smoke
与 equivalence。修补后的 learned wrapper 实际执行 39 次 forward,父/包装器 15 组
科学字段仍全部 exact。
结果后 Grok 在只读 sandbox 中从 13 个 raw 与三个 historical references 独立复算
identity、自哈希、selector、pairing、两项主指标、BPC、replay 与 `I67`。它报告
`0 mismatch`、`blocking_errors=0`,确认 analyzer status 与 claim boundary。
## 6. 最终 claim boundary
可以说:在这一固定缩小协议内,联合 group 6+7 的 train-time uniform-forward
architecture ablation 相对历史同 seed reference 达到预注册 attenuation + BPC 门控。
不能说:
- 已定位真实 Kimi-K3 的训练尖峰;
- 已复现 K3 报告 Figure 5(c);
- 已把 forward、natural backward 与 optimizer update 分离成纯因果效应;
- 已证明下游能力等价、总体统计显著性、可加性或因果 interaction。
@@ -0,0 +1,69 @@
# Round 08 Grok Headless 对抗审阅与处置
审阅日期:2026-07-30
审阅会话:`019fb1ce-7ffa-70e3-b860-c4a31a4c6621`
身份:**外部模型的只读方法学审稿,不是论文证据源**
## 1. 调用边界
Grok CLI 使用 single/headless + plan permission 读取:
- `research/K3_ATTNRES_FORWARD_TRAINING_SCOPING.md`
- `research/K3_ATTNRES_FORWARD_TRAINING_PROTOCOL.md`
- `experiments/k3/attnres_forward/manifest.json`
- Round 04 / 05 父 runners
关闭 web search、禁止 subagents;它没有修改文件,也没有运行训练。
## 2. Blocking findings 与处置
| finding | 风险 | 处置 |
|---|---|---|
| `mean` 与父 `softmax+einsum` 的 FP32 归约顺序不保证 byte-exact | step-0 会假失败 | **采纳**:选中路径改成参数无关 constant-zero logits,并复用同一 `softmax+einsum` kernel |
| pairing 要求了父 JSON 不存在的 initial full hash,并把 GPU 名写成 exact | 科学身份正确却因 metadata 失败 | **采纳**:只 hard-gate 初始化、数据、拓扑、optimizer 与确定性/autocast 合同 |
| replay 只排除三个字段,path 会制造假差异 | exact replay 假失败 | **采纳**:冻结 scientific canonical payload 与 path normalization |
| wrapper 若把新 protocol ID 写入 `window_start` salt,会静默换数据 | 历史逐 seed 配对失效 | **采纳**:父 salt 只由父 manifest/runner 管,新 ID 只进 wrapper output |
| 主判定没有在协议正文再写 final-only | analyzer 可能误读六 checkpoints | **采纳**:主 attenuation 与 BPC 只读 step 8,000 |
| spike layer numbering 未 machine-readable 固定 | 可能整体平移一层 | **采纳**:manifest 新增 `spike_layers_1based`,协议钉死 Python index = layer−1 |
## 3. Non-blocking findings 与处置
全部采纳:
- selected 参数留在 AdamW param groups,但因 graph 不可达而没有 state entry,不能写成
“训练了但没动”;
- group 6 / 7 indices 与 source counts 经独立复算正确;
- reference 统一称 historical paired reference;
- layers 26–28 位于 contrast 分母 `R`,必须拆报 `mean(S)` / `mean(R)`;
- `I67` 明确是三次独立训练之间的 log residual;
- missing / non-finite / structural error 统一 `contract_failed`,不能冒充科学失败;
- wrapper 禁止改变 FP32 residual、AdamW grouping、parent salt 与 empty-selector path。
## 4. 算术复核
```text
8,000 × 32 × 256 = 65,536,000 bytes / cell
12 formal cells = 786,432,000 bytes
+ primary replay = 851,968,000 newly processed bytes
historical refs = 196,608,000 bytes(不重跑)
```
selector:
```text
group 6 = 40..47; source N = 6,7,7,7,7,7,7,7
group 7 = 48..55; source N = 7,8,8,8,8,8,8,8
group 7 MLP = 49,51,53,55
groups 6+7 = 40..55
```
主 attenuation 为 `3 seeds × {contrast, peak} = 6` cells;quality 为三个
per-seed BPC gates 加一个 mean gate,共 4 项。两者合取,且只读 final step。
## 5. 复核结论
Grok 判断研究身份、矩阵、source counts 与主公式骨架可以保留;主要风险来自浮点
arithmetic identity、wrapper salt、over-exact metadata 与层号歧义。以上项目已在任何
正式输出出现前全部修订,manifest 状态随后改为 `frozen-before-model-output`。
审稿意见不会进入实验结果、论文事实或官网证据等级;它只用于结果前强化协议。
@@ -0,0 +1,89 @@
# Round 08 runner / analyzer 实现审阅与处置
审阅日期:2026-07-30
Grok 会话:`019fb1f0-bb24-7202-8123-295edda7518f`
身份:**正式文件完成前的外部模型只读实现审计,不是结果或论文证据**
## 1. 审阅边界
Grok Headless 只读检查:
- 冻结协议与 manifest;
- `train.py` / `verify.py` / `analyze.py` / `run_matrix.py`;
- 父 runner 的 salt、schema 与 DepthMixer 算术路径。
明确禁止读取 Round 08 raw formal results、运行训练、修改文件、web search 与 subagents。
## 2. 对正式 4×3 路径的确认
审阅确认:
- 新 protocol ID 没有进入父 `window_start` salt;
- selected 路径用 constant-zero FP32 logits 和父 `softmax+einsum` kernel;
- 非空 selector 的 query / key norm hook、AdamW state 与 final=initial gate 自洽;
- historical pairing 比较的字段在父 JSON 中真实存在;
- replay payload 正确排除 run kind、timing、path、parent self hash 与 GPU/version metadata,
同时保留模型、optimizer、diagnostics、history、selector 与 reachability;
- analyzer 强制 4×3 identity、step 8,000 主判定、1-based layers 21–25、finite/positive、
`D=(ref-variant)/ref`、BPC `variant-ref` 与 descriptive `I67`;
- 12 formal + 1 replay、65,536,000 bytes/cell 与最大两进程算术正确。
没有发现会改变正在运行的四个非空 formal variants 数值语义的 blocking error。
## 3. Blocking finding:smoke-only empty selector 的空审计
`learned_reference` 直接调用父 `forward`,没有递增 wrapper 的 visit counters。因此:
```text
forward_calls = 0
all depth/output visits = 0
visit_gate = all(0 == 0) = true
selected reachability = all([]) = true
```
这不会影响四个正式变体,它们全部是非空 selector;而 empty selector 另有 20-step
parent-equivalence gate,模型/optimizer/evaluations/diagnostics/history/hash 已 exact。
但 `forward_intervention.passed` 本身不能在修复前被当作 learned census 证据。
处置:
- formal matrix 完成后,删除 `selected_indices` 为空时的父路径 early return;
- 让 empty selector 也走同一 copied forward,其中 `mix` 对 64 个节点逐个调用父
`DepthMixer.forward`;
- 加入 `forward_calls > 0`;
- 重跑 step-0、parent smoke、wrapper smoke 与 parent equivalence;
- 只有 copied path 仍逐字段 exact 才保留。
这项修复只强化 smoke audit,不更改任何正式 variant 的 selector 或 forward。
## 4. Non-blocking findings 与处置
| finding | 处置 |
|---|---|
| `smoke_compare` 整体比较 `manifest.path` | 规范化 path 后比较 scientific manifest fields |
| unselected gate 额外要求 optimizer state | 保留为强实现 gate,但在文档中标成 protocol 之外的额外审计,不用于科学 status |
| GPU/version 差异只有 `metadata_equal`,没有 warnings 数组 | aggregate 增加显式 metadata warnings |
| `run_matrix` 失败后只 terminate、不 wait/kill,invalid file 会阻塞重跑 | 加入 terminate→wait→kill 清理,并在 cell failure 时标明 exact invalid target |
| step-0 CE 只比较 Python float | 追加 scalar tensor SHA-256 |
| `EXPECTED_SOURCE_COUNTS` 未使用 | 用于 runner↔manifest 交叉校验 |
上述修订不读取结果、不改变冻结阈值或主公式。
## 5. 矩阵结束后的处置结果
13 个单元全部退出后才应用上述修订;四个正式非空 selector 的 raw 文件未被重写。
修补后的前置闸门结果:
- step-zero 五个 variant 的 logits、loss tensor、evaluation 与 diagnostic exact;
- smoke-only learned wrapper 实际执行 `39` 次 forward,64 个 depth mixer 与
output mixer 的 visit census 全部非零且 exact;
- parent 与 wrapper 的 architecture、seed、schedule、model、optimizer、hash、
evaluation、diagnostic、history、gradient gate 与 scientific environment 共
15 组字段全部 exact;
- primary smoke 的 selected 参数仍为 0 hook、无 optimizer state、final=initial;
- unselected optimizer-state 条件继续作为额外实现闸门,不进入科学 status。
结果后 Grok 会话 `019fb28b-a9e1-7643-8e43-06f5e16a2077` 在只读 sandbox 中独立
复算 13 个 raw、三个 historical references 与 aggregate,报告
`blocking_errors=0`、`status_confirmed=true`、`replay_confirmed=true`。详细数字与
claim boundary 见 `research/K3_ATTNRES_FORWARD_TRAINING_AUDIT.md`。
@@ -0,0 +1,389 @@
# K3 Attention Residuals 训练期前向干预协议
协议 ID:`llm-atlas-k3-attnres-forward-training-v1`
冻结日期:2026-07-30
协议状态:**结果前预注册 frozen;任何语义变更必须更换 protocol ID**
父协议:`llm-atlas-k3-attnres-gradient-scale-v1`
## 0. 研究身份
这是 Round 07 定向线索之后的训练期架构消融。选中 depth mixer 在每一次 train / eval /
diagnostic forward 都用 source states 的算术平均,完全绕过该 mixer 的
`query + key_norm + softmax` 路径。
允许回答:
1. 固定 groups 6+7 的 uniform forward 训练变体,能否在不触发预注册 BPC 失败护栏时,
material 地降低最终固定 activation-gradient spike?
2. group 6、group 7 与 joint 的训练轨迹呈现什么非加性关系?
3. Round 07 指向的 group 7 MLP-only 路径能否独立产生 material response?
不允许回答:
- 真实 Kimi K3 2.8T checkpoint 的梯度或训练动力学;
- 论文 Figure 5(c) 未公开 telemetry 的复现;
- “forward effect” 与 natural backward/update effect 的分离;
- selected query/key 参数如果继续训练会怎样;
- 三 seed 外的总体显著性、置信区间或 p-value;
- 下游能力保持、通用质量等价或最优 AttnRes 设计;
- 单组 effects 的可加性、Shapley value、方差贡献或因果交互;
- 与 Round 07 value-coefficient intervention 同构的“纯 forward”因果复制;
- K3 `A_log` 的官方修复裁决。
## 1. 冻结训练与数据合同
| 字段 | 固定值 |
|---|---|
| architecture | Block AttnRes |
| Transformer depth | 32 |
| aggregation groups | 8 |
| blocks / group | 4 |
| depth / output mixers | 64 / 1 |
| width / heads / FFN | 192 / 6 / 768 |
| context / vocabulary | 256 / byte-256 |
| seeds | 2026073001 / 2026073002 / 2026073003 |
| steps / batch | 8,000 / 32 |
| target bytes / new formal cell | 65,536,000 |
| optimizer | AdamW |
| peak / min LR | 3e-4 / 3e-5 |
| warmup | 400 |
| weight decay | 0.1 for ndim ≥ 2 |
| betas / epsilon | 0.9, 0.95 / 1e-8 |
| clip | global norm 1.0 |
| forward | CUDA BF16 autocast |
| residual accumulation | explicit FP32 |
| validation | fixed 64 × 256-byte windows |
| diagnostic | fixed 16 × 256-byte windows |
| checkpoints | 0 / 100 / 500 / 2,000 / 4,000 / 8,000 |
| concurrency | at most two independent processes |
训练输入 schedule **逐 step 复用父协议**。`window_start` 使用父
`llm-atlas-k3-attnres-gradient-scale-v1` 的 salt;新 protocol ID 只写入 wrapper output
和 study manifest,绝不能进入 `round04.PROTOCOL_ID` 或训练窗口散列。父 manifest
负责 bytes / windows / schedule,新 manifest 只负责 variants / selector / thresholds。
每个 variant / seed 的初始化、optimizer input、validation 与 diagnostic tensors 必须
exact 相同。
runner 必须继承 Round 05 `GradientLanguageModel` 的 explicit FP32 Block residual
accumulation;不得退回 Round 04 的旧累加路径。
新正式处理量:
```text
4 variants × 3 seeds × 65,536,000 = 786,432,000 target bytes
1 primary replay 65,536,000 target bytes
total newly processed 851,968,000 target bytes
historical learned reference 196,608,000 target bytes(不重跑)
```
每格必须使用全新 Python process。最多并行两个;不能共享 model、optimizer、RNG、
CUDA graph 或 output file。
## 2. 冻结正式矩阵
正式 variants:
| variant | exact selected depth indices | layers / branch |
|---|---|---|
| `uniform_group_6_forward` | 40–47 | 21–24 / both |
| `uniform_group_7_forward` | 48–55 | 25–28 / both |
| `uniform_groups_6_7_forward` | 40–55 | 21–28 / both |
| `uniform_group_7_mlp_forward` | 49, 51, 53, 55 | 25–28 / MLP |
`learned_reference` 只允许 smoke,不进入新正式矩阵。output mixer index 64 永远 learned。
正式运行 12 格;另从初始化 replay:
```text
replay / uniform_groups_6_7_forward / seed 2026073001
```
## 3. 唯一 selector 与 forward 语义
runner 必须只有一个 machine-readable selector:
```text
selected(variant, depth_mixer_index) -> bool
```
不得把四个 variant 分叉成四份 model forward。
对未选中 depth mixer 和 output mixer,逐调用父 `DepthMixer.forward`。对选中 mixer,
为了让初始化负控制复用相同浮点归约顺序,使用参数无关的零 logits,但仍走父
`softmax + einsum` 数值 kernel:
```text
values = stack(sources, dim=0)
logits = zeros([N, batch, tokens], dtype=FP32)
weights = softmax(logits, dim=0)
output = einsum("nbt,nbtd->btd", weights, values.float()).to(values.dtype)
```
capture summary 必须仍使用父 schema:
```text
mean_weights = [1 / N] × N
entropy_mean = ln(N)
sources = N
```
选中路径不得调用 `query`、`key_norm` 或 source-dependent logits,也不得用
stop-gradient trick 让这些参数看似参与。这里保留的 softmax 只把常数零 logits 变成
`1/N`,目的是与父路径保持同一 arithmetic kernel;它没有可训练参数。自然结果是选中
mixer 的 `query` 和 `key_norm.weight`:
- gradient hook call count 必须为 0;
- optimizer state entry 必须不存在;
- final tensor 必须与 initial tensor byte-exact。
所有未选中 depth mixer 和 output mixer 的两个参数都必须有正的 gradient hook call
count;这只证明图可达,不要求它们的梯度非零或最终 tensor 一定变化。hook census
从 model 构造后开始,覆盖所有 training backward 与 diagnostic backward;eval
forward 不计 hook。selected 参数允许保留在原 AdamW param groups,但 state entry 必须
不存在,不能表述为“训练了但没有移动”。
## 4. 结果前实现闸门
### 4.1 empty-selector 父等价
`learned_reference` smoke 必须直接走父 forward,不得走常数 uniform 分支;它与父
Round 05 runner 在相同 seed / 20 steps 下必须:
- initial/final model hashes exact;
- final optimizer hash exact;
- evaluations、diagnostics 与 training history exact;
- gradient gate exact;
- input tensor hashes exact。
允许不同字段只限 protocol wrapper identity、study-manifest wrapper、timing、规范化
后的 manifest path、output path 与 self-hash。AdamW 参数分组必须保持父语义;
empty-selector 不能删除任何 mixer 参数。
### 4.2 step-0 identity negative control
父模型所有 depth mixer query 初始化为 0,因此 learned softmax 在 step 0 是 exact
uniform。选中分支用同 dtype 的 constant-zero logits 和同一个
`softmax + einsum` kernel。四个 variant 与 learned reference 在固定 input 上必须:
- logits byte-exact;
- CE byte-exact;
- activation-gradient spectrum byte-exact;
- validation metrics byte-exact;
- selected 的**归约前 weight tensor** 等于 FP32 `1/N`,max absolute error
`≤ 1e-12`;capture 的 `mean_weights` 因 FP32 大规模 mean 可有约 `1e-8` 的归约舍入,
但必须与父 learned capture summary byte-exact;
- 若任何跨 variant byte-exact 比较失败,hard-fail;不得在结果后改成容差 gate。
这个负控制只约束初始化;训练开始后 forward 必须允许分化。
### 4.3 selector census
每个 forward 的 64 个 depth index 必须各访问一次,output 访问一次且保持 learned。
每个 variant 的 selected set 必须与第 2 节 exact。group 内 source counts 必须满足:
```text
group 6: index 40 has N=6; indices 41..47 have N=7
group 7: index 48 has N=7; indices 49..55 have N=8
```
推导前提是 `completed` 含 embedding,且每 8 个 depth mixer 才把 `partial` 聚合为一个
completed group。
正式 output 保存 exact selected indices、实际 visit census、source counts 与
uniform-weight max error。任何漏访、重访、越界或 output 被选中都失败。
### 4.4 数据、有限性与梯度尺度
- 父 manifest、train/validation/diagnostic bytes 与 schedule hashes exact;
- step 0 / 1 / 7,999 optimizer input gate hashes exact;
- loss、logits、所有主 activation gradients 全部 finite;
- global clip 后每一步都执行 optimizer update,不允许 skip;
- smoke 的 diagnostic loss `×2` 时,每层 activation-gradient RMS 比值在
`2 ± 1e-5`,normalized spectrum max delta `≤ 1e-6`。
## 5. 历史 reference 配对合同
reference 固定为:
```text
experiments/k3/attnres_gradient/results/raw/
formal-depth-32-block-seed-{seed}.json
```
analyzer 的 pairing hard gates:
- `protocol_id = llm-atlas-k3-attnres-gradient-scale-v1`;
- formal / block / depth 32 / 8,000 steps / batch 32;
- seed exact;
- initial public 与 mixer hashes 跟对应新 variant exact;
- formal schedule、validation tensor、diagnostic tensor 和三个 input gate hashes exact;
- model topology、optimizer hyperparameters、CUBLAS workspace、deterministic flags 与
autocast 语义 exact。
以下字段明确**不参与 pairing equality**:
- 所有 final hashes、evaluations、diagnostics、training history 与 gradient gate;
- timing、run kind、self-hash、output path 与 manifest path 字符串;
- GPU 名称、driver / CUDA / torch version 的 minor 差异。
环境完整记录;若数值栈变化,aggregate 给出 metadata warning,但只要上述确定性与
autocast 合同相同就不将其误判为 pairing failure。父 JSON 没有 initial full-state 字段,
不得假定它存在;public ∪ mixer 的完整性只用结构/元素 census 自洽。
reference 是历史配对基线,不得写成同期随机对照。若任何合同不等,整轮 aggregate
失败,而不是降级为“近似比较”。
## 6. 固定主对象与公式
每个 diagnostic checkpoint 从:
```text
activation_grad_rms_by_block = [g1, ..., g32]
layer_id ∈ {1,...,32}
g[layer_id] = activation_grad_rms_by_block[layer_id - 1]
S = {21,22,23,24,25} # 1-based
R = {1,...,32} \ S
```
计算:
```text
C = mean(g[S]) / mean(g[R]) # spike contrast
P = max(g) / mean(g) # peak normalized
```
所有 `g`、`C`、`P` 必须 finite 且严格大于 `1e-30`。groups 6+7 还改写落在 `R`
中的 layers 26–28,所以 analyzer 同时报告 `mean(g[S])` 与 `mean(g[R])`,但不把它们
加入主 status。
对同 seed reference `X_ref` 与 variant `X_v`:
```text
D_X(v) = (X_ref - X_v) / X_ref
```
`D>0` 表示 attenuation,`D<0` 表示 amplification。不得取绝对值,不得更换分母。
## 7. 预注册判定
### 7.1 主判定
`uniform_groups_6_7_forward` 的主 attenuation gate **只读取 step=8,000**:
```text
D_C >= 0.20 AND D_P >= 0.20
for all 3 seeds
```
质量 gate 也只读取 `evaluations[step=8000].bits_per_byte`:
```text
delta_bpc(seed) = final_bpc_variant - final_bpc_reference
delta_bpc(seed) <= 0.05 for all 3 seeds
mean(delta_bpc) <= 0.03
```
只有 attenuation 6/6 与 quality 4/4 同时通过,正式 status 才是:
```text
forward_training_attenuation_established_within_reduced_protocol
```
否则按失败位置使用:
```text
attenuation_not_established
quality_guard_failed
attenuation_and_quality_failed
```
不能用次级变体补救主判定。任何 step 8,000 缺失/重复、数组长度错误、hash 不配对、
selector / reachability 失败、`g/C/P/BPC/D/log` 缺失或非 finite 都是
`contract_failed` 并让 analyzer non-zero exit;不能把结构失败包装成上面的科学状态。
### 7.2 次级 material response
group 6、group 7、group 7 MLP-only 各自使用同一 `20% / 3-seed / 2-metric` attenuation
threshold 和同一 quality guard,分别报告:
```text
secondary_material_response / secondary_response_not_established
```
它们不改变主 status,也不升级成 localization。
### 7.3 BPC 护栏的解释
`+0.05 per seed / +0.03 mean` 是预注册的 catastrophic-degradation screen:
- 失败说明不能把 spike 下降当成健康训练的证据;
- 通过不说明能力、校准或下游任务等价;
- BPC 改善也不说明总体架构更优。
## 8. 非加性交互与轨迹
对 `X ∈ {C,P}`、每个 seed、每个 checkpoint:
```text
E6 = ln(X_ref / X_group6)
E7 = ln(X_ref / X_group7)
E67 = ln(X_ref / X_groups6+7)
I67 = E67 - E6 - E7
```
保存 `E6/E7/E67/I67` 原值、对应 `D_C/D_P` 和三 seed mean/range。没有通过阈值、
p-value 或 CI。它是三套独立训练在相同 checkpoint 的跨-run log residual;
`I67` 不能写成可加贡献、独立作用、Shapley value 或因果 interaction estimate。
同时全量保存:
- 六 checkpoints 的 32-layer raw / normalized spectra;
- peak layer、top-five layers;
- validation BPC 与 train-loss trajectory;
- selected/unselected mixer weight summaries;
- selected-parameter reachability audit;
- per-cell timing 与显存(不进入数值结论)。
## 9. replay 与 analyzer 合同
primary seed-1 replay 使用 analyzer 定义的 scientific canonical payload。先删除:
```text
run_kind
timing
canonical_sha256_without_self
manifest.path
study_manifest.path
```
再比较以下固定字段 exact:protocol / variant / architecture / depth / seed / steps / batch /
target bytes、manifest scientific hashes、model、optimizer、initial/final hashes、
evaluations、diagnostics、training history、selector 与 reachability audits,以及
environment 中 deterministic / autocast scientific subset。GPU/版本 metadata 保留在
两份文件中单独展示,不进入 canonical equality。
所有主指标、阈值、status、interaction map 与 compact website artifact 只能由单一
`experiments/k3/attnres_forward/analyze.py` 生成。网站不能在 TypeScript 中重新计算
另一套结论。
analyzer 在任何结构、hash、selector、reachability、finite、reference pairing、
replay 或 threshold contract 失败时必须 non-zero exit,不得输出部分通过结论。
## 10. 报告语言红线
允许:
- “在这个固定缩小模型与训练协议内,局部 uniform-forward 变体……”
- “selected mixer 参数在此架构消融中结构性不可达……”
- “joint log effect 呈现正/负 interaction residual……”
禁止:
- “证明 K3 的训练尖峰来自 group 6/7”
- “只改变 forward,所以这是纯 forward 因果效应”
- “BPC gate 通过,所以能力不受影响”
- “interaction residual 是两个 group 的真实贡献”
- “contrast 下降证明尖峰层本身下降”(未同时检查 `S` / `R` 分拆)
- “step-0 exact 说明训练期始终与 learned forward 恒等”
- “复现了 K3 Figure 5(c)”
- “已经验证官方 2.8T checkpoint”
@@ -0,0 +1,195 @@
# K3 Attention Residuals 训练期前向干预:Round 08 前置定位
研究日期:2026-07-30
阶段身份:**Round 07 后的定向 scoping,不是 Round 08 结果**
上游协议:`llm-atlas-k3-attnres-local-path-v1`
## 1. 为什么还需要一次训练期实验
Round 06 / 07 都保持 learned forward 完全不变,只在固定 diagnostic 的 backward 中
替换 source value coefficients。它们回答的是:
- 全部 65 个 mixer 的 uniform value backward 能不能压低固定尖峰;
- group 6 / 7 的 16 个 depth mixer 在 learned 背景上是否足以复现全局下降;
- 从 all-uniform 背景恢复这些 mixer 是否能反向恢复尖峰。
Round 07 的正式结论是:
```text
groups 6+7 sufficiency:6 / 6 seed×metric cells 通过
groups 6+7 restoration:3 / 6 cells 通过
formal status:one_sided_evidence_localization_not_established
```
这已经足以排除“局部 mask 完全没有反应”,但还不能回答:
> 如果训练的每一次 forward 都真的把这段 depth routing 改成算术平均,模型会怎样适应?
Round 08 把 intervention 放进 optimizer path。它不再追求 forward-identical,而是让
选中 mixer 的输出在训练、验证与诊断中始终为所有 source states 的等权平均。
## 2. 这不是“只改变 forward”
选中 mixer 的 learned 路径原本是:
```text
keys = RMSNorm(sources)
logits = query · keys
weights = softmax(logits over source-depth)
output = Σ weights_i × source_i
```
Round 08 的选中路径是:
```text
output = (1 / N) × Σ source_i
```
因此 intervention 同时改变:
1. forward 的 branch input;
2. 由新 forward 自然产生的 source gradients;
3. 下游 activation、loss 与所有后续 optimizer updates;
4. 选中 mixer 的参数可达性:`query` 与 `key_norm.weight` 不参与图,不得到梯度。
这是一项**训练期架构消融**,不是“只改变 forward、不改变 backward”的可分离因果实验。
结果不能被翻译成 query/key 路径的纯因果效应。
## 3. 为什么只选四个新变体
固定 depth-32 / Block AttnRes 拓扑:
| scope | layers | 0-based depth mixer indices | count |
|---|---:|---:|---:|
| group 6 | 21–24 | 40–47 | 8 |
| group 7 | 25–28 | 48–55 | 8 |
| groups 6+7 | 21–28 | 40–55 | 16 |
| group 7 MLP | 25–28 | 49, 51, 53, 55 | 4 |
四个新训练变体固定为:
1. `uniform_group_6_forward`
2. `uniform_group_7_forward`
3. `uniform_groups_6_7_forward`(主变体)
4. `uniform_group_7_mlp_forward`
选择依据不是 Round 08 结果:
- joint 6+7 是 Round 07 的固定主 scope;
- 单 group 6 / 7 用来构成交互图;
- group 7 MLP-only 是 Round 07 的 branch-level 次级线索;
- output mixer 保持 learned,避免把 local depth intervention 扩成全局 readout 改写。
不加入 attention-only、output-only、all-depth 或 all-65,是为了把本轮正式算力集中在
“局部训练适应”而不是重新展开 14-mode backward census。
## 4. 冻结参考,而不是重新挑基线
paired learned reference 复用 Round 05 已公开的三个:
```text
formal / depth-32 / block / seeds 2026073001..3
```
它们与新变体使用相同:
- 初始化过程;
- 8,000-step deterministic byte schedule;
- 64 validation windows;
- 16 diagnostic windows;
- optimizer、LR schedule、BF16 forward 和 FP32 residual accumulation;
- 六个 diagnostic checkpoints。
runner 的 `learned_reference` smoke mode 必须逐字段复现父 runner 数值。正式 analyzer
还必须验证新旧 cell 的 initial public/mixer hashes、输入 gate tensor hashes、
validation tensor hash、diagnostic tensor hash 与 deterministic / autocast 数值合同。
GPU 名称、驱动 minor 与 timing 只记录,不作为配对 hard gate。
复用参考的意义是避免为已冻结、可复现的三格再消耗 196,608,000 target bytes。报告必须
把它写成**历史配对参考**,不能写成同期随机对照。
## 5. 主问题与质量护栏
主对象沿用 Round 05–07,并且主判定只读取最终 step 8,000:
```text
step = 8,000
position = post-MLP Transformer-block output
reduction = activation-gradient element RMS
S = layers 21–25
R = other 27 layers
C = mean(g[S]) / mean(g[R])
P = max(g) / mean(g)
```
对变体 `v` 和同 seed learned reference:
```text
D_X(v) = (X_ref - X_v) / X_ref
```
主变体 `uniform_groups_6_7_forward` 只有在:
- `C` 与 `P` 的 `D_X ≥ 0.20`;
- 三个 seed 的六个 cell 全部通过;
- 每个 seed 的 final validation BPC 相对 reference 不劣化超过 `+0.05`;
- 三 seed mean BPC delta 不超过 `+0.03`;
时才得到 “forward-training attenuation established within reduced protocol”。
质量阈值在结果前固定。它不是“能力保持”的通用标准,只用于阻止把明显训练失败造成的
梯度塌缩计成正结果。即使通过,也不能推出下游能力等价。
## 6. 非加性局部交互图
对每个 seed、checkpoint 和指标:
```text
E6 = ln(X_ref / X_group6)
E7 = ln(X_ref / X_group7)
E67 = ln(X_ref / X_groups6+7)
I67 = E67 - E6 - E7
```
- `I67 > 0`:joint log attenuation 大于两个 single effects 的和;
- `I67 < 0`:joint log attenuation 小于两个 single effects 的和;
- `I67 = 0`:只是在这个定义下恰好 log-additive。
`I67` 没有预注册显著性阈值,不是 Shapley value、方差分解、独立性检验或因果交互估计。
三个 effect 来自三套独立训练,它只是跨 run 的 log-attenuation residual。它的用途是把
训练轨迹中的补偿/放大关系画清楚,而不是制造一个新的“通过/失败”结论。
groups 6+7 覆盖 layers 21–28,而固定尖峰窗只到 layer 25;layers 26–28 落在 `R`。
所以 `C` 的变化可能同时来自 `S` 下降与 `R` 上升。正式结果必须把两者拆开报告,不能把
contrast 下降单独翻译成“尖峰层被关闭”。
## 7. 真实 K3 checkpoint 的同期边界
截至本轮预检,官方 Kimi-K3 Hugging Face main 仍停在 revision
`9f62e4e9fffbd0a83ddd60e1c209d828994b3569`,remote code 仍按 96 heads 初始化
`A_log`,公开 checkpoint header 仍为 `[128]`。社区 PR #144 / #150 仍是两个未合并、
语义不同的候选修复;没有官方裁决。
所以本轮不下载约 1.56 TB 权重,不声称对真实 K3 forward 做了验证。缩小实验只继承
Block AttnRes 的拓扑动机,不是 K3 checkpoint 的数值替身。
一手状态页:
- [Kimi-K3 official main](https://huggingface.co/moonshotai/Kimi-K3/tree/main)
- [main `modeling_kimi_linear.py`](https://huggingface.co/moonshotai/Kimi-K3/blob/main/modeling_kimi_linear.py)
- [community PR #144](https://huggingface.co/moonshotai/Kimi-K3/discussions/144)
- [community PR #150](https://huggingface.co/moonshotai/Kimi-K3/discussions/150)
## 8. 本轮可证伪交付
Round 08 将在查看正式结果前完成:
1. 冻结协议与 machine-readable manifest;
2. 实现一个 selector,而不是四份分叉 forward;
3. 通过 empty-selector 父等价、step-0 uniform identity、selector census、参数不可达性、
loss-scale 与输入 hash 闸门;
4. 运行 4 variants × 3 seeds × 8,000 steps;
5. 从初始化 replay 主变体 seed 2026073001;
6. 由单一 analyzer 生成主判定、质量闸门、轨迹与非加性交互;
7. 独立审阅机器可读结果;
8. 以五视图交互实验接入网站、开源并发布。
+302
View File
@@ -0,0 +1,302 @@
# K3 Attention Residuals 局部 mixer 路径:Round 07 结果审计
研究日期:2026-07-30
协议:`llm-atlas-k3-attnres-local-path-v1`
预注册 commit:`6911efc`
结果前 runner / analyzer commit:`39a9ad6`
研究身份:**受 Round 05 / 06 启发的定向 reduced-model mechanism probe**
## 0. 一句话结论
> group 6+7 的 16 个 depth mixers 在 learned 背景上的局部 uniform intervention,
> 足以复现全局 log-gap reduction 的至少一半;但从 all-uniform 背景只恢复这 16 个
> mixer 时,contrast 恢复超过一半,peak 只恢复约 35.5%–41.3%。因此本轮得到强的
> **one-sided evidence**,但没有通过预注册的双向 localization 门。
这不是一句保守套话,而是协议第 13 节的直接判定:
```text
groups 6+7 sufficiency = PASS 6 / 6
groups 6+7 restoration = FAIL 3 / 6
localization = NOT ESTABLISHED
```
## 1. 运行与输入闸门
正式网格:
| run | seed | steps | target bytes | final validation BPC |
|---|---:|---:|---:|---:|
| formal | 2026073001 | 8,000 | 65,536,000 | 1.7123525941 |
| formal | 2026073002 | 8,000 | 65,536,000 | 1.7093240656 |
| formal | 2026073003 | 8,000 | 65,536,000 | 1.7030966813 |
| replay | 2026073001 | 8,000 | 65,536,000 | 1.7123525941 |
正式三格合计 196,608,000 target bytes,含 replay 为 262,144,000。
全部通过:
- 三 seed final model-state 与 Round 06 exact;
- 三 seed final optimizer-state 与 Round 06 exact;
- 六个 validation BPC、training history、六个 parent `learned` diagnostics exact;
- step 0 / 8,000 的 `detached_learned` 与 Round 06 同名 endpoint exact;
- step 0 / 8,000 的 `uniform_all` 与 Round 06
`uniform_value_backward` endpoint exact;
- 14 modes 的 logits、loss、六位置 activations、父 mixer summaries exact;
- 65 次 selector visits 的 identity、顺序、唯一性、exact mask 与 census 全部通过;
- step-0 14-mode negative control、parent learned-vs-detached control、
`loss ×1 / ×2` scale gate 全部通过;
- seed-1 从初始化完整 replay exact。
因此后续差异来自同一 forward state 上的预注册 backward coefficient masks,不是不同训练
状态、batch、loss、activation 或 selector 漂移。
## 2. 全局端点先复现
最终 `post_mlp_state / element_rms`:
| metric | detached learned(3-seed mean) | uniform all(3-seed mean) | Round 06 per-seed mean relative drop |
|---|---:|---:|---:|
| spike contrast | 2.8065 | 0.7859 | 70.2% |
| peak / layer mean | 3.0651 | 1.8964 | 37.0% |
每个 seed、两个指标的 `G_X = ln(X_ref / X_uniform_all)` 都严格为正,raw relative
drop 6 / 6 超过 20%。global gap gate 完整成立。
这一步重要,因为局部 score 的分母不是任意“改善空间”,而是同一 seed、同一指标的
实测 global log gap。若任一 global gap 不成立,本轮主判定就必须停止;实际没有触发该
停止规则。
## 3. groups 6+7:充分性很强
只把 mixer indices 40–55 改成 uniform,其他 49 个 mixer 保持 detached learned:
| seed | `S_contrast` | `S_peak` | 50% × two metrics |
|---:|---:|---:|---:|
| 2026073001 | 0.697 | 1.817 | PASS |
| 2026073002 | 0.676 | 1.783 | PASS |
| 2026073003 | 0.658 | 1.501 | PASS |
| **mean** | **0.677** | **1.700** | **6 / 6** |
raw metric 的三 seed mean 也从 reference 的 `2.8065 / 3.0651` 变为
`1.1737 / 1.3450`。
`S_peak > 1` 不是 170% 因果贡献。它只表示在 log ratio 上,局部 uniform groups 6+7
把 peak 推得比 all-65 uniform endpoint 还低。这是很直接的 non-additivity / interaction
信号,也是协议坚持“不裁剪 score 到 [0,1]”的原因。
允许结论:
> groups 6+7 在本 diagnostic 中足以复现至少一半 global log-gap reduction。
禁止结论:
- “这 16 个 mixer 解释了 67.7% / 170.0% 的尖峰”;
- “剩下 49 个 mixer 只贡献 32.3% / −70.0%”;
- “group 6+7 是唯一原因”。
## 4. 反向 restoration 没有给出同样答案
从 all-uniform 背景出发,只把 groups 6+7 恢复为 detached-learned coefficients;
其他 49 个 mixer 仍为 uniform:
| seed | `R_contrast` | `R_peak` | 50% × two metrics |
|---:|---:|---:|---:|
| 2026073001 | 0.649 PASS | 0.372 FAIL | FAIL |
| 2026073002 | 0.621 PASS | 0.355 FAIL | FAIL |
| 2026073003 | 0.680 PASS | 0.413 FAIL | FAIL |
| **mean** | **0.650** | **0.380** | **3 / 6** |
raw metric 的三 seed mean 从 all-uniform 的 `0.7859 / 1.8964` 恢复为
`1.7693 / 2.2658`。contrast 明显朝 reference 回升,但 peak 的 log-gap recovery
没有一个 seed 达到 50%。
这说明同一个 scope 的作用强烈依赖其他 mixer 处于 learned 还是 uniform 背景:
- learned 背景中 uniformize groups 6+7,足以大幅压低 contrast 与 peak;
- uniform 背景中 restore groups 6+7,足以恢复 contrast,却不足以恢复 peak;
- 两个方向不对称,不能用单侧 sufficiency 替代双向 localization。
因此正式 verdict 是:
```text
one_sided_evidence_localization_not_established
```
不是 “almost passed”,也不因 `R_peak` mean 约 0.38 而软化 0.50 阈值。
## 5. 单 group 结果:同样显示交互
### 5.1 sufficiency
| scope | mean `S_contrast` | mean `S_peak` | 20% gate |
|---|---:|---:|---:|
| group 6 | 0.281 | 1.044 | PASS 6 / 6 |
| group 7 | 0.438 | 0.843 | PASS 6 / 6 |
两个单 group 都在两个指标、三个 seed 通过 material local sufficiency。
但:
```text
S(group6) + S(group7) ≠ S(groups6+7)
```
尤其 peak 上,两个单 group 与联合 scope 都可能超过 global endpoint,不能按 mixer
数量或 score 相加做贡献账。
### 5.2 restoration
| restored scope | mean `R_contrast` | mean `R_peak` | 20% gate |
|---|---:|---:|---:|
| group 6 | +0.248 | −0.146 | MIXED / FAIL 3 / 6 |
| group 7 | +0.409 | −0.191 | MIXED / FAIL 3 / 6 |
恢复单个 group 时,contrast 在三 seed 都超过 20%,peak 却在三 seed 全为负:相对
all-uniform,恢复一个 group 的 learned coefficients 反而让最高层 / 均值更低。
这不是 “group 没作用”,而是 effect direction 随 metric 与背景改变。它进一步反对
简单、可加的局部归因故事。
## 6. attention vs MLP:只有 group 7 过闸
### group 6
MLP-only 在 6 个 branch cells 中赢 5 个;seed 2026073003 的 contrast
`S=0.177 < 0.20`。因此:
```text
group 6 branch dominance = NOT ESTABLISHED
```
不能因为 margin 大、均值高,忽略 material threshold 的单格失败。
### group 7
| branch | mean `S_contrast` | mean `S_peak` |
|---|---:|---:|
| attention-only | 0.015 | 0.026 |
| MLP-only | 0.426 | 0.823 |
MLP-only 自身 material,且在两个指标、三个 seed 都比 attention-only 高至少
15 percentage points,因此:
```text
group 7 MLP branch-dominant at the preregistered margin
```
这是协议第 14 节的**次级、sufficiency-only、探索性**判定;没有 branch-level
restoration,不得升级为第 13 节的双向 localization。
## 7. output / depth controls
| control | mean `S_contrast` | mean `S_peak` | verdict |
|---|---:|---:|---|
| output-only(1 mixer) | 0.131 | 0.187 | 0 / 6 at 50% |
| all-depth(64 mixers) | 0.949 | 1.145 | near / beyond global endpoint |
output mixer 单独无法解释 global gap 的一半。all-depth 已复现绝大多数 contrast gap,
peak 甚至超过 all-65 endpoint;把 output 与 depth scores 相加会产生负
`interaction_residual`。该 residual 只是 bookkeeping,不预期为 0,不是统计交互检验。
## 8. 32-layer 谱的直观变化
三个 seed 的 reference peak 都在 layer 21;all-uniform peak 都迁到 layer 2。
groups 6+7 only:
- seed 1 peak → layer 5;
- seed 2 peak → layer 25;
- seed 3 peak → layer 6。
restore groups 6+7 on uniform background:
- 三 seed peak 都回到 layer 21;
- 但 peak / mean 的恢复比例仍只有 0.355–0.413。
“peak layer 回来了”与“peak 强度恢复超过一半”不是同一判据。网站会同时展示谱与
预注册 score,避免只凭最高点位置讲故事。
## 9. replay 与 artifact 链
seed 2026073001 从初始化完整重跑。排除 `run_kind`、timing 与 self canonical hash 后:
```text
formal seed1 == replay
compare SHA-256 = 7dbd15ad03fbd357c5d91e159706d63b24703722f76c492ed1dc733535d6b9cf
```
它覆盖训练状态、六 checkpoints、14-mode 两端矩阵、六位置 gradient reductions、
selector visits 与所有 gates,不只是 final BPC。
冻结物理 hashes:
| artifact | SHA-256 |
|---|---|
| manifest | `db01e92e…9139` |
| runner | `b42879e2…b03d` |
| analyzer | `e0921562…d02b` |
| packager | `6a9ada0b…9196` |
| aggregate | `bb0ec9fc…160e` |
| compact | `3bb6c158…9c08` |
| reproduction | `524a6883…5db` |
canonical hashes:
```text
aggregate b86d119cd2f106e2cbee8a35760ed3244336a2fcfeb9178ea1e7dab13fc6f215
compact 2aff9288f52d3d41bb1f59c64d9a07518ad3e2120c61615478087b24aaabd835
reproduction 6f5d98fce6446fecc966dd2675f272f2c4f0c9a39a5741fabc4ffad6852ca7f4
```
## 10. 两次 Grok Headless 审阅
### 结果前
session `019fb151-9627-76c1-b7d7-53012874f85c` 找出 selector API、output identity、
“restore to learned”歧义、parent learned 调度、20% 分母和 negative control 六个硬问题。
全部在预注册 freeze 前修正并留档。
### 结果后
session `019fb19d-94a3-7231-9a63-3a1ef33a9892` 只读对照 protocol、manifest、
analyzer 与 aggregate,独立复算代表性 `G/S/R` cells:
- 阻断实现错误:0;
- one-sided verdict:确认正确;
- 不允许修改阈值;
- 指出单 group restoration 的 `score_sign_split` reason 是跨指标汇总,因此 reason
wording 略宽;`mixed / fail` 本身仍由 C/P pass split 独立成立,主 verdict 不受影响。
Grok 是方法学审稿人,不是论文或实验事实来源;正式证据仍是冻结代码与 raw outputs。
## 11. `A_log` 工件边界同步更新
本轮仍不是 K3 2.8T checkpoint forward。截止 2026-07-30 12:35 CST:
- official main 仍是 `9f62e4e9fffbd0a83ddd60e1c209d828994b3569`,96 vs 128
mismatch 未修;
- community PR #144 把 parameter 改成 128,但没有独立 forward 验证;
- community PR #150 保留 96,并在加载时验证 / 裁掉 32 个全零尾项;提交者报告检查
69 层并完成 disk-offloaded end-to-end generation;
- 两个 PR 都未合并,Moonshot 尚未给出官方裁决。
因此“没有任何公开候选解释”已经过时;“官方 contract 已解决”同样不成立。
## 12. 最强允许结论
可以说:
> 在本缩小 Block AttnRes 模型的同前向 diagnostic backward 中,global learned-value
> coefficient sensitivity 对 groups 6+7 的局部 uniformization 具有强 sufficiency;
> restoration 只在 contrast 上超过一半,在 peak 上稳定不足一半,因此预注册的双向
> localization 未建立。group 7 的 MLP mixer 在次级 sufficiency-only branch 判定中占优。
不能说:
- “证明 K3 的尖峰来自 group 6 / 7”;
- “groups 6+7 解释了 67.7% / 170.0%”;
- “MLP 是唯一原因”;
- “只要把这些 mixer 训练成 uniform 就会更稳定”;
- “复现了 K3 Figure 5(c)”;
- “community PR #150 已经是官方 `A_log` 修复”。
@@ -0,0 +1,64 @@
# Round 07 Grok Headless 对抗审阅与处置
审阅日期:2026-07-30
审阅会话:`019fb151-9627-76c1-b7d7-53012874f85c`
身份:**外部模型的只读方法学审稿,不是论文证据源**
## 1. 调用边界
Grok CLI 使用 single/headless 方式读取:
- `research/K3_ATTNRES_LOCAL_PATH_SCOPING.md`
- `research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md`
- `experiments/k3/attnres_spike/train.py`
关闭 web search、禁止 subagents、使用 plan permission;它没有修改文件。审稿任务是找
selector、公式、endpoint exactness、negative control、replay 与父 runner API 的冲突。
## 2. Blocking findings 与处置
| finding | 风险 | 处置 |
|---|---|---|
| 父 runner 只有 global mode,局部 selector 尚不可实现 | 不同实现可能选错 indices 或混入 full autograd | **采纳**:协议冻结 14 个 exact identity sets;所有矩阵 mode 只用同一个 `RoutedSourceBackward` |
| 新 output `index=64` 与父 summary 的 `mixer_index=null` 冲突 | 改 schema 会破坏 Round 06 summary hash | **采纳**:父 `trace.mixers` 不动;新建平行 `selector_visits`,64 只作为新 schema alias |
| “restore to learned” 会被误解成恢复 query/key/softmax autograd | 分母是 detached learned,实验会回答不同问题 | **采纳**:统一改为 “restore to detached-learned value coefficients” |
| 14-mode 合同没明确保留六个 parent `learned` diagnostics | 可能削弱或破坏 Round 06 exactness | **采纳**:六 checkpoints 跑 parent learned;0/8000 追加 14-mode matrix |
| 20% raw drop 没写明确分母 | gap gate 可能用不同公式 | **采纳**:冻结 `(X_ref-X_all)/X_ref`,并定义缺失格如何使 3×2 主 gate 失败 |
| step-0 只做 custom-mode 互比不够 | 同一种 surrogate 错误可全体一致 | **采纳**:双 endpoint exact、learned-vs-detached 负控制、detached `×1/×2` loss-scale gate |
## 3. Non-blocking findings 与处置
全部采纳:
- 统一符号为 `X_ref = X_detached_learned`;
- 在 protocol 正文列出 exact indices,而不只依赖 scoping 公式;
- branch dominance 标为 sufficiency-only 次级探索;
- interaction residual 明确“不预期为 0、不是检验”;
- 明示 group 7 包含固定 `S` 外的 layers 26–28;
- 精确定义 `mixed`;
- replay 限定 same-host environment;
- 所有 score / gate 只由单一 `analyze.py` 生成。
“单 group 20% 与 joint 50% 不按 mixer 数量成比例”保留为预注册决策;协议已禁止把它解释为
per-mixer rate。
## 4. 复核结论
Grok 复核 14-mode arithmetic:
```text
1 reference + 10 sufficiency + 3 restoration = 14
```
uniform census `0 / 1 / 4 / 8 / 16 / 49 / 57 / 64 / 65` 与 64 depth + 1 output
拓扑一致。真正风险是 identity/schema,不是计数;本轮修订已把两者写成独立 exact gate。
最终公式被审阅为内部自洽:
```text
G_X = ln(X_ref / X_all)
S_X(m) = ln(X_ref / X_m) / G_X
R_X(r) = ln(X_r / X_all) / G_X
```
审稿意见不会进入实验结果、论文事实或官网证据等级;它只用于在结果出现前强化协议。
+539
View File
@@ -0,0 +1,539 @@
# K3 Attention Residuals 局部 mixer 路径干预协议
协议 ID:`llm-atlas-k3-attnres-local-path-v1`
冻结日期:2026-07-30
协议状态:**结果前预注册 frozen;任何语义变更必须更换 protocol ID**
父协议:`llm-atlas-k3-attnres-spike-path-v1`
## 0. 研究身份
本轮是 Round 06 结果后的**定向机制追踪**,不是盲发现。
已知:
- depth-32 / Block 的固定尖峰集合是 layers 21–25;
- 全部 65 个 mixer 的 source value backward coefficients 从 learned 改成 uniform,
在三 seed 上 material 地降低最终 `post_mlp_state` 的 spike contrast 和 peak;
- 该全局干预保持 forward exact;
- group 6 覆盖 layers 21–24,group 7 覆盖 layers 25–28。
未知:
- 全局下降是否主要集中在 group 6 / 7 的 16 个 depth mixers;
- group 6 与 group 7 是否各自有稳定 effect;
- attention 与 MLP mixer 是否能在预注册阈值下区分;
- output mixer 是否解释了大量全局 effect;
- 局部 sufficiency 与反向 restoration 是否给出一致证据。
前置证据、拓扑和同期 artifact audit 固定在
`research/K3_ATTNRES_LOCAL_PATH_SCOPING.md`。任何结果不得倒写成事前未知。
## 1. 允许回答的问题
1. 在相同模型、batch、loss、activation 与 learned forward weights 下,只改变某个固定
mixer scope 的 source-gradient coefficients,能复现多少全局 log gap?
2. 从 all-uniform 背景只恢复 group 6 / 7 的 learned coefficients,能恢复多少
global log gap?
3. groups 6+7 是否在 sufficiency 与 restoration 两个方向、两个 spike 指标、三 seed
同时通过 50% 阈值?
4. 单独 group 6 或 group 7 是否在两个指标、三 seed 通过 20% 阈值?
5. attention-only 与 MLP-only 是否达到预注册的 branch dominance 规则?
6. output-only 与 all-depth 控制是否显示 effect 主要来自最终 output mixer?
## 2. 明确不回答的问题
- Kimi K3 2.8T checkpoint 的真实训练梯度;
- 论文 Figure 5(c) 未公开 telemetry 的精确定义;
- 哪个 layer、source 或 operator “产生”尖峰;
- learned source weight 的语义归因;
- 重新训练局部 uniform variant 的最终能力;
- intervention effect 的可加性、Shapley value 或方差分解;
- 三 seed 外的总体显著性、置信区间或 p-value;
- K3 `A_log` 两个社区修复中哪个已经得到官方认可;
- checkpoint conversion、推理正确性或部署可用性。
## 3. 冻结训练与数据合同
完整复用 Round 06:
| 字段 | 固定值 |
|---|---|
| architecture | Block AttnRes |
| Transformer depth | 32 |
| aggregation groups | 8 |
| blocks / group | 4 |
| depth mixers / output mixers | 64 / 1 |
| width / heads / FFN | 192 / 6 / 768 |
| context / vocabulary | 256 / byte-256 |
| seeds | 2026073001 / 2026073002 / 2026073003 |
| steps / batch | 8,000 / 32 |
| target bytes / formal cell | 65,536,000 |
| optimizer | AdamW |
| peak / min LR | 3e-4 / 3e-5 |
| warmup | 400 |
| weight decay | 0.1 for ndim ≥ 2 |
| betas / epsilon | 0.9, 0.95 / 1e-8 |
| clip | global norm 1.0 |
| forward | CUDA BF16 autocast |
| residual accumulation | explicit FP32 |
| diagnostic CE | fixed 16 × 256 token-mean FP32 CE |
| parent diagnostic steps | 0 / 100 / 500 / 2,000 / 4,000 / 8,000 |
| local matrix steps | 0 / 8,000 |
训练路径必须逐调用父 runner 原始 forward;局部 custom autograd 只能存在于 optimizer
step 外的 diagnostic。数据 bytes、schedule、validation tensor、diagnostic tensor 与三
seed 的每个 optimizer input 都必须与 Round 06 exact。
诊断调度明确分成两条:
- 六个 parent diagnostic steps 都运行原始 `learned` mode,用于 Round 06 等价;
- step 0 / 8,000 另外运行下述 14-mode local matrix;
- localization 公式只读取 14-mode matrix,绝不把 full-autograd `learned` 混入分母。
## 4. 正式矩阵与 replay
正式运行:
```text
depth-32 / block / seed-2026073001
depth-32 / block / seed-2026073002
depth-32 / block / seed-2026073003
```
另从初始化完整重跑:
```text
replay / depth-32 / block / seed-2026073001
```
正式三格处理 196,608,000 target bytes;含 replay 共 262,144,000 bytes。每格使用
全新 Python 进程。最多并行两个进程;不能共享 model、optimizer、RNG 或 CUDA graph。
wall-time 不进入数值复现合同。
## 5. 固定主对象与指标
主对象固定为最终 step 8,000:
```text
position = post_mlp_state
reduction = element_rms
S = layers 21, 22, 23, 24, 25
R = other 27 layers
```
对 mode `m`:
```text
C_m = mean(metric[S]) / mean(metric[R]) # spike contrast
P_m = max(metric) / mean(metric) # peak normalized
```
`C_m` 与 `P_m` 必须 finite 且严格大于 `1e-30`。后文统一用 `X_ref` 表示
`X_detached_learned`;不用 `detached_reference` 等其他别名。不允许用其他位置、reduction、layer
集合或 metric 替换主对象。32-layer raw spectrum、normalized spectrum、peak layer 和
top-five layers 全量报告,但不参与主阈值。
## 6. 14 种冻结模式
所有模式调用同一个 parent learned forward。custom Function 的 forward 直接返回
parent output,只有 backward 对 source tensors 使用选定 coefficients。
14-mode matrix **全部**使用 `RoutedSourceBackward`:每个 mixer 只在
`stopgrad(w)` 与 `1/N` 两种 source value coefficients 中选择。完整
query / key / softmax autograd 的 `learned` 不属于这 14 种模式,只用于训练和父诊断
等价。实现不得把某个 restoration scope 切回 full-autograd `learned`。
### 6.1 reference
`detached_learned`
- 65 个 mixer 全部使用 learned `w` 作为 source value backward coefficients;
- `w` detach,不走 query / key / softmax derivative path;
- 必须 exact reproduce Round 06 的同名 mode。
### 6.2 learned 背景上的局部 uniform:sufficiency family
未选 mixer 使用 detached learned coefficients;选中 mixer 使用 `1/N`:
1. `uniform_group_6_only`
2. `uniform_group_7_only`
3. `uniform_groups_6_7_only`
4. `uniform_group_6_attention_only`
5. `uniform_group_6_mlp_only`
6. `uniform_group_7_attention_only`
7. `uniform_group_7_mlp_only`
8. `uniform_output_only`
9. `uniform_depth_all`
10. `uniform_all`
`uniform_all` 必须 exact reproduce Round 06 的 `uniform_value_backward`。
### 6.3 all-uniform 背景上的 detached-learned restoration family
选中 scope 恢复 detached learned coefficients,其余保持 uniform:
1. `uniform_except_group_6`
2. `uniform_except_group_7`
3. `uniform_except_groups_6_7`
名字中的 `except` 表示该 scope **不是 uniform**。报告和网站必须同时展示人话标签
“restore ... to detached-learned value coefficients”,避免误读。
## 7. selector 的唯一合同
为了同时满足新 selector audit 和 Round 06 endpoint exactness,保留两个互不混写的
schema:
1. 父 `trace.mixers` summary **逐字段不变**;output 仍使用父 schema 的
`mixer_index=null`,并继续参与父 `mixer_summary_sha256`;
2. 新增平行 `selector_visits`,只用于 local mask audit,不写入父 summary。
`selector_visits` 的 depth mixer identity 用:
```text
(kind="depth", index=0..63, layer=1..32,
group=1..8, branch in {"attention","mlp"})
```
`selector_visits` 的 output mixer identity 用:
```text
(kind="output", index=64, layer=null, group=null, branch="output")
```
这里 `index=64` 只是新 selector schema 的稳定别名,不得回写父 summary。
令 `D_i` 表示 `kind=depth,index=i`,`O` 表示 output。14 种 mode 的 uniform identity
集合冻结如下:
| mode | exact uniform set |
|---|---|
| `detached_learned` | `∅` |
| `uniform_group_6_only` | `{D40,…,D47}` |
| `uniform_group_7_only` | `{D48,…,D55}` |
| `uniform_groups_6_7_only` | `{D40,…,D55}` |
| `uniform_group_6_attention_only` | `{D40,D42,D44,D46}` |
| `uniform_group_6_mlp_only` | `{D41,D43,D45,D47}` |
| `uniform_group_7_attention_only` | `{D48,D50,D52,D54}` |
| `uniform_group_7_mlp_only` | `{D49,D51,D53,D55}` |
| `uniform_output_only` | `{O}` |
| `uniform_depth_all` | `{D0,…,D63}` |
| `uniform_all` | `{D0,…,D63,O}` |
| `uniform_except_group_6` | `{D0,…,D39,D48,…,D63,O}` |
| `uniform_except_group_7` | `{D0,…,D47,D56,…,D63,O}` |
| `uniform_except_groups_6_7` | `{D0,…,D39,D56,…,D63,O}` |
runner 必须把这些 set 编码为一个 frozen selector 函数;不能散落在 mode-specific
if/else 中。manifest 同时保存 machine-readable exact index lists。runner 通过
override `_mix` 或等价 hook 做 set lookup,并替换父 runner 中只接受三种 global mode
的 mode validation、bundle loop 和相关 gate;训练 forward 继续直接调用父路径。
每次 forward 必须验证:
- exactly 65 个 mixer visits;
- identity 不重复;
- identity 顺序与 reference exact;
- 父 `trace.mixers` schema 与 hash 路径没有新字段;
- uniform census 与 manifest exact;
- selected identity list 与 selector rule exact;
- reference 的 uniform count 为 0;
- branch-only 4,group-only 8,groups 6+7 为 16;
- output-only 1,all-depth 64,all 65;
- except-one-group 57,except-two-groups 49。
任一 gate 失败,cell invalid;不得只改结果 JSON。
## 8. forward identity 与 parent exactness
### 8.1 所有 14 模式的 forward identity
同 seed / step 相对 `detached_learned` 必须满足:
- logits tensor SHA-256 exact;
- loss FP32 value exact;
- 六位置 activation tensor hashes exact;
- 65 个 mixer forward summaries exact。
任一 mode 失败,整格 invalid。
### 8.2 Round 06 endpoint exactness
对 step 0 / 8,000:
- `detached_learned` 的 logits、loss、六位置 activation、未改 schema 的 mixer
summaries 和六位置 gradient reductions 必须与对应 Round 06
raw output exact;
- `uniform_all` 的同一组字段和六位置 gradient reductions必须与对应 Round 06
`uniform_value_backward` exact;
- 正式训练的 final model hash、optimizer hash、六个 validation BPC、training
history 与六个 parent `learned` diagnostics 必须与 Round 06 exact;
- 新 `selector_visits` 不参与旧 `mixer_summary_sha256`,而由独立 canonical hash
和 exact-set gate 管理。
runner / protocol / scoping / manifest 物理 hash 在运行前冻结。父 raw 文件同时检查 physical
SHA-256、canonical SHA-256、final model hash 和 final optimizer hash。
## 9. 初始化负控制
step 0 的 mixer query 为零,learned `w` 是 uniform。14 模式在六个位置的
`element_rms` 必须:
- 32 个 raw values 全部 finite、strictly positive;
- 相对 reference 的逐层 raw relative error `≤1e-6`;
- normalized absolute error `≤1e-6`。
此外:
- `detached_learned` 与 `uniform_all` 必须分别与 Round 06 step-0 endpoint exact;
- parent full-autograd `learned` 与 `detached_learned` 必须按 Round 06 负控制在
`1e-6` tolerance 内一致;
- `detached_learned` 另执行同一 loss 的 `×1 / ×2` backward,六位置、七 reductions
都必须通过父协议相同的 scale 与 normalized-spectrum gate。
失败表示 selector 或 surrogate 没有隔离预期路径;正式结果无效。
## 10. global log gap
对每个 seed 和每个指标 `X ∈ {C,P}`:
```text
G_X = ln(X_ref / X_uniform_all)
relative_drop_X = (X_ref - X_uniform_all) / X_ref
```
只有同时满足以下条件才允许解释局部比例:
1. `G_C > 0` 且 `G_P > 0`;
2. 上式 `relative_drop_X ≥0.20`;
3. Round 06 endpoint exactness 通过。
seed `s` 的 metric `X` 任一条件不满足,则该 `(s,X)` 称为
`global gap not established`,不计算该格 `S_X / R_X`。groups 6+7 的主 gate 要求
3 seed × 2 metrics 全部存在,因此任一 required cell 缺失都会使主 localization
判定失败;仍公开 raw matrix,不使用事后替代分母。
log ratio 用于让相同的乘法变化在两个方向可比。所有归一化值按原值报告,**不裁剪到
[0,1]**;负值表示反方向,超过 1 表示局部 intervention 超过 all-uniform endpoint。
## 11. sufficiency score
对 sufficiency mode `m`:
```text
S_X(m) = ln(X_ref / X_m) / G_X
```
### 11.1 groups 6+7 主判定
只有 `uniform_groups_6_7_only` 对 `C` 和 `P` 都满足:
```text
S_X(m) ≥ 0.50
```
且三个 formal seed 6 / 6 全部达标,才记为:
> groups 6+7 的 16 个 depth mixers 在本 diagnostic 中,足以复现至少一半
> all-65 uniform intervention 的预注册 log-gap reduction。
任一失败记为 `not sufficient at the preregistered 50% threshold`。`mixed` 精确定义为:
seed 通过/失败不一致、`C/P` 通过/失败不一致,或 score 的正负号跨 seed 不一致;可同时
附加多个原因,不得降低阈值。
### 11.2 单 group
group 6 / group 7 分别对 `C` 和 `P`、三 seed 全部满足:
```text
S_X(m) ≥ 0.20
```
才称为 `material local sufficiency at the 20% threshold`。没过阈值不等于 effect 为零。
## 12. restoration score
对 restoration mode `r`:
```text
R_X(r) = ln(X_r / X_uniform_all) / G_X
```
### 12.1 groups 6+7 主判定
只有 `uniform_except_groups_6_7` 对 `C` 和 `P`、三 seed全部满足:
```text
R_X(r) ≥ 0.50
```
才称为:
> 从 all-uniform 背景只恢复 groups 6+7 的 learned coefficients,恢复了至少一半
> 预注册 global log gap。
这仍是同前向 backward-rule restoration sensitivity,不是严格 causal necessity。
### 12.2 单 group
`uniform_except_group_6` / `uniform_except_group_7` 分别以 `≥0.20`、两个指标、三 seed
作为 material restoration threshold。
## 13. localization 总闸门
只有以下两项同时通过:
1. groups 6+7 sufficiency:`S_C,S_P ≥0.50`,3 / 3 seeds;
2. groups 6+7 restoration:`R_C,R_P ≥0.50`,3 / 3 seeds;
才允许写:
> 在本缩小模型、固定训练状态和 diagnostic backward 下,全局 value-coefficient
> sensitivity 的主要部分 localization 到 group 6 / 7 mixer path。
即使通过,也必须紧邻注明:
- “主要部分”由 50% 双向阈值定义;
- effect non-additive;
- 不是唯一来源或 layer-origin;
- 不是真实 K3 checkpoint 结论。
一侧通过一侧失败,统一写成 `one-sided evidence, localization not established`。
## 14. attention vs MLP branch 判定
每个 group 独立比较 attention-only 与 MLP-only sufficiency score。只有某 branch:
1. `S_C ≥0.20` 且 `S_P ≥0.20`;
2. 在 `C` 与 `P` 上都比 sibling 高至少 `0.15`;
3. 三 seed 全部满足前两项;
才称为 `branch-dominant at the preregistered margin`。
若 group-level sufficiency 未通过 20% 阈值,不允许宣称其内部 branch dominance。
branch-only scores 可能交互、超加或相互抵消,不能相加成 group score。
本节只有 sufficiency 方向,没有 branch-level restoration,属于预注册的次级探索性
判定,证据层级低于 §13 双向 localization。
## 15. output 与 depth 控制
`uniform_output_only` 和 `uniform_depth_all` 不进入 group localization 主判定。
探索性报告:
```text
S_X(output)
S_X(depth_all)
interaction_residual_X =
1 - S_X(output) - S_X(depth_all)
```
`interaction_residual` 只是 log-gap bookkeeping,不是统计交互估计或贡献分解。
它不预期接近 0,也不是 hypothesis test。
只有 output-only 对两个指标、三 seed 都 `≥0.50`,才标记
`output mixer alone captures at least half the global gap`。即使如此,也不否定
groups 6+7;两者可能重叠、串联或超加。
## 16. 报告顺序与反 cherry-picking
固定报告顺序:
1. input / parent / endpoint exactness;
2. step-0 negative control;
3. 每 seed 的 raw `C` / `P` 矩阵;
4. global gaps;
5. groups 6+7 sufficiency;
6. groups 6+7 restoration;
7. localization gate;
8. single-group scores;
9. branch scores;
10. output / depth controls;
11. full 32-layer spectra;
12. replay;
13. limitations。
所有 14 modes、两个指标、三个 seed 都公开。不得只展示通过阈值的 scope。不得用跨 seed
均值替代 3 / 3 gate;均值只用于视觉摘要。
## 17. replay 与复现闸门
seed 2026073001 从初始化独立 replay,比较去除以下字段后的 canonical content:
- `run_kind`;
- wall-clock timing;
- output path;
- self canonical hash。
至少以下字段必须 exact:
- input tensor hashes;
- initial/final model 与 optimizer hashes;
- evaluations / training history;
- parent diagnostics;
- 14-mode step-0 / step-8,000 forward hashes;
- selector census / identities;
- 六位置 raw gradient reductions;
- global gaps / local scores / gates。
若正式 seed1 与 replay 不 exact,Round 07 数值结论无效。
replay 固定在与 formal 相同 host、GPU、Python、PyTorch、CUDA 和
`CUBLAS_WORKSPACE_CONFIG` 环境;本协议不声称跨硬件 bit exact。
## 18. 预期失败与停止规则
以下任一项使 cell invalid:
- CUDA deterministic contract 未开启;
- parent manifest / runner / protocol / scoping / raw hash 不匹配;
- 训练等价失败;
- diagnostic 改变 optimizer state;
- forward identity 失败;
- selector identity / census 失败;
- Round 06 endpoint exactness 失败;
- step-0 negative control 失败;
- raw gradient missing、non-finite 或 non-positive;
- global gap denominator 不成立。
程序错误修复必须:
1. 保存失败日志;
2. 修改 runner;
3. 更新 runner hash;
4. 明确判断协议语义是否改变;
5. 若改变 selector、mode、metric、threshold 或 aggregation,创建新 protocol ID;
6. 全部受影响 cell 从初始化重跑。
## 19. 结果语言边界
允许:
- “在同前向 diagnostic backward 下,uniformizing scope X 改变了固定尖峰指标”;
- “groups 6+7 在预注册 50% 双向阈值下建立 / 未建立 localization”;
- “branch effect mixed / below threshold”;
- “这是一项 reduced-model mechanism probe”。
禁止:
- “证明 K3 的尖峰来自第 6 组”;
- “这些 mixer 贡献了 X% 梯度”;
- “group effect 加总为 100%”;
- “uniform mixer 更适合训练”;
- “复现了 Figure 5(c)”;
- “验证了 K3 2.8T checkpoint”;
- “社区 PR #144 或 #150 已成为官方修复”。
## 20. 冻结清单
在任何 formal 结果产生前必须完成:
- [x] scoping 文件完成;
- [x] protocol 状态改为 frozen;
- [ ] 14 modes 与 selector census 写入 manifest;
- [ ] thresholds / formulas 写入 manifest;
- [ ] Round 06 父 artifact physical / canonical hashes 写入 manifest;
- [ ] runner、protocol、scoping、manifest hashes 固定;
- [x] Grok Headless 对抗审阅完成,采纳/拒绝理由留档;
- [ ] step-0 smoke 全门通过;
- [ ] formal 命令与环境写入 README;
- [ ] 单一 `analyze.py` 实现所有 score / gate,网站只消费其冻结输出;
- [ ] protocol commit 早于 formal result commit。
+197
View File
@@ -0,0 +1,197 @@
# K3 Attention Residuals 局部 mixer 路径:Round 07 前置定位
研究日期:2026-07-30
阶段身份:**定向 scoping,不是 Round 07 预注册结果**
上游协议:`llm-atlas-k3-attnres-spike-path-v1`
## 1. 已知到什么程度
Round 06 在同一个 depth-32 / Block AttnRes 缩小模型上,把前向保持为 learned
weights,只改写 mixer 的反向规则。三 seed 的最终 `post_mlp_state / element_rms`
结果为:
| backward rule | spike contrast(3-seed mean) | 相对 detached learned | peak normalized(3-seed mean) | 相对 detached learned |
|---|---:|---:|---:|---:|
| learned | 2.754 | — | 4.803 | — |
| detached learned | 2.812 | reference | 4.956 | reference |
| uniform value / all 65 mixers | 0.837 | **−70.2%** | 3.122 | **−37.0%** |
其中:
- `learned → detached learned` 没有降低尖峰,contrast 反而平均增加约 2.0%;
- `detached learned → uniform value backward` 在 contrast 和 peak 上都 3 / 3 seed
超过预注册的 20% material threshold;
- 三种模式的 logits、loss、六位置 activation 和 mixer forward 摘要全部 exact;
- 这证明的是**全局 backward-rule sensitivity**,不是训练变体,也不是局部归因。
因此 Round 07 不再重复问“learned value coefficients 是否重要”,而是问:
> 65 个 mixer 全局改写带来的下降,主要能否由尖峰邻近的 group 6 / 7
> depth mixers 复现,并能否从反方向恢复?
## 2. 固定拓扑,而不是结果后挑层
depth 32 的 Block AttnRes 有 8 个 aggregation groups,每组 4 个 Transformer
blocks。每层有 attention 和 MLP 两个 depth mixers,合计 64 个;模型末尾还有一个
独立 output mixer,合计 65 个 intervention nodes。
对 1-based layer `l`:
```text
group = floor((l - 1) / 4) + 1
attention mixer index = 2 × (l - 1) # 0-based
MLP mixer index = 2 × (l - 1) + 1 # 0-based
```
所以:
| scope | layers | 0-based depth mixer indices | mixer count |
|---|---:|---:|---:|
| group 6 | 21–24 | 40–47 | 8 |
| group 7 | 25–28 | 48–55 | 8 |
| groups 6+7 | 21–28 | 40–55 | 16 |
| output | — | separate node | 1 |
Round 05 已在看过数据后冻结尖峰集合 `S = layers 21–25`。它覆盖完整 group 6 和
group 7 的首层。因此 Round 07 明确是**定向邻域追踪**,不能称为盲发现;group 6 / 7
也不能结果后替换成更好看的范围。
## 3. 为什么需要两个方向
只在 learned 背景把 group 6 / 7 改成 uniform,回答的是:
> 只改这段是否足以复现全局干预的一大部分下降?
但 mixer 路径有串联、分流和 nonlinear interaction,单侧结果可能被其他 learned
路径补偿。反过来,在 all-uniform 背景只把 group 6 / 7 恢复为 learned,回答的是:
> 只恢复这段是否足以让尖峰朝 reference 回升?
两种值都不是“贡献百分比”,也不要求相加为 100%。Round 07 用相同的 global log gap
归一化两种方向,只把双向、跨 seed 稳定的结果称为 localization evidence。
## 4. 冻结候选范围
从 `detached_learned` 背景出发的 sufficiency scopes:
1. group 6;
2. group 7;
3. groups 6+7;
4. group 6 attention-only;
5. group 6 MLP-only;
6. group 7 attention-only;
7. group 7 MLP-only;
8. output-only;
9. all 64 depth mixers;
10. all 65 mixers。
从 all-uniform 背景出发的 restoration scopes:
1. restore group 6 to detached-learned value coefficients;
2. restore group 7 to detached-learned value coefficients;
3. restore groups 6+7 to detached-learned value coefficients。
加上 `detached_learned` reference,共 14 种模式。预期 uniform selector census 为:
| mode | uniform mixers |
|---|---:|
| detached reference | 0 |
| group branch only | 4 |
| group only | 8 |
| groups 6+7 | 16 |
| output only | 1 |
| all depth | 64 |
| all | 65 |
| uniform except group 6 / 7 | 57 |
| uniform except groups 6+7 | 49 |
每次 diagnostic 都必须保存实际选中的 mixer identity;不能只信 mode 名称。
group 7 的 scope 包含 layers 26–28,它们不在固定尖峰集合 `S=21–25` 中,所以
group 7 是预先定义的**完整邻接 group intervention**,不是 spike-layer-only
intervention。
## 5. 不把局部 intervention 误译成什么
即使 groups 6+7 双向通过,结论也只限于:
- 固定训练状态;
- 固定 diagnostic batch 与 loss;
- 固定 `S = 21–25` 指标;
- 同前向、替代 source-gradient coefficient 的 diagnostic backward。
它不等于:
- 这些层“产生”了尖峰;
- group 6 / 7 是唯一原因;
- 真实 K3 checkpoint 有相同梯度路径;
- 把 mixer 训练成 uniform 会有同样结果;
- 局部 effect 可加,或可解释成方差分解;
- 论文 Figure 5(c) 的未公开 telemetry 已被复现。
保留 output-only 和 all-depth 两个控制,是为了看清最终 readout 与 depth path 的关系;
它们不进入 group 6 / 7 localization 的主判定。
## 6. 同期 artifact 状态审计:`A_log`
这一问题与缩小实验的局部梯度机制**相互独立**,但会限制任何真实 K3 checkpoint
验证,因此在冻结 Round 07 前重新检查官方模型仓库。
截至 **2026-07-30 12:35 CST**:
- 官方 Hugging Face main commit 仍为
`9f62e4e9fffbd0a83ddd60e1c209d828994b3569`;
- main 的 `modeling_kimi_linear.py` 仍以 `num_heads=96` 初始化 `A_log`;
- 已发布 checkpoint 中该张量的公开 shape 是 `[128]`,与 main 存在加载不匹配;
- 官方 main 尚未合并修复或给出 conversion contract。
同时出现了两个**未合并、互相竞争的社区 PR**:
### PR #144:把参数改成 128
- 一行把初始化从 `self.num_heads` 改为 `self.head_dim`;
- 提交者报告所有 shards 能加载;
- 提交者明确说没有独立验证 forward;
- 它把 checkpoint shape 当作权威语义。
### PR #150:保留 96,加载时验证并裁零尾
- 保持模型参数为 `[num_heads]=[96]`;
- `_load_from_state_dict` 检查 `[96:128]` 全为零后再裁掉;
- 提交者报告检查了 69 个 KDA 层,所有 32 项尾部都 exact zero;
- 提交者还报告经过 disk-offloaded MoE 的完整生成;
- 这些 checkpoint 全量扫描和生成是**提交者报告**,本项目没有下载约 1.56 TB
权重独立复核;本项目只核对了 PR diff、main 代码路径和 PR 状态。
PR #150 进一步指出,forward 中 `v` 被 reshape 为 96 heads,kernel 随后接收
`A_log`;若直接采用 #144 的 128 元素参数,现有 `view(H, 1)` 路径会在 96 heads 下
失败。这个论证比单看 checkpoint shape 更完整,但在官方合并或独立复核前,仍必须标成
高可信社区解释,而不是 Kimi 官方结论。
当前准确状态应写成:
> official main 仍然不匹配;社区已有两个竞争性候选修复,其中 #150 提供了更完整的
> checkpoint-tail 与 end-to-end 证据,但尚无官方裁决。
来源:
- [Kimi-K3 official main](https://huggingface.co/moonshotai/Kimi-K3/tree/main)
- [main `modeling_kimi_linear.py`](https://huggingface.co/moonshotai/Kimi-K3/blob/main/modeling_kimi_linear.py)
- [community PR #144](https://huggingface.co/moonshotai/Kimi-K3/discussions/144)
- [community PR #150](https://huggingface.co/moonshotai/Kimi-K3/discussions/150)
## 7. Round 07 的可证伪问题
Round 07 将:
1. exact replay Round 06 的三 seed 训练;
2. exact reproduce Round 06 的 `detached_learned` 与 `uniform_all` 两个端点;
3. 在 step 0 对 14 种模式做负控制,在 step 8,000 做正式矩阵;
4. 同时测 `spike_contrast` 与 `peak_normalized`;
5. 用 groups 6+7 的 sufficiency 与 restoration 两个方向预注册 50% log-gap
localization threshold;
6. 用单 group 的 20% 阈值和 attention-vs-MLP 的 15 percentage-point margin
作更细分的层级判定;
7. 从初始化完整 replay seed 2026073001。
完整模式、公式、失败规则和复现合同见
`research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md`。
+503
View File
@@ -0,0 +1,503 @@
# K3 Attention Residuals 局部梯度尖峰与反向路由审计
研究日期:2026-07-30
协议 ID:`llm-atlas-k3-attnres-spike-path-v1`
父协议:`llm-atlas-k3-attnres-gradient-scale-v1`
研究身份:**Round 05 已知异常驱动的定向机制追踪,不是盲发现**
## 0. 一句话结果
在与 Round 05 **model / optimizer / BPC / post-MLP gradient 全部 exact** 的
depth-32 Block AttnRes 三 seed 重放中:
1. layer 21–25 的局部峰在六个捕获位置最终都 3 / 3 可见;
2. 四种预注册 Token-gradient reductions 的 12 / 12 单元全部过闸;
3. 全局切断 64 个 depth mixers 与 output mixer 的 softmax / query / key
source-gradient path,峰没有下降,反而在 3 / 3 seed 略增;
4. 保持 learned forward 完全不变、只把所有 mixer 的 value backward coefficients
从 learned `w` 改为 `1/N` 后,spike contrast 平均下降 **70.2%**、peak normalized
平均下降 **37.0%**,两项都在 3 / 3 seed 超过预注册 20% 阈值;
5. seed-1 从初始化完整重训 8,000 steps 的 replay,连六位置、七 reductions、三模式和
65 个 mixer quantile summaries 都 exact。
最窄、最准确的结论是:
> 在这个已训练的缩小 Block AttnRes 图上,最终 layer 21–25 峰对 learned mixer 的
> **value-backward routing coefficients** 有预注册阈值下的强全局敏感性;它不是由
> softmax/key source-gradient derivative path 单独解释的。
这不等于:
- uniform backward 是更好的训练方法;
- learned value weights 让模型训练变差;
- 某一个 layer 或同层 MLP mixer 是唯一原因;
- Kimi K3 真实 checkpoint 存在同样尖峰。
## 1. 研究问题从哪里来
Round 05 已知:
- Block 的首/末四分位失衡在 6 / 6 配对中改善;
- 全层 CV 却在 6 / 6 配对中恶化;
- depth-32 三 seed 平均的相对高值集中在 layer 21–25;
- post-MLP gradient 与 MLP latest-source mean weight 在 96 个
`seed × layer` 点上 Pearson `r≈0.651`,在 layer 19–28 的 30 点上 `r≈0.690`。
这些 observational correlations 不能回答:
- 是 softmax / key 的导数路径重要,还是 value coefficients 重要?
- 峰是否只是 element RMS 的 reduction 产物?
- 峰在一个 block 内到哪个位置才出现?
因此 Round 06 在看见新结果前固定:
- 六个 activation 位置;
- 四个主 reductions、两个 cancellation-sensitive reductions 和一个代数控制;
- 三种 same-forward backward rules;
- fixed spike set `S={21,22,23,24,25}`;
- `contrast≥1.5`、top-5 overlap、Spearman、20% material drop;
- 三 formal seeds 和一个完整 replay。
前置 scoping、协议与 runner 分别在提交 `204f599`、`54c80fb` 前后进入公开仓库,
formal 训练随后才开始。
## 2. 对抗审查如何改变协议
本机 Grok Headless 只用于一次只读、对抗性方法审查,不提供事实来源,也没有修改文件。
主代理采纳并在结果前修正的关键问题包括:
- 把“最早生成位置”降级为“最早可见 tensor”;
- 主判定时点固定为 step 8,000;
- intervention 明确作用于全部 64 个 depth mixers 加 output mixer;
- 结论从局部因果分解降级为全局 backward-rule sensitivity;
- material 判定固定只用 `element_rms`;
- median / p95 固定 Hyndman–Fan Type 7;
- top-5 ties、Spearman ties、正分母与 invalid 条件全部写死;
- 增加六位置 loss×2、optimizer isolation 和训练不得进入 custom autograd 的闸门。
这次审查的价值是发现协议漏洞,不是为结果背书。
## 3. 第一次 smoke 为什么被拦住
最初 intervention custom backward 直接用 elementwise multiplication 分配
`weights × grad_output`。forward identity 已经 exact,但 step-0 uniform negative
control 失败:
```text
六位置 max raw relative difference:约 0.0037–0.0046
六位置 max normalized difference:约 0.0011–0.0020
```
这说明虽然代数公式一致,手写 backward 没有复用父 runner autocast einsum 的实际数值
路径,不能满足预注册的“query=0 时三模式等价”。
正式训练尚未开始。实现随后改为:
1. forward 直接返回父 `DepthMixer.forward` 已算出的 `y_parent`;
2. backward 在 `torch.enable_grad()` 下,用冻结的 learned / uniform weights
重新走同一个 BF16-autocast einsum surrogate;
3. 只把 surrogate 对 source values 的梯度返回;
4. 不向 `y_parent`、query、key path 回传。
修正后:
```text
三模式 logits / loss / 六位置 activations:byte-exact
65 个 mixer summaries:exact
step-0 六位置 raw gradient spectra:max difference 0
step-0 normalized spectra:max difference 0
六位置 × 七 reductions loss×2:scale / shape error 0
两个全新进程非计时字段:exact
```
这个失败不是需要隐藏的“调试噪声”;它正是负控制在正式结果前发挥作用的证据。
## 4. 训练合同与工作量
Round 06 没有创建新的训练分布:
| 字段 | 值 |
|---|---:|
| architecture | Block AttnRes |
| depth | 32 Transformer blocks |
| groups | 8,每组 4 blocks |
| width / heads / FFN | 192 / 6 / 768 |
| context / batch | 256 / 32 |
| steps | 8,000 |
| formal seeds | 2026073001 / 2 / 3 |
| replay | seed-1,从初始化完整 8,000 steps |
| formal target bytes | 196,608,000 |
| 含 replay target bytes | 262,144,000 |
| diagnostic tensor | 固定 16 × 256 |
| diagnostic steps | 0 / 100 / 500 / 2,000 / 4,000 / 8,000 |
| intervention steps | 0 / 8,000 |
数据与父协议 exact,完整值为:
```text
formal schedule:
5041e09b167f229248d2462324e8c254b8f5938975f135dcd8192b00a54a4f4e
validation:
f459316f13078a163b47c133511bb7181e05170ab89516e196490113893ce338
diagnostic:
21117e31db302b10d67b63f035665dc8f220b879d216ccd12b7d2ba86e7b1716
```
## 5. 父协议训练等价闸门
三个 formal seed 全部满足:
| 字段 | seed-1 | seed-2 | seed-3 |
|---|---:|---:|---:|
| final model-state hash | exact | exact | exact |
| final optimizer-state hash | exact | exact | exact |
| 六个 validation BPC | exact | exact | exact |
| 六条 post-MLP element-RMS arrays | exact | exact | exact |
| training history | exact | exact | exact |
最终 BPC:
| seed | Round 06 | Round 05 |
|---:|---:|---:|
| 2026073001 | 1.7123525940760473 | exact |
| 2026073002 | 1.7093240655856785 | exact |
| 2026073003 | 1.7030966812870931 | exact |
因此新增 capture / diagnostics 没有改变 optimizer trajectory。Round 06 不是一组“看起来
相近”的新模型,而是同一冻结训练的独立数值重放。
## 6. 峰在训练中形成,不是初始化常数
最终主指标是 post-MLP element-RMS:
```text
spike_contrast =
mean(layers 21–25) / mean(other 27 layers)
```
时间轨迹:
| step | seed-1 | seed-2 | seed-3 |
|---:|---:|---:|---:|
| 0 | 0.782 | 0.691 | 0.797 |
| 100 | 0.560 | 0.556 | 0.575 |
| 500 | 0.522 | 0.573 | 0.478 |
| 2,000 | 2.497 | 2.740 | **1.461** |
| 4,000 | 2.851 | 3.157 | 1.773 |
| 8,000 | **3.046** | **3.333** | **1.881** |
step 500 时三个 seed 的目标层都低于其他层均值;到 step 2,000,seed-1/2 已形成明显峰,
seed-3 仍略低于预注册 1.5 阈值;seed-3 到 step 4,000 才跨过。
所以允许说:
> 尖峰在这个训练过程中发展出来,且 onset 有 seed 差异。
不允许说:
> aggregation boundary 本身在初始化就必然产生 layer 21 峰。
## 7. 六个位置:处处可见,不等于找到 origin
最终 element-RMS spike contrast:
| 位置 | seed-1 | seed-2 | seed-3 | 三 seed mean | 3/3 ≥1.5 |
|---|---:|---:|---:|---:|---:|
| pre-attention input | 3.724 | 3.189 | 2.040 | 2.984 | 是 |
| attention branch output | 3.200 | 3.662 | 2.005 | 2.955 | 是 |
| post-attention state | 3.200 | 3.662 | 2.005 | 2.955 | 是 |
| pre-MLP input | 3.829 | 4.527 | 2.978 | **3.778** | 是 |
| MLP branch output | 3.046 | 3.333 | 1.881 | 2.754 | 是 |
| post-MLP state | 3.046 | 3.333 | 1.881 | 2.754 | 是 |
协议顺序中的第一个可见 tensor 是 `pre_attention_input`。但 activation gradient
本来就汇总全部下游路径,所以这只说明:
> 在进入该层 attention 之前,固定深度模式已经能在损失对该 tensor 的敏感性中看到。
它不说明峰在 pre-attention mixer “生成”。六个位置全部可见,反而证明单靠位置 trace
无法定位唯一 origin。
值得注意的是,`pre_mlp_input` 的平均 contrast 最高,但这仍是读数,不是“MLP 是唯一
原因”的消融证据。
## 8. reduction robustness:12 / 12 过闸
主 family:
1. element RMS;
2. mean token RMS;
3. median token RMS;
4. p95 token RMS。
每个 seed / reduction 必须同时满足:
- contrast ≥1.5;
- top-5 与固定集合至少重合 3;
- 与 element-RMS 32-layer Spearman ≥0.8。
结果:
| reduction | seed contrasts | 最低 top-5 overlap | 最低 Spearman | verdict |
|---|---|---:|---:|---:|
| element RMS | 3.046 / 3.333 / 1.881 | 3 | 1.000 | pass |
| token RMS mean | 2.869 / 3.182 / 1.808 | 3 | 0.958 | pass |
| token RMS median | 2.646 / 2.905 / 1.666 | 3 | **0.895** | pass |
| token RMS p95 | 3.143 / 3.532 / 2.012 | 3 | 0.993 | pass |
总判定:
```text
12 / 12 cells pass
robust within the preregistered reduction family
```
`global_l2` 与 element RMS 的固定 shape 代数控制也通过。batch-mean / token-mean
reduction 同样保存,但它们允许正负梯度先抵消,按协议只作探索性读数,不进入 robustness
判定。
这个结果支持:
> layer 21–25 峰不是只由“先平方再平均”的单一 reduction 选择制造的。
它不支持:
> 论文所有可能的 gradient magnitude 定义都会给出同一曲线。
## 9. same-forward interventions 到底改了什么
三模式:
### learned
原计算图:
```text
w = softmax(qᵀ RMSNorm(sources))
y = Σ w_i source_i
```
### detached learned
forward 的 `w`、`y`、logits、loss 完全相同;backward 对 sources 保留 learned
value coefficients,但切断 softmax / query / key 的 source-gradient path。
### uniform value backward
forward 仍逐字节返回 learned `y`;backward 对每个 source 返回 `grad_y/N`,不向
softmax / query / key 回传。
作用域是全部 64 个 depth mixers 加 output mixer。它不是只改 layer 21 附近,也不是一次
训练消融。
三个 seed 的最终 forward identity 均满足:
```text
logits hash exact
loss exact
六位置 activation hashes exact
65 mixer summary hash exact
```
所以三模式的梯度差不能归因于不同 forward activation 或不同预测。
## 10. softmax / key path:没有降低峰
learned → detached learned:
| seed | contrast | detached | relative drop | peak relative drop |
|---:|---:|---:|---:|---:|
| 2026073001 | 3.046 | 3.093 | −1.54% | −0.67% |
| 2026073002 | 3.333 | 3.397 | −1.93% | −0.38% |
| 2026073003 | 1.881 | 1.929 | −2.53% | −1.47% |
| mean | — | — | **−2.00%** | **−0.84%** |
“relative drop”为负表示 detaching 后峰略高。预注册规则要求 contrast 与 peak 都下降至少
20%、3 / 3 seed 同向;实际是两项在 3 / 3 seed 都朝相反方向。
结论:
> 全局移除 softmax / query / key 的 source-gradient derivative path,没有在本阈值下
> 降低固定峰;它在三个 seed 中都使峰略增。
这不证明该路径处处为零,也不证明它对训练没有作用。它只否定本轮预注册的
“material peak reduction”方向。
## 11. learned value coefficients:3 / 3 过 material 闸门
detached learned → uniform value backward:
| seed | detached contrast | uniform contrast | contrast drop | peak drop |
|---:|---:|---:|---:|---:|
| 2026073001 | 3.093 | 0.780 | **74.8%** | **39.3%** |
| 2026073002 | 3.397 | 0.786 | **76.9%** | **43.2%** |
| 2026073003 | 1.929 | 0.792 | **58.9%** | **28.6%** |
| mean | — | — | **70.2%** | **37.0%** |
六个相对下降全部超过预注册 20%。uniform value backward 下:
- 固定 target-layer contrast 都低于 0.8;
- peak layer 从 learned / detached 的 layer 21 移到 layer 2;
- forward logits、loss 与 activation 完全不变。
预注册判定:
```text
material sensitivity at the preregistered threshold
3 / 3 seeds
```
允许说:
> 在最终已训练计算图上,learned source value-backward coefficients 是该局部梯度形状的
> material carrier / amplifier。
禁止说:
- “把训练改成 uniform backward 一定更好”;
- “learned mixer 导致 BPC 下降或上升”;
- “峰完全由同层 MLP latest weight造成”;
- “70.2% 是因果贡献率”。
相对下降是对一个**全局反向规则干预**的指标变化,不是可加的路径归因百分比。
## 12. mixer association 与 intervention 如何一起读
最终 observational association:
| scope | metric | Pearson | Spearman |
|---|---|---:|---:|
| 96 seed×layer | MLP latest mass | 0.651 | 0.692 |
| 96 seed×layer | MLP normalized entropy | −0.274 | −0.417 |
| layer 19–28 | MLP latest mass | 0.690 | 0.693 |
| layer 19–28 | MLP max mass | 0.702 | 0.705 |
| layer 19–28 | MLP normalized entropy | −0.636 | −0.640 |
| layer 19–28 | attention latest mass | 0.158 | 0.339 |
Round 06 的 intervention 让“value coefficients 值得关注”从单纯相关线索升级为
same-forward backward sensitivity。但它仍然是**全部 mixers 同时改写**,所以不能把
全局结果定向归因给表中同层 MLP latest mass。
更窄的下一步需要:
- 逐 group / 逐 mixer intervention;
- output mixer 与 64 个 depth mixers 分开;
- current partial、completed group、embedding source 分开;
- 保持 forward exact 的局部 backward path ablation。
这些没有在本轮结果后追加,以免事后挑选 layer。
## 13. replay 与完整复现
seed-1 replay 从初始化完整训练 8,000 steps,不加载 formal checkpoint。
exact 字段:
```text
architecture / depth / seed / steps / batch / target bytes
manifest / model / optimizer contracts
model and optimizer state hashes
six validation evaluations
all training-history checkpoints
six diagnostic steps
six positions
seven reductions
three intervention modes at step 0 / 8,000
65 mixer summaries with quantiles
forward identity / negative control / loss-scale gates
environment / artifact identities
Round 05 equivalence record
```
冻结 compare SHA-256:
```text
05396eaf56429ca8d87815ecf00a5f78be5a9b2ec7cc96014856c09dcedcbb87
```
排除计时、run kind 和输出路径。replay verdict:`exact`。
## 14. 成本数字的正确解释
formal 三个进程的 mean step time 都约 224 ms;peak allocated 约 12.84 GB,
reserved 约 13.92 GB。
它们与 Round 05 单进程约 109 ms 不可直接比较,因为本轮两进程并行竞争同一 RTX 5090,
GPU 利用率 100%。训练 step 本身走父 runner exact 路径,新增诊断只发生在冻结时点。
因此:
- 计时证明实验真实执行;
- 计时不用于评价新诊断 overhead;
- 更不能外推到 K3 训练系统。
## 15. 工件与哈希
核心源码:
| 工件 | SHA-256 |
|---|---|
| manifest | `d5302a249249a07d362819134763d14e7d32307f22cff416c665ed9606142fef` |
| protocol | `6cb101b8760d9f1c81caeb2f16880b16152da103867224a06761a75a12984a16` |
| scoping | `590166bd62580bb8238293823cfcc39bc0a465fec4c697025343f3f1138abd27` |
| runner | `77298081d3c491d2e88e4705995174b9879ef377f520eb5fe5ea107e7a1da084` |
| analyzer | `b9dd7f723e8c5765ecffd9ae3050ff7a95643f95853adaeeaa2c2c323d0ca057` |
raw physical hashes:
| 文件 | SHA-256 |
|---|---|
| formal seed-1 | `e39e93b7a7fce3c56f5f14f95cfdc04afdce53628affee1202fe62bd1bdb7f71` |
| formal seed-2 | `1c6f6c731030ec0adb2a8e7a4d586e0c4005cc3319568a7ac83c08c2a4b8eaf8` |
| formal seed-3 | `115f8245577ece6dfaaa8ada68445c186e6523a7f3b26efcc3eb4c0c4ce82406` |
| replay seed-1 | `4f91c7a2698662ea69af9aa3425e144107834fd9b6b717fb82b84f153b867dc5` |
聚合工件:
| 工件 | physical SHA-256 | canonical SHA-256 |
|---|---|---|
| full aggregate | `75058f9e…c1e90` | `1d0faef2…fc8b4` |
| compact | `7b3a3749…8ef6c` | `a430dd26…26cd` |
| reproduction | `86ac20ee…879a` | `7c522314…3515` |
完整值保存在文件自身和 `reproduction.json`,缩写只用于阅读。
## 16. 允许与禁止的最终表述
### 允许
> 固定 layer 21–25 峰在四种预注册 Token-gradient reductions 中 12 / 12 过闸,
> 所以它在本轮 reduction family 内稳健。
> 在同一个 learned forward 上,全局 value-backward coefficient uniformization 使
> contrast / peak 在 3 / 3 seed 超过 20% 下降阈值。
> 全局移除 softmax/key source-gradient path 没有降低峰,反而在 3 / 3 seed 略增。
> 峰从训练中发展,且 seed-3 的形成更晚、更弱。
### 禁止
- “找到了梯度尖峰的唯一来源”;
- “pre-attention 是尖峰生成点”;
- “论文 Figure 5(c) 已复现”;
- “Kimi K3 checkpoint 有 layer 21 尖峰”;
- “uniform backward 会提高训练质量”;
- “learned mixer 的 70.2% 梯度是坏的”;
- “softmax/key path 没有任何作用”;
- 从 3 seeds 推断总体显著性;
- 从 width 192 / depth 32 外推到 2.8T 参数。
## 17. 下一轮最有价值的实验
1. 把全局 intervention 拆成 output mixer、group 6、group 7 和其他 mixers 四组;
2. 对 completed-group value path 与 current-partial value path 分开做 same-forward backward
ablation;
3. 预先固定 local intervention grid,不根据本轮最大的单层事后选择;
4. 比较“只改诊断 backward”与“从初始化按替代 backward 训练”的身份差异;
5. 若 Attention Residuals 官方发布 Figure 5 telemetry,再开独立 protocol 对齐其 tensor、
norm、reduction、batch 与 AMP 时点。
真实 K3 forward 仍受 `A_log [128]→[96]` 官方转换合同缺失阻断;本轮不绕过该边界。
+406
View File
@@ -0,0 +1,406 @@
# K3 Attention Residuals 局部梯度尖峰与归约敏感性协议
协议 ID:`llm-atlas-k3-attnres-spike-path-v1`
冻结日期:2026-07-30
协议状态:**结果前预注册**
父协议:`llm-atlas-k3-attnres-gradient-scale-v1`
## 0. 研究身份
本轮是 Round 05 的**定向机制追踪**,不是盲发现:
- 已知 depth-32 / Block 的三 seed 平均 layer 21–25 normalized post-MLP
activation-gradient RMS 较高;
- 已知 observationally,MLP mixer latest-source mass 与该梯度谱相关;
- 未知尖峰最早在哪个 block position 出现;
- 未知 mixer 的 softmax/key derivative path 与 learned value coefficients 分别贡献多少;
- 未知更换 raw gradient tensor 的公开 reduction 后,layer 21–25 是否仍构成稳定局部峰。
前置已知结果和相关分析固定在
`research/K3_ATTNRES_SPIKE_SCOPING.md`。任何 Round 06 输出不得被倒写成“事前未知”。
## 1. 允许回答的问题
1. 在同一个缩小 Block AttnRes 模型里,layer 21–25 的相对高值在哪些
attention / MLP 位置已经可见?
2. 在前向完全相同的条件下,对**全部 mixer** 切断 weight / key 导数路径是否降低尖峰?
3. 在前向完全相同的条件下,把**全部 mixer** 的 source value 反向系数改为均匀权重,
是否降低尖峰?
4. element RMS、token-RMS mean / median / p95 是否给出一致的 spike 结论?
5. Round 06 的新增观测是否保持 Round 05 训练状态、BPC 和 post-MLP global RMS exact?
## 2. 明确不回答的问题
- 论文 Figure 5(c) 的未公开 telemetry 定义;
- Kimi K3 2.8T checkpoint 的真实梯度;
- learned mixer 对大模型最终能力的因果收益;
- 改写 backward rule 后重新训练会发生什么;
- 哪个 source 具有可命名的语义;
- 三个 seed 之外的总体显著性或置信区间;
- 同 FLOPs、wall time 或参数量公平性。
## 3. 冻结训练合同
Round 06 不创建新训练任务分布,完整复用 Round 05:
| 字段 | 固定值 |
|---|---|
| architecture | Block AttnRes |
| Transformer depth | 32 |
| aggregation groups | 8 |
| blocks / group | 4 |
| width / heads / FFN | 192 / 6 / 768 |
| context / vocabulary | 256 / byte-256 |
| formal seeds | 2026073001 / 2026073002 / 2026073003 |
| steps / batch | 8,000 / 32 |
| target bytes / formal cell | 65,536,000 |
| optimizer | AdamW |
| peak / min LR | 3e-4 / 3e-5 |
| warmup | 400 |
| weight decay | 0.1 for ndim ≥ 2 |
| betas / epsilon | 0.9, 0.95 / 1e-8 |
| clip | global norm 1.0 |
| forward | CUDA BF16 autocast |
| residual accumulation | explicit FP32 |
| diagnostic CE | fixed 16 × 256 token-mean FP32 CE |
| diagnostic steps | 0 / 100 / 500 / 2,000 / 4,000 / 8,000 |
数据 bytes、training schedule、validation tensor 和 diagnostic tensor hashes 必须与父协议
manifest exact。训练窗口仍由父协议 ID 派生;Round 06 ID 只标识新增 telemetry,不能改变
任一 optimizer input。
## 4. 正式网格与 replay
正式运行:
```text
depth-32 / block / seed-2026073001
depth-32 / block / seed-2026073002
depth-32 / block / seed-2026073003
```
另从初始化完整重跑:
```text
replay / depth-32 / block / seed-2026073001
```
正式三格共处理 196,608,000 target bytes;含 replay 共 262,144,000 bytes。
每格都必须在全新 Python 进程中运行。可并行两个进程,但不能共享 model、optimizer、RNG
或 CUDA graph。性能计时不进入数值复现合同。
## 5. 六个 activation 位置
对每个 Transformer block `l` 捕获:
1. `pre_attention_input`:attention RMSNorm 的输入;Block 中是 attention mixer 输出;
2. `attention_branch_output`:attention projection 输出;
3. `post_attention_state`:attention branch 加入后的 FP32 partial state;
4. `pre_mlp_input`:MLP RMSNorm 的输入;Block 中是独立 MLP mixer 输出;
5. `mlp_branch_output`:MLP down projection 输出;
6. `post_mlp_state`:MLP branch 加入后的 FP32 partial state,即 Round 05 主对象。
每个位置必须有 32 个不同计算节点,shape 为 `[16,256,192]`,梯度全部 present / finite。
跨位置允许计算图语义上的共享来源,但同一位置的 32 个条目不得意外复用同一 storage。
捕获在一次 learned-mode diagnostic forward / backward 中完成;不能把六个位置拆成六次
不同 loss 的 backward:
- `attention_branch_output` / `mlp_branch_output` 在原 module 输出、转成 FP32 residual
之前捕获,保留其实际 autocast dtype;
- `post_attention_state` / `post_mlp_state` 在显式 FP32 residual accumulation 后捕获;
- 所有 `retain_grad()` 只允许出现在 diagnostic `capture=True` 路径;
- 8,000 个 optimizer training steps 必须走父 runner 的原始 `DepthMixer.forward`,
`capture=False`,不得进入 intervention custom autograd;
- 每次 diagnostic / intervention 前后都 `zero_grad(set_to_none=True)`;
- diagnostic 前后 optimizer-state tensor hash 必须 exact。
### 5.1 尖峰可见位置
固定目标集合:
```text
S = layers 21, 22, 23, 24, 25
R = other 27 layers
```
对每个位置和 seed:
```text
spike_contrast = mean(metric[S]) / mean(metric[R])
peak_normalized = max(metric) / mean(metric)
```
主判定**只使用 step 8,000**。若某个位置 `spike_contrast ≥ 1.5` 在 3 / 3 seed
成立,则称“该位置已可见局部峰”。六个位置按上述 forward 顺序报告;第一个满足者只标为
**earliest tensor where the pattern is observed**,禁止写成 origin、injection point 或
“在该算子生成”。其他 diagnostic steps 只展示轨迹,不参与位置判定。若没有位置 3 / 3
达标,结论为 position-mixed。
## 6. 同一 raw gradient tensor 的 reductions
令某层某位置梯度为 `g ∈ R[B,T,D]`,其中 `B=16,T=256,D=192`。
### 6.1 主 sensitivity family
1. `element_rms = sqrt(mean_btd(g²))`
2. `token_rms_mean = mean_bt(sqrt(mean_d(g²)))`
3. `token_rms_median = median_bt(sqrt(mean_d(g²)))`
4. `token_rms_p95 = p95_bt(sqrt(mean_d(g²)))`
这四项都测每 Token 梯度长度的分布,只改变平方根与 Token reduction 的次序/统计量。
median / p95 统一使用排序后的 **Hyndman–Fan Type 7 linear interpolation**:
```text
h = (N - 1) × p
j = floor(h)
q_p = x_sorted[j] + (h - j) × (x_sorted[j + 1] - x_sorted[j])
```
`p=.5/.95`,零基下标;若 `h` 为整数则直接取 `x_sorted[h]`。实现不得依赖 numpy /
torch 版本相关的默认 quantile 方法。
### 6.2 cancellation-sensitive diagnostics
5. `batch_mean_rms = sqrt(mean_td((mean_b g)²))`
6. `token_mean_rms = sqrt(mean_bd((mean_t g)²))`
它们允许正负梯度先抵消,测的是更相干的方向信号,只作探索性诊断,不进入主 robustness
判定。
### 6.3 代数控制
7. `global_l2 = sqrt(sum_btd(g²))`
固定 shape 下它应满足:
```text
global_l2 = element_rms × sqrt(B×T×D)
```
逐层 normalized spectrum、CV、spike contrast 应与 element RMS 在 `1e-6` 内相同。
### 6.4 reduction robustness 闸门
对最终 `post_mlp_state`,每个主 family reduction、每个 seed 必须同时满足:
1. `spike_contrast ≥ 1.5`;
2. 该 reduction 的 top-5 layers 与固定集合 `S` 至少重合 3 层;
3. 与 element RMS 的 32-layer Spearman `ρ ≥ 0.8`。
四种 reductions、三个 seed 全部满足才记为
`robust within the preregistered reduction family`。任何一格失败即为 `mixed`;
全数不满足才记为 `not robust at this threshold`。不添加事后替代阈值。
top-5 固定按 `metric descending, layer index ascending` 排序;Spearman 对并列值使用
average ranks。cancellation-sensitive diagnostics 不得进入本节判定。
## 7. mixer backward-path interventions
三种模式使用完全相同的 learned forward weights 与 model state。intervention 的作用域固定
为**全部 64 个 depth mixers 加最终 output mixer**;不允许结果后只改 layer 21–25 邻域。
因此本节回答的是“全局改写 mixer backward rule 后,固定尖峰指标是否变化”,不是把某个
局部 mixer 宣布为唯一原因。
### 7.1 `learned`
原始 mixer:
```text
w = softmax(qᵀ RMSNorm(sources))
y = Σ w_i source_i
```
反向同时经过 value coefficients 和 softmax / query / key 路径。
### 7.2 `detached_learned`
前向仍用同一 `w`,但 `w` 在反向中 detach:
```text
y = Σ stopgrad(w_i) source_i
```
它保留 learned value coefficients,切断 softmax / query / key 对 source gradient 的路径。
### 7.3 `uniform_value_backward`
使用自定义 autograd:
- 先调用父 runner 原始 `DepthMixer.forward` 得到 `y_parent`;
- custom Function 的 forward 直接返回已经计算出的 `y_parent`,不重算或做
`y + z - z` 式浮点抵消;
- backward 对每个 source 返回 `grad_y / N`;
- 不向 learned weights / query / key 回传。
因此 forward logits、loss、所有 activation 值必须与 `learned` **逐元素 exact**,但 source
value 的 backward coefficient 变为均匀。
`detached_learned` 同样以 custom Function 原样返回 `y_parent`,backward 才按 learned
`w_i` 把 `grad_y` 分配给 source。两种 custom Function 都只允许在 step 0 / 8,000
diagnostic 使用。
这些是**全局 backward-rule sensitivity diagnostics**,不是可训练模型变体,也不声称是
合理部署方案。
## 8. intervention 判定
只在 step 0 和 8,000 运行三模式。
### 8.1 forward identity gate
同 seed / step 三模式必须满足:
- logits tensor SHA-256 exact;
- loss FP32 value exact;
- 六位置 activation tensor hashes exact;
- mixer forward mean / quantile summaries exact。
任一失败,正式结果无效。
learned mode 还必须证明:训练和普通 diagnostic 的每个 mixer output 使用父 runner 数值
路径;custom Function 不得被训练 step 调用。
### 8.2 初始化负控制
step 0 的 mixer query 全为零,learned weights 是均匀分布。三模式的六位置
`element_rms`:
- 32 个 raw values 必须 finite 且严格大于 0;
- raw spectra 的逐层相对差必须 ≤ `1e-6`;
- normalized spectra 最大绝对差必须 ≤ `1e-6`。
失败则说明 intervention 实现没有隔离预期路径。
### 8.3 softmax / key derivative path
最终 `post_mlp_state`,且 reduction 固定为 `element_rms`:
```text
relative_drop_contrast =
(contrast_learned - contrast_detached) / contrast_learned
relative_drop_peak =
(peak_learned - peak_detached) / peak_learned
```
`contrast_learned` 与 `peak_learned` 必须 finite 且 `>1e-30`,否则本格 invalid 并停止
聚合。若两项都 `≥20%` 且 3 / 3 seed 同向,记为:
> 全局切断所有 mixer 的 softmax / query / key source-gradient path,在本阈值下对固定
> 尖峰指标有 material sensitivity。
禁止缩写成“layer 21–25 由 softmax/key path 造成”。
### 8.4 learned value coefficients
用 `detached_learned → uniform_value_backward` 的同一公式,分母也必须 finite 且
`>1e-30`。若 contrast 和 peak 都下降 `≥20%` 且 3 / 3 seed 同向,记为:
> 全部 mixer 的 value backward coefficients 从 learned `w` 改成 `1/N`,在本阈值下
> 对固定尖峰指标有 material sensitivity。
`N` 随 source count 变化;均匀系数不等于各 source 的数值贡献均匀。禁止写成“learned
value weights 是尖峰唯一原因”。
未过阈值只表示本协议没有达到“material”规则,不证明路径贡献为零。seed 或两个指标方向
分裂时统一记为 `mixed`。
## 9. mixer 分布摘要
每个子层 mixer 保存:
- source count;
- source labels;
- 每 source mean / p05 / median / p95;
- entropy mean;
- normalized entropy;
- max source mass;
- latest-source mass;
- group / layer / attention-or-MLP 身份。
相关分析报告 Pearson 与 Spearman,但永远标为 association;只有第 7–8 节的同前向
backward intervention 可以支持路径贡献判断。
不保存可还原语料内容的逐 Token weight arrays。
## 10. 训练等价与复现闸门
每个 formal seed 必须与对应 Round 05 depth-32 / Block raw 结果满足:
- final model-state hash exact;
- final optimizer-state hash exact;
- 六个 validation BPC exact;
- 六个 Round 05 post-MLP element-RMS arrays exact;
- training-history 冻结字段 exact。
此外,step-0 smoke 对六个位置逐一执行同一 loss 的 `×1 / ×2` backward:
- 每层每 reduction 中线性尺度应乘 2 的指标,其比值在 `2±1e-5`;
- normalized spectrum、CV、spike contrast 的绝对差 ≤ `1e-6`;
- shape、dtype、gradient presence / finite 与同位置 storage uniqueness 全部过闸。
普通 diagnostic 与三种 intervention 前后 optimizer-state hash 必须 exact;训练 step
禁止 `capture=True`。
seed-1 replay 还必须与 Round 06 formal seed-1 的以下字段 exact:
- 全部训练等价字段;
- 六位置、七 reductions、三 intervention 的全部数值;
- mixer quantile summaries;
- logits / activation tensor hashes;
- final model / optimizer hashes。
排除:
- run kind / output path;
- wall time / step time;
- peak allocated / reserved memory;
- process / host 瞬时字段。
任一训练等价字段失败,不能聚合机制结果;replay 失败则公开失败并停止网站结论。
## 11. 正式工件
```text
experiments/k3/attnres_spike/
README.md
manifest.json
scope.py
train.py
analyze.py
reproduction.json
results/raw/*.json
research/
K3_ATTNRES_SPIKE_SCOPING.md
K3_ATTNRES_SPIKE_PROTOCOL.md
K3_ATTNRES_SPIKE_AUDIT.md
src/data/
k3-attnres-spike.json
k3-attnres-spike-compact.json
```
网站必须把四种证据身份分开:
- Round 05 已知峰;
- observational mixer association;
- same-forward backward intervention;
- reduction robustness / failure。
## 12. 停止规则
出现以下任一情况立即停止正式聚合:
- 训练 hash 不匹配 Round 05;
- forward identity gate 失败;
- step-0 uniform negative control 失败;
- 六位置 loss×2 或 optimizer isolation gate 失败;
- activation gradient 缺失、非 finite 或 shape 错误;
- replay 数值不 exact;
- 正式 raw 文件缺失或 canonical hash 不闭合。
遇到反结果时不修改集合 `S`、20% 阈值、1.5 contrast、top-5 overlap 或 Spearman 阈值。
+120
View File
@@ -0,0 +1,120 @@
# K3 Attention Residuals 局部梯度尖峰:Round 06 前置定位
研究日期:2026-07-30
阶段身份:**探索性 scoping,不是 Round 06 预注册结果**
上游协议:`llm-atlas-k3-attnres-gradient-scale-v1`
## 1. 为什么先做 scoping
Round 05 已经公开了一个方向分裂:
- Block AttnRes 在 depth 16 / 32、三个 seed 的 6 / 6 配对中都改善首/末四分位失衡;
- 但它也在 6 / 6 配对中提高全层 activation-gradient CV;
- depth 32 的三 seed 平均峰值集中在 layer 21–25。
这意味着下一步不该再问一个笼统的“梯度是否均匀”,而应该问:
1. 尖峰在 attention / MLP block 的哪个位置已经出现?
2. 它是否与 learned mixer 的 source 权重有关?
3. 相关来自 mixer softmax / key 的导数路径,还是来自 learned value coefficients?
4. 换一种公开、合理的 gradient reduction 后,尖峰是否还存在?
Round 05 已保存每个诊断点的逐层 post-MLP gradient RMS 和 64 个子层 mixer
分布。这里先用那些**已经见过的数据**定位假设,再冻结新协议。Round 06 的因果式
backward-path intervention 不能被描述成盲验证。
## 2. 对齐规则
depth 32 的 Block AttnRes 使用 8 个 aggregation groups,每组 4 个 Transformer
blocks、8 个 attention / MLP residual sublayers。
对 1-based Transformer layer `l`:
```text
attention mixer index = 2 × (l - 1)
MLP mixer index = 2 × (l - 1) + 1
group = floor((l - 1) / 4) + 1
offset in group = ((l - 1) mod 4) + 1
```
`latest source mass` 定义为对应 mixer `mean_weights` 的最后一个元素。它在不同
group offset 的语义并不完全相同:
- group 首层 attention mixer 没有 current partial,最后源是最近完成的 group;
- 其他 attention mixer 的最后源是 current partial;
- 每层 MLP mixer 的最后源都是刚加入 attention branch 后的 current partial。
因此不能把所有 `last` 自动命名为同一种“最近层贡献”。
## 3. layer 19–28 的三 seed 均值
下表的 gradient 是每个 seed 先按该 seed 的 32 层均值归一化,再跨 seed 平均。
权重和 normalized entropy 也跨三个 seed 平均。
| layer | group / offset | gradient / layer mean | attn last | MLP last | attn H/logN | MLP H/logN |
|---:|---:|---:|---:|---:|---:|---:|
| 19 | 5 / 3 | 1.063 | 0.349 | 0.452 | 0.929 | 0.842 |
| 20 | 5 / 4 | 1.014 | 0.316 | 0.419 | 0.946 | 0.863 |
| 21 | 6 / 1 | **3.042** | 0.250 | **0.513** | 0.969 | 0.763 |
| 22 | 6 / 2 | **2.405** | 0.436 | **0.563** | 0.851 | 0.721 |
| 23 | 6 / 3 | **1.910** | 0.400 | **0.551** | 0.879 | 0.737 |
| 24 | 6 / 4 | **1.548** | 0.384 | **0.515** | 0.889 | 0.771 |
| 25 | 7 / 1 | **1.769** | 0.317 | 0.358 | 0.934 | 0.766 |
| 26 | 7 / 2 | 1.369 | 0.343 | 0.420 | 0.876 | 0.764 |
| 27 | 7 / 3 | 0.986 | 0.325 | 0.402 | 0.891 | 0.775 |
| 28 | 7 / 4 | 0.802 | 0.296 | 0.304 | 0.902 | 0.822 |
layer 21–24 正好是第 6 个 group,layer 25 是第 7 个 group 的首层。梯度峰并不
只是一个 group boundary 单点;它在第 6 组内部递减,并在下一组首层出现较小的第二峰。
## 4. 相关线索
把三个 seed 的 32 层合成 96 个点,post-MLP normalized gradient 与 mixer 摘要的
Pearson 相关为:
| 变量 | 96 点 `r` | layer 19–28 的 30 点 `r` |
|---|---:|---:|
| attention latest-source mass | 0.066 | 0.158 |
| MLP latest-source mass | **0.651** | **0.690** |
| attention normalized entropy | −0.296 | −0.057 |
| MLP normalized entropy | −0.274 | **−0.636** |
| attention max source mass | −0.024 | 0.158 |
| MLP max source mass | 0.230 | **0.702** |
这些数值只支持:
> 尖峰层与更集中的 MLP mixer、尤其较大的 latest-partial mean weight 同时出现。
它们不支持:
- “MLP latest weight 导致梯度尖峰”;
- “第 6 group 是唯一原因”;
- “降低 entropy 就一定增加梯度”;
- “K3 真实 checkpoint 也有相同模式”。
同一个 learned mixer 同时改变 forward activation、value-path gradient coefficient
和 softmax/key derivative path。单看 observational correlation 无法分解这三者。
## 5. Round 06 要冻结的可证伪问题
Round 06 将复跑与 Round 05 完全相同的 depth-32 / Block 三 seed 训练,并要求最终
model / optimizer hash 与 Round 05 exact。新增诊断只在 optimizer step 之外执行。
它将同时测量:
- pre-attention input;
- attention branch output;
- post-attention partial state;
- pre-MLP input;
- MLP branch output;
- post-MLP partial state;
- 同一 raw gradient tensor 的多种 reduction;
- learned backward;
- learned weights detached backward;
- learned forward + uniform value backward。
其中后两项保持**前向 logits、loss 与 activation 完全相同**,只改变反向路径。这样才能
区分“相关”与“哪条 backward path 对尖峰有实质贡献”。
完整阈值、失败规则和复现合同见
`research/K3_ATTNRES_SPIKE_PROTOCOL.md`。
+1 -1
View File
@@ -228,7 +228,7 @@ if (numeric(reliability.initial.passAt) <= numeric(reliability.k2.passAt) || num
if (reliability.nonIdempotent.sideRisk === "LOW") failures.push("非幂等写操作风险没有提升"); if (reliability.nonIdempotent.sideRisk === "LOW") failures.push("非幂等写操作风险没有提升");
if (numeric(rl.wait.utilization) >= numeric(rl.full.utilization) || numeric(rl.wait.lostWork) <= numeric(rl.full.lostWork)) failures.push("wait-all 长尾/重算方向异常"); if (numeric(rl.wait.utilization) >= numeric(rl.full.utilization) || numeric(rl.wait.lostWork) <= numeric(rl.full.lostWork)) failures.push("wait-all 长尾/重算方向异常");
if (!rl.wait.takeaway.includes("wait-all") || rl.keyboardSelected !== "rl" || rl.keyboardVisible !== "rl") failures.push("长程 RL 解释或键盘导航异常"); if (!rl.wait.takeaway.includes("wait-all") || rl.keyboardSelected !== "rl" || rl.keyboardVisible !== "rl") failures.push("长程 RL 解释或键盘导航异常");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") failures.push("首页 K3 首发入口异常");
if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasAgentFilter || papers.agentVisible < 52) failures.push("论文库 Agent 标签或论文总数异常"); if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasAgentFilter || papers.agentVisible < 52) failures.push("论文库 Agent 标签或论文总数异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`); if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
+1 -1
View File
@@ -226,7 +226,7 @@ if (!update.steps[0].includes("Fixed preference")) failures.push("DPO 更新流
if (!recipe.family.includes("Multi-effort") || !recipe.regime.includes("9 RL experts") || !recipe.constraints.includes("verbosity")) failures.push("K3 配方合同异常"); if (!recipe.family.includes("Multi-effort") || !recipe.regime.includes("9 RL experts") || !recipe.constraints.includes("verbosity")) failures.push("K3 配方合同异常");
if (!recipe.path.some((step) => step.includes("3 domains × 3 efforts")) || !recipe.path.some((step) => step.includes("MOPD"))) failures.push("K3 配方路径异常"); if (!recipe.path.some((step) => step.includes("3 domains × 3 efforts")) || !recipe.path.some((step) => step.includes("MOPD"))) failures.push("K3 配方路径异常");
if (recipe.keyboardSelected !== "recipe" || recipe.keyboardVisible !== "recipe") failures.push("实验 tab 键盘导航异常"); if (recipe.keyboardSelected !== "recipe" || recipe.keyboardVisible !== "recipe") failures.push("实验 tab 键盘导航异常");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") failures.push("首页 K3 首发入口异常");
if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasAlignmentFilter || papers.alignmentVisible < 35) failures.push("论文库后训练标签或论文总数异常"); if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasAlignmentFilter || papers.alignmentVisible < 35) failures.push("论文库后训练标签或论文总数异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`); if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
+1 -1
View File
@@ -234,7 +234,7 @@ if (layout.navLinks !== 20 || mobile.mobileLinks !== 20 || home.navLinks !== 20)
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出"); if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`); if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`);
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用"); if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") { if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") {
failures.push("首页 Transformer 新章入口异常"); failures.push("首页 Transformer 新章入口异常");
} }
if (home.paperCount !== "486") failures.push(`首页论文总数异常:${home.paperCount}`); if (home.paperCount !== "486") failures.push(`首页论文总数异常:${home.paperCount}`);
+1 -1
View File
@@ -1318,7 +1318,7 @@ if (completionDepth.tasks.panel !== "tasks" || completionDepth.tasks.mathCards !
if (completionDepth.hidden.panel !== "hidden" || completionDepth.hidden.stages !== 29 || completionDepth.hidden.selected !== "layer_07" || completionDepth.hidden.points !== 29 || completionDepth.hidden.exact === "1,537 / 1,537" || numeric(completionDepth.hidden.relative) <= 0) failures.push("29 阶段隐藏状态曲线或交互异常"); if (completionDepth.hidden.panel !== "hidden" || completionDepth.hidden.stages !== 29 || completionDepth.hidden.selected !== "layer_07" || completionDepth.hidden.points !== 29 || completionDepth.hidden.exact === "1,537 / 1,537" || numeric(completionDepth.hidden.relative) <= 0) failures.push("29 阶段隐藏状态曲线或交互异常");
if (completionDepth.router.panel !== "router" || completionDepth.router.layers !== 26 || completionDepth.router.selected !== "layer 24" || completionDepth.router.points !== 26 || !completionDepth.router.ordered.includes("%") || !completionDepth.router.setExact.includes("%") || numeric(completionDepth.router.tv) <= 0 || completionDepth.router.reproCards !== 4 || completionDepth.router.links !== 4) failures.push("26 层 MoE 路由曲线或复跑证据异常"); if (completionDepth.router.panel !== "router" || completionDepth.router.layers !== 26 || completionDepth.router.selected !== "layer 24" || completionDepth.router.points !== 26 || !completionDepth.router.ordered.includes("%") || !completionDepth.router.setExact.includes("%") || numeric(completionDepth.router.tv) <= 0 || completionDepth.router.reproCards !== 4 || completionDepth.router.links !== 4) failures.push("26 层 MoE 路由曲线或复跑证据异常");
if (completionDepth.keyboardSelected !== "tasks" || completionDepth.keyboardVisible !== "tasks") failures.push("完成度与全深度实验键盘 tab 导航异常"); if (completionDepth.keyboardSelected !== "tasks" || completionDepth.keyboardVisible !== "tasks") failures.push("完成度与全深度实验键盘 tab 导航异常");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/" || home.paperCount !== "486") failures.push("首页 DeepSeek 首发入口或论文数异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/" || home.paperCount !== "486") failures.push("首页 K3 首发入口或论文数异常");
if (papers.total !== 486 || !papers.hasFilter || papers.visible < 20 || !papers.hasCoder || !papers.hasEngram) failures.push("论文库 DeepSeek 聚光异常"); if (papers.total !== 486 || !papers.hasFilter || papers.visible < 20 || !papers.hasCoder || !papers.hasEngram) failures.push("论文库 DeepSeek 聚光异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4 || mobile.artifactTabs !== 13 || mobile.behaviorTabs !== 4 || mobile.behaviorSources !== 16 || mobile.behaviorEdges !== 10 || mobile.behaviorDeviceCells !== 29 || mobile.completionDepthTabs !== 4 || mobile.completionDepthHiddenStages !== 29 || mobile.completionDepthRouterLayers !== 26 || mobile.artifactHeatCells !== 64 || mobile.corpusCohorts !== 3 || mobile.lengthDeltaCards !== 4 || mobile.templateLayers !== 6 || mobile.templateScopes !== 2 || mobile.templateModes !== 2 || mobile.templateDomainCards !== 4 || mobile.templateDepthCells !== 24 || mobile.historyLayers !== 6 || mobile.historyScopes !== 2 || mobile.historyModes !== 2 || mobile.historyEffects !== 3 || mobile.historyDomainCards !== 4 || mobile.historyDepthCells !== 24 || mobile.distanceLayers !== 6 || mobile.distanceScopes !== 2 || mobile.distanceModes !== 2 || mobile.distanceContrasts !== 2 || mobile.distanceDomainCards !== 4 || mobile.distanceDepthCells !== 24 || mobile.boundaryLayers !== 6 || mobile.boundaryScopes !== 2 || mobile.boundaryModes !== 2 || mobile.boundaryContrasts !== 3 || mobile.boundaryTokenCards !== 4 || mobile.boundaryDomainCards !== 4 || mobile.boundaryDepthCells !== 24 || mobile.roleLayers !== 6 || mobile.roleScopes !== 2 || mobile.roleModes !== 2 || mobile.roleContrasts !== 3 || mobile.roleLevelCards !== 4 || mobile.roleDomainCards !== 4 || mobile.roleDepthCells !== 24 || mobile.specialLayers !== 6 || mobile.specialScopes !== 2 || mobile.specialModes !== 2 || mobile.specialContrasts !== 4 || mobile.specialTokenCards !== 4 || mobile.specialDomainCards !== 4 || mobile.specialDepthCells !== 24 || mobile.roleBlockLayers !== 6 || mobile.roleBlockScopes !== 2 || mobile.roleBlockModes !== 2 || mobile.roleBlockEffects !== 3 || mobile.roleBlockMatrixCards !== 4 || mobile.roleBlockDomainCards !== 4 || mobile.roleBlockDepthCells !== 24) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4 || mobile.artifactTabs !== 13 || mobile.behaviorTabs !== 4 || mobile.behaviorSources !== 16 || mobile.behaviorEdges !== 10 || mobile.behaviorDeviceCells !== 29 || mobile.completionDepthTabs !== 4 || mobile.completionDepthHiddenStages !== 29 || mobile.completionDepthRouterLayers !== 26 || mobile.artifactHeatCells !== 64 || mobile.corpusCohorts !== 3 || mobile.lengthDeltaCards !== 4 || mobile.templateLayers !== 6 || mobile.templateScopes !== 2 || mobile.templateModes !== 2 || mobile.templateDomainCards !== 4 || mobile.templateDepthCells !== 24 || mobile.historyLayers !== 6 || mobile.historyScopes !== 2 || mobile.historyModes !== 2 || mobile.historyEffects !== 3 || mobile.historyDomainCards !== 4 || mobile.historyDepthCells !== 24 || mobile.distanceLayers !== 6 || mobile.distanceScopes !== 2 || mobile.distanceModes !== 2 || mobile.distanceContrasts !== 2 || mobile.distanceDomainCards !== 4 || mobile.distanceDepthCells !== 24 || mobile.boundaryLayers !== 6 || mobile.boundaryScopes !== 2 || mobile.boundaryModes !== 2 || mobile.boundaryContrasts !== 3 || mobile.boundaryTokenCards !== 4 || mobile.boundaryDomainCards !== 4 || mobile.boundaryDepthCells !== 24 || mobile.roleLayers !== 6 || mobile.roleScopes !== 2 || mobile.roleModes !== 2 || mobile.roleContrasts !== 3 || mobile.roleLevelCards !== 4 || mobile.roleDomainCards !== 4 || mobile.roleDepthCells !== 24 || mobile.specialLayers !== 6 || mobile.specialScopes !== 2 || mobile.specialModes !== 2 || mobile.specialContrasts !== 4 || mobile.specialTokenCards !== 4 || mobile.specialDomainCards !== 4 || mobile.specialDepthCells !== 24 || mobile.roleBlockLayers !== 6 || mobile.roleBlockScopes !== 2 || mobile.roleBlockModes !== 2 || mobile.roleBlockEffects !== 3 || mobile.roleBlockMatrixCards !== 4 || mobile.roleBlockDomainCards !== 4 || mobile.roleBlockDepthCells !== 24) failures.push("移动端导航或实验异常");
if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`); if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`);
+1 -1
View File
@@ -277,7 +277,7 @@ if (numeric(system.initial.success) <= numeric(system.initial.model) || numeric(
if (numeric(system.cheap.success) >= numeric(system.initial.success) || numeric(system.cheap.cost) !== 4) failures.push("低预算没有降低成功率 / 成本"); if (numeric(system.cheap.success) >= numeric(system.initial.success) || numeric(system.cheap.cost) !== 4) failures.push("低预算没有降低成功率 / 成本");
if (numeric(system.locked.unsafe) !== 0 || numeric(system.locked.overrefusal) <= numeric(system.initial.overrefusal)) failures.push("安全壳没有展现危险服从 / 过拒权衡"); if (numeric(system.locked.unsafe) !== 0 || numeric(system.locked.overrefusal) <= numeric(system.initial.overrefusal)) failures.push("安全壳没有展现危险服从 / 过拒权衡");
if (system.keyboardSelected !== "judge" || system.keyboardVisible !== "judge") failures.push("实验键盘 tab 导航异常"); if (system.keyboardSelected !== "judge" || system.keyboardVisible !== "judge") failures.push("实验键盘 tab 导航异常");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") failures.push("首页 K3 首发入口异常");
if (home.paperCount !== "486" || home.topicCount !== "17" || papers.total !== 486 || !papers.hasFilter || papers.visible < 80) failures.push("首页 / 论文库评测索引异常"); if (home.paperCount !== "486" || home.topicCount !== "17" || papers.total !== 486 || !papers.hasFilter || papers.visible < 80) failures.push("首页 / 论文库评测索引异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`); if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`);
+1 -1
View File
@@ -264,7 +264,7 @@ if (!fleet.k3.avoided.includes("320K") || fleet.k3.shortSlo !== "PROTECTED") fai
if (!fleet.failed.state.includes("SECONDARY RE-PREFILL") || !fleet.failed.recompute.includes("FAILED PRIMARY")) failures.push("缓存故障没有触发原子失效后的重算"); if (!fleet.failed.state.includes("SECONDARY RE-PREFILL") || !fleet.failed.recompute.includes("FAILED PRIMARY")) failures.push("缓存故障没有触发原子失效后的重算");
if (fleet.bursty.shortSlo !== "VIOLATED") failures.push("平均并发阈值没有暴露长请求突发"); if (fleet.bursty.shortSlo !== "VIOLATED") failures.push("平均并发阈值没有暴露长请求突发");
if (fleet.keyboardSelected !== "phase" || fleet.keyboardVisible !== "phase") failures.push("实验键盘 tab 导航异常"); if (fleet.keyboardSelected !== "phase" || fleet.keyboardVisible !== "phase") failures.push("实验键盘 tab 导航异常");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") failures.push("首页 K3 首发入口异常");
if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasFilter || papers.visible !== 46) failures.push("论文库推理服务标签或总数异常"); if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasFilter || papers.visible !== 46) failures.push("论文库推理服务标签或总数异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`); if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`);
@@ -0,0 +1,257 @@
import { writeFileSync } from "node:fs";
const cdpPort = process.env.CDP_PORT ?? "9231";
const baseUrl = process.env.SITE_URL ?? "http://127.0.0.1:4330";
const pages = await fetch(`http://127.0.0.1:${cdpPort}/json/list`).then((response) => response.json());
const page = pages.find((entry) => entry.type === "page");
if (!page) throw new Error(`CDP ${cdpPort} 没有可用页面`);
const socket = new WebSocket(page.webSocketDebuggerUrl);
await new Promise((resolve, reject) => {
socket.addEventListener("open", resolve, { once: true });
socket.addEventListener("error", reject, { once: true });
});
let nextId = 0;
const pending = new Map();
const exceptions = [];
socket.addEventListener("message", (event) => {
const message = JSON.parse(event.data);
if (message.id && pending.has(message.id)) {
const { resolve, reject } = pending.get(message.id);
pending.delete(message.id);
if (message.error) reject(new Error(message.error.message));
else resolve(message.result);
}
if (message.method === "Runtime.exceptionThrown") {
exceptions.push(message.params.exceptionDetails.exception?.description ?? message.params.exceptionDetails.text);
}
});
const command = (method, params = {}) => new Promise((resolve, reject) => {
const id = ++nextId;
pending.set(id, { resolve, reject });
socket.send(JSON.stringify({ id, method, params }));
});
const pause = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
const evaluate = async (expression) => {
const result = await command("Runtime.evaluate", { expression, returnByValue: true, awaitPromise: true });
if (result.exceptionDetails) throw new Error(result.exceptionDetails.exception?.description ?? result.exceptionDetails.text);
return result.result.value;
};
const navigate = async (path) => {
await command("Page.navigate", { url: `${baseUrl}${path}` });
for (let attempt = 0; attempt < 100; attempt += 1) {
await pause(100);
if (await evaluate("document.readyState === 'complete'")) return;
}
throw new Error(`${path} 加载超时`);
};
const screenshot = async (path) => {
const result = await command("Page.captureScreenshot", { format: "png", captureBeyondViewport: false });
writeFileSync(path, Buffer.from(result.data, "base64"));
};
await command("Page.enable");
await command("Runtime.enable");
await command("Emulation.setDeviceMetricsOverride", {
width: 1440,
height: 1100,
deviceScaleFactor: 1,
mobile: false,
});
await navigate("/k3/");
const desktop = await evaluate(`(() => {
const root = document.querySelector("[data-forward-lab]");
root.scrollIntoView({ block: "start", behavior: "instant" });
window.scrollBy(0, -78);
const text = (selector) => root.querySelector(selector)?.textContent.replace(/\\s+/g, " ").trim();
const panel = () => root.querySelector("[data-forward-panel]:not([hidden])")?.dataset.forwardPanel;
const setSelect = (selector, value) => {
const node = root.querySelector(selector);
node.value = value;
node.dispatchEvent(new Event("change", { bubbles: true }));
};
const initial = {
panel: panel(),
tabs: root.querySelectorAll("[data-forward-tab]").length,
panels: root.querySelectorAll("[data-forward-panel]").length,
ledger: root.querySelectorAll(".forward-ledger article").length,
targetGroups: root.querySelectorAll(".group-map article.target").length,
spikeLayers: root.querySelectorAll(".group-map i.spike").length,
variants: root.querySelectorAll(".variant-grid article").length,
boundary: root.textContent.includes("K3 的训练尖峰已被定位") &&
root.textContent.includes("Figure 5(c)") &&
root.textContent.includes("训练期架构消融"),
};
root.querySelector('[data-forward-tab="trajectory"]').click();
const trajectoryInitial = {
panel: panel(),
state: text("[data-forward-trajectory-state]"),
points: root.querySelectorAll("[data-forward-trajectory-series] circle").length,
lines: root.querySelectorAll("[data-forward-trajectory-series] polyline").length,
readouts: [...root.querySelectorAll("[data-forward-trajectory-readout] article")]
.map((node) => node.textContent.replace(/\\s+/g, " ").trim()),
};
setSelect("[data-forward-trajectory-seed]", "2026073002");
root.querySelector('[data-forward-trajectory-metric="peak_normalized"]').click();
const trajectoryChanged = {
state: text("[data-forward-trajectory-state]"),
readouts: [...root.querySelectorAll("[data-forward-trajectory-readout] article")]
.map((node) => node.textContent.replace(/\\s+/g, " ").trim()),
};
root.querySelector('[data-forward-tab="gate"]').click();
const gate = {
panel: panel(),
status: text(".status-banner"),
rows: root.querySelectorAll(".gate-table tbody tr").length,
passedRows: root.querySelectorAll(".gate-table tbody td.good:last-child").length,
quality: text(".gate-layout aside"),
claims: root.querySelectorAll(".claim-pair article").length,
};
root.querySelector('[data-forward-tab="interaction"]').click();
const interactionInitial = {
panel: panel(),
state: text("[data-forward-interaction-state]"),
cells: root.querySelectorAll("[data-forward-interaction-cells] article").length,
summary: text("[data-forward-interaction-summary]"),
};
root.querySelector('[data-forward-interaction-metric="peak_normalized"]').click();
const interactionChanged = {
state: text("[data-forward-interaction-state]"),
summary: text("[data-forward-interaction-summary]"),
};
root.querySelector('[data-forward-tab="spectrum"]').click();
const spectrumInitial = {
panel: panel(),
state: text("[data-forward-spectrum-state]"),
points: root.querySelectorAll("[data-forward-spectrum-points] circle").length,
contrast: text("[data-forward-spectrum-contrast]"),
peak: text("[data-forward-spectrum-peak]"),
layer: text("[data-forward-spectrum-layer]"),
audits: root.querySelectorAll(".audit-grid article").length,
replay: root.textContent.includes("scientific exact"),
};
setSelect("[data-forward-spectrum-seed]", "2026073002");
setSelect("[data-forward-spectrum-variant]", "uniform_groups_6_7_forward");
const spectrumChanged = {
state: text("[data-forward-spectrum-state]"),
points: root.querySelectorAll("[data-forward-spectrum-points] circle").length,
contrast: text("[data-forward-spectrum-contrast]"),
peak: text("[data-forward-spectrum-peak]"),
layer: text("[data-forward-spectrum-layer]"),
};
const first = root.querySelector('[data-forward-tab="contract"]');
first.focus();
first.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowRight", bubbles: true }));
const keyboard = {
selected: root.querySelector('[data-forward-tab][aria-selected="true"]').dataset.forwardTab,
panel: panel(),
};
return {
initial, trajectoryInitial, trajectoryChanged, gate,
interactionInitial, interactionChanged, spectrumInitial, spectrumChanged, keyboard,
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
rootOverflow: root.scrollWidth - root.clientWidth,
};
})()`);
await pause(180);
await screenshot("/tmp/llm-atlas-k3-attnres-forward-desktop.png");
await command("Emulation.setDeviceMetricsOverride", {
width: 390,
height: 844,
deviceScaleFactor: 1,
mobile: true,
});
await navigate("/k3/");
const mobile = await evaluate(`(() => {
const root = document.querySelector("[data-forward-lab]");
root.scrollIntoView({ block: "start", behavior: "instant" });
window.scrollBy(0, -64);
root.querySelector('[data-forward-tab="gate"]').click();
const table = root.querySelector(".gate-table-wrap");
const gateTableScrolls = table.scrollWidth > table.clientWidth;
root.querySelector('[data-forward-tab="spectrum"]').click();
return {
tabs: root.querySelectorAll("[data-forward-tab]").length,
visiblePanel: root.querySelector("[data-forward-panel]:not([hidden])")?.dataset.forwardPanel,
points: root.querySelectorAll("[data-forward-spectrum-points] circle").length,
audits: root.querySelectorAll(".audit-grid article").length,
gateTableScrolls,
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
rootOverflow: root.scrollWidth - root.clientWidth,
};
})()`);
await pause(180);
await screenshot("/tmp/llm-atlas-k3-attnres-forward-mobile.png");
const failures = [];
if (desktop.initial.panel !== "contract" || desktop.initial.tabs !== 5 || desktop.initial.panels !== 5 ||
desktop.initial.ledger !== 6 || desktop.initial.targetGroups !== 2 ||
desktop.initial.spikeLayers !== 5 || desktop.initial.variants !== 4) {
failures.push("五视图、账本、group 或 selector map 结构异常");
}
if (!desktop.initial.boundary) failures.push("reduced-model / K3 / Figure 5(c) claim boundary 缺失");
if (desktop.trajectoryInitial.panel !== "trajectory" || desktop.trajectoryInitial.points !== 24 ||
desktop.trajectoryInitial.lines !== 4 || desktop.trajectoryInitial.readouts.length !== 4 ||
!desktop.trajectoryInitial.state.includes("2026073001") ||
!desktop.trajectoryInitial.readouts.some((value) => value.includes("+73.5%"))) {
failures.push("seed 1 contrast 训练轨迹异常");
}
if (!desktop.trajectoryChanged.state.includes("2026073002") ||
!desktop.trajectoryChanged.state.includes("PEAK / MEAN") ||
!desktop.trajectoryChanged.readouts.some((value) => value.includes("+62.0%"))) {
failures.push("trajectory seed / metric 切换异常");
}
if (desktop.gate.panel !== "gate" || desktop.gate.rows !== 6 || desktop.gate.passedRows !== 6 ||
desktop.gate.claims !== 2 || !desktop.gate.status.includes("ATTENUATION ESTABLISHED") ||
!desktop.gate.status.includes("6 / 6") || !desktop.gate.status.includes("4 / 4") ||
!desktop.gate.quality.includes("+0.0066")) {
failures.push("冻结主门或 BPC quality readout 异常");
}
if (desktop.interactionInitial.panel !== "interaction" || desktop.interactionInitial.cells !== 3 ||
!desktop.interactionInitial.state.includes("8,000") ||
!desktop.interactionInitial.summary.includes("-0.367") ||
!desktop.interactionChanged.state.includes("PEAK / MEAN") ||
!desktop.interactionChanged.summary.includes("-0.170")) {
failures.push("I67 描述性 residual 切换异常");
}
if (desktop.spectrumInitial.panel !== "spectrum" || desktop.spectrumInitial.points !== 32 ||
desktop.spectrumInitial.contrast !== "3.046×" || desktop.spectrumInitial.peak !== "3.219×" ||
desktop.spectrumInitial.layer !== "L21" || desktop.spectrumInitial.audits !== 5 ||
!desktop.spectrumInitial.replay) {
failures.push("reference 32 层谱或 replay 审计异常");
}
if (desktop.spectrumChanged.points !== 32 || !desktop.spectrumChanged.state.includes("2026073002") ||
!desktop.spectrumChanged.state.includes("GROUPS 6+7") ||
desktop.spectrumChanged.contrast !== "0.758×" || desktop.spectrumChanged.peak !== "1.375×" ||
desktop.spectrumChanged.layer !== "L9") {
failures.push("joint variant spectrum 切换异常");
}
if (desktop.keyboard.selected !== "trajectory" || desktop.keyboard.panel !== "trajectory") {
failures.push("键盘 tab 切换异常");
}
if (desktop.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("页面出现文档级横向溢出");
if (desktop.rootOverflow > 1 || mobile.rootOverflow > 1) failures.push("Round 08 实验室出现横向溢出");
if (mobile.tabs !== 5 || mobile.visiblePanel !== "spectrum" || mobile.points !== 32 ||
mobile.audits !== 5 || !mobile.gateTableScrolls) {
failures.push("390px 移动端布局或局部表格滚动异常");
}
if (exceptions.length) failures.push(`运行时异常:${exceptions.join(" | ")}`);
console.log(JSON.stringify({ desktop, mobile, exceptions }, null, 2));
socket.close();
if (failures.length) {
console.error(`FAIL K3 AttnRes forward-training browser\n- ${failures.join("\n- ")}`);
process.exit(1);
}
console.log("PASS K3 AttnRes forward-training browser interactions");
+106
View File
@@ -0,0 +1,106 @@
import { createHash } from "node:crypto";
import { readdirSync, readFileSync } from "node:fs";
const hash = (bytes) => createHash("sha256").update(bytes).digest("hex");
const read = (path) => {
const bytes = readFileSync(new URL(path, import.meta.url));
return { bytes, json: JSON.parse(bytes), sha256: hash(bytes) };
};
const aggregate = read("../src/data/k3-attnres-forward.json");
const compact = read("../src/data/k3-attnres-forward-compact.json");
const reproduction = read("../experiments/k3/attnres_forward/reproduction.json");
const manifest = read("../experiments/k3/attnres_forward/manifest.json");
const rawDirectory = new URL("../experiments/k3/attnres_forward/results/raw/", import.meta.url);
const failures = [];
const expect = (condition, message) => {
if (!condition) failures.push(message);
};
const close = (actual, expected, tolerance = 1e-15) =>
Math.abs(actual - expected) <= tolerance;
expect(aggregate.sha256 === "f8df928adb8a563d851bb3c1abbf40bcada33626d9177821e4341d854980a097", "aggregate physical SHA-256 changed");
expect(compact.sha256 === "664f6d6226df7c0c9aba6314d54a1cb8ae90f6922016dfa7c7a823729606f0c1", "compact physical SHA-256 changed");
expect(reproduction.sha256 === "b029c333596dd1de957efc2b42f7ebfc1d32f241c687f06021c4d08ffa27b1d0", "reproduction physical SHA-256 changed");
expect(manifest.sha256 === "49546ed5baf36bcb30885062b7c671fafe4ccff2e606624cd7dbe23717f9a712", "manifest physical SHA-256 changed");
expect(aggregate.json.canonical_sha256_without_self === "eecf05c623e473ec5eba8afa54d555d2733a0af2fc5488398f1da095206d50c8", "aggregate canonical SHA-256 changed");
expect(compact.json.canonical_sha256_without_self === "c3e672adb6879f99efccf3bd2e0fabaab1a1159225a8ddaf4f5f9ea7a002784b", "compact canonical SHA-256 changed");
expect(reproduction.json.canonical_sha256_without_self === "57346df80c0d76bd1d306d5fa213feed74ac2094237c22ebcb49f16ea16437e0", "reproduction canonical SHA-256 changed");
expect(compact.json.protocol_id === "llm-atlas-k3-attnres-forward-training-v1", "protocol identity mismatch");
expect(compact.json.status === "forward_training_attenuation_established_within_reduced_protocol", "frozen status changed");
expect(compact.json.primary_step === 8000, "primary step changed");
expect(compact.json.spike_layers_1based.join(",") === "21,22,23,24,25", "spike window changed");
expect(compact.json.thresholds.material_relative_drop === 0.2, "attenuation threshold changed");
expect(compact.json.processed_target_bytes.formal_12_cells === 786432000, "formal target bytes changed");
expect(compact.json.processed_target_bytes.primary_replay === 65536000, "replay target bytes changed");
expect(compact.json.processed_target_bytes.total === 851968000, "new target bytes changed");
expect(compact.json.aggregate_sha256 === aggregate.json.canonical_sha256_without_self, "compact→aggregate canonical link mismatch");
expect(reproduction.json.scientific_payload_sha256 === compact.json.replay.scientific_payload_sha256, "reproduction→replay hash link mismatch");
expect(reproduction.json.passed && compact.json.replay.passed, "full replay is not exact");
expect(reproduction.json.post_result_grok_review.blocking_errors === 0, "post-result audit reports a blocking error");
expect(reproduction.json.post_result_grok_review.status_confirmed, "post-result audit did not confirm status");
expect(reproduction.json.post_result_grok_review.replay_confirmed, "post-result audit did not confirm replay");
expect(compact.json.metadata_warnings.length === 0, "historical pairing metadata warning appeared");
const rawNames = readdirSync(rawDirectory).filter((name) => name.endsWith(".json")).sort();
expect(rawNames.length === 13, "raw matrix is not 12 formal + 1 replay");
const expectedRaw = [
...aggregate.json.input_files.formal.map((item) => item),
{ ...aggregate.json.input_files.replay, variant: "uniform_groups_6_7_forward", seed: 2026073001 },
];
for (const item of expectedRaw) {
const name = item.path.split("/").at(-1);
const raw = read(`../experiments/k3/attnres_forward/results/raw/${name}`);
expect(raw.sha256 === item.sha256, `${name} physical SHA-256 mismatch`);
expect(/^[0-9a-f]{64}$/.test(raw.json.canonical_sha256_without_self), `${name} canonical self-hash missing`);
expect(raw.json.forward_intervention.passed, `${name} forward audit failed`);
expect(raw.json.forward_intervention.forward_calls === 8054, `${name} forward census changed`);
expect(raw.json.target_bytes_seen === 65536000, `${name} target bytes changed`);
}
const primary = compact.json.primary;
expect(primary.material_response_passed, "primary material response failed");
expect(primary.passed_cells === 6 && primary.required_cells === 6, "primary attenuation is not 6/6");
expect(primary.cells.every((cell) => cell.relative_drop >= 0.2 && cell.passed), "a primary attenuation cell fell below 20%");
expect(primary.quality.passed && primary.quality.passed_checks === 4, "BPC quality gate is not 4/4");
expect(close(primary.quality.mean_delta_bpc, 0.0065784582165467525), "mean BPC delta changed");
expect(close(Math.min(...primary.cells.filter((cell) => cell.metric === "spike_contrast").map((cell) => cell.relative_drop)), 0.6211080443849011), "minimum contrast drop changed");
expect(close(Math.min(...primary.cells.filter((cell) => cell.metric === "peak_normalized").map((cell) => cell.relative_drop)), 0.3234439150908039), "minimum peak drop changed");
expect(Object.values(compact.json.secondary_status).every((value) => value === "secondary_material_response"), "secondary status changed");
expect(compact.json.trajectories.length === 90, "trajectory cell count changed");
expect(compact.json.final_spectra.length === 15, "final spectrum count changed");
expect(compact.json.final_spectra.every((item) => item.normalized.length === 32), "a final spectrum is not 32 layers");
expect(compact.json.interaction.cells.length === 36, "interaction cell count changed");
expect(compact.json.interaction.summaries.length === 12, "interaction summary count changed");
expect(close(
compact.json.interaction.summaries.find((item) =>
item.step === 8000 && item.metric === "spike_contrast").mean_interaction_residual,
-0.3670379672721313,
), "final contrast I67 changed");
expect(close(
compact.json.interaction.summaries.find((item) =>
item.step === 8000 && item.metric === "peak_normalized").mean_interaction_residual,
-0.17044758609080035,
), "final peak I67 changed");
if (failures.length) {
console.error(`FAIL K3 AttnRes forward-training data\n- ${failures.join("\n- ")}`);
process.exit(1);
}
console.log(JSON.stringify({
protocol: compact.json.protocol_id,
status: compact.json.status,
rawRuns: rawNames.length,
attenuation: `${primary.passed_cells}/${primary.required_cells}`,
quality: `${primary.quality.passed_checks}/${primary.quality.required_checks}`,
replayExact: compact.json.replay.passed,
bytes: compact.json.processed_target_bytes,
hashes: {
aggregate: aggregate.sha256,
compact: compact.sha256,
reproduction: reproduction.sha256,
},
}, null, 2));
console.log("PASS K3 AttnRes forward-training frozen data");
@@ -0,0 +1,292 @@
import { writeFileSync } from "node:fs";
const cdpPort = process.env.CDP_PORT ?? "9230";
const baseUrl = process.env.SITE_URL ?? "http://127.0.0.1:4329";
const pages = await fetch(`http://127.0.0.1:${cdpPort}/json/list`).then((response) => response.json());
const page = pages.find((entry) => entry.type === "page");
if (!page) throw new Error(`CDP ${cdpPort} 没有可用页面`);
const socket = new WebSocket(page.webSocketDebuggerUrl);
await new Promise((resolve, reject) => {
socket.addEventListener("open", resolve, { once: true });
socket.addEventListener("error", reject, { once: true });
});
let nextId = 0;
const pending = new Map();
const exceptions = [];
socket.addEventListener("message", (event) => {
const message = JSON.parse(event.data);
if (message.id && pending.has(message.id)) {
const { resolve, reject } = pending.get(message.id);
pending.delete(message.id);
if (message.error) reject(new Error(message.error.message));
else resolve(message.result);
}
if (message.method === "Runtime.exceptionThrown") {
exceptions.push(message.params.exceptionDetails.exception?.description ?? message.params.exceptionDetails.text);
}
});
const command = (method, params = {}) => new Promise((resolve, reject) => {
const id = ++nextId;
pending.set(id, { resolve, reject });
socket.send(JSON.stringify({ id, method, params }));
});
const pause = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
const evaluate = async (expression) => {
const result = await command("Runtime.evaluate", { expression, returnByValue: true, awaitPromise: true });
if (result.exceptionDetails) throw new Error(result.exceptionDetails.exception?.description ?? result.exceptionDetails.text);
return result.result.value;
};
const navigate = async (path) => {
await command("Page.navigate", { url: `${baseUrl}${path}` });
for (let attempt = 0; attempt < 100; attempt += 1) {
await pause(100);
if (await evaluate("document.readyState === 'complete'")) return;
}
throw new Error(`${path} 加载超时`);
};
const screenshot = async (path) => {
const result = await command("Page.captureScreenshot", { format: "png", captureBeyondViewport: false });
writeFileSync(path, Buffer.from(result.data, "base64"));
};
await command("Page.enable");
await command("Runtime.enable");
await command("Emulation.setDeviceMetricsOverride", {
width: 1440,
height: 1100,
deviceScaleFactor: 1,
mobile: false,
});
await navigate("/k3/");
const desktop = await evaluate(`(() => {
const root = document.querySelector("[data-local-path-lab]");
root.scrollIntoView({ block: "start", behavior: "instant" });
window.scrollBy(0, -78);
const text = (selector) => root.querySelector(selector)?.textContent.trim();
const panel = () => root.querySelector("[data-local-panel]:not([hidden])")?.dataset.localPanel;
const setSelect = (selector, value) => {
const node = root.querySelector(selector);
node.value = value;
node.dispatchEvent(new Event("change", { bubbles: true }));
};
const findMatrix = (needle) => [...root.querySelectorAll(".matrix-bars article")]
.find((node) => node.textContent.includes(needle))?.textContent.replace(/\\s+/g, " ").trim();
const initial = {
panel: panel(),
tabs: root.querySelectorAll("[data-local-tab]").length,
panels: root.querySelectorAll("[data-local-panel]").length,
ledger: root.querySelectorAll(".local-ledger article").length,
groups: root.querySelectorAll(".group-map article.target").length,
spikeLayers: root.querySelectorAll(".group-map i.spike").length,
boundary: root.textContent.includes("LOCALIZATION NOT ESTABLISHED") &&
root.textContent.includes("不是贡献率") &&
root.textContent.includes("不是 K3 checkpoint"),
};
root.querySelector('[data-local-tab="matrix"]').click();
const matrixInitial = {
panel: panel(),
rows: root.querySelectorAll(".matrix-bars article").length,
state: text("[data-local-matrix-state]"),
group67: findMatrix("仅 Groups 6+7 uniform"),
restoration67: findMatrix("恢复 Groups 6+7"),
};
setSelect("[data-local-matrix-seed]", "2026073002");
root.querySelector('[data-local-matrix-metric="peak_normalized"]').click();
const matrixChanged = {
rows: root.querySelectorAll(".matrix-bars article").length,
state: text("[data-local-matrix-state]"),
group67: findMatrix("仅 Groups 6+7 uniform"),
restoration67: findMatrix("恢复 Groups 6+7"),
};
root.querySelector('[data-local-tab="dual"]').click();
const dual = {
panel: panel(),
rows: root.querySelectorAll(".dual-table tbody tr").length,
good: root.querySelectorAll(".dual-table td.good").length,
bad: root.querySelectorAll(".dual-table td.bad").length,
verdict: text(".verdict-banner"),
mean: root.querySelector(".dual-table tr.mean")?.textContent.replace(/\\s+/g, " ").trim(),
};
root.querySelector('[data-local-tab="branch"]').click();
const branchInitial = {
panel: panel(),
state: text("[data-local-branch-state]"),
values: [...root.querySelectorAll("[data-local-branch-value]")].map((node) => node.textContent.trim()),
passed: root.dataset.branchPassed,
};
root.querySelector('[data-local-branch-group="7"]').click();
const branchChanged = {
state: text("[data-local-branch-state]"),
values: [...root.querySelectorAll("[data-local-branch-value]")].map((node) => node.textContent.trim()),
labels: [...root.querySelectorAll("[data-local-branch-label]")].map((node) => node.textContent.trim()),
passed: root.dataset.branchPassed,
};
root.querySelector('[data-local-tab="spectrum"]').click();
const spectrumInitial = {
panel: panel(),
points: root.querySelectorAll("[data-local-spectrum-points] circle").length,
line: root.querySelector("[data-local-spectrum-line]").getAttribute("points"),
state: text("[data-local-spectrum-state]"),
count: text("[data-local-spectrum-count]"),
contrast: text("[data-local-spectrum-contrast]"),
peak: text("[data-local-spectrum-peak]"),
layer: text("[data-local-spectrum-layer]"),
};
setSelect("[data-local-spectrum-seed]", "2026073002");
setSelect("[data-local-spectrum-mode]", "uniform_groups_6_7_only");
const spectrumGroup67 = {
points: root.querySelectorAll("[data-local-spectrum-points] circle").length,
line: root.querySelector("[data-local-spectrum-line]").getAttribute("points"),
state: text("[data-local-spectrum-state]"),
count: text("[data-local-spectrum-count]"),
contrast: text("[data-local-spectrum-contrast]"),
peak: text("[data-local-spectrum-peak]"),
layer: text("[data-local-spectrum-layer]"),
selectedPeakRadius: root.querySelector('[data-local-spectrum-points] circle[data-layer="25"]')?.getAttribute("r"),
};
setSelect("[data-local-spectrum-seed]", "2026073003");
setSelect("[data-local-spectrum-mode]", "uniform_all");
const spectrumAll = {
points: root.querySelectorAll("[data-local-spectrum-points] circle").length,
state: text("[data-local-spectrum-state]"),
count: text("[data-local-spectrum-count]"),
layer: text("[data-local-spectrum-layer]"),
selectedPeakRadius: root.querySelector('[data-local-spectrum-points] circle[data-layer="2"]')?.getAttribute("r"),
};
const first = root.querySelector('[data-local-tab="scope"]');
first.focus();
first.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowRight", bubbles: true }));
const keyboard = {
selected: root.querySelector('[data-local-tab][aria-selected="true"]').dataset.localTab,
panel: panel(),
};
return {
initial, matrixInitial, matrixChanged, dual, branchInitial, branchChanged,
spectrumInitial, spectrumGroup67, spectrumAll, keyboard,
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
rootOverflow: root.scrollWidth - root.clientWidth,
};
})()`);
await pause(180);
await screenshot("/tmp/llm-atlas-k3-attnres-local-path-desktop.png");
await command("Emulation.setDeviceMetricsOverride", {
width: 390,
height: 844,
deviceScaleFactor: 1,
mobile: true,
});
await navigate("/k3/");
const mobile = await evaluate(`(() => {
const root = document.querySelector("[data-local-path-lab]");
root.scrollIntoView({ block: "start", behavior: "instant" });
window.scrollBy(0, -64);
root.querySelector('[data-local-tab="dual"]').click();
const dual = {
panel: root.querySelector("[data-local-panel]:not([hidden])")?.dataset.localPanel,
rows: root.querySelectorAll(".dual-table tbody tr").length,
wrapperOverflow: root.querySelector(".dual-table-wrap").scrollWidth -
root.querySelector(".dual-table-wrap").clientWidth,
};
root.querySelector('[data-local-tab="spectrum"]').click();
return {
tabs: root.querySelectorAll("[data-local-tab]").length,
ledger: root.querySelectorAll(".local-ledger article").length,
visiblePanel: root.querySelector("[data-local-panel]:not([hidden])")?.dataset.localPanel,
points: root.querySelectorAll("[data-local-spectrum-points] circle").length,
dual,
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
rootOverflow: root.scrollWidth - root.clientWidth,
};
})()`);
await pause(180);
await screenshot("/tmp/llm-atlas-k3-attnres-local-path-mobile.png");
const report = { desktop, mobile, exceptions };
console.log(JSON.stringify(report, null, 2));
const numeric = (value) => Number.parseFloat(value.replace("−", "-").replace("×", ""));
const failures = [];
if (desktop.initial.panel !== "scope" || desktop.initial.tabs !== 5 || desktop.initial.panels !== 5 ||
desktop.initial.ledger !== 6 || desktop.initial.groups !== 2 || desktop.initial.spikeLayers !== 5) {
failures.push("五视图、六项账本或预注册路径地图结构异常");
}
if (!desktop.initial.boundary) failures.push("localization / contribution / reduced-model claim boundary 缺失");
if (desktop.matrixInitial.panel !== "matrix" || desktop.matrixInitial.rows !== 14 ||
!desktop.matrixInitial.state.includes("3-SEED MEAN") ||
!desktop.matrixInitial.group67.includes("+0.677") ||
!desktop.matrixInitial.restoration67.includes("+0.650")) {
failures.push("14-mask mean contrast 矩阵异常");
}
if (desktop.matrixChanged.rows !== 14 || !desktop.matrixChanged.state.includes("2026073002") ||
!desktop.matrixChanged.state.includes("PEAK / MEAN") ||
!desktop.matrixChanged.group67.includes("+1.783") ||
!desktop.matrixChanged.restoration67.includes("+0.355")) {
failures.push("matrix seed / metric 切换异常");
}
if (desktop.dual.panel !== "dual" || desktop.dual.rows !== 4 || desktop.dual.good !== 9 ||
desktop.dual.bad !== 6 || !desktop.dual.verdict.includes("LOCALIZATION NOT ESTABLISHED") ||
!desktop.dual.mean.includes("0.677") || !desktop.dual.mean.includes("0.380") ||
!desktop.dual.mean.includes("3 / 6")) {
failures.push("双向主门表格或冻结判定异常");
}
if (desktop.branchInitial.panel !== "branch" || desktop.branchInitial.passed !== "false" ||
!desktop.branchInitial.state.includes("5 / 6") ||
desktop.branchChanged.passed !== "true" || !desktop.branchChanged.state.includes("6 / 6 PASS") ||
desktop.branchChanged.labels.some((value) => value !== "GROUP 7") ||
Math.abs(numeric(desktop.branchChanged.values[0]) - .015) > .001 ||
Math.abs(numeric(desktop.branchChanged.values[1]) - .026) > .001 ||
Math.abs(numeric(desktop.branchChanged.values[2]) - .426) > .001 ||
Math.abs(numeric(desktop.branchChanged.values[3]) - .823) > .001) {
failures.push("group 6 / 7 branch gate 切换异常");
}
if (desktop.spectrumInitial.panel !== "spectrum" || desktop.spectrumInitial.points !== 32 ||
desktop.spectrumInitial.count !== "0 / 65" || numeric(desktop.spectrumInitial.contrast) !== 3.093 ||
numeric(desktop.spectrumInitial.peak) !== 3.241 || desktop.spectrumInitial.layer !== "21") {
failures.push("reference 32 层谱异常");
}
if (desktop.spectrumGroup67.points !== 32 || desktop.spectrumGroup67.line === desktop.spectrumInitial.line ||
!desktop.spectrumGroup67.state.includes("2026073002") ||
desktop.spectrumGroup67.count !== "16 / 65" ||
numeric(desktop.spectrumGroup67.contrast) !== 1.263 ||
numeric(desktop.spectrumGroup67.peak) !== 1.326 ||
desktop.spectrumGroup67.layer !== "25" || desktop.spectrumGroup67.selectedPeakRadius !== "4.5") {
failures.push("groups 6+7 spectrum seed / mode / peak 切换异常");
}
if (desktop.spectrumAll.points !== 32 || desktop.spectrumAll.count !== "65 / 65" ||
desktop.spectrumAll.layer !== "2" || desktop.spectrumAll.selectedPeakRadius !== "4.5") {
failures.push("all-uniform spectrum selector census 或 peak 异常");
}
if (desktop.keyboard.selected !== "matrix" || desktop.keyboard.panel !== "matrix") {
failures.push("键盘 tab 导航异常");
}
if (desktop.documentOverflow > 1 || desktop.rootOverflow > 1 ||
mobile.documentOverflow > 1 || mobile.rootOverflow > 1) {
failures.push("桌面或移动端出现文档级横向溢出");
}
if (mobile.tabs !== 5 || mobile.ledger !== 6 || mobile.visiblePanel !== "spectrum" ||
mobile.points !== 32 || mobile.dual.panel !== "dual" || mobile.dual.rows !== 4 ||
mobile.dual.wrapperOverflow <= 0) {
failures.push("移动端交互结构或局部可滚动表格异常");
}
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
if (failures.length) {
console.error(`\nFAIL\n- ${failures.join("\n- ")}`);
process.exitCode = 1;
} else {
console.log("\nPASS K3 AttnRes local-path browser regression");
}
socket.close();
@@ -0,0 +1,106 @@
import { createHash } from "node:crypto";
import { readdirSync, readFileSync } from "node:fs";
const hash = (bytes) => createHash("sha256").update(bytes).digest("hex");
const read = (path) => {
const bytes = readFileSync(new URL(path, import.meta.url));
return { bytes, json: JSON.parse(bytes), sha256: hash(bytes) };
};
const aggregate = read("../src/data/k3-attnres-local-path.json");
const compact = read("../src/data/k3-attnres-local-path-compact.json");
const reproduction = read("../experiments/k3/attnres_local_path/reproduction.json");
const manifest = read("../experiments/k3/attnres_local_path/manifest.json");
const rawDirectory = new URL("../experiments/k3/attnres_local_path/results/raw/", import.meta.url);
const failures = [];
const expect = (condition, message) => {
if (!condition) failures.push(message);
};
const close = (actual, expected, tolerance = 1e-15) =>
Math.abs(actual - expected) <= tolerance;
expect(aggregate.sha256 === "bb0ec9fce5b30d50ad5c50c4b95af7a892d614f205a2c20cfc2662125e10160e", "aggregate physical SHA-256 changed");
expect(compact.sha256 === "3bb6c15815f37d2109386fb87f9642246d41e636242ca3a9ba3c17ae8f419c08", "compact physical SHA-256 changed");
expect(reproduction.sha256 === "524a6883c08941c1be908834d7bcf915eb32f0bdb501aae09f3cec761fdd65db", "reproduction physical SHA-256 changed");
expect(manifest.sha256 === "db01e92ef2cf0896212fcd529429bd94a344de0e1195db7f87b9a56dc3449139", "manifest physical SHA-256 changed");
expect(aggregate.json.canonical_sha256_without_self === "b86d119cd2f106e2cbee8a35760ed3244336a2fcfeb9178ea1e7dab13fc6f215", "aggregate canonical SHA-256 changed");
expect(compact.json.canonical_sha256_without_self === "2aff9288f52d3d41bb1f59c64d9a07518ad3e2120c61615478087b24aaabd835", "compact canonical SHA-256 changed");
expect(reproduction.json.canonical_sha256_without_self === "6f5d98fce6446fecc966dd2675f272f2c4f0c9a39a5741fabc4ffad6852ca7f4", "reproduction canonical SHA-256 changed");
expect(compact.json.protocol_id === "llm-atlas-k3-attnres-local-path-v1", "protocol identity mismatch");
expect(compact.json.study.seeds.length === 3, "formal seed count changed");
expect(compact.json.study.steps === 8000, "formal step budget changed");
expect(compact.json.study.modes.length === 14, "matrix mode count changed");
expect(compact.json.study.spike_layers.join(",") === "21,22,23,24,25", "fixed spike set changed");
expect(compact.json.study.formal_target_bytes === 196608000, "formal target-byte count changed");
expect(compact.json.study.total_target_bytes_with_replay === 262144000, "total target-byte count changed");
expect(readdirSync(rawDirectory).filter((name) => name.endsWith(".json")).length === 4, "raw run count is not four");
for (const [name, expected] of Object.entries(reproduction.json.raw_files)) {
const raw = read(`../experiments/k3/attnres_local_path/results/raw/${name}`);
expect(raw.sha256 === expected.file_sha256, `${name} physical hash mismatch`);
expect(raw.json.canonical_sha256_without_self === expected.canonical_sha256, `${name} canonical hash mismatch`);
expect(raw.json.round06_equivalence.passed, `${name} Round 06 equivalence failed`);
}
expect(compact.json.hashes.aggregate_canonical_sha256 === aggregate.json.canonical_sha256_without_self, "compact→aggregate canonical link mismatch");
expect(compact.json.hashes.reproduction_canonical_sha256 === reproduction.json.canonical_sha256_without_self, "compact→reproduction canonical link mismatch");
expect(reproduction.json.aggregate.canonical_sha256 === aggregate.json.canonical_sha256_without_self, "reproduction→aggregate canonical link mismatch");
expect(reproduction.json.replay_gate.passed, "full replay is not exact");
expect(reproduction.json.replay_gate.frozen_compare_sha256 === "7dbd15ad03fbd357c5d91e159706d63b24703722f76c492ed1dc733535d6b9cf", "replay compare hash changed");
expect(reproduction.json.post_result_grok_review.blocking_errors === 0, "post-result audit reports a blocking error");
expect(reproduction.json.post_result_grok_review.localization_status_confirmed, "post-result audit did not confirm the status");
const gates = compact.json.gates;
expect(gates.all_input_and_parent_gates_passed, "input or parent gate failed");
expect(gates.global_gap.passed && gates.global_gap.passed_cells === 6, "global gap gate changed");
expect(gates.sufficiency.groups_6_7.passed && gates.sufficiency.groups_6_7.passed_cells === 6, "groups 6+7 sufficiency gate failed");
expect(!gates.restoration.groups_6_7.passed && gates.restoration.groups_6_7.passed_cells === 3, "groups 6+7 restoration verdict changed");
expect(!gates.localization.passed, "localization unexpectedly passed");
expect(gates.localization.status === "one_sided_evidence_localization_not_established", "localization status changed");
expect(gates.sufficiency.group_6.passed && gates.sufficiency.group_7.passed, "single-group sufficiency gate changed");
expect(!gates.restoration.group_6.passed && !gates.restoration.group_7.passed, "single-group restoration unexpectedly passed");
expect(!gates.sufficiency.output_half_gap.passed && gates.sufficiency.output_half_gap.passed_cells === 0, "output half-gap control changed");
expect(!gates.branch_dominance.group_6.passed, "group 6 branch dominance unexpectedly passed");
expect(gates.branch_dominance.group_7.passed && gates.branch_dominance.group_7.dominant_branch === "mlp", "group 7 MLP dominance changed");
expect(close(compact.json.means.sufficiency.uniform_groups_6_7_only.spike_contrast, 0.6769080107044138), "groups 6+7 mean contrast sufficiency changed");
expect(close(compact.json.means.sufficiency.uniform_groups_6_7_only.peak_normalized, 1.7003398291502192), "groups 6+7 mean peak sufficiency changed");
expect(close(compact.json.means.restoration.uniform_except_groups_6_7.spike_contrast, 0.6499896714884171), "groups 6+7 mean contrast restoration changed");
expect(close(compact.json.means.restoration.uniform_except_groups_6_7.peak_normalized, 0.3801045652660336), "groups 6+7 mean peak restoration changed");
expect(close(compact.json.means.sufficiency.uniform_output_only.spike_contrast, 0.13066777032743535), "output-only mean contrast score changed");
expect(close(compact.json.means.sufficiency.uniform_output_only.peak_normalized, 0.18724852586461135), "output-only mean peak score changed");
for (const spectrum of compact.json.final_spectra) {
expect(Object.keys(spectrum.modes).length === 14, `seed ${spectrum.seed} spectrum mode count changed`);
expect(spectrum.modes.detached_learned.uniform_count === 0, `seed ${spectrum.seed} reference census changed`);
expect(spectrum.modes.uniform_groups_6_7_only.uniform_count === 16, `seed ${spectrum.seed} groups 6+7 census changed`);
expect(spectrum.modes.uniform_except_groups_6_7.uniform_count === 49, `seed ${spectrum.seed} restoration census changed`);
expect(spectrum.modes.uniform_all.uniform_count === 65, `seed ${spectrum.seed} all-uniform census changed`);
for (const mode of Object.values(spectrum.modes)) {
expect(mode.normalized.length === 32, `seed ${spectrum.seed} normalized spectrum length changed`);
}
}
if (failures.length) {
console.error(`FAIL K3 AttnRes local-path data\n- ${failures.join("\n- ")}`);
process.exit(1);
}
console.log(JSON.stringify({
protocol: compact.json.protocol_id,
modes: compact.json.study.modes.length,
formalRuns: compact.json.study.seeds.length,
replayExact: reproduction.json.replay_gate.passed,
globalGap: gates.global_gap,
sufficiency: gates.sufficiency.groups_6_7,
restoration: gates.restoration.groups_6_7,
localization: gates.localization,
branch: gates.branch_dominance,
hashes: {
aggregate: aggregate.sha256,
compact: compact.sha256,
reproduction: reproduction.sha256,
},
}, null, 2));
console.log("PASS K3 AttnRes local-path frozen data");
+216
View File
@@ -0,0 +1,216 @@
import { writeFileSync } from "node:fs";
const cdpPort = process.env.CDP_PORT ?? "9229";
const baseUrl = process.env.SITE_URL ?? "http://127.0.0.1:4329";
const pages = await fetch(`http://127.0.0.1:${cdpPort}/json/list`).then((response) => response.json());
const page = pages.find((entry) => entry.type === "page");
if (!page) throw new Error(`CDP ${cdpPort} 没有可用页面`);
const socket = new WebSocket(page.webSocketDebuggerUrl);
await new Promise((resolve, reject) => {
socket.addEventListener("open", resolve, { once: true });
socket.addEventListener("error", reject, { once: true });
});
let nextId = 0;
const pending = new Map();
const exceptions = [];
socket.addEventListener("message", (event) => {
const message = JSON.parse(event.data);
if (message.id && pending.has(message.id)) {
const { resolve, reject } = pending.get(message.id);
pending.delete(message.id);
if (message.error) reject(new Error(message.error.message));
else resolve(message.result);
}
if (message.method === "Runtime.exceptionThrown") {
exceptions.push(message.params.exceptionDetails.exception?.description ?? message.params.exceptionDetails.text);
}
});
const command = (method, params = {}) => new Promise((resolve, reject) => {
const id = ++nextId;
pending.set(id, { resolve, reject });
socket.send(JSON.stringify({ id, method, params }));
});
const pause = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
const evaluate = async (expression) => {
const result = await command("Runtime.evaluate", { expression, returnByValue: true, awaitPromise: true });
if (result.exceptionDetails) throw new Error(result.exceptionDetails.exception?.description ?? result.exceptionDetails.text);
return result.result.value;
};
const navigate = async (path) => {
await command("Page.navigate", { url: `${baseUrl}${path}` });
for (let attempt = 0; attempt < 100; attempt += 1) {
await pause(100);
if (await evaluate("document.readyState === 'complete'")) return;
}
throw new Error(`${path} 加载超时`);
};
const screenshot = async (path) => {
const result = await command("Page.captureScreenshot", { format: "png", captureBeyondViewport: false });
writeFileSync(path, Buffer.from(result.data, "base64"));
};
await command("Page.enable");
await command("Runtime.enable");
await command("Emulation.setDeviceMetricsOverride", {
width: 1440,
height: 1100,
deviceScaleFactor: 1,
mobile: false,
});
await navigate("/k3/");
const desktop = await evaluate(`(() => {
const root = document.querySelector("[data-spike-lab]");
root.scrollIntoView({ block: "start", behavior: "instant" });
window.scrollBy(0, -78);
const text = (selector) => root.querySelector(selector)?.textContent.trim();
const panel = () => root.querySelector("[data-spike-panel]:not([hidden])")?.dataset.spikePanel;
const points = (selector) => root.querySelector(selector)?.getAttribute("points");
const setSelect = (selector, value) => {
const node = root.querySelector(selector);
node.value = value;
node.dispatchEvent(new Event("change", { bubbles: true }));
};
const initial = {
panel: panel(),
tabs: root.querySelectorAll("[data-spike-tab]").length,
panels: root.querySelectorAll("[data-spike-panel]").length,
ledger: root.querySelectorAll(".spike-ledger article").length,
trajectoryPoints: root.querySelectorAll("[data-spike-time-points] circle").length,
trajectoryLine: points("[data-spike-time-line]"),
boundary: root.textContent.includes("不是训练出的 uniform 模型") &&
root.textContent.includes("不是 70.2% 因果贡献"),
};
root.querySelector('[data-spike-time-metric="population_cv"]').click();
setSelect("[data-spike-time-seed]", "2026073003");
const trajectoryChanged = {
title: text("[data-spike-time-title]"),
state: text("[data-spike-time-state]"),
line: points("[data-spike-time-line]"),
thresholdHidden: getComputedStyle(root.querySelector("[data-spike-time-threshold]")).display === "none",
};
root.querySelector('[data-spike-tab="positions"]').click();
root.querySelector('[data-spike-position="pre_mlp_input"]').click();
setSelect("[data-spike-position-seed]", "2026073002");
const positions = {
panel: panel(),
buttons: root.querySelectorAll("[data-spike-position]").length,
state: text("[data-spike-position-state]"),
contrast: text("[data-spike-position-contrast]"),
peak: text("[data-spike-position-peak]"),
peakValue: text("[data-spike-position-peak-value]"),
pointCount: root.querySelectorAll("[data-spike-position-points] circle").length,
};
root.querySelector('[data-spike-tab="reductions"]').click();
setSelect("[data-spike-reduction-seed]", "2026073003");
setSelect("[data-spike-reduction]", "token_rms_median");
const reductions = {
panel: panel(),
rows: root.querySelectorAll(".reduction-table tbody tr").length,
state: text("[data-spike-reduction-state]"),
contrast: text("[data-spike-reduction-contrast]"),
overlap: text("[data-spike-reduction-overlap]"),
rho: text("[data-spike-reduction-rho]"),
pointCount: root.querySelectorAll("[data-spike-reduction-points] circle").length,
};
root.querySelector('[data-spike-tab="intervention"]').click();
setSelect("[data-spike-intervention-seed]", "2026073002");
const intervention = {
panel: panel(),
lines: root.querySelectorAll("[data-spike-intervention-line]").length,
keyDrop: text("[data-spike-key-drop]"),
valueDrop: text("[data-spike-value-drop]"),
peakShift: text("[data-spike-peak-shift]"),
linePoints: [...root.querySelectorAll("[data-spike-intervention-line]")].map((node) => node.getAttribute("points").split(" ").length),
forwardIdentity: root.textContent.includes("same logits · same loss"),
};
root.querySelector('[data-spike-tab="mixer"]').click();
const mixerInitial = {
panel: panel(),
points: root.querySelectorAll("[data-spike-mixer-points] circle").length,
pearson: text("[data-spike-mixer-pearson]"),
spearman: text("[data-spike-mixer-spearman]"),
title: text("[data-spike-mixer-title]"),
};
root.querySelector('[data-spike-mixer-metric="mlp_entropy"]').click();
const mixerChanged = {
points: root.querySelectorAll("[data-spike-mixer-points] circle").length,
pearson: text("[data-spike-mixer-pearson]"),
spearman: text("[data-spike-mixer-spearman]"),
title: text("[data-spike-mixer-title]"),
};
const first = root.querySelector('[data-spike-tab="trajectory"]');
first.focus();
first.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowRight", bubbles: true }));
const keyboard = {
selected: root.querySelector('[data-spike-tab][aria-selected="true"]').dataset.spikeTab,
panel: panel(),
};
return {
initial, trajectoryChanged, positions, reductions, intervention, mixerInitial, mixerChanged, keyboard,
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
rootOverflow: root.scrollWidth - root.clientWidth,
};
})()`);
await pause(180);
await screenshot("/tmp/llm-atlas-k3-attnres-spike-desktop.png");
await command("Emulation.setDeviceMetricsOverride", {
width: 390,
height: 844,
deviceScaleFactor: 1,
mobile: true,
});
await navigate("/k3/");
const mobile = await evaluate(`(() => {
const root = document.querySelector("[data-spike-lab]");
root.scrollIntoView({ block: "start", behavior: "instant" });
window.scrollBy(0, -64);
root.querySelector('[data-spike-tab="intervention"]').click();
return {
tabs: root.querySelectorAll("[data-spike-tab]").length,
ledger: root.querySelectorAll(".spike-ledger article").length,
visiblePanel: root.querySelector("[data-spike-panel]:not([hidden])")?.dataset.spikePanel,
lines: root.querySelectorAll("[data-spike-intervention-line]").length,
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
rootOverflow: root.scrollWidth - root.clientWidth,
};
})()`);
await pause(180);
await screenshot("/tmp/llm-atlas-k3-attnres-spike-mobile.png");
const report = { desktop, mobile, exceptions };
console.log(JSON.stringify(report, null, 2));
const numeric = (value) => Number.parseFloat(value.replace("−", "-").replace("%", ""));
const failures = [];
if (desktop.initial.panel !== "trajectory" || desktop.initial.tabs !== 5 || desktop.initial.panels !== 5 || desktop.initial.ledger !== 6 || desktop.initial.trajectoryPoints !== 6) failures.push("五视图或六点轨迹初始结构异常");
if (!desktop.initial.boundary) failures.push("因果与训练变体边界缺失");
if (!desktop.trajectoryChanged.title.includes("POPULATION CV") || !desktop.trajectoryChanged.state.includes("2026073003") || desktop.trajectoryChanged.line === desktop.initial.trajectoryLine || !desktop.trajectoryChanged.thresholdHidden) failures.push("轨迹 seed / metric 切换异常");
if (desktop.positions.panel !== "positions" || desktop.positions.buttons !== 6 || !desktop.positions.state.includes("MLP 前输入") || Math.abs(numeric(desktop.positions.contrast) - 4.527) > .001 || numeric(desktop.positions.peak) !== 21 || desktop.positions.pointCount !== 32) failures.push("六位置谱或 seed 切换异常");
if (desktop.reductions.panel !== "reductions" || desktop.reductions.rows !== 12 || !desktop.reductions.state.includes("Token RMS 中位数") || Math.abs(numeric(desktop.reductions.contrast) - 1.666) > .001 || desktop.reductions.overlap !== "3 / 5" || Math.abs(numeric(desktop.reductions.rho) - .934) > .001 || desktop.reductions.pointCount !== 32) failures.push("reduction robustness 视图异常");
if (desktop.intervention.panel !== "intervention" || desktop.intervention.lines !== 3 || Math.abs(numeric(desktop.intervention.keyDrop) + 1.93) > .01 || Math.abs(numeric(desktop.intervention.valueDrop) - 76.87) > .01 || desktop.intervention.peakShift !== "21 → 2" || desktop.intervention.linePoints.some((count) => count !== 32) || !desktop.intervention.forwardIdentity) failures.push("same-forward backward 干预视图异常");
if (desktop.mixerInitial.panel !== "mixer" || desktop.mixerInitial.points !== 30 || Math.abs(numeric(desktop.mixerInitial.pearson) - .690) > .001 || Math.abs(numeric(desktop.mixerInitial.spearman) - .693) > .001 || desktop.mixerChanged.points !== 30 || Math.abs(numeric(desktop.mixerChanged.pearson) + .636) > .001 || Math.abs(numeric(desktop.mixerChanged.spearman) + .640) > .001 || desktop.mixerInitial.title === desktop.mixerChanged.title) failures.push("mixer association 散点或指标切换异常");
if (desktop.keyboard.selected !== "positions" || desktop.keyboard.panel !== "positions") failures.push("键盘 tab 导航异常");
if (desktop.documentOverflow > 1 || desktop.rootOverflow > 1 || mobile.documentOverflow > 1 || mobile.rootOverflow > 1) failures.push("桌面或移动端出现文档级横向溢出");
if (mobile.tabs !== 5 || mobile.ledger !== 6 || mobile.visiblePanel !== "intervention" || mobile.lines !== 3) failures.push("移动端交互结构异常");
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
if (failures.length) {
console.error(`\nFAIL\n- ${failures.join("\n- ")}`);
process.exitCode = 1;
} else {
console.log("\nPASS K3 AttnRes spike browser regression");
}
socket.close();
+87
View File
@@ -0,0 +1,87 @@
import { createHash } from "node:crypto";
import { readdirSync, readFileSync } from "node:fs";
const hash = (bytes) => createHash("sha256").update(bytes).digest("hex");
const read = (path) => {
const bytes = readFileSync(new URL(path, import.meta.url));
return { bytes, json: JSON.parse(bytes), sha256: hash(bytes) };
};
const aggregate = read("../src/data/k3-attnres-spike.json");
const compact = read("../src/data/k3-attnres-spike-compact.json");
const reproduction = read("../experiments/k3/attnres_spike/reproduction.json");
const manifest = read("../experiments/k3/attnres_spike/manifest.json");
const rawDirectory = new URL("../experiments/k3/attnres_spike/results/raw/", import.meta.url);
const failures = [];
const expect = (condition, message) => {
if (!condition) failures.push(message);
};
expect(aggregate.sha256 === "75058f9ef1f89593436960428378ab920569017f1758f9c15a2a0b1e081c1e90", "aggregate physical SHA-256 changed");
expect(compact.sha256 === "7b3a3749d1344e56fb61039d2678a4e2aefe11a0a1e279516d6f27ff3bb8ef6c", "compact physical SHA-256 changed");
expect(reproduction.sha256 === "86ac20ee06189a924eed2b2614e535660b536447f5ad019f4c1fd99cf4aa879a", "reproduction physical SHA-256 changed");
expect(manifest.sha256 === "d5302a249249a07d362819134763d14e7d32307f22cff416c665ed9606142fef", "manifest physical SHA-256 changed");
expect(aggregate.json.canonical_sha256_without_self === "1d0faef297682d61d35ce2ab54da3868e18a84e9ebc6e44c617d08016e4fc8b4", "aggregate canonical SHA-256 changed");
expect(compact.json.canonical_sha256_without_self === "a430dd26f512db7d64c933df65d9a97e6899028099333fe0a1212728f13126cd", "compact canonical SHA-256 changed");
expect(reproduction.json.canonical_sha256_without_self === "7c522314742920961be84b750d823ce41814f10fd6129f0531208116e2fe3515", "reproduction canonical SHA-256 changed");
expect(compact.json.protocol_id === "llm-atlas-k3-attnres-spike-path-v1", "protocol identity mismatch");
expect(compact.json.study.seeds.length === 3, "formal seed count changed");
expect(compact.json.study.steps === 8000, "formal step budget changed");
expect(compact.json.study.spike_layers.join(",") === "21,22,23,24,25", "fixed spike set changed");
expect(readdirSync(rawDirectory).filter((name) => name.endsWith(".json")).length === 4, "raw run count is not four");
for (const [name, expected] of Object.entries(reproduction.json.raw_files)) {
const raw = read(`../experiments/k3/attnres_spike/results/raw/${name}`);
expect(raw.sha256 === expected.file_sha256, `${name} physical hash mismatch`);
expect(raw.json.canonical_sha256_without_self === expected.canonical_sha256, `${name} canonical hash mismatch`);
}
expect(compact.json.hashes.aggregate_canonical_sha256 === aggregate.json.canonical_sha256_without_self, "compact→aggregate canonical link mismatch");
expect(compact.json.hashes.reproduction_canonical_sha256 === reproduction.json.canonical_sha256_without_self, "compact→reproduction canonical link mismatch");
expect(aggregate.json.reproduction_canonical_sha256 === reproduction.json.canonical_sha256_without_self, "aggregate→reproduction canonical link mismatch");
expect(reproduction.json.replay_gate.passed, "full replay is not exact");
expect(reproduction.json.replay_gate.frozen_compare_sha256 === "05396eaf56429ca8d87815ecf00a5f78be5a9b2ec7cc96014856c09dcedcbb87", "replay compare hash changed");
expect(compact.json.gates.round05_equivalence.every((item) => item.passed), "Round 05 equivalence gate failed");
expect(compact.json.gates.forward_identity.every((item) => item.passed), "forward identity gate failed");
expect(compact.json.gates.initialization_negative_control.every((item) => item.passed), "initialization negative control failed");
expect(compact.json.gates.loss_scale.every((item) => item.passed), "loss-scale gate failed");
const verdicts = compact.json.verdicts;
expect(verdicts.reduction_robustness.verdict === "robust within the preregistered reduction family", "reduction robustness verdict changed");
expect(verdicts.reduction_robustness.passed_cells === 12, "reduction pass count changed");
expect(verdicts.visible_positions.visible_positions.length === 6, "visible-position count changed");
expect(verdicts.visible_positions.earliest_observed_tensor === "pre_attention_input", "earliest observed tensor changed");
expect(verdicts.interventions.softmax_key_path.opposite_direction_3_of_3, "softmax/key opposite-direction result changed");
expect(!verdicts.interventions.softmax_key_path.passed_3_of_3, "softmax/key path unexpectedly passed material threshold");
expect(verdicts.interventions.value_coefficients.passed_3_of_3, "value-coefficient material threshold failed");
expect(Math.abs(verdicts.interventions.value_coefficients.mean_relative_drop_contrast - 0.701998806119879) < 1e-15, "value contrast drop changed");
expect(Math.abs(verdicts.interventions.value_coefficients.mean_relative_drop_peak - 0.3702754647109951) < 1e-15, "value peak drop changed");
for (const run of compact.json.final_arrays) {
expect(Object.keys(run.position_element_rms).length === 6, `seed ${run.seed} position count changed`);
expect(Object.keys(run.post_mlp_reductions).length === 7, `seed ${run.seed} reduction count changed`);
expect(Object.keys(run.interventions).length === 3, `seed ${run.seed} intervention count changed`);
}
if (failures.length) {
console.error(`FAIL K3 AttnRes spike data\n- ${failures.join("\n- ")}`);
process.exit(1);
}
console.log(JSON.stringify({
protocol: compact.json.protocol_id,
formalRuns: compact.json.study.seeds.length,
replayExact: reproduction.json.replay_gate.passed,
reduction: verdicts.reduction_robustness.verdict,
visiblePositions: verdicts.visible_positions.visible_positions.length,
softmaxKey: verdicts.interventions.softmax_key_path.verdict,
valueCoefficients: verdicts.interventions.value_coefficients.verdict,
hashes: {
aggregate: aggregate.sha256,
compact: compact.sha256,
reproduction: reproduction.sha256,
},
}, null, 2));
console.log("PASS K3 AttnRes spike frozen data");
+18 -3
View File
@@ -83,6 +83,15 @@ const overview = await evaluate(`(() => ({
attnresPanels: document.querySelectorAll("[data-attnres-panel]").length, attnresPanels: document.querySelectorAll("[data-attnres-panel]").length,
gradientTabs: document.querySelectorAll("[data-gradient-tab]").length, gradientTabs: document.querySelectorAll("[data-gradient-tab]").length,
gradientPanels: document.querySelectorAll("[data-gradient-panel]").length, gradientPanels: document.querySelectorAll("[data-gradient-panel]").length,
spikeTabs: document.querySelectorAll("[data-spike-tab]").length,
spikePanels: document.querySelectorAll("[data-spike-panel]").length,
localPathTabs: document.querySelectorAll("[data-local-tab]").length,
localPathPanels: document.querySelectorAll("[data-local-panel]").length,
localPathVerdict: document.querySelector("#attnres-local-path")?.textContent.includes("localization 未建立"),
forwardTabs: document.querySelectorAll("[data-forward-tab]").length,
forwardPanels: document.querySelectorAll("[data-forward-panel]").length,
forwardVerdict: document.querySelector("#attnres-forward")?.textContent.includes("6 / 6 PASS") &&
document.querySelector("#attnres-forward")?.textContent.includes("reduced protocol only"),
nativeVisionCorrected: document.body.textContent.includes("MoonViT‑V2 从头训练") && nativeVisionCorrected: document.body.textContent.includes("MoonViT‑V2 从头训练") &&
document.body.textContent.includes("同一个 next-token prediction objective"), document.body.textContent.includes("同一个 next-token prediction objective"),
staleVisionClaim: document.body.textContent.includes("先固定语言模型训练视觉组件"), staleVisionClaim: document.body.textContent.includes("先固定语言模型训练视觉组件"),
@@ -293,8 +302,11 @@ const mobile = await evaluate(`(() => {
artifactLayers: document.querySelectorAll("[data-layer-cell]").length, artifactLayers: document.querySelectorAll("[data-layer-cell]").length,
attnresTabs: document.querySelectorAll("[data-attnres-tab]").length, attnresTabs: document.querySelectorAll("[data-attnres-tab]").length,
gradientTabs: document.querySelectorAll("[data-gradient-tab]").length, gradientTabs: document.querySelectorAll("[data-gradient-tab]").length,
spikeTabs: document.querySelectorAll("[data-spike-tab]").length,
localPathTabs: document.querySelectorAll("[data-local-tab]").length,
forwardTabs: document.querySelectorAll("[data-forward-tab]").length,
offenders: [...document.querySelectorAll("body *")] offenders: [...document.querySelectorAll("body *")]
.filter((node) => !node.closest(".paper-chain, .spec-table-wrap, .cache-strip, .architecture-explorer, [data-k3-lab], [data-k3-artifact-lab], [data-attnres-lab], [data-gradient-lab]")) .filter((node) => !node.closest(".paper-chain, .spec-table-wrap, .cache-strip, .architecture-explorer, [data-k3-lab], [data-k3-artifact-lab], [data-attnres-lab], [data-gradient-lab], [data-spike-lab], [data-local-path-lab], [data-forward-lab]"))
.filter((node) => node.getBoundingClientRect().right > document.documentElement.clientWidth + 1) .filter((node) => node.getBoundingClientRect().right > document.documentElement.clientWidth + 1)
.slice(0, 15) .slice(0, 15)
.map((node) => ({ .map((node) => ({
@@ -325,13 +337,16 @@ console.log(JSON.stringify(report, null, 2));
const numeric = (text) => Number.parseFloat(text.replaceAll(",", "").replace("−", "-")); const numeric = (text) => Number.parseFloat(text.replaceAll(",", "").replace("−", "-"));
const failures = []; const failures = [];
if (!overview.title.includes("因果环节")) failures.push("K3 二轮标题异常"); if (!overview.title.includes("因果环节")) failures.push("K3 二轮标题异常");
if (overview.sections !== 34 || overview.tocLinks !== 34) failures.push("33 个编号专题加阅读链的目录结构异常"); if (overview.sections !== 37 || overview.tocLinks !== 37) failures.push("36 个编号专题加阅读链的目录结构异常");
if (overview.ledgers !== 32 || overview.reportMap !== 9) failures.push("32 张问题账或报告地图异常"); if (overview.ledgers !== 32 || overview.reportMap !== 9) failures.push("32 张问题账或报告地图异常");
if (overview.figureAtlas !== 21 || overview.paperLinks !== 100 || overview.paperGroups < 12) failures.push("图表审计或 100 节点阅读链异常"); if (overview.figureAtlas !== 21 || overview.paperLinks !== 100 || overview.paperGroups < 12) failures.push("图表审计或 100 节点阅读链异常");
if (overview.labTabs !== 8 || overview.labPanels !== 8) failures.push("八联实验结构异常"); if (overview.labTabs !== 8 || overview.labPanels !== 8) failures.push("八联实验结构异常");
if (overview.artifactTabs !== 4 || overview.artifactPanels !== 4 || overview.artifactLayers !== 93 || !overview.artifactMismatch) failures.push("开放工件四视图、93 层条带或形状冲突异常"); if (overview.artifactTabs !== 4 || overview.artifactPanels !== 4 || overview.artifactLayers !== 93 || !overview.artifactMismatch) failures.push("开放工件四视图、93 层条带或形状冲突异常");
if (overview.attnresTabs !== 5 || overview.attnresPanels !== 5) failures.push("AttnRes 独立实验五视图异常"); if (overview.attnresTabs !== 5 || overview.attnresPanels !== 5) failures.push("AttnRes 独立实验五视图异常");
if (overview.gradientTabs !== 5 || overview.gradientPanels !== 5) failures.push("AttnRes 梯度定义扩展五视图异常"); if (overview.gradientTabs !== 5 || overview.gradientPanels !== 5) failures.push("AttnRes 梯度定义扩展五视图异常");
if (overview.spikeTabs !== 5 || overview.spikePanels !== 5) failures.push("AttnRes 尖峰路径五视图异常");
if (overview.localPathTabs !== 5 || overview.localPathPanels !== 5 || !overview.localPathVerdict) failures.push("AttnRes 局部路径五视图或冻结判定异常");
if (overview.forwardTabs !== 5 || overview.forwardPanels !== 5 || !overview.forwardVerdict) failures.push("AttnRes 训练期前向五视图或冻结判定异常");
if (!overview.nativeVisionCorrected || overview.staleVisionClaim) failures.push("原生多模态纠错未生效或旧错误残留"); if (!overview.nativeVisionCorrected || overview.staleVisionClaim) failures.push("原生多模态纠错未生效或旧错误残留");
if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出"); if (overview.documentOverflow > 1 || mobile.documentOverflow > 1) failures.push("桌面或移动端存在文档级横向溢出");
if (labs.memoryInitial.panel !== "memory" || numeric(labs.memoryInitial.additiveError) <= numeric(labs.memoryInitial.deltaError)) failures.push("Delta memory 初始递推异常"); if (labs.memoryInitial.panel !== "memory" || numeric(labs.memoryInitial.additiveError) <= numeric(labs.memoryInitial.deltaError)) failures.push("Delta memory 初始递推异常");
@@ -356,7 +371,7 @@ if (artifacts.parameterChanged.shape !== "[96,128] F32" || !artifacts.parameterC
if (artifacts.reproductionInitial.panel !== "reproduction" || numeric(artifacts.reproductionInitial.speedup) !== 1.85 || numeric(artifacts.reproductionInitial.localMean) < 2.6 || !artifacts.reproductionInitial.exactSuite || numeric(artifacts.reproductionInitial.cv) < 2) failures.push("FlashKDA H20、本机 exact suite 或 router 初始探针异常"); if (artifacts.reproductionInitial.panel !== "reproduction" || numeric(artifacts.reproductionInitial.speedup) !== 1.85 || numeric(artifacts.reproductionInitial.localMean) < 2.6 || !artifacts.reproductionInitial.exactSuite || numeric(artifacts.reproductionInitial.cv) < 2) failures.push("FlashKDA H20、本机 exact suite 或 router 初始探针异常");
if (numeric(artifacts.reproductionChanged.speedup) !== 3.27 || numeric(artifacts.reproductionChanged.flash) !== 0.7064 || numeric(artifacts.reproductionChanged.localMean) >= numeric(artifacts.reproductionInitial.localMean) || !artifacts.reproductionChanged.localMode.includes("FP32 state") || numeric(artifacts.reproductionChanged.cv) <= numeric(artifacts.reproductionInitial.cv) || numeric(artifacts.reproductionChanged.zero) <= numeric(artifacts.reproductionInitial.zero)) failures.push("GB200 benchmark、本机 varlen/state 或 synthetic router counterexample 未更新"); if (numeric(artifacts.reproductionChanged.speedup) !== 3.27 || numeric(artifacts.reproductionChanged.flash) !== 0.7064 || numeric(artifacts.reproductionChanged.localMean) >= numeric(artifacts.reproductionInitial.localMean) || !artifacts.reproductionChanged.localMode.includes("FP32 state") || numeric(artifacts.reproductionChanged.cv) <= numeric(artifacts.reproductionInitial.cv) || numeric(artifacts.reproductionChanged.zero) <= numeric(artifacts.reproductionInitial.zero)) failures.push("GB200 benchmark、本机 varlen/state 或 synthetic router counterexample 未更新");
if (artifacts.keyboardSelected !== "tensors" || artifacts.keyboardVisible !== "tensors") failures.push("开放工件键盘 tab 导航异常"); if (artifacts.keyboardSelected !== "tensors" || artifacts.keyboardVisible !== "tensors") failures.push("开放工件键盘 tab 导航异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 8 || mobile.artifactTabs !== 4 || mobile.artifactLayers !== 93 || mobile.attnresTabs !== 5 || mobile.gradientTabs !== 5) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 8 || mobile.artifactTabs !== 4 || mobile.artifactLayers !== 93 || mobile.attnresTabs !== 5 || mobile.gradientTabs !== 5 || mobile.spikeTabs !== 5 || mobile.localPathTabs !== 5 || mobile.forwardTabs !== 5) failures.push("移动端导航或实验异常");
if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`); if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`);
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`); if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
+1 -1
View File
@@ -246,7 +246,7 @@ if (ocr.unreported.status !== "OUT OF EVIDENCE" || ocr.unreported.accuracy !== "
if (loop.toolsStart.state !== "OPEN" || loop.toolsEnd.state !== "VERIFIED" || loop.toolsEnd.evidence !== "97%" || loop.toolsEnd.tools !== "3") failures.push("vision-in-the-loop 终局异常"); if (loop.toolsStart.state !== "OPEN" || loop.toolsEnd.state !== "VERIFIED" || loop.toolsEnd.evidence !== "97%" || loop.toolsEnd.tools !== "3") failures.push("vision-in-the-loop 终局异常");
if (loop.cotEnd.state !== "FAILED" || !loop.cotEnd.takeaway.includes("不能凭空增加")) failures.push("文字 CoT 与新观察没有分开"); if (loop.cotEnd.state !== "FAILED" || !loop.cotEnd.takeaway.includes("不能凭空增加")) failures.push("文字 CoT 与新观察没有分开");
if (loop.keyboardSelected !== "connector" || loop.keyboardVisible !== "connector") failures.push("实验键盘 tab 导航异常"); if (loop.keyboardSelected !== "connector" || loop.keyboardVisible !== "connector") failures.push("实验键盘 tab 导航异常");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") failures.push("首页 K3 首发入口异常");
if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasFilter || papers.multimodalVisible < 59) failures.push("论文库多模态标签或总数异常"); if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasFilter || papers.multimodalVisible < 59) failures.push("论文库多模态标签或总数异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`); if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`);
+1 -1
View File
@@ -277,7 +277,7 @@ if (layout.navLinks !== 20 || mobile.mobileLinks !== 20 || home.navLinks !== 20)
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出"); if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`); if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`);
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用"); if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") { if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") {
failures.push("首页 Transformer 新章入口异常"); failures.push("首页 Transformer 新章入口异常");
} }
if (home.paperCount !== "486") failures.push(`首页论文总数异常:${home.paperCount}`); if (home.paperCount !== "486") failures.push(`首页论文总数异常:${home.paperCount}`);
+1 -1
View File
@@ -289,7 +289,7 @@ if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentO
} }
if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`); if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`);
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用"); if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要")) failures.push("首页评测新章入口异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后")) failures.push("首页 K3 首发入口异常");
if (exceptions.length) failures.push(`浏览器脚本异常:${exceptions.join("; ")}`); if (exceptions.length) failures.push(`浏览器脚本异常:${exceptions.join("; ")}`);
socket.close(); socket.close();
+1 -1
View File
@@ -288,7 +288,7 @@ if (numeric(residual.attnres.states) !== 9 || !residual.attnres.routeExplain.inc
if (!residual.clamp.activation.includes("V4") || !residual.clamp.bound.includes("100")) failures.push("DeepSeek-V4 clamp 展示异常"); if (!residual.clamp.activation.includes("V4") || !residual.clamp.bound.includes("100")) failures.push("DeepSeek-V4 clamp 展示异常");
if (!residual.situ.activation.includes("KIMI") || !residual.situ.bound.includes("100")) failures.push("K3 SiTU 上界展示异常"); if (!residual.situ.activation.includes("KIMI") || !residual.situ.bound.includes("100")) failures.push("K3 SiTU 上界展示异常");
if (residual.keyboardSelected !== "position" || residual.keyboardVisible !== "position") failures.push("实验键盘 tab 导航异常"); if (residual.keyboardSelected !== "position" || residual.keyboardVisible !== "position") failures.push("实验键盘 tab 导航异常");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页表示新章入口异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") failures.push("首页 K3 首发入口异常");
if (home.paperCount !== "486" || home.topicCount !== "17" || papers.total !== 486 || !papers.hasFilter || papers.visible < 30) failures.push("首页 / 论文库表示索引异常"); if (home.paperCount !== "486" || home.topicCount !== "17" || papers.total !== 486 || !papers.hasFilter || papers.visible < 30) failures.push("首页 / 论文库表示索引异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`); if (mobile.offenders.length) failures.push(`移动端越界元素:${JSON.stringify(mobile.offenders)}`);
+1 -1
View File
@@ -273,7 +273,7 @@ if (layout.navLinks !== 20 || mobile.mobileLinks !== 20 || home.navLinks !== 20)
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出"); if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`); if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`);
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用"); if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") { if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") {
failures.push("首页 Transformer 新章入口异常"); failures.push("首页 Transformer 新章入口异常");
} }
if (home.paperCount !== "486") failures.push(`首页论文总数异常:${home.paperCount}`); if (home.paperCount !== "486") failures.push(`首页论文总数异常:${home.paperCount}`);
+1 -1
View File
@@ -233,7 +233,7 @@ if (layout.articleSections !== 16 || layout.paperLinks !== 37 || layout.labTabs
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出"); if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`); if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`);
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用"); if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要")) failures.push("首页评测新章入口异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后")) failures.push("首页 K3 首发入口异常");
if (exceptions.length) failures.push(`浏览器脚本异常:${exceptions.join("; ")}`); if (exceptions.length) failures.push(`浏览器脚本异常:${exceptions.join("; ")}`);
socket.close(); socket.close();
+1 -1
View File
@@ -236,7 +236,7 @@ if (block.family.trim() !== "Hybrid MoE" || !block.kv.includes("3 KDA : 1 Gated
if (!block.path.some((step) => step.includes("KDA × 3")) || !block.note.includes("AttnRes")) failures.push("K3 Block 路径异常"); if (!block.path.some((step) => step.includes("KDA × 3")) || !block.note.includes("AttnRes")) failures.push("K3 Block 路径异常");
if (block.context.trim() !== "128K" || numeric(block.mha) !== 400 || numeric(block.kda) !== 1) failures.push("KV 成本缩放异常"); if (block.context.trim() !== "128K" || numeric(block.mha) !== 400 || numeric(block.kda) !== 1) failures.push("KV 成本缩放异常");
if (block.keyboardSelected !== "block" || block.keyboardVisible !== "block") failures.push("实验 tab 键盘导航异常"); if (block.keyboardSelected !== "block" || block.keyboardVisible !== "block") failures.push("实验 tab 键盘导航异常");
if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常"); if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") failures.push("首页 K3 首发入口异常");
if (home.paperCount !== "486" || papers.total !== 486 || papers.transformerVisible < 30) failures.push("论文库或首页论文数量异常"); if (home.paperCount !== "486" || papers.total !== 486 || papers.transformerVisible < 30) failures.push("论文库或首页论文数量异常");
if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常"); if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`); if (exceptions.length) failures.push(`浏览器异常:${exceptions.join(" | ")}`);
+1 -1
View File
@@ -334,7 +334,7 @@ const benchmarkDevices = [
<p><span>TOP-16 OVERLAP</span><b>{probe.membership_overlap_mean.toFixed(2)} / 16</b></p> <p><span>TOP-16 OVERLAP</span><b>{probe.membership_overlap_mean.toFixed(2)} / 16</b></p>
</div> </div>
</div> </div>
<div class="boundary"><b>X/S/U boundary</b><p>本机 kernel 实测只验证公开 FlashKDA API 与合成合法 shape;没有加载 K3 checkpoint,也不解决 checkpoint `A_log [128]` 与 API `[96]` 的冲突。Router counterexample 仍只证明 hidden distribution 不可省略。</p></div> <div class="boundary"><b>X/S/U boundary</b><p>本机 kernel 实测只验证公开 FlashKDA API 与合成合法 shape;没有加载 K3 checkpoint。官方 main 仍存在 `A_log [128]` 与 API `[96]` 的冲突;社区 #144 与 #150 给出两种未合并候选,本站不替官方裁决。Router counterexample 仍只证明 hidden distribution 不可省略。</p></div>
</section> </section>
<footer class="evidence-strip"> <footer class="evidence-strip">
+703
View File
@@ -0,0 +1,703 @@
---
import rawLab from "@/data/k3-attnres-forward-compact.json";
const lab = rawLab as any;
const json = JSON.stringify(lab).replaceAll("<", "\\u003c");
const seeds = [...new Set(lab.final_spectra.map((item: any) => item.seed))] as number[];
const variants = [
"uniform_group_6_forward",
"uniform_group_7_forward",
"uniform_groups_6_7_forward",
"uniform_group_7_mlp_forward",
];
const variantLabels: Record<string, string> = {
learned_reference: "HISTORICAL LEARNED REFERENCE",
uniform_group_6_forward: "GROUP 6 · UNIFORM FORWARD",
uniform_group_7_forward: "GROUP 7 · UNIFORM FORWARD",
uniform_groups_6_7_forward: "GROUPS 6+7 · UNIFORM FORWARD",
uniform_group_7_mlp_forward: "GROUP 7 MLP · UNIFORM FORWARD",
};
const statusLabels: Record<string, string> = {
forward_training_attenuation_established_within_reduced_protocol:
"ATTENUATION ESTABLISHED · REDUCED PROTOCOL",
attenuation_not_established: "ATTENUATION NOT ESTABLISHED",
quality_guard_failed: "QUALITY GUARD FAILED",
attenuation_and_quality_failed: "ATTENUATION + QUALITY FAILED",
};
const statusLabel = statusLabels[lab.status] ?? lab.status;
const primary = lab.primary;
const quality = primary.quality;
const meanDrop = (effect: any, metric: string) => {
const cells = effect.cells.filter((cell: any) => cell.metric === metric);
return cells.reduce((sum: number, cell: any) => sum + cell.relative_drop, 0) / cells.length;
};
const percent = (value: number) => `${value >= 0 ? "+" : "−"}${Math.abs(value * 100).toFixed(1)}%`;
const shortHash = (value: string) => `${value.slice(0, 10)}…${value.slice(-8)}`;
---
<figure class="forward-lab" data-forward-lab>
<figcaption>
<span>ROUND 08 / TRAIN-TIME FORWARD</span>
<div>
<h3>不再只改 diagnostic backward:让局部 uniform routing 真正进入 8,000-step 训练</h3>
<p>4 variants × 3 seeds · 1 historical paired reference · 1 full replay · frozen analyzer</p>
</div>
<em>REDUCED-MODEL ARCHITECTURE ABLATION</em>
</figcaption>
<div class="forward-ledger">
<article><span>NEW BYTES</span><b>851.968M</b><p>12 formal + 1 replay</p></article>
<article><span>PRIMARY CELLS</span><b>{primary.passed_cells} / {primary.required_cells}</b><p>contrast + peak</p></article>
<article><span>QUALITY</span><b>{quality.passed_checks} / {quality.required_checks}</b><p>BPC degradation screen</p></article>
<article><span>REPLAY</span><b>{lab.replay.passed ? "EXACT" : "FAILED"}</b><p>scientific payload</p></article>
<article><span>SELECTED PARAMS</span><b>UNREACHABLE</b><p>0 hooks · 0 optimizer state</p></article>
<article class:list={{ pass: primary.material_response_passed, warn: !primary.material_response_passed }}>
<span>STATUS</span><b>{primary.material_response_passed ? "PASS" : "NOT ESTABLISHED"}</b><p>reduced protocol only</p>
</article>
</div>
<div class="forward-tabs" role="tablist" aria-label="Round 08 训练期前向干预实验视图">
<button type="button" role="tab" data-forward-tab="contract" aria-selected="true">01 / FORWARD CONTRACT</button>
<button type="button" role="tab" data-forward-tab="trajectory" aria-selected="false">02 / TRAINING TRAJECTORY</button>
<button type="button" role="tab" data-forward-tab="gate" aria-selected="false">03 / FINAL GATE</button>
<button type="button" role="tab" data-forward-tab="interaction" aria-selected="false">04 / NON-ADDITIVITY</button>
<button type="button" role="tab" data-forward-tab="spectrum" aria-selected="false">05 / SPECTRUM × AUDIT</button>
</div>
<section class="forward-panel" data-forward-panel="contract">
<div class="panel-lead">
<div><span>I / WHAT ACTUALLY CHANGED</span><h4>这是训练期架构消融,不是“只改变 forward”的纯因果实验</h4></div>
<p>选中 mixer 每一次 train / eval / diagnostic 都改成参数无关的均匀读取;新的表示自然改变 backward 与后续更新。</p>
</div>
<div class="equation-pair">
<article>
<span>LEARNED DEPTH MIXER</span>
<code>keys = RMSNorm(sources)</code>
<code>w = softmax(query · keys)</code>
<b>output = Σ wᵢ · sourceᵢ</b>
</article>
<i>→</i>
<article class="uniform">
<span>SELECTED UNIFORM MIXER</span>
<code>logits = zeros(N, B, T)</code>
<code>w = softmax(logits) = 1 / N</code>
<b>output = Σ sourceᵢ / N</b>
</article>
</div>
<div class="group-map">
{[1,2,3,4,5,6,7,8].map((group) => (
<article class:list={{ target: group === 6 || group === 7 }}>
<span>GROUP {group}</span>
<b>L{(group - 1) * 4 + 1}–{group * 4}</b>
<div>
{[0,1,2,3].map((offset) => {
const layer = (group - 1) * 4 + offset + 1;
return <i class:list={{ spike: layer >= 21 && layer <= 25 }}>{layer}</i>;
})}
</div>
<small>{group === 6 ? "#40–47" : group === 7 ? "#48–55" : "LEARNED"}</small>
</article>
))}
</div>
<div class="variant-grid">
{variants.map((variant) => {
const effect = lab.effects[variant];
return (
<article class:list={{ primary: variant === "uniform_groups_6_7_forward" }}>
<span>{variant === "uniform_groups_6_7_forward" ? "PRIMARY" : "SECONDARY"}</span>
<b>{variantLabels[variant]}</b>
<dl>
<div><dt>CONTRAST</dt><dd>{percent(meanDrop(effect, "spike_contrast"))}</dd></div>
<div><dt>PEAK</dt><dd>{percent(meanDrop(effect, "peak_normalized"))}</dd></div>
<div><dt>STATUS</dt><dd>{effect.material_response_passed ? "PASS" : "NOT EST."}</dd></div>
</dl>
</article>
);
})}
</div>
<div class="reachability-flow">
<article><span>SELECTED QUERY / KEY NORM</span><b>留在 AdamW param groups</b><p>保持 optimizer 结构与 reference 一致。</p></article>
<i>×</i>
<article><span>COMPUTATION GRAPH</span><b>没有被 forward 调用</b><p>不是 stop-gradient;参数结构性不可达。</p></article>
<i>→</i>
<article><span>FINAL AUDIT</span><b>0 hook · 0 state · byte-exact init</b><p>不能写成“训练了但没有移动”。</p></article>
</div>
</section>
<section class="forward-panel" data-forward-panel="trajectory" hidden>
<div class="panel-lead">
<div><span>II / SIX FROZEN CHECKPOINTS</span><h4>不是只看终点:局部前向干预从什么时候开始分化?</h4></div>
<p>纵轴是相对同 seed historical learned reference 的下降;正值表示 attenuation,负值表示 amplification。</p>
</div>
<div class="forward-controls">
<label>SEED
<select data-forward-trajectory-seed>
{seeds.map((seed) => <option value={String(seed)}>{seed}</option>)}
</select>
</label>
<div>
<button type="button" data-forward-trajectory-metric="spike_contrast" aria-pressed="true">SPIKE CONTRAST</button>
<button type="button" data-forward-trajectory-metric="peak_normalized" aria-pressed="false">PEAK / MEAN</button>
</div>
<span data-forward-trajectory-state></span>
</div>
<div class="trajectory-chart">
<header><b>RELATIVE DROP VS PAIRED REFERENCE</b><span>checkpoints are equally spaced; labels preserve actual steps</span></header>
<svg viewBox="0 0 960 390" role="img" aria-label="四个训练变体的尖峰指标轨迹" data-forward-trajectory-chart>
<g data-forward-trajectory-grid></g>
<g data-forward-trajectory-series></g>
</svg>
<div class="series-key">
<span><i class="g6"></i>GROUP 6</span>
<span><i class="g7"></i>GROUP 7</span>
<span><i class="joint"></i>GROUPS 6+7</span>
<span><i class="mlp"></i>GROUP 7 MLP</span>
</div>
</div>
<div class="trajectory-readout" data-forward-trajectory-readout></div>
<div class="boundary-note">
<b>轨迹不参与主闸门</b>
<p>主 status 只读取 step 8,000;中间 checkpoint 用来观察适应过程,不能挑一个最好看的时点替代终点。</p>
</div>
</section>
<section class="forward-panel" data-forward-panel="gate" hidden>
<div class="panel-lead">
<div><span>III / PREREGISTERED FINAL VERDICT</span><h4>20% attenuation 与 BPC degradation screen 必须同时通过</h4></div>
<p>任何结构、hash、selector、finite 或 replay 错误都会让 analyzer 直接失败,不进入下表的科学状态。</p>
</div>
<div class:list={{ "status-banner": true, pass: primary.material_response_passed, warn: !primary.material_response_passed }}>
<span>FROZEN ANALYZER STATUS</span>
<b>{statusLabel}</b>
<p>ATTENUATION {primary.passed_cells} / {primary.required_cells} · QUALITY {quality.passed_checks} / {quality.required_checks}</p>
</div>
<div class="gate-layout">
<div class="gate-table-wrap">
<table class="gate-table">
<thead><tr><th>SEED</th><th>METRIC</th><th>REFERENCE</th><th>VARIANT</th><th>DROP</th><th>≥20%</th></tr></thead>
<tbody>
{primary.cells.map((cell: any) => (
<tr>
<th>{cell.seed}</th>
<td>{cell.metric === "spike_contrast" ? "CONTRAST" : "PEAK"}</td>
<td>{cell.reference.toFixed(3)}×</td>
<td>{cell.variant.toFixed(3)}×</td>
<td class:list={{ good: cell.passed, bad: !cell.passed }}>{percent(cell.relative_drop)}</td>
<td class:list={{ good: cell.passed, bad: !cell.passed }}>{cell.passed ? "PASS" : "FAIL"}</td>
</tr>
))}
</tbody>
</table>
</div>
<aside>
<span>QUALITY SCREEN</span>
<b>final validation BPC</b>
{Object.entries(quality.per_seed).map(([seed, item]: [string, any]) => (
<div><em>{seed}</em><strong class:list={{ good: item.passed, bad: !item.passed }}>{item.delta_bpc >= 0 ? "+" : ""}{item.delta_bpc.toFixed(4)}</strong><small>≤ +.050</small></div>
))}
<div class="mean"><em>3-SEED MEAN</em><strong class:list={{ good: quality.mean_passed, bad: !quality.mean_passed }}>{quality.mean_delta_bpc >= 0 ? "+" : ""}{quality.mean_delta_bpc.toFixed(4)}</strong><small>≤ +.030</small></div>
</aside>
</div>
<div class="numerator-denominator">
<article><span>SPIKE WINDOW</span><b>S = layers 21–25</b><p>报告 mean(g[S]),但不单独作为 status。</p></article>
<i>÷</i>
<article><span>REFERENCE WINDOW</span><b>R = other 27 layers</b><p>layers 26–28 也被 joint intervention 改写。</p></article>
<i>=</i>
<article class="warning"><span>CONTRAST</span><b>不是“尖峰层关闭”</b><p>下降可能来自 S 降、R 升,或两者同时发生。</p></article>
</div>
<div class="claim-pair">
<article class="yes"><span>可以说</span><b>{statusLabel}</b><p>只限这个固定缩小模型、数据、预算、seed 与 operational metric。</p></article>
<article class="no"><span>不能说</span><b>K3 的训练尖峰已被定位</b><p>没有真实 2.8T checkpoint forward,也没有复现未公开 Figure 5(c) telemetry。</p></article>
</div>
</section>
<section class="forward-panel" data-forward-panel="interaction" hidden>
<div class="panel-lead">
<div><span>IV / DESCRIPTIVE LOG RESIDUAL</span><h4>joint effect 等不等于两个 single-run effects 相加?</h4></div>
<p>三条 effect 来自三套独立训练;这里画的是跨 run 的 bookkeeping residual,不是因果 interaction 或 Shapley contribution。</p>
</div>
<div class="forward-controls">
<label>STEP
<select data-forward-interaction-step>
{[0,100,500,2000,4000,8000].map((step) => <option value={String(step)}>{step.toLocaleString()}</option>)}
</select>
</label>
<div>
<button type="button" data-forward-interaction-metric="spike_contrast" aria-pressed="true">SPIKE CONTRAST</button>
<button type="button" data-forward-interaction-metric="peak_normalized" aria-pressed="false">PEAK / MEAN</button>
</div>
<span data-forward-interaction-state></span>
</div>
<div class="interaction-equation">
<span>E₆₇</span><i>−</i><span>E₆</span><i>−</i><span>E₇</span><b>= I₆₇</b>
<p>E = ln(Xref / Xvariant);I &gt; 0 表示 joint log attenuation 大于两个 single-run effects 的和。</p>
</div>
<div class="interaction-cells" data-forward-interaction-cells></div>
<div class="interaction-summary" data-forward-interaction-summary></div>
<div class="boundary-note">
<b>没有预注册通过线</b>
<p>I₆₇ 只报告原值、三 seed mean 与 range;正负都不能翻译成 group 6 / 7 的真实贡献。</p>
</div>
</section>
<section class="forward-panel" data-forward-panel="spectrum" hidden>
<div class="panel-lead">
<div><span>V / 32-LAYER SHAPE × REPRODUCTION</span><h4>终点平均数之外:峰值移动、S/R 分拆与完整 replay</h4></div>
<p>每条谱按自身 32-layer mean 归一化;选择 reference 或任一训练变体,不改变 analyzer 的正式 status。</p>
</div>
<div class="forward-controls spectrum-controls">
<label>SEED
<select data-forward-spectrum-seed>
{seeds.map((seed) => <option value={String(seed)}>{seed}</option>)}
</select>
</label>
<label>VARIANT
<select data-forward-spectrum-variant>
{["learned_reference", ...variants].map((variant) => <option value={variant}>{variantLabels[variant]}</option>)}
</select>
</label>
<span data-forward-spectrum-state></span>
</div>
<div class="spectrum-layout">
<div class="spectrum-chart">
<header><b>FINAL NORMALIZED ACTIVATION-GRADIENT RMS</b><span>layer mean = 1</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="训练期前向变体的 32 层梯度谱" data-forward-spectrum-chart>
<rect class="spike-zone" x="0" y="26" width="0" height="280" data-forward-spectrum-zone></rect>
<g data-forward-spectrum-grid></g>
<polyline points="" data-forward-spectrum-line></polyline>
<g data-forward-spectrum-points></g>
</svg>
</div>
<aside>
<span>SELECTED READOUT</span>
<b data-forward-spectrum-label></b>
<dl>
<div><dt>SPIKE MEAN</dt><dd data-forward-spectrum-smean></dd></div>
<div><dt>R MEAN</dt><dd data-forward-spectrum-rmean></dd></div>
<div><dt>CONTRAST</dt><dd data-forward-spectrum-contrast></dd></div>
<div><dt>PEAK / MEAN</dt><dd data-forward-spectrum-peak></dd></div>
<div><dt>PEAK LAYER</dt><dd data-forward-spectrum-layer></dd></div>
</dl>
</aside>
</div>
<div class="audit-grid">
<article><span>STEP-0 IDENTITY</span><b>5 / 5 exact</b><p>logits、CE、validation、32-layer diagnostic 与 capture summary。</p></article>
<article><span>EMPTY SELECTOR</span><b>parent exact</b><p>20-step model、optimizer、history、evaluations 与 hashes。</p></article>
<article><span>PRIMARY REPLAY</span><b>{lab.replay.passed ? "scientific exact" : "FAILED"}</b><p><code>{shortHash(lab.replay.scientific_payload_sha256)}</code></p></article>
<article><span>PROCESSED</span><b>851,968,000 bytes</b><p>历史 references 的 196,608,000 bytes 单列、不重复计入。</p></article>
<article class="boundary"><span>REAL K3</span><b>not executed</b><p><code>A_log [128]↔[96]</code> 尚无官方裁决。</p></article>
</div>
<div class="hash-strip">
<span>PROTOCOL <code>{lab.protocol_id}</code></span>
<span>AGGREGATE <code>{shortHash(lab.aggregate_sha256)}</code></span>
<span>COMPACT <code>{shortHash(lab.canonical_sha256_without_self)}</code></span>
</div>
</section>
<script is:inline type="application/json" data-forward-payload set:html={json}></script>
</figure>
<script>
const initializeForwardLab = (root: HTMLElement) => {
if (root.dataset.ready === "true") return;
root.dataset.ready = "true";
const payload = root.querySelector<HTMLScriptElement>("[data-forward-payload]");
if (!payload) return;
const data = JSON.parse(payload.textContent || "{}");
const variants = [
"uniform_group_6_forward",
"uniform_group_7_forward",
"uniform_groups_6_7_forward",
"uniform_group_7_mlp_forward",
];
const labels: Record<string, string> = {
learned_reference: "HISTORICAL LEARNED REFERENCE",
uniform_group_6_forward: "GROUP 6",
uniform_group_7_forward: "GROUP 7",
uniform_groups_6_7_forward: "GROUPS 6+7",
uniform_group_7_mlp_forward: "GROUP 7 MLP",
};
const colors: Record<string, string> = {
uniform_group_6_forward: "#c98a58",
uniform_group_7_forward: "#8794aa",
uniform_groups_6_7_forward: "#8aae8d",
uniform_group_7_mlp_forward: "#b487a7",
};
const tabs = [...root.querySelectorAll<HTMLButtonElement>("[data-forward-tab]")];
const panels = [...root.querySelectorAll<HTMLElement>("[data-forward-panel]")];
const activate = (name: string, focus = false) => {
tabs.forEach((tab) => {
const selected = tab.dataset.forwardTab === name;
tab.setAttribute("aria-selected", String(selected));
if (selected && focus) tab.focus();
});
panels.forEach((panel) => panel.hidden = panel.dataset.forwardPanel !== name);
};
tabs.forEach((tab, index) => {
tab.addEventListener("click", () => activate(tab.dataset.forwardTab || "contract"));
tab.addEventListener("keydown", (event) => {
if (!["ArrowLeft", "ArrowRight", "Home", "End"].includes(event.key)) return;
event.preventDefault();
let next = index;
if (event.key === "ArrowRight") next = (index + 1) % tabs.length;
if (event.key === "ArrowLeft") next = (index - 1 + tabs.length) % tabs.length;
if (event.key === "Home") next = 0;
if (event.key === "End") next = tabs.length - 1;
activate(tabs[next].dataset.forwardTab || "contract", true);
});
});
const ns = "http://www.w3.org/2000/svg";
const make = (name: string, attributes: Record<string, string>) => {
const node = document.createElementNS(ns, name);
Object.entries(attributes).forEach(([key, value]) => node.setAttribute(key, value));
return node;
};
const setText = (selector: string, value: string) => {
const node = root.querySelector<HTMLElement>(selector);
if (node) node.textContent = value;
};
let trajectoryMetric = "spike_contrast";
const trajectorySeed = root.querySelector<HTMLSelectElement>("[data-forward-trajectory-seed]");
const renderTrajectory = () => {
if (!trajectorySeed) return;
const seed = Number(trajectorySeed.value);
const rows = data.trajectories.filter((row: any) => row.seed === seed && variants.includes(row.variant));
const steps = [...new Set(rows.map((row: any) => row.step))] as number[];
const values = rows.map((row: any) => row.relative_drop[trajectoryMetric]);
const minimum = Math.min(-.05, ...values);
const maximum = Math.max(.25, ...values);
const left = 75, right = 925, top = 35, bottom = 330;
const x = (index: number) => left + index * (right - left) / (steps.length - 1);
const y = (value: number) => bottom - (value - minimum) / (maximum - minimum) * (bottom - top);
const grid = root.querySelector<SVGGElement>("[data-forward-trajectory-grid]");
const series = root.querySelector<SVGGElement>("[data-forward-trajectory-series]");
if (!grid || !series) return;
grid.innerHTML = ""; series.innerHTML = "";
[minimum, 0, .2, maximum].filter((value, index, all) => all.indexOf(value) === index).forEach((value) => {
grid.append(make("line", { x1: String(left), x2: String(right), y1: String(y(value)), y2: String(y(value)), class: value === 0 ? "zero" : "" }));
const label = make("text", { x: "12", y: String(y(value) + 4) });
label.textContent = `${value >= 0 ? "+" : ""}${(value * 100).toFixed(0)}%`;
grid.append(label);
});
steps.forEach((step, index) => {
const label = make("text", { x: String(x(index)), y: "360", "text-anchor": "middle" });
label.textContent = step.toLocaleString();
grid.append(label);
});
variants.forEach((variant) => {
const selected = rows.filter((row: any) => row.variant === variant).sort((a: any, b: any) => a.step - b.step);
const points = selected.map((row: any, index: number) => `${x(index)},${y(row.relative_drop[trajectoryMetric])}`).join(" ");
series.append(make("polyline", { points, fill: "none", stroke: colors[variant], "stroke-width": variant.includes("groups_6_7") ? "3" : "2" }));
selected.forEach((row: any, index: number) => {
const circle = make("circle", { cx: String(x(index)), cy: String(y(row.relative_drop[trajectoryMetric])), r: "4", fill: colors[variant], "data-variant": variant, "data-step": String(row.step) });
series.append(circle);
});
});
const metricLabel = trajectoryMetric === "spike_contrast" ? "SPIKE CONTRAST" : "PEAK / MEAN";
setText("[data-forward-trajectory-state]", `${seed} · ${metricLabel}`);
const readout = root.querySelector<HTMLElement>("[data-forward-trajectory-readout]");
if (readout) {
readout.innerHTML = variants.map((variant) => {
const final = rows.find((row: any) => row.variant === variant && row.step === 8000);
const value = final.relative_drop[trajectoryMetric];
return `<article><span>${labels[variant]}</span><b>${value >= 0 ? "+" : "−"}${Math.abs(value * 100).toFixed(1)}%</b><p>step 8,000 vs same-seed reference</p></article>`;
}).join("");
}
};
trajectorySeed?.addEventListener("change", renderTrajectory);
root.querySelectorAll<HTMLButtonElement>("[data-forward-trajectory-metric]").forEach((button) => {
button.addEventListener("click", () => {
trajectoryMetric = button.dataset.forwardTrajectoryMetric || "spike_contrast";
root.querySelectorAll<HTMLButtonElement>("[data-forward-trajectory-metric]").forEach((candidate) =>
candidate.setAttribute("aria-pressed", String(candidate === button)));
renderTrajectory();
});
});
renderTrajectory();
let interactionMetric = "spike_contrast";
const interactionStep = root.querySelector<HTMLSelectElement>("[data-forward-interaction-step]");
if (interactionStep) interactionStep.value = "8000";
const renderInteraction = () => {
if (!interactionStep) return;
const step = Number(interactionStep.value);
const cells = data.interaction.cells.filter((cell: any) => cell.step === step && cell.metric === interactionMetric);
const container = root.querySelector<HTMLElement>("[data-forward-interaction-cells]");
if (container) {
container.innerHTML = cells.map((cell: any) => {
const residual = cell.interaction_residual;
return `<article data-sign="${residual >= 0 ? "positive" : "negative"}"><span>SEED ${cell.seed}</span><div><em>E6</em><b>${cell.log_effects.group6.toFixed(3)}</b></div><div><em>E7</em><b>${cell.log_effects.group7.toFixed(3)}</b></div><div><em>E67</em><b>${cell.log_effects.groups6_7.toFixed(3)}</b></div><strong>I67 ${residual >= 0 ? "+" : ""}${residual.toFixed(3)}</strong></article>`;
}).join("");
}
const summary = data.interaction.summaries.find((item: any) => item.step === step && item.metric === interactionMetric);
const summaryNode = root.querySelector<HTMLElement>("[data-forward-interaction-summary]");
if (summaryNode && summary) {
summaryNode.innerHTML = `<span>3-SEED DESCRIPTIVE SUMMARY</span><b>mean I67 ${summary.mean_interaction_residual >= 0 ? "+" : ""}${summary.mean_interaction_residual.toFixed(3)}</b><p>range ${summary.minimum.toFixed(3)} → ${summary.maximum.toFixed(3)}</p>`;
}
setText("[data-forward-interaction-state]", `${step.toLocaleString()} · ${interactionMetric === "spike_contrast" ? "SPIKE CONTRAST" : "PEAK / MEAN"}`);
};
interactionStep?.addEventListener("change", renderInteraction);
root.querySelectorAll<HTMLButtonElement>("[data-forward-interaction-metric]").forEach((button) => {
button.addEventListener("click", () => {
interactionMetric = button.dataset.forwardInteractionMetric || "spike_contrast";
root.querySelectorAll<HTMLButtonElement>("[data-forward-interaction-metric]").forEach((candidate) =>
candidate.setAttribute("aria-pressed", String(candidate === button)));
renderInteraction();
});
});
renderInteraction();
const spectrumSeed = root.querySelector<HTMLSelectElement>("[data-forward-spectrum-seed]");
const spectrumVariant = root.querySelector<HTMLSelectElement>("[data-forward-spectrum-variant]");
const renderSpectrum = () => {
if (!spectrumSeed || !spectrumVariant) return;
const seed = Number(spectrumSeed.value);
const variant = spectrumVariant.value;
const record = data.final_spectra.find((item: any) => item.seed === seed && item.variant === variant);
if (!record) return;
const values = record.normalized;
const left = 48, right = 915, top = 28, bottom = 305;
const maximum = Math.max(2, ...values) * 1.08;
const x = (index: number) => left + index * (right - left) / 31;
const y = (value: number) => bottom - value / maximum * (bottom - top);
const grid = root.querySelector<SVGGElement>("[data-forward-spectrum-grid]");
const points = root.querySelector<SVGGElement>("[data-forward-spectrum-points]");
const line = root.querySelector<SVGPolylineElement>("[data-forward-spectrum-line]");
const zone = root.querySelector<SVGRectElement>("[data-forward-spectrum-zone]");
if (!grid || !points || !line || !zone) return;
grid.innerHTML = ""; points.innerHTML = "";
[0, 1, Math.ceil(maximum)].forEach((value) => {
grid.append(make("line", { x1: String(left), x2: String(right), y1: String(y(value)), y2: String(y(value)) }));
const label = make("text", { x: "8", y: String(y(value) + 4) });
label.textContent = `${value}×`; grid.append(label);
});
[1,5,9,13,17,21,25,29,32].forEach((layer) => {
const label = make("text", { x: String(x(layer - 1)), y: "330", "text-anchor": "middle" });
label.textContent = String(layer); grid.append(label);
});
zone.setAttribute("x", String(x(20) - 8));
zone.setAttribute("width", String(x(24) - x(20) + 16));
line.setAttribute("points", values.map((value: number, index: number) => `${x(index)},${y(value)}`).join(" "));
values.forEach((value: number, index: number) => {
points.append(make("circle", { cx: String(x(index)), cy: String(y(value)), r: index + 1 === record.peak_layer_1based ? "4.5" : "2.6", "data-layer": String(index + 1) }));
});
setText("[data-forward-spectrum-state]", `${seed} · ${labels[variant]}`);
setText("[data-forward-spectrum-label]", labels[variant]);
setText("[data-forward-spectrum-smean]", record.spike_mean.toExponential(3));
setText("[data-forward-spectrum-rmean]", record.reference_mean.toExponential(3));
setText("[data-forward-spectrum-contrast]", `${record.spike_contrast.toFixed(3)}×`);
setText("[data-forward-spectrum-peak]", `${record.peak_normalized.toFixed(3)}×`);
setText("[data-forward-spectrum-layer]", `L${record.peak_layer_1based}`);
};
spectrumSeed?.addEventListener("change", renderSpectrum);
spectrumVariant?.addEventListener("change", renderSpectrum);
renderSpectrum();
};
document.querySelectorAll<HTMLElement>("[data-forward-lab]").forEach(initializeForwardLab);
document.addEventListener("astro:page-load", () => {
document.querySelectorAll<HTMLElement>("[data-forward-lab]").forEach(initializeForwardLab);
});
</script>
<style>
.forward-lab {
--ink: #d9d7ce;
--muted: #8e918d;
--line: rgba(217,215,206,.14);
--panel: rgba(14,17,16,.78);
--green: #8aae8d;
--copper: #c98a58;
--blue: #8794aa;
--pink: #b487a7;
--red: #c77768;
margin: 32px 0;
border: 1px solid var(--line);
background: #0d0f0e;
color: var(--ink);
}
.forward-lab h3, .forward-lab h4 { color: var(--ink); }
.forward-lab figcaption { display: grid; grid-template-columns: 150px 1fr auto; gap: 24px; align-items: start; padding: 25px; border-bottom: 1px solid var(--line); }
.forward-lab figcaption > span, .forward-lab figcaption > em { color: var(--copper); font: normal .52rem var(--mono); letter-spacing: .12em; }
.forward-lab figcaption > em { color: var(--muted); text-align: right; }
.forward-lab figcaption h3 { margin: 0 0 8px; font-size: 1.02rem; line-height: 1.35; }
.forward-lab figcaption p { margin: 0; color: var(--muted); font: .56rem/1.5 var(--mono); }
.forward-ledger { display: grid; grid-template-columns: repeat(6, 1fr); border-bottom: 1px solid var(--line); }
.forward-ledger article { min-height: 96px; padding: 15px; border-right: 1px solid var(--line); }
.forward-ledger article:last-child { border-right: 0; }
.forward-ledger span, .variant-grid span, .audit-grid span { color: var(--muted); font: .48rem var(--mono); }
.forward-ledger b { display: block; margin: 11px 0 6px; font: 800 .66rem var(--mono); }
.forward-ledger p { margin: 0; color: var(--muted); font-size: .51rem; }
.forward-ledger .pass b { color: var(--green); }
.forward-ledger .warn b { color: var(--red); }
.forward-tabs { display: grid; grid-template-columns: repeat(5, 1fr); border-bottom: 1px solid var(--line); }
.forward-tabs button { min-height: 48px; border: 0; border-right: 1px solid var(--line); background: transparent; color: var(--muted); font: .51rem var(--mono); cursor: pointer; }
.forward-tabs button:last-child { border-right: 0; }
.forward-tabs button[aria-selected="true"] { background: rgba(201,138,88,.09); color: var(--copper); box-shadow: inset 0 -2px var(--copper); }
.forward-tabs button:focus-visible { outline: 2px solid var(--green); outline-offset: -3px; }
.forward-panel { padding: 26px; }
.panel-lead { display: grid; grid-template-columns: 1fr minmax(260px, 42%); gap: 32px; margin-bottom: 22px; }
.panel-lead span { color: var(--copper); font: .5rem var(--mono); }
.panel-lead h4 { margin: 8px 0 0; font-size: .9rem; }
.panel-lead p { margin: 0; color: var(--muted); font-size: .64rem; line-height: 1.65; }
.equation-pair { display: grid; grid-template-columns: 1fr auto 1fr; gap: 14px; align-items: center; }
.equation-pair article { padding: 18px; border: 1px solid var(--line); background: var(--panel); }
.equation-pair article.uniform { border-color: rgba(138,174,141,.45); }
.equation-pair span { color: var(--muted); font: .5rem var(--mono); }
.equation-pair code, .equation-pair b { display: block; margin-top: 10px; font: .61rem var(--mono); }
.equation-pair code { color: var(--muted); }
.equation-pair b { color: var(--green); }
.equation-pair > i, .reachability-flow > i, .numerator-denominator > i { color: var(--copper); font-style: normal; }
.group-map { display: grid; grid-template-columns: repeat(8, 1fr); margin-top: 18px; border: 1px solid var(--line); }
.group-map article { min-height: 118px; padding: 12px; border-right: 1px solid var(--line); }
.group-map article:last-child { border-right: 0; }
.group-map article.target { background: rgba(201,138,88,.08); box-shadow: inset 0 3px var(--copper); }
.group-map span, .group-map small { color: var(--muted); font: .45rem var(--mono); }
.group-map b { display: block; margin: 9px 0; font: .64rem var(--mono); }
.group-map div { display: grid; grid-template-columns: repeat(4, 1fr); gap: 3px; }
.group-map i { display: grid; place-items: center; aspect-ratio: 1; border: 1px solid var(--line); color: var(--muted); font: normal .44rem var(--mono); }
.group-map i.spike { border-color: rgba(138,174,141,.6); color: var(--green); }
.group-map small { display: block; margin-top: 8px; }
.variant-grid { display: grid; grid-template-columns: repeat(4, 1fr); gap: 12px; margin-top: 18px; }
.variant-grid article { padding: 15px; border: 1px solid var(--line); }
.variant-grid article.primary { border-color: rgba(138,174,141,.5); }
.variant-grid b { display: block; min-height: 34px; margin: 8px 0 12px; font: .6rem/1.4 var(--mono); }
.variant-grid dl { margin: 0; }
.variant-grid dl div { display: flex; justify-content: space-between; padding: 7px 0; border-top: 1px solid var(--line); }
.variant-grid dt, .variant-grid dd { margin: 0; font: .48rem var(--mono); }
.variant-grid dt { color: var(--muted); }
.variant-grid dd { color: var(--green); }
.reachability-flow, .numerator-denominator { display: grid; grid-template-columns: 1fr auto 1fr auto 1fr; gap: 12px; align-items: center; margin-top: 18px; }
.reachability-flow article, .numerator-denominator article { min-height: 108px; padding: 15px; border: 1px solid var(--line); }
.reachability-flow span, .numerator-denominator span { color: var(--muted); font: .47rem var(--mono); }
.reachability-flow b, .numerator-denominator b { display: block; margin: 9px 0; font: .64rem var(--mono); }
.reachability-flow p, .numerator-denominator p { margin: 0; color: var(--muted); font-size: .56rem; line-height: 1.5; }
.numerator-denominator .warning { border-color: rgba(199,119,104,.45); }
.forward-controls { display: flex; align-items: stretch; margin-bottom: 16px; border: 1px solid var(--line); }
.forward-controls label { display: flex; align-items: center; gap: 9px; padding: 10px 13px; border-right: 1px solid var(--line); color: var(--muted); font: .48rem var(--mono); }
.forward-controls select { max-width: 260px; border: 1px solid var(--line); background: #0d0f0e; color: var(--ink); font: .53rem var(--mono); }
.forward-controls > div { display: flex; }
.forward-controls button { border: 0; border-right: 1px solid var(--line); background: transparent; color: var(--muted); font: .48rem var(--mono); }
.forward-controls button[aria-pressed="true"] { color: var(--green); background: rgba(138,174,141,.08); }
.forward-controls > span { display: grid; place-items: center; margin-left: auto; padding: 0 14px; color: var(--green); font: .49rem var(--mono); }
.trajectory-chart, .spectrum-chart { border: 1px solid var(--line); background: var(--panel); }
.trajectory-chart header, .spectrum-chart header { display: flex; justify-content: space-between; padding: 11px 14px; border-bottom: 1px solid var(--line); }
.trajectory-chart header b, .trajectory-chart header span, .spectrum-chart header b, .spectrum-chart header span { font: .49rem var(--mono); }
.trajectory-chart header span, .spectrum-chart header span { color: var(--muted); }
.trajectory-chart svg, .spectrum-chart svg { display: block; width: 100%; height: auto; }
[data-forward-trajectory-grid] :global(line), [data-forward-spectrum-grid] :global(line) { stroke: var(--line); stroke-width: 1; }
[data-forward-trajectory-grid] :global(line.zero) { stroke: rgba(217,215,206,.5); stroke-dasharray: 4 4; }
[data-forward-trajectory-grid] :global(text), [data-forward-spectrum-grid] :global(text) { fill: var(--muted); font: 10px var(--mono); }
.series-key { display: flex; flex-wrap: wrap; gap: 18px; padding: 10px 14px; border-top: 1px solid var(--line); color: var(--muted); font: .48rem var(--mono); }
.series-key span { display: flex; align-items: center; gap: 7px; }
.series-key i { width: 16px; height: 2px; }
.series-key .g6 { background: var(--copper); }
.series-key .g7 { background: var(--blue); }
.series-key .joint { background: var(--green); }
.series-key .mlp { background: var(--pink); }
.trajectory-readout { display: grid; grid-template-columns: repeat(4, 1fr); gap: 12px; margin-top: 14px; }
.trajectory-readout :global(article) { padding: 14px; border: 1px solid var(--line); }
.trajectory-readout :global(span) { color: var(--muted); font: .47rem var(--mono); }
.trajectory-readout :global(b) { display: block; margin: 8px 0; color: var(--green); font: .7rem var(--mono); }
.trajectory-readout :global(p) { margin: 0; color: var(--muted); font-size: .53rem; }
.boundary-note { margin-top: 16px; padding: 15px; border-left: 3px solid var(--copper); background: rgba(201,138,88,.07); }
.boundary-note b { font-size: .64rem; }
.boundary-note p { margin: 6px 0 0; color: var(--muted); font-size: .58rem; line-height: 1.55; }
.status-banner { display: grid; grid-template-columns: 180px 1fr auto; gap: 18px; align-items: center; padding: 16px; border: 1px solid var(--line); }
.status-banner.pass { border-color: rgba(138,174,141,.5); }
.status-banner.warn { border-color: rgba(199,119,104,.5); }
.status-banner span, .status-banner p { color: var(--muted); font: .49rem var(--mono); }
.status-banner b { color: var(--green); font: .72rem var(--mono); }
.status-banner.warn b { color: var(--red); }
.gate-layout { display: grid; grid-template-columns: 1fr 230px; gap: 15px; margin-top: 16px; }
.gate-table-wrap { overflow-x: auto; border: 1px solid var(--line); }
.gate-table { width: 100%; border-collapse: collapse; min-width: 680px; font: .53rem var(--mono); }
.gate-table th, .gate-table td { padding: 12px 10px; border-right: 1px solid var(--line); border-bottom: 1px solid var(--line); text-align: right; }
.gate-table thead { color: var(--muted); }
.good { color: var(--green) !important; }
.bad { color: var(--red) !important; }
.gate-layout aside { padding: 15px; border: 1px solid var(--line); }
.gate-layout aside > span { color: var(--muted); font: .48rem var(--mono); }
.gate-layout aside > b { display: block; margin: 8px 0 14px; font: .63rem var(--mono); }
.gate-layout aside > div { display: grid; grid-template-columns: 1fr auto; gap: 4px 8px; padding: 9px 0; border-top: 1px solid var(--line); }
.gate-layout aside em, .gate-layout aside strong, .gate-layout aside small { font: normal .49rem var(--mono); }
.gate-layout aside small { grid-column: 1 / -1; color: var(--muted); }
.gate-layout aside .mean { margin-top: 5px; }
.claim-pair { display: grid; grid-template-columns: 1fr 1fr; gap: 14px; margin-top: 16px; }
.claim-pair article { padding: 16px; border: 1px solid var(--line); }
.claim-pair span { color: var(--muted); font: .49rem var(--mono); }
.claim-pair b { display: block; margin: 9px 0; font: .65rem var(--mono); }
.claim-pair p { margin: 0; color: var(--muted); font-size: .57rem; line-height: 1.5; }
.claim-pair .yes { border-color: rgba(138,174,141,.45); }
.claim-pair .yes b { color: var(--green); }
.claim-pair .no { border-color: rgba(199,119,104,.45); }
.claim-pair .no b { color: var(--red); }
.interaction-equation { display: flex; flex-wrap: wrap; align-items: center; gap: 12px; padding: 17px; border: 1px solid var(--line); }
.interaction-equation span, .interaction-equation b { padding: 8px 12px; background: var(--panel); font: .68rem var(--mono); }
.interaction-equation i { color: var(--copper); font-style: normal; }
.interaction-equation b { color: var(--green); }
.interaction-equation p { flex-basis: 100%; margin: 0; color: var(--muted); font-size: .56rem; }
.interaction-cells { display: grid; grid-template-columns: repeat(3, 1fr); gap: 14px; margin-top: 16px; }
.interaction-cells :global(article) { padding: 16px; border: 1px solid var(--line); }
.interaction-cells :global(article[data-sign="positive"]) { border-color: rgba(138,174,141,.4); }
.interaction-cells :global(article[data-sign="negative"]) { border-color: rgba(199,119,104,.4); }
.interaction-cells :global(span) { color: var(--muted); font: .48rem var(--mono); }
.interaction-cells :global(div) { display: flex; justify-content: space-between; margin-top: 9px; padding-top: 8px; border-top: 1px solid var(--line); }
.interaction-cells :global(em), .interaction-cells :global(b) { font: normal .52rem var(--mono); }
.interaction-cells :global(em) { color: var(--muted); }
.interaction-cells :global(strong) { display: block; margin-top: 12px; color: var(--green); font: .65rem var(--mono); }
.interaction-summary { margin-top: 14px; padding: 16px; border: 1px solid var(--line); }
.interaction-summary :global(span) { color: var(--muted); font: .48rem var(--mono); }
.interaction-summary :global(b) { display: block; margin: 8px 0; font: .7rem var(--mono); }
.interaction-summary :global(p) { margin: 0; color: var(--muted); font: .52rem var(--mono); }
.spectrum-controls label:nth-child(2) { flex: 1; }
.spectrum-controls label:nth-child(2) select { width: 100%; max-width: none; }
.spectrum-layout { display: grid; grid-template-columns: 1fr 225px; gap: 15px; }
.spike-zone { fill: rgba(201,138,88,.1); }
[data-forward-spectrum-line] { fill: none; stroke: var(--green); stroke-width: 2; }
[data-forward-spectrum-points] :global(circle) { fill: var(--green); stroke: #0d0f0e; stroke-width: 1; }
.spectrum-layout aside { padding: 16px; border: 1px solid var(--line); background: var(--panel); }
.spectrum-layout aside > span { color: var(--muted); font: .48rem var(--mono); }
.spectrum-layout aside > b { display: block; margin: 9px 0 16px; font: .61rem/1.4 var(--mono); }
.spectrum-layout dl { margin: 0; }
.spectrum-layout dl div { display: flex; justify-content: space-between; gap: 8px; padding: 10px 0; border-top: 1px solid var(--line); }
.spectrum-layout dt, .spectrum-layout dd { margin: 0; font: .49rem var(--mono); }
.spectrum-layout dt { color: var(--muted); }
.spectrum-layout dd { color: var(--green); }
.audit-grid { display: grid; grid-template-columns: repeat(5, 1fr); gap: 12px; margin-top: 16px; }
.audit-grid article { padding: 14px; border: 1px solid var(--line); }
.audit-grid b { display: block; margin: 9px 0; font: .63rem var(--mono); }
.audit-grid p { margin: 0; color: var(--muted); font-size: .53rem; line-height: 1.5; }
.audit-grid .boundary { border-color: rgba(199,119,104,.45); }
.hash-strip { display: flex; flex-wrap: wrap; gap: 18px; margin-top: 14px; padding: 12px 14px; border: 1px solid var(--line); color: var(--muted); font: .48rem var(--mono); }
.hash-strip code { color: var(--green); }
@media (max-width: 980px) {
.forward-ledger { grid-template-columns: repeat(3, 1fr); }
.forward-tabs { grid-template-columns: repeat(3, 1fr); }
.group-map { grid-template-columns: repeat(4, 1fr); }
.variant-grid, .trajectory-readout { grid-template-columns: repeat(2, 1fr); }
.gate-layout, .spectrum-layout { grid-template-columns: 1fr; }
.audit-grid { grid-template-columns: repeat(3, 1fr); }
}
@media (max-width: 680px) {
.forward-lab figcaption { grid-template-columns: 1fr; padding: 20px; }
.forward-lab figcaption > span { order: -1; }
.forward-lab figcaption > em { text-align: left; }
.forward-ledger { grid-template-columns: repeat(2, 1fr); }
.forward-tabs { display: flex; overflow-x: auto; }
.forward-tabs button { min-width: 165px; }
.forward-panel { padding: 20px 14px; }
.panel-lead { grid-template-columns: 1fr; gap: 12px; }
.equation-pair, .reachability-flow, .numerator-denominator { grid-template-columns: 1fr; }
.equation-pair > i, .reachability-flow > i, .numerator-denominator > i { text-align: center; transform: rotate(90deg); }
.group-map { grid-template-columns: repeat(2, 1fr); }
.variant-grid, .trajectory-readout, .claim-pair, .interaction-cells, .audit-grid { grid-template-columns: 1fr; }
.forward-controls { flex-wrap: wrap; }
.forward-controls > span { width: 100%; min-height: 34px; border-top: 1px solid var(--line); }
.status-banner { grid-template-columns: 1fr; }
.gate-table { min-width: 680px; }
}
</style>
+707
View File
@@ -0,0 +1,707 @@
---
import rawLab from "@/data/k3-attnres-local-path-compact.json";
const lab = rawLab as any;
const json = JSON.stringify(lab).replaceAll("<", "\\u003c");
const seeds = lab.study.seeds as number[];
const shortHash = (value: string) => `${value.slice(0, 10)}…${value.slice(-8)}`;
const modeLabels: Record<string, string> = {
detached_learned: "REFERENCE · 全部 detached learned",
uniform_group_6_only: "仅 Group 6 uniform",
uniform_group_7_only: "仅 Group 7 uniform",
uniform_groups_6_7_only: "仅 Groups 6+7 uniform",
uniform_group_6_attention_only: "Group 6 · Attention only",
uniform_group_6_mlp_only: "Group 6 · MLP only",
uniform_group_7_attention_only: "Group 7 · Attention only",
uniform_group_7_mlp_only: "Group 7 · MLP only",
uniform_output_only: "仅 Output mixer uniform",
uniform_depth_all: "64 个 depth mixers uniform",
uniform_all: "全部 65 个 mixers uniform",
uniform_except_group_6: "恢复 Group 6 → detached learned",
uniform_except_group_7: "恢复 Group 7 → detached learned",
uniform_except_groups_6_7: "恢复 Groups 6+7 → detached learned",
};
const score = (family: "sufficiency" | "restoration", mode: string, seed: number, metric: string) =>
lab.scores[family][mode][String(seed)][metric].score;
---
<figure class="local-lab" data-local-path-lab>
<figcaption>
<span>ROUND 07 / LOCAL MIXER PATHS</span>
<div>
<h3>把“全局敏感”缩到 16 个 mixer:为什么单侧很强,双向门仍然不让过?</h3>
<p>同一 forward · 14 个冻结 mask · 3 seeds · sufficiency × restoration · 完整 replay</p>
</div>
<em>REDUCED-MODEL DIAGNOSTIC</em>
</figcaption>
<div class="local-ledger">
<article><span>TOPOLOGY</span><b>64 + 1</b><p>depth mixers + output</p></article>
<article><span>MATRIX</span><b>14 modes</b><p>exact selector sets</p></article>
<article><span>GLOBAL GAP</span><b>6 / 6</b><p>two metrics × three seeds</p></article>
<article class="pass"><span>SUFFICIENCY</span><b>6 / 6</b><p>Groups 6+7 ≥ 50%</p></article>
<article class="warn"><span>RESTORATION</span><b>3 / 6</b><p>contrast pass · peak fail</p></article>
<article class="boundary"><span>LOCALIZATION</span><b>NOT ESTABLISHED</b><p>one-sided evidence</p></article>
</div>
<div class="local-tabs" role="tablist" aria-label="Round 07 局部 mixer 路径实验视图">
<button type="button" role="tab" data-local-tab="scope" aria-selected="true">01 / PATH MAP</button>
<button type="button" role="tab" data-local-tab="matrix" aria-selected="false">02 / 14 MASKS</button>
<button type="button" role="tab" data-local-tab="dual" aria-selected="false">03 / TWO-WAY GATE</button>
<button type="button" role="tab" data-local-tab="branch" aria-selected="false">04 / BRANCH × OUTPUT</button>
<button type="button" role="tab" data-local-tab="spectrum" aria-selected="false">05 / SPECTRUM × AUDIT</button>
</div>
<section class="local-panel" data-local-panel="scope">
<div class="panel-lead">
<div><span>I / FIXED TOPOLOGY</span><h4>先画清 65 个 intervention nodes,再谈“局部”</h4></div>
<p>
layer 21–25 是 Round 05 看过数据后冻结的 spike set;Round 07 预先选择完整
group 6 / 7。group 7 还包含 S 外的 layers 26–28,所以不是事后只挑尖峰层。
</p>
</div>
<div class="group-map">
{[1,2,3,4,5,6,7,8].map((group) => (
<article class:list={{ target: group === 6 || group === 7 }}>
<span>GROUP {group}</span>
<b>L{(group - 1) * 4 + 1}–{group * 4}</b>
<div>
{[0,1,2,3].map((offset) => {
const layer = (group - 1) * 4 + offset + 1;
return <i class:list={{ spike: layer >= 21 && layer <= 25 }}>{layer}</i>;
})}
</div>
<small>8 MIXERS</small>
</article>
))}
<i class="map-arrow">→</i>
<article class="output-node"><span>OUTPUT</span><b>#65</b><p>9 sources</p></article>
</div>
<div class="scope-key">
<span><i class="target"></i>预注册局部 scope:groups 6+7 / 16 mixers</span>
<span><i class="spike"></i>固定 spike set:layers 21–25</span>
<span><i class="plain"></i>其他 49 mixers</span>
</div>
<div class="direction-pair">
<article>
<span>A / SUFFICIENCY</span>
<div><b>其他 49:learned</b><i>+</i><b class="accent">G6+7:uniform</b></div>
<code>Sₓ = ln(Xref / Xm) ÷ Gₓ</code>
<p>只改这 16 个,能否复现全局下降的一半?</p>
</article>
<i>⇄</i>
<article>
<span>B / RESTORATION</span>
<div><b>其他 49:uniform</b><i>+</i><b class="accent">G6+7:learned</b></div>
<code>Rₓ = ln(Xr / Xall) ÷ Gₓ</code>
<p>只恢复这 16 个,能否把全局 gap 恢复一半?</p>
</article>
</div>
<div class="plain-rule">
<b>为什么要两个方向?</b>
<p>mixer 路径非线性交互。一个 scope 在 learned 背景“足够强”,不代表它在 uniform 背景“恢复得回来”。</p>
</div>
</section>
<section class="local-panel" data-local-panel="matrix" hidden>
<div class="panel-lead">
<div><span>II / FROZEN MATRIX</span><h4>把 14 个 mask 全部摆出来,不只展示通过的 scope</h4></div>
<p>条形长度是 global log gap 的归一化 score,不是贡献率;负值和大于 1 都保留。</p>
</div>
<div class="local-controls">
<label>SEED
<select data-local-matrix-seed>
<option value="mean">3-SEED MEAN</option>
{seeds.map((seed) => <option value={String(seed)}>{seed}</option>)}
</select>
</label>
<div>
<button type="button" data-local-matrix-metric="spike_contrast" aria-pressed="true">SPIKE CONTRAST</button>
<button type="button" data-local-matrix-metric="peak_normalized" aria-pressed="false">PEAK / MEAN</button>
</div>
<span data-local-matrix-state>3-SEED MEAN · SPIKE CONTRAST</span>
</div>
<div class="matrix-axis"><span>−</span><i></i><b>0</b><i></i><b>.5</b><i></i><b>1.0</b><i></i><span>2.0+</span></div>
<div class="matrix-bars" data-local-matrix-bars></div>
<div class="matrix-note">
<article><b>S</b><p>learned 背景 → scope uniform</p></article>
<article><b>R</b><p>uniform 背景 → scope learned</p></article>
<article><b>&gt;1</b><p>超过 all-uniform endpoint;不是 &gt;100% 贡献</p></article>
</div>
</section>
<section class="local-panel" data-local-panel="dual" hidden>
<div class="panel-lead">
<div><span>III / PREREGISTERED VERDICT</span><h4>contrast 双向过线;peak 只在 sufficiency 方向过线</h4></div>
<p>主门要求两个方向、两个指标、三个 seed 全部 ≥ .50;均值只用于视觉摘要。</p>
</div>
<div class="verdict-banner">
<span>ONE-SIDED EVIDENCE</span>
<b>LOCALIZATION NOT ESTABLISHED</b>
<p>SUFFICIENCY 6 / 6 PASS · RESTORATION 3 / 6 FAIL</p>
</div>
<div class="dual-table-wrap">
<table class="dual-table">
<thead><tr><th>SEED</th><th>S / CONTRAST</th><th>S / PEAK</th><th>R / CONTRAST</th><th>R / PEAK</th><th>DUAL</th></tr></thead>
<tbody>
{seeds.map((seed) => {
const sc = score("sufficiency", "uniform_groups_6_7_only", seed, "spike_contrast");
const sp = score("sufficiency", "uniform_groups_6_7_only", seed, "peak_normalized");
const rc = score("restoration", "uniform_except_groups_6_7", seed, "spike_contrast");
const rp = score("restoration", "uniform_except_groups_6_7", seed, "peak_normalized");
return (
<tr>
<th>{seed}</th>
<td class="good">{sc.toFixed(3)} ✓</td>
<td class="good">{sp.toFixed(3)} ✓</td>
<td class="good">{rc.toFixed(3)} ✓</td>
<td class="bad">{rp.toFixed(3)} ×</td>
<td class="bad">FAIL</td>
</tr>
);
})}
<tr class="mean">
<th>MEAN</th>
<td>{lab.means.sufficiency.uniform_groups_6_7_only.spike_contrast.toFixed(3)}</td>
<td>{lab.means.sufficiency.uniform_groups_6_7_only.peak_normalized.toFixed(3)}</td>
<td>{lab.means.restoration.uniform_except_groups_6_7.spike_contrast.toFixed(3)}</td>
<td>{lab.means.restoration.uniform_except_groups_6_7.peak_normalized.toFixed(3)}</td>
<td>3 / 6</td>
</tr>
</tbody>
</table>
</div>
<div class="dual-visual">
<article class="pass">
<span>LEARNED BACKGROUND</span><b>只 uniform G6+7</b>
<div><i style="--score:.677"></i><em>CONTRAST .677</em></div>
<div><i style="--score:1"></i><em>PEAK 1.700</em></div>
<p>16 个 mixer 足以把两项都推过 50% global log gap。</p>
</article>
<article class="fail">
<span>UNIFORM BACKGROUND</span><b>只 restore G6+7</b>
<div><i style="--score:.650"></i><em>CONTRAST .650</em></div>
<div><i style="--score:.380"></i><em>PEAK .380</em></div>
<p>contrast 回升;peak 三 seed 都稳定停在 50% 以下。</p>
</article>
</div>
<div class="boundary-pair">
<article class="yes"><span>可以说</span><b>强局部 sufficiency</b><p>Groups 6+7 在 learned 背景足以复现全局下降的大部分。</p></article>
<article class="no"><span>不能说</span><b>尖峰 localization 到 G6+7</b><p>预注册 restoration peak 门没有通过;双向证据不闭合。</p></article>
</div>
</section>
<section class="local-panel" data-local-panel="branch" hidden>
<div class="panel-lead">
<div><span>IV / SECONDARY CONTROLS</span><h4>group 7 的 MLP 分支过闸;group 6 差一格,output 远不到一半</h4></div>
<p>branch 判定只有 sufficiency 方向,证据等级低于双向 localization。</p>
</div>
<div class="local-controls">
<div>
<button type="button" data-local-branch-group="6" aria-pressed="true">GROUP 6</button>
<button type="button" data-local-branch-group="7" aria-pressed="false">GROUP 7</button>
</div>
<span data-local-branch-state>GROUP 6 · NO DOMINANT BRANCH · MLP 5 / 6</span>
</div>
<div class="branch-chart">
<article>
<span>ATTENTION-ONLY</span><b data-local-branch-label="attention">GROUP 6</b>
<div><em>CONTRAST</em><i data-local-branch-bar="attention:spike_contrast"></i><strong data-local-branch-value="attention:spike_contrast">.082</strong></div>
<div><em>PEAK</em><i data-local-branch-bar="attention:peak_normalized"></i><strong data-local-branch-value="attention:peak_normalized">.246</strong></div>
</article>
<article class="mlp">
<span>MLP-ONLY</span><b data-local-branch-label="mlp">GROUP 6</b>
<div><em>CONTRAST</em><i data-local-branch-bar="mlp:spike_contrast"></i><strong data-local-branch-value="mlp:spike_contrast">.250</strong></div>
<div><em>PEAK</em><i data-local-branch-bar="mlp:peak_normalized"></i><strong data-local-branch-value="mlp:peak_normalized">.950</strong></div>
</article>
</div>
<div class="branch-verdicts">
<article><span>GROUP 6</span><b>MLP 5 / 6</b><p>seed 3 contrast S=.177;低于 material .20,不能宣布 dominance。</p></article>
<article class="pass"><span>GROUP 7</span><b>MLP 6 / 6 PASS</b><p>两个指标、三个 seed 自身 material,且领先 attention 至少 15pp。</p></article>
</div>
<div class="control-grid">
<article><span>OUTPUT ONLY · 1 MIXER</span><b>.131 / .187</b><p>contrast / peak mean sufficiency;0 / 6 达到 50%。</p></article>
<article><span>ALL DEPTH · 64 MIXERS</span><b>.949 / 1.145</b><p>depth path 已复现绝大多数 gap;peak 超过 global endpoint。</p></article>
<article class="warning"><span>INTERACTION RESIDUAL</span><b>NEGATIVE</b><p>只作 log-gap bookkeeping;不是加和分解或 hypothesis test。</p></article>
</div>
</section>
<section class="local-panel" data-local-panel="spectrum" hidden>
<div class="panel-lead">
<div><span>V / RAW SHAPE × REPLAY</span><h4>同一个 score 背后,32 层峰值究竟移到哪里?</h4></div>
<p>谱按每个 mode 自己的 layer mean 归一化。位置变化帮助理解,但不替代双向 score gate。</p>
</div>
<div class="local-controls spectrum-controls">
<label>SEED
<select data-local-spectrum-seed>
{seeds.map((seed) => <option value={String(seed)}>{seed}</option>)}
</select>
</label>
<label>MODE
<select data-local-spectrum-mode>
{lab.study.modes.map((mode: string) => <option value={mode}>{modeLabels[mode]}</option>)}
</select>
</label>
<span data-local-spectrum-state>2026073001 · REFERENCE</span>
</div>
<div class="spectrum-layout">
<div class="chart-shell">
<header><b>POST-MLP · NORMALIZED ELEMENT RMS</b><span>layer mean = 1</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="局部 mixer mask 下的 32 层梯度谱" data-local-spectrum-chart>
<rect class="spike-zone" x="0" y="26" width="0" height="280" data-local-spectrum-zone></rect>
<g data-local-spectrum-grid></g>
<polyline class="series" points="" data-local-spectrum-line></polyline>
<g data-local-spectrum-points></g>
</svg>
<div class="spectrum-key"><span><i></i>normalized layer gradient</span><span><i class="zone"></i>S = layers 21–25</span></div>
</div>
<aside>
<span>SELECTED READOUT</span>
<b data-local-spectrum-label>REFERENCE</b>
<dl>
<div><dt>UNIFORM MIXERS</dt><dd data-local-spectrum-count>0 / 65</dd></div>
<div><dt>SPIKE CONTRAST</dt><dd data-local-spectrum-contrast>3.093×</dd></div>
<div><dt>PEAK / MEAN</dt><dd data-local-spectrum-peak>3.241×</dd></div>
<div><dt>PEAK LAYER</dt><dd data-local-spectrum-layer>21</dd></div>
</dl>
</aside>
</div>
<div class="spectrum-story">
<article><span>REFERENCE</span><b>L21 · 3 / 3</b><p>三个 seed 的最高层都是 layer 21。</p></article>
<i>→</i>
<article><span>G6+7 UNIFORM</span><b>L5 / L25 / L6</b><p>peak 强度压到 global endpoint 以下,但位置不统一。</p></article>
<i>→</i>
<article><span>RESTORE G6+7</span><b>L21 · 3 / 3</b><p>位置回归不等于强度恢复超过 50%。</p></article>
</div>
<div class="audit-grid">
<article><span>ROUND 06 EQUIVALENCE</span><b>3 / 3 exact</b><p>model、optimizer、history、BPC 与 parent diagnostics。</p></article>
<article><span>FULL REPLAY</span><b>canonical exact</b><p><code>{shortHash(lab.replay.frozen_compare_sha256)}</code></p></article>
<article><span>FORWARD IDENTITY</span><b>14 / 14</b><p>logits、loss、activations 与 parent summaries exact。</p></article>
<article class="boundary"><span>CLAIM BOUNDARY</span><b>diagnostic only</b><p>不是 K3 checkpoint、训练变体或 additive attribution。</p></article>
</div>
<div class="hash-strip">
<span>PROTOCOL <code>{lab.protocol_id}</code></span>
<span>AGGREGATE <code>{shortHash(lab.hashes.aggregate_canonical_sha256)}</code></span>
<span>COMPACT <code>{shortHash(lab.canonical_sha256_without_self)}</code></span>
</div>
</section>
<script is:inline type="application/json" data-local-payload set:html={json}></script>
</figure>
<script>
const initializeLocalPathLab = (root: HTMLElement) => {
if (root.dataset.ready === "true") return;
root.dataset.ready = "true";
const payload = root.querySelector<HTMLScriptElement>("[data-local-payload]");
if (!payload) return;
const data = JSON.parse(payload.textContent || "{}");
const modeLabels: Record<string, string> = {
detached_learned: "REFERENCE · 全部 detached learned",
uniform_group_6_only: "仅 Group 6 uniform",
uniform_group_7_only: "仅 Group 7 uniform",
uniform_groups_6_7_only: "仅 Groups 6+7 uniform",
uniform_group_6_attention_only: "Group 6 · Attention only",
uniform_group_6_mlp_only: "Group 6 · MLP only",
uniform_group_7_attention_only: "Group 7 · Attention only",
uniform_group_7_mlp_only: "Group 7 · MLP only",
uniform_output_only: "仅 Output mixer uniform",
uniform_depth_all: "64 个 depth mixers uniform",
uniform_all: "全部 65 个 mixers uniform",
uniform_except_group_6: "恢复 Group 6 → detached learned",
uniform_except_group_7: "恢复 Group 7 → detached learned",
uniform_except_groups_6_7: "恢复 Groups 6+7 → detached learned",
};
const metricLabels: Record<string, string> = {
spike_contrast: "SPIKE CONTRAST",
peak_normalized: "PEAK / MEAN",
};
const tabs = [...root.querySelectorAll<HTMLButtonElement>("[data-local-tab]")];
const panels = [...root.querySelectorAll<HTMLElement>("[data-local-panel]")];
const activate = (name: string, focus = false) => {
tabs.forEach((tab) => {
const selected = tab.dataset.localTab === name;
tab.setAttribute("aria-selected", String(selected));
if (selected && focus) tab.focus();
});
panels.forEach((panel) => {
panel.hidden = panel.dataset.localPanel !== name;
});
};
tabs.forEach((tab, index) => {
tab.addEventListener("click", () => activate(tab.dataset.localTab || "scope"));
tab.addEventListener("keydown", (event) => {
if (!["ArrowLeft", "ArrowRight", "Home", "End"].includes(event.key)) return;
event.preventDefault();
let next = index;
if (event.key === "ArrowRight") next = (index + 1) % tabs.length;
if (event.key === "ArrowLeft") next = (index - 1 + tabs.length) % tabs.length;
if (event.key === "Home") next = 0;
if (event.key === "End") next = tabs.length - 1;
activate(tabs[next].dataset.localTab || "scope", true);
});
});
let matrixMetric = "spike_contrast";
const matrixSeed = root.querySelector<HTMLSelectElement>("[data-local-matrix-seed]");
const matrixBars = root.querySelector<HTMLElement>("[data-local-matrix-bars]");
const matrixState = root.querySelector<HTMLElement>("[data-local-matrix-state]");
const family = (mode: string) => mode.startsWith("uniform_except_") ? "restoration" : "sufficiency";
const matrixValue = (mode: string, seed: string, metric: string) => {
if (mode === "detached_learned") return 0;
const kind = family(mode);
if (seed === "mean") return data.means[kind][mode][metric];
return data.scores[kind][mode][seed][metric].score;
};
const rawValue = (mode: string, seed: string, metric: string) => {
if (seed === "mean") return data.means.mode_metrics[mode][metric];
return data.formal_cells.find((cell: any) => String(cell.seed) === seed).mode_metrics[mode][metric];
};
const uniformCount = (mode: string) => data.final_spectra[0].modes[mode].uniform_count;
const renderMatrix = () => {
if (!matrixBars || !matrixSeed || !matrixState) return;
const selectedSeed = matrixSeed.value;
matrixState.textContent = `${selectedSeed === "mean" ? "3-SEED MEAN" : selectedSeed} · ${metricLabels[matrixMetric]}`;
matrixBars.innerHTML = "";
data.study.modes.forEach((mode: string) => {
const value = matrixValue(mode, selectedSeed, matrixMetric);
const row = document.createElement("article");
const kind = family(mode);
row.dataset.family = mode === "detached_learned" ? "reference" : kind;
const magnitude = Math.min(Math.abs(value) / 2, 1) * 100;
row.innerHTML = `
<div><span>${mode === "detached_learned" ? "REF" : kind === "sufficiency" ? "S" : "R"}</span><b>${modeLabels[mode]}</b><small>${uniformCount(mode)} uniform</small></div>
<div class="matrix-track"><i class="${value < 0 ? "negative" : ""}" style="--magnitude:${magnitude}%"></i></div>
<strong>${value >= 0 ? "+" : "−"}${Math.abs(value).toFixed(3)}</strong>
<em>RAW ${rawValue(mode, selectedSeed, matrixMetric).toFixed(3)}×</em>`;
matrixBars.appendChild(row);
});
};
matrixSeed?.addEventListener("change", renderMatrix);
root.querySelectorAll<HTMLButtonElement>("[data-local-matrix-metric]").forEach((button) => {
button.addEventListener("click", () => {
matrixMetric = button.dataset.localMatrixMetric || "spike_contrast";
root.querySelectorAll<HTMLButtonElement>("[data-local-matrix-metric]").forEach((item) =>
item.setAttribute("aria-pressed", String(item === button)));
renderMatrix();
});
});
renderMatrix();
const renderBranch = (group: string) => {
root.querySelectorAll<HTMLButtonElement>("[data-local-branch-group]").forEach((button) =>
button.setAttribute("aria-pressed", String(button.dataset.localBranchGroup === group)));
const gate = data.gates.branch_dominance[`group_${group}`];
const state = root.querySelector<HTMLElement>("[data-local-branch-state]");
if (state) state.textContent = group === "7"
? "GROUP 7 · MLP DOMINANT · 6 / 6 PASS"
: "GROUP 6 · NO DOMINANT BRANCH · MLP 5 / 6";
(["attention", "mlp"] as const).forEach((branch) => {
root.querySelectorAll<HTMLElement>(`[data-local-branch-label="${branch}"]`).forEach((node) => node.textContent = `GROUP ${group}`);
(["spike_contrast", "peak_normalized"] as const).forEach((metric) => {
const mode = `uniform_group_${group}_${branch}_only`;
const value = data.means.sufficiency[mode][metric];
const bar = root.querySelector<HTMLElement>(`[data-local-branch-bar="${branch}:${metric}"]`);
const label = root.querySelector<HTMLElement>(`[data-local-branch-value="${branch}:${metric}"]`);
if (bar) bar.style.setProperty("--branch-score", String(Math.min(value, 1.2) / 1.2));
if (label) label.textContent = value.toFixed(3);
});
});
root.dataset.branchPassed = String(gate.passed);
};
root.querySelectorAll<HTMLButtonElement>("[data-local-branch-group]").forEach((button) =>
button.addEventListener("click", () => renderBranch(button.dataset.localBranchGroup || "6")));
renderBranch("6");
const spectrumSeed = root.querySelector<HTMLSelectElement>("[data-local-spectrum-seed]");
const spectrumMode = root.querySelector<HTMLSelectElement>("[data-local-spectrum-mode]");
const spectrumState = root.querySelector<HTMLElement>("[data-local-spectrum-state]");
const chart = root.querySelector<SVGElement>("[data-local-spectrum-chart]");
const grid = root.querySelector<SVGGElement>("[data-local-spectrum-grid]");
const line = root.querySelector<SVGPolylineElement>("[data-local-spectrum-line]");
const points = root.querySelector<SVGGElement>("[data-local-spectrum-points]");
const zone = root.querySelector<SVGRectElement>("[data-local-spectrum-zone]");
const svg = (name: string) => document.createElementNS("http://www.w3.org/2000/svg", name);
const renderSpectrum = () => {
if (!spectrumSeed || !spectrumMode || !chart || !grid || !line || !points || !zone) return;
const seed = spectrumSeed.value;
const mode = spectrumMode.value;
const run = data.final_spectra.find((item: any) => String(item.seed) === seed);
const selected = run.modes[mode];
const values = selected.normalized;
const left = 56, right = 908, top = 28, bottom = 306;
const maximum = Math.max(2, ...values) * 1.08;
const x = (index: number) => left + index * (right - left) / 31;
const y = (value: number) => bottom - value / maximum * (bottom - top);
grid.innerHTML = "";
[0, .5, 1, 1.5, 2].filter((value) => value <= maximum).forEach((value) => {
const guide = svg("line");
guide.setAttribute("x1", String(left));
guide.setAttribute("x2", String(right));
guide.setAttribute("y1", String(y(value)));
guide.setAttribute("y2", String(y(value)));
grid.appendChild(guide);
const label = svg("text");
label.setAttribute("x", "48");
label.setAttribute("y", String(y(value) + 4));
label.setAttribute("text-anchor", "end");
label.textContent = value.toFixed(1);
grid.appendChild(label);
});
zone.setAttribute("x", String(x(20) - 8));
zone.setAttribute("width", String(x(24) - x(20) + 16));
line.setAttribute("points", values.map((value: number, index: number) => `${x(index)},${y(value)}`).join(" "));
points.innerHTML = "";
values.forEach((value: number, index: number) => {
const point = svg("circle");
point.setAttribute("cx", String(x(index)));
point.setAttribute("cy", String(y(value)));
point.setAttribute("r", index + 1 === selected.peak_layer ? "4.5" : "2.3");
point.dataset.layer = String(index + 1);
points.appendChild(point);
});
if (spectrumState) spectrumState.textContent = `${seed} · ${modeLabels[mode]}`;
const bind = (selector: string, value: string) => {
const node = root.querySelector<HTMLElement>(selector);
if (node) node.textContent = value;
};
bind("[data-local-spectrum-label]", modeLabels[mode]);
bind("[data-local-spectrum-count]", `${selected.uniform_count} / 65`);
bind("[data-local-spectrum-contrast]", `${selected.spike_contrast.toFixed(3)}×`);
bind("[data-local-spectrum-peak]", `${selected.peak_normalized.toFixed(3)}×`);
bind("[data-local-spectrum-layer]", String(selected.peak_layer));
};
spectrumSeed?.addEventListener("change", renderSpectrum);
spectrumMode?.addEventListener("change", renderSpectrum);
renderSpectrum();
};
document.querySelectorAll<HTMLElement>("[data-local-path-lab]").forEach(initializeLocalPathLab);
</script>
<style>
.local-lab {
--ink: #e8e5dd;
--muted: #9a9890;
--line: rgba(232,229,221,.14);
--panel: #121413;
--panel-2: #171a18;
--green: #8aae8d;
--copper: #c98a58;
--red: #c77768;
margin: 42px 0 0;
border: 1px solid var(--line);
background: #0d0f0e;
color: var(--ink);
overflow: hidden;
}
.local-lab figcaption { display: grid; grid-template-columns: auto 1fr auto; gap: 24px; align-items: start; padding: 26px 28px; border-bottom: 1px solid var(--line); }
.local-lab figcaption > span, .panel-lead span { color: var(--copper); font: .58rem/1.3 var(--mono); letter-spacing: .1em; }
.local-lab figcaption h3 { margin: 0; color: var(--ink); font-size: 1.08rem; line-height: 1.35; }
.local-lab figcaption p { margin: 8px 0 0; color: var(--muted); font: .62rem/1.5 var(--mono); }
.local-lab figcaption em { color: var(--muted); font: normal .55rem var(--mono); letter-spacing: .08em; }
.local-ledger { display: grid; grid-template-columns: repeat(6, 1fr); border-bottom: 1px solid var(--line); }
.local-ledger article { min-width: 0; padding: 16px 14px; border-right: 1px solid var(--line); }
.local-ledger article:last-child { border-right: 0; }
.local-ledger span { color: var(--muted); font: .52rem var(--mono); letter-spacing: .08em; }
.local-ledger b { display: block; margin-top: 10px; font: 800 .78rem var(--mono); }
.local-ledger p { margin: 7px 0 0; color: var(--muted); font: .55rem/1.4 var(--mono); }
.local-ledger .pass b { color: var(--green); }
.local-ledger .warn b { color: var(--copper); }
.local-ledger .boundary { background: rgba(199,119,104,.07); }
.local-ledger .boundary b { color: var(--red); font-size: .68rem; }
.local-tabs { display: grid; grid-template-columns: repeat(5, 1fr); border-bottom: 1px solid var(--line); }
.local-tabs button, .local-controls button { appearance: none; border: 0; border-right: 1px solid var(--line); background: transparent; color: var(--muted); cursor: pointer; font: .57rem var(--mono); letter-spacing: .06em; }
.local-tabs button { padding: 16px 10px; }
.local-tabs button[aria-selected="true"], .local-controls button[aria-pressed="true"] { background: rgba(201,138,88,.11); color: var(--copper); box-shadow: inset 0 -2px var(--copper); }
.local-panel { padding: 28px; }
.panel-lead { display: grid; grid-template-columns: 1fr minmax(260px, .7fr); gap: 34px; align-items: end; margin-bottom: 26px; }
.panel-lead h4 { margin: 9px 0 0; color: var(--ink); font-size: 1rem; line-height: 1.4; }
.panel-lead > p { margin: 0; color: var(--muted); font-size: .68rem; line-height: 1.7; }
.group-map { display: grid; grid-template-columns: repeat(8, minmax(72px,1fr)) 22px minmax(82px,.8fr); align-items: stretch; border: 1px solid var(--line); }
.group-map article { position: relative; min-width: 0; padding: 14px 10px; border-right: 1px solid var(--line); background: var(--panel); }
.group-map article.target { background: rgba(201,138,88,.11); box-shadow: inset 0 3px var(--copper); }
.group-map span { color: var(--muted); font: .5rem var(--mono); }
.group-map b { display: block; margin: 12px 0; font: .7rem var(--mono); }
.group-map article > div { display: grid; grid-template-columns: repeat(2, 1fr); gap: 4px; }
.group-map article i { display: grid; place-items: center; aspect-ratio: 1; border: 1px solid var(--line); color: var(--muted); font: normal .55rem var(--mono); }
.group-map article i.spike { border-color: var(--copper); background: var(--copper); color: #16110d; }
.group-map small { display: block; margin-top: 11px; color: var(--muted); font: .45rem var(--mono); }
.map-arrow { display: grid; place-items: center; color: var(--muted); font-style: normal; }
.group-map .output-node { border-left: 1px solid var(--line); border-right: 0; background: rgba(138,174,141,.08); }
.group-map .output-node b { color: var(--green); font-size: 1rem; }
.group-map .output-node p { color: var(--muted); font: .5rem var(--mono); }
.scope-key { display: flex; flex-wrap: wrap; gap: 18px; margin: 14px 0 24px; color: var(--muted); font: .54rem var(--mono); }
.scope-key span { display: inline-flex; align-items: center; gap: 7px; }
.scope-key i { width: 12px; height: 12px; border: 1px solid var(--line); }
.scope-key i.target { background: rgba(201,138,88,.2); border-color: var(--copper); }
.scope-key i.spike { background: var(--copper); }
.scope-key i.plain { background: var(--panel); }
.direction-pair { display: grid; grid-template-columns: 1fr auto 1fr; gap: 18px; align-items: center; }
.direction-pair > article { padding: 20px; border: 1px solid var(--line); background: var(--panel); }
.direction-pair > article > span { color: var(--copper); font: .55rem var(--mono); }
.direction-pair article div { display: flex; gap: 10px; align-items: center; margin: 18px 0; }
.direction-pair article div b { padding: 9px 10px; border: 1px solid var(--line); font: .6rem var(--mono); }
.direction-pair article div b.accent { border-color: var(--copper); color: var(--copper); }
.direction-pair article div i, .direction-pair > i { color: var(--muted); font-style: normal; }
.direction-pair code { color: var(--green); font-size: .65rem; }
.direction-pair p { margin: 12px 0 0; color: var(--muted); font-size: .63rem; }
.plain-rule { margin-top: 20px; padding: 17px 20px; border-left: 3px solid var(--copper); background: rgba(201,138,88,.07); }
.plain-rule b { font-size: .68rem; }
.plain-rule p { margin: 6px 0 0; color: var(--muted); font-size: .65rem; line-height: 1.6; }
.local-controls { display: flex; gap: 12px; align-items: stretch; min-height: 40px; margin-bottom: 18px; border: 1px solid var(--line); }
.local-controls label { display: flex; align-items: center; gap: 10px; padding: 0 12px; border-right: 1px solid var(--line); color: var(--muted); font: .54rem var(--mono); }
.local-controls select { max-width: 310px; border: 0; background: transparent; color: var(--ink); font: .58rem var(--mono); }
.local-controls select option { background: #151715; }
.local-controls > div { display: flex; }
.local-controls button { padding: 0 16px; border-left: 1px solid var(--line); }
.local-controls > span { margin-left: auto; display: flex; align-items: center; padding: 0 14px; color: var(--muted); font: .55rem var(--mono); }
.matrix-axis { display: grid; grid-template-columns: auto 1fr auto 1fr auto 1fr auto 1fr auto; gap: 8px; align-items: center; padding: 0 170px 8px 245px; color: var(--muted); font: .47rem var(--mono); }
.matrix-axis i { height: 1px; background: var(--line); }
.matrix-bars { border: 1px solid var(--line); }
.matrix-bars :global(article) { display: grid; grid-template-columns: 235px 1fr 66px 86px; gap: 12px; align-items: center; min-height: 44px; padding: 5px 12px; border-bottom: 1px solid var(--line); }
.matrix-bars :global(article:last-child) { border-bottom: 0; }
.matrix-bars :global(article[data-family="restoration"]) { background: rgba(138,174,141,.035); }
.matrix-bars :global(article > div:first-child) { display: grid; grid-template-columns: 24px 1fr auto; gap: 8px; align-items: center; min-width: 0; }
.matrix-bars :global(article > div:first-child span) { display: grid; place-items: center; width: 22px; height: 22px; border: 1px solid var(--line); color: var(--copper); font: .52rem var(--mono); }
.matrix-bars :global(article[data-family="restoration"] > div:first-child span) { color: var(--green); }
.matrix-bars :global(article > div:first-child b) { overflow: hidden; color: var(--ink); text-overflow: ellipsis; white-space: nowrap; font: .58rem var(--mono); }
.matrix-bars :global(article > div:first-child small) { color: var(--muted); font: .45rem var(--mono); }
.matrix-bars :global(.matrix-track) { position: relative; height: 12px; background: rgba(232,229,221,.045); }
.matrix-bars :global(.matrix-track::after) { content: ""; position: absolute; left: 13%; top: -4px; bottom: -4px; width: 1px; background: rgba(232,229,221,.4); }
.matrix-bars :global(.matrix-track i) { position: absolute; left: 13%; top: 2px; height: 8px; width: calc(var(--magnitude) * .87); background: var(--copper); }
.matrix-bars :global(article[data-family="restoration"] .matrix-track i) { background: var(--green); }
.matrix-bars :global(.matrix-track i.negative) { right: 87%; left: auto; width: calc(var(--magnitude) * .13); background: var(--red) !important; }
.matrix-bars :global(strong) { text-align: right; color: var(--ink); font: .64rem var(--mono); }
.matrix-bars :global(em) { color: var(--muted); font: normal .48rem var(--mono); }
.matrix-note { display: grid; grid-template-columns: repeat(3, 1fr); margin-top: 14px; border: 1px solid var(--line); }
.matrix-note article { display: grid; grid-template-columns: 30px 1fr; gap: 10px; align-items: center; padding: 12px; border-right: 1px solid var(--line); }
.matrix-note article:last-child { border-right: 0; }
.matrix-note b { color: var(--copper); font: .75rem var(--mono); }
.matrix-note p { margin: 0; color: var(--muted); font-size: .56rem; }
.verdict-banner { display: grid; grid-template-columns: auto 1fr auto; align-items: center; gap: 18px; padding: 18px 20px; border: 1px solid rgba(199,119,104,.5); background: rgba(199,119,104,.08); }
.verdict-banner span, .verdict-banner p { color: var(--muted); font: .54rem var(--mono); }
.verdict-banner b { color: var(--red); font: 800 .9rem var(--mono); }
.verdict-banner p { margin: 0; text-align: right; }
.dual-table-wrap { margin-top: 18px; overflow-x: auto; border: 1px solid var(--line); }
.dual-table { width: 100%; border-collapse: collapse; font: .59rem var(--mono); }
.dual-table th, .dual-table td { padding: 13px 11px; border-right: 1px solid var(--line); border-bottom: 1px solid var(--line); text-align: right; }
.dual-table th:first-child { text-align: left; }
.dual-table thead { color: var(--muted); }
.dual-table .good { color: var(--green); }
.dual-table .bad { color: var(--red); }
.dual-table .mean { background: rgba(232,229,221,.04); }
.dual-visual { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; margin-top: 18px; }
.dual-visual article { padding: 18px; border: 1px solid var(--line); background: var(--panel); }
.dual-visual span { color: var(--muted); font: .52rem var(--mono); }
.dual-visual b { display: block; margin: 9px 0 16px; font: .75rem var(--mono); }
.dual-visual article > div { position: relative; height: 22px; margin: 8px 0; background: rgba(232,229,221,.05); }
.dual-visual article > div::after { content: "50%"; position: absolute; left: 50%; top: -14px; bottom: -2px; border-left: 1px dashed var(--muted); color: var(--muted); font: .42rem var(--mono); }
.dual-visual article i { display: block; width: calc(min(var(--score), 1) * 100%); height: 100%; background: var(--green); }
.dual-visual article.fail i { background: var(--copper); }
.dual-visual em { position: absolute; left: 9px; top: 5px; color: #0d0f0e; font: normal 800 .52rem var(--mono); }
.dual-visual p { margin: 13px 0 0; color: var(--muted); font-size: .62rem; line-height: 1.55; }
.boundary-pair { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; margin-top: 18px; }
.boundary-pair article { padding: 18px; border: 1px solid var(--line); }
.boundary-pair span { color: var(--muted); font: .52rem var(--mono); }
.boundary-pair b { display: block; margin: 9px 0; font-size: .72rem; }
.boundary-pair p { margin: 0; color: var(--muted); font-size: .61rem; line-height: 1.5; }
.boundary-pair .yes { border-color: rgba(138,174,141,.45); }
.boundary-pair .yes b { color: var(--green); }
.boundary-pair .no { border-color: rgba(199,119,104,.45); }
.boundary-pair .no b { color: var(--red); }
.branch-chart { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; }
.branch-chart article { padding: 18px; border: 1px solid var(--line); background: var(--panel); }
.branch-chart article > span { color: var(--muted); font: .52rem var(--mono); }
.branch-chart article > b { display: block; margin: 8px 0 18px; font: .75rem var(--mono); }
.branch-chart article > div { display: grid; grid-template-columns: 74px 1fr 46px; gap: 9px; align-items: center; margin: 10px 0; }
.branch-chart em { color: var(--muted); font: normal .5rem var(--mono); }
.branch-chart i { height: 14px; width: calc(var(--branch-score, .1) * 100%); background: var(--copper); }
.branch-chart .mlp i { background: var(--green); }
.branch-chart strong { text-align: right; font: .58rem var(--mono); }
.branch-verdicts, .control-grid, .audit-grid { display: grid; gap: 14px; margin-top: 16px; }
.branch-verdicts { grid-template-columns: 1fr 1fr; }
.control-grid { grid-template-columns: repeat(3, 1fr); }
.branch-verdicts article, .control-grid article, .audit-grid article { padding: 16px; border: 1px solid var(--line); }
.branch-verdicts span, .control-grid span, .audit-grid span { color: var(--muted); font: .5rem var(--mono); }
.branch-verdicts b, .control-grid b, .audit-grid b { display: block; margin: 9px 0; font: .7rem var(--mono); }
.branch-verdicts p, .control-grid p, .audit-grid p { margin: 0; color: var(--muted); font-size: .59rem; line-height: 1.5; }
.branch-verdicts .pass { border-color: rgba(138,174,141,.45); }
.branch-verdicts .pass b { color: var(--green); }
.control-grid .warning { border-color: rgba(201,138,88,.4); }
.spectrum-controls label:nth-child(2) { flex: 1; }
.spectrum-controls label:nth-child(2) select { width: 100%; max-width: none; }
.spectrum-layout { display: grid; grid-template-columns: 1fr 220px; gap: 16px; }
.chart-shell { border: 1px solid var(--line); background: var(--panel); }
.chart-shell header { display: flex; justify-content: space-between; padding: 12px 15px; border-bottom: 1px solid var(--line); }
.chart-shell header b, .chart-shell header span { font: .53rem var(--mono); }
.chart-shell header span { color: var(--muted); }
.chart-shell svg { display: block; width: 100%; height: auto; }
[data-local-spectrum-grid] :global(line) { stroke: var(--line); stroke-width: 1; }
[data-local-spectrum-grid] :global(text) { fill: var(--muted); font: 10px var(--mono); }
.spike-zone { fill: rgba(201,138,88,.08); }
.series { fill: none; stroke: var(--green); stroke-width: 2; }
[data-local-spectrum-points] :global(circle) { fill: var(--green); stroke: #0d0f0e; stroke-width: 1; }
.spectrum-key { display: flex; gap: 18px; padding: 10px 14px; border-top: 1px solid var(--line); color: var(--muted); font: .5rem var(--mono); }
.spectrum-key span { display: flex; gap: 6px; align-items: center; }
.spectrum-key i { width: 14px; height: 2px; background: var(--green); }
.spectrum-key i.zone { height: 10px; background: rgba(201,138,88,.25); }
.spectrum-layout aside { padding: 18px; border: 1px solid var(--line); background: var(--panel); }
.spectrum-layout aside > span { color: var(--muted); font: .5rem var(--mono); }
.spectrum-layout aside > b { display: block; margin: 10px 0 18px; font: .67rem/1.4 var(--mono); }
.spectrum-layout dl { margin: 0; }
.spectrum-layout dl div { display: flex; justify-content: space-between; gap: 8px; padding: 10px 0; border-top: 1px solid var(--line); }
.spectrum-layout dt, .spectrum-layout dd { margin: 0; font: .53rem var(--mono); }
.spectrum-layout dt { color: var(--muted); }
.spectrum-layout dd { color: var(--green); }
.spectrum-story { display: grid; grid-template-columns: 1fr auto 1fr auto 1fr; gap: 12px; align-items: center; margin-top: 16px; }
.spectrum-story article { min-height: 105px; padding: 15px; border: 1px solid var(--line); }
.spectrum-story span { color: var(--muted); font: .5rem var(--mono); }
.spectrum-story b { display: block; margin: 9px 0; font: .7rem var(--mono); }
.spectrum-story p { margin: 0; color: var(--muted); font-size: .58rem; line-height: 1.5; }
.spectrum-story > i { color: var(--muted); font-style: normal; }
.audit-grid { grid-template-columns: repeat(4, 1fr); }
.audit-grid .boundary { border-color: rgba(199,119,104,.45); }
.hash-strip { display: flex; flex-wrap: wrap; gap: 18px; margin-top: 14px; padding: 12px 14px; border: 1px solid var(--line); color: var(--muted); font: .48rem var(--mono); }
.hash-strip code { color: var(--green); }
@media (max-width: 980px) {
.local-ledger { grid-template-columns: repeat(3, 1fr); }
.local-ledger article:nth-child(3) { border-right: 0; }
.local-tabs { grid-template-columns: repeat(3, 1fr); }
.group-map { grid-template-columns: repeat(4, 1fr); }
.group-map .map-arrow { display: none; }
.group-map .output-node { grid-column: span 4; border-left: 0; border-top: 1px solid var(--line); }
.matrix-bars :global(article) { grid-template-columns: 210px 1fr 58px; }
.matrix-bars :global(em) { display: none; }
.matrix-axis { display: none; }
.spectrum-layout { grid-template-columns: 1fr; }
.audit-grid { grid-template-columns: repeat(2, 1fr); }
}
@media (max-width: 680px) {
.local-lab figcaption { grid-template-columns: 1fr; padding: 20px; }
.local-lab figcaption > span { order: -1; }
.local-ledger { grid-template-columns: repeat(2, 1fr); }
.local-ledger article:nth-child(3) { border-right: 1px solid var(--line); }
.local-ledger article:nth-child(even) { border-right: 0; }
.local-tabs { display: flex; overflow-x: auto; }
.local-tabs button { min-width: 145px; }
.local-panel { padding: 20px 14px; }
.panel-lead { grid-template-columns: 1fr; gap: 14px; }
.group-map { grid-template-columns: repeat(2, 1fr); }
.group-map .output-node { grid-column: span 2; }
.direction-pair { grid-template-columns: 1fr; }
.direction-pair > i { transform: rotate(90deg); text-align: center; }
.local-controls { flex-wrap: wrap; }
.local-controls > span { width: 100%; min-height: 34px; border-top: 1px solid var(--line); }
.matrix-bars :global(article) { grid-template-columns: 1fr 58px; padding: 9px; }
.matrix-bars :global(article > div:first-child) { grid-column: 1 / -1; }
.matrix-bars :global(.matrix-track) { min-width: 0; }
.matrix-note, .dual-visual, .boundary-pair, .branch-chart, .branch-verdicts, .control-grid, .audit-grid { grid-template-columns: 1fr; }
.verdict-banner { grid-template-columns: 1fr; }
.verdict-banner p { text-align: left; }
.dual-table { min-width: 670px; }
.spectrum-story { grid-template-columns: 1fr; }
.spectrum-story > i { transform: rotate(90deg); text-align: center; }
}
</style>
+892
View File
@@ -0,0 +1,892 @@
---
import rawLab from "@/data/k3-attnres-spike-compact.json";
const lab = rawLab as any;
const json = JSON.stringify(lab).replaceAll("<", "\\u003c");
const reductions = lab.verdicts.reduction_robustness.cells;
const replayHash = lab.gates.replay.frozen_compare_sha256;
const shortHash = (value: string) => `${value.slice(0, 10)}…${value.slice(-8)}`;
const reductionLabels: Record<string, string> = {
element_rms: "全元素 RMS",
token_rms_mean: "Token RMS 均值",
token_rms_median: "Token RMS 中位数",
token_rms_p95: "Token RMS P95",
};
---
<figure class="spike-lab" data-spike-lab>
<header class="spike-head">
<div>
<p>ROUND 06 / SPIKE TRAJECTORY × BACKWARD PATH</p>
<h3>尖峰不是出生时就有;它在训练中形成,并对 value 路径敏感</h3>
</div>
<p>
固定 Round 05 的 depth-32 Block 训练格,只在 optimizer 之外增加诊断。
3 个正式 seed、1 个完整重放;所有前向值完全相同,只改变同一次反向传播的局部导数规则。
</p>
</header>
<div class="spike-ledger">
<article><span>FORMAL TARGET BYTES</span><b>196.608M</b><p>3 × 8,000 steps</p></article>
<article><span>FULL REPLAY</span><b>exact</b><p>16 组冻结字段</p></article>
<article><span>VISIBLE POSITIONS</span><b>6 / 6</b><p>每个位置 3 / 3 seed</p></article>
<article class="pass"><span>REDUCTIONS</span><b>12 / 12</b><p>预注册家族内稳健</p></article>
<article class="counter"><span>KEY / SOFTMAX PATH</span><b>−2.0%</b><p>移除后反而略升</p></article>
<article class="pass"><span>VALUE COEFFICIENTS</span><b>−70.2%</b><p>平均 contrast 降幅</p></article>
</div>
<div class="spike-tabs" role="tablist" aria-label="选择 AttnRes 尖峰路径实验视图">
<button type="button" role="tab" data-spike-tab="trajectory" aria-selected="true">
<span>01</span><b>尖峰何时形成</b><small>step 0 → 8,000</small>
</button>
<button type="button" role="tab" data-spike-tab="positions" aria-selected="false">
<span>02</span><b>六个位置都看见什么</b><small>visible ≠ origin</small>
</button>
<button type="button" role="tab" data-spike-tab="reductions" aria-selected="false">
<span>03</span><b>换一种算法还在吗</b><small>12 / 12 cells</small>
</button>
<button type="button" role="tab" data-spike-tab="intervention" aria-selected="false">
<span>04</span><b>哪条反向路径敏感</b><small>same forward · 3 rules</small>
</button>
<button type="button" role="tab" data-spike-tab="mixer" aria-selected="false">
<span>05</span><b>Mixer 线索与证据边界</b><small>association ≠ attribution</small>
</button>
</div>
<section class="spike-panel" data-spike-panel="trajectory">
<div class="panel-lead">
<div><span>I / DEVELOPMENT</span><h4>先问“训练到什么时候出现”,再问“由哪条路径放大”</h4></div>
<p>
尖峰集合 S 固定为 layer 21–25。纵轴 contrast = S 内均值 ÷ 其余层均值:
1 表示没有集中,越大表示梯度越集中在这五层。
</p>
</div>
<div class="lab-controls">
<label>SEED
<select data-spike-time-seed>
<option value="mean">3-SEED MEAN</option>
<option value="2026073001">2026073001</option>
<option value="2026073002">2026073002</option>
<option value="2026073003">2026073003</option>
</select>
</label>
<div>
<button type="button" data-spike-time-metric="spike_contrast" aria-pressed="true">SPIKE CONTRAST</button>
<button type="button" data-spike-time-metric="population_cv" aria-pressed="false">LAYER CV</button>
<button type="button" data-spike-time-metric="peak_normalized" aria-pressed="false">PEAK / MEAN</button>
</div>
</div>
<div class="chart-shell">
<header><b data-spike-time-title>SPIKE CONTRAST · S / REST</b><span data-spike-time-state>3-SEED MEAN</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="六个训练检查点上的尖峰轨迹" data-spike-time-chart>
<g data-spike-time-grid></g>
<line class="threshold" x1="58" x2="900" y1="0" y2="0" data-spike-time-threshold></line>
<polyline class="series copper" points="" data-spike-time-line></polyline>
<g data-spike-time-points></g>
</svg>
<div class="chart-legend"><span><i class="copper"></i>post-MLP activation-gradient</span><span><i class="threshold-key"></i>预注册可见阈值 1.5</span></div>
</div>
<div class="trajectory-story">
<article><span>STEP 0</span><b>0.757×</b><p>三 seed 均低于 1;layer 21–25 并不特殊。</p></article>
<i>→</i>
<article><span>STEP 500</span><b>0.524×</b><p>学习已经发生,但尖峰尚未出现。</p></article>
<i>→</i>
<article class="accent"><span>STEP 2,000</span><b>2.233×</b><p>峰值层同时迁移到 layer 21,结构开始显形。</p></article>
<i>→</i>
<article><span>STEP 8,000</span><b>2.754×</b><p>seed 3 只有 1.881×:方向一致,强度并不整齐。</p></article>
</div>
<div class="plain-rule">
<b>通俗读法</b>
<p>这不像一个由初始化直接写死的“坏层”;它更像训练动力学与 Block mixer 共同塑出的深度模式。</p>
</div>
</section>
<section class="spike-panel" data-spike-panel="positions" hidden>
<div class="panel-lead">
<div><span>II / LOCALIZATION</span><h4>从一个 Block 向前追:尖峰在六个候选张量上都已经可见</h4></div>
<p>
“最早观测到”是沿本站采样点的相对顺序;两个采样点之间仍有 mixer、归一化与残差运算,
所以它不是尖峰的物理出生点。
</p>
</div>
<div class="tensor-path" aria-label="一个 Transformer block 的六个梯度采样位置">
<button type="button" data-spike-position="pre_attention_input" aria-pressed="true"><span>01</span><b>Attn 前输入</b><small>3.724 / 3.189 / 2.040</small></button>
<i>→</i>
<button type="button" data-spike-position="attention_branch_output" aria-pressed="false"><span>02</span><b>Attn 分支输出</b><small>3.200 / 3.662 / 2.005</small></button>
<i>→</i>
<button type="button" data-spike-position="post_attention_state" aria-pressed="false"><span>03</span><b>Attn 后状态</b><small>3.200 / 3.662 / 2.005</small></button>
<i>→</i>
<button type="button" data-spike-position="pre_mlp_input" aria-pressed="false"><span>04</span><b>MLP 前输入</b><small>3.829 / 4.527 / 2.978</small></button>
<i>→</i>
<button type="button" data-spike-position="mlp_branch_output" aria-pressed="false"><span>05</span><b>MLP 分支输出</b><small>3.046 / 3.333 / 1.881</small></button>
<i>→</i>
<button type="button" data-spike-position="post_mlp_state" aria-pressed="false"><span>06</span><b>MLP 后状态</b><small>3.046 / 3.333 / 1.881</small></button>
</div>
<div class="lab-controls position-controls">
<label>SEED
<select data-spike-position-seed>
<option value="2026073001">2026073001</option>
<option value="2026073002">2026073002</option>
<option value="2026073003">2026073003</option>
</select>
</label>
<span data-spike-position-state>ATTENTION 前输入 · SEED 2026073001</span>
</div>
<div class="position-layout">
<div class="chart-shell">
<header><b>NORMALIZED GRADIENT BY LAYER</b><span>layer mean = 1</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="所选张量位置的 32 层归一化梯度谱" data-spike-position-chart>
<rect class="spike-zone" x="0" y="24" width="0" height="280" data-spike-position-zone></rect>
<g data-spike-position-grid></g>
<polyline class="series green" points="" data-spike-position-line></polyline>
<g data-spike-position-points></g>
</svg>
<div class="chart-legend"><span><i class="green"></i>所选位置</span><span><i class="zone-key"></i>S = layer 21–25</span></div>
</div>
<aside>
<span>SELECTED READOUT</span>
<b data-spike-position-label>Attention 前输入</b>
<dl>
<div><dt>SPIKE CONTRAST</dt><dd data-spike-position-contrast>3.724×</dd></div>
<div><dt>PEAK LAYER</dt><dd data-spike-position-peak>22</dd></div>
<div><dt>PEAK / MEAN</dt><dd data-spike-position-peak-value>4.233×</dd></div>
</dl>
<p>六个位置都满足 contrast ≥ 1.5 的 3 / 3 seed 规则。</p>
</aside>
</div>
<div class="boundary-pair">
<article class="yes"><span>可以说</span><b>最早采样点已经看见尖峰</b><p><code>pre_attention_input</code> 是本站六点链中最早的可见位置。</p></article>
<article class="no"><span>不能说</span><b>尖峰起源于 Attention 输入</b><p>观测链不是干预链;更早 mixer 与跨层回传已经作用于该张量。</p></article>
</div>
</section>
<section class="spike-panel" data-spike-panel="reductions" hidden>
<div class="panel-lead">
<div><span>III / ROBUSTNESS</span><h4>把 “RMS” 拆成四种合理算法,layer 21–25 仍然突出</h4></div>
<p>
预注册主家族只包含四种 reduction。每个 seed × reduction 必须同时满足:
contrast ≥ 1.5、top-5 与 S 至少重叠 3 层、相对 element RMS 的 Spearman ≥ .8。
</p>
</div>
<div class="lab-controls">
<label>SEED
<select data-spike-reduction-seed>
<option value="2026073001">2026073001</option>
<option value="2026073002">2026073002</option>
<option value="2026073003">2026073003</option>
</select>
</label>
<label>REDUCTION
<select data-spike-reduction>
<option value="element_rms">全元素 RMS</option>
<option value="token_rms_mean">Token RMS 均值</option>
<option value="token_rms_median">Token RMS 中位数</option>
<option value="token_rms_p95">Token RMS P95</option>
</select>
</label>
<span data-spike-reduction-state>SEED 2026073001 · 全元素 RMS</span>
</div>
<div class="position-layout reduction-layout">
<div class="chart-shell">
<header><b>POST-MLP · NORMALIZED BY LAYER MEAN</b><span>32 layers</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="所选 reduction 下的归一化层梯度谱" data-spike-reduction-chart>
<rect class="spike-zone" x="0" y="24" width="0" height="280" data-spike-reduction-zone></rect>
<g data-spike-reduction-grid></g>
<polyline class="series copper" points="" data-spike-reduction-line></polyline>
<g data-spike-reduction-points></g>
</svg>
</div>
<aside>
<span>THREE GATES</span>
<dl>
<div><dt>CONTRAST ≥ 1.5</dt><dd data-spike-reduction-contrast>3.046×</dd></div>
<div><dt>TOP-5 OVERLAP ≥ 3</dt><dd data-spike-reduction-overlap>4 / 5</dd></div>
<div><dt>SPEARMAN ≥ .8</dt><dd data-spike-reduction-rho>1.000</dd></div>
</dl>
<p class="pass-copy">PASS · 三个条件必须同时成立</p>
</aside>
</div>
<div class="reduction-table-wrap">
<table class="reduction-table">
<thead><tr><th>SEED</th><th>REDUCTION</th><th>CONTRAST</th><th>TOP-5 ∩ S</th><th>SPEARMAN</th><th>VERDICT</th></tr></thead>
<tbody>
{reductions.map((cell: any) => (
<tr>
<th>{cell.seed}</th>
<td>{reductionLabels[cell.reduction]}</td>
<td>{cell.spike_contrast.toFixed(3)}×</td>
<td>{cell.top_five_spike_overlap} / 5</td>
<td>{cell.spearman_vs_element_rms.toFixed(3)}</td>
<td class="good">PASS</td>
</tr>
))}
</tbody>
</table>
</div>
<div class="plain-rule">
<b>判定原文</b><p><code>robust within the preregistered reduction family</code>——只对这四种、这三个 seed、这个诊断 batch 成立。</p>
</div>
</section>
<section class="spike-panel" data-spike-panel="intervention" hidden>
<div class="panel-lead">
<div><span>IV / SAME-FORWARD INTERVENTION</span><h4>前向数值一字不动,只问梯度经过 mixer 时沿哪条边回去</h4></div>
<p>
干预覆盖全部 64 个 depth mixer 与 output mixer。它能定位“反向路径敏感性”,
不能替代重新训练,更不能把降幅解释成因果贡献百分比。
</p>
</div>
<div class="backward-diagram">
<div class="forward"><span>FORWARD / ALL MODES EXACT</span><b>α = softmax(q·k)</b><i>→</i><b>y = Σ αᵢvᵢ</b><i>→</i><b>same logits · same loss</b></div>
<div class="paths">
<article><span>LEARNED</span><b>∂q, ∂k, ∂v</b><p>正常回传:权重路径和值路径都保留。</p></article>
<article><span>DETACHED LEARNED</span><b><s>∂q, ∂k</s> · ∂v</b><p>α 数值不变,但切断 softmax / query / key 源梯度。</p></article>
<article class="accent"><span>UNIFORM VALUE BACKWARD</span><b><s>α</s> → 1/N</b><p>前向仍用 learned α;仅 value-backward 系数改成均匀。</p></article>
</div>
</div>
<div class="lab-controls">
<label>SEED
<select data-spike-intervention-seed>
<option value="2026073001">2026073001</option>
<option value="2026073002">2026073002</option>
<option value="2026073003">2026073003</option>
</select>
</label>
<span data-spike-intervention-state>SEED 2026073001 · THREE BACKWARD RULES</span>
</div>
<div class="chart-shell">
<header><b>POST-MLP NORMALIZED GRADIENT</b><span>same forward · different backward</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="三种反向规则下的 32 层梯度谱" data-spike-intervention-chart>
<rect class="spike-zone" x="0" y="24" width="0" height="280" data-spike-intervention-zone></rect>
<g data-spike-intervention-grid></g>
<polyline class="series gray" points="" data-spike-intervention-line="learned"></polyline>
<polyline class="series copper dash" points="" data-spike-intervention-line="detached_learned"></polyline>
<polyline class="series green" points="" data-spike-intervention-line="uniform_value_backward"></polyline>
</svg>
<div class="chart-legend">
<span><i class="gray"></i>learned</span>
<span><i class="copper dashed"></i>detached learned</span>
<span><i class="green"></i>uniform value backward</span>
</div>
</div>
<div class="intervention-results">
<article class="counter">
<span>LEARNED → DETACHED</span>
<b data-spike-key-drop>−1.54%</b>
<p>contrast “降幅”为负;移除 key / softmax 路径后,峰值在 3 / 3 seed 略微上升。</p>
</article>
<article class="pass">
<span>DETACHED → UNIFORM VALUE</span>
<b data-spike-value-drop>74.79%</b>
<p>contrast 在 3 / 3 seed 至少下降 20%,达到预注册 material sensitivity 阈值。</p>
</article>
<article>
<span>PEAK LAYER</span>
<b data-spike-peak-shift>21 → 2</b>
<p>均匀 value-backward 后,三个 seed 的最高层都从中后段移到 layer 2。</p>
</article>
</div>
<div class="causal-warning">
<b>最强允许结论</b>
<p>该尖峰对“全局 learned value-backward coefficients”具有材料级敏感性;softmax / query / key 源梯度路径没有显示同方向材料级作用。</p>
<span>不是训练出的 uniform 模型 · 不是 70.2% 因果贡献 · 不是某个局部 mixer 的归因</span>
</div>
</section>
<section class="spike-panel" data-spike-panel="mixer" hidden>
<div class="panel-lead">
<div><span>V / ASSOCIATION × AUDIT</span><h4>mixer 权重给出一条可追的线索;干预才把它推进到“路径敏感”</h4></div>
<p>
每个点是 layer 19–28 的一个 layer × seed。横轴是 MLP mixer 的统计量,
纵轴是归一化梯度;相关性仍然只是观察关系。
</p>
</div>
<div class="lab-controls">
<div>
<button type="button" data-spike-mixer-metric="mlp_latest" aria-pressed="true">LATEST-SOURCE WEIGHT</button>
<button type="button" data-spike-mixer-metric="mlp_entropy" aria-pressed="false">NORMALIZED ENTROPY</button>
</div>
<span data-spike-mixer-state>MLP LATEST · LAYERS 19–28 · N=30</span>
</div>
<div class="mixer-layout">
<div class="chart-shell">
<header><b data-spike-mixer-title>MLP LATEST-SOURCE WEIGHT × NORMALIZED GRADIENT</b><span>3 seeds</span></header>
<svg viewBox="0 0 940 390" role="img" aria-label="Mixer 统计量和层梯度的散点关系" data-spike-mixer-chart>
<g data-spike-mixer-grid></g>
<line class="fit" x1="0" x2="0" y1="0" y2="0" data-spike-mixer-fit></line>
<g data-spike-mixer-points></g>
</svg>
</div>
<aside>
<span>LAYERS 19–28</span>
<dl>
<div><dt>PEARSON</dt><dd data-spike-mixer-pearson>+.690</dd></div>
<div><dt>SPEARMAN</dt><dd data-spike-mixer-spearman>+.693</dd></div>
<div><dt>POINTS</dt><dd>30</dd></div>
</dl>
<p data-spike-mixer-copy>越依赖最新 residual source 的 MLP mixer,局部梯度往往越高。</p>
</aside>
</div>
<div class="evidence-ladder">
<article><span>1 / OBSERVE</span><b>layer 21–25 集中</b><p>六位置、四 reduction、三 seed 都能看见。</p></article>
<i>→</i>
<article><span>2 / ASSOCIATE</span><b>MLP latest r≈.69</b><p>相关性提供候选机制,不承担因果结论。</p></article>
<i>→</i>
<article class="accent"><span>3 / INTERVENE</span><b>value route −70.2%</b><p>全局反向规则干预支持路径敏感性。</p></article>
<i>→</i>
<article><span>4 / LOCAL FOLLOW-UP</span><b>单侧 6 / 6 · 双向 3 / 6</b><p>groups 6+7 的 sufficiency 通过;restoration peak 未过,localization 未建立。</p></article>
</div>
<div class="audit-grid">
<article><span>ROUND 05 EQUIVALENCE</span><b>3 / 3 exact</b><p>model、optimizer、history、六个 BPC 与全部 post-MLP 数组一致。</p></article>
<article><span>FULL REPLAY</span><b>16 / 16 exact</b><p>compare hash <code>{shortHash(replayHash)}</code></p></article>
<article><span>NEGATIVE CONTROL</span><b>step 0 exact</b><p>query-zero 与 loss×2 gate 在三 seed 全部通过。</p></article>
<article class="boundary"><span>EVIDENCE IDENTITY</span><b>targeted follow-up</b><p>尖峰集合来自 Round 05;本轮不是盲发现研究。</p></article>
</div>
<div class="hash-strip">
<span>PROTOCOL <code>{lab.protocol_id}</code></span>
<span>REPLAY <code>{shortHash(replayHash)}</code></span>
<span>COMPACT <code>{shortHash(lab.canonical_sha256_without_self)}</code></span>
</div>
</section>
<script is:inline type="application/json" data-spike-payload set:html={json}></script>
</figure>
<script>
const initializeSpikeLab = (root: HTMLElement) => {
if (root.dataset.ready === "true") return;
root.dataset.ready = "true";
const payload = root.querySelector<HTMLScriptElement>("[data-spike-payload]");
if (!payload) return;
const data = JSON.parse(payload.textContent || "{}");
const ns = "http://www.w3.org/2000/svg";
const seeds = data.study.seeds;
const steps = data.trajectory[0].points.map((point: any) => point.step);
const labels: Record<string, string> = {
pre_attention_input: "Attention 前输入",
attention_branch_output: "Attention 分支输出",
post_attention_state: "Attention 后状态",
pre_mlp_input: "MLP 前输入",
mlp_branch_output: "MLP 分支输出",
post_mlp_state: "MLP 后状态",
element_rms: "全元素 RMS",
token_rms_mean: "Token RMS 均值",
token_rms_median: "Token RMS 中位数",
token_rms_p95: "Token RMS P95",
};
const colors = ["#ba603b", "#163f3b", "#77746b"];
const average = (values: number[]) => values.reduce((sum, value) => sum + value, 0) / values.length;
const svgNode = (name: string, attrs: Record<string, string | number>) => {
const node = document.createElementNS(ns, name);
Object.entries(attrs).forEach(([key, value]) => node.setAttribute(key, String(value)));
return node;
};
const seedFinal = (seed: number) => data.final_arrays.find((row: any) => row.seed === seed);
const formatSignedPct = (value: number) => `${value < 0 ? "−" : "+"}${Math.abs(value * 100).toFixed(2)}%`;
const tabs = Array.from(root.querySelectorAll<HTMLButtonElement>("[data-spike-tab]"));
const selectTab = (name: string) => {
tabs.forEach((tab) => tab.setAttribute("aria-selected", String(tab.dataset.spikeTab === name)));
root.querySelectorAll<HTMLElement>("[data-spike-panel]").forEach((panel) => {
panel.hidden = panel.dataset.spikePanel !== name;
});
};
tabs.forEach((tab, index) => {
tab.addEventListener("click", () => selectTab(tab.dataset.spikeTab || "trajectory"));
tab.addEventListener("keydown", (event) => {
if (!["ArrowLeft", "ArrowRight", "Home", "End"].includes(event.key)) return;
event.preventDefault();
const next = event.key === "Home" ? 0
: event.key === "End" ? tabs.length - 1
: (index + (event.key === "ArrowRight" ? 1 : -1) + tabs.length) % tabs.length;
tabs[next].focus();
selectTab(tabs[next].dataset.spikeTab || "trajectory");
});
});
const drawLine = (
svg: SVGSVGElement,
grid: SVGGElement,
series: Record<string, number[]>,
xLabels: string[],
options: {
formatter?: (value: number) => string;
zero?: boolean;
pointGroup?: SVGGElement | null;
threshold?: number;
thresholdLine?: SVGLineElement | null;
zone?: SVGRectElement | null;
} = {},
) => {
grid.replaceChildren();
const allValues = Object.values(series).flat();
const rawMin = options.zero === false ? Math.min(...allValues) : 0;
const rawMax = Math.max(...allValues, options.threshold || 0);
const padding = Math.max(1e-8, (rawMax - rawMin) * .08);
const min = options.zero === false ? rawMin - padding : 0;
const max = rawMax + padding;
const left = 58;
const right = 900;
const top = 24;
const bottom = 304;
const xAt = (index: number) => left + index / Math.max(1, xLabels.length - 1) * (right - left);
const yAt = (value: number) => top + (max - value) / Math.max(1e-30, max - min) * (bottom - top);
const formatter = options.formatter || ((value: number) => value.toFixed(2));
for (let index = 0; index < 5; index += 1) {
const value = max - index / 4 * (max - min);
const y = top + index / 4 * (bottom - top);
grid.append(svgNode("line", { x1: left, x2: right, y1: y, y2: y, class: "grid-line" }));
const text = svgNode("text", { x: left - 10, y: y + 4, class: "axis-label y" });
text.textContent = formatter(value);
grid.append(text);
}
xLabels.forEach((label, index) => {
if (xLabels.length > 12 && ![0, 3, 7, 11, 15, 19, 20, 24, 27, 31].includes(index)) return;
const text = svgNode("text", { x: xAt(index), y: 328, class: "axis-label x" });
text.textContent = label;
grid.append(text);
});
if (options.thresholdLine && options.threshold !== undefined) {
const y = yAt(options.threshold);
options.thresholdLine.setAttribute("y1", String(y));
options.thresholdLine.setAttribute("y2", String(y));
}
if (options.zone && xLabels.length === 32) {
const start = xAt(20) - (right - left) / 62;
const end = xAt(24) + (right - left) / 62;
options.zone.setAttribute("x", String(start));
options.zone.setAttribute("width", String(end - start));
}
Object.entries(series).forEach(([key, values], seriesIndex) => {
const line = svg.querySelector<SVGPolylineElement>(`[data-line-key="${key}"]`)
|| svg.querySelectorAll<SVGPolylineElement>("polyline")[seriesIndex];
const points = values.map((value, index) => `${xAt(index)},${yAt(value)}`).join(" ");
line?.setAttribute("points", points);
if (seriesIndex === 0 && options.pointGroup) {
options.pointGroup.replaceChildren();
values.forEach((value, index) => {
const circle = svgNode("circle", { cx: xAt(index), cy: yAt(value), r: 4, fill: colors[seriesIndex] });
const title = svgNode("title", {});
title.textContent = `${xLabels[index]} · ${formatter(value)}`;
circle.append(title);
options.pointGroup?.append(circle);
});
}
});
return { min, max, xAt, yAt };
};
let timeMetric = "spike_contrast";
const timeSeed = root.querySelector<HTMLSelectElement>("[data-spike-time-seed]")!;
const timeChart = root.querySelector<SVGSVGElement>("[data-spike-time-chart]")!;
const updateTime = () => {
const selected = timeSeed.value === "mean"
? data.trajectory
: data.trajectory.filter((row: any) => String(row.seed) === timeSeed.value);
const values = steps.map((_: number, index: number) =>
average(selected.map((row: any) => row.points[index][timeMetric])));
const names: Record<string, string> = {
spike_contrast: "SPIKE CONTRAST · S / REST",
population_cv: "POPULATION CV ACROSS 32 LAYERS",
peak_normalized: "PEAK / LAYER MEAN",
};
root.querySelector<HTMLElement>("[data-spike-time-title]")!.textContent = names[timeMetric];
root.querySelector<HTMLElement>("[data-spike-time-state]")!.textContent =
timeSeed.value === "mean" ? "3-SEED MEAN" : `SEED ${timeSeed.value}`;
drawLine(
timeChart,
timeChart.querySelector("[data-spike-time-grid]")!,
{ trajectory: values },
steps.map((step: number) => step >= 1000 ? `${step / 1000}K` : String(step)),
{
formatter: (value) => value.toFixed(2),
pointGroup: timeChart.querySelector<SVGGElement>("[data-spike-time-points]"),
threshold: timeMetric === "spike_contrast" ? 1.5 : undefined,
thresholdLine: timeChart.querySelector<SVGLineElement>("[data-spike-time-threshold]"),
},
);
const threshold = timeChart.querySelector<SVGLineElement>("[data-spike-time-threshold]");
if (threshold) threshold.style.display = timeMetric === "spike_contrast" ? "" : "none";
};
root.querySelectorAll<HTMLButtonElement>("[data-spike-time-metric]").forEach((button) => button.addEventListener("click", () => {
timeMetric = button.dataset.spikeTimeMetric || "spike_contrast";
root.querySelectorAll<HTMLButtonElement>("[data-spike-time-metric]").forEach((peer) =>
peer.setAttribute("aria-pressed", String(peer === button)));
updateTime();
}));
timeSeed.addEventListener("change", updateTime);
updateTime();
let positionName = "pre_attention_input";
const positionSeed = root.querySelector<HTMLSelectElement>("[data-spike-position-seed]")!;
const positionChart = root.querySelector<SVGSVGElement>("[data-spike-position-chart]")!;
const updatePosition = () => {
const seed = Number(positionSeed.value);
const cell = seedFinal(seed).position_element_rms[positionName];
drawLine(
positionChart,
positionChart.querySelector("[data-spike-position-grid]")!,
{ position: cell.statistics.normalized },
Array.from({ length: 32 }, (_, index) => String(index + 1)),
{
formatter: (value) => `${value.toFixed(1)}×`,
pointGroup: positionChart.querySelector<SVGGElement>("[data-spike-position-points]"),
zone: positionChart.querySelector<SVGRectElement>("[data-spike-position-zone]"),
},
);
root.querySelector<HTMLElement>("[data-spike-position-state]")!.textContent = `${labels[positionName].toUpperCase()} · SEED ${seed}`;
root.querySelector<HTMLElement>("[data-spike-position-label]")!.textContent = labels[positionName];
root.querySelector<HTMLElement>("[data-spike-position-contrast]")!.textContent = `${cell.statistics.spike_contrast.toFixed(3)}×`;
root.querySelector<HTMLElement>("[data-spike-position-peak]")!.textContent = String(cell.statistics.peak_layer);
root.querySelector<HTMLElement>("[data-spike-position-peak-value]")!.textContent = `${cell.statistics.peak_normalized.toFixed(3)}×`;
};
root.querySelectorAll<HTMLButtonElement>("[data-spike-position]").forEach((button) => button.addEventListener("click", () => {
positionName = button.dataset.spikePosition || "pre_attention_input";
root.querySelectorAll<HTMLButtonElement>("[data-spike-position]").forEach((peer) =>
peer.setAttribute("aria-pressed", String(peer === button)));
updatePosition();
}));
positionSeed.addEventListener("change", updatePosition);
updatePosition();
const reductionSeed = root.querySelector<HTMLSelectElement>("[data-spike-reduction-seed]")!;
const reductionSelect = root.querySelector<HTMLSelectElement>("[data-spike-reduction]")!;
const reductionChart = root.querySelector<SVGSVGElement>("[data-spike-reduction-chart]")!;
const updateReduction = () => {
const seed = Number(reductionSeed.value);
const name = reductionSelect.value;
const cell = seedFinal(seed).post_mlp_reductions[name];
const verdict = data.verdicts.reduction_robustness.cells.find((row: any) => row.seed === seed && row.reduction === name);
drawLine(
reductionChart,
reductionChart.querySelector("[data-spike-reduction-grid]")!,
{ reduction: cell.statistics.normalized },
Array.from({ length: 32 }, (_, index) => String(index + 1)),
{
formatter: (value) => `${value.toFixed(1)}×`,
pointGroup: reductionChart.querySelector<SVGGElement>("[data-spike-reduction-points]"),
zone: reductionChart.querySelector<SVGRectElement>("[data-spike-reduction-zone]"),
},
);
root.querySelector<HTMLElement>("[data-spike-reduction-state]")!.textContent = `SEED ${seed} · ${labels[name]}`;
root.querySelector<HTMLElement>("[data-spike-reduction-contrast]")!.textContent = `${verdict.spike_contrast.toFixed(3)}×`;
root.querySelector<HTMLElement>("[data-spike-reduction-overlap]")!.textContent = `${verdict.top_five_spike_overlap} / 5`;
root.querySelector<HTMLElement>("[data-spike-reduction-rho]")!.textContent = verdict.spearman_vs_element_rms.toFixed(3);
};
reductionSeed.addEventListener("change", updateReduction);
reductionSelect.addEventListener("change", updateReduction);
updateReduction();
const interventionSeed = root.querySelector<HTMLSelectElement>("[data-spike-intervention-seed]")!;
const interventionChart = root.querySelector<SVGSVGElement>("[data-spike-intervention-chart]")!;
const updateIntervention = () => {
const seed = Number(interventionSeed.value);
const cell = seedFinal(seed);
const modes = ["learned", "detached_learned", "uniform_value_backward"];
const series = Object.fromEntries(modes.map((mode) => [mode, cell.interventions[mode].statistics.normalized]));
modes.forEach((mode) => interventionChart.querySelector(`[data-spike-intervention-line="${mode}"]`)?.setAttribute("data-line-key", mode));
drawLine(
interventionChart,
interventionChart.querySelector("[data-spike-intervention-grid]")!,
series,
Array.from({ length: 32 }, (_, index) => String(index + 1)),
{
formatter: (value) => `${value.toFixed(1)}×`,
zone: interventionChart.querySelector<SVGRectElement>("[data-spike-intervention-zone]"),
},
);
const keyRow = data.verdicts.interventions.softmax_key_path.per_seed.find((row: any) => row.seed === seed);
const valueRow = data.verdicts.interventions.value_coefficients.per_seed.find((row: any) => row.seed === seed);
root.querySelector<HTMLElement>("[data-spike-intervention-state]")!.textContent = `SEED ${seed} · THREE BACKWARD RULES`;
root.querySelector<HTMLElement>("[data-spike-key-drop]")!.textContent = formatSignedPct(keyRow.relative_drop_contrast);
root.querySelector<HTMLElement>("[data-spike-value-drop]")!.textContent = `${(valueRow.relative_drop_contrast * 100).toFixed(2)}%`;
root.querySelector<HTMLElement>("[data-spike-peak-shift]")!.textContent =
`${cell.interventions.detached_learned.statistics.peak_layer} → ${cell.interventions.uniform_value_backward.statistics.peak_layer}`;
};
interventionSeed.addEventListener("change", updateIntervention);
updateIntervention();
let mixerMetric = "mlp_latest";
const mixerChart = root.querySelector<SVGSVGElement>("[data-spike-mixer-chart]")!;
const mixerPoints = data.mixer_associations.target_layers.flatMap((row: any) =>
row.per_seed_normalized_gradient.map((gradient: number, index: number) => ({
x: row[mixerMetric],
y: gradient,
seed: seeds[index],
layer: row.layer,
})));
const updateMixer = () => {
const points = data.mixer_associations.target_layers.flatMap((row: any) =>
row.per_seed_normalized_gradient.map((gradient: number, index: number) => ({
x: row[mixerMetric],
y: gradient,
seed: seeds[index],
layer: row.layer,
})));
const group = mixerChart.querySelector<SVGGElement>("[data-spike-mixer-points]")!;
const grid = mixerChart.querySelector<SVGGElement>("[data-spike-mixer-grid]")!;
const fit = mixerChart.querySelector<SVGLineElement>("[data-spike-mixer-fit]")!;
group.replaceChildren();
grid.replaceChildren();
const left = 72, right = 900, top = 24, bottom = 330;
const xs = points.map((point: any) => point.x);
const ys = points.map((point: any) => point.y);
const minX = Math.min(...xs) * .94;
const maxX = Math.max(...xs) * 1.04;
const minY = Math.max(0, Math.min(...ys) * .88);
const maxY = Math.max(...ys) * 1.08;
const xAt = (value: number) => left + (value - minX) / (maxX - minX) * (right - left);
const yAt = (value: number) => top + (maxY - value) / (maxY - minY) * (bottom - top);
for (let index = 0; index < 5; index += 1) {
const yValue = maxY - index / 4 * (maxY - minY);
const y = top + index / 4 * (bottom - top);
grid.append(svgNode("line", { x1: left, x2: right, y1: y, y2: y, class: "grid-line" }));
const yText = svgNode("text", { x: left - 10, y: y + 4, class: "axis-label y" });
yText.textContent = `${yValue.toFixed(1)}×`;
grid.append(yText);
const xValue = minX + index / 4 * (maxX - minX);
const x = left + index / 4 * (right - left);
const xText = svgNode("text", { x, y: 355, class: "axis-label x" });
xText.textContent = xValue.toFixed(2);
grid.append(xText);
}
points.forEach((point: any, index: number) => {
const circle = svgNode("circle", {
cx: xAt(point.x), cy: yAt(point.y), r: point.layer >= 21 && point.layer <= 25 ? 6 : 4,
fill: colors[index % 3], opacity: point.layer >= 21 && point.layer <= 25 ? .95 : .55,
});
const title = svgNode("title", {});
title.textContent = `L${point.layer} · ${point.seed} · x=${point.x.toFixed(3)} · gradient=${point.y.toFixed(3)}×`;
circle.append(title);
group.append(circle);
});
const meanX = average(xs);
const meanY = average(ys);
const slope = xs.reduce((sum: number, x: number, index: number) => sum + (x - meanX) * (ys[index] - meanY), 0)
/ xs.reduce((sum: number, x: number) => sum + (x - meanX) ** 2, 0);
const intercept = meanY - slope * meanX;
fit.setAttribute("x1", String(xAt(minX)));
fit.setAttribute("x2", String(xAt(maxX)));
fit.setAttribute("y1", String(yAt(intercept + slope * minX)));
fit.setAttribute("y2", String(yAt(intercept + slope * maxX)));
const stats = data.mixer_associations.layers_19_28[mixerMetric];
root.querySelector<HTMLElement>("[data-spike-mixer-pearson]")!.textContent = `${stats.pearson >= 0 ? "+" : "−"}${Math.abs(stats.pearson).toFixed(3)}`;
root.querySelector<HTMLElement>("[data-spike-mixer-spearman]")!.textContent = `${stats.spearman >= 0 ? "+" : "−"}${Math.abs(stats.spearman).toFixed(3)}`;
const latest = mixerMetric === "mlp_latest";
root.querySelector<HTMLElement>("[data-spike-mixer-state]")!.textContent =
`${latest ? "MLP LATEST" : "MLP ENTROPY"} · LAYERS 19–28 · N=30`;
root.querySelector<HTMLElement>("[data-spike-mixer-title]")!.textContent =
`${latest ? "MLP LATEST-SOURCE WEIGHT" : "MLP NORMALIZED ENTROPY"} × NORMALIZED GRADIENT`;
root.querySelector<HTMLElement>("[data-spike-mixer-copy]")!.textContent = latest
? "越依赖最新 residual source 的 MLP mixer,局部梯度往往越高。"
: "Mixer 越分散、熵越高,局部梯度往往越低;这仍不是局部因果归因。";
};
root.querySelectorAll<HTMLButtonElement>("[data-spike-mixer-metric]").forEach((button) => button.addEventListener("click", () => {
mixerMetric = button.dataset.spikeMixerMetric || "mlp_latest";
root.querySelectorAll<HTMLButtonElement>("[data-spike-mixer-metric]").forEach((peer) =>
peer.setAttribute("aria-pressed", String(peer === button)));
updateMixer();
}));
void mixerPoints;
updateMixer();
};
document.querySelectorAll<HTMLElement>("[data-spike-lab]").forEach(initializeSpikeLab);
document.addEventListener("astro:page-load", () => {
document.querySelectorAll<HTMLElement>("[data-spike-lab]").forEach(initializeSpikeLab);
});
</script>
<style>
.spike-lab {
--s-ink: #1c201e;
--s-muted: #747168;
--s-line: rgba(28, 32, 30, .16);
--s-paper: #f4f0e7;
--s-raised: #faf7ef;
--s-copper: #ba603b;
--s-green: #163f3b;
width: min(1120px, 100%);
margin: 42px 0;
color: var(--s-ink);
border: 1px solid var(--s-line);
background: var(--s-paper);
box-shadow: 0 30px 80px rgba(28, 32, 30, .09);
}
.spike-head {
display: grid;
grid-template-columns: minmax(0, 1.45fr) minmax(260px, .7fr);
gap: 44px;
padding: 30px;
color: #f5efe4;
background: var(--s-green);
}
.spike-head p { margin: 0; color: rgba(245,239,228,.7); font: .65rem/1.7 var(--mono); }
.spike-head div > p { color: #d58a68; letter-spacing: .08em; }
.spike-head h3 { max-width: 760px; margin: 14px 0 0; color: inherit; font-size: clamp(1.15rem, 2.2vw, 1.75rem); line-height: 1.35; }
.spike-ledger { display: grid; grid-template-columns: repeat(6, 1fr); border-bottom: 1px solid var(--s-line); }
.spike-ledger article { min-height: 126px; padding: 18px 15px; border-right: 1px solid var(--s-line); }
.spike-ledger article:last-child { border-right: 0; }
.spike-ledger span, .panel-lead span { color: var(--s-muted); font: .56rem/1.2 var(--mono); letter-spacing: .08em; }
.spike-ledger b { display: block; margin-top: 23px; font: 700 .95rem/1 var(--mono); }
.spike-ledger p { margin: 8px 0 0; color: var(--s-muted); font-size: .6rem; line-height: 1.45; }
.spike-ledger .pass { color: #f7f0e6; background: var(--s-green); }
.spike-ledger .counter { color: #f7f0e6; background: var(--s-copper); }
.spike-ledger .pass span, .spike-ledger .pass p, .spike-ledger .counter span, .spike-ledger .counter p { color: rgba(247,240,230,.72); }
.spike-tabs { display: grid; grid-template-columns: repeat(5, 1fr); border-bottom: 1px solid var(--s-line); background: #e9e4da; }
.spike-tabs button { min-height: 116px; padding: 16px; text-align: left; color: inherit; border: 0; border-right: 1px solid var(--s-line); background: transparent; cursor: pointer; }
.spike-tabs button:last-child { border-right: 0; }
.spike-tabs button[aria-selected="true"] { color: #f7f0e6; background: var(--s-copper); }
.spike-tabs span, .spike-tabs small { display: block; color: var(--s-muted); font: .54rem/1.25 var(--mono); }
.spike-tabs b { display: block; margin: 15px 0 8px; font-size: .69rem; line-height: 1.35; }
.spike-tabs button[aria-selected="true"] span, .spike-tabs button[aria-selected="true"] small { color: rgba(247,240,230,.72); }
.spike-panel { padding: 30px; }
.panel-lead { display: grid; grid-template-columns: 1.05fr .95fr; gap: 48px; align-items: end; margin-bottom: 28px; }
.panel-lead h4 { max-width: 680px; margin: 10px 0 0; font-size: 1.2rem; line-height: 1.4; }
.panel-lead p { margin: 0; color: var(--s-muted); font-size: .7rem; line-height: 1.7; }
.lab-controls { display: flex; flex-wrap: wrap; gap: 10px 18px; align-items: end; margin-bottom: 20px; }
.lab-controls > div { display: flex; }
.lab-controls button, .lab-controls select { min-height: 38px; padding: 10px 12px; color: var(--s-muted); font: 700 .56rem/1 var(--mono); border: 1px solid var(--s-line); background: var(--s-raised); }
.lab-controls button { cursor: pointer; }
.lab-controls button + button { border-left: 0; }
.lab-controls button[aria-pressed="true"] { color: #fff9ef; background: var(--s-green); }
.lab-controls label { display: grid; gap: 6px; color: var(--s-muted); font: .52rem/1 var(--mono); }
.lab-controls > span { margin: 0 0 11px auto; color: var(--s-copper); font: .55rem/1 var(--mono); }
.chart-shell { min-width: 0; padding: 18px; border: 1px solid var(--s-line); background: var(--s-raised); }
.chart-shell header { display: flex; justify-content: space-between; gap: 12px; color: var(--s-muted); font: .55rem/1 var(--mono); }
.chart-shell svg { display: block; width: 100%; height: auto; margin-top: 10px; overflow: visible; }
.series { fill: none; stroke-width: 3; stroke-linejoin: round; stroke-linecap: round; }
.series.copper { stroke: var(--s-copper); }
.series.green { stroke: var(--s-green); }
.series.gray { stroke: #77746b; }
.series.dash { stroke-dasharray: 8 7; }
.grid-line { stroke: rgba(28,32,30,.1); stroke-width: 1; }
.threshold { stroke: var(--s-copper); stroke-width: 1.5; stroke-dasharray: 6 5; }
.spike-zone { fill: rgba(186,96,59,.11); }
.axis-label { fill: #8b867c; font: 11px var(--mono); }
.axis-label.y { text-anchor: end; }
.axis-label.x { text-anchor: middle; }
.chart-legend { display: flex; flex-wrap: wrap; gap: 18px; margin-top: 4px; color: var(--s-muted); font: .56rem/1 var(--mono); }
.chart-legend span { display: inline-flex; gap: 7px; align-items: center; }
.chart-legend i { width: 22px; height: 3px; }
.chart-legend i.copper { background: var(--s-copper); }
.chart-legend i.green { background: var(--s-green); }
.chart-legend i.gray { background: #77746b; }
.chart-legend i.dashed { height: 0; border-top: 3px dashed var(--s-copper); background: transparent; }
.chart-legend i.threshold-key { height: 0; border-top: 2px dashed var(--s-copper); }
.chart-legend i.zone-key { width: 18px; height: 12px; background: rgba(186,96,59,.16); }
.trajectory-story, .evidence-ladder { display: grid; grid-template-columns: 1fr 26px 1fr 26px 1fr 26px 1fr; gap: 7px; align-items: center; margin-top: 20px; }
.trajectory-story article, .evidence-ladder article { min-height: 150px; padding: 18px; border: 1px solid var(--s-line); background: var(--s-raised); }
.trajectory-story article.accent, .evidence-ladder article.accent { color: #f7f0e6; background: var(--s-green); }
.trajectory-story span, .evidence-ladder span, .position-layout aside > span, .backward-diagram span, .intervention-results span, .causal-warning span, .audit-grid span {
color: var(--s-copper); font: .56rem/1 var(--mono); letter-spacing: .06em;
}
.trajectory-story .accent span, .trajectory-story .accent p, .evidence-ladder .accent span, .evidence-ladder .accent p { color: rgba(247,240,230,.7); }
.trajectory-story b, .evidence-ladder b { display: block; margin-top: 25px; font-size: .78rem; }
.trajectory-story p, .evidence-ladder p { color: var(--s-muted); font-size: .61rem; line-height: 1.5; }
.trajectory-story > i, .evidence-ladder > i, .tensor-path > i { color: var(--s-copper); font-style: normal; text-align: center; }
.plain-rule { display: grid; grid-template-columns: 120px 1fr; margin-top: 20px; color: #f7f0e6; background: var(--s-green); }
.plain-rule > * { margin: 0; padding: 17px; }
.plain-rule b { color: #d58a68; font: .6rem/1.6 var(--mono); }
.plain-rule p { font-size: .67rem; line-height: 1.6; }
.tensor-path { display: grid; grid-template-columns: repeat(11, auto); align-items: stretch; margin-bottom: 22px; overflow-x: auto; }
.tensor-path button { min-width: 130px; min-height: 145px; padding: 16px; text-align: left; color: inherit; border: 1px solid var(--s-line); background: var(--s-raised); cursor: pointer; }
.tensor-path button[aria-pressed="true"] { color: #f7f0e6; background: var(--s-green); }
.tensor-path span, .tensor-path small { display: block; color: var(--s-muted); font: .5rem/1.3 var(--mono); }
.tensor-path b { display: block; margin: 24px 0 10px; font-size: .68rem; }
.tensor-path button[aria-pressed="true"] span, .tensor-path button[aria-pressed="true"] small { color: rgba(247,240,230,.68); }
.tensor-path > i { align-self: center; padding: 0 6px; }
.position-layout, .mixer-layout { display: grid; grid-template-columns: minmax(0, 1fr) 235px; border: 1px solid var(--s-line); background: var(--s-raised); }
.position-layout .chart-shell, .mixer-layout .chart-shell { border: 0; border-right: 1px solid var(--s-line); }
.position-layout aside, .mixer-layout aside { padding: 20px 18px; }
.position-layout aside > b { display: block; margin-top: 18px; font-size: .86rem; }
.position-layout dl, .mixer-layout dl { margin: 24px 0 0; }
.position-layout dl div, .mixer-layout dl div { display: flex; justify-content: space-between; gap: 12px; padding: 15px 0; border-bottom: 1px solid var(--s-line); }
.position-layout dt, .mixer-layout dt { color: var(--s-muted); font: .5rem/1.2 var(--mono); }
.position-layout dd, .mixer-layout dd { margin: 0; font: 700 .68rem/1 var(--mono); }
.position-layout aside p, .mixer-layout aside p { color: var(--s-muted); font-size: .61rem; line-height: 1.55; }
.pass-copy { color: var(--s-green) !important; font: 700 .55rem/1.5 var(--mono) !important; }
.boundary-pair { display: grid; grid-template-columns: repeat(2, 1fr); margin-top: 20px; }
.boundary-pair article { min-height: 170px; padding: 22px; }
.boundary-pair .yes { color: #f7f0e6; background: var(--s-green); }
.boundary-pair .no { background: #e7d8ca; }
.boundary-pair span { color: var(--s-copper); font: .56rem/1 var(--mono); }
.boundary-pair .yes span, .boundary-pair .yes p { color: rgba(247,240,230,.72); }
.boundary-pair b { display: block; margin-top: 25px; font-size: .83rem; }
.boundary-pair p { color: var(--s-muted); font-size: .63rem; line-height: 1.55; }
.reduction-table-wrap { margin-top: 20px; overflow-x: auto; }
.reduction-table { width: 100%; min-width: 790px; border-collapse: collapse; font-size: .62rem; }
.reduction-table th, .reduction-table td { padding: 13px 11px; border-bottom: 1px solid var(--s-line); text-align: left; }
.reduction-table thead th { color: var(--s-muted); font: .52rem/1.3 var(--mono); }
.reduction-table tbody th, .reduction-table .good { font: 700 .61rem/1 var(--mono); }
.reduction-table .good { color: var(--s-green); }
.backward-diagram { border: 1px solid var(--s-line); background: var(--s-raised); }
.backward-diagram .forward { display: flex; flex-wrap: wrap; gap: 14px; align-items: center; padding: 18px; color: #f7f0e6; background: var(--s-green); }
.backward-diagram .forward span { margin-right: auto; color: #d58a68; }
.backward-diagram .forward b { font: 700 .65rem/1 var(--mono); }
.backward-diagram .forward i { color: #d58a68; font-style: normal; }
.backward-diagram .paths { display: grid; grid-template-columns: repeat(3, 1fr); }
.backward-diagram article { min-height: 180px; padding: 20px; border-right: 1px solid var(--s-line); }
.backward-diagram article:last-child { border-right: 0; }
.backward-diagram article.accent { background: #e7d8ca; }
.backward-diagram article b { display: block; margin-top: 30px; font-size: .85rem; }
.backward-diagram article p { color: var(--s-muted); font-size: .62rem; line-height: 1.55; }
.intervention-results { display: grid; grid-template-columns: repeat(3, 1fr); margin-top: 20px; }
.intervention-results article { min-height: 185px; padding: 20px; border: 1px solid var(--s-line); background: var(--s-raised); }
.intervention-results .counter { color: #f7f0e6; background: var(--s-copper); }
.intervention-results .pass { color: #f7f0e6; background: var(--s-green); }
.intervention-results .counter span, .intervention-results .counter p, .intervention-results .pass span, .intervention-results .pass p { color: rgba(247,240,230,.72); }
.intervention-results b { display: block; margin-top: 28px; font: 700 1.1rem/1 var(--mono); }
.intervention-results p { color: var(--s-muted); font-size: .62rem; line-height: 1.55; }
.causal-warning { display: grid; grid-template-columns: 145px 1fr; margin-top: 20px; padding: 20px; border-left: 5px solid var(--s-copper); background: #e7d8ca; }
.causal-warning b { font-size: .78rem; }
.causal-warning p { margin: 0; font-size: .67rem; line-height: 1.6; }
.causal-warning span { grid-column: 2; margin-top: 13px; line-height: 1.5; }
.fit { stroke: var(--s-copper); stroke-width: 2; stroke-dasharray: 8 6; }
.audit-grid { display: grid; grid-template-columns: repeat(4, 1fr); margin-top: 20px; }
.audit-grid article { min-height: 175px; padding: 18px; border: 1px solid var(--s-line); background: var(--s-raised); }
.audit-grid .boundary { color: #f7f0e6; background: var(--s-green); }
.audit-grid .boundary span, .audit-grid .boundary p { color: rgba(247,240,230,.7); }
.audit-grid b { display: block; margin-top: 25px; font-size: .75rem; }
.audit-grid p { color: var(--s-muted); font-size: .59rem; line-height: 1.55; overflow-wrap: anywhere; }
.audit-grid code { font-size: .52rem; }
.hash-strip { display: flex; flex-wrap: wrap; gap: 12px 28px; margin-top: 20px; padding: 16px; background: #e9e4da; }
.hash-strip span { color: var(--s-muted); font: .5rem/1.5 var(--mono); }
.hash-strip code { color: var(--s-green); }
@media (max-width: 920px) {
.spike-ledger { grid-template-columns: repeat(3, 1fr); }
.spike-ledger article:nth-child(3) { border-right: 0; }
.spike-tabs { grid-template-columns: repeat(3, 1fr); }
.trajectory-story, .evidence-ladder { grid-template-columns: 1fr 24px 1fr; }
.trajectory-story > i:nth-of-type(n+3), .evidence-ladder > i:nth-of-type(n+3) { display: none; }
.audit-grid { grid-template-columns: repeat(2, 1fr); }
}
@media (max-width: 680px) {
.spike-head, .panel-lead { grid-template-columns: 1fr; gap: 20px; }
.spike-head, .spike-panel { padding: 20px; }
.spike-ledger { grid-template-columns: repeat(2, 1fr); }
.spike-ledger article:nth-child(3) { border-right: 1px solid var(--s-line); }
.spike-ledger article:nth-child(even) { border-right: 0; }
.spike-tabs { display: flex; overflow-x: auto; }
.spike-tabs button { flex: 0 0 190px; }
.lab-controls { align-items: stretch; }
.lab-controls > div, .lab-controls label { flex: 1 0 100%; }
.lab-controls button { flex: 1; }
.lab-controls select { width: 100%; }
.lab-controls > span { margin: 5px 0 0; line-height: 1.4; }
.chart-shell { padding: 12px 8px; }
.chart-shell header { padding: 0 8px; }
.axis-label { font-size: 9px; }
.trajectory-story, .evidence-ladder { grid-template-columns: 1fr; }
.trajectory-story > i, .evidence-ladder > i { display: block !important; transform: rotate(90deg); }
.plain-rule, .causal-warning { grid-template-columns: 1fr; }
.plain-rule p, .causal-warning span { grid-column: 1; }
.position-layout, .mixer-layout { grid-template-columns: 1fr; }
.position-layout .chart-shell, .mixer-layout .chart-shell { border-right: 0; border-bottom: 1px solid var(--s-line); }
.boundary-pair, .backward-diagram .paths, .intervention-results, .audit-grid { grid-template-columns: 1fr; }
.backward-diagram article { border-right: 0; border-bottom: 1px solid var(--s-line); }
.tensor-path { padding-bottom: 8px; }
.chart-shell svg { min-width: 620px; }
.chart-shell { overflow-x: auto; }
}
</style>
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+9 -9
View File
@@ -128,20 +128,20 @@ const paths = [
<div class="release-grid"> <div class="release-grid">
<a class="release-card k3-release" href="/k3/"> <a class="release-card k3-release" href="/k3/">
<div> <div>
<p class="eyebrow"><span>NEW / K3 ROUND 04</span> ATTENTION RESIDUALS · PREREGISTERED PROBE</p> <p class="eyebrow"><span>NEW / K3 ROUND 08</span> TRAIN-TIME FORWARD · PREREGISTERED GATE</p>
<h2>真实 K3 权重仍有冲突:先把一个可证伪的深度路由问题完整做完</h2> <h2>局部 uniform routing 进入完整训练后,六个尖峰指标格全部衰减</h2>
<p> <p>
不裁剪 <code>A_log [128]</code> 冒充 96-head K3 forward;冻结相同主干、初始化、数据窗口和预算, 四个前向架构变体各跑三个 8,000-step seed,再完整 replay 主格:groups 6+7 的
从零训练 Baseline / Full / Block 共 9 个 2,000-step 格。两种 AttnRes 的三 seed BPC contrast / peak 六格降幅为 32.3%–77.3%,BPC 质量门 4 / 4 通过。结论只限缩小
配对方向都为负,但梯度均匀性指标没有复现论文叙述;支持与反证在同一实验室展示。 depth-32 Block 协议,不是真实 K3 checkpoint 或 Figure 5(c) 复现。
</p> </p>
</div> </div>
<dl> <dl>
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div> <div><dt>MATRIX</dt><dd>12 formal + 1 replay</dd></div>
<div><dt>GRID</dt><dd>3 structures × 3 seeds</dd></div> <div><dt>ATTENUATION</dt><dd>6 / 6 pass</dd></div>
<div><dt>REPLAY</dt><dd>2,000 steps · 8/8 exact</dd></div> <div><dt>QUALITY</dt><dd>4 / 4 pass</dd></div>
</dl> </dl>
<span class="release-arrow" aria-hidden="true">进入训练曲线、深度权重与梯度反证 →</span> <span class="release-arrow" aria-hidden="true">进入训练轨迹、主门、non-additivity 与 32 层谱 →</span>
</a> </a>
<a class="release-card deepseek-release" href="/deepseek/"> <a class="release-card deepseek-release" href="/deepseek/">
<div> <div>
+91 -9
View File
@@ -2,7 +2,10 @@
import BaseLayout from "@/layouts/BaseLayout.astro"; import BaseLayout from "@/layouts/BaseLayout.astro";
import ArchitectureExplorer from "@/components/ArchitectureExplorer.astro"; import ArchitectureExplorer from "@/components/ArchitectureExplorer.astro";
import K3ArtifactLab from "@/components/K3ArtifactLab.astro"; import K3ArtifactLab from "@/components/K3ArtifactLab.astro";
import K3AttnResForwardLab from "@/components/K3AttnResForwardLab.astro";
import K3AttnResGradientLab from "@/components/K3AttnResGradientLab.astro"; import K3AttnResGradientLab from "@/components/K3AttnResGradientLab.astro";
import K3AttnResLocalPathLab from "@/components/K3AttnResLocalPathLab.astro";
import K3AttnResSpikeLab from "@/components/K3AttnResSpikeLab.astro";
import K3AttnResTraceLab from "@/components/K3AttnResTraceLab.astro"; import K3AttnResTraceLab from "@/components/K3AttnResTraceLab.astro";
import K3ReportLab from "@/components/K3ReportLab.astro"; import K3ReportLab from "@/components/K3ReportLab.astro";
import { k3FigureAtlas, k3Ledgers, k3PaperChain, k3ReportMap } from "@/data/k3"; import { k3FigureAtlas, k3Ledgers, k3PaperChain, k3ReportMap } from "@/data/k3";
@@ -40,7 +43,10 @@ const toc = [
["29", "artifacts", "开放权重工件审计"], ["29", "artifacts", "开放权重工件审计"],
["30", "attnres-reduced", "AttnRes 缩小机制实验"], ["30", "attnres-reduced", "AttnRes 缩小机制实验"],
["31", "attnres-gradient", "梯度定义与深度扩展"], ["31", "attnres-gradient", "梯度定义与深度扩展"],
["32", "audit", "21 张图表审计"], ["32", "attnres-spike", "尖峰轨迹与反向路径"],
["33", "attnres-local-path", "局部 mixer 双向干预"],
["34", "attnres-forward", "训练期前向干预"],
["35", "audit", "21 张图表审计"],
["↳", "papers", "100 节点阅读链"], ["↳", "papers", "100 节点阅读链"],
]; ];
@@ -109,13 +115,13 @@ const paperGroups = [
<BaseLayout <BaseLayout
title="Kimi K3 技术报告完整深读:架构、训练、RL、系统与评测" title="Kimi K3 技术报告完整深读:架构、训练、RL、系统与评测"
description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图、两轮十个 AttnRes 独立实验视图与一百个一手阅读节点,逐节读懂 Kimi K3。" description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图、五轮二十五个 AttnRes 独立实验视图与一百个一手阅读节点,逐节读懂 Kimi K3。"
section="k3" section="k3"
> >
<header class="page-hero k3-hero"> <header class="page-hero k3-hero">
<div class="page-hero-inner"> <div class="page-hero-inner">
<div> <div>
<p class="eyebrow"><span>ANCHOR REPORT / ROUND 05</span> KIMI K3 · REPORT → ARTIFACTS → INDEPENDENT PROBE</p> <p class="eyebrow"><span>ANCHOR REPORT / ROUND 08</span> KIMI K3 · REPORT → ARTIFACTS → INDEPENDENT PROBE</p>
<h1>不把报告压成摘要<br />把每个因果环节<br />重新展开</h1> <h1>不把报告压成摘要<br />把每个因果环节<br />重新展开</h1>
<p class="lead"> <p class="lead">
K3 同时扩展序列、深度、宽度、视觉与 Agent 轨迹。真正值得读的不是 2.8T 这个最大数字, K3 同时扩展序列、深度、宽度、视觉与 Agent 轨迹。真正值得读的不是 2.8T 这个最大数字,
@@ -125,11 +131,11 @@ const paperGroups = [
<dl class="page-facts"> <dl class="page-facts">
<div><dt>QUESTIONS</dt><dd>32 张问题账</dd></div> <div><dt>QUESTIONS</dt><dd>32 张问题账</dd></div>
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div> <div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div>
<div><dt>LABS</dt><dd>8 + 4 + 5 + 5 个交互视图</dd></div> <div><dt>LABS</dt><dd>8 + 4 + 5 + 5 + 5 + 5 个交互视图</dd></div>
<div><dt>READING</dt><dd>100 个一手 / 官方节点</dd></div> <div><dt>READING</dt><dd>100 个一手 / 官方节点</dd></div>
<div><dt>MODEL</dt><dd>2.78T total / 104.2B active</dd></div> <div><dt>MODEL</dt><dd>2.78T total / 104.2B active</dd></div>
<div><dt>ARTIFACTS</dt><dd>96 shards · 497,220 tensors</dd></div> <div><dt>ARTIFACTS</dt><dd>96 shards · 497,220 tensors</dd></div>
<div><dt>STATUS</dt><dd>K3 五轮 · 梯度定义闭环</dd></div> <div><dt>STATUS</dt><dd>K3 八轮 · 训练期消融审计</dd></div>
</dl> </dl>
</div> </div>
</header> </header>
@@ -864,13 +870,14 @@ const paperGroups = [
<article><span>O / OBSERVED</span><b>1.4196 TiB tensor data</b><p>96 shards、497,220 entries;不是运行显存,也不是参数量口径。</p></article> <article><span>O / OBSERVED</span><b>1.4196 TiB tensor data</b><p>96 shards、497,220 entries;不是运行显存,也不是参数量口径。</p></article>
<article><span>D / CLOSED LOOP</span><b>69 KDA · 24 MLA · 92 MoE</b><p>配置、tensor names 与 header shape 三方闭合。</p></article> <article><span>D / CLOSED LOOP</span><b>69 KDA · 24 MLA · 92 MoE</b><p>配置、tensor names 与 header shape 三方闭合。</p></article>
<article><span>X / RTX 5090</span><b>exact 6/6 · max error 0</b><p>官方 torch reference;fixed BF16 mean 2.6210 ms。</p></article> <article><span>X / RTX 5090</span><b>exact 6/6 · max error 0</b><p>官方 torch reference;fixed BF16 mean 2.6210 ms。</p></article>
<article class="warning"><span>U / UNRESOLVED</span><b>A_log [128] ≠ expected [96]</b><p>checkpoint 与公开代码 / kernel API 的形状冲突保留在主视区,不擅自解释。</p></article> <article class="warning"><span>U / MAIN UNRESOLVED</span><b>A_log [128] ≠ expected [96]</b><p>main 未修;#144 改成 128,#150 验零后裁成 96,两个社区 PR 都未合并。</p></article>
</div> </div>
<K3ArtifactLab /> <K3ArtifactLab />
<div class="hero-actions"> <div class="hero-actions">
<a class="button primary" href="https://huggingface.co/moonshotai/Kimi-K3">打开官方开放权重</a> <a class="button primary" href="https://huggingface.co/moonshotai/Kimi-K3">打开官方开放权重</a>
<a class="button" href="https://github.com/MoonshotAI/FlashKDA">打开 FlashKDA 官方实现</a> <a class="button" href="https://github.com/MoonshotAI/FlashKDA">打开 FlashKDA 官方实现</a>
<a class="button" href="https://github.com/MoonshotAI/FlashKDA/blob/master/BENCHMARK_GB200.md">核对作者 GB200 benchmark</a> <a class="button" href="https://github.com/MoonshotAI/FlashKDA/blob/master/BENCHMARK_GB200.md">核对作者 GB200 benchmark</a>
<a class="button" href="https://huggingface.co/moonshotai/Kimi-K3/discussions/150">审阅社区 PR #150</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/flashkda">复跑本站 RTX 5090 探针</a> <a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/flashkda">复跑本站 RTX 5090 探针</a>
</div> </div>
</section> </section>
@@ -879,8 +886,8 @@ const paperGroups = [
<p class="eyebrow"><span>30</span> REDUCED ATTENTION RESIDUALS STUDY</p> <p class="eyebrow"><span>30</span> REDUCED ATTENTION RESIDUALS STUDY</p>
<h2>真实 K3 权重还不能诚实 forward;先把一个可证伪的 AttnRes 问题完整做完</h2> <h2>真实 K3 权重还不能诚实 forward;先把一个可证伪的 AttnRes 问题完整做完</h2>
<p class="lede"> <p class="lede">
checkpoint 的 <code>A_log [128]</code> 与 config、remote code、FlashKDA、vLLM 和 SGLang checkpoint 的 <code>A_log [128]</code> 与 main 代码期望的 96 heads 仍没有官方转换合同;
期望的 96 heads 仍没有公开转换合同。本轮不裁剪权重冒充 K3,而是预注册一个从零训练的缩小实验: 社区 #144 / #150 提出相反修复,均未合并。本轮不把候选 patch 冒充 K3 官方 forward,而是预注册一个从零训练的缩小实验:
相同 16-block Transformer、相同数据窗口与相同初始化,只改变 residual source 的读取拓扑。 相同 16-block Transformer、相同数据窗口与相同初始化,只改变 residual source 的读取拓扑。
</p> </p>
<div class="artifact-callout"> <div class="artifact-callout">
@@ -922,8 +929,83 @@ const paperGroups = [
</div> </div>
</section> </section>
<section class="article-section" id="attnres-spike">
<p class="eyebrow"><span>32</span> SPIKE TRAJECTORY × BACKWARD PATH</p>
<h2>layer 21–25 的尖峰从什么时候出现,又对 mixer 的哪条反向路径敏感?</h2>
<p class="lede">
第六轮不把上一轮的局部尖峰直接解释成机制。本站预注册六个训练时点、六个张量位置、
四种 confirmatory reduction 与三种 same-forward backward rule;正式训练严格复用
Round 05 depth-32 Block 的模型、数据、optimizer 与 schedule,诊断全部位于 optimizer 之外。
结果显示尖峰在 step 500 后才形成、六个位置均可见;切断 softmax / query / key 源梯度没有降低尖峰,
而把全局 learned value-backward coefficients 改成均匀后,contrast 平均下降 70.2%。
</p>
<div class="artifact-callout">
<article><span>F / FROZEN</span><b>3 formal + 1 replay</b><p>196,608,000 formal target bytes;完整重放再加 65,536,000。</p></article>
<article><span>X / DEVELOPMENT</span><b>step 500 → 2,000</b><p>三 seed 的尖峰 contrast 从 0.524× 跃到平均 2.233×。</p></article>
<article><span>X / ROBUSTNESS</span><b>12 / 12 PASS</b><p>四种预注册 reduction 都保留 layer 21–25 集中。</p></article>
<article class="warning"><span>B / BOUNDARY</span><b>global sensitivity</b><p>不是训练变体、因果贡献百分比或局部 mixer 归因。</p></article>
</div>
<K3AttnResSpikeLab />
<div class="hero-actions">
<a class="button primary" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_SPIKE_AUDIT.md">阅读完整结果审计</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_SPIKE_PROTOCOL.md">核对预注册协议</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_SPIKE_SCOPING.md">查看范围冻结</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/attnres_spike">复跑实验与分析</a>
</div>
</section>
<section class="article-section" id="attnres-local-path">
<p class="eyebrow"><span>33</span> LOCAL MIXER PATH × BIDIRECTIONAL GATE</p>
<h2>全局 value-route 很敏感;能不能把它诚实地缩到 group 6 / 7?</h2>
<p class="lede">
第七轮先冻结 14 种 same-forward mask:在 learned 背景只把固定 scope 改成 uniform,
测 sufficiency;再从 all-uniform 背景只恢复同一 scope 的 detached-learned coefficients,
测 restoration。groups 6+7 的 16 个 depth mixers 在 sufficiency 的两个指标、三个 seed
全部复现至少一半 global log gap;但 restoration 只在 contrast 通过,peak 三 seed 均低于 50%。
因而主结论不是“定位成功”,而是强单侧证据与未闭合的双向 localization。
</p>
<div class="artifact-callout">
<article><span>F / FROZEN</span><b>14 masks · 65 visits</b><p>每次 forward 审计 exact identity set、顺序、唯一性与 census。</p></article>
<article><span>X / SUFFICIENCY</span><b>6 / 6 PASS</b><p>groups 6+7 mean S:contrast .677;peak 1.700。</p></article>
<article class="warning"><span>X / RESTORATION</span><b>3 / 6 FAIL</b><p>contrast mean .650;peak 仅 .380,三 seed 均未过 .50。</p></article>
<article class="warning"><span>B / VERDICT</span><b>one-sided evidence</b><p>双向 localization 未建立;group 7 MLP 只作次级 sufficiency 发现。</p></article>
</div>
<K3AttnResLocalPathLab />
<div class="hero-actions">
<a class="button primary" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_LOCAL_PATH_AUDIT.md">阅读完整结果审计</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md">核对预注册协议</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_LOCAL_PATH_GROK_REVIEW.md">查看结果前对抗审阅</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/attnres_local_path">复跑矩阵与 replay</a>
</div>
</section>
<section class="article-section" id="attnres-forward">
<p class="eyebrow"><span>34</span> TRAIN-TIME FORWARD INTERVENTION</p>
<h2>诊断期的局部敏感性进入完整训练后,尖峰指标还会衰减吗?</h2>
<p class="lede">
第八轮不再只改 diagnostic backward,而是让 group 6 / 7 的 parameter-free
uniform mixer 进入每一次 train、eval 与 diagnostic forward。四个变体各跑三个
8,000-step seed,并把同 seed 的 Round 05 learned run 锁为历史配对 reference;
主变体 groups 6+7 的 contrast 与 peak 六格降幅全部超过 20%,同时 BPC 质量门
4 / 4 通过,指定 seed 的完整重训 scientific payload exact。
</p>
<div class="artifact-callout">
<article><span>F / FROZEN</span><b>12 formal + 1 replay</b><p>851,968,000 个新 target bytes;每格 65,536,000。</p></article>
<article><span>X / ATTENUATION</span><b>6 / 6 PASS</b><p>contrast drop 62.1%–77.3%;peak drop 32.3%–62.0%。</p></article>
<article><span>X / QUALITY</span><b>4 / 4 PASS</b><p>三 seed ΔBPC 最大 +0.00960;均值 +0.00658。</p></article>
<article class="warning"><span>B / BOUNDARY</span><b>reduced protocol only</b><p>训练期架构消融;不是 K3 checkpoint 或 Figure 5(c) 复现。</p></article>
</div>
<K3AttnResForwardLab />
<div class="hero-actions">
<a class="button primary" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_FORWARD_TRAINING_AUDIT.md">阅读完整结果审计</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_FORWARD_TRAINING_PROTOCOL.md">核对预注册协议</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_FORWARD_TRAINING_IMPLEMENTATION_REVIEW.md">查看两阶段实现审阅</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/attnres_forward">复跑训练、分析与 replay</a>
</div>
</section>
<section class="article-section" id="audit"> <section class="article-section" id="audit">
<p class="eyebrow"><span>32</span> FIGURE & TABLE AUDIT</p> <p class="eyebrow"><span>35</span> FIGURE & TABLE AUDIT</p>
<h2>Figure 1–16、Table 1–5:每张图究竟支持什么,不能支持什么</h2> <h2>Figure 1–16、Table 1–5:每张图究竟支持什么,不能支持什么</h2>
<div class="figure-atlas"> <div class="figure-atlas">
{k3FigureAtlas.map(([id, report, title, contract]) => ( {k3FigureAtlas.map(([id, report, title, contract]) => (
+11 -5
View File
@@ -9,7 +9,7 @@ const researching = chapters.filter((chapter) => ["researching", "drafting"].inc
const workstreams = [ const workstreams = [
{ label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" }, { label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" },
{ label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" }, { label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" },
{ label: "Kimi K3 深读", value: 98, next: "对齐 layer 21–25 梯度尖峰与 mixer weights;等待 A_log 官方转换合同" }, { label: "Kimi K3 深读", value: 99, next: "设计前向训练变体,并等待 A_log 社区方案的官方裁决" },
{ label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau,并加入真实小语料复现" }, { label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau,并加入真实小语料复现" },
{ label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" }, { label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" },
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" }, { label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
@@ -50,7 +50,7 @@ const workstreams = [
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div> <div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div> <div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div> <div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-30 10:05 CST</dd></div> <div><dt>UPDATED</dt><dd>2026-07-30 14:42 CST</dd></div>
<div><dt>MODE</dt><dd>持续迭代,不锁死版本</dd></div> <div><dt>MODE</dt><dd>持续迭代,不锁死版本</dd></div>
</dl> </dl>
</div> </div>
@@ -97,7 +97,7 @@ const workstreams = [
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article> <article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全,包含先修依赖和三条贯穿案例。</p></article> <article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全,包含先修依赖和三条贯穿案例。</p></article>
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article> <article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
<article><span>✓</span><h3>九十九个原创交互视图</h3><p>K3 三轴图、八联报告实验、四联开放工件实验与两轮十联 AttnRes 独立实验,DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验,以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article> <article><span>✓</span><h3>一百一十四个原创交互视图</h3><p>K3 三轴图、八联报告实验、四联开放工件实验与五轮二十五联 AttnRes 独立实验,DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验,以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article> <article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article> <article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article> <article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
@@ -107,6 +107,9 @@ const workstreams = [
<article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions,审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes;四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article> <article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions,审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes;四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
<article><span>✓</span><h3>Kimi K3 四轮 AttnRes 独立实验</h3><p>冻结三结构 × 三 seed 的 9 个 2,000-step 格;Full / Block 相对 Baseline 的平均 paired delta 为 −0.01457 / −0.04247 BPC,但核心参数梯度 CV 没有复现论文叙述。指定正式格全新进程八字段 exact,五视图同时展示结果、反证、成本与 claim boundary。</p></article> <article><span>✓</span><h3>Kimi K3 四轮 AttnRes 独立实验</h3><p>冻结三结构 × 三 seed 的 9 个 2,000-step 格;Full / Block 相对 Baseline 的平均 paired delta 为 −0.01457 / −0.04247 BPC,但核心参数梯度 CV 没有复现论文叙述。指定正式格全新进程八字段 exact,五视图同时展示结果、反证、成本与 claim boundary。</p></article>
<article><span>✓</span><h3>Kimi K3 五轮梯度定义与深度扩展</h3><p>先确认 Figure 5 没有公开唯一 gradient telemetry 合同,再冻结 16/32 blocks × Baseline/Block × 3 seeds 的 12 个 8,000-step 格。Block 的首尾失衡 6/6 改善但全层 CV 6/6 恶化,两个深度都判为 mixed;指定 32 层格完整重训的模型、优化器与全部冻结字段 exact。</p></article> <article><span>✓</span><h3>Kimi K3 五轮梯度定义与深度扩展</h3><p>先确认 Figure 5 没有公开唯一 gradient telemetry 合同,再冻结 16/32 blocks × Baseline/Block × 3 seeds 的 12 个 8,000-step 格。Block 的首尾失衡 6/6 改善但全层 CV 6/6 恶化,两个深度都判为 mixed;指定 32 层格完整重训的模型、优化器与全部冻结字段 exact。</p></article>
<article><span>✓</span><h3>Kimi K3 六轮尖峰轨迹与反向路径</h3><p>严格复用 Round 05 depth-32 Block 的三个正式格:尖峰在 step 500 后形成,六个位置 3/3 seed 可见,四种 reduction 12/12 格稳健。切断 key/softmax 源梯度没有降低尖峰;uniform value-backward 让 contrast 平均下降 70.2%,只判为全局 backward-rule sensitivity。完整 replay 的 16 组冻结字段 exact。</p></article>
<article><span>✓</span><h3>Kimi K3 七轮局部路径双向审计</h3><p>冻结 14 个 same-forward mask,把 groups 6+7 的 16 个 depth mixers 同时放进 sufficiency 与 restoration 两个方向。充分性 6/6 过 50%,恢复性却只有 contrast 3/3 通过、peak 0/3 通过,因此正式状态为 one-sided evidence / localization not established。三个正式格、完整 replay、selector 与 forward identity 全部 exact。</p></article>
<article><span>✓</span><h3>Kimi K3 八轮训练期前向干预</h3><p>四个 forward architecture variants × 三 seed × 8,000 steps,加一格完整 replay;groups 6+7 的 contrast / peak 六格降幅全部超过 20%,BPC 质量门 4/4 通过。13 个 raw、自哈希、historical pairing 与 scientific replay 全部过闸;Grok 结果后复算 blocking error 为 0。结论只限固定缩小协议,不是真实 K3 checkpoint 或 Figure 5(c) 复现。</p></article>
<article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel;6/6 官方参考逐元素相等,并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article> <article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel;6/6 官方参考逐元素相等,并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article> <article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
<article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article> <article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article>
@@ -135,7 +138,6 @@ const workstreams = [
</div> </div>
<div class="queue-table"> <div class="queue-table">
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div> <div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
<div><span>P0</span><strong>K3 五轮后续</strong><p>对齐 layer 21–25 尖峰、pre-attention / pre-MLP 与 mixer source weights → 等待 A_log 官方合同后进入真实 checkpoint forward</p><em>局部机制 + 工件边界</em></div>
<div><span>P0</span><strong>DeepSeek 八轮后续</strong><p>干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div> <div><span>P0</span><strong>DeepSeek 八轮后续</strong><p>干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div> <div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div> <div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
@@ -217,7 +219,7 @@ const workstreams = [
<div><time>2026-07-29</time><b>A_log 形状冲突保持未决</b><p>checkpoint 的 [128] 与 config / remote code / FlashKDA API 期待的 [96] 并列展示;不宣布权重损坏,也不把 channel-wise 假设写成真实 forward。</p></div> <div><time>2026-07-29</time><b>A_log 形状冲突保持未决</b><p>checkpoint 的 [128] 与 config / remote code / FlashKDA API 期待的 [96] 并列展示;不宣布权重损坏,也不把 channel-wise 假设写成真实 forward。</p></div>
<div><time>2026-07-29</time><b>FlashKDA 编译与执行永久分两道闸门</b><p>容器产出 sm_120a wheel 只证明可编译;RTX 5090 的 6/6 official-reference exact suite 通过后,才把证据升级为本机执行 X。</p></div> <div><time>2026-07-29</time><b>FlashKDA 编译与执行永久分两道闸门</b><p>容器产出 sm_120a wheel 只证明可编译;RTX 5090 的 6/6 official-reference exact suite 通过后,才把证据升级为本机执行 X。</p></div>
<div><time>2026-07-29</time><b>作者表与 RTX 5090 表永久分账</b><p>H20 / GB200 保持 O;本站只报告独立环境、协议、300 samples/mode 和延迟分布,未跑本机 FLA 就不写本机 speedup。</p></div> <div><time>2026-07-29</time><b>作者表与 RTX 5090 表永久分账</b><p>H20 / GB200 保持 O;本站只报告独立环境、协议、300 samples/mode 和延迟分布,未跑本机 FLA 就不写本机 speedup。</p></div>
<div><time>2026-07-30</time><b>K3 权重冲突不靠裁剪“解决”</b><p>HF / FlashKDA / vLLM / SGLang 仍没有公开 A_log 128→96 转换;真实 K3 forward 继续标为未决。</p></div> <div><time>2026-07-30</time><b>K3 权重冲突仍没有官方裁决</b><p>官方 main 仍保留 128↔96 不匹配;社区 #144 把 head 数改成 128,#150 验证尾部全零后按 96 裁入,两案都未合并。真实 K3 forward 继续标为未决。</p></div>
<div><time>2026-07-30</time><b>AttnRes 缩小实验先冻结、后运行</b><p>三结构共享公共主干、初始化、窗口与优化器;只按三个 paired seed 和预注册 −0.010 BPC 阈值给出本协议内方向判断。</p></div> <div><time>2026-07-30</time><b>AttnRes 缩小实验先冻结、后运行</b><p>三结构共享公共主干、初始化、窗口与优化器;只按三个 paired seed 和预注册 −0.010 BPC 阈值给出本协议内方向判断。</p></div>
<div><time>2026-07-30</time><b>支持结果与梯度反结果同时进入主视区</b><p>Full / Block 的最终 BPC 同向改善;核心参数 gradient RMS CV 却高于 Baseline,不换指标掩盖。</p></div> <div><time>2026-07-30</time><b>支持结果与梯度反结果同时进入主视区</b><p>Full / Block 的最终 BPC 同向改善;核心参数 gradient RMS CV 却高于 Baseline,不换指标掩盖。</p></div>
<div><time>2026-07-30</time><b>正式重放不把 wall time 纳入 exact</b><p>Block / seed-1 的模型、优化器、曲线、历史、诊断和环境八字段 exact;计时受调度影响,单独报告。</p></div> <div><time>2026-07-30</time><b>正式重放不把 wall time 纳入 exact</b><p>Block / seed-1 的模型、优化器、曲线、历史、诊断和环境八字段 exact;计时受调度影响,单独报告。</p></div>
@@ -225,6 +227,10 @@ const workstreams = [
<div><time>2026-07-30</time><b>首尾平衡与全层 CV 永久分账</b><p>Block 在 6/6 配对中改善 first/last,却因中后段局部尖峰让 CV 在 6/6 配对中恶化;联合判定保持 mixed。</p></div> <div><time>2026-07-30</time><b>首尾平衡与全层 CV 永久分账</b><p>Block 在 6/6 配对中改善 first/last,却因中后段局部尖峰让 CV 在 6/6 配对中恶化;联合判定保持 mixed。</p></div>
<div><time>2026-07-30</time><b>绝对梯度尺度必须与归一化谱同屏</b><p>Block mean gradient 约为 Baseline 的 54%–57%;更接近 1 的首尾比不能偷换成各层信号更强。</p></div> <div><time>2026-07-30</time><b>绝对梯度尺度必须与归一化谱同屏</b><p>Block mean gradient 约为 Baseline 的 54%–57%;更接近 1 的首尾比不能偷换成各层信号更强。</p></div>
<div><time>2026-07-30</time><b>32 层完整重放扩到状态哈希</b><p>8,000-step fresh replay 的全部冻结字段以及 model / optimizer state hashes exact;额外 replay bytes 单列,不混入 formal 预算。</p></div> <div><time>2026-07-30</time><b>32 层完整重放扩到状态哈希</b><p>8,000-step fresh replay 的全部冻结字段以及 model / optimizer state hashes exact;额外 replay bytes 单列,不混入 formal 预算。</p></div>
<div><time>2026-07-30</time><b>尖峰是定向复查,不是盲发现</b><p>layer 21–25 来自 Round 05;Round 06 先固定目标集合,再检查训练时点、张量位置、reduction 与反向路径。</p></div>
<div><time>2026-07-30</time><b>最早可见不等于物理起源</b><p>pre-attention input 是六个采样点中最早可见位置;更早 mixer 与跨层回传已经作用,不能写成尖峰从这里注入。</p></div>
<div><time>2026-07-30</time><b>同一前向只识别反向规则敏感性</b><p>三模式的 logits、loss、activations 与 mixer summaries exact;uniform value-backward 的 70.2% contrast 降幅不是训练变体或因果贡献百分比。</p></div>
<div><time>2026-07-30</time><b>局部充分性不自动成为双向定位</b><p>groups 6+7 的 sufficiency 6/6 通过,但 restoration peak 0/3 通过;非线性交互让两个方向不同,正式结论保持 localization not established。</p></div>
<div><time>2026-07-29</time><b>32-token 对照改为同源 16→24</b><p>TNEWS 只有 105/10,000 条达到 32 tokens,强行统一会落入约 1% 极端长尾;24-token eligibility 仍保留 1,609 条中文候选。</p></div> <div><time>2026-07-29</time><b>32-token 对照改为同源 16→24</b><p>TNEWS 只有 105/10,000 条达到 32 tokens,强行统一会落入约 1% 极端长尾;24-token eligibility 仍保留 1,609 条中文候选。</p></div>
<div><time>2026-07-29</time><b>长度敏感性必须成对重采样</b><p>16-token 输入严格是 24-token 输入前缀,2,000 次 bootstrap 共用 prompt indices;结果只描述固定 cohort 的长度敏感性。</p></div> <div><time>2026-07-29</time><b>长度敏感性必须成对重采样</b><p>16-token 输入严格是 24-token 输入前缀,2,000 次 bootstrap 共用 prompt indices;结果只描述固定 cohort 的长度敏感性。</p></div>
<div><time>2026-07-29</time><b>三类 cohort 永久分身份</b><p>自然长度回答本批样本如何路由;matched-16 / 24 回答同一 prompt 多看 8 tokens 后如何变化,不把二者混成内容因果。</p></div> <div><time>2026-07-29</time><b>三类 cohort 永久分身份</b><p>自然长度回答本批样本如何路由;matched-16 / 24 回答同一 prompt 多看 8 tokens 后如何变化,不把二者混成内容因果。</p></div>