docs: record K3 Round 08 release

PROGRESS.md

@@ -8,7 +8,7 @@
 |---|---:|---:|---|
 | 研究框架与规范 | 进行中 | 83% | Scaling Laws 二轮拟合复现与逐图精读 |
 | 网站设计系统 | 进行中 | 89% | 打印样式与更多通用可视化组件 |
-| Kimi K3 深读 | 三轮实证进行中 | 94% | 接入真实 hidden-state / expert-load / cache traces，并重绘报告数值图 |
+| Kimi K3 深读 | 八轮实证已收敛 | 100% | 稳定维护；真实 forward 等待 `A_log` 官方裁决 |
 | 语言模型前史 | 完成首版 | 78% | Kneser–Ney、LSTM、Bahdanau 逐图精读与真实小语料复现 |
 | Transformer 基础 | 完成首版 | 79% | 多头电路、归一化 traces 与真实 kernel / KV 配置 |
 | 表示、位置与残差高速公路 | 完成首版 | 81% | 真实 hidden-state / norm traces、长上下文位置外推与深层稳定性消融 |
@@ -41,7 +41,7 @@
 - [x] 完成 486 篇关键论文索引，覆盖 16 个标签专题与 Kimi/DeepSeek 聚光主线。
 - [x] 完成可检索、可按专题筛选的论文库页面。
 - [x] 完成 K3、语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全十七篇首版长文。
-- [x] 完成 K3 三轴架构、八联报告实验与四联开放工件实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 二十二联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等八十九个原创交互视图。
+- [x] 完成 K3 三轴架构、八联报告实验、四联开放工件实验、Round 04 / 05 / 06 / 07 / 08 各五联 AttnRes 独立实验、语言模型前史四联实验、Transformer 四联实验、表示深度四联实验、DeepSeek 二十二联实验、长上下文、MoE 路由、推理三页签，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment、Agent、原生多模态与评测安全专题各四页签等一百一十四个原创交互视图。
 - [x] 完成长上下文首版：五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
 - [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文，并建立长上下文研究账本。
 - [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文，并建立 MoE 研究账本。
@@ -272,10 +272,49 @@
 - [x] 第二十二个 DeepSeek 交互实验用五页签讲解显式 CRN sampler、选定任务 bootstrap、32 题浏览器、4×4 tape 诊断与十二字段复现/失败/偏差账；正式协议、manifest、runner、独立 evaluator、分析、复现与完整/compact 数据均已进入开源树。
 - [x] Round 08 本地闸门通过：88 个 Astro 文件零诊断，21 个页面、1,151 个站内引用、12 个跨页锚点零失败；数据合同、DeepSeek 全量历史、Round 06/07/08 三套 sampling 专项真实 Chrome 回归均通过，桌面与 390px 移动端无文档级溢出。
 - [x] DeepSeek Round 08 以源提交 `975ed3d`、不可变镜像 `20260729T221654Z-975ed3d` 发布；OCI index digest `sha256:759a8446…21452b`，复用 NAS `12010→8080`、NPM host 31 / cert 41 与门户 `LLM ATLAS / projects / 180`。容器 healthy、0 次重启，21/21 公网路由、HTTPS/2、gzip / immutable assets、Round 08 专项与 DeepSeek 全量生产 Chrome 回归通过；保留 `20260729T195952Z-9211333` 回滚。
+- [x] K3 Round 04 在任何正式结果前冻结 `llm-atlas-k3-attnres-reduced-v1`：WikiText-2 固定 revision 与 byte 拼接、192,000-window SHA-256 日程、16-block / 32-sublayer / width-192 公共主干、Baseline / Full / Block 三结构、三个初始化 seed、2,000-step 预算、主 BPC 判据、RMS / mixer / gradient 诊断与独立 replay 合同全部先落盘。
+- [x] 九个正式训练格全部完成，共消费 147,456,000 target bytes；同一 seed 的 9,541,824 个公共 core 参数初始化 hash 在三结构间 exact。Full − Baseline 三个 paired delta 为 `−.015996 / −.013207 / −.014498`，Block 为 `−.052660 / −.041349 / −.033388`，分别以均值 `−.014567 / −.042466 BPC` 满足预注册方向支持规则。
+- [x] 反结果同时进入主审计：核心参数 gradient RMS 的 16-block mean CV 为 Baseline `0.3447`、Full `0.5087`、Block `0.6306`；本 reduced metric 没有复现论文的大模型梯度均匀性叙述，不改换指标掩盖。
+- [x] 指定 Block / seed 2026073001 进行全新进程 2,000-step replay；manifest、model、optimizer、hashes、evaluations、training history、diagnostic、environment 八字段全部 exact，最终 BPC 同为 `1.947877975922777`；timing 明确不要求 exact。
+- [x] K3 Round 04 五视图实验室完成：三 seed BPC 曲线、Residual RMS / Block 锯齿、Full / Block depth-weight heatmap、梯度反证、成本/哈希/claim boundary 分开展示；完整 9-run JSON、compact 数据、复现清单、协议、审计、训练与聚合代码进入公开仓库。
+- [x] Round 04 本地闸门通过：91 个受检文件零诊断/提示，21 个页面、1,151 个站内引用、12 个跨页锚点零失败；冻结数据、AttnRes 专项与 K3 全量真实 Chrome 回归通过，桌面/390px 移动端零文档级溢出、零运行时异常。
+- [x] K3 Round 04 以功能源提交 `4ce780d`、不可变镜像 `20260729T233142Z-4ce780d` 发布；OCI index digest `sha256:6e89f802…25f582`，复用 NAS `12010→8080`、NPM host 31 / cert 41 与门户 `LLM ATLAS / projects / 180`。容器 healthy、0 次重启，21/21 公网页面、HTTPS/2、gzip / immutable assets、AttnRes 专项与 K3 全量生产 Chrome 回归通过；保留 `20260729T221654Z-975ed3d` 回滚。
+- [x] K3 Round 05 一手定义审计确认 Figure 5(c) 未公开 gradient tensor、norm、reduction、diagnostic batch、AMP / clipping 时点或统计代码；Round 04 参数梯度与 Round 05 post-MLP output activation gradient 永久分对象记账，不把本站 operationalization 冒充作者实现。
+- [x] 在任何 formal 输出前冻结 16 / 32 blocks、Baseline / Block、三个 seed、8,000 steps、六个诊断点、CV + 首尾四分位 imbalance 联合判据、FP32 residual accumulator、20-step 双 smoke 与 depth-32 Block 完整 replay；768,000-window schedule SHA-256 为 `5041e09b…f4e`。
+- [x] 12 个 formal 格全部完成，共 786,432,000 target bytes；公共主干与三个 gate input tensor hashes 在 paired 架构间 exact。Block 的验证 BPC 在 6 / 6 配对中更低，depth-16 / 32 mean delta 为 `−.008938 / −.009872`，但不追加事后 BPC support 阈值。
+- [x] activation-gradient 结果分裂：首/末四分位 imbalance 在 6 / 6 配对改善，depth-16 / 32 均值为 `+61.0% / +72.0%`；全层 CV 却在 6 / 6 配对恶化，均值相对 reduction 为 `−10.3% / −60.0%`。两个 depth 都按预注册规则判为 `mixed / inconclusive`，总判定 `depth-dependent or inconclusive`。
+- [x] 绝对 gradient mean 仅为 Baseline 的 `57.4% / 54.4%`；参数 gradient CV 从 `0.416→0.683 / 0.397→0.772`，继续保留反结果。Output RMS 最后/第一层比则由 Baseline `4.59× / 6.08×` 降至 Block `1.17× / 1.89×`。
+- [x] 指定 depth-32 / Block / seed-2026073001 从初始化完整重训 8,000 steps；排除 run-kind / timing 后冻结字段 compare SHA-256 同为 `46300a45…4817`，model / optimizer state hashes exact。正式/compact/reproduction 物理 SHA-256 为 `ad461cbe…a8d / 5377a5e7…68e3 / aedcde6a…dea6`。
+- [x] K3 Round 05 五视图实验室完成：论文定义已知/未定义、绝对/归一化深度谱、六 checkpoint 时间轨迹、Output RMS 组节律、activation/parameter/BPC/成本/重放联合账全部可切换；21 个 raw JSON、完整 aggregate、compact、runner、analyzer、协议与审计进入公开树。
+- [x] Round 05 本地闸门通过：94 个 Astro 文件零诊断/提示，21 个页面、1,151 个站内引用、12 个跨页锚点零失败；冻结数据、新专项、Round 04 与 K3 全量真实 Chrome 回归通过，桌面/390px 移动端零文档级溢出、零运行时异常。
+- [x] K3 Round 05 以功能提交 `f7670ef`、不可变镜像 `20260730T022610Z-f7670ef` 发布；OCI digest `sha256:02f902c1a64f976c6ed5d8614a826fca487b483f9addf2392637ea3f021cc09b`，NAS healthy / 零重启、21/21 公网路由、HTTPS/2、gzip/immutable 与三套 K3 生产 Chrome 回归全部通过；保留 `20260729T233142Z-4ce780d` 回滚。
+- [x] K3 Round 06 在正式训练前冻结 `llm-atlas-k3-attnres-spike-path-v1`：固定 Round 05 depth-32 Block、三个 seed、8,000 steps、六个 checkpoint、layer 21–25 目标集合、六个张量位置、四种 confirmatory reduction、三条 same-forward backward rule、20% 材料级阈值与完整 seed-1 replay；研究身份明确为受 Round 05 启发的 targeted follow-up，不冒充 blind discovery。
+- [x] 三个正式格共消费 196,608,000 target bytes；最终模型、优化器、六个 BPC、全部训练历史与 Round 05 post-MLP 数组逐 seed exact。完整 replay 再消费 65,536,000 bytes，16 组冻结字段全部 exact，compare SHA-256 为 `05396eaf…cbb87`。
+- [x] 尖峰发展轨迹显示 layer 21–25 不是初始化时固有：三 seed mean contrast 在 step 0 / 100 / 500 为 `0.757 / 0.563 / 0.524×`，到 step 2,000 / 4,000 / 8,000 升为 `2.233 / 2.594 / 2.754×`；seed 3 的最终 `1.881×` 同向但更弱。
+- [x] 六个张量位置的最终 mean contrast 均超过 1.5，且每个位置 3 / 3 seed 可见；最早采样点 `pre_attention_input` 已可见，但观测顺序不升级为物理起源或注入位置。
+- [x] 预注册 reduction 家族 12 / 12 格同时满足 contrast ≥ 1.5、top-5 与目标集合至少重叠 3 层、相对 element RMS 的 Spearman ≥ .8；正式结论严格写作 `robust within the preregistered reduction family`。
+- [x] 全局切断 64 个 depth mixer 与 output mixer 的 softmax/query/key source-gradient paths 后，contrast / peak 平均相对“降幅”为 `−2.00% / −0.84%`，三 seed 均略微上升，不支持材料级降低。将 learned value-backward coefficients 换成 `1/N` 后，contrast / peak 平均下降 `70.20% / 37.03%`，三 seed 都超过 20% 阈值，peak layer 全部移到 layer 2。
+- [x] mixer 观察关联与反向干预分层：layers 19–28 的 MLP latest weight 对 normalized gradient Pearson / Spearman 为 `.690 / .693`，MLP entropy 为 `−.636 / −.640`；相关性只提供候选机制，全局干预只支持 value-route sensitivity，不冒充局部 mixer attribution。
+- [x] K3 Round 06 五视图实验室完成：训练轨迹、六位置谱、12 格 reduction 稳健性、same-forward 三规则干预与 mixer 散点/证据阶梯可交互；protocol、scoping、audit、runner、analyzer、raw/aggregate/compact/reproduction 全部进入公开树。
+- [x] Round 06 本地闸门通过：97 个 Astro 文件零诊断，21 个页面、1,151 个站内引用、12 个跨页锚点零失败；Round 04/05/06 三套冻结数据、三套专项、K3 全量与全站 22 套真实 Chrome 回归通过，桌面/390px 移动端零文档级溢出、零 offender、零运行时异常；首页首发卡与陈旧跨页断言同步到 Round 06。
+- [x] K3 Round 06 以运行源提交 `b393780`、不可变镜像 `20260730T042651Z-b393780` 发布；OCI index digest `sha256:4e4cb2e065526f50c10cf759bab80a5a871177812ca9fa5c9da77c662c81a63a`，NAS healthy / 0 次重启、Compose Manager 标签、VPS→NAS、NPM host 31 / cert 41、DNS、HTTPS/2、首页/K3 公网内容、门户 `LLM ATLAS / projects / 180` 与全站 22 套生产 Chrome 回归全部通过；保留 Round 05 `20260730T022610Z-f7670ef` 与中间 `20260730T042110Z-a1a52d4` 回滚点。
+- [x] K3 Round 07 在正式输出前冻结 `llm-atlas-k3-attnres-local-path-v1`：固定 Round 06 三个 depth-32 Block 正式格、layer 21–25 spike set、groups 6+7、14 个 exact selector masks、sufficiency / restoration 双向 log-gap score、主门 50%、单组/分支/输出控制与完整 seed-1 replay；Grok 只读对抗审阅提出的六个 blocking protocol 问题全部在冻结前修正。
+- [x] 三个正式格与 replay 共消费 262,144,000 target bytes；Round 06 parent equivalence、forward identity、65-node census、selector identity / order / uniqueness、负对照与 loss-scale 闸门全部通过。正式最终 BPC 为 `1.7123525941 / 1.7093240656 / 1.7030966813`。
+- [x] global gap 在两个指标 × 三 seed 的 6 / 6 格通过。groups 6+7 sufficiency 的 mean score 为 contrast `.677`、peak `1.700`，6 / 6 ≥ `.50`；restoration mean 为 contrast `.650`、peak `.380`，contrast 3 / 3 通过而 peak 0 / 3 通过，正式状态固定为 `one_sided_evidence_localization_not_established`。
+- [x] 次级控制显示 group 7 MLP-only branch 在 sufficiency 方向 6 / 6 通过 material + margin 门；group 6 MLP-only 为 5 / 6，不能宣布 dominance。output-only mean sufficiency 仅 `.131 / .187`、0 / 6 过 50%；all-depth 为 `.949 / 1.145`。所有 score 都是非加性 log-gap 诊断，不写成贡献率。
+- [x] K3 Round 07 五视图实验室完成：65-node 路径图、14-mask 全矩阵、双向主门、branch/output controls 与 32 层原始谱/replay 审计可交互；protocol、scoping、Grok 结果前审阅、audit、runner、analyzer、packager、四个 raw JSON、aggregate、compact 与 reproduction 全部进入公开树。
+- [x] Round 07 本地闸门通过：100 个 Astro 文件零诊断，21 个页面、1,151 个站内引用、12 个跨页锚点零失败；Round 04/05/06/07 四套冻结数据、四套 K3 专项、K3 全量与全站 23 套真实 Chrome 回归通过。动态生成矩阵的 scoped CSS 退化由截图复查发现并修复；桌面/390px 移动端零文档级溢出、零 offender、零运行时异常。
+- [x] K3 Round 07 以功能源提交 `dc8ec30`、不可变镜像 `20260730T063811Z-dc8ec30` 发布；OCI index digest `sha256:98d441412408774628195ce23f431d25034bb1590210c11230d884513898e71d`。NAS `12010→8080` healthy / 0 次重启、Compose Manager 标签、VPS→NAS、NPM host 31 / cert 41、DNS、HTTPS/2、gzip、immutable asset、首页/K3 公网内容、门户 `LLM ATLAS / projects / 180` 与全站 23 套生产 Chrome 回归全部通过；保留 Round 06 `20260730T042651Z-b393780` 回滚点。
+- [x] K3 Round 08 在正式输出前冻结 `llm-atlas-k3-attnres-forward-training-v1`：四个 train-time forward variants × 三 seed × 8,000 steps、同 seed Round 05 historical pairing、step 8,000 的 layers 21–25 contrast / peak 双指标 20% 主门、BPC 每 seed / mean 护栏、描述性 `I67` 与完整 primary replay；不把架构消融写成 pure-forward 因果实验。
+- [x] 12 formal + 1 replay 全部一次完成，每格 65,536,000 target bytes；新处理总量 851,968,000，历史 references 196,608,000 单列。联合 groups 6+7 的 contrast / peak 六格降幅为 62.1%–77.3% / 32.3%–62.0%，6 / 6 通过；BPC 三 seed 最大 `+.009598`、均值 `+.006578`，质量门 4 / 4 通过。
+- [x] primary seed-1 replay scientific payload exact，SHA-256 为 `b85563ca…c051`；冻结主状态为 `forward_training_attenuation_established_within_reduced_protocol`。`I67` 的 step-8,000 mean 为 contrast `−.3670`、peak `−.1704`，只保留为跨独立训练的描述性 log residual。
+- [x] 结果前 Grok 实现审阅指出 smoke-only empty selector 的空 census；矩阵结束后删除 early return、加入 `forward_calls > 0`，修补后的 learned wrapper 实际执行 39 次 forward，父/包装器 15 组科学字段仍 exact。结果后 Grok 只读复算报告 `blocking_errors=0`、status / replay 均确认。
+- [x] K3 Round 08 五视图实验室完成：forward contract、六 checkpoint 训练轨迹、attenuation+BPC 主门、non-additivity map 与 32 层谱/replay audit 可交互；protocol、scoping、两阶段 Grok 审阅、runner、analyzer、13 raw、aggregate、compact、reproduction 与结果审计全部进入公开树。
+- [x] Round 08 本地闸门通过：101 个 Astro 文件零诊断，21 个页面、1,151 个站内引用、12 个跨页锚点零失败；Round 04–08 五套冻结数据、五套 K3 专项、K3 全量与全站 24 套真实 Chrome 回归通过。截图复查修复黑色实验室标题对比度；桌面/390px 移动端零文档级溢出、零 offender、零运行时异常。
+- [x] K3 Round 08 以功能源提交 `f177fa6`、不可变镜像 `20260730T104228Z-f177fa6` 发布；OCI index digest `sha256:0b5b9d9cbf1b4538a37b01727ba1d16be13f991408d3d60e20bc76023b9008bb`。NAS `12010→8080` healthy / 0 次重启、Compose Manager、VPS→NAS、NPM host 31 / cert 41、DNS、HTTPS/2、gzip、首页/K3/进度页、门户 `LLM ATLAS / projects / 180` 与全站 24 套生产 Chrome 回归全部通过；保留 Round 07 `20260730T064331Z-badf498` 回滚点。
 
 ## 正在进行
 
-- [ ] K3 三轮下一闸门：获得真实 token hidden states、expert load 与 cache traces，解释或修订 `A_log [128]` 工件冲突，再做 Figure 3/4/5 数值重绘和独立小模型复现。
 - [ ] DeepSeek 八轮下一闸门：推进干预式 mediation、SM90 FlashMLA、FP8 / pipeline traces 与 R1-like RL 小模型复现。
 - [ ] 表示、位置与残差二轮：真实 hidden-state / norm traces、长上下文位置外推复现与 mHC / AttnRes 深层稳定性消融。
 - [ ] 评测安全二轮：真实 cross-harness / pass@k 复跑、Judge 元评测、动态污染与过拒案例。
@@ -464,6 +503,31 @@
 | 2026-07-30 | 输出长度揭示强任务域交互 | system-at-period 在 Code 为负、Math 为正，两个选定任务带都不跨零；不从长度外推能力 |
 | 2026-07-30 | Round 08 十二字段重放过闸 | 64/64 exact；uniform hash、完整 token IDs、文本、stop 与 RNG 一并进入复现合同，评分 gold 提前加载的流程偏差公开保留 |
 | 2026-07-30 | DeepSeek Round 08 任务 bootstrap / CRN 里程碑发布 | 源提交 `975ed3d`、镜像 `20260729T221654Z-975ed3d`、OCI `sha256:759a8446…21452b`；21/21 公网路由与生产专项/全量 Chrome 通过，保留 Round 07 回滚点 |
+| 2026-07-30 | K3 `A_log [128]` 冲突仍没有官方裁决 | 官方 main 仍期望 96；社区 #144 改为 128，#150 验证 tail zero 后裁为 96，两案都未合并。不把候选 patch 冒充官方 K3 forward |
+| 2026-07-30 | AttnRes 缩小实验先冻结再训练 | 三结构共享公共主干、初始化、窗口与优化器；三 seed paired BPC 只按预注册 `3/3 same direction + mean≤−.010` 判为本协议内方向支持 |
+| 2026-07-30 | 主结果与机制反结果同时发布 | Full / Block BPC 方向支持；核心参数 gradient RMS CV 却高于 Baseline，明确写成未复现论文梯度叙述 |
+| 2026-07-30 | 独立重放按数值合同而非计时合同验收 | Block / seed-1 的八组冻结字段 2,000 steps exact；wall time 受调度影响，不要求或声称 bit-exact |
+| 2026-07-30 | K3 Round 04 缩小 AttnRes 里程碑发布 | 功能源 `4ce780d`、镜像 `20260729T233142Z-4ce780d`、OCI `sha256:6e89f802…25f582`；21/21 公网页面与生产专项/全量 Chrome 通过，保留 Round 08 回滚点 |
+| 2026-07-30 | AttnRes 的“梯度”先按公开证据拆对象 | 论文 Figure 5 没有公开唯一 telemetry 合同；参数梯度与 post-MLP activation gradient 不再互相代称 |
+| 2026-07-30 | “更均匀”拆成首尾失衡与全层 CV | Block 6/6 改善 first/last，却 6/6 恶化 CV；局部尖峰与系统性早层隆起必须分开解释 |
+| 2026-07-30 | 绝对尺度与归一化形状永久同报 | Block activation-gradient mean 约为 Baseline 54%–57%；不能把更接近 1 的首尾比自动解释为各层信号更强 |
+| 2026-07-30 | Round 05 完整重放过闸 | depth-32 Block seed-1 从零重训 8,000 steps；全部冻结字段与 model/optimizer state hashes exact，timing 仍单独报告 |
+| 2026-07-30 | K3 Round 05 梯度尺度里程碑发布 | 功能源 `f7670ef`、镜像 `20260730T022610Z-f7670ef`、OCI `sha256:02f902c1…cc09b`；复用 NAS `12010→8080`、NPM 31 / cert 41、门户 order 180，保留 Round 04 回滚点 |
+| 2026-07-30 | 尖峰集合是 targeted follow-up，不是盲发现 | layer 21–25 来自 Round 05；Round 06 固定集合后才检查时间、位置、reduction 与路径，避免用同一结果发现并确认 |
+| 2026-07-30 | “最早可见”与“物理起源”永久分账 | pre-attention input 是六个采样点中最早可见张量；更早 mixer 与反向传播已经作用，不能写成尖峰从这里注入 |
+| 2026-07-30 | 同一前向只识别 backward-rule sensitivity | learned / detached / uniform 三模式的 logits、loss、activations 与 mixer summaries exact；结果不升级为重训模型或性能结论 |
+| 2026-07-30 | value-route 降幅不写成因果贡献百分比 | 全局 uniform value-backward 让 contrast 平均下降 70.2%，只支持预注册阈值下的材料级敏感性；不声称 value 路径“解释了 70.2%” |
+| 2026-07-30 | reduction 稳健性限定在预注册家族 | element RMS、token RMS mean/median/P95 的 12/12 格通过；探索性 reduction 和其他 batch 不被纳入确认性外推 |
+| 2026-07-30 | K3 Round 06 尖峰路径里程碑发布 | 运行源 `b393780`、镜像 `20260730T042651Z-b393780`、OCI `sha256:4e4cb2e0…1a63a`；21/21 公网页面链路与全站 22 套生产 Chrome 通过，保留 Round 05 与中间 Round 06 回滚点 |
+| 2026-07-30 | 局部路径必须同时通过 sufficiency 与 restoration | groups 6+7 的 sufficiency 6/6 过 `.50`，restoration 仅 contrast 3/3 通过、peak 0/3 通过；强单侧证据不升级为 localization |
+| 2026-07-30 | 局部 score 不写成可加贡献率 | 同一 scope 在 learned 与 uniform 背景的响应不同，`S_peak > 1` 与负 interaction residual 都是非加性诊断，不是 170% 贡献或方差分解 |
+| 2026-07-30 | branch 与 output 控制保持次级证据身份 | group 7 MLP-only 的 6/6 只属于 sufficiency branch gate；group 6 为 5/6，output-only 为 0/6，都不能补救失败的双向主门 |
+| 2026-07-30 | K3 Round 07 局部路径里程碑发布 | 功能源 `dc8ec30`、镜像 `20260730T063811Z-dc8ec30`、OCI `sha256:98d44141…e71d`；21/21 公网页面链路与全站 23 套生产 Chrome 通过，保留 Round 06 回滚点 |
+| 2026-07-30 | 训练期前向干预属于架构消融 | selected uniform mixer 同时改变 train/eval forward、natural backward 与后续 updates；不能写成只改 forward 的路径因果 |
+| 2026-07-30 | Round 08 主门与质量门同时成立 | groups 6+7 六个 attenuation 格 6/6 ≥20%；三 seed ΔBPC mean `+.006578`，4/4 过闸；结论只限固定缩小协议 |
+| 2026-07-30 | non-additivity 永久保留描述身份 | `I67` 来自三套独立训练，只是 cross-run log residual，不是因果 interaction、Shapley 或贡献率 |
+| 2026-07-30 | K3 研究线在 Round 08 主动收敛 | 不启动 Round 09；公开 `A_log [128]↔[96]` 冲突继续等待官方裁决，现有五轮实证停在可复现、可回滚的稳定边界 |
+| 2026-07-30 | K3 Round 08 训练期前向里程碑发布 | 功能源 `f177fa6`、镜像 `20260730T104228Z-f177fa6`、OCI `sha256:0b5b9d9c…008bb`；21/21 公网页面链路与全站 24 套生产 Chrome 通过，保留 Round 07 回滚点 |
 
 ## 未决问题
 

README.md

@@ -19,7 +19,7 @@
 
 当前里程碑包含 17 专题学习地图、486 篇关键论文索引、Kimi K3 完整导读，
 语言模型前史、Transformer 基础、表示/位置/残差、DeepSeek 技术谱系、Scaling Laws、数据工程、长上下文、MoE、指令微调与人类偏好、推理、工具使用与长程 Agent、原生多模态、训练系统、推理服务、数值优化，以及评测与安全深度专题，
-以及 89 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
+以及 109 个覆盖核心机制的原创交互视图。K3 二轮导读以 32 张问题账、16 图 / 5 表审计、
 8 个交互实验和 100 个一手/官方节点，完整覆盖架构、预训练、后训练、系统、评测、案例与附录。
 第三轮已完成开放工件与首个真实 kernel 里程碑：固定官方模型与 FlashKDA revisions，审计 96 个 checkpoint shards、
 497,220 个 tensor entries、真实 KDA / MLA / MoE / MoonViT shapes 与小范围参数统计，并用 4 个新视图
@@ -27,6 +27,57 @@
 `sm_120a` wheel，在 RTX 5090 上完成 6/6 官方参考 exact-match 和 K3 fixed / varlen 形状计时。详见
 [K3_ARTIFACT_AUDIT.md](./research/K3_ARTIFACT_AUDIT.md) 与
 [checkpoint_probe.py](./experiments/k3/checkpoint_probe.py)、[FlashKDA probe](./experiments/k3/flashkda/)。
+第四轮不裁剪仍未解释的 `A_log [128]` 去伪造 K3 forward，而是先冻结
+`llm-atlas-k3-attnres-reduced-v1`，用相同 16-block / 32-sublayer Transformer 主干、
+相同 WikiText-2 byte windows 与三个共同初始化 seed，从零训练 Baseline、Full AttnRes
+与 Block AttnRes 共 9 个 2,000-step 正式格。最终 Full / Block 相对 Baseline 的三 seed
+平均 paired delta 分别为 `−0.01457 / −0.04247 BPC`，都满足预注册的本协议内方向支持规则；
+但核心参数梯度 RMS 的跨 block CV 为 `0.3447 / 0.5087 / 0.6306`，没有复现论文式
+梯度均匀性叙述。指定 Block / seed-1 的全新进程 2,000-step replay 在八组数值与哈希字段
+上全部 exact，计时不要求 exact。完整 9-run JSON、compact payload、复现清单、训练/
+聚合代码与五视图实验室均进入开源树。详见
+[K3_ATTNRES_REDUCED_PROTOCOL.md](./research/K3_ATTNRES_REDUCED_PROTOCOL.md)、
+[K3_ATTNRES_REDUCED_AUDIT.md](./research/K3_ATTNRES_REDUCED_AUDIT.md) 与
+[AttnRes experiment](./experiments/k3/attnres/)。
+第五轮先审计 Attention Residuals Figure 5 的公开定义边界，再冻结
+`llm-atlas-k3-attnres-gradient-scale-v1`：以 post-MLP block output activation gradient
+为公开 operationalization，把深度扩为 16 / 32 blocks、预算扩为每格 8,000 steps，
+完成 Baseline / Block × 三 seed 共 12 格、786,432,000 formal target bytes。结果把
+“更均匀”拆成两个相反方向：Block 在 6 / 6 配对中把首/末四分位失衡改善 56%–81%，
+却因中后段局部尖峰让全层 CV 在 6 / 6 配对中恶化；两个深度都按预注册联合规则判为
+mixed / inconclusive。验证 BPC 仍在 6 / 6 配对中更低，但实际 step time 约 2.6×、
+peak allocated memory 约 2.2×，不冒充同算力优势。指定 depth-32 / Block / seed-1
+从零重训完整 8,000 steps，全部冻结字段以及 model / optimizer state hashes exact。
+详见
+[K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md](./research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md)、
+[K3_ATTNRES_GRADIENT_SCALE_AUDIT.md](./research/K3_ATTNRES_GRADIENT_SCALE_AUDIT.md) 与
+[gradient experiment](./experiments/k3/attnres_gradient/)。
+第六轮冻结 `llm-atlas-k3-attnres-spike-path-v1`，严格复用 Round 05 的 depth-32 Block
+训练格，在 optimizer 之外追踪六个 checkpoint、六个 block 内张量位置与四种预注册
+reduction，并在同一前向值下比较 learned、detached-learned 与 uniform-value-backward
+三条反向规则。layer 21–25 的 post-MLP contrast 在 step 500 仍仅为三 seed 平均 `0.524×`，
+到 step 2,000 升至 `2.233×`，说明尖峰在训练中形成；六个位置全部在 3/3 seed 可见，
+四种 reduction 的 12/12 格同时通过 contrast、top-5 overlap 与 Spearman 闸门。
+切断全局 softmax/query/key 源梯度后 contrast 反而平均增加约 2.0%；把 learned
+value-backward coefficients 换成均匀系数后，contrast 平均下降 `70.2%`、peak/mean
+下降 `37.0%`，三 seed 都达到预注册材料级敏感性阈值。该结果只支持全局反向路径敏感性，
+不冒充训练变体、因果贡献百分比或局部 mixer 归因。正式三格与完整 replay 共
+262,144,000 target bytes，Round 05 等价字段与 16 组 replay 字段全部 exact。详见
+[K3_ATTNRES_SPIKE_PROTOCOL.md](./research/K3_ATTNRES_SPIKE_PROTOCOL.md)、
+[K3_ATTNRES_SPIKE_AUDIT.md](./research/K3_ATTNRES_SPIKE_AUDIT.md) 与
+[spike-path experiment](./experiments/k3/attnres_spike/)。
+第七轮冻结 `llm-atlas-k3-attnres-local-path-v1`，把 Round 06 的全局 value-route
+sensitivity 收缩为 14 个 exact selector masks，并同时从 learned 背景测 sufficiency、
+从 all-uniform 背景测 restoration。groups 6+7 的 16 个 depth mixers 在充分性方向
+两个指标 × 三 seed 的 `6 / 6` 格全部超过 `.50` global log gap；恢复性却只有 contrast
+`3 / 3` 通过，peak `0 / 3` 通过，mean peak restoration 仅 `.380`。因此冻结结论是
+`one_sided_evidence_localization_not_established`，不是“定位成功”。group 7 的
+MLP-only 次级门 6 / 6 通过，但只属于 sufficiency 探索；output-only 为 0 / 6。
+三个正式格与完整 replay 共处理 262,144,000 target bytes，Round 06 equivalence、
+14 / 14 forward identity、65-node census、selector 与 canonical replay 全部 exact。
+详见 [K3_ATTNRES_LOCAL_PATH_PROTOCOL.md](./research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md)、
+[K3_ATTNRES_LOCAL_PATH_AUDIT.md](./research/K3_ATTNRES_LOCAL_PATH_AUDIT.md) 与
+[local-path experiment](./experiments/k3/attnres_local_path/)。
 DeepSeek 八轮专题以 24 张问题账、10 次技术转向、
 22 个交互实验和 60 个一手/官方节点，串起 Dense、MoE、MLA、V3 协同、R1 与 V4；
 并固定官方 V2-Lite revision，在 RTX 5090 上连续执行 7/27 层，记录 3,240 次真实专家选择、

experiments/k3/attnres/README.md

@@ -0,0 +1,64 @@
+# Reduced Attention Residuals reproduction
+
+This directory implements protocol
+`llm-atlas-k3-attnres-reduced-v1`, frozen in
+`research/K3_ATTNRES_REDUCED_PROTOCOL.md`.
+
+The experiment is a reduced independent mechanism probe. It is not a Kimi K3
+checkpoint forward pass and not a reproduction of the paper-scale training run.
+
+## Environment
+
+The pinned execution environment used by this project is:
+
+```text
+Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
+PyTorch 2.11.0+cu128
+GPU NVIDIA GeForce RTX 5090
+CUBLAS_WORKSPACE_CONFIG=:4096:8
+```
+
+## Build the frozen dataset
+
+```bash
+python experiments/k3/attnres/build_dataset.py \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1 \
+  --manifest experiments/k3/attnres/manifest.json
+```
+
+## Run one cell
+
+```bash
+CUBLAS_WORKSPACE_CONFIG=:4096:8 \
+python experiments/k3/attnres/train.py \
+  --architecture baseline \
+  --seed 2026073001 \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1 \
+  --manifest experiments/k3/attnres/manifest.json \
+  --output /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/runs/baseline-2026073001.json
+```
+
+Raw parquet and checkpoints stay in the local cache. Frozen manifests, metric
+JSON, analyses, code, checksums, and a compact website payload enter the public
+repository.
+
+## Validate and aggregate the complete study
+
+After the nine formal cells, the preregistered replay, and the paired smoke runs
+exist in the cache:
+
+```bash
+python experiments/k3/attnres/analyze.py \
+  --formal-dir /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/formal \
+  --smoke-dir /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/smoke \
+  --replay /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/replay/block-2026073001.json \
+  --manifest experiments/k3/attnres/manifest.json \
+  --output src/data/k3-attnres-reduced.json \
+  --compact-output src/data/k3-attnres-reduced-compact.json \
+  --reproduction-output experiments/k3/attnres/reproduction.json
+```
+
+The aggregator fails closed on protocol identity, grid completeness, byte
+budget, schedule hashes, shared initialization, non-finite metrics, diagnostic
+shape, smoke mismatch, or formal replay mismatch. Timing is recorded but is not
+required to replay bit-for-bit.

experiments/k3/attnres/analyze.py

@@ -0,0 +1,536 @@
+#!/usr/bin/env python3
+"""Validate, aggregate, and compact the reduced Attention Residuals study."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import math
+import os
+import statistics
+from pathlib import Path
+from typing import Any, Iterable
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-reduced-v1"
+ARCHITECTURES = ("baseline", "full", "block")
+SEEDS = (2026073001, 2026073002, 2026073003)
+REPLAY_FIELDS = (
+    "manifest",
+    "model",
+    "optimizer",
+    "hashes",
+    "evaluations",
+    "training_history",
+    "diagnostic",
+    "environment",
+)
+SMOKE_FIELDS = REPLAY_FIELDS
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--formal-dir", type=Path, required=True)
+    parser.add_argument("--smoke-dir", type=Path, required=True)
+    parser.add_argument("--replay", type=Path, required=True)
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument("--compact-output", type=Path, required=True)
+    parser.add_argument("--reproduction-output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def read_json(path: Path) -> dict[str, Any]:
+    return json.loads(path.read_text())
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def canonical_sha256(value: Any) -> str:
+    payload = json.dumps(
+        value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+    ).encode()
+    return hashlib.sha256(payload).hexdigest()
+
+
+def write_json(path: Path, value: dict[str, Any]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(path.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    os.replace(temporary, path)
+
+
+def mean(values: Iterable[float]) -> float:
+    return statistics.fmean(values)
+
+
+def elementwise_summary(rows: list[list[float]]) -> dict[str, list[float]]:
+    length = len(rows[0])
+    if any(len(row) != length for row in rows):
+        raise ValueError("array lengths do not match")
+    return {
+        "mean": [mean(row[index] for row in rows) for index in range(length)],
+        "min": [min(row[index] for row in rows) for index in range(length)],
+        "max": [max(row[index] for row in rows) for index in range(length)],
+    }
+
+
+def coefficient_of_variation(values: list[float]) -> float:
+    average = mean(values)
+    variance = mean((value - average) ** 2 for value in values)
+    return math.sqrt(variance) / average
+
+
+def verdict(deltas: list[float]) -> dict[str, Any]:
+    average = mean(deltas)
+    if all(delta < 0 for delta in deltas) and average <= -0.010:
+        label = "directional support in this reduced protocol"
+    elif all(delta > 0 for delta in deltas) and average >= 0.010:
+        label = "directional concern in this reduced protocol"
+    else:
+        label = "inconclusive at this budget"
+    return {
+        "paired_deltas_bpc": deltas,
+        "mean_delta_bpc": average,
+        "min_delta_bpc": min(deltas),
+        "max_delta_bpc": max(deltas),
+        "same_direction": len({delta < 0 for delta in deltas}) == 1,
+        "threshold_bpc": 0.010,
+        "verdict": label,
+    }
+
+
+def average_depth_weights(runs: list[dict[str, Any]]) -> dict[str, Any]:
+    rows_by_run = [run["diagnostic"]["depth_weights"] for run in runs]
+    layer_count = len(rows_by_run[0])
+    if any(len(rows) != layer_count for rows in rows_by_run):
+        raise ValueError("depth-weight layer counts differ")
+    rows = []
+    max_sources = 0
+    for layer in range(layer_count):
+        source_count = rows_by_run[0][layer]["sources"]
+        if any(rows[layer]["sources"] != source_count for rows in rows_by_run):
+            raise ValueError("source count differs across seeds")
+        weights = [
+            mean(rows_by_run[seed_index][layer]["mean_weights"][source]
+                 for seed_index in range(len(runs)))
+            for source in range(source_count)
+        ]
+        entropies = [
+            rows_by_run[seed_index][layer]["entropy_mean"]
+            for seed_index in range(len(runs))
+        ]
+        rows.append(
+            {
+                "sublayer": layer + 1,
+                "sources": source_count,
+                "mean_weights": weights,
+                "entropy_mean": mean(entropies),
+                "entropy_min": min(entropies),
+                "entropy_max": max(entropies),
+            }
+        )
+        max_sources = max(max_sources, source_count)
+
+    output_rows = [run["diagnostic"]["output_weights"] for run in runs]
+    output_source_count = output_rows[0]["sources"]
+    output_weights = [
+        mean(row["mean_weights"][source] for row in output_rows)
+        for source in range(output_source_count)
+    ]
+    return {
+        "rows": rows,
+        "max_sources": max_sources,
+        "output": {
+            "sources": output_source_count,
+            "mean_weights": output_weights,
+            "entropy_mean": mean(row["entropy_mean"] for row in output_rows),
+            "entropy_min": min(row["entropy_mean"] for row in output_rows),
+            "entropy_max": max(row["entropy_mean"] for row in output_rows),
+        },
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    manifest = read_json(args.manifest)
+    if manifest["protocol_id"] != PROTOCOL_ID:
+        raise ValueError("manifest protocol mismatch")
+
+    runs: dict[tuple[str, int], dict[str, Any]] = {}
+    formal_file_hashes: dict[str, str] = {}
+    for seed in SEEDS:
+        for architecture in ARCHITECTURES:
+            path = args.formal_dir / f"{architecture}-{seed}.json"
+            run = read_json(path)
+            if run["protocol_id"] != PROTOCOL_ID:
+                raise ValueError(f"protocol mismatch: {path}")
+            if run["run_kind"] != "formal":
+                raise ValueError(f"not a formal run: {path}")
+            if run["architecture"] != architecture or run["seed"] != seed:
+                raise ValueError(f"cell identity mismatch: {path}")
+            if run["steps"] != 2000 or run["batch_size"] != 32:
+                raise ValueError(f"formal budget mismatch: {path}")
+            if run["target_bytes_seen"] != 16_384_000:
+                raise ValueError(f"target byte count mismatch: {path}")
+            if run["manifest"]["file_sha256"] != file_sha256(args.manifest):
+                raise ValueError(f"manifest file hash mismatch: {path}")
+            if run["manifest"]["formal_schedule_sha256"] != (
+                manifest["windows"]["formal_schedule_sha256"]
+            ):
+                raise ValueError(f"schedule mismatch: {path}")
+            if run["evaluations"][-1]["step"] != 2000:
+                raise ValueError(f"missing final evaluation: {path}")
+            if any(
+                not math.isfinite(value)
+                for evaluation in run["evaluations"]
+                for value in (
+                    evaluation["cross_entropy_nats"],
+                    evaluation["bits_per_byte"],
+                )
+            ):
+                raise ValueError(f"non-finite evaluation: {path}")
+            if len(run["diagnostic"]["layer_input_rms"]) != 32:
+                raise ValueError(f"diagnostic depth mismatch: {path}")
+            if len(run["diagnostic"]["core_parameter_grad_rms_by_block"]) != 16:
+                raise ValueError(f"gradient depth mismatch: {path}")
+            runs[(architecture, seed)] = run
+            formal_file_hashes[path.name] = file_sha256(path)
+
+    common_initial_exact = {}
+    for seed in SEEDS:
+        hashes = {
+            architecture: runs[(architecture, seed)]["hashes"][
+                "initial_common_parameters"
+            ]
+            for architecture in ARCHITECTURES
+        }
+        common_initial_exact[str(seed)] = {
+            "hashes": hashes,
+            "exact": len(set(hashes.values())) == 1,
+        }
+        if not common_initial_exact[str(seed)]["exact"]:
+            raise ValueError(f"common initialization mismatch for seed {seed}")
+
+    by_seed = []
+    for seed in SEEDS:
+        values = {
+            architecture: runs[(architecture, seed)]["evaluations"][-1][
+                "bits_per_byte"
+            ]
+            for architecture in ARCHITECTURES
+        }
+        by_seed.append(
+            {
+                "seed": seed,
+                "final_bpc": values,
+                "full_minus_baseline": values["full"] - values["baseline"],
+                "block_minus_baseline": values["block"] - values["baseline"],
+                "block_minus_full": values["block"] - values["full"],
+            }
+        )
+
+    final = {
+        "by_seed": by_seed,
+        "means": {
+            architecture: mean(
+                runs[(architecture, seed)]["evaluations"][-1]["bits_per_byte"]
+                for seed in SEEDS
+            )
+            for architecture in ARCHITECTURES
+        },
+        "full_contrast": verdict(
+            [row["full_minus_baseline"] for row in by_seed]
+        ),
+        "block_contrast": verdict(
+            [row["block_minus_baseline"] for row in by_seed]
+        ),
+        "block_minus_full": {
+            "paired_deltas_bpc": [row["block_minus_full"] for row in by_seed],
+            "mean_delta_bpc": mean(row["block_minus_full"] for row in by_seed),
+        },
+    }
+
+    evaluation_steps = [
+        evaluation["step"] for evaluation in runs[("baseline", SEEDS[0])]["evaluations"]
+    ]
+    curves = {}
+    for architecture in ARCHITECTURES:
+        curve = []
+        for index, step in enumerate(evaluation_steps):
+            values = [
+                runs[(architecture, seed)]["evaluations"][index]["bits_per_byte"]
+                for seed in SEEDS
+            ]
+            if any(
+                runs[(architecture, seed)]["evaluations"][index]["step"] != step
+                for seed in SEEDS
+            ):
+                raise ValueError("evaluation step mismatch")
+            curve.append(
+                {
+                    "step": step,
+                    "mean_bpc": mean(values),
+                    "min_bpc": min(values),
+                    "max_bpc": max(values),
+                    "by_seed": values,
+                }
+            )
+        curves[architecture] = curve
+
+    timing = {}
+    for architecture in ARCHITECTURES:
+        cells = [runs[(architecture, seed)]["timing"] for seed in SEEDS]
+        timing[architecture] = {
+            "mean_step_ms": mean(cell["mean_ms"] for cell in cells),
+            "median_step_ms": mean(cell["median_ms"] for cell in cells),
+            "p95_step_ms": mean(cell["p95_ms"] for cell in cells),
+            "mean_peak_allocated_bytes": mean(
+                cell["peak_allocated_bytes"] for cell in cells
+            ),
+            "mean_peak_reserved_bytes": mean(
+                cell["peak_reserved_bytes"] for cell in cells
+            ),
+            "by_seed": cells,
+        }
+    timing["relative_to_baseline"] = {
+        architecture: {
+            "step_time_ratio": timing[architecture]["mean_step_ms"]
+            / timing["baseline"]["mean_step_ms"],
+            "allocated_memory_ratio": timing[architecture][
+                "mean_peak_allocated_bytes"
+            ]
+            / timing["baseline"]["mean_peak_allocated_bytes"],
+        }
+        for architecture in ("full", "block")
+    }
+
+    parameters = {
+        architecture: runs[(architecture, SEEDS[0])]["model"]["parameters"]
+        for architecture in ARCHITECTURES
+    }
+    parameters["mixer_overhead_fraction_of_baseline"] = (
+        parameters["full"]["mixer"] / parameters["baseline"]["total"]
+    )
+
+    traces = {}
+    gradients = {}
+    for architecture in ARCHITECTURES:
+        architecture_runs = [runs[(architecture, seed)] for seed in SEEDS]
+        traces[architecture] = {
+            key: elementwise_summary(
+                [run["diagnostic"][key] for run in architecture_runs]
+            )
+            for key in (
+                "layer_input_rms",
+                "branch_output_rms",
+                "stream_state_rms",
+            )
+        }
+        gradient_rows = [
+            run["diagnostic"]["core_parameter_grad_rms_by_block"]
+            for run in architecture_runs
+        ]
+        gradients[architecture] = {
+            "by_block": elementwise_summary(gradient_rows),
+            "cv_by_seed": [
+                coefficient_of_variation(row) for row in gradient_rows
+            ],
+            "mean_cv": mean(coefficient_of_variation(row) for row in gradient_rows),
+            "first_last_ratio_by_seed": [
+                row[0] / row[-1] for row in gradient_rows
+            ],
+            "mean_first_last_ratio": mean(row[0] / row[-1] for row in gradient_rows),
+        }
+
+    mixers = {
+        architecture: average_depth_weights(
+            [runs[(architecture, seed)] for seed in SEEDS]
+        )
+        for architecture in ("full", "block")
+    }
+
+    full_branch = traces["full"]["branch_output_rms"]["mean"]
+    largest_index = max(range(len(full_branch)), key=full_branch.__getitem__)
+    # Full output source 0 is the embedding; branch l is source l+1.
+    largest_source_weight = mixers["full"]["output"]["mean_weights"][
+        largest_index + 1
+    ]
+    uniform_output_weight = 1 / mixers["full"]["output"]["sources"]
+    posthoc = {
+        "label": "post-hoc descriptive callout; not a preregistered endpoint",
+        "largest_full_branch_sublayer": largest_index + 1,
+        "largest_full_branch_rms": full_branch[largest_index],
+        "corresponding_final_output_weight": largest_source_weight,
+        "uniform_final_output_weight": uniform_output_weight,
+        "weight_over_uniform": largest_source_weight / uniform_output_weight,
+    }
+
+    replay = read_json(args.replay)
+    formal_replay_source = runs[("block", 2026073001)]
+    replay_exact = {
+        field: formal_replay_source[field] == replay[field]
+        for field in REPLAY_FIELDS
+    }
+    if not all(replay_exact.values()):
+        raise ValueError(f"formal replay mismatch: {replay_exact}")
+
+    smoke = {}
+    for architecture in ARCHITECTURES:
+        first_path = args.smoke_dir / f"{architecture}-2026073001-a.json"
+        second_path = args.smoke_dir / f"{architecture}-2026073001-b.json"
+        first = read_json(first_path)
+        second = read_json(second_path)
+        exact = {field: first[field] == second[field] for field in SMOKE_FIELDS}
+        if not all(exact.values()):
+            raise ValueError(f"smoke mismatch for {architecture}: {exact}")
+        smoke[architecture] = {
+            "fields": exact,
+            "all_exact": True,
+            "first_sha256": file_sha256(first_path),
+            "second_sha256": file_sha256(second_path),
+        }
+
+    reproduction = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "manifest_sha256": file_sha256(args.manifest),
+        "formal_files": formal_file_hashes,
+        "common_initial_parameters": common_initial_exact,
+        "smoke": smoke,
+        "formal_replay": {
+            "architecture": "block",
+            "seed": 2026073001,
+            "fields": replay_exact,
+            "all_numeric_and_hash_fields_exact": all(replay_exact.values()),
+            "timing_exact_required": False,
+            "timing_exact_observed": formal_replay_source["timing"] == replay["timing"],
+            "formal_file_sha256": formal_file_hashes[
+                "block-2026073001.json"
+            ],
+            "replay_file_sha256": file_sha256(args.replay),
+        },
+    }
+    reproduction["canonical_sha256_without_self"] = canonical_sha256(reproduction)
+
+    analysis = {
+        "final_validation": final,
+        "evaluation_curves": curves,
+        "timing": timing,
+        "parameters": parameters,
+        "traces": traces,
+        "gradients": gradients,
+        "mixers": mixers,
+        "posthoc": posthoc,
+        "interpretation": {
+            "primary": (
+                "Both Full and Block AttnRes satisfy the preregistered "
+                "directional-support rule in this reduced protocol."
+            ),
+            "bounded_depth_pattern": (
+                "Block partial-state RMS resets every four residual sublayers; "
+                "the complete 32-point vectors are reported."
+            ),
+            "gradient_boundary": (
+                "The preregistered core-parameter gradient RMS is not more "
+                "uniform for AttnRes here; this metric and scale do not reproduce "
+                "the paper's large-model gradient-magnitude result."
+            ),
+            "scope": (
+                "Reduced byte-level WikiText-2 mechanism probe; not a K3 "
+                "checkpoint run, paper-scale reproduction, benchmark, or "
+                "same-FLOP comparison."
+            ),
+        },
+    }
+
+    raw = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "manifest": manifest,
+        "provenance": {
+            "manifest_file_sha256": file_sha256(args.manifest),
+            "formal_file_sha256": formal_file_hashes,
+            "reproduction_sha256": reproduction[
+                "canonical_sha256_without_self"
+            ],
+        },
+        "formal_runs": [
+            runs[(architecture, seed)]
+            for seed in SEEDS
+            for architecture in ARCHITECTURES
+        ],
+        "analysis": analysis,
+        "reproduction": reproduction,
+    }
+    raw["canonical_sha256_without_self"] = canonical_sha256(raw)
+
+    compact = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "dataset": {
+            "repository": manifest["dataset"]["repository"],
+            "revision": manifest["dataset"]["revision"],
+            "train_bytes": manifest["dataset"]["splits"]["train"][
+                "concatenated_bytes"
+            ],
+            "schedule_sha256": manifest["windows"]["formal_schedule_sha256"],
+            "validation_sha256": manifest["windows"][
+                "validation_tensor_sha256"
+            ],
+        },
+        "grid": {
+            "architectures": list(ARCHITECTURES),
+            "seeds": list(SEEDS),
+            "runs": 9,
+            "steps_per_run": 2000,
+            "target_bytes_per_run": 16_384_000,
+            "target_bytes_total": 9 * 16_384_000,
+        },
+        "final_validation": final,
+        "evaluation_curves": curves,
+        "timing": timing,
+        "parameters": parameters,
+        "traces": traces,
+        "gradients": gradients,
+        "mixers": mixers,
+        "posthoc": posthoc,
+        "interpretation": analysis["interpretation"],
+        "reproduction": reproduction,
+        "source_sha256": raw["canonical_sha256_without_self"],
+    }
+    compact["canonical_sha256_without_self"] = canonical_sha256(compact)
+
+    write_json(args.reproduction_output, reproduction)
+    write_json(args.output, raw)
+    write_json(args.compact_output, compact)
+    print(
+        json.dumps(
+            {
+                "output": str(args.output),
+                "compact_output": str(args.compact_output),
+                "reproduction_output": str(args.reproduction_output),
+                "raw_sha256": file_sha256(args.output),
+                "compact_sha256": file_sha256(args.compact_output),
+                "reproduction_sha256": file_sha256(args.reproduction_output),
+                "full": final["full_contrast"],
+                "block": final["block_contrast"],
+                "formal_replay": reproduction["formal_replay"],
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres/build_dataset.py

@@ -0,0 +1,192 @@
+#!/usr/bin/env python3
+"""Download and freeze the byte-level WikiText-2 corpus for the AttnRes study."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import os
+import urllib.request
+from pathlib import Path
+from typing import Any
+
+import pyarrow.parquet as pq
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-reduced-v1"
+DATASET_REPO = "Salesforce/wikitext"
+DATASET_REVISION = "b08601e04326c79dfdd32d625aee71d232d685c3"
+DATASET_VARIANT = "wikitext-2-raw-v1"
+SPLITS = ("train", "validation", "test")
+SEEDS = (2026073001, 2026073002, 2026073003)
+CONTEXT = 256
+FORMAL_STEPS = 2000
+FORMAL_BATCH = 32
+VALIDATION_WINDOWS = 64
+DIAGNOSTIC_WINDOWS = 16
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--cache-dir", type=Path, required=True)
+    parser.add_argument("--manifest", type=Path, required=True)
+    return parser.parse_args()
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def bytes_sha256(payload: bytes) -> str:
+    return hashlib.sha256(payload).hexdigest()
+
+
+def atomic_json(path: Path, value: dict[str, Any]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(path.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    os.replace(temporary, path)
+
+
+def download(url: str, path: Path) -> None:
+    if path.exists():
+        return
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(path.suffix + ".part")
+    request = urllib.request.Request(
+        url,
+        headers={"User-Agent": "llm-atlas-k3-attnres-reduced/1.0"},
+    )
+    with urllib.request.urlopen(request, timeout=120) as response:
+        with temporary.open("wb") as output:
+            while block := response.read(1024 * 1024):
+                output.write(block)
+    os.replace(temporary, path)
+
+
+def window_start(label: str, index: int, corpus_length: int, seed: int | None = None) -> int:
+    fields = [PROTOCOL_ID, label]
+    if seed is not None:
+        fields.append(str(seed))
+    fields.append(str(index))
+    payload = "\0".join(fields).encode()
+    value = int.from_bytes(hashlib.sha256(payload).digest()[:8], "big")
+    return value % (corpus_length - (CONTEXT + 1))
+
+
+def train_window_start(seed: int, step: int, row: int, corpus_length: int) -> int:
+    payload = "\0".join(
+        [PROTOCOL_ID, "train-window", str(seed), str(step), str(row)]
+    ).encode()
+    value = int.from_bytes(hashlib.sha256(payload).digest()[:8], "big")
+    return value % (corpus_length - (CONTEXT + 1))
+
+
+def concatenate_split(parquet_path: Path) -> tuple[bytes, int]:
+    table = pq.read_table(parquet_path, columns=["text"])
+    rows = table.column("text").to_pylist()
+    payload = b"".join(((row or "") + "\n").encode("utf-8") for row in rows)
+    return payload, len(rows)
+
+
+def main() -> None:
+    args = parse_args()
+    args.cache_dir.mkdir(parents=True, exist_ok=True)
+
+    split_manifest: dict[str, Any] = {}
+    split_bytes: dict[str, bytes] = {}
+    for split in SPLITS:
+        relative = f"{DATASET_VARIANT}/{split}-00000-of-00001.parquet"
+        url = (
+            f"https://huggingface.co/datasets/{DATASET_REPO}/resolve/"
+            f"{DATASET_REVISION}/{relative}"
+        )
+        parquet_path = args.cache_dir / f"{split}.parquet"
+        download(url, parquet_path)
+        payload, rows = concatenate_split(parquet_path)
+        binary_path = args.cache_dir / f"{split}.bin"
+        if not binary_path.exists() or binary_path.read_bytes() != payload:
+            temporary = binary_path.with_suffix(".bin.tmp")
+            temporary.write_bytes(payload)
+            os.replace(temporary, binary_path)
+        split_bytes[split] = payload
+        split_manifest[split] = {
+            "source_path": relative,
+            "source_url": url,
+            "parquet_bytes": parquet_path.stat().st_size,
+            "parquet_sha256": file_sha256(parquet_path),
+            "rows": rows,
+            "concatenated_bytes": len(payload),
+            "concatenated_sha256": bytes_sha256(payload),
+            "binary_path": str(binary_path),
+            "binary_sha256": file_sha256(binary_path),
+        }
+
+    train = split_bytes["train"]
+    validation = split_bytes["validation"]
+
+    schedule_digest = hashlib.sha256()
+    schedule_cells = 0
+    for seed in SEEDS:
+        for step in range(1, FORMAL_STEPS + 1):
+            for row in range(FORMAL_BATCH):
+                start = train_window_start(seed, step, row, len(train))
+                schedule_digest.update(start.to_bytes(8, "big"))
+                schedule_cells += 1
+
+    validation_starts = [
+        window_start("validation-window", index, len(validation))
+        for index in range(VALIDATION_WINDOWS)
+    ]
+    diagnostic_starts = [
+        window_start("diagnostic-window", index, len(validation))
+        for index in range(DIAGNOSTIC_WINDOWS)
+    ]
+
+    def tensor_hash(starts: list[int]) -> str:
+        digest = hashlib.sha256()
+        for start in starts:
+            digest.update(validation[start : start + CONTEXT + 1])
+        return digest.hexdigest()
+
+    manifest = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "status": "frozen-before-model-output",
+        "dataset": {
+            "repository": DATASET_REPO,
+            "revision": DATASET_REVISION,
+            "variant": DATASET_VARIANT,
+            "preprocessing": (
+                "parquet row order; (text or empty string) + LF; UTF-8; "
+                "no normalization; vocabulary is raw bytes 0..255"
+            ),
+            "splits": split_manifest,
+        },
+        "windows": {
+            "context": CONTEXT,
+            "target_bytes_per_window": CONTEXT,
+            "seeds": list(SEEDS),
+            "formal_steps": FORMAL_STEPS,
+            "formal_batch": FORMAL_BATCH,
+            "formal_schedule_cells": schedule_cells,
+            "formal_schedule_sha256": schedule_digest.hexdigest(),
+            "validation_starts": validation_starts,
+            "validation_tensor_sha256": tensor_hash(validation_starts),
+            "diagnostic_starts": diagnostic_starts,
+            "diagnostic_tensor_sha256": tensor_hash(diagnostic_starts),
+        },
+    }
+    atomic_json(args.manifest, manifest)
+    print(json.dumps(manifest, ensure_ascii=False, indent=2))
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres/manifest.json

@@ -0,0 +1,145 @@
+{
+  "dataset": {
+    "preprocessing": "parquet row order; (text or empty string) + LF; UTF-8; no normalization; vocabulary is raw bytes 0..255",
+    "repository": "Salesforce/wikitext",
+    "revision": "b08601e04326c79dfdd32d625aee71d232d685c3",
+    "splits": {
+      "test": {
+        "binary_path": "/home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/test.bin",
+        "binary_sha256": "bfe9eb16ab9987fb88bde4ea9a30a00f2a45db01dfc14bad78d05325789c4f12",
+        "concatenated_bytes": 1292014,
+        "concatenated_sha256": "bfe9eb16ab9987fb88bde4ea9a30a00f2a45db01dfc14bad78d05325789c4f12",
+        "parquet_bytes": 732610,
+        "parquet_sha256": "5f1bea067869d04849c0f975a2b29c4ff47d867f484f5010ea5e861eab246d91",
+        "rows": 4358,
+        "source_path": "wikitext-2-raw-v1/test-00000-of-00001.parquet",
+        "source_url": "https://huggingface.co/datasets/Salesforce/wikitext/resolve/b08601e04326c79dfdd32d625aee71d232d685c3/wikitext-2-raw-v1/test-00000-of-00001.parquet"
+      },
+      "train": {
+        "binary_path": "/home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/train.bin",
+        "binary_sha256": "0ca7d3e74dbe44564ea5942b85232f1bbcb525c9cd481cd5d28a87ee90e7e9b4",
+        "concatenated_bytes": 10951563,
+        "concatenated_sha256": "0ca7d3e74dbe44564ea5942b85232f1bbcb525c9cd481cd5d28a87ee90e7e9b4",
+        "parquet_bytes": 6357543,
+        "parquet_sha256": "e83889baabc497075506f91975be5fac0d45c5290b6b20582c8cd1e853d0c9f7",
+        "rows": 36718,
+        "source_path": "wikitext-2-raw-v1/train-00000-of-00001.parquet",
+        "source_url": "https://huggingface.co/datasets/Salesforce/wikitext/resolve/b08601e04326c79dfdd32d625aee71d232d685c3/wikitext-2-raw-v1/train-00000-of-00001.parquet"
+      },
+      "validation": {
+        "binary_path": "/home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/validation.bin",
+        "binary_sha256": "a42356f6a8ff1d25daf25ec9db49e10a537c265581b61c74604bb63231dee719",
+        "concatenated_bytes": 1148008,
+        "concatenated_sha256": "a42356f6a8ff1d25daf25ec9db49e10a537c265581b61c74604bb63231dee719",
+        "parquet_bytes": 657209,
+        "parquet_sha256": "204929b7ff9d6184953f867dedb860e40aa69c078fc1e54b3baaa8fb28511c4c",
+        "rows": 3760,
+        "source_path": "wikitext-2-raw-v1/validation-00000-of-00001.parquet",
+        "source_url": "https://huggingface.co/datasets/Salesforce/wikitext/resolve/b08601e04326c79dfdd32d625aee71d232d685c3/wikitext-2-raw-v1/validation-00000-of-00001.parquet"
+      }
+    },
+    "variant": "wikitext-2-raw-v1"
+  },
+  "protocol_id": "llm-atlas-k3-attnres-reduced-v1",
+  "schema_version": 1,
+  "status": "frozen-before-model-output",
+  "windows": {
+    "context": 256,
+    "diagnostic_starts": [
+      611936,
+      81370,
+      284950,
+      884010,
+      436549,
+      319425,
+      664210,
+      2929,
+      1120568,
+      1130204,
+      567111,
+      471404,
+      178798,
+      773999,
+      8739,
+      567335
+    ],
+    "diagnostic_tensor_sha256": "d970af9b0c656c9826f369b5fe6e3869a6f6cfeccfa5a922fe94ed1d24b86818",
+    "formal_batch": 32,
+    "formal_schedule_cells": 192000,
+    "formal_schedule_sha256": "81521a70ec61f3717968f160cb711e50c5f52a665a6961538d339360cb695f48",
+    "formal_steps": 2000,
+    "seeds": [
+      2026073001,
+      2026073002,
+      2026073003
+    ],
+    "target_bytes_per_window": 256,
+    "validation_starts": [
+      990074,
+      30961,
+      68731,
+      926982,
+      644250,
+      11726,
+      878377,
+      825572,
+      874166,
+      182830,
+      392129,
+      822765,
+      583693,
+      287605,
+      117755,
+      463343,
+      1005482,
+      27886,
+      1107742,
+      1025132,
+      190867,
+      563499,
+      375410,
+      611922,
+      576979,
+      959290,
+      139748,
+      855267,
+      439281,
+      268378,
+      205337,
+      638978,
+      916468,
+      1059268,
+      1098648,
+      863779,
+      1028257,
+      340396,
+      547037,
+      682617,
+      217662,
+      744004,
+      288924,
+      890860,
+      856857,
+      312424,
+      12454,
+      1135148,
+      385677,
+      954097,
+      490766,
+      164873,
+      620587,
+      329982,
+      1014567,
+      236028,
+      356220,
+      603142,
+      879272,
+      511832,
+      472270,
+      823189,
+      653568,
+      159430
+    ],
+    "validation_tensor_sha256": "5f71fda757fc75010ed16e7636bc394c69f55b34a3713b3b5a7ef8e03eae3c20"
+  }
+}

experiments/k3/attnres/reproduction.json

@@ -0,0 +1,109 @@
+{
+  "canonical_sha256_without_self": "89a127e625ccefbd9b749e6ffca4ba72868e217e0c7a24773024bd2fafde427a",
+  "common_initial_parameters": {
+    "2026073001": {
+      "exact": true,
+      "hashes": {
+        "baseline": "af2724a1c34bcfd61d8a8bef402246430898c815e56b6e5c5257949a4eb0e7b1",
+        "block": "af2724a1c34bcfd61d8a8bef402246430898c815e56b6e5c5257949a4eb0e7b1",
+        "full": "af2724a1c34bcfd61d8a8bef402246430898c815e56b6e5c5257949a4eb0e7b1"
+      }
+    },
+    "2026073002": {
+      "exact": true,
+      "hashes": {
+        "baseline": "9fd4f04212ab5cea822f469902d8e80ecc368da329f3c20abacfe6b7a50ed523",
+        "block": "9fd4f04212ab5cea822f469902d8e80ecc368da329f3c20abacfe6b7a50ed523",
+        "full": "9fd4f04212ab5cea822f469902d8e80ecc368da329f3c20abacfe6b7a50ed523"
+      }
+    },
+    "2026073003": {
+      "exact": true,
+      "hashes": {
+        "baseline": "7a565cd353efdb3b95e9b8b1844581082c029991ea18d438b4b18566970c8c61",
+        "block": "7a565cd353efdb3b95e9b8b1844581082c029991ea18d438b4b18566970c8c61",
+        "full": "7a565cd353efdb3b95e9b8b1844581082c029991ea18d438b4b18566970c8c61"
+      }
+    }
+  },
+  "formal_files": {
+    "baseline-2026073001.json": "f5cddecace6a70ee3824f272811d2a6336ab6dc24ad1100b18292f4413692e9a",
+    "baseline-2026073002.json": "4c3b9e1ea2ed6fd3078f1212984327364832a1146bfcbdb3a6f2e61b589bfaa8",
+    "baseline-2026073003.json": "e5ce5a8944dad0c4811f8bf19b4cfec8ce4a9283858fd59cd739dbc37626707b",
+    "block-2026073001.json": "5df870369d9a86ccb4ba4191fbd1d6f3642893dd47a60f8f6d1143006bdfbdaf",
+    "block-2026073002.json": "be1f109a630e27c8469438e33b53806cdaecac2f712af4f885da83f00ef6843b",
+    "block-2026073003.json": "949ec51ca3a219a11f260171da01f737296241536f5e139cf07d444c27efba92",
+    "full-2026073001.json": "648cb25ea98da1868779733da155275e9a16aad5efbbeeced7812c19d75817d5",
+    "full-2026073002.json": "1fa715bbd81a3a04a5aa0ba0fc27feb8883c44a01e9f28e281a309a394447979",
+    "full-2026073003.json": "4e34fa35bbfec460cabc94bce7a08f44fc188d908705ae2e256500f00813b4e3"
+  },
+  "formal_replay": {
+    "all_numeric_and_hash_fields_exact": true,
+    "architecture": "block",
+    "fields": {
+      "diagnostic": true,
+      "environment": true,
+      "evaluations": true,
+      "hashes": true,
+      "manifest": true,
+      "model": true,
+      "optimizer": true,
+      "training_history": true
+    },
+    "formal_file_sha256": "5df870369d9a86ccb4ba4191fbd1d6f3642893dd47a60f8f6d1143006bdfbdaf",
+    "replay_file_sha256": "e74d3323e5fe31378bb8aad7a8efa2fb91995cd7224c158cf03006466cdea2a7",
+    "seed": 2026073001,
+    "timing_exact_observed": false,
+    "timing_exact_required": false
+  },
+  "manifest_sha256": "9778ade5b1c9dd7676d2cdc52b4e4e7ff5ae513cb56c667974e2422702f9dc2b",
+  "protocol_id": "llm-atlas-k3-attnres-reduced-v1",
+  "schema_version": 1,
+  "smoke": {
+    "baseline": {
+      "all_exact": true,
+      "fields": {
+        "diagnostic": true,
+        "environment": true,
+        "evaluations": true,
+        "hashes": true,
+        "manifest": true,
+        "model": true,
+        "optimizer": true,
+        "training_history": true
+      },
+      "first_sha256": "fd8b14f70a6a14978e65f9899b694164ba91753bf82eead436a40837c251e82c",
+      "second_sha256": "fd8b14f70a6a14978e65f9899b694164ba91753bf82eead436a40837c251e82c"
+    },
+    "block": {
+      "all_exact": true,
+      "fields": {
+        "diagnostic": true,
+        "environment": true,
+        "evaluations": true,
+        "hashes": true,
+        "manifest": true,
+        "model": true,
+        "optimizer": true,
+        "training_history": true
+      },
+      "first_sha256": "980a4a534e458199d95b5864b008a81f51b565e05a7b2f24a644b36d2134eccc",
+      "second_sha256": "980a4a534e458199d95b5864b008a81f51b565e05a7b2f24a644b36d2134eccc"
+    },
+    "full": {
+      "all_exact": true,
+      "fields": {
+        "diagnostic": true,
+        "environment": true,
+        "evaluations": true,
+        "hashes": true,
+        "manifest": true,
+        "model": true,
+        "optimizer": true,
+        "training_history": true
+      },
+      "first_sha256": "c6e32b737c7bf7fdc9b647650ebd7c9d2f9f8550f66a5b84f168f16fa7d2eacb",
+      "second_sha256": "c6e32b737c7bf7fdc9b647650ebd7c9d2f9f8550f66a5b84f168f16fa7d2eacb"
+    }
+  }
+}

experiments/k3/attnres/train.py

@@ -0,0 +1,717 @@
+#!/usr/bin/env python3
+"""Train one frozen residual variant for the reduced Attention Residuals study."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import math
+import os
+import platform
+import statistics
+import time
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Iterable
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-reduced-v1"
+ARCHITECTURES = ("baseline", "full", "block")
+EXPECTED_SEEDS = (2026073001, 2026073002, 2026073003)
+EVAL_STEPS = (0, 100, 250, 500, 1000, 1500, 2000)
+VOCABULARY = 256
+CONTEXT = 256
+LAYERS = 16
+SUBLAYERS = LAYERS * 2
+BLOCKS = 8
+SUBLAYERS_PER_BLOCK = SUBLAYERS // BLOCKS
+D_MODEL = 192
+HEADS = 6
+D_HEAD = D_MODEL // HEADS
+D_FF = 768
+RMS_EPS = 1e-6
+PEAK_LR = 3e-4
+MIN_LR = 3e-5
+WARMUP_STEPS = 100
+WEIGHT_DECAY = 0.1
+BETAS = (0.9, 0.95)
+ADAM_EPS = 1e-8
+GRAD_CLIP = 1.0
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--architecture", choices=ARCHITECTURES, required=True)
+    parser.add_argument("--seed", type=int, required=True)
+    parser.add_argument("--steps", type=int, default=2000)
+    parser.add_argument("--batch-size", type=int, default=32)
+    parser.add_argument("--cache-dir", type=Path, required=True)
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument("--validation-windows", type=int, default=64)
+    parser.add_argument("--diagnostic-windows", type=int, default=16)
+    parser.add_argument("--eval-batch-size", type=int, default=8)
+    parser.add_argument("--timing-warmup", type=int, default=20)
+    parser.add_argument("--run-kind", choices=("smoke", "formal", "replay"), default="formal")
+    return parser.parse_args()
+
+
+def configure_determinism(seed: int) -> None:
+    if os.environ.get("CUBLAS_WORKSPACE_CONFIG") != ":4096:8":
+        raise RuntimeError("CUBLAS_WORKSPACE_CONFIG must be :4096:8 before Python starts")
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed_all(seed)
+    torch.use_deterministic_algorithms(True)
+    torch.backends.cudnn.benchmark = False
+    torch.backends.cudnn.deterministic = True
+    torch.backends.cuda.matmul.allow_tf32 = False
+    torch.backends.cudnn.allow_tf32 = False
+    torch.set_float32_matmul_precision("highest")
+
+
+def canonical_json_sha256(value: Any) -> str:
+    payload = json.dumps(
+        value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+    ).encode()
+    return hashlib.sha256(payload).hexdigest()
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def tensor_bytes(tensor: torch.Tensor) -> bytes:
+    value = tensor.detach().cpu().contiguous()
+    header = f"{value.dtype}|{tuple(value.shape)}|".encode()
+    return header + value.view(torch.uint8).numpy().tobytes()
+
+
+def state_hash(
+    model: nn.Module,
+    *,
+    include_mixers: bool | None,
+) -> str:
+    digest = hashlib.sha256()
+    for name, tensor in sorted(model.state_dict().items()):
+        is_mixer = name.startswith("mixers.") or name.startswith("output_mixer.")
+        if include_mixers is not None and is_mixer != include_mixers:
+            continue
+        digest.update(name.encode())
+        digest.update(b"\0")
+        digest.update(tensor_bytes(tensor))
+    return digest.hexdigest()
+
+
+def window_start(seed: int, step: int, row: int, corpus_length: int) -> int:
+    payload = "\0".join(
+        [PROTOCOL_ID, "train-window", str(seed), str(step), str(row)]
+    ).encode()
+    value = int.from_bytes(hashlib.sha256(payload).digest()[:8], "big")
+    return value % (corpus_length - (CONTEXT + 1))
+
+
+class ByteCorpus:
+    def __init__(self, cache_dir: Path, manifest: dict[str, Any], device: torch.device):
+        self.device = device
+        self.train = np.memmap(cache_dir / "train.bin", dtype=np.uint8, mode="r")
+        self.validation = np.memmap(
+            cache_dir / "validation.bin", dtype=np.uint8, mode="r"
+        )
+        self.validation_starts = manifest["windows"]["validation_starts"]
+        self.diagnostic_starts = manifest["windows"]["diagnostic_starts"]
+
+    def training_batch(
+        self, seed: int, step: int, batch_size: int
+    ) -> tuple[torch.Tensor, torch.Tensor]:
+        rows = np.empty((batch_size, CONTEXT + 1), dtype=np.int64)
+        for row in range(batch_size):
+            start = window_start(seed, step, row, len(self.train))
+            rows[row] = self.train[start : start + CONTEXT + 1]
+        tensor = torch.from_numpy(rows).to(self.device, non_blocking=False)
+        return tensor[:, :-1], tensor[:, 1:]
+
+    def fixed_batch(
+        self, starts: list[int], begin: int, end: int
+    ) -> tuple[torch.Tensor, torch.Tensor]:
+        chosen = starts[begin:end]
+        rows = np.empty((len(chosen), CONTEXT + 1), dtype=np.int64)
+        for row, start in enumerate(chosen):
+            rows[row] = self.validation[start : start + CONTEXT + 1]
+        tensor = torch.from_numpy(rows).to(self.device, non_blocking=False)
+        return tensor[:, :-1], tensor[:, 1:]
+
+
+class RMSNorm(nn.Module):
+    def __init__(self, dimension: int):
+        super().__init__()
+        self.weight = nn.Parameter(torch.ones(dimension))
+
+    def forward(self, value: torch.Tensor) -> torch.Tensor:
+        normalized = value.float() * torch.rsqrt(
+            value.float().square().mean(dim=-1, keepdim=True) + RMS_EPS
+        )
+        return normalized.to(value.dtype) * self.weight
+
+
+class CausalAttention(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.qkv = nn.Linear(D_MODEL, 3 * D_MODEL, bias=False)
+        self.o_proj = nn.Linear(D_MODEL, D_MODEL, bias=False)
+        mask = torch.triu(torch.ones(CONTEXT, CONTEXT, dtype=torch.bool), diagonal=1)
+        self.register_buffer("causal_mask", mask, persistent=False)
+
+    def forward(self, value: torch.Tensor) -> torch.Tensor:
+        batch, sequence, _ = value.shape
+        qkv = self.qkv(value).view(batch, sequence, 3, HEADS, D_HEAD)
+        query, key, content = qkv.unbind(dim=2)
+        query = query.transpose(1, 2)
+        key = key.transpose(1, 2)
+        content = content.transpose(1, 2)
+        scores = torch.matmul(query, key.transpose(-1, -2)).float() / math.sqrt(D_HEAD)
+        scores = scores.masked_fill(
+            self.causal_mask[:sequence, :sequence], float("-inf")
+        )
+        probabilities = torch.softmax(scores, dim=-1).to(query.dtype)
+        mixed = torch.matmul(probabilities, content)
+        mixed = mixed.transpose(1, 2).contiguous().view(batch, sequence, D_MODEL)
+        return self.o_proj(mixed)
+
+
+class SwiGLU(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.gate = nn.Linear(D_MODEL, D_FF, bias=False)
+        self.up = nn.Linear(D_MODEL, D_FF, bias=False)
+        self.down = nn.Linear(D_FF, D_MODEL, bias=False)
+
+    def forward(self, value: torch.Tensor) -> torch.Tensor:
+        return self.down(F.silu(self.gate(value)) * self.up(value))
+
+
+class TransformerBlock(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.attention_norm = RMSNorm(D_MODEL)
+        self.attention = CausalAttention()
+        self.mlp_norm = RMSNorm(D_MODEL)
+        self.mlp = SwiGLU()
+
+
+class DepthMixer(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.query = nn.Parameter(torch.zeros(D_MODEL))
+        self.key_norm = RMSNorm(D_MODEL)
+
+    def forward(
+        self, sources: list[torch.Tensor], capture: bool = False
+    ) -> tuple[torch.Tensor, dict[str, Any] | None]:
+        values = torch.stack(sources, dim=0)
+        keys = self.key_norm(values)
+        logits = torch.einsum("d,nbtd->nbt", self.query, keys.float())
+        weights = torch.softmax(logits, dim=0)
+        output = torch.einsum("nbt,nbtd->btd", weights, values.float()).to(
+            values.dtype
+        )
+        if not capture:
+            return output, None
+        entropy = -(weights * torch.log(weights.clamp_min(1e-30))).sum(dim=0)
+        return output, {
+            "mean_weights": weights.mean(dim=(1, 2)).detach().cpu().tolist(),
+            "entropy_mean": entropy.mean().detach().cpu().item(),
+            "sources": len(sources),
+        }
+
+
+@dataclass
+class TraceAccumulator:
+    layer_input_rms: list[float]
+    branch_output_rms: list[float]
+    stream_state_rms: list[float]
+    depth_weights: list[dict[str, Any]]
+    output_weights: dict[str, Any] | None = None
+
+
+def rms(value: torch.Tensor) -> float:
+    return value.float().square().mean().sqrt().detach().cpu().item()
+
+
+class ReducedLanguageModel(nn.Module):
+    def __init__(self, architecture: str):
+        super().__init__()
+        self.architecture = architecture
+        self.token_embedding = nn.Embedding(VOCABULARY, D_MODEL)
+        self.position_embedding = nn.Embedding(CONTEXT, D_MODEL)
+        self.blocks = nn.ModuleList([TransformerBlock() for _ in range(LAYERS)])
+        self.final_norm = RMSNorm(D_MODEL)
+        if architecture == "baseline":
+            self.mixers = nn.ModuleList()
+            self.output_mixer = None
+        else:
+            self.mixers = nn.ModuleList([DepthMixer() for _ in range(SUBLAYERS)])
+            self.output_mixer = DepthMixer()
+        self.reset_parameters()
+
+    def reset_parameters(self) -> None:
+        for module in self.modules():
+            if isinstance(module, nn.Embedding):
+                nn.init.normal_(module.weight, mean=0.0, std=0.02)
+            elif isinstance(module, nn.Linear):
+                nn.init.normal_(module.weight, mean=0.0, std=0.02)
+            elif isinstance(module, RMSNorm):
+                nn.init.ones_(module.weight)
+        scaled = 0.02 / math.sqrt(2 * LAYERS)
+        for block in self.blocks:
+            nn.init.normal_(block.attention.o_proj.weight, mean=0.0, std=scaled)
+            nn.init.normal_(block.mlp.down.weight, mean=0.0, std=scaled)
+        for mixer in self.mixers:
+            nn.init.zeros_(mixer.query)
+            nn.init.ones_(mixer.key_norm.weight)
+        if self.output_mixer is not None:
+            nn.init.zeros_(self.output_mixer.query)
+            nn.init.ones_(self.output_mixer.key_norm.weight)
+
+    def embed(self, input_ids: torch.Tensor) -> torch.Tensor:
+        positions = torch.arange(input_ids.shape[1], device=input_ids.device)
+        return self.token_embedding(input_ids) + self.position_embedding(positions)[None]
+
+    def forward(
+        self, input_ids: torch.Tensor, capture: bool = False
+    ) -> tuple[torch.Tensor, TraceAccumulator | None]:
+        embedded = self.embed(input_ids)
+        trace = (
+            TraceAccumulator([], [], [], [])
+            if capture
+            else None
+        )
+
+        if self.architecture == "baseline":
+            hidden = embedded
+            for block in self.blocks:
+                attention_input = hidden
+                attention_output = block.attention(block.attention_norm(attention_input))
+                hidden = hidden + attention_output
+                if trace is not None:
+                    trace.layer_input_rms.append(rms(attention_input))
+                    trace.branch_output_rms.append(rms(attention_output))
+                    trace.stream_state_rms.append(rms(hidden))
+                mlp_input = hidden
+                mlp_output = block.mlp(block.mlp_norm(mlp_input))
+                hidden = hidden + mlp_output
+                if trace is not None:
+                    trace.layer_input_rms.append(rms(mlp_input))
+                    trace.branch_output_rms.append(rms(mlp_output))
+                    trace.stream_state_rms.append(rms(hidden))
+        elif self.architecture == "full":
+            sources = [embedded]
+            mixer_index = 0
+            for block in self.blocks:
+                attention_input, weights = self.mixers[mixer_index](sources, capture)
+                mixer_index += 1
+                attention_output = block.attention(block.attention_norm(attention_input))
+                sources.append(attention_output)
+                if trace is not None:
+                    trace.layer_input_rms.append(rms(attention_input))
+                    trace.branch_output_rms.append(rms(attention_output))
+                    trace.stream_state_rms.append(
+                        rms(torch.stack(sources, dim=0))
+                    )
+                    trace.depth_weights.append(weights or {})
+                mlp_input, weights = self.mixers[mixer_index](sources, capture)
+                mixer_index += 1
+                mlp_output = block.mlp(block.mlp_norm(mlp_input))
+                sources.append(mlp_output)
+                if trace is not None:
+                    trace.layer_input_rms.append(rms(mlp_input))
+                    trace.branch_output_rms.append(rms(mlp_output))
+                    trace.stream_state_rms.append(
+                        rms(torch.stack(sources, dim=0))
+                    )
+                    trace.depth_weights.append(weights or {})
+            assert self.output_mixer is not None
+            hidden, output_weights = self.output_mixer(sources, capture)
+            if trace is not None:
+                trace.output_weights = output_weights
+        else:
+            completed = [embedded]
+            partial: torch.Tensor | None = None
+            mixer_index = 0
+            for block in self.blocks:
+                for branch_index in range(2):
+                    sources = completed + ([] if partial is None else [partial])
+                    branch_input, weights = self.mixers[mixer_index](sources, capture)
+                    mixer_index += 1
+                    if branch_index == 0:
+                        branch_output = block.attention(
+                            block.attention_norm(branch_input)
+                        )
+                    else:
+                        branch_output = block.mlp(block.mlp_norm(branch_input))
+                    partial = (
+                        branch_output if partial is None else partial + branch_output
+                    )
+                    if trace is not None:
+                        trace.layer_input_rms.append(rms(branch_input))
+                        trace.branch_output_rms.append(rms(branch_output))
+                        trace.stream_state_rms.append(rms(partial))
+                        trace.depth_weights.append(weights or {})
+                    if mixer_index % SUBLAYERS_PER_BLOCK == 0:
+                        completed.append(partial)
+                        partial = None
+            assert partial is None
+            assert len(completed) == BLOCKS + 1
+            assert self.output_mixer is not None
+            hidden, output_weights = self.output_mixer(completed, capture)
+            if trace is not None:
+                trace.output_weights = output_weights
+
+        normalized = self.final_norm(hidden)
+        logits = F.linear(normalized, self.token_embedding.weight)
+        return logits, trace
+
+
+def learning_rate(step: int, total_steps: int) -> float:
+    if step <= WARMUP_STEPS:
+        return PEAK_LR * step / WARMUP_STEPS
+    progress = (step - WARMUP_STEPS) / max(1, total_steps - WARMUP_STEPS)
+    cosine = 0.5 * (1 + math.cos(math.pi * progress))
+    return MIN_LR + (PEAK_LR - MIN_LR) * cosine
+
+
+def cross_entropy(logits: torch.Tensor, targets: torch.Tensor) -> torch.Tensor:
+    return F.cross_entropy(
+        logits.float().reshape(-1, VOCABULARY), targets.reshape(-1)
+    )
+
+
+@torch.no_grad()
+def evaluate(
+    model: ReducedLanguageModel,
+    corpus: ByteCorpus,
+    starts: list[int],
+    window_count: int,
+    eval_batch_size: int,
+) -> dict[str, float]:
+    model.eval()
+    loss_sum = 0.0
+    target_count = 0
+    for begin in range(0, window_count, eval_batch_size):
+        end = min(begin + eval_batch_size, window_count)
+        inputs, targets = corpus.fixed_batch(starts, begin, end)
+        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+            logits, _ = model(inputs)
+        loss = F.cross_entropy(
+            logits.float().reshape(-1, VOCABULARY),
+            targets.reshape(-1),
+            reduction="sum",
+        )
+        loss_sum += loss.detach().cpu().item()
+        target_count += targets.numel()
+    nats = loss_sum / target_count
+    return {"cross_entropy_nats": nats, "bits_per_byte": nats / math.log(2)}
+
+
+def percentile(values: list[float], quantile: float) -> float:
+    return float(np.quantile(np.asarray(values, dtype=np.float64), quantile))
+
+
+def core_parameter_gradient_rms(model: ReducedLanguageModel) -> list[float]:
+    values = []
+    for block in model.blocks:
+        sum_square = 0.0
+        count = 0
+        for parameter in block.parameters():
+            if parameter.grad is None:
+                continue
+            gradient = parameter.grad.detach().float()
+            sum_square += gradient.square().sum().detach().cpu().item()
+            count += gradient.numel()
+        values.append(math.sqrt(sum_square / count))
+    return values
+
+
+def diagnostic(
+    model: ReducedLanguageModel,
+    corpus: ByteCorpus,
+    window_count: int,
+) -> dict[str, Any]:
+    model.eval()
+    model.zero_grad(set_to_none=True)
+    inputs, targets = corpus.fixed_batch(
+        corpus.diagnostic_starts, 0, window_count
+    )
+    with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+        logits, trace = model(inputs, capture=True)
+        loss = cross_entropy(logits, targets)
+    loss.backward()
+    gradients = core_parameter_gradient_rms(model)
+    assert trace is not None
+    return {
+        "loss_nats": loss.detach().cpu().item(),
+        "bits_per_byte": loss.detach().cpu().item() / math.log(2),
+        "layer_input_rms": trace.layer_input_rms,
+        "branch_output_rms": trace.branch_output_rms,
+        "stream_state_rms": trace.stream_state_rms,
+        "core_parameter_grad_rms_by_block": gradients,
+        "depth_weights": trace.depth_weights,
+        "output_weights": trace.output_weights,
+    }
+
+
+def parameter_inventory(model: ReducedLanguageModel) -> dict[str, int]:
+    total = sum(parameter.numel() for parameter in model.parameters())
+    mixer = sum(
+        parameter.numel()
+        for name, parameter in model.named_parameters()
+        if name.startswith("mixers.") or name.startswith("output_mixer.")
+    )
+    embedding = model.token_embedding.weight.numel() + model.position_embedding.weight.numel()
+    return {
+        "total": total,
+        "core": total - mixer,
+        "mixer": mixer,
+        "embedding": embedding,
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    if not torch.cuda.is_available():
+        raise RuntimeError("CUDA is required by the frozen protocol")
+    if args.run_kind != "smoke" and args.seed not in EXPECTED_SEEDS:
+        raise ValueError(f"formal/replay seed is not preregistered: {args.seed}")
+    configure_determinism(args.seed)
+    device = torch.device("cuda")
+
+    manifest = json.loads(args.manifest.read_text())
+    if manifest["protocol_id"] != PROTOCOL_ID:
+        raise ValueError("manifest protocol mismatch")
+    if manifest["dataset"]["revision"] != (
+        "b08601e04326c79dfdd32d625aee71d232d685c3"
+    ):
+        raise ValueError("dataset revision mismatch")
+    corpus = ByteCorpus(args.cache_dir, manifest, device)
+
+    model = ReducedLanguageModel(args.architecture).to(device)
+    initial_common_hash = state_hash(model, include_mixers=False)
+    initial_mixer_hash = (
+        state_hash(model, include_mixers=True)
+        if args.architecture != "baseline"
+        else None
+    )
+    inventory = parameter_inventory(model)
+
+    decay_parameters: list[nn.Parameter] = []
+    no_decay_parameters: list[nn.Parameter] = []
+    for parameter in model.parameters():
+        if parameter.ndim >= 2:
+            decay_parameters.append(parameter)
+        else:
+            no_decay_parameters.append(parameter)
+    optimizer = torch.optim.AdamW(
+        [
+            {"params": decay_parameters, "weight_decay": WEIGHT_DECAY},
+            {"params": no_decay_parameters, "weight_decay": 0.0},
+        ],
+        lr=PEAK_LR,
+        betas=BETAS,
+        eps=ADAM_EPS,
+    )
+
+    evaluation_steps = sorted(
+        set(step for step in EVAL_STEPS if step <= args.steps) | {0, args.steps}
+    )
+    evaluations = [
+        {
+            "step": 0,
+            **evaluate(
+                model,
+                corpus,
+                corpus.validation_starts,
+                args.validation_windows,
+                args.eval_batch_size,
+            ),
+        }
+    ]
+    training_history: list[dict[str, float | int]] = []
+    step_times: list[float] = []
+    model.train()
+
+    for step in range(1, args.steps + 1):
+        lr = learning_rate(step, args.steps)
+        for group in optimizer.param_groups:
+            group["lr"] = lr
+        inputs, targets = corpus.training_batch(args.seed, step, args.batch_size)
+        optimizer.zero_grad(set_to_none=True)
+
+        torch.cuda.synchronize()
+        started = time.perf_counter()
+        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+            logits, _ = model(inputs)
+            loss = cross_entropy(logits, targets)
+        if not torch.isfinite(loss):
+            raise RuntimeError(f"non-finite loss at step {step}: {loss}")
+        loss.backward()
+        unclipped_norm = torch.nn.utils.clip_grad_norm_(
+            model.parameters(), GRAD_CLIP
+        )
+        optimizer.step()
+        torch.cuda.synchronize()
+        elapsed_ms = (time.perf_counter() - started) * 1000
+
+        if step == args.timing_warmup:
+            torch.cuda.reset_peak_memory_stats()
+        elif step > args.timing_warmup:
+            step_times.append(elapsed_ms)
+
+        if step == 1 or step % 10 == 0 or step == args.steps:
+            training_history.append(
+                {
+                    "step": step,
+                    "loss_nats": loss.detach().cpu().item(),
+                    "bits_per_byte": loss.detach().cpu().item() / math.log(2),
+                    "learning_rate": lr,
+                    "unclipped_grad_norm": float(unclipped_norm.detach().cpu()),
+                }
+            )
+
+        if step in evaluation_steps and step != 0:
+            evaluations.append(
+                {
+                    "step": step,
+                    **evaluate(
+                        model,
+                        corpus,
+                        corpus.validation_starts,
+                        args.validation_windows,
+                        args.eval_batch_size,
+                    ),
+                }
+            )
+            model.train()
+
+    training_peak_allocated = torch.cuda.max_memory_allocated()
+    training_peak_reserved = torch.cuda.max_memory_reserved()
+    diagnostic_result = diagnostic(model, corpus, args.diagnostic_windows)
+    final_common_hash = state_hash(model, include_mixers=False)
+    final_mixer_hash = (
+        state_hash(model, include_mixers=True)
+        if args.architecture != "baseline"
+        else None
+    )
+    timing = {
+        "warmup_steps_excluded": args.timing_warmup,
+        "measured_steps": len(step_times),
+        "mean_ms": statistics.fmean(step_times) if step_times else None,
+        "median_ms": statistics.median(step_times) if step_times else None,
+        "p95_ms": percentile(step_times, 0.95) if step_times else None,
+        "peak_allocated_bytes": training_peak_allocated,
+        "peak_reserved_bytes": training_peak_reserved,
+    }
+
+    result = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "run_kind": args.run_kind,
+        "architecture": args.architecture,
+        "seed": args.seed,
+        "steps": args.steps,
+        "batch_size": args.batch_size,
+        "target_bytes_seen": args.steps * args.batch_size * CONTEXT,
+        "manifest": {
+            "path": str(args.manifest),
+            "file_sha256": file_sha256(args.manifest),
+            "formal_schedule_sha256": manifest["windows"][
+                "formal_schedule_sha256"
+            ],
+            "validation_tensor_sha256": manifest["windows"][
+                "validation_tensor_sha256"
+            ],
+            "diagnostic_tensor_sha256": manifest["windows"][
+                "diagnostic_tensor_sha256"
+            ],
+        },
+        "model": {
+            "layers": LAYERS,
+            "sublayers": SUBLAYERS,
+            "blocks_for_block_attnres": BLOCKS,
+            "sublayers_per_attnres_block": SUBLAYERS_PER_BLOCK,
+            "d_model": D_MODEL,
+            "heads": HEADS,
+            "d_head": D_HEAD,
+            "d_ff": D_FF,
+            "context": CONTEXT,
+            "vocabulary": VOCABULARY,
+            "parameters": inventory,
+        },
+        "optimizer": {
+            "name": "AdamW",
+            "betas": list(BETAS),
+            "epsilon": ADAM_EPS,
+            "weight_decay_ndim_ge_2": WEIGHT_DECAY,
+            "peak_lr": PEAK_LR,
+            "min_lr": MIN_LR,
+            "warmup_steps": WARMUP_STEPS,
+            "grad_clip": GRAD_CLIP,
+        },
+        "hashes": {
+            "initial_common_parameters": initial_common_hash,
+            "initial_mixer_parameters": initial_mixer_hash,
+            "final_common_parameters": final_common_hash,
+            "final_mixer_parameters": final_mixer_hash,
+        },
+        "evaluations": evaluations,
+        "training_history": training_history,
+        "diagnostic": diagnostic_result,
+        "timing": timing,
+        "environment": {
+            "python": platform.python_version(),
+            "torch": torch.__version__,
+            "cuda": torch.version.cuda,
+            "gpu": torch.cuda.get_device_name(0),
+            "compute_capability": list(torch.cuda.get_device_capability(0)),
+            "cublas_workspace_config": os.environ["CUBLAS_WORKSPACE_CONFIG"],
+            "deterministic_algorithms": torch.are_deterministic_algorithms_enabled(),
+            "autocast": "cuda-bfloat16",
+            "compile": False,
+        },
+    }
+    result["canonical_sha256_without_self"] = canonical_json_sha256(result)
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    temporary = args.output.with_suffix(args.output.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    os.replace(temporary, args.output)
+    print(
+        json.dumps(
+            {
+                "output": str(args.output),
+                "architecture": args.architecture,
+                "seed": args.seed,
+                "steps": args.steps,
+                "final_bpc": evaluations[-1]["bits_per_byte"],
+                "initial_common_hash": initial_common_hash,
+                "final_common_hash": final_common_hash,
+                "canonical_sha256": result["canonical_sha256_without_self"],
+                "timing": timing,
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_forward/README.md

@@ -0,0 +1,70 @@
+# Attention Residuals train-time forward intervention
+
+This directory implements preregistered protocol
+`llm-atlas-k3-attnres-forward-training-v1`:
+
+- `research/K3_ATTNRES_FORWARD_TRAINING_SCOPING.md`
+- `research/K3_ATTNRES_FORWARD_TRAINING_PROTOCOL.md`
+- `research/K3_ATTNRES_FORWARD_TRAINING_GROK_REVIEW.md`
+- `research/K3_ATTNRES_FORWARD_TRAINING_IMPLEMENTATION_REVIEW.md`
+- `research/K3_ATTNRES_FORWARD_TRAINING_AUDIT.md`
+
+It is a depth-32 reduced Block AttnRes architecture ablation. It is not a
+Kimi-K3 checkpoint forward pass and does not claim to recover unpublished
+Figure 5 telemetry.
+
+## Frozen environment
+
+```text
+Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
+PyTorch 2.11.0+cu128
+GPU NVIDIA GeForce RTX 5090
+CUBLAS_WORKSPACE_CONFIG=:4096:8
+maximum concurrency 2
+```
+
+## Pre-result gates
+
+The checked-in gate artifacts must pass before formal output:
+
+```bash
+CUBLAS_WORKSPACE_CONFIG=:4096:8 \
+/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
+  experiments/k3/attnres_forward/verify.py step-zero \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
+  --parent-manifest experiments/k3/attnres_gradient/manifest.json \
+  --study-manifest experiments/k3/attnres_forward/manifest.json \
+  --output experiments/k3/attnres_forward/results/gates/step-zero.json
+```
+
+`learned_reference` is smoke-only. Its 20-step result is compared with a
+fresh parent Round 05 smoke using `verify.py smoke-compare`.
+
+## Formal matrix
+
+```bash
+/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
+  experiments/k3/attnres_forward/run_matrix.py \
+  --python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
+  --parent-manifest experiments/k3/attnres_gradient/manifest.json \
+  --study-manifest experiments/k3/attnres_forward/manifest.json \
+  --output-dir experiments/k3/attnres_forward/results/raw \
+  --phase all \
+  --concurrency 2
+```
+
+This runs 12 formal cells and one full replay. The analyzer reads all cells,
+the frozen historical paired references, and generates the only authoritative
+status, interaction map, and website compact artifact.
+
+The checked-in Round 08 release contains:
+
+- 13 raw results under `results/raw/`;
+- `reproduction.json` with the exact primary scientific-payload hash;
+- aggregate / compact website data under `src/data/`;
+- a frozen-data checker and real-Chrome five-view regression in `scripts/`.
+
+The established status is deliberately scoped to this reduced protocol. It is
+not a real Kimi-K3 checkpoint result or a reproduction of unpublished Figure
+5(c) telemetry.

experiments/k3/attnres_forward/analyze.py

@@ -0,0 +1,699 @@
+#!/usr/bin/env python3
+"""Aggregate and gate preregistered Round 08 forward-training results."""
+
+from __future__ import annotations
+
+import argparse
+import copy
+import hashlib
+import json
+import math
+import statistics
+from pathlib import Path
+from typing import Any, Iterable
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-forward-training-v1"
+PARENT_PROTOCOL_ID = "llm-atlas-k3-attnres-gradient-scale-v1"
+METRICS = ("spike_contrast", "peak_normalized")
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument("--formal", type=Path, action="append", required=True)
+    parser.add_argument("--replay", type=Path, required=True)
+    parser.add_argument("--reference-dir", type=Path, required=True)
+    parser.add_argument("--aggregate-output", type=Path, required=True)
+    parser.add_argument("--compact-output", type=Path, required=True)
+    parser.add_argument("--reproduction-output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def canonical_sha256(value: Any) -> str:
+    return hashlib.sha256(
+        json.dumps(
+            value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+        ).encode()
+    ).hexdigest()
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def mean(values: Iterable[float]) -> float:
+    return statistics.fmean(values)
+
+
+def read_result(path: Path, expected_protocol: str) -> dict[str, Any]:
+    value = json.loads(path.read_text())
+    if value.get("protocol_id") != expected_protocol:
+        raise RuntimeError(f"protocol mismatch: {path}")
+    expected = value.get("canonical_sha256_without_self")
+    payload = {
+        key: item
+        for key, item in value.items()
+        if key != "canonical_sha256_without_self"
+    }
+    if not isinstance(expected, str) or canonical_sha256(payload) != expected:
+        raise RuntimeError(f"canonical self-hash mismatch: {path}")
+    return value
+
+
+def exactly_one(values: list[dict[str, Any]], step: int) -> dict[str, Any]:
+    matches = [value for value in values if value["step"] == step]
+    if len(matches) != 1:
+        raise RuntimeError(f"step {step} missing or duplicated")
+    return matches[0]
+
+
+def spectrum_metrics(
+    diagnostic: dict[str, Any],
+    spike_layers: tuple[int, ...],
+    epsilon: float,
+) -> dict[str, Any]:
+    values = [
+        float(value)
+        for value in diagnostic["activation_grad_rms_by_block"]
+    ]
+    if len(values) != 32:
+        raise RuntimeError("activation-gradient spectrum must have 32 layers")
+    if any(not math.isfinite(value) or value <= epsilon for value in values):
+        raise RuntimeError("activation-gradient spectrum is non-finite/non-positive")
+    spike_indices = {layer - 1 for layer in spike_layers}
+    spike_values = [
+        value for index, value in enumerate(values) if index in spike_indices
+    ]
+    reference_values = [
+        value for index, value in enumerate(values) if index not in spike_indices
+    ]
+    spike_mean = mean(spike_values)
+    reference_mean = mean(reference_values)
+    global_mean = mean(values)
+    contrast = spike_mean / reference_mean
+    peak = max(values) / global_mean
+    if any(
+        not math.isfinite(value) or value <= epsilon
+        for value in (spike_mean, reference_mean, contrast, peak)
+    ):
+        raise RuntimeError("derived spike metric is non-finite/non-positive")
+    ordered = sorted(range(32), key=lambda index: (-values[index], index))
+    return {
+        "values": values,
+        "normalized": [value / global_mean for value in values],
+        "spike_mean": spike_mean,
+        "reference_mean": reference_mean,
+        "global_mean": global_mean,
+        "spike_contrast": contrast,
+        "peak_normalized": peak,
+        "peak_layer_1based": ordered[0] + 1,
+        "top_five_layers_1based": [index + 1 for index in ordered[:5]],
+    }
+
+
+def final_bpc(value: dict[str, Any], step: int) -> float:
+    result = float(exactly_one(value["evaluations"], step)["bits_per_byte"])
+    if not math.isfinite(result):
+        raise RuntimeError("final validation BPC is non-finite")
+    return result
+
+
+def stable_environment(value: dict[str, Any]) -> dict[str, Any]:
+    keys = (
+        "cublas_workspace_config",
+        "deterministic_algorithms",
+        "autocast",
+        "compile",
+    )
+    return {key: value["environment"][key] for key in keys}
+
+
+def pairing_checks(
+    run: dict[str, Any], reference: dict[str, Any]
+) -> dict[str, bool]:
+    manifest_fields = (
+        "formal_schedule_sha256",
+        "validation_tensor_sha256",
+        "diagnostic_tensor_sha256",
+        "input_gate_tensor_hashes",
+    )
+    checks = {
+        "seed": run["seed"] == reference["seed"],
+        "architecture": (
+            run["architecture"] == reference["architecture"] == "block"
+        ),
+        "depth": run["depth"] == reference["depth"] == 32,
+        "steps": run["steps"] == reference["steps"] == 8000,
+        "batch_size": run["batch_size"] == reference["batch_size"] == 32,
+        "initial_public_parameters": (
+            run["hashes"]["initial_public_parameters"]
+            == reference["hashes"]["initial_public_parameters"]
+        ),
+        "initial_mixer_parameters": (
+            run["hashes"]["initial_mixer_parameters"]
+            == reference["hashes"]["initial_mixer_parameters"]
+        ),
+        "model_topology": run["model"] == reference["model"],
+        "optimizer_hyperparameters": (
+            run["optimizer"] == reference["optimizer"]
+        ),
+        "scientific_environment": (
+            stable_environment(run) == stable_environment(reference)
+        ),
+    }
+    for field in manifest_fields:
+        checks[f"manifest.{field}"] = (
+            run["manifest"][field] == reference["manifest"][field]
+        )
+    return checks
+
+
+def scientific_replay_payload(value: dict[str, Any]) -> dict[str, Any]:
+    payload = copy.deepcopy(value)
+    for key in (
+        "run_kind",
+        "timing",
+        "canonical_sha256_without_self",
+        "parent_runner_canonical_sha256",
+    ):
+        payload.pop(key, None)
+    payload["manifest"].pop("path", None)
+    payload["study_manifest"].pop("path", None)
+    payload["environment"] = stable_environment(value)
+    return payload
+
+
+def quality_gate(
+    variant_runs: dict[int, dict[str, Any]],
+    references: dict[int, dict[str, Any]],
+    *,
+    step: int,
+    per_seed_maximum: float,
+    mean_maximum: float,
+) -> dict[str, Any]:
+    per_seed = {}
+    for seed, run in sorted(variant_runs.items()):
+        variant_bpc = final_bpc(run, step)
+        reference_bpc = final_bpc(references[seed], step)
+        delta = variant_bpc - reference_bpc
+        per_seed[str(seed)] = {
+            "variant_bpc": variant_bpc,
+            "reference_bpc": reference_bpc,
+            "delta_bpc": delta,
+            "passed": delta <= per_seed_maximum,
+        }
+    mean_delta = mean(item["delta_bpc"] for item in per_seed.values())
+    per_seed_passed = all(item["passed"] for item in per_seed.values())
+    mean_passed = mean_delta <= mean_maximum
+    return {
+        "passed": per_seed_passed and mean_passed,
+        "passed_checks": (
+            sum(item["passed"] for item in per_seed.values())
+            + int(mean_passed)
+        ),
+        "required_checks": 4,
+        "per_seed_maximum": per_seed_maximum,
+        "mean_maximum": mean_maximum,
+        "mean_delta_bpc": mean_delta,
+        "mean_passed": mean_passed,
+        "per_seed": per_seed,
+    }
+
+
+def variant_effect(
+    variant: str,
+    runs: dict[int, dict[str, Any]],
+    references: dict[int, dict[str, Any]],
+    metrics_by_cell: dict[tuple[str, int, int], dict[str, Any]],
+    *,
+    step: int,
+    threshold: float,
+    quality: dict[str, Any],
+) -> dict[str, Any]:
+    cells = []
+    for seed in sorted(runs):
+        candidate = metrics_by_cell[(variant, seed, step)]
+        reference = metrics_by_cell[("learned_reference", seed, step)]
+        for metric in METRICS:
+            reference_value = reference[metric]
+            candidate_value = candidate[metric]
+            relative_drop = (
+                reference_value - candidate_value
+            ) / reference_value
+            cells.append(
+                {
+                    "seed": seed,
+                    "metric": metric,
+                    "reference": reference_value,
+                    "variant": candidate_value,
+                    "relative_drop": relative_drop,
+                    "passed": relative_drop >= threshold,
+                }
+            )
+    attenuation_passed = all(cell["passed"] for cell in cells)
+    return {
+        "variant": variant,
+        "threshold": threshold,
+        "passed_cells": sum(cell["passed"] for cell in cells),
+        "required_cells": len(cells),
+        "attenuation_passed": attenuation_passed,
+        "quality": quality,
+        "material_response_passed": (
+            attenuation_passed and quality["passed"]
+        ),
+        "cells": cells,
+    }
+
+
+def interaction_map(
+    metrics_by_cell: dict[tuple[str, int, int], dict[str, Any]],
+    seeds: tuple[int, ...],
+    steps: tuple[int, ...],
+) -> dict[str, Any]:
+    cells = []
+    for step in steps:
+        for seed in seeds:
+            reference = metrics_by_cell[
+                ("learned_reference", seed, step)
+            ]
+            group6 = metrics_by_cell[
+                ("uniform_group_6_forward", seed, step)
+            ]
+            group7 = metrics_by_cell[
+                ("uniform_group_7_forward", seed, step)
+            ]
+            joint = metrics_by_cell[
+                ("uniform_groups_6_7_forward", seed, step)
+            ]
+            for metric in METRICS:
+                ref = reference[metric]
+                effects = {
+                    "group6": math.log(ref / group6[metric]),
+                    "group7": math.log(ref / group7[metric]),
+                    "groups6_7": math.log(ref / joint[metric]),
+                }
+                residual = (
+                    effects["groups6_7"]
+                    - effects["group6"]
+                    - effects["group7"]
+                )
+                cells.append(
+                    {
+                        "step": step,
+                        "seed": seed,
+                        "metric": metric,
+                        "log_effects": effects,
+                        "interaction_residual": residual,
+                        "relative_drops": {
+                            "group6": (ref - group6[metric]) / ref,
+                            "group7": (ref - group7[metric]) / ref,
+                            "groups6_7": (ref - joint[metric]) / ref,
+                        },
+                    }
+                )
+    summaries = []
+    for step in steps:
+        for metric in METRICS:
+            selected = [
+                cell
+                for cell in cells
+                if cell["step"] == step and cell["metric"] == metric
+            ]
+            residuals = [
+                cell["interaction_residual"] for cell in selected
+            ]
+            summaries.append(
+                {
+                    "step": step,
+                    "metric": metric,
+                    "mean_interaction_residual": mean(residuals),
+                    "minimum": min(residuals),
+                    "maximum": max(residuals),
+                }
+            )
+    return {
+        "definition": "I67=ln(Xref/X67)-ln(Xref/X6)-ln(Xref/X7)",
+        "interpretation": (
+            "descriptive cross-run log-attenuation residual from three "
+            "independently trained variants; not a causal interaction"
+        ),
+        "cells": cells,
+        "summaries": summaries,
+    }
+
+
+def environment_metadata(value: dict[str, Any]) -> dict[str, Any]:
+    return {
+        key: value["environment"].get(key)
+        for key in ("gpu", "torch", "cuda", "compute_capability")
+    }
+
+
+def write_hashed(path: Path, value: dict[str, Any]) -> None:
+    value["canonical_sha256_without_self"] = canonical_sha256(value)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(
+        json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+
+
+def main() -> None:
+    args = parse_args()
+    manifest = json.loads(args.manifest.read_text())
+    if (
+        manifest["protocol_id"] != PROTOCOL_ID
+        or manifest["status"] != "frozen-before-model-output"
+    ):
+        raise RuntimeError("manifest is not the frozen Round 08 contract")
+    variants = tuple(manifest["variants"].keys())
+    seeds = tuple(manifest["formal_seeds"])
+    steps = tuple(manifest["diagnostic_steps"])
+    primary_step = manifest["primary_step"]
+    epsilon = manifest["thresholds"]["positive_denominator_epsilon"]
+    spike_layers = tuple(manifest["spike_layers_1based"])
+    expected_cells = {(variant, seed) for variant in variants for seed in seeds}
+    if len(args.formal) != len(expected_cells):
+        raise RuntimeError("formal path count does not match the 4×3 matrix")
+
+    runs: dict[tuple[str, int], dict[str, Any]] = {}
+    run_paths: dict[tuple[str, int], Path] = {}
+    pairing: dict[str, Any] = {}
+    references: dict[int, dict[str, Any]] = {}
+    reference_paths: dict[int, Path] = {}
+    for seed in seeds:
+        path = args.reference_dir / (
+            f"formal-depth-32-block-seed-{seed}.json"
+        )
+        references[seed] = read_result(path, PARENT_PROTOCOL_ID)
+        reference_paths[seed] = path
+
+    for path in args.formal:
+        value = read_result(path, PROTOCOL_ID)
+        identity = (value["variant"], value["seed"])
+        if identity in runs:
+            raise RuntimeError(f"duplicate formal cell: {identity}")
+        if (
+            value["run_kind"] != "formal"
+            or value["steps"] != manifest["formal_steps"]
+            or not value["forward_intervention"]["passed"]
+        ):
+            raise RuntimeError(f"invalid formal cell: {path}")
+        runs[identity] = value
+        run_paths[identity] = path
+    if set(runs) != expected_cells:
+        raise RuntimeError("formal matrix identities do not match manifest")
+
+    for (variant, seed), value in sorted(runs.items()):
+        checks = pairing_checks(value, references[seed])
+        if not all(checks.values()):
+            raise RuntimeError(
+                f"historical reference pairing failed: "
+                f"{variant}/{seed}: {checks}"
+            )
+        pairing[f"{variant}:{seed}"] = {
+            "passed": True,
+            "checks": checks,
+            "run_environment": environment_metadata(value),
+            "reference_environment": environment_metadata(references[seed]),
+            "metadata_equal": (
+                environment_metadata(value)
+                == environment_metadata(references[seed])
+            ),
+        }
+    metadata_warnings = [
+        {
+            "cell": cell,
+            "message": (
+                "GPU/version metadata differs from the historical paired "
+                "reference; frozen scientific-environment fields still match"
+            ),
+            "run_environment": item["run_environment"],
+            "reference_environment": item["reference_environment"],
+        }
+        for cell, item in pairing.items()
+        if not item["metadata_equal"]
+    ]
+
+    replay = read_result(args.replay, PROTOCOL_ID)
+    replay_contract = manifest["replay"]
+    if (
+        replay["run_kind"] != "replay"
+        or replay["variant"] != replay_contract["variant"]
+        or replay["seed"] != replay_contract["seed"]
+        or replay["steps"] != manifest["formal_steps"]
+        or not replay["forward_intervention"]["passed"]
+    ):
+        raise RuntimeError("invalid replay identity/audit")
+    formal_primary = runs[
+        (replay_contract["variant"], replay_contract["seed"])
+    ]
+    formal_payload = scientific_replay_payload(formal_primary)
+    replay_payload = scientific_replay_payload(replay)
+    replay_exact = formal_payload == replay_payload
+    if not replay_exact:
+        raise RuntimeError("primary formal/replay scientific payload mismatch")
+
+    metrics_by_cell: dict[tuple[str, int, int], dict[str, Any]] = {}
+    for seed, reference in references.items():
+        for step in steps:
+            metrics_by_cell[("learned_reference", seed, step)] = (
+                spectrum_metrics(
+                    exactly_one(reference["diagnostics"], step),
+                    spike_layers,
+                    epsilon,
+                )
+            )
+    for (variant, seed), value in runs.items():
+        if tuple(item["step"] for item in value["diagnostics"]) != steps:
+            raise RuntimeError(f"diagnostic schedule drift: {variant}/{seed}")
+        for step in steps:
+            metrics_by_cell[(variant, seed, step)] = spectrum_metrics(
+                exactly_one(value["diagnostics"], step),
+                spike_layers,
+                epsilon,
+            )
+
+    runs_by_variant = {
+        variant: {seed: runs[(variant, seed)] for seed in seeds}
+        for variant in variants
+    }
+    qualities = {
+        variant: quality_gate(
+            variant_runs,
+            references,
+            step=primary_step,
+            per_seed_maximum=manifest["thresholds"][
+                "final_bpc_delta_per_seed_maximum"
+            ],
+            mean_maximum=manifest["thresholds"][
+                "final_bpc_delta_mean_maximum"
+            ],
+        )
+        for variant, variant_runs in runs_by_variant.items()
+    }
+    effects = {
+        variant: variant_effect(
+            variant,
+            variant_runs,
+            references,
+            metrics_by_cell,
+            step=primary_step,
+            threshold=manifest["thresholds"]["material_relative_drop"],
+            quality=qualities[variant],
+        )
+        for variant, variant_runs in runs_by_variant.items()
+    }
+    primary = effects[manifest["primary_variant"]]
+    if primary["attenuation_passed"] and primary["quality"]["passed"]:
+        status = (
+            "forward_training_attenuation_established_within_reduced_protocol"
+        )
+    elif primary["attenuation_passed"]:
+        status = "quality_guard_failed"
+    elif primary["quality"]["passed"]:
+        status = "attenuation_not_established"
+    else:
+        status = "attenuation_and_quality_failed"
+    secondary = {
+        variant: (
+            "secondary_material_response"
+            if effect["material_response_passed"]
+            else "secondary_response_not_established"
+        )
+        for variant, effect in effects.items()
+        if variant != manifest["primary_variant"]
+    }
+    interaction = interaction_map(metrics_by_cell, seeds, steps)
+
+    trajectories = []
+    final_spectra = []
+    for variant in ("learned_reference",) + variants:
+        for seed in seeds:
+            for step in steps:
+                record = metrics_by_cell[(variant, seed, step)]
+                reference = metrics_by_cell[
+                    ("learned_reference", seed, step)
+                ]
+                trajectories.append(
+                    {
+                        "variant": variant,
+                        "seed": seed,
+                        "step": step,
+                        "spike_mean": record["spike_mean"],
+                        "reference_mean": record["reference_mean"],
+                        "spike_contrast": record["spike_contrast"],
+                        "peak_normalized": record["peak_normalized"],
+                        "relative_drop": {
+                            metric: (
+                                reference[metric] - record[metric]
+                            )
+                            / reference[metric]
+                            for metric in METRICS
+                        },
+                    }
+                )
+            final = metrics_by_cell[(variant, seed, primary_step)]
+            final_spectra.append(
+                {
+                    "variant": variant,
+                    "seed": seed,
+                    **final,
+                }
+            )
+
+    input_files = {
+        "manifest": {
+            "path": str(args.manifest),
+            "sha256": file_sha256(args.manifest),
+        },
+        "formal": [
+            {
+                "variant": variant,
+                "seed": seed,
+                "path": str(run_paths[(variant, seed)]),
+                "sha256": file_sha256(run_paths[(variant, seed)]),
+            }
+            for variant, seed in sorted(runs)
+        ],
+        "references": [
+            {
+                "seed": seed,
+                "path": str(reference_paths[seed]),
+                "sha256": file_sha256(reference_paths[seed]),
+            }
+            for seed in seeds
+        ],
+        "replay": {
+            "path": str(args.replay),
+            "sha256": file_sha256(args.replay),
+        },
+    }
+    aggregate = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "status": status,
+        "scope": (
+            "depth-32 reduced Block AttnRes train-time architecture "
+            "ablation; not a real Kimi-K3 checkpoint result"
+        ),
+        "primary_step": primary_step,
+        "spike_layers_1based": list(spike_layers),
+        "thresholds": manifest["thresholds"],
+        "input_files": input_files,
+        "historical_pairing": pairing,
+        "metadata_warnings": metadata_warnings,
+        "replay": {
+            "passed": replay_exact,
+            "scientific_payload_sha256": canonical_sha256(formal_payload),
+            "excluded": [
+                "run_kind",
+                "timing",
+                "self hashes",
+                "manifest path strings",
+                "GPU/version metadata",
+            ],
+        },
+        "primary": primary,
+        "secondary_status": secondary,
+        "effects": effects,
+        "interaction": interaction,
+        "trajectories": trajectories,
+        "final_spectra": final_spectra,
+        "processed_target_bytes": manifest["new_target_bytes"],
+        "historical_reference_target_bytes": (
+            manifest["historical_reference_target_bytes"]
+        ),
+        "reporting_boundary": (
+            "C can change through spike-window numerator and the 27-layer "
+            "reference denominator; layers 26-28 are intervened but belong "
+            "to the denominator."
+        ),
+    }
+    write_hashed(args.aggregate_output, aggregate)
+
+    compact = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "status": status,
+        "primary_step": primary_step,
+        "spike_layers_1based": list(spike_layers),
+        "thresholds": manifest["thresholds"],
+        "primary": primary,
+        "secondary_status": secondary,
+        "effects": effects,
+        "interaction": interaction,
+        "trajectories": trajectories,
+        "final_spectra": final_spectra,
+        "replay": aggregate["replay"],
+        "metadata_warnings": metadata_warnings,
+        "processed_target_bytes": manifest["new_target_bytes"],
+        "reporting_boundary": aggregate["reporting_boundary"],
+        "aggregate_sha256": aggregate["canonical_sha256_without_self"],
+    }
+    write_hashed(args.compact_output, compact)
+
+    reproduction = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "passed": replay_exact,
+        "formal_variant": replay_contract["variant"],
+        "seed": replay_contract["seed"],
+        "formal_file_sha256": file_sha256(
+            run_paths[
+                (replay_contract["variant"], replay_contract["seed"])
+            ]
+        ),
+        "replay_file_sha256": file_sha256(args.replay),
+        "scientific_payload_sha256": canonical_sha256(formal_payload),
+        "excluded_fields": aggregate["replay"]["excluded"],
+    }
+    write_hashed(args.reproduction_output, reproduction)
+    print(
+        json.dumps(
+            {
+                "status": status,
+                "primary_attenuation": {
+                    "passed_cells": primary["passed_cells"],
+                    "required_cells": primary["required_cells"],
+                },
+                "primary_quality": {
+                    "passed_checks": primary["quality"]["passed_checks"],
+                    "required_checks": primary["quality"]["required_checks"],
+                },
+                "replay_exact": replay_exact,
+                "aggregate": str(args.aggregate_output),
+                "compact": str(args.compact_output),
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_forward/manifest.json

@@ -0,0 +1,89 @@
+{
+  "schema_version": 1,
+  "protocol_id": "llm-atlas-k3-attnres-forward-training-v1",
+  "status": "frozen-before-model-output",
+  "parent_protocol_id": "llm-atlas-k3-attnres-gradient-scale-v1",
+  "parent_manifest": "experiments/k3/attnres_gradient/manifest.json",
+  "architecture": "block",
+  "depth": 32,
+  "formal_steps": 8000,
+  "smoke_steps": 20,
+  "batch_size": 32,
+  "formal_seeds": [
+    2026073001,
+    2026073002,
+    2026073003
+  ],
+  "diagnostic_steps": [
+    0,
+    100,
+    500,
+    2000,
+    4000,
+    8000
+  ],
+  "primary_step": 8000,
+  "spike_layers_1based": [21, 22, 23, 24, 25],
+  "historical_reference": {
+    "directory": "experiments/k3/attnres_gradient/results/raw",
+    "filename_template": "formal-depth-32-block-seed-{seed}.json",
+    "identity": "historical-paired-reference-not-contemporaneous-randomized-control"
+  },
+  "variants": {
+    "uniform_group_6_forward": {
+      "selected_depth_indices": [40, 41, 42, 43, 44, 45, 46, 47]
+    },
+    "uniform_group_7_forward": {
+      "selected_depth_indices": [48, 49, 50, 51, 52, 53, 54, 55]
+    },
+    "uniform_groups_6_7_forward": {
+      "selected_depth_indices": [40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55]
+    },
+    "uniform_group_7_mlp_forward": {
+      "selected_depth_indices": [49, 51, 53, 55]
+    }
+  },
+  "smoke_only_variants": [
+    "learned_reference"
+  ],
+  "primary_variant": "uniform_groups_6_7_forward",
+  "replay": {
+    "variant": "uniform_groups_6_7_forward",
+    "seed": 2026073001
+  },
+  "selected_source_counts": {
+    "40": 6,
+    "41": 7,
+    "42": 7,
+    "43": 7,
+    "44": 7,
+    "45": 7,
+    "46": 7,
+    "47": 7,
+    "48": 7,
+    "49": 8,
+    "50": 8,
+    "51": 8,
+    "52": 8,
+    "53": 8,
+    "54": 8,
+    "55": 8
+  },
+  "thresholds": {
+    "positive_denominator_epsilon": 1e-30,
+    "material_relative_drop": 0.2,
+    "final_bpc_delta_per_seed_maximum": 0.05,
+    "final_bpc_delta_mean_maximum": 0.03,
+    "uniform_weight_max_abs_error": 1e-12,
+    "loss_scale_ratio_abs_error": 1e-5,
+    "loss_scale_shape_abs_error": 1e-6
+  },
+  "new_target_bytes": {
+    "per_cell": 65536000,
+    "formal_12_cells": 786432000,
+    "primary_replay": 65536000,
+    "total": 851968000
+  },
+  "historical_reference_target_bytes": 196608000,
+  "concurrency_maximum": 2
+}

experiments/k3/attnres_forward/reproduction.json

@@ -0,0 +1,25 @@
+{
+  "canonical_sha256_without_self": "57346df80c0d76bd1d306d5fa213feed74ac2094237c22ebcb49f16ea16437e0",
+  "excluded_fields": [
+    "run_kind",
+    "timing",
+    "self hashes",
+    "manifest path strings",
+    "GPU/version metadata"
+  ],
+  "formal_file_sha256": "0962ebd1a00a11e61ac795282bfa99412166c8752f2a3731f137030d7f134dc1",
+  "formal_variant": "uniform_groups_6_7_forward",
+  "passed": true,
+  "post_result_grok_review": {
+    "blocking_errors": 0,
+    "claim_boundary_confirmed": true,
+    "replay_confirmed": true,
+    "session_id": "019fb28b-a9e1-7643-8e43-06f5e16a2077",
+    "status_confirmed": true
+  },
+  "protocol_id": "llm-atlas-k3-attnres-forward-training-v1",
+  "replay_file_sha256": "b85ac8062b2b0b8b3f2305d4b22a7c0212466fbfb28a9d63099cadd0a6917c8e",
+  "schema_version": 1,
+  "scientific_payload_sha256": "b85563ca5cb53e60b39c3801d372376206105b8a089a8633b3e81973a7f0c051",
+  "seed": 2026073001
+}

experiments/k3/attnres_forward/results/gates/parent-equivalence.json

@@ -0,0 +1,39 @@
+{
+  "canonical_sha256_without_self": "99aeefc0ba35c199725ed7af377450ecbb5cc5f1aabdda9ac2345a30f03b444f",
+  "excluded_fields": [
+    "protocol wrapper fields",
+    "timing",
+    "self hash",
+    "parent runner self hash",
+    "study manifest"
+  ],
+  "field_checks": {
+    "architecture": true,
+    "batch_size": true,
+    "depth": true,
+    "diagnostics": true,
+    "environment": true,
+    "evaluations": true,
+    "gradient_gate": true,
+    "hashes": true,
+    "manifest": true,
+    "model": true,
+    "optimizer": true,
+    "seed": true,
+    "steps": true,
+    "target_bytes_seen": true,
+    "training_history": true
+  },
+  "gate": "empty-selector-parent-equivalence",
+  "parent_file": "/home/wuyang/Code/K3/experiments/k3/attnres_forward/results/gates/parent-smoke.json",
+  "passed": true,
+  "protocol_id": "llm-atlas-k3-attnres-forward-training-v1",
+  "schema_version": 1,
+  "wrapper_file": "/home/wuyang/Code/K3/experiments/k3/attnres_forward/results/gates/wrapper-smoke.json",
+  "wrapper_identity": {
+    "forward_audit": true,
+    "parent_protocol": true,
+    "protocol": true,
+    "variant": true
+  }
+}

experiments/k3/attnres_forward/run_matrix.py

@@ -0,0 +1,220 @@
+#!/usr/bin/env python3
+"""Run the frozen Round 08 matrix with at most two isolated processes."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import subprocess
+import time
+from pathlib import Path
+from typing import Any
+
+
+VARIANTS = (
+    "uniform_group_6_forward",
+    "uniform_group_7_forward",
+    "uniform_groups_6_7_forward",
+    "uniform_group_7_mlp_forward",
+)
+SEEDS = (2026073001, 2026073002, 2026073003)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--python", type=Path, required=True)
+    parser.add_argument("--cache-dir", type=Path, required=True)
+    parser.add_argument("--parent-manifest", type=Path, required=True)
+    parser.add_argument("--study-manifest", type=Path, required=True)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument(
+        "--phase", choices=("formal", "replay", "all"), default="all"
+    )
+    parser.add_argument("--concurrency", type=int, default=2)
+    return parser.parse_args()
+
+
+def cell_output(
+    output_dir: Path, variant: str, seed: int, run_kind: str
+) -> Path:
+    return output_dir / (
+        f"{run_kind}-{variant}-seed-{seed}.json"
+    )
+
+
+def command_for(
+    args: argparse.Namespace, variant: str, seed: int, run_kind: str
+) -> list[str]:
+    runner = Path(__file__).resolve().parent / "train.py"
+    return [
+        str(args.python),
+        str(runner),
+        "--variant",
+        variant,
+        "--study-manifest",
+        str(args.study_manifest),
+        "--run-kind",
+        run_kind,
+        "--architecture",
+        "block",
+        "--depth",
+        "32",
+        "--seed",
+        str(seed),
+        "--cache-dir",
+        str(args.cache_dir),
+        "--manifest",
+        str(args.parent_manifest),
+        "--output",
+        str(cell_output(args.output_dir, variant, seed, run_kind)),
+    ]
+
+
+def validate_manifest(args: argparse.Namespace) -> None:
+    manifest = json.loads(args.study_manifest.read_text())
+    if (
+        manifest["status"] != "frozen-before-model-output"
+        or tuple(manifest["variants"]) != VARIANTS
+        or tuple(manifest["formal_seeds"]) != SEEDS
+        or manifest["concurrency_maximum"] != 2
+    ):
+        raise RuntimeError("study manifest matrix/concurrency drift")
+    if args.concurrency < 1 or args.concurrency > 2:
+        raise ValueError("the frozen protocol permits one or two processes")
+
+
+def stop_processes(items: list[dict[str, Any]]) -> None:
+    for item in items:
+        if item["process"].poll() is None:
+            item["process"].terminate()
+    for item in items:
+        process = item["process"]
+        if process.poll() is not None:
+            continue
+        try:
+            process.wait(timeout=10)
+        except subprocess.TimeoutExpired:
+            process.kill()
+            process.wait()
+
+
+def quarantine_failed_output(
+    output_dir: Path, variant: str, seed: int, run_kind: str
+) -> str | None:
+    output = cell_output(output_dir, variant, seed, run_kind)
+    if not output.exists():
+        return None
+    failed = output.with_suffix(".failed.json")
+    if failed.exists():
+        failed = output.with_suffix(f".failed-{time.time_ns()}.json")
+    output.replace(failed)
+    return str(failed)
+
+
+def run_cells(
+    args: argparse.Namespace,
+    cells: list[tuple[str, int, str]],
+) -> None:
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+    for variant, seed, run_kind in cells:
+        output = cell_output(args.output_dir, variant, seed, run_kind)
+        if output.exists():
+            raise FileExistsError(
+                f"refusing to overwrite existing result: {output}"
+            )
+
+    environment = dict(os.environ)
+    environment["CUBLAS_WORKSPACE_CONFIG"] = ":4096:8"
+    pending = list(cells)
+    running: list[dict[str, Any]] = []
+    completed = 0
+    while pending or running:
+        while pending and len(running) < args.concurrency:
+            variant, seed, run_kind = pending.pop(0)
+            command = command_for(args, variant, seed, run_kind)
+            process = subprocess.Popen(command, env=environment)
+            running.append(
+                {
+                    "identity": (variant, seed, run_kind),
+                    "process": process,
+                    "started": time.monotonic(),
+                }
+            )
+            print(
+                json.dumps(
+                    {
+                        "event": "cell_started",
+                        "variant": variant,
+                        "seed": seed,
+                        "run_kind": run_kind,
+                        "pid": process.pid,
+                        "active": len(running),
+                        "remaining": len(pending),
+                    },
+                    sort_keys=True,
+                ),
+                flush=True,
+            )
+        time.sleep(1)
+        survivors = []
+        for item in running:
+            return_code = item["process"].poll()
+            if return_code is None:
+                survivors.append(item)
+                continue
+            variant, seed, run_kind = item["identity"]
+            elapsed = time.monotonic() - item["started"]
+            if return_code != 0:
+                stop_processes(
+                    [candidate for candidate in running if candidate is not item]
+                )
+                quarantined = quarantine_failed_output(
+                    args.output_dir, variant, seed, run_kind
+                )
+                raise RuntimeError(
+                    f"cell failed: {variant}/{seed}/{run_kind}: {return_code}; "
+                    f"quarantined_output={quarantined}"
+                )
+            completed += 1
+            print(
+                json.dumps(
+                    {
+                        "event": "cell_completed",
+                        "variant": variant,
+                        "seed": seed,
+                        "run_kind": run_kind,
+                        "elapsed_seconds": elapsed,
+                        "completed": completed,
+                        "total": len(cells),
+                    },
+                    sort_keys=True,
+                ),
+                flush=True,
+            )
+        running = survivors
+
+
+def main() -> None:
+    args = parse_args()
+    validate_manifest(args)
+    formal = [
+        (variant, seed, "formal")
+        for variant in VARIANTS
+        for seed in SEEDS
+    ]
+    replay = [
+        ("uniform_groups_6_7_forward", 2026073001, "replay")
+    ]
+    cells = (
+        formal
+        if args.phase == "formal"
+        else replay
+        if args.phase == "replay"
+        else formal + replay
+    )
+    run_cells(args, cells)
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_forward/train.py

@@ -0,0 +1,449 @@
+#!/usr/bin/env python3
+"""Run one preregistered Round 08 train-time uniform-forward cell."""
+
+from __future__ import annotations
+
+import importlib.util
+import json
+import math
+import os
+import sys
+from pathlib import Path
+from typing import Any
+
+import torch
+import torch.nn.functional as F
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-forward-training-v1"
+PARENT_PROTOCOL_ID = "llm-atlas-k3-attnres-gradient-scale-v1"
+VARIANTS = {
+    "learned_reference": (),
+    "uniform_group_6_forward": tuple(range(40, 48)),
+    "uniform_group_7_forward": tuple(range(48, 56)),
+    "uniform_groups_6_7_forward": tuple(range(40, 56)),
+    "uniform_group_7_mlp_forward": (49, 51, 53, 55),
+}
+FORMAL_VARIANTS = tuple(name for name in VARIANTS if name != "learned_reference")
+EXPECTED_SOURCE_COUNTS = {
+    **{40: 6},
+    **{index: 7 for index in range(41, 49)},
+    **{index: 8 for index in range(49, 56)},
+}
+
+
+def load_parent_module() -> Any:
+    path = Path(__file__).resolve().parents[1] / "attnres_gradient" / "train.py"
+    spec = importlib.util.spec_from_file_location("k3_attnres_round05_train", path)
+    if spec is None or spec.loader is None:
+        raise RuntimeError(f"cannot import Round 05 runner from {path}")
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[spec.name] = module
+    spec.loader.exec_module(module)
+    return module
+
+
+parent = load_parent_module()
+ACTIVE_VARIANT = "learned_reference"
+LAST_MODEL: ForwardInterventionLanguageModel | None = None
+LAST_OPTIMIZER: torch.optim.Optimizer | None = None
+
+
+def extract_wrapper_argument(name: str) -> str:
+    try:
+        index = sys.argv.index(name)
+    except ValueError as error:
+        raise ValueError(f"missing required wrapper argument: {name}") from error
+    if index + 1 >= len(sys.argv):
+        raise ValueError(f"missing value for wrapper argument: {name}")
+    value = sys.argv[index + 1]
+    del sys.argv[index : index + 2]
+    return value
+
+
+def argument_value(name: str, default: str | None = None) -> str | None:
+    try:
+        index = sys.argv.index(name)
+    except ValueError:
+        return default
+    if index + 1 >= len(sys.argv):
+        raise ValueError(f"missing value for argument: {name}")
+    return sys.argv[index + 1]
+
+
+def parameter_names_for_indices(indices: tuple[int, ...]) -> tuple[str, ...]:
+    names = []
+    for index in indices:
+        names.extend(
+            (
+                f"mixers.{index}.query",
+                f"mixers.{index}.key_norm.weight",
+            )
+        )
+    return tuple(names)
+
+
+class ForwardInterventionLanguageModel(parent.GradientLanguageModel):
+    """Round 05 model with one frozen selector and parameter-free uniform mixers."""
+
+    def __init__(self, architecture: str):
+        super().__init__(architecture)
+        global LAST_MODEL
+        if architecture != "block":
+            raise ValueError("Round 08 only permits the block architecture")
+        if ACTIVE_VARIANT not in VARIANTS:
+            raise ValueError(f"unknown Round 08 variant: {ACTIVE_VARIANT}")
+        self.forward_variant = ACTIVE_VARIANT
+        self.selected_indices = tuple(VARIANTS[ACTIVE_VARIANT])
+        self.selected_set = frozenset(self.selected_indices)
+        self.forward_calls = 0
+        self.depth_visits = [0] * len(self.mixers)
+        self.output_visits = 0
+        self.source_counts: dict[int, set[int]] = {
+            index: set() for index in range(len(self.mixers))
+        }
+        self.uniform_weight_max_abs_error = 0.0
+        selected_names = parameter_names_for_indices(self.selected_indices)
+        named_parameters = dict(self.named_parameters())
+        self.selected_initial_tensors = {
+            name: named_parameters[name].detach().cpu().clone()
+            for name in selected_names
+        }
+        self.gradient_hook_calls = {
+            name: 0
+            for name in named_parameters
+            if name.startswith("mixers.") or name.startswith("output_mixer.")
+        }
+        self._gradient_hooks = []
+        for name, parameter in named_parameters.items():
+            if name not in self.gradient_hook_calls:
+                continue
+
+            def count_hook(
+                gradient: torch.Tensor, *, parameter_name: str = name
+            ) -> torch.Tensor:
+                self.gradient_hook_calls[parameter_name] += 1
+                return gradient
+
+            self._gradient_hooks.append(parameter.register_hook(count_hook))
+        LAST_MODEL = self
+
+    def mix(
+        self,
+        mixer_index: int,
+        sources: list[torch.Tensor],
+        capture: bool,
+    ) -> tuple[torch.Tensor, dict[str, Any] | None]:
+        self.depth_visits[mixer_index] += 1
+        self.source_counts[mixer_index].add(len(sources))
+        if mixer_index not in self.selected_set:
+            return self.mixers[mixer_index](sources, capture)
+
+        values = torch.stack(sources, dim=0)
+        logits = torch.zeros(
+            values.shape[0],
+            values.shape[1],
+            values.shape[2],
+            dtype=torch.float32,
+            device=values.device,
+        )
+        weights = torch.softmax(logits, dim=0)
+        expected = torch.tensor(
+            1.0 / len(sources), dtype=weights.dtype, device=weights.device
+        )
+        error = (weights - expected).abs().max().detach().cpu().item()
+        self.uniform_weight_max_abs_error = max(
+            self.uniform_weight_max_abs_error, error
+        )
+        output = torch.einsum(
+            "nbt,nbtd->btd", weights, values.float()
+        ).to(values.dtype)
+        if not capture:
+            return output, None
+        entropy = -(weights * torch.log(weights.clamp_min(1e-30))).sum(dim=0)
+        return output, {
+            "mean_weights": weights.mean(dim=(1, 2)).detach().cpu().tolist(),
+            "entropy_mean": entropy.mean().detach().cpu().item(),
+            "sources": len(sources),
+        }
+
+    def forward(
+        self, input_ids: torch.Tensor, capture: bool = False
+    ) -> tuple[torch.Tensor, parent.ActivationTrace | None]:
+        self.forward_calls += 1
+        embedded = self.embed(input_ids)
+        trace = parent.ActivationTrace([], [], [], [], []) if capture else None
+        completed = [embedded]
+        partial: torch.Tensor | None = None
+        mixer_index = 0
+        for block in self.blocks:
+            for branch_index in range(2):
+                sources = completed + ([] if partial is None else [partial])
+                branch_input, weights = self.mix(
+                    mixer_index, sources, capture
+                )
+                mixer_index += 1
+                if branch_index == 0:
+                    branch_output = block.attention(
+                        block.attention_norm(branch_input)
+                    )
+                else:
+                    branch_output = block.mlp(block.mlp_norm(branch_input))
+                branch_for_residual = branch_output.float()
+                partial = (
+                    branch_for_residual
+                    if partial is None
+                    else partial + branch_for_residual
+                )
+                if trace is not None:
+                    trace.layer_input_rms.append(parent.rms(branch_input))
+                    trace.branch_output_rms.append(parent.rms(branch_output))
+                    trace.stream_state_rms.append(parent.rms(partial))
+                    trace.depth_weights.append(weights or {})
+                    if branch_index == 1:
+                        partial.retain_grad()
+                        trace.block_outputs.append(partial)
+                if mixer_index % parent.round04.SUBLAYERS_PER_BLOCK == 0:
+                    completed.append(partial)
+                    partial = None
+        if partial is not None or len(completed) != parent.BLOCK_GROUPS + 1:
+            raise RuntimeError("Round 08 Block AttnRes aggregation failed")
+        if self.output_mixer is None:
+            raise RuntimeError("Round 08 output mixer missing")
+        self.output_visits += 1
+        hidden, output_weights = self.output_mixer(completed, capture)
+        if trace is not None:
+            trace.output_weights = output_weights
+        normalized = self.final_norm(hidden)
+        logits = F.linear(normalized, self.token_embedding.weight)
+        return logits, trace
+
+
+def tensor_exact(left: torch.Tensor, right: torch.Tensor) -> bool:
+    return (
+        left.dtype == right.dtype
+        and tuple(left.shape) == tuple(right.shape)
+        and torch.equal(left.detach().cpu(), right.detach().cpu())
+    )
+
+
+def build_intervention_audit(
+    model: ForwardInterventionLanguageModel,
+    optimizer: torch.optim.Optimizer,
+    study_manifest: dict[str, Any],
+) -> dict[str, Any]:
+    selected = tuple(model.selected_indices)
+    selected_names = set(parameter_names_for_indices(selected))
+    mixer_parameters = {
+        name: parameter
+        for name, parameter in model.named_parameters()
+        if name.startswith("mixers.") or name.startswith("output_mixer.")
+    }
+    optimizer_parameters = {
+        parameter
+        for group in optimizer.param_groups
+        for parameter in group["params"]
+    }
+    selected_parameter_checks = {}
+    for name in sorted(selected_names):
+        parameter = mixer_parameters[name]
+        selected_parameter_checks[name] = {
+            "gradient_hook_calls": model.gradient_hook_calls[name],
+            "in_optimizer_param_group": parameter in optimizer_parameters,
+            "optimizer_state_present": parameter in optimizer.state,
+            "final_equals_initial": tensor_exact(
+                parameter, model.selected_initial_tensors[name]
+            ),
+        }
+    unselected_parameter_checks = {}
+    for name, parameter in sorted(mixer_parameters.items()):
+        if name in selected_names:
+            continue
+        unselected_parameter_checks[name] = {
+            "gradient_hook_calls": model.gradient_hook_calls[name],
+            "in_optimizer_param_group": parameter in optimizer_parameters,
+            "optimizer_state_present": parameter in optimizer.state,
+        }
+
+    source_counts = {
+        str(index): sorted(values)
+        for index, values in model.source_counts.items()
+    }
+    selected_source_gate = {
+        str(index): (
+            source_counts[str(index)]
+            == [study_manifest["selected_source_counts"][str(index)]]
+            == [EXPECTED_SOURCE_COUNTS[index]]
+        )
+        for index in selected
+    }
+    visit_gate = (
+        model.forward_calls > 0
+        and all(value == model.forward_calls for value in model.depth_visits)
+        and model.output_visits == model.forward_calls
+    )
+    selected_parameter_gate = all(
+        check["gradient_hook_calls"] == 0
+        and check["in_optimizer_param_group"]
+        and not check["optimizer_state_present"]
+        and check["final_equals_initial"]
+        for check in selected_parameter_checks.values()
+    )
+    unselected_parameter_gate = all(
+        check["gradient_hook_calls"] > 0
+        and check["in_optimizer_param_group"]
+        and check["optimizer_state_present"]
+        for check in unselected_parameter_checks.values()
+    )
+    expected_selected = tuple(
+        study_manifest["variants"]
+        .get(model.forward_variant, {"selected_depth_indices": []})[
+            "selected_depth_indices"
+        ]
+    )
+    selector_gate = (
+        selected == expected_selected
+        and 64 not in selected
+        and selected_source_gate == {
+            str(index): True for index in selected
+        }
+    )
+    threshold = study_manifest["thresholds"][
+        "uniform_weight_max_abs_error"
+    ]
+    uniform_gate = model.uniform_weight_max_abs_error <= threshold
+    passed = (
+        visit_gate
+        and selector_gate
+        and selected_parameter_gate
+        and unselected_parameter_gate
+        and uniform_gate
+    )
+    return {
+        "passed": passed,
+        "variant": model.forward_variant,
+        "selected_depth_indices": list(selected),
+        "output_mixer_selected": False,
+        "forward_calls": model.forward_calls,
+        "depth_visit_counts": model.depth_visits,
+        "output_visit_count": model.output_visits,
+        "visit_gate": visit_gate,
+        "source_counts_by_depth_index": source_counts,
+        "selected_source_count_checks": selected_source_gate,
+        "selector_gate": selector_gate,
+        "uniform_weight_max_abs_error": model.uniform_weight_max_abs_error,
+        "uniform_weight_threshold": threshold,
+        "uniform_weight_gate": uniform_gate,
+        "selected_parameters": selected_parameter_checks,
+        "selected_parameter_reachability_gate": selected_parameter_gate,
+        "unselected_parameters": unselected_parameter_checks,
+        "unselected_parameter_reachability_gate": unselected_parameter_gate,
+        "semantics": (
+            "selected depth mixers use parameter-free constant-zero logits "
+            "with the parent softmax+einsum arithmetic kernel"
+        ),
+    }
+
+
+def rewrite_result(
+    output_path: Path,
+    study_manifest_path: Path,
+    study_manifest: dict[str, Any],
+) -> None:
+    if LAST_MODEL is None or LAST_OPTIMIZER is None:
+        raise RuntimeError("runner capture state missing")
+    result = json.loads(output_path.read_text())
+    parent_self_hash = result.pop("canonical_sha256_without_self")
+    if result["protocol_id"] != PARENT_PROTOCOL_ID:
+        raise RuntimeError("parent runner protocol drift")
+    result["schema_version"] = 2
+    result["protocol_id"] = PROTOCOL_ID
+    result["parent_protocol_id"] = PARENT_PROTOCOL_ID
+    result["variant"] = ACTIVE_VARIANT
+    result["parent_runner_canonical_sha256"] = parent_self_hash
+    result["study_manifest"] = {
+        "path": str(study_manifest_path),
+        "file_sha256": parent.file_sha256(study_manifest_path),
+        "status": study_manifest["status"],
+    }
+    result["forward_intervention"] = build_intervention_audit(
+        LAST_MODEL, LAST_OPTIMIZER, study_manifest
+    )
+    result["canonical_sha256_without_self"] = parent.canonical_sha256(result)
+    temporary = output_path.with_suffix(output_path.suffix + ".round08.tmp")
+    temporary.write_text(
+        json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    os.replace(temporary, output_path)
+    if not result["forward_intervention"]["passed"]:
+        raise RuntimeError(
+            f"forward intervention audit failed: "
+            f"{result['forward_intervention']}"
+        )
+
+
+def main() -> None:
+    global ACTIVE_VARIANT, LAST_OPTIMIZER
+    variant = extract_wrapper_argument("--variant")
+    study_manifest_path = Path(
+        extract_wrapper_argument("--study-manifest")
+    ).resolve()
+    if variant not in VARIANTS:
+        raise ValueError(f"unknown variant: {variant}")
+    run_kind = argument_value("--run-kind", "formal")
+    if run_kind in ("formal", "replay") and variant not in FORMAL_VARIANTS:
+        raise ValueError("learned_reference is smoke-only")
+    if argument_value("--architecture") != "block":
+        raise ValueError("Round 08 requires --architecture block")
+    if argument_value("--depth") != "32":
+        raise ValueError("Round 08 requires --depth 32")
+    if run_kind == "replay" and variant != "uniform_groups_6_7_forward":
+        raise ValueError("the frozen replay uses the primary joint variant")
+
+    study_manifest = json.loads(study_manifest_path.read_text())
+    if (
+        study_manifest["protocol_id"] != PROTOCOL_ID
+        or study_manifest["status"] != "frozen-before-model-output"
+    ):
+        raise ValueError("study manifest is not the frozen Round 08 contract")
+    expected = tuple(
+        study_manifest["variants"]
+        .get(variant, {"selected_depth_indices": []})[
+            "selected_depth_indices"
+        ]
+    )
+    if expected != VARIANTS[variant]:
+        raise ValueError("study manifest selector drift")
+    for index, source_count in EXPECTED_SOURCE_COUNTS.items():
+        if (
+            study_manifest["selected_source_counts"].get(str(index))
+            != source_count
+        ):
+            raise ValueError(
+                f"study manifest source-count drift at depth index {index}"
+            )
+
+    output_value = argument_value("--output")
+    if output_value is None:
+        raise ValueError("--output is required")
+    output_path = Path(output_value).resolve()
+    ACTIVE_VARIANT = variant
+    parent.GradientLanguageModel = ForwardInterventionLanguageModel
+
+    original_adamw = torch.optim.AdamW
+
+    def capture_adamw(*args: Any, **kwargs: Any) -> torch.optim.Optimizer:
+        global LAST_OPTIMIZER
+        LAST_OPTIMIZER = original_adamw(*args, **kwargs)
+        return LAST_OPTIMIZER
+
+    torch.optim.AdamW = capture_adamw  # type: ignore[assignment]
+    try:
+        parent.main()
+    finally:
+        torch.optim.AdamW = original_adamw  # type: ignore[assignment]
+    rewrite_result(output_path, study_manifest_path, study_manifest)
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_forward/verify.py

@@ -0,0 +1,294 @@
+#!/usr/bin/env python3
+"""Run pre-result Round 08 identity gates."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import importlib.util
+import json
+import os
+import sys
+from pathlib import Path
+from typing import Any
+
+import torch
+
+
+def load_runner() -> Any:
+    path = Path(__file__).resolve().parent / "train.py"
+    spec = importlib.util.spec_from_file_location("k3_attnres_round08_train", path)
+    if spec is None or spec.loader is None:
+        raise RuntimeError(f"cannot import Round 08 runner from {path}")
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[spec.name] = module
+    spec.loader.exec_module(module)
+    return module
+
+
+runner = load_runner()
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    subparsers = parser.add_subparsers(dest="command", required=True)
+
+    step_zero = subparsers.add_parser("step-zero")
+    step_zero.add_argument("--cache-dir", type=Path, required=True)
+    step_zero.add_argument("--parent-manifest", type=Path, required=True)
+    step_zero.add_argument("--study-manifest", type=Path, required=True)
+    step_zero.add_argument("--output", type=Path, required=True)
+    step_zero.add_argument("--seed", type=int, default=2026073001)
+
+    smoke = subparsers.add_parser("smoke-compare")
+    smoke.add_argument("--parent", type=Path, required=True)
+    smoke.add_argument("--wrapper", type=Path, required=True)
+    smoke.add_argument("--output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def canonical_sha256(value: Any) -> str:
+    return hashlib.sha256(
+        json.dumps(
+            value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+        ).encode()
+    ).hexdigest()
+
+
+def tensor_sha256(value: torch.Tensor) -> str:
+    return hashlib.sha256(runner.parent.tensor_bytes(value)).hexdigest()
+
+
+def read_and_verify(path: Path) -> dict[str, Any]:
+    value = json.loads(path.read_text())
+    expected = value["canonical_sha256_without_self"]
+    payload = {
+        key: item
+        for key, item in value.items()
+        if key != "canonical_sha256_without_self"
+    }
+    if canonical_sha256(payload) != expected:
+        raise RuntimeError(f"canonical self-hash failed: {path}")
+    return value
+
+
+def smoke_compare(args: argparse.Namespace) -> None:
+    parent_result = read_and_verify(args.parent)
+    wrapper_result = read_and_verify(args.wrapper)
+    fields = (
+        "architecture",
+        "depth",
+        "seed",
+        "steps",
+        "batch_size",
+        "target_bytes_seen",
+        "manifest",
+        "model",
+        "optimizer",
+        "hashes",
+        "evaluations",
+        "diagnostics",
+        "training_history",
+        "gradient_gate",
+        "environment",
+    )
+    checks = {}
+    for field in fields:
+        parent_value = exact_structure(parent_result[field])
+        wrapper_value = exact_structure(wrapper_result[field])
+        if field == "manifest":
+            parent_value.pop("path", None)
+            wrapper_value.pop("path", None)
+        checks[field] = parent_value == wrapper_value
+    wrapper_identity = {
+        "protocol": wrapper_result["protocol_id"] == runner.PROTOCOL_ID,
+        "parent_protocol": (
+            wrapper_result["parent_protocol_id"]
+            == runner.PARENT_PROTOCOL_ID
+        ),
+        "variant": wrapper_result["variant"] == "learned_reference",
+        "forward_audit": wrapper_result["forward_intervention"]["passed"],
+    }
+    passed = all(checks.values()) and all(wrapper_identity.values())
+    result = {
+        "schema_version": 1,
+        "protocol_id": runner.PROTOCOL_ID,
+        "gate": "empty-selector-parent-equivalence",
+        "passed": passed,
+        "field_checks": checks,
+        "wrapper_identity": wrapper_identity,
+        "excluded_fields": [
+            "protocol wrapper fields",
+            "timing",
+            "self hash",
+            "parent runner self hash",
+            "study manifest",
+        ],
+        "parent_file": str(args.parent.resolve()),
+        "wrapper_file": str(args.wrapper.resolve()),
+    }
+    result["canonical_sha256_without_self"] = canonical_sha256(result)
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(
+        json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    if not passed:
+        raise RuntimeError(f"empty-selector parent equivalence failed: {checks}")
+
+
+def exact_structure(value: Any) -> Any:
+    return json.loads(
+        json.dumps(value, ensure_ascii=False, sort_keys=True)
+    )
+
+
+def step_zero(args: argparse.Namespace) -> None:
+    if not torch.cuda.is_available():
+        raise RuntimeError("CUDA is required by the frozen step-zero gate")
+    study_manifest = json.loads(args.study_manifest.read_text())
+    if study_manifest["protocol_id"] != runner.PROTOCOL_ID:
+        raise RuntimeError("study manifest mismatch")
+    parent_manifest = json.loads(args.parent_manifest.read_text())
+    if parent_manifest["protocol_id"] != runner.PARENT_PROTOCOL_ID:
+        raise RuntimeError("parent manifest mismatch")
+
+    parent = runner.parent
+    parent.configure_round04_globals(32)
+    device = torch.device("cuda")
+    corpus = parent.round04.ByteCorpus(
+        args.cache_dir, parent_manifest, device
+    )
+    inputs, targets = corpus.fixed_batch(
+        corpus.diagnostic_starts, 0, 16
+    )
+
+    variants = ("learned_reference",) + tuple(
+        study_manifest["variants"].keys()
+    )
+    observations: dict[str, Any] = {}
+    reference_payload: dict[str, Any] | None = None
+    for variant in variants:
+        parent.configure_determinism(args.seed)
+        runner.ACTIVE_VARIANT = variant
+        model = runner.ForwardInterventionLanguageModel("block").to(device)
+        initial_public = parent.named_state_hash(
+            model, include_mixers=False
+        )
+        initial_mixer = parent.named_state_hash(
+            model, include_mixers=True
+        )
+        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+            logits, trace = model(inputs, capture=True)
+            loss = parent.cross_entropy(logits, targets)
+        if trace is None:
+            raise RuntimeError("step-zero trace missing")
+        evaluation = parent.evaluate(model, corpus, 64, 8)
+        diagnostic = parent.diagnostic(model, corpus, 16)
+        payload = {
+            "initial_public_hash": initial_public,
+            "initial_mixer_hash": initial_mixer,
+            "logits_sha256": tensor_sha256(logits),
+            "loss_nats": loss.detach().cpu().item(),
+            "loss_tensor_sha256": tensor_sha256(loss),
+            "evaluation": exact_structure(evaluation),
+            "diagnostic": exact_structure(diagnostic),
+        }
+        if reference_payload is None:
+            reference_payload = payload
+        exact_checks = {
+            key: payload[key] == reference_payload[key]
+            for key in payload
+        }
+        selected = tuple(runner.VARIANTS[variant])
+        capture_checks = {}
+        for index in selected:
+            summary = trace.depth_weights[index]
+            source_count = summary["sources"]
+            capture_checks[str(index)] = {
+                "source_count": source_count,
+                "expected_source_count": study_manifest[
+                    "selected_source_counts"
+                ][str(index)],
+                "capture_summary_exact_vs_learned": (
+                    payload["diagnostic"]["depth_weights"][index]
+                    == reference_payload["diagnostic"]["depth_weights"][index]
+                ),
+                "passed": (
+                    source_count
+                    == study_manifest["selected_source_counts"][str(index)]
+                    and payload["diagnostic"]["depth_weights"][index]
+                    == reference_payload["diagnostic"]["depth_weights"][index]
+                ),
+            }
+        runtime_uniform_gate = (
+            model.uniform_weight_max_abs_error
+            <= study_manifest["thresholds"][
+                "uniform_weight_max_abs_error"
+            ]
+        )
+        observations[variant] = {
+            "payload": payload,
+            "exact_vs_learned_reference": exact_checks,
+            "selected_capture_checks": capture_checks,
+            "pre_reduction_uniform_weight_max_abs_error": (
+                model.uniform_weight_max_abs_error
+            ),
+            "pre_reduction_uniform_weight_gate": runtime_uniform_gate,
+            "passed": (
+                all(exact_checks.values())
+                and all(
+                    item["passed"] for item in capture_checks.values()
+                )
+                and runtime_uniform_gate
+            ),
+        }
+        del model, logits, loss, trace
+        torch.cuda.empty_cache()
+
+    passed = all(item["passed"] for item in observations.values())
+    result = {
+        "schema_version": 1,
+        "protocol_id": runner.PROTOCOL_ID,
+        "gate": "step-zero-cross-variant-byte-exact",
+        "seed": args.seed,
+        "passed": passed,
+        "variants": observations,
+        "parent_manifest_sha256": runner.parent.file_sha256(
+            args.parent_manifest
+        ),
+        "study_manifest_sha256": runner.parent.file_sha256(
+            args.study_manifest
+        ),
+        "environment": {
+            "gpu": torch.cuda.get_device_name(0),
+            "torch": torch.__version__,
+            "cuda": torch.version.cuda,
+            "cublas_workspace_config": os.environ.get(
+                "CUBLAS_WORKSPACE_CONFIG"
+            ),
+        },
+    }
+    result["canonical_sha256_without_self"] = canonical_sha256(result)
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(
+        json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    if not passed:
+        failed = [
+            name
+            for name, value in observations.items()
+            if not value["passed"]
+        ]
+        raise RuntimeError(f"step-zero exactness failed: {failed}")
+
+
+def main() -> None:
+    args = parse_args()
+    if args.command == "step-zero":
+        step_zero(args)
+    else:
+        smoke_compare(args)
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_gradient/README.md

@@ -0,0 +1,57 @@
+# Attention Residuals activation-gradient/depth study
+
+This directory implements preregistered protocol
+`llm-atlas-k3-attnres-gradient-scale-v1`:
+
+- `research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md`
+- `research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md`
+
+It is an independent reduced mechanism experiment. It is not a Kimi K3
+checkpoint forward pass and does not claim to recover the paper's unpublished
+Figure 5 telemetry definition.
+
+## Frozen environment
+
+```text
+Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
+PyTorch 2.11.0+cu128
+GPU NVIDIA GeForce RTX 5090
+CUBLAS_WORKSPACE_CONFIG=:4096:8
+```
+
+## Build the manifest
+
+```bash
+python experiments/k3/attnres_gradient/build_dataset.py \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
+  --manifest experiments/k3/attnres_gradient/manifest.json
+```
+
+## Run a smoke cell
+
+```bash
+CUBLAS_WORKSPACE_CONFIG=:4096:8 \
+python experiments/k3/attnres_gradient/train.py \
+  --run-kind smoke \
+  --architecture block \
+  --depth 32 \
+  --seed 2026073001 \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
+  --manifest experiments/k3/attnres_gradient/manifest.json \
+  --output /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1/smoke-a/depth-32-block.json
+```
+
+Smoke is fixed to 20 steps. Formal and replay runs are fixed to 8,000 steps;
+the runner rejects alternative budgets. The same command uses
+`--run-kind formal` or `--run-kind replay` and omits an explicit `--steps`.
+
+Formal output keys use:
+
+```text
+formal/depth-{16|32}-{baseline|block}-seed-{seed}.json
+replay/depth-32-block-seed-2026073001.json
+```
+
+Raw parquet/binary files and working runs remain in the local cache. The
+manifest, runner, complete result JSON, compact website payload, reproduction
+hashes, protocol, and audit enter the public repository.

experiments/k3/attnres_gradient/analyze.py

@@ -0,0 +1,685 @@
+#!/usr/bin/env python3
+"""Validate, aggregate, and publish K3 AttnRes Round 05 experiment data."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import math
+import os
+import shutil
+import statistics
+from pathlib import Path
+from typing import Any, Iterable
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-gradient-scale-v1"
+ARCHITECTURES = ("baseline", "block")
+DEPTHS = (16, 32)
+SEEDS = (2026073001, 2026073002, 2026073003)
+STEPS = (0, 100, 500, 2000, 4000, 8000)
+FORMAL_STEPS = 8000
+FORMAL_BATCH = 32
+TARGET_BYTES_PER_RUN = 65_536_000
+EXPECTED_TOTAL_TARGET_BYTES = 786_432_000
+SMOKE_COMPARE_FIELDS = (
+    "protocol_id",
+    "run_kind",
+    "architecture",
+    "depth",
+    "seed",
+    "steps",
+    "batch_size",
+    "target_bytes_seen",
+    "manifest",
+    "model",
+    "optimizer",
+    "hashes",
+    "evaluations",
+    "diagnostics",
+    "training_history",
+    "gradient_gate",
+    "environment",
+)
+REPLAY_COMPARE_FIELDS = tuple(
+    field for field in SMOKE_COMPARE_FIELDS if field != "run_kind"
+)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--formal-dir", type=Path, required=True)
+    parser.add_argument("--smoke-a-dir", type=Path, required=True)
+    parser.add_argument("--smoke-b-dir", type=Path, required=True)
+    parser.add_argument("--replay", type=Path, required=True)
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument("--raw-output-dir", type=Path, required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument("--compact-output", type=Path, required=True)
+    parser.add_argument("--reproduction-output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def read_json(path: Path) -> dict[str, Any]:
+    return json.loads(path.read_text())
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def canonical_sha256(value: Any) -> str:
+    payload = json.dumps(
+        value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+    ).encode()
+    return hashlib.sha256(payload).hexdigest()
+
+
+def atomic_json(path: Path, value: dict[str, Any]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(path.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    os.replace(temporary, path)
+
+
+def mean(values: Iterable[float]) -> float:
+    return statistics.fmean(values)
+
+
+def require_finite(value: Any, path: str = "root") -> None:
+    if isinstance(value, float):
+        if not math.isfinite(value):
+            raise ValueError(f"non-finite float at {path}")
+    elif isinstance(value, dict):
+        for key, child in value.items():
+            require_finite(child, f"{path}.{key}")
+    elif isinstance(value, list):
+        for index, child in enumerate(value):
+            require_finite(child, f"{path}[{index}]")
+
+
+def selected(run: dict[str, Any], fields: tuple[str, ...]) -> dict[str, Any]:
+    return {field: run[field] for field in fields}
+
+
+def final_diagnostic(run: dict[str, Any]) -> dict[str, Any]:
+    diagnostic = run["diagnostics"][-1]
+    if diagnostic["step"] != FORMAL_STEPS:
+        raise ValueError("final diagnostic is not step 8000")
+    return diagnostic
+
+
+def validate_run(
+    run: dict[str, Any],
+    *,
+    path: Path,
+    depth: int,
+    architecture: str,
+    seed: int,
+    manifest: dict[str, Any],
+    manifest_hash: str,
+) -> None:
+    if run["protocol_id"] != PROTOCOL_ID or run["run_kind"] != "formal":
+        raise ValueError(f"formal protocol/kind mismatch: {path}")
+    if (
+        run["depth"] != depth
+        or run["architecture"] != architecture
+        or run["seed"] != seed
+    ):
+        raise ValueError(f"formal identity mismatch: {path}")
+    if (
+        run["steps"] != FORMAL_STEPS
+        or run["batch_size"] != FORMAL_BATCH
+        or run["target_bytes_seen"] != TARGET_BYTES_PER_RUN
+    ):
+        raise ValueError(f"formal budget mismatch: {path}")
+    if run["manifest"]["file_sha256"] != manifest_hash:
+        raise ValueError(f"manifest file hash mismatch: {path}")
+    for key in (
+        "formal_schedule_sha256",
+        "validation_tensor_sha256",
+        "diagnostic_tensor_sha256",
+    ):
+        if run["manifest"][key] != manifest["windows"][key]:
+            raise ValueError(f"manifest {key} mismatch: {path}")
+    if [row["step"] for row in run["evaluations"]] != list(STEPS):
+        raise ValueError(f"evaluation steps mismatch: {path}")
+    if [row["step"] for row in run["diagnostics"]] != list(STEPS):
+        raise ValueError(f"diagnostic steps mismatch: {path}")
+    if run["model"]["layers"] != depth:
+        raise ValueError(f"model depth mismatch: {path}")
+
+    for diagnostic in run["diagnostics"]:
+        capture = diagnostic["capture"]
+        if (
+            capture["count"] != depth
+            or capture["shape"] != [16, 256, 192]
+            or set(capture["dtypes"]) != {"torch.float32"}
+            or not capture["all_gradients_finite"]
+            or not capture["all_gradients_present"]
+            or not capture["storage_unique"]
+        ):
+            raise ValueError(f"activation capture gate mismatch: {path}")
+        for field in (
+            "activation_grad_rms_by_block",
+            "activation_output_rms_by_block",
+            "core_parameter_grad_rms_by_block",
+        ):
+            if len(diagnostic[field]) != depth:
+                raise ValueError(f"{field} length mismatch: {path}")
+        for field in (
+            "layer_input_rms_by_sublayer",
+            "branch_output_rms_by_sublayer",
+            "stream_state_rms_by_sublayer",
+        ):
+            if len(diagnostic[field]) != depth * 2:
+                raise ValueError(f"{field} length mismatch: {path}")
+        if architecture == "baseline":
+            if diagnostic["depth_weights"] or diagnostic["output_weights"] is not None:
+                raise ValueError(f"unexpected Baseline mixer trace: {path}")
+        else:
+            if (
+                len(diagnostic["depth_weights"]) != depth * 2
+                or diagnostic["output_weights"]["sources"] != 9
+            ):
+                raise ValueError(f"Block mixer trace mismatch: {path}")
+    require_finite(run, path.name)
+
+
+def relative_reduction(baseline: float, block: float) -> float:
+    return (baseline - block) / baseline
+
+
+def depth_verdict(rows: list[dict[str, Any]]) -> dict[str, Any]:
+    cv_reductions = [row["relative_cv_reduction"] for row in rows]
+    imbalance_reductions = [
+        row["relative_imbalance_reduction"] for row in rows
+    ]
+    mean_cv = mean(cv_reductions)
+    mean_imbalance = mean(imbalance_reductions)
+    support = (
+        all(value > 0 for value in cv_reductions)
+        and mean_cv >= 0.20
+        and all(value > 0 for value in imbalance_reductions)
+        and mean_imbalance >= 0.20
+    )
+    concern = (
+        all(value < 0 for value in cv_reductions)
+        and -mean_cv >= 0.20
+        and all(value < 0 for value in imbalance_reductions)
+        and -mean_imbalance >= 0.20
+    )
+    if support:
+        label = "joint directional support at this depth"
+    elif concern:
+        label = "joint directional concern at this depth"
+    else:
+        label = "mixed / inconclusive at this depth"
+    return {
+        "label": label,
+        "threshold_relative": 0.20,
+        "cv_reductions": cv_reductions,
+        "mean_cv_reduction": mean_cv,
+        "imbalance_reductions": imbalance_reductions,
+        "mean_imbalance_reduction": mean_imbalance,
+        "all_cv_improve": all(value > 0 for value in cv_reductions),
+        "all_imbalance_improve": all(
+            value > 0 for value in imbalance_reductions
+        ),
+    }
+
+
+def summarize_depth(
+    depth: int, runs: dict[tuple[int, str, int], dict[str, Any]]
+) -> dict[str, Any]:
+    rows = []
+    for seed in SEEDS:
+        baseline = runs[(depth, "baseline", seed)]
+        block = runs[(depth, "block", seed)]
+        baseline_diagnostic = final_diagnostic(baseline)
+        block_diagnostic = final_diagnostic(block)
+        baseline_activation = baseline_diagnostic[
+            "activation_grad_statistics"
+        ]
+        block_activation = block_diagnostic["activation_grad_statistics"]
+        baseline_bpc = baseline["evaluations"][-1]["bits_per_byte"]
+        block_bpc = block["evaluations"][-1]["bits_per_byte"]
+        rows.append(
+            {
+                "seed": seed,
+                "baseline_bpc": baseline_bpc,
+                "block_bpc": block_bpc,
+                "block_minus_baseline_bpc": block_bpc - baseline_bpc,
+                "baseline_activation_grad_mean": baseline_activation["mean"],
+                "block_activation_grad_mean": block_activation["mean"],
+                "block_to_baseline_activation_grad_mean": (
+                    block_activation["mean"] / baseline_activation["mean"]
+                ),
+                "baseline_cv": baseline_activation["population_cv"],
+                "block_cv": block_activation["population_cv"],
+                "relative_cv_reduction": relative_reduction(
+                    baseline_activation["population_cv"],
+                    block_activation["population_cv"],
+                ),
+                "baseline_first_to_last_ratio": baseline_activation[
+                    "first_to_last_ratio"
+                ],
+                "block_first_to_last_ratio": block_activation[
+                    "first_to_last_ratio"
+                ],
+                "baseline_imbalance": baseline_activation[
+                    "imbalance_abs_log_ratio"
+                ],
+                "block_imbalance": block_activation[
+                    "imbalance_abs_log_ratio"
+                ],
+                "relative_imbalance_reduction": relative_reduction(
+                    baseline_activation["imbalance_abs_log_ratio"],
+                    block_activation["imbalance_abs_log_ratio"],
+                ),
+                "baseline_parameter_grad_cv": baseline_diagnostic[
+                    "core_parameter_grad_statistics"
+                ]["population_cv"],
+                "block_parameter_grad_cv": block_diagnostic[
+                    "core_parameter_grad_statistics"
+                ]["population_cv"],
+            }
+        )
+    verdict = depth_verdict(rows)
+    return {
+        "depth": depth,
+        "by_seed": rows,
+        "means": {
+            "baseline_bpc": mean(row["baseline_bpc"] for row in rows),
+            "block_bpc": mean(row["block_bpc"] for row in rows),
+            "block_minus_baseline_bpc": mean(
+                row["block_minus_baseline_bpc"] for row in rows
+            ),
+            "baseline_cv": mean(row["baseline_cv"] for row in rows),
+            "block_cv": mean(row["block_cv"] for row in rows),
+            "relative_cv_reduction": mean(
+                row["relative_cv_reduction"] for row in rows
+            ),
+            "baseline_imbalance": mean(
+                row["baseline_imbalance"] for row in rows
+            ),
+            "block_imbalance": mean(
+                row["block_imbalance"] for row in rows
+            ),
+            "relative_imbalance_reduction": mean(
+                row["relative_imbalance_reduction"] for row in rows
+            ),
+            "block_to_baseline_activation_grad_mean": mean(
+                row["block_to_baseline_activation_grad_mean"] for row in rows
+            ),
+            "baseline_parameter_grad_cv": mean(
+                row["baseline_parameter_grad_cv"] for row in rows
+            ),
+            "block_parameter_grad_cv": mean(
+                row["block_parameter_grad_cv"] for row in rows
+            ),
+        },
+        "verdict": verdict,
+    }
+
+
+def compact_cell(run: dict[str, Any]) -> dict[str, Any]:
+    diagnostics = []
+    for row in run["diagnostics"]:
+        compact = {
+            "step": row["step"],
+            "loss_nats": row["loss_nats"],
+            "bits_per_byte": row["bits_per_byte"],
+            "activation_grad_rms_by_block": row[
+                "activation_grad_rms_by_block"
+            ],
+            "activation_grad_statistics": row[
+                "activation_grad_statistics"
+            ],
+            "activation_output_rms_by_block": row[
+                "activation_output_rms_by_block"
+            ],
+            "activation_output_statistics": row[
+                "activation_output_statistics"
+            ],
+            "core_parameter_grad_rms_by_block": row[
+                "core_parameter_grad_rms_by_block"
+            ],
+            "core_parameter_grad_statistics": row[
+                "core_parameter_grad_statistics"
+            ],
+        }
+        if row["depth_weights"]:
+            compact["depth_weights"] = row["depth_weights"]
+            compact["output_weights"] = row["output_weights"]
+        diagnostics.append(compact)
+    return {
+        "architecture": run["architecture"],
+        "depth": run["depth"],
+        "seed": run["seed"],
+        "evaluations": run["evaluations"],
+        "diagnostics": diagnostics,
+        "timing": run["timing"],
+        "parameters": run["model"]["parameters"],
+        "hashes": run["hashes"],
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    manifest = read_json(args.manifest)
+    manifest_hash = file_sha256(args.manifest)
+    if manifest["protocol_id"] != PROTOCOL_ID:
+        raise ValueError("manifest protocol mismatch")
+    if (
+        manifest["windows"]["formal_schedule_cells"] != 768_000
+        or manifest["windows"]["formal_steps"] != FORMAL_STEPS
+        or manifest["windows"]["formal_batch"] != FORMAL_BATCH
+    ):
+        raise ValueError("manifest schedule budget mismatch")
+
+    runs: dict[tuple[int, str, int], dict[str, Any]] = {}
+    source_paths: dict[str, Path] = {}
+    formal_hashes: dict[str, str] = {}
+    for depth in DEPTHS:
+        for architecture in ARCHITECTURES:
+            for seed in SEEDS:
+                name = f"depth-{depth}-{architecture}-seed-{seed}.json"
+                path = args.formal_dir / name
+                run = read_json(path)
+                validate_run(
+                    run,
+                    path=path,
+                    depth=depth,
+                    architecture=architecture,
+                    seed=seed,
+                    manifest=manifest,
+                    manifest_hash=manifest_hash,
+                )
+                runs[(depth, architecture, seed)] = run
+                public_name = f"formal-{name}"
+                source_paths[public_name] = path
+                formal_hashes[public_name] = file_sha256(path)
+
+    if sum(run["target_bytes_seen"] for run in runs.values()) != (
+        EXPECTED_TOTAL_TARGET_BYTES
+    ):
+        raise ValueError("formal total target-byte budget mismatch")
+
+    common_initial_exact: dict[str, Any] = {}
+    input_gate_exact: dict[str, Any] = {}
+    for depth in DEPTHS:
+        for seed in SEEDS:
+            baseline = runs[(depth, "baseline", seed)]
+            block = runs[(depth, "block", seed)]
+            public_fields = (
+                "initial_public_parameter_structure",
+                "initial_public_parameter_tensors",
+                "initial_public_parameter_elements",
+                "initial_public_parameters",
+            )
+            exact = all(
+                baseline["hashes"][field] == block["hashes"][field]
+                for field in public_fields
+            )
+            gate_exact = (
+                baseline["manifest"]["input_gate_tensor_hashes"]
+                == block["manifest"]["input_gate_tensor_hashes"]
+            )
+            key = f"depth-{depth}-seed-{seed}"
+            common_initial_exact[key] = {
+                "exact": exact,
+                "baseline": {
+                    field: baseline["hashes"][field] for field in public_fields
+                },
+                "block": {
+                    field: block["hashes"][field] for field in public_fields
+                },
+            }
+            input_gate_exact[key] = {
+                "exact": gate_exact,
+                "hashes": baseline["manifest"]["input_gate_tensor_hashes"],
+            }
+            if not exact or not gate_exact:
+                raise ValueError(f"paired equality gate failed: {key}")
+
+    smoke_exact: dict[str, Any] = {}
+    smoke_hashes: dict[str, str] = {}
+    for depth in DEPTHS:
+        for architecture in ARCHITECTURES:
+            name = f"depth-{depth}-{architecture}.json"
+            left_path = args.smoke_a_dir / name
+            right_path = args.smoke_b_dir / name
+            left = read_json(left_path)
+            right = read_json(right_path)
+            left_selected = selected(left, SMOKE_COMPARE_FIELDS)
+            right_selected = selected(right, SMOKE_COMPARE_FIELDS)
+            exact = left_selected == right_selected
+            if (
+                not exact
+                or left["run_kind"] != "smoke"
+                or left["steps"] != 20
+                or not left["gradient_gate"]["passed"]
+            ):
+                raise ValueError(f"smoke gate failed: {name}")
+            key = f"depth-{depth}-{architecture}"
+            smoke_exact[key] = {
+                "exact": exact,
+                "compare_sha256": canonical_sha256(left_selected),
+                "gradient_gate": left["gradient_gate"],
+            }
+            for label, path in (("a", left_path), ("b", right_path)):
+                public_name = f"smoke-{label}-{name}"
+                source_paths[public_name] = path
+                smoke_hashes[public_name] = file_sha256(path)
+
+    replay = read_json(args.replay)
+    replay_formal = runs[(32, "block", 2026073001)]
+    replay_left = selected(replay_formal, REPLAY_COMPARE_FIELDS)
+    replay_right = selected(replay, REPLAY_COMPARE_FIELDS)
+    replay_exact = replay_left == replay_right
+    if (
+        replay["run_kind"] != "replay"
+        or replay["depth"] != 32
+        or replay["architecture"] != "block"
+        or replay["seed"] != 2026073001
+        or not replay_exact
+    ):
+        raise ValueError("formal replay gate failed")
+    replay_public_name = "replay-depth-32-block-seed-2026073001.json"
+    source_paths[replay_public_name] = args.replay
+
+    depth_summaries = {
+        str(depth): summarize_depth(depth, runs) for depth in DEPTHS
+    }
+    depth_labels = [
+        depth_summaries[str(depth)]["verdict"]["label"] for depth in DEPTHS
+    ]
+    if all(
+        label == "joint directional support at this depth"
+        for label in depth_labels
+    ):
+        overall_verdict = (
+            "scale-consistent directional support in this operationalization"
+        )
+    elif all(
+        label == "joint directional concern at this depth"
+        for label in depth_labels
+    ):
+        overall_verdict = (
+            "scale-consistent directional concern in this operationalization"
+        )
+    else:
+        overall_verdict = "depth-dependent or inconclusive"
+
+    full = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "manifest": manifest,
+        "study": {
+            "architectures": list(ARCHITECTURES),
+            "depths": list(DEPTHS),
+            "seeds": list(SEEDS),
+            "diagnostic_steps": list(STEPS),
+            "formal_runs": len(runs),
+            "formal_target_bytes": EXPECTED_TOTAL_TARGET_BYTES,
+            "replay_target_bytes": TARGET_BYTES_PER_RUN,
+            "gradient_object": (
+                "RMS of d(mean token CE)/d(post-MLP Transformer-block output) "
+                "over batch×time×channel"
+            ),
+        },
+        "depth_summaries": depth_summaries,
+        "overall_verdict": overall_verdict,
+        "gates": {
+            "common_initial_parameters": common_initial_exact,
+            "paired_input_tensors": input_gate_exact,
+            "smoke_exact": smoke_exact,
+            "replay": {
+                "exact": replay_exact,
+                "compare_fields": list(REPLAY_COMPARE_FIELDS),
+                "formal_compare_sha256": canonical_sha256(replay_left),
+                "replay_compare_sha256": canonical_sha256(replay_right),
+                "formal_final_model_state": replay_formal["hashes"][
+                    "final_model_state"
+                ],
+                "replay_final_model_state": replay["hashes"][
+                    "final_model_state"
+                ],
+                "formal_final_optimizer_state": replay_formal["hashes"][
+                    "final_optimizer_state"
+                ],
+                "replay_final_optimizer_state": replay["hashes"][
+                    "final_optimizer_state"
+                ],
+            },
+        },
+        "runs": {
+            f"depth-{depth}-{architecture}-seed-{seed}": run
+            for (depth, architecture, seed), run in sorted(runs.items())
+        },
+    }
+    full["canonical_sha256_without_self"] = canonical_sha256(full)
+
+    compact = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "study": full["study"],
+        "manifest_summary": {
+            "file_sha256": manifest_hash,
+            "dataset_revision": manifest["dataset"]["revision"],
+            "train_bytes_sha256": manifest["dataset"]["splits"]["train"][
+                "concatenated_sha256"
+            ],
+            "formal_schedule_sha256": manifest["windows"][
+                "formal_schedule_sha256"
+            ],
+            "validation_tensor_sha256": manifest["windows"][
+                "validation_tensor_sha256"
+            ],
+            "diagnostic_tensor_sha256": manifest["windows"][
+                "diagnostic_tensor_sha256"
+            ],
+        },
+        "depth_summaries": depth_summaries,
+        "overall_verdict": overall_verdict,
+        "replay_exact": replay_exact,
+        "cells": [
+            compact_cell(runs[(depth, architecture, seed)])
+            for depth in DEPTHS
+            for architecture in ARCHITECTURES
+            for seed in SEEDS
+        ],
+    }
+    compact["canonical_sha256_without_self"] = canonical_sha256(compact)
+
+    args.raw_output_dir.mkdir(parents=True, exist_ok=True)
+    for public_name, source_path in sorted(source_paths.items()):
+        target = args.raw_output_dir / public_name
+        temporary = target.with_suffix(target.suffix + ".tmp")
+        shutil.copyfile(source_path, temporary)
+        os.replace(temporary, target)
+
+    reproduction = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "manifest": {
+            "path": str(args.manifest),
+            "sha256": manifest_hash,
+        },
+        "protocol_sha256": file_sha256(
+            Path("research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md")
+        ),
+        "definition_audit_sha256": file_sha256(
+            Path("research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md")
+        ),
+        "runner_sha256": file_sha256(
+            Path("experiments/k3/attnres_gradient/train.py")
+        ),
+        "analyzer_sha256": file_sha256(Path(__file__)),
+        "formal_raw_sha256": formal_hashes,
+        "smoke_raw_sha256": smoke_hashes,
+        "replay_raw_sha256": {
+            replay_public_name: file_sha256(args.replay)
+        },
+        "formal_target_bytes": EXPECTED_TOTAL_TARGET_BYTES,
+        "replay_target_bytes": TARGET_BYTES_PER_RUN,
+        "smoke_exact": smoke_exact,
+        "replay_exact": {
+            "exact": replay_exact,
+            "compare_sha256": canonical_sha256(replay_left),
+            "final_model_state": replay["hashes"]["final_model_state"],
+            "final_optimizer_state": replay["hashes"][
+                "final_optimizer_state"
+            ],
+        },
+        "aggregate_sha256": full["canonical_sha256_without_self"],
+        "compact_sha256": compact["canonical_sha256_without_self"],
+        "overall_verdict": overall_verdict,
+    }
+    reproduction["canonical_sha256_without_self"] = canonical_sha256(
+        reproduction
+    )
+
+    atomic_json(args.output, full)
+    atomic_json(args.compact_output, compact)
+    atomic_json(args.reproduction_output, reproduction)
+    print(
+        json.dumps(
+            {
+                "formal_runs": len(runs),
+                "formal_target_bytes": EXPECTED_TOTAL_TARGET_BYTES,
+                "smoke_exact": all(
+                    row["exact"] for row in smoke_exact.values()
+                ),
+                "replay_exact": replay_exact,
+                "depth_verdicts": {
+                    depth: depth_summaries[str(depth)]["verdict"]["label"]
+                    for depth in DEPTHS
+                },
+                "overall_verdict": overall_verdict,
+                "aggregate_sha256": full[
+                    "canonical_sha256_without_self"
+                ],
+                "compact_sha256": compact[
+                    "canonical_sha256_without_self"
+                ],
+                "reproduction_sha256": reproduction[
+                    "canonical_sha256_without_self"
+                ],
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_gradient/build_dataset.py

@@ -0,0 +1,206 @@
+#!/usr/bin/env python3
+"""Freeze the byte-level corpus and window schedule for K3 AttnRes Round 05."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import os
+import urllib.request
+from pathlib import Path
+from typing import Any
+
+import pyarrow.parquet as pq
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-gradient-scale-v1"
+DATASET_REPO = "Salesforce/wikitext"
+DATASET_REVISION = "b08601e04326c79dfdd32d625aee71d232d685c3"
+DATASET_VARIANT = "wikitext-2-raw-v1"
+SPLITS = ("train", "validation", "test")
+SEEDS = (2026073001, 2026073002, 2026073003)
+CONTEXT = 256
+FORMAL_STEPS = 8000
+FORMAL_BATCH = 32
+VALIDATION_WINDOWS = 64
+DIAGNOSTIC_WINDOWS = 16
+GATE_STEPS = (0, 1, 7999)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--cache-dir", type=Path, required=True)
+    parser.add_argument("--manifest", type=Path, required=True)
+    return parser.parse_args()
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def atomic_json(path: Path, value: dict[str, Any]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(path.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    os.replace(temporary, path)
+
+
+def download(url: str, path: Path) -> None:
+    if path.exists():
+        return
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(path.suffix + ".part")
+    request = urllib.request.Request(
+        url,
+        headers={"User-Agent": "llm-atlas-k3-attnres-gradient-scale/1.0"},
+    )
+    with urllib.request.urlopen(request, timeout=120) as response:
+        with temporary.open("wb") as output:
+            while block := response.read(1024 * 1024):
+                output.write(block)
+    os.replace(temporary, path)
+
+
+def hashed_start(fields: list[str], corpus_length: int) -> int:
+    value = int.from_bytes(
+        hashlib.sha256("\0".join(fields).encode()).digest()[:8], "big"
+    )
+    return value % (corpus_length - (CONTEXT + 1))
+
+
+def fixed_window_start(label: str, index: int, corpus_length: int) -> int:
+    return hashed_start([PROTOCOL_ID, label, str(index)], corpus_length)
+
+
+def train_window_start(seed: int, step: int, row: int, corpus_length: int) -> int:
+    return hashed_start(
+        [PROTOCOL_ID, "train-window", str(seed), str(step), str(row)],
+        corpus_length,
+    )
+
+
+def concatenate_split(parquet_path: Path) -> tuple[bytes, int]:
+    table = pq.read_table(parquet_path, columns=["text"])
+    rows = table.column("text").to_pylist()
+    payload = b"".join(((row or "") + "\n").encode("utf-8") for row in rows)
+    return payload, len(rows)
+
+
+def tensor_hash(payload: bytes, starts: list[int]) -> str:
+    digest = hashlib.sha256()
+    for start in starts:
+        digest.update(payload[start : start + CONTEXT + 1])
+    return digest.hexdigest()
+
+
+def main() -> None:
+    args = parse_args()
+    args.cache_dir.mkdir(parents=True, exist_ok=True)
+
+    split_manifest: dict[str, Any] = {}
+    split_bytes: dict[str, bytes] = {}
+    for split in SPLITS:
+        relative = f"{DATASET_VARIANT}/{split}-00000-of-00001.parquet"
+        url = (
+            f"https://huggingface.co/datasets/{DATASET_REPO}/resolve/"
+            f"{DATASET_REVISION}/{relative}"
+        )
+        parquet_path = args.cache_dir / f"{split}.parquet"
+        download(url, parquet_path)
+        payload, rows = concatenate_split(parquet_path)
+        binary_path = args.cache_dir / f"{split}.bin"
+        if not binary_path.exists() or file_sha256(binary_path) != hashlib.sha256(
+            payload
+        ).hexdigest():
+            temporary = binary_path.with_suffix(".bin.tmp")
+            temporary.write_bytes(payload)
+            os.replace(temporary, binary_path)
+        split_bytes[split] = payload
+        split_manifest[split] = {
+            "source_path": relative,
+            "source_url": url,
+            "parquet_bytes": parquet_path.stat().st_size,
+            "parquet_sha256": file_sha256(parquet_path),
+            "rows": rows,
+            "concatenated_bytes": len(payload),
+            "concatenated_sha256": hashlib.sha256(payload).hexdigest(),
+            "binary_path": str(binary_path),
+            "binary_sha256": file_sha256(binary_path),
+        }
+
+    train = split_bytes["train"]
+    validation = split_bytes["validation"]
+    schedule_digest = hashlib.sha256()
+    schedule_cells = 0
+    for seed in SEEDS:
+        for step in range(1, FORMAL_STEPS + 1):
+            for row in range(FORMAL_BATCH):
+                start = train_window_start(seed, step, row, len(train))
+                schedule_digest.update(start.to_bytes(8, "big"))
+                schedule_cells += 1
+
+    validation_starts = [
+        fixed_window_start("validation-window", index, len(validation))
+        for index in range(VALIDATION_WINDOWS)
+    ]
+    diagnostic_starts = [
+        fixed_window_start("diagnostic-window", index, len(validation))
+        for index in range(DIAGNOSTIC_WINDOWS)
+    ]
+    gate_tensor_hashes: dict[str, dict[str, str]] = {}
+    for seed in SEEDS:
+        gate_tensor_hashes[str(seed)] = {}
+        for step in GATE_STEPS:
+            starts = [
+                train_window_start(seed, step, row, len(train))
+                for row in range(FORMAL_BATCH)
+            ]
+            gate_tensor_hashes[str(seed)][str(step)] = tensor_hash(train, starts)
+
+    manifest = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "status": "frozen-before-model-output",
+        "dataset": {
+            "repository": DATASET_REPO,
+            "revision": DATASET_REVISION,
+            "variant": DATASET_VARIANT,
+            "preprocessing": (
+                "parquet row order; (text or empty string) + LF; UTF-8; "
+                "no normalization; vocabulary is raw bytes 0..255"
+            ),
+            "splits": split_manifest,
+        },
+        "windows": {
+            "context": CONTEXT,
+            "target_bytes_per_window": CONTEXT,
+            "seeds": list(SEEDS),
+            "formal_steps": FORMAL_STEPS,
+            "formal_batch": FORMAL_BATCH,
+            "formal_schedule_cells": schedule_cells,
+            "formal_schedule_sha256": schedule_digest.hexdigest(),
+            "validation_starts": validation_starts,
+            "validation_tensor_sha256": tensor_hash(
+                validation, validation_starts
+            ),
+            "diagnostic_starts": diagnostic_starts,
+            "diagnostic_tensor_sha256": tensor_hash(
+                validation, diagnostic_starts
+            ),
+            "gate_steps": list(GATE_STEPS),
+            "gate_training_tensor_sha256": gate_tensor_hashes,
+        },
+    }
+    atomic_json(args.manifest, manifest)
+    print(json.dumps(manifest, ensure_ascii=False, indent=2))
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_gradient/manifest.json

@@ -0,0 +1,167 @@
+{
+  "dataset": {
+    "preprocessing": "parquet row order; (text or empty string) + LF; UTF-8; no normalization; vocabulary is raw bytes 0..255",
+    "repository": "Salesforce/wikitext",
+    "revision": "b08601e04326c79dfdd32d625aee71d232d685c3",
+    "splits": {
+      "test": {
+        "binary_path": "/home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1/test.bin",
+        "binary_sha256": "bfe9eb16ab9987fb88bde4ea9a30a00f2a45db01dfc14bad78d05325789c4f12",
+        "concatenated_bytes": 1292014,
+        "concatenated_sha256": "bfe9eb16ab9987fb88bde4ea9a30a00f2a45db01dfc14bad78d05325789c4f12",
+        "parquet_bytes": 732610,
+        "parquet_sha256": "5f1bea067869d04849c0f975a2b29c4ff47d867f484f5010ea5e861eab246d91",
+        "rows": 4358,
+        "source_path": "wikitext-2-raw-v1/test-00000-of-00001.parquet",
+        "source_url": "https://huggingface.co/datasets/Salesforce/wikitext/resolve/b08601e04326c79dfdd32d625aee71d232d685c3/wikitext-2-raw-v1/test-00000-of-00001.parquet"
+      },
+      "train": {
+        "binary_path": "/home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1/train.bin",
+        "binary_sha256": "0ca7d3e74dbe44564ea5942b85232f1bbcb525c9cd481cd5d28a87ee90e7e9b4",
+        "concatenated_bytes": 10951563,
+        "concatenated_sha256": "0ca7d3e74dbe44564ea5942b85232f1bbcb525c9cd481cd5d28a87ee90e7e9b4",
+        "parquet_bytes": 6357543,
+        "parquet_sha256": "e83889baabc497075506f91975be5fac0d45c5290b6b20582c8cd1e853d0c9f7",
+        "rows": 36718,
+        "source_path": "wikitext-2-raw-v1/train-00000-of-00001.parquet",
+        "source_url": "https://huggingface.co/datasets/Salesforce/wikitext/resolve/b08601e04326c79dfdd32d625aee71d232d685c3/wikitext-2-raw-v1/train-00000-of-00001.parquet"
+      },
+      "validation": {
+        "binary_path": "/home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1/validation.bin",
+        "binary_sha256": "a42356f6a8ff1d25daf25ec9db49e10a537c265581b61c74604bb63231dee719",
+        "concatenated_bytes": 1148008,
+        "concatenated_sha256": "a42356f6a8ff1d25daf25ec9db49e10a537c265581b61c74604bb63231dee719",
+        "parquet_bytes": 657209,
+        "parquet_sha256": "204929b7ff9d6184953f867dedb860e40aa69c078fc1e54b3baaa8fb28511c4c",
+        "rows": 3760,
+        "source_path": "wikitext-2-raw-v1/validation-00000-of-00001.parquet",
+        "source_url": "https://huggingface.co/datasets/Salesforce/wikitext/resolve/b08601e04326c79dfdd32d625aee71d232d685c3/wikitext-2-raw-v1/validation-00000-of-00001.parquet"
+      }
+    },
+    "variant": "wikitext-2-raw-v1"
+  },
+  "protocol_id": "llm-atlas-k3-attnres-gradient-scale-v1",
+  "schema_version": 1,
+  "status": "frozen-before-model-output",
+  "windows": {
+    "context": 256,
+    "diagnostic_starts": [
+      399861,
+      210983,
+      449025,
+      1098024,
+      323754,
+      152932,
+      1091551,
+      1078021,
+      415985,
+      612288,
+      910624,
+      530272,
+      827285,
+      765798,
+      1086876,
+      1035447
+    ],
+    "diagnostic_tensor_sha256": "21117e31db302b10d67b63f035665dc8f220b879d216ccd12b7d2ba86e7b1716",
+    "formal_batch": 32,
+    "formal_schedule_cells": 768000,
+    "formal_schedule_sha256": "5041e09b167f229248d2462324e8c254b8f5938975f135dcd8192b00a54a4f4e",
+    "formal_steps": 8000,
+    "gate_steps": [
+      0,
+      1,
+      7999
+    ],
+    "gate_training_tensor_sha256": {
+      "2026073001": {
+        "0": "52fdd6885cc2bef8e29cedec8c293e1ea71f63fe3cd0b83f627fe640e19a95c5",
+        "1": "9d0a960595a3f57cd18834d880bde56fa8dcbb0b1cbed26bcc9bbf773a67949c",
+        "7999": "8f4f04a889d1f8c9eb75e917dc7cd6ddef1196bdea94e87466270db7be1289ee"
+      },
+      "2026073002": {
+        "0": "156813ff7ab93736c8dba340711a9633b6c952d0f54df30f277e254705cc8b82",
+        "1": "9ee5a434bdf417d658a1485135d48749e98c86f03239eef86dbd81a9d1309da0",
+        "7999": "bca78ffefa000dc3693a790d65933251646c70facc64b42007d500c19bb90977"
+      },
+      "2026073003": {
+        "0": "8ba13ad55eb54501ec443430446793ef41dea21fd067b2a5b0c84eda202ed0ba",
+        "1": "326a12f07637fe70fd39fa2758b389e91dacc7f2f2ede708b80e09c9d4d1c39d",
+        "7999": "7d6b10c3febfb697444909f695d3a45297789022714f8b71dadfec4d6236aea6"
+      }
+    },
+    "seeds": [
+      2026073001,
+      2026073002,
+      2026073003
+    ],
+    "target_bytes_per_window": 256,
+    "validation_starts": [
+      19785,
+      600807,
+      1029500,
+      319878,
+      652204,
+      1072662,
+      679625,
+      1027264,
+      500250,
+      944134,
+      187313,
+      834295,
+      968556,
+      550645,
+      239009,
+      452519,
+      356354,
+      134015,
+      64555,
+      397632,
+      203140,
+      346032,
+      314812,
+      10817,
+      1141274,
+      807645,
+      417975,
+      870687,
+      377265,
+      635426,
+      597238,
+      805324,
+      12300,
+      264343,
+      84743,
+      596894,
+      188690,
+      992517,
+      854512,
+      427504,
+      94167,
+      296670,
+      760313,
+      912279,
+      1054297,
+      81970,
+      419690,
+      971472,
+      1041491,
+      669963,
+      735537,
+      434513,
+      169153,
+      6229,
+      136413,
+      1098303,
+      400950,
+      457810,
+      659776,
+      911665,
+      909832,
+      532969,
+      555820,
+      1019138
+    ],
+    "validation_tensor_sha256": "f459316f13078a163b47c133511bb7181e05170ab89516e196490113893ce338"
+  }
+}

experiments/k3/attnres_gradient/reproduction.json

@@ -0,0 +1,217 @@
+{
+  "aggregate_sha256": "69be133c8f5b11f8a58e831bc03f5432e737ed21efe89bf186784ead7c7f7b51",
+  "analyzer_sha256": "017d38d92bfd5bc26e10028015f9bf85d28516fb5c5bae06d7bf315b5eca2fc8",
+  "canonical_sha256_without_self": "addb2e5990af4bbec4b88b4654fc72464f4df5b57fd00e46cad1febbb042f68a",
+  "compact_sha256": "8cdb71808a429e5f9513c1c6c9426bdc93e30f237001a1b47a2a2fba574a044f",
+  "definition_audit_sha256": "79221c5648ba280d9177386d2fcfb2fa653b3dd14405185b5997754db2961cc6",
+  "formal_raw_sha256": {
+    "formal-depth-16-baseline-seed-2026073001.json": "c9144ddb458010868668ab0bb1272d4e5051300936d9b21dccee75c32a9e46de",
+    "formal-depth-16-baseline-seed-2026073002.json": "939cad5054960ba8a72b50b4b250b94c3d22a6b46796f669c6e4f033d59fb9de",
+    "formal-depth-16-baseline-seed-2026073003.json": "afd9a60ff323e97d21b24b413d9b9fa97b8d5f960bfc88d2040502aa5ad96a1e",
+    "formal-depth-16-block-seed-2026073001.json": "23e3c68e54ec55d981066ad5730316db9084a6637a8064763952ba109ea1f715",
+    "formal-depth-16-block-seed-2026073002.json": "027f8786f530d6ad5d910f0aad3d0f287887d6457c670bc8b50af7248be1487d",
+    "formal-depth-16-block-seed-2026073003.json": "254b637f403a4c61e8e8bb2dd083349406b6baa7a3d8a616e37a9ab6f8ebaeb9",
+    "formal-depth-32-baseline-seed-2026073001.json": "232ed3dfbbbc898c42622c4a9aee8400f76cb773c576bf1c522cdbb19ead998c",
+    "formal-depth-32-baseline-seed-2026073002.json": "f4fb5fca608a6623ce4c9707714a8a9ebd2c10f6e63f54e243ec3c06d18b1fff",
+    "formal-depth-32-baseline-seed-2026073003.json": "1b0bd279b165c3a2431685d8c1e6bc36ff72eb55333d5de689dca4f89506bb5e",
+    "formal-depth-32-block-seed-2026073001.json": "29e1d638b481619c7b32de402122523db8b17cc1fc67a8881fa1ba132a1d5c38",
+    "formal-depth-32-block-seed-2026073002.json": "21199deb2199395061e51220fd8c7afd04a1135a6381e406da9b5795e3ad5032",
+    "formal-depth-32-block-seed-2026073003.json": "c0d7f1bcfa9fa7f8f3134ca4571bdf23a951182d03b7de9611a6b4b89667d4ac"
+  },
+  "formal_target_bytes": 786432000,
+  "manifest": {
+    "path": "experiments/k3/attnres_gradient/manifest.json",
+    "sha256": "080afb17d1e036c0bba0a799fdb8b98ee4ad652bd42dd1b3b67110dd2ede6371"
+  },
+  "overall_verdict": "depth-dependent or inconclusive",
+  "protocol_id": "llm-atlas-k3-attnres-gradient-scale-v1",
+  "protocol_sha256": "f772629b3b82975b6756721c3a3bb57cc4171dfa26ba5b1e8043dc91c9dcce22",
+  "replay_exact": {
+    "compare_sha256": "46300a452840a9dc6a5180efe7949cf3d81cc4471ed4a942da407e9343064817",
+    "exact": true,
+    "final_model_state": "3f0b97ece3a15571ba3d656f589f512ca0bb9e20083c9f58a42ccaee14892f59",
+    "final_optimizer_state": "ed03e6fbd4a12d8b063dcb22e0437754285f54d585374cd52fbd534f05d24637"
+  },
+  "replay_raw_sha256": {
+    "replay-depth-32-block-seed-2026073001.json": "5cea76d68a642c2b4b4f189a813b2e4af9361a939ddd31a68dfdb0ab090b4aad"
+  },
+  "replay_target_bytes": 65536000,
+  "runner_sha256": "04ae69e10c58972c9193c2d31c7e09d924a0d0e834107afa4ba128c64ac5800f",
+  "schema_version": 1,
+  "smoke_exact": {
+    "depth-16-baseline": {
+      "compare_sha256": "525cdcd9a79ec61096ecf64fedcbf93144d3168b434e2f8fd384c4374cbebd2a",
+      "exact": true,
+      "gradient_gate": {
+        "first_to_last_ratio_abs_delta": 0.0,
+        "max_abs_scale_ratio_error": 0.0,
+        "normalized_spectrum_max_abs_delta": 0.0,
+        "passed": true,
+        "per_block_scale_ratios": [
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0
+        ],
+        "population_cv_abs_delta": 0.0,
+        "thresholds": {
+          "scale_ratio_abs": 1e-05,
+          "shape_abs": 1e-06
+        }
+      }
+    },
+    "depth-16-block": {
+      "compare_sha256": "e4330a0d878d10b474b9aeb0be58131f801ebe2e5dd9553d4713c3d16b6590cc",
+      "exact": true,
+      "gradient_gate": {
+        "first_to_last_ratio_abs_delta": 0.0,
+        "max_abs_scale_ratio_error": 0.0,
+        "normalized_spectrum_max_abs_delta": 0.0,
+        "passed": true,
+        "per_block_scale_ratios": [
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0
+        ],
+        "population_cv_abs_delta": 0.0,
+        "thresholds": {
+          "scale_ratio_abs": 1e-05,
+          "shape_abs": 1e-06
+        }
+      }
+    },
+    "depth-32-baseline": {
+      "compare_sha256": "8ee0dc37e88742b6705968f5f2845a3a1df1250f5e0fd77be6b909236ea28e56",
+      "exact": true,
+      "gradient_gate": {
+        "first_to_last_ratio_abs_delta": 0.0,
+        "max_abs_scale_ratio_error": 0.0,
+        "normalized_spectrum_max_abs_delta": 0.0,
+        "passed": true,
+        "per_block_scale_ratios": [
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0
+        ],
+        "population_cv_abs_delta": 0.0,
+        "thresholds": {
+          "scale_ratio_abs": 1e-05,
+          "shape_abs": 1e-06
+        }
+      }
+    },
+    "depth-32-block": {
+      "compare_sha256": "289073b1940d4747b7e049f8d767f9c22a49867d37403ff6ef7746a55bd1734c",
+      "exact": true,
+      "gradient_gate": {
+        "first_to_last_ratio_abs_delta": 0.0,
+        "max_abs_scale_ratio_error": 0.0,
+        "normalized_spectrum_max_abs_delta": 0.0,
+        "passed": true,
+        "per_block_scale_ratios": [
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0,
+          2.0
+        ],
+        "population_cv_abs_delta": 0.0,
+        "thresholds": {
+          "scale_ratio_abs": 1e-05,
+          "shape_abs": 1e-06
+        }
+      }
+    }
+  },
+  "smoke_raw_sha256": {
+    "smoke-a-depth-16-baseline.json": "fe0ff3ed21bf9b86d238d7a22e99b6e07ba99135b58d29eb6b205f2602e64556",
+    "smoke-a-depth-16-block.json": "faa06a2bd02135858d3a2503e1ecfb34187b4ee3b4ce3428ed473a55b15b6705",
+    "smoke-a-depth-32-baseline.json": "e327a6f5db87515c1b29b2808205e23b81228b08ce356b98804717e2e55eef1c",
+    "smoke-a-depth-32-block.json": "2d9a97c2d6272c928d5b5bc01febd142d4b2f2ad3080330e849f2731a8d080c2",
+    "smoke-b-depth-16-baseline.json": "fe0ff3ed21bf9b86d238d7a22e99b6e07ba99135b58d29eb6b205f2602e64556",
+    "smoke-b-depth-16-block.json": "faa06a2bd02135858d3a2503e1ecfb34187b4ee3b4ce3428ed473a55b15b6705",
+    "smoke-b-depth-32-baseline.json": "e327a6f5db87515c1b29b2808205e23b81228b08ce356b98804717e2e55eef1c",
+    "smoke-b-depth-32-block.json": "2d9a97c2d6272c928d5b5bc01febd142d4b2f2ad3080330e849f2731a8d080c2"
+  }
+}

experiments/k3/attnres_gradient/results/raw/smoke-a-depth-16-baseline.json

@@ -0,0 +1,700 @@
+{
+  "architecture": "baseline",
+  "batch_size": 32,
+  "canonical_sha256_without_self": "0ae9ad1697ca15ec4c84270ad82b9fa98056f14f20fc7372c4b9886c29845415",
+  "depth": 16,
+  "diagnostics": [
+    {
+      "activation_grad_rms_by_block": [
+        0.0002865509013645351,
+        0.00024785567075014114,
+        0.00021551651298068464,
+        0.00019683866412378848,
+        0.0001823508064262569,
+        0.0001679604029050097,
+        0.00015584587526973337,
+        0.00014737028686795384,
+        0.00014130656199995428,
+        0.00013612695329356939,
+        0.00013129493163432926,
+        0.00012807638267986476,
+        0.00012550120300147682,
+        0.00012324050476308912,
+        0.00012108208466088399,
+        0.00011888353037647903
+      ],
+      "activation_grad_statistics": {
+        "first_quartile_mean": 0.00023669043730478734,
+        "first_to_last_ratio": 1.9372775995887173,
+        "imbalance_abs_log_ratio": 0.6612836883477485,
+        "last_quartile_mean": 0.00012217683070048224,
+        "mean": 0.00016411257956860936,
+        "normalized": [
+          1.7460629899168627,
+          1.5102783187106137,
+          1.313223602646409,
+          1.1994124072706904,
+          1.1111324123086055,
+          1.0234462424910682,
+          0.9496278449793059,
+          0.8979828801383491,
+          0.8610343117596252,
+          0.8294729974472175,
+          0.8000296624393728,
+          0.7804178266926869,
+          0.7647262832098098,
+          0.7509509940495869,
+          0.7377989242455608,
+          0.7244023016942358
+        ],
+        "population_cv": 0.29361872380036635
+      },
+      "activation_output_rms_by_block": [
+        0.02866268903017044,
+        0.029099803417921066,
+        0.02953805774450302,
+        0.030088091269135475,
+        0.030644793063402176,
+        0.03131929785013199,
+        0.03213750571012497,
+        0.03286394104361534,
+        0.033869802951812744,
+        0.03499744459986687,
+        0.03588006645441055,
+        0.037365153431892395,
+        0.03789033368229866,
+        0.039445169270038605,
+        0.04050002992153168,
+        0.04121527820825577
+      ],
+      "activation_output_statistics": {
+        "first_quartile_mean": 0.0293471603654325,
+        "first_to_last_ratio": 0.7380574840395957,
+        "imbalance_abs_log_ratio": 0.3037335657624931,
+        "last_quartile_mean": 0.03976270277053118,
+        "mean": 0.034094841103069484,
+        "normalized": [
+          0.8406752488894867,
+          0.8534957922212247,
+          0.8663497699023965,
+          0.8824822259232266,
+          0.8988102619619861,
+          0.9185934539320196,
+          0.942591449919727,
+          0.9638977622528551,
+          0.9933996421752943,
+          1.0264733158329964,
+          1.0523605710888722,
+          1.0959180985456622,
+          1.1113216092650298,
+          1.1569248600043318,
+          1.1878638706395246,
+          1.2088420674453664
+        ],
+        "population_cv": 0.11952987076282337
+      },
+      "bits_per_byte": 8.096463027059821,
+      "branch_output_rms_by_sublayer": [
+        0.0033055038657039404,
+        0.0037518907338380814,
+        0.0033751516602933407,
+        0.003911525942385197,
+        0.004229962360113859,
+        0.0038461871445178986,
+        0.004243654198944569,
+        0.0038366704247891903,
+        0.004372371360659599,
+        0.003851204412057996,
+        0.005078981164842844,
+        0.003818925702944398,
+        0.0054668826051056385,
+        0.0038749484810978174,
+        0.005823117680847645,
+        0.0037566579412668943,
+        0.0065947300754487514,
+        0.003876061411574483,
+        0.006763988174498081,
+        0.003795720636844635,
+        0.006863070651888847,
+        0.0038706199266016483,
+        0.007856340147554874,
+        0.003839300014078617,
+        0.00830968376249075,
+        0.003927029203623533,
+        0.009655521251261234,
+        0.0038513424806296825,
+        0.009004893712699413,
+        0.0039484030567109585,
+        0.008608299307525158,
+        0.003791053779423237
+      ],
+      "capture": {
+        "all_gradients_finite": true,
+        "all_gradients_present": true,
+        "count": 16,
+        "dtypes": [
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32"
+        ],
+        "position": "post-MLP Transformer-block output; Block AttnRes is captured before aggregation-partial reset",
+        "shape": [
+          16,
+          256,
+          192
+        ],
+        "storage_unique": true
+      },
+      "core_parameter_grad_rms_by_block": [
+        0.00809059897248305,
+        0.007313812062277777,
+        0.007090710224412361,
+        0.0067033738367094624,
+        0.006535407867454384,
+        0.006785021795736036,
+        0.006295993569919972,
+        0.005509300326230647,
+        0.006231736048910802,
+        0.006047390980174007,
+        0.005629405359532731,
+        0.005784042737956458,
+        0.0059591736113607996,
+        0.0061983173180850636,
+        0.006571690039252182,
+        0.005060205437758869
+      ],
+      "core_parameter_grad_statistics": {
+        "first_quartile_mean": 0.007299623773970663,
+        "first_to_last_ratio": 1.227374872012571,
+        "imbalance_abs_log_ratio": 0.2048776382299483,
+        "last_quartile_mean": 0.005947346601614228,
+        "mean": 0.006362886261765913,
+        "normalized": [
+          1.271529717747562,
+          1.1494488132258316,
+          1.1143858200043408,
+          1.05351149791715,
+          1.0271137340180259,
+          1.066343404015675,
+          0.9894870520870547,
+          0.8658492545019393,
+          0.9793882512652816,
+          0.9504163254515969,
+          0.8847251275509024,
+          0.909028151691524,
+          0.9365519618304368,
+          0.9741361173356609,
+          1.0328158902888074,
+          0.7952688810682109
+        ],
+        "population_cv": 0.11384977604868386
+      },
+      "depth_weights": [],
+      "layer_input_rms_by_sublayer": [
+        0.02823694422841072,
+        0.028404271230101585,
+        0.02866268903017044,
+        0.02886480651795864,
+        0.029099803417921066,
+        0.029365191236138344,
+        0.02953805774450302,
+        0.029881296679377556,
+        0.030088091269135475,
+        0.030441828072071075,
+        0.030644793063402176,
+        0.03106885403394699,
+        0.03131929785013199,
+        0.031892918050289154,
+        0.03213750571012497,
+        0.03266792371869087,
+        0.03286394104361534,
+        0.03364725783467293,
+        0.033869802951812744,
+        0.03485054895281792,
+        0.03499744459986687,
+        0.03562505170702934,
+        0.03588006645441055,
+        0.03719272464513779,
+        0.037365153431892395,
+        0.03768136352300644,
+        0.03789033368229866,
+        0.03927159309387207,
+        0.039445169270038605,
+        0.04041972756385803,
+        0.04050002992153168,
+        0.04110744968056679
+      ],
+      "loss_nats": 5.6120405197143555,
+      "loss_scale": 1.0,
+      "output_weights": null,
+      "step": 0,
+      "stream_state_rms_by_sublayer": [
+        0.028404271230101585,
+        0.02866268903017044,
+        0.02886480651795864,
+        0.029099803417921066,
+        0.029365191236138344,
+        0.02953805774450302,
+        0.029881296679377556,
+        0.030088091269135475,
+        0.030441828072071075,
+        0.030644793063402176,
+        0.03106885403394699,
+        0.03131929785013199,
+        0.031892918050289154,
+        0.03213750571012497,
+        0.03266792371869087,
+        0.03286394104361534,
+        0.03364725783467293,
+        0.033869802951812744,
+        0.03485054895281792,
+        0.03499744459986687,
+        0.03562505170702934,
+        0.03588006645441055,
+        0.03719272464513779,
+        0.037365153431892395,
+        0.03768136352300644,
+        0.03789033368229866,
+        0.03927159309387207,
+        0.039445169270038605,
+        0.04041972756385803,
+        0.04050002992153168,
+        0.04110744968056679,
+        0.04121527820825577
+      ]
+    },
+    {
+      "activation_grad_rms_by_block": [
+        5.69471885683015e-05,
+        5.527245593839325e-05,
+        5.393734318204224e-05,
+        5.292302375892177e-05,
+        5.23102717124857e-05,
+        5.178990977583453e-05,
+        5.1564093155320734e-05,
+        5.129844430484809e-05,
+        5.121947833686136e-05,
+        5.116340616950765e-05,
+        5.109804988023825e-05,
+        5.104695082991384e-05,
+        5.09646451973822e-05,
+        5.100828275317326e-05,
+        5.106439857627265e-05,
+        5.11144389747642e-05
+      ],
+      "activation_grad_statistics": {
+        "first_quartile_mean": 5.477000286191469e-05,
+        "first_to_last_ratio": 1.0731232762518041,
+        "imbalance_abs_log_ratio": 0.07057334637995329,
+        "last_quartile_mean": 5.103794137539808e-05,
+        "mean": 5.2170148819641327e-05,
+        "normalized": [
+          1.0915665348238701,
+          1.0594651767139285,
+          1.033873669184083,
+          1.0144311441756324,
+          1.002685882559561,
+          0.9927115591500164,
+          0.988383094968431,
+          0.9832911246274795,
+          0.9817775010367221,
+          0.9807027069519371,
+          0.9794499543578176,
+          0.9784704852268963,
+          0.9768928467805085,
+          0.9777292936141551,
+          0.9788049244944386,
+          0.9797641013345227
+        ],
+        "population_cv": 0.032815404485150704
+      },
+      "activation_output_rms_by_block": [
+        0.028743742033839226,
+        0.02942933700978756,
+        0.030596865341067314,
+        0.03249936178326607,
+        0.03542664647102356,
+        0.03894684836268425,
+        0.04318666458129883,
+        0.04936420917510986,
+        0.05370106175541878,
+        0.06048284471035004,
+        0.0664696991443634,
+        0.07231792062520981,
+        0.07627613097429276,
+        0.07997097074985504,
+        0.08494995534420013,
+        0.08972473442554474
+      ],
+      "activation_output_statistics": {
+        "first_quartile_mean": 0.03031732654199004,
+        "first_to_last_ratio": 0.3664591129538783,
+        "imbalance_abs_log_ratio": 1.0038683247140607,
+        "last_quartile_mean": 0.08273044787347317,
+        "mean": 0.05450543703045696,
+        "normalized": [
+          0.5273555006590916,
+          0.5399339701348108,
+          0.5613543713807885,
+          0.5962590808162097,
+          0.6499653686150171,
+          0.7145497859400932,
+          0.7923368187501488,
+          0.9056749539963465,
+          0.9852422929002714,
+          1.1096662646068314,
+          1.21950584686113,
+          1.326801958945847,
+          1.399422427007981,
+          1.4672108895332452,
+          1.5585592919240543,
+          1.6461611779281335
+        ],
+        "population_cv": 0.3808757530834043
+      },
+      "bits_per_byte": 6.7509657011324835,
+      "branch_output_rms_by_sublayer": [
+        0.003436450148001313,
+        0.003791899885982275,
+        0.003964710980653763,
+        0.003938332665711641,
+        0.0056790695525705814,
+        0.003919641952961683,
+        0.006960175931453705,
+        0.0038887569680809975,
+        0.008646421134471893,
+        0.003928068559616804,
+        0.009785857982933521,
+        0.0038193853106349707,
+        0.010625048540532589,
+        0.003996006678789854,
+        0.012892307713627815,
+        0.0039014811627566814,
+        0.012757784686982632,
+        0.003861474571749568,
+        0.014726920053362846,
+        0.003962590359151363,
+        0.012820222415030003,
+        0.004551138263195753,
+        0.01401793584227562,
+        0.004161422606557608,
+        0.01284075528383255,
+        0.004107494372874498,
+        0.012963366694748402,
+        0.0034911984112113714,
+        0.014392351731657982,
+        0.004168955609202385,
+        0.013408888131380081,
+        0.003909120801836252
+      ],
+      "capture": {
+        "all_gradients_finite": true,
+        "all_gradients_present": true,
+        "count": 16,
+        "dtypes": [
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32"
+        ],
+        "position": "post-MLP Transformer-block output; Block AttnRes is captured before aggregation-partial reset",
+        "shape": [
+          16,
+          256,
+          192
+        ],
+        "storage_unique": true
+      },
+      "core_parameter_grad_rms_by_block": [
+        0.0004507690637370199,
+        0.0004530669153600251,
+        0.0005174622333717066,
+        0.0005579942111201043,
+        0.0006494724560479704,
+        0.0007477190266084469,
+        0.0008433132451421535,
+        0.0008983201732199723,
+        0.001008715304343127,
+        0.0011143118429534733,
+        0.0010428854825857738,
+        0.0010916116075341876,
+        0.0011085001103672855,
+        0.0012404769719650786,
+        0.0013545740271648265,
+        0.0014243271893498689
+      ],
+      "core_parameter_grad_statistics": {
+        "first_quartile_mean": 0.000494823105897214,
+        "first_to_last_ratio": 0.385986622193018,
+        "imbalance_abs_log_ratio": 0.9519525676489338,
+        "last_quartile_mean": 0.001281969574711765,
+        "mean": 0.0009064699913044387,
+        "normalized": [
+          0.49727963204644987,
+          0.4998145771025995,
+          0.5708542349284638,
+          0.6155683215912455,
+          0.7164853357289404,
+          0.824869034585972,
+          0.930326710461313,
+          0.99100927977468,
+          1.112795033503044,
+          1.2292870736404011,
+          1.1504909071341995,
+          1.2042446170372658,
+          1.2228756836970622,
+          1.3684699812069823,
+          1.494339625314625,
+          1.571289952246756
+        ],
+        "population_cv": 0.338812252534038
+      },
+      "depth_weights": [],
+      "layer_input_rms_by_sublayer": [
+        0.028241552412509918,
+        0.028533460572361946,
+        0.028743742033839226,
+        0.029236938804388046,
+        0.02942933700978756,
+        0.03035305254161358,
+        0.030596865341067314,
+        0.032195623964071274,
+        0.03249936178326607,
+        0.03506496921181679,
+        0.03542664647102356,
+        0.03852483630180359,
+        0.03894684836268425,
+        0.04251723736524582,
+        0.04318666458129883,
+        0.04861506074666977,
+        0.04936420917510986,
+        0.052915990352630615,
+        0.05370106175541878,
+        0.0598941408097744,
+        0.06048284471035004,
+        0.06538087129592896,
+        0.0664696991443634,
+        0.07142146676778793,
+        0.07231792062520981,
+        0.07521561533212662,
+        0.07627613097429276,
+        0.07935076206922531,
+        0.07997097074985504,
+        0.08368266373872757,
+        0.08494995534420013,
+        0.08860929310321808
+      ],
+      "loss_nats": 4.679412841796875,
+      "loss_scale": 1.0,
+      "output_weights": null,
+      "step": 20,
+      "stream_state_rms_by_sublayer": [
+        0.028533460572361946,
+        0.028743742033839226,
+        0.029236938804388046,
+        0.02942933700978756,
+        0.03035305254161358,
+        0.030596865341067314,
+        0.032195623964071274,
+        0.03249936178326607,
+        0.03506496921181679,
+        0.03542664647102356,
+        0.03852483630180359,
+        0.03894684836268425,
+        0.04251723736524582,
+        0.04318666458129883,
+        0.04861506074666977,
+        0.04936420917510986,
+        0.052915990352630615,
+        0.05370106175541878,
+        0.0598941408097744,
+        0.06048284471035004,
+        0.06538087129592896,
+        0.0664696991443634,
+        0.07142146676778793,
+        0.07231792062520981,
+        0.07521561533212662,
+        0.07627613097429276,
+        0.07935076206922531,
+        0.07997097074985504,
+        0.08368266373872757,
+        0.08494995534420013,
+        0.08860929310321808,
+        0.08972473442554474
+      ]
+    }
+  ],
+  "environment": {
+    "autocast": "cuda-bfloat16-forward-fp32-cross-entropy",
+    "compile": false,
+    "compute_capability": [
+      12,
+      0
+    ],
+    "cublas_workspace_config": ":4096:8",
+    "cuda": "12.8",
+    "deterministic_algorithms": true,
+    "gpu": "NVIDIA GeForce RTX 5090",
+    "python": "3.10.14",
+    "torch": "2.11.0+cu128"
+  },
+  "evaluations": [
+    {
+      "bits_per_byte": 8.076786148009306,
+      "cross_entropy_nats": 5.5984015464782715,
+      "step": 0
+    },
+    {
+      "bits_per_byte": 6.752401670275862,
+      "cross_entropy_nats": 4.680408179759979,
+      "step": 20
+    }
+  ],
+  "gradient_gate": {
+    "first_to_last_ratio_abs_delta": 0.0,
+    "max_abs_scale_ratio_error": 0.0,
+    "normalized_spectrum_max_abs_delta": 0.0,
+    "passed": true,
+    "per_block_scale_ratios": [
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0
+    ],
+    "population_cv_abs_delta": 0.0,
+    "thresholds": {
+      "scale_ratio_abs": 1e-05,
+      "shape_abs": 1e-06
+    }
+  },
+  "hashes": {
+    "final_mixer_parameters": null,
+    "final_model_state": "d53dab0fa7c76215ab91de676b2aef3f9ef14cb0cc1819b7c4a887915bed97c0",
+    "final_optimizer_state": "a7bce44db1478ce53933758aa5033bbb1e0aa21296e9615c33146920f30a2057",
+    "final_public_parameters": "d53dab0fa7c76215ab91de676b2aef3f9ef14cb0cc1819b7c4a887915bed97c0",
+    "initial_mixer_parameters": null,
+    "initial_public_parameter_elements": 9541824,
+    "initial_public_parameter_structure": "e732db766f25e01f6ce1772cc182ced9de2c56c4a2384130117242b9444f0abe",
+    "initial_public_parameter_tensors": 115,
+    "initial_public_parameters": "af2724a1c34bcfd61d8a8bef402246430898c815e56b6e5c5257949a4eb0e7b1"
+  },
+  "manifest": {
+    "diagnostic_tensor_sha256": "21117e31db302b10d67b63f035665dc8f220b879d216ccd12b7d2ba86e7b1716",
+    "file_sha256": "080afb17d1e036c0bba0a799fdb8b98ee4ad652bd42dd1b3b67110dd2ede6371",
+    "formal_schedule_sha256": "5041e09b167f229248d2462324e8c254b8f5938975f135dcd8192b00a54a4f4e",
+    "input_gate_tensor_hashes": {
+      "0": "65136111a29a042e61a7909132560d95cd4bcf0f9b52f64d0fb2e57773856434",
+      "1": "d995676b4e7dec8f661cd8c2345fe7fc7a513c17f528c02fc946a441a6995a94",
+      "7999": "2345e7ac3decca2bdaebf13094fdc92fcabef42e3e461a2fefd6e8e7e76baccc"
+    },
+    "path": "experiments/k3/attnres_gradient/manifest.json",
+    "validation_tensor_sha256": "f459316f13078a163b47c133511bb7181e05170ab89516e196490113893ce338"
+  },
+  "model": {
+    "attnres_aggregation_groups": 8,
+    "context": 256,
+    "d_ff": 768,
+    "d_head": 32,
+    "d_model": 192,
+    "heads": 6,
+    "layers": 16,
+    "parameters": {
+      "core": 9541824,
+      "embedding": 98304,
+      "mixer": 0,
+      "total": 9541824
+    },
+    "sublayers": 32,
+    "sublayers_per_attnres_group": 4,
+    "transformer_blocks_per_attnres_group": 2,
+    "vocabulary": 256
+  },
+  "optimizer": {
+    "betas": [
+      0.9,
+      0.95
+    ],
+    "epsilon": 1e-08,
+    "grad_clip": 1.0,
+    "min_lr": 3e-05,
+    "name": "AdamW",
+    "peak_lr": 0.0003,
+    "warmup_steps": 400,
+    "weight_decay_ndim_ge_2": 0.1
+  },
+  "protocol_id": "llm-atlas-k3-attnres-gradient-scale-v1",
+  "run_kind": "smoke",
+  "schema_version": 1,
+  "seed": 2026073001,
+  "steps": 20,
+  "target_bytes_seen": 163840,
+  "timing": {
+    "mean_ms": null,
+    "measured_steps": 0,
+    "median_ms": null,
+    "p95_ms": null,
+    "peak_allocated_bytes": 1648265728,
+    "peak_reserved_bytes": 3282042880,
+    "warmup_steps_excluded": 20
+  },
+  "training_history": [
+    {
+      "bits_per_byte": 8.088097790921855,
+      "learning_rate": 7.499999999999999e-07,
+      "loss_nats": 5.6062421798706055,
+      "step": 1,
+      "unclipped_grad_norm": 19.475919723510742
+    },
+    {
+      "bits_per_byte": 7.474302716882146,
+      "learning_rate": 7.499999999999999e-06,
+      "loss_nats": 5.180791854858398,
+      "step": 10,
+      "unclipped_grad_norm": 12.938376426696777
+    },
+    {
+      "bits_per_byte": 6.789615018295581,
+      "learning_rate": 1.4999999999999999e-05,
+      "loss_nats": 4.706202507019043,
+      "step": 20,
+      "unclipped_grad_norm": 4.482712745666504
+    }
+  ]
+}

experiments/k3/attnres_gradient/results/raw/smoke-b-depth-16-baseline.json

@@ -0,0 +1,700 @@
+{
+  "architecture": "baseline",
+  "batch_size": 32,
+  "canonical_sha256_without_self": "0ae9ad1697ca15ec4c84270ad82b9fa98056f14f20fc7372c4b9886c29845415",
+  "depth": 16,
+  "diagnostics": [
+    {
+      "activation_grad_rms_by_block": [
+        0.0002865509013645351,
+        0.00024785567075014114,
+        0.00021551651298068464,
+        0.00019683866412378848,
+        0.0001823508064262569,
+        0.0001679604029050097,
+        0.00015584587526973337,
+        0.00014737028686795384,
+        0.00014130656199995428,
+        0.00013612695329356939,
+        0.00013129493163432926,
+        0.00012807638267986476,
+        0.00012550120300147682,
+        0.00012324050476308912,
+        0.00012108208466088399,
+        0.00011888353037647903
+      ],
+      "activation_grad_statistics": {
+        "first_quartile_mean": 0.00023669043730478734,
+        "first_to_last_ratio": 1.9372775995887173,
+        "imbalance_abs_log_ratio": 0.6612836883477485,
+        "last_quartile_mean": 0.00012217683070048224,
+        "mean": 0.00016411257956860936,
+        "normalized": [
+          1.7460629899168627,
+          1.5102783187106137,
+          1.313223602646409,
+          1.1994124072706904,
+          1.1111324123086055,
+          1.0234462424910682,
+          0.9496278449793059,
+          0.8979828801383491,
+          0.8610343117596252,
+          0.8294729974472175,
+          0.8000296624393728,
+          0.7804178266926869,
+          0.7647262832098098,
+          0.7509509940495869,
+          0.7377989242455608,
+          0.7244023016942358
+        ],
+        "population_cv": 0.29361872380036635
+      },
+      "activation_output_rms_by_block": [
+        0.02866268903017044,
+        0.029099803417921066,
+        0.02953805774450302,
+        0.030088091269135475,
+        0.030644793063402176,
+        0.03131929785013199,
+        0.03213750571012497,
+        0.03286394104361534,
+        0.033869802951812744,
+        0.03499744459986687,
+        0.03588006645441055,
+        0.037365153431892395,
+        0.03789033368229866,
+        0.039445169270038605,
+        0.04050002992153168,
+        0.04121527820825577
+      ],
+      "activation_output_statistics": {
+        "first_quartile_mean": 0.0293471603654325,
+        "first_to_last_ratio": 0.7380574840395957,
+        "imbalance_abs_log_ratio": 0.3037335657624931,
+        "last_quartile_mean": 0.03976270277053118,
+        "mean": 0.034094841103069484,
+        "normalized": [
+          0.8406752488894867,
+          0.8534957922212247,
+          0.8663497699023965,
+          0.8824822259232266,
+          0.8988102619619861,
+          0.9185934539320196,
+          0.942591449919727,
+          0.9638977622528551,
+          0.9933996421752943,
+          1.0264733158329964,
+          1.0523605710888722,
+          1.0959180985456622,
+          1.1113216092650298,
+          1.1569248600043318,
+          1.1878638706395246,
+          1.2088420674453664
+        ],
+        "population_cv": 0.11952987076282337
+      },
+      "bits_per_byte": 8.096463027059821,
+      "branch_output_rms_by_sublayer": [
+        0.0033055038657039404,
+        0.0037518907338380814,
+        0.0033751516602933407,
+        0.003911525942385197,
+        0.004229962360113859,
+        0.0038461871445178986,
+        0.004243654198944569,
+        0.0038366704247891903,
+        0.004372371360659599,
+        0.003851204412057996,
+        0.005078981164842844,
+        0.003818925702944398,
+        0.0054668826051056385,
+        0.0038749484810978174,
+        0.005823117680847645,
+        0.0037566579412668943,
+        0.0065947300754487514,
+        0.003876061411574483,
+        0.006763988174498081,
+        0.003795720636844635,
+        0.006863070651888847,
+        0.0038706199266016483,
+        0.007856340147554874,
+        0.003839300014078617,
+        0.00830968376249075,
+        0.003927029203623533,
+        0.009655521251261234,
+        0.0038513424806296825,
+        0.009004893712699413,
+        0.0039484030567109585,
+        0.008608299307525158,
+        0.003791053779423237
+      ],
+      "capture": {
+        "all_gradients_finite": true,
+        "all_gradients_present": true,
+        "count": 16,
+        "dtypes": [
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32"
+        ],
+        "position": "post-MLP Transformer-block output; Block AttnRes is captured before aggregation-partial reset",
+        "shape": [
+          16,
+          256,
+          192
+        ],
+        "storage_unique": true
+      },
+      "core_parameter_grad_rms_by_block": [
+        0.00809059897248305,
+        0.007313812062277777,
+        0.007090710224412361,
+        0.0067033738367094624,
+        0.006535407867454384,
+        0.006785021795736036,
+        0.006295993569919972,
+        0.005509300326230647,
+        0.006231736048910802,
+        0.006047390980174007,
+        0.005629405359532731,
+        0.005784042737956458,
+        0.0059591736113607996,
+        0.0061983173180850636,
+        0.006571690039252182,
+        0.005060205437758869
+      ],
+      "core_parameter_grad_statistics": {
+        "first_quartile_mean": 0.007299623773970663,
+        "first_to_last_ratio": 1.227374872012571,
+        "imbalance_abs_log_ratio": 0.2048776382299483,
+        "last_quartile_mean": 0.005947346601614228,
+        "mean": 0.006362886261765913,
+        "normalized": [
+          1.271529717747562,
+          1.1494488132258316,
+          1.1143858200043408,
+          1.05351149791715,
+          1.0271137340180259,
+          1.066343404015675,
+          0.9894870520870547,
+          0.8658492545019393,
+          0.9793882512652816,
+          0.9504163254515969,
+          0.8847251275509024,
+          0.909028151691524,
+          0.9365519618304368,
+          0.9741361173356609,
+          1.0328158902888074,
+          0.7952688810682109
+        ],
+        "population_cv": 0.11384977604868386
+      },
+      "depth_weights": [],
+      "layer_input_rms_by_sublayer": [
+        0.02823694422841072,
+        0.028404271230101585,
+        0.02866268903017044,
+        0.02886480651795864,
+        0.029099803417921066,
+        0.029365191236138344,
+        0.02953805774450302,
+        0.029881296679377556,
+        0.030088091269135475,
+        0.030441828072071075,
+        0.030644793063402176,
+        0.03106885403394699,
+        0.03131929785013199,
+        0.031892918050289154,
+        0.03213750571012497,
+        0.03266792371869087,
+        0.03286394104361534,
+        0.03364725783467293,
+        0.033869802951812744,
+        0.03485054895281792,
+        0.03499744459986687,
+        0.03562505170702934,
+        0.03588006645441055,
+        0.03719272464513779,
+        0.037365153431892395,
+        0.03768136352300644,
+        0.03789033368229866,
+        0.03927159309387207,
+        0.039445169270038605,
+        0.04041972756385803,
+        0.04050002992153168,
+        0.04110744968056679
+      ],
+      "loss_nats": 5.6120405197143555,
+      "loss_scale": 1.0,
+      "output_weights": null,
+      "step": 0,
+      "stream_state_rms_by_sublayer": [
+        0.028404271230101585,
+        0.02866268903017044,
+        0.02886480651795864,
+        0.029099803417921066,
+        0.029365191236138344,
+        0.02953805774450302,
+        0.029881296679377556,
+        0.030088091269135475,
+        0.030441828072071075,
+        0.030644793063402176,
+        0.03106885403394699,
+        0.03131929785013199,
+        0.031892918050289154,
+        0.03213750571012497,
+        0.03266792371869087,
+        0.03286394104361534,
+        0.03364725783467293,
+        0.033869802951812744,
+        0.03485054895281792,
+        0.03499744459986687,
+        0.03562505170702934,
+        0.03588006645441055,
+        0.03719272464513779,
+        0.037365153431892395,
+        0.03768136352300644,
+        0.03789033368229866,
+        0.03927159309387207,
+        0.039445169270038605,
+        0.04041972756385803,
+        0.04050002992153168,
+        0.04110744968056679,
+        0.04121527820825577
+      ]
+    },
+    {
+      "activation_grad_rms_by_block": [
+        5.69471885683015e-05,
+        5.527245593839325e-05,
+        5.393734318204224e-05,
+        5.292302375892177e-05,
+        5.23102717124857e-05,
+        5.178990977583453e-05,
+        5.1564093155320734e-05,
+        5.129844430484809e-05,
+        5.121947833686136e-05,
+        5.116340616950765e-05,
+        5.109804988023825e-05,
+        5.104695082991384e-05,
+        5.09646451973822e-05,
+        5.100828275317326e-05,
+        5.106439857627265e-05,
+        5.11144389747642e-05
+      ],
+      "activation_grad_statistics": {
+        "first_quartile_mean": 5.477000286191469e-05,
+        "first_to_last_ratio": 1.0731232762518041,
+        "imbalance_abs_log_ratio": 0.07057334637995329,
+        "last_quartile_mean": 5.103794137539808e-05,
+        "mean": 5.2170148819641327e-05,
+        "normalized": [
+          1.0915665348238701,
+          1.0594651767139285,
+          1.033873669184083,
+          1.0144311441756324,
+          1.002685882559561,
+          0.9927115591500164,
+          0.988383094968431,
+          0.9832911246274795,
+          0.9817775010367221,
+          0.9807027069519371,
+          0.9794499543578176,
+          0.9784704852268963,
+          0.9768928467805085,
+          0.9777292936141551,
+          0.9788049244944386,
+          0.9797641013345227
+        ],
+        "population_cv": 0.032815404485150704
+      },
+      "activation_output_rms_by_block": [
+        0.028743742033839226,
+        0.02942933700978756,
+        0.030596865341067314,
+        0.03249936178326607,
+        0.03542664647102356,
+        0.03894684836268425,
+        0.04318666458129883,
+        0.04936420917510986,
+        0.05370106175541878,
+        0.06048284471035004,
+        0.0664696991443634,
+        0.07231792062520981,
+        0.07627613097429276,
+        0.07997097074985504,
+        0.08494995534420013,
+        0.08972473442554474
+      ],
+      "activation_output_statistics": {
+        "first_quartile_mean": 0.03031732654199004,
+        "first_to_last_ratio": 0.3664591129538783,
+        "imbalance_abs_log_ratio": 1.0038683247140607,
+        "last_quartile_mean": 0.08273044787347317,
+        "mean": 0.05450543703045696,
+        "normalized": [
+          0.5273555006590916,
+          0.5399339701348108,
+          0.5613543713807885,
+          0.5962590808162097,
+          0.6499653686150171,
+          0.7145497859400932,
+          0.7923368187501488,
+          0.9056749539963465,
+          0.9852422929002714,
+          1.1096662646068314,
+          1.21950584686113,
+          1.326801958945847,
+          1.399422427007981,
+          1.4672108895332452,
+          1.5585592919240543,
+          1.6461611779281335
+        ],
+        "population_cv": 0.3808757530834043
+      },
+      "bits_per_byte": 6.7509657011324835,
+      "branch_output_rms_by_sublayer": [
+        0.003436450148001313,
+        0.003791899885982275,
+        0.003964710980653763,
+        0.003938332665711641,
+        0.0056790695525705814,
+        0.003919641952961683,
+        0.006960175931453705,
+        0.0038887569680809975,
+        0.008646421134471893,
+        0.003928068559616804,
+        0.009785857982933521,
+        0.0038193853106349707,
+        0.010625048540532589,
+        0.003996006678789854,
+        0.012892307713627815,
+        0.0039014811627566814,
+        0.012757784686982632,
+        0.003861474571749568,
+        0.014726920053362846,
+        0.003962590359151363,
+        0.012820222415030003,
+        0.004551138263195753,
+        0.01401793584227562,
+        0.004161422606557608,
+        0.01284075528383255,
+        0.004107494372874498,
+        0.012963366694748402,
+        0.0034911984112113714,
+        0.014392351731657982,
+        0.004168955609202385,
+        0.013408888131380081,
+        0.003909120801836252
+      ],
+      "capture": {
+        "all_gradients_finite": true,
+        "all_gradients_present": true,
+        "count": 16,
+        "dtypes": [
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32",
+          "torch.float32"
+        ],
+        "position": "post-MLP Transformer-block output; Block AttnRes is captured before aggregation-partial reset",
+        "shape": [
+          16,
+          256,
+          192
+        ],
+        "storage_unique": true
+      },
+      "core_parameter_grad_rms_by_block": [
+        0.0004507690637370199,
+        0.0004530669153600251,
+        0.0005174622333717066,
+        0.0005579942111201043,
+        0.0006494724560479704,
+        0.0007477190266084469,
+        0.0008433132451421535,
+        0.0008983201732199723,
+        0.001008715304343127,
+        0.0011143118429534733,
+        0.0010428854825857738,
+        0.0010916116075341876,
+        0.0011085001103672855,
+        0.0012404769719650786,
+        0.0013545740271648265,
+        0.0014243271893498689
+      ],
+      "core_parameter_grad_statistics": {
+        "first_quartile_mean": 0.000494823105897214,
+        "first_to_last_ratio": 0.385986622193018,
+        "imbalance_abs_log_ratio": 0.9519525676489338,
+        "last_quartile_mean": 0.001281969574711765,
+        "mean": 0.0009064699913044387,
+        "normalized": [
+          0.49727963204644987,
+          0.4998145771025995,
+          0.5708542349284638,
+          0.6155683215912455,
+          0.7164853357289404,
+          0.824869034585972,
+          0.930326710461313,
+          0.99100927977468,
+          1.112795033503044,
+          1.2292870736404011,
+          1.1504909071341995,
+          1.2042446170372658,
+          1.2228756836970622,
+          1.3684699812069823,
+          1.494339625314625,
+          1.571289952246756
+        ],
+        "population_cv": 0.338812252534038
+      },
+      "depth_weights": [],
+      "layer_input_rms_by_sublayer": [
+        0.028241552412509918,
+        0.028533460572361946,
+        0.028743742033839226,
+        0.029236938804388046,
+        0.02942933700978756,
+        0.03035305254161358,
+        0.030596865341067314,
+        0.032195623964071274,
+        0.03249936178326607,
+        0.03506496921181679,
+        0.03542664647102356,
+        0.03852483630180359,
+        0.03894684836268425,
+        0.04251723736524582,
+        0.04318666458129883,
+        0.04861506074666977,
+        0.04936420917510986,
+        0.052915990352630615,
+        0.05370106175541878,
+        0.0598941408097744,
+        0.06048284471035004,
+        0.06538087129592896,
+        0.0664696991443634,
+        0.07142146676778793,
+        0.07231792062520981,
+        0.07521561533212662,
+        0.07627613097429276,
+        0.07935076206922531,
+        0.07997097074985504,
+        0.08368266373872757,
+        0.08494995534420013,
+        0.08860929310321808
+      ],
+      "loss_nats": 4.679412841796875,
+      "loss_scale": 1.0,
+      "output_weights": null,
+      "step": 20,
+      "stream_state_rms_by_sublayer": [
+        0.028533460572361946,
+        0.028743742033839226,
+        0.029236938804388046,
+        0.02942933700978756,
+        0.03035305254161358,
+        0.030596865341067314,
+        0.032195623964071274,
+        0.03249936178326607,
+        0.03506496921181679,
+        0.03542664647102356,
+        0.03852483630180359,
+        0.03894684836268425,
+        0.04251723736524582,
+        0.04318666458129883,
+        0.04861506074666977,
+        0.04936420917510986,
+        0.052915990352630615,
+        0.05370106175541878,
+        0.0598941408097744,
+        0.06048284471035004,
+        0.06538087129592896,
+        0.0664696991443634,
+        0.07142146676778793,
+        0.07231792062520981,
+        0.07521561533212662,
+        0.07627613097429276,
+        0.07935076206922531,
+        0.07997097074985504,
+        0.08368266373872757,
+        0.08494995534420013,
+        0.08860929310321808,
+        0.08972473442554474
+      ]
+    }
+  ],
+  "environment": {
+    "autocast": "cuda-bfloat16-forward-fp32-cross-entropy",
+    "compile": false,
+    "compute_capability": [
+      12,
+      0
+    ],
+    "cublas_workspace_config": ":4096:8",
+    "cuda": "12.8",
+    "deterministic_algorithms": true,
+    "gpu": "NVIDIA GeForce RTX 5090",
+    "python": "3.10.14",
+    "torch": "2.11.0+cu128"
+  },
+  "evaluations": [
+    {
+      "bits_per_byte": 8.076786148009306,
+      "cross_entropy_nats": 5.5984015464782715,
+      "step": 0
+    },
+    {
+      "bits_per_byte": 6.752401670275862,
+      "cross_entropy_nats": 4.680408179759979,
+      "step": 20
+    }
+  ],
+  "gradient_gate": {
+    "first_to_last_ratio_abs_delta": 0.0,
+    "max_abs_scale_ratio_error": 0.0,
+    "normalized_spectrum_max_abs_delta": 0.0,
+    "passed": true,
+    "per_block_scale_ratios": [
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0,
+      2.0
+    ],
+    "population_cv_abs_delta": 0.0,
+    "thresholds": {
+      "scale_ratio_abs": 1e-05,
+      "shape_abs": 1e-06
+    }
+  },
+  "hashes": {
+    "final_mixer_parameters": null,
+    "final_model_state": "d53dab0fa7c76215ab91de676b2aef3f9ef14cb0cc1819b7c4a887915bed97c0",
+    "final_optimizer_state": "a7bce44db1478ce53933758aa5033bbb1e0aa21296e9615c33146920f30a2057",
+    "final_public_parameters": "d53dab0fa7c76215ab91de676b2aef3f9ef14cb0cc1819b7c4a887915bed97c0",
+    "initial_mixer_parameters": null,
+    "initial_public_parameter_elements": 9541824,
+    "initial_public_parameter_structure": "e732db766f25e01f6ce1772cc182ced9de2c56c4a2384130117242b9444f0abe",
+    "initial_public_parameter_tensors": 115,
+    "initial_public_parameters": "af2724a1c34bcfd61d8a8bef402246430898c815e56b6e5c5257949a4eb0e7b1"
+  },
+  "manifest": {
+    "diagnostic_tensor_sha256": "21117e31db302b10d67b63f035665dc8f220b879d216ccd12b7d2ba86e7b1716",
+    "file_sha256": "080afb17d1e036c0bba0a799fdb8b98ee4ad652bd42dd1b3b67110dd2ede6371",
+    "formal_schedule_sha256": "5041e09b167f229248d2462324e8c254b8f5938975f135dcd8192b00a54a4f4e",
+    "input_gate_tensor_hashes": {
+      "0": "65136111a29a042e61a7909132560d95cd4bcf0f9b52f64d0fb2e57773856434",
+      "1": "d995676b4e7dec8f661cd8c2345fe7fc7a513c17f528c02fc946a441a6995a94",
+      "7999": "2345e7ac3decca2bdaebf13094fdc92fcabef42e3e461a2fefd6e8e7e76baccc"
+    },
+    "path": "experiments/k3/attnres_gradient/manifest.json",
+    "validation_tensor_sha256": "f459316f13078a163b47c133511bb7181e05170ab89516e196490113893ce338"
+  },
+  "model": {
+    "attnres_aggregation_groups": 8,
+    "context": 256,
+    "d_ff": 768,
+    "d_head": 32,
+    "d_model": 192,
+    "heads": 6,
+    "layers": 16,
+    "parameters": {
+      "core": 9541824,
+      "embedding": 98304,
+      "mixer": 0,
+      "total": 9541824
+    },
+    "sublayers": 32,
+    "sublayers_per_attnres_group": 4,
+    "transformer_blocks_per_attnres_group": 2,
+    "vocabulary": 256
+  },
+  "optimizer": {
+    "betas": [
+      0.9,
+      0.95
+    ],
+    "epsilon": 1e-08,
+    "grad_clip": 1.0,
+    "min_lr": 3e-05,
+    "name": "AdamW",
+    "peak_lr": 0.0003,
+    "warmup_steps": 400,
+    "weight_decay_ndim_ge_2": 0.1
+  },
+  "protocol_id": "llm-atlas-k3-attnres-gradient-scale-v1",
+  "run_kind": "smoke",
+  "schema_version": 1,
+  "seed": 2026073001,
+  "steps": 20,
+  "target_bytes_seen": 163840,
+  "timing": {
+    "mean_ms": null,
+    "measured_steps": 0,
+    "median_ms": null,
+    "p95_ms": null,
+    "peak_allocated_bytes": 1648265728,
+    "peak_reserved_bytes": 3282042880,
+    "warmup_steps_excluded": 20
+  },
+  "training_history": [
+    {
+      "bits_per_byte": 8.088097790921855,
+      "learning_rate": 7.499999999999999e-07,
+      "loss_nats": 5.6062421798706055,
+      "step": 1,
+      "unclipped_grad_norm": 19.475919723510742
+    },
+    {
+      "bits_per_byte": 7.474302716882146,
+      "learning_rate": 7.499999999999999e-06,
+      "loss_nats": 5.180791854858398,
+      "step": 10,
+      "unclipped_grad_norm": 12.938376426696777
+    },
+    {
+      "bits_per_byte": 6.789615018295581,
+      "learning_rate": 1.4999999999999999e-05,
+      "loss_nats": 4.706202507019043,
+      "step": 20,
+      "unclipped_grad_norm": 4.482712745666504
+    }
+  ]
+}

experiments/k3/attnres_gradient/train.py

@@ -0,0 +1,827 @@
+#!/usr/bin/env python3
+"""Run one preregistered AttnRes activation-gradient/depth experiment cell."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import importlib.util
+import json
+import math
+import os
+import platform
+import statistics
+import sys
+import time
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Iterable
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-gradient-scale-v1"
+ARCHITECTURES = ("baseline", "block")
+DEPTHS = (16, 32)
+EXPECTED_SEEDS = (2026073001, 2026073002, 2026073003)
+DIAGNOSTIC_STEPS = (0, 100, 500, 2000, 4000, 8000)
+FORMAL_STEPS = 8000
+SMOKE_STEPS = 20
+CONTEXT = 256
+VOCABULARY = 256
+BLOCK_GROUPS = 8
+PEAK_LR = 3e-4
+MIN_LR = 3e-5
+WARMUP_STEPS = 400
+WEIGHT_DECAY = 0.1
+BETAS = (0.9, 0.95)
+ADAM_EPS = 1e-8
+GRAD_CLIP = 1.0
+
+
+def load_round04_module() -> Any:
+    path = Path(__file__).resolve().parents[1] / "attnres" / "train.py"
+    spec = importlib.util.spec_from_file_location("k3_attnres_round04_train", path)
+    if spec is None or spec.loader is None:
+        raise RuntimeError(f"cannot import Round 04 runner from {path}")
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[spec.name] = module
+    spec.loader.exec_module(module)
+    return module
+
+
+round04 = load_round04_module()
+
+
+def configure_round04_globals(depth: int) -> None:
+    round04.PROTOCOL_ID = PROTOCOL_ID
+    round04.LAYERS = depth
+    round04.SUBLAYERS = depth * 2
+    round04.BLOCKS = BLOCK_GROUPS
+    round04.SUBLAYERS_PER_BLOCK = (depth * 2) // BLOCK_GROUPS
+    round04.WARMUP_STEPS = WARMUP_STEPS
+    round04.EVAL_STEPS = DIAGNOSTIC_STEPS
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--architecture", choices=ARCHITECTURES, required=True)
+    parser.add_argument("--depth", type=int, choices=DEPTHS, required=True)
+    parser.add_argument("--seed", type=int, required=True)
+    parser.add_argument("--steps", type=int)
+    parser.add_argument("--batch-size", type=int, default=32)
+    parser.add_argument("--cache-dir", type=Path, required=True)
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument("--validation-windows", type=int, default=64)
+    parser.add_argument("--diagnostic-windows", type=int, default=16)
+    parser.add_argument("--eval-batch-size", type=int, default=8)
+    parser.add_argument("--timing-warmup", type=int, default=20)
+    parser.add_argument(
+        "--run-kind", choices=("smoke", "formal", "replay"), default="formal"
+    )
+    args = parser.parse_args()
+    expected_steps = SMOKE_STEPS if args.run_kind == "smoke" else FORMAL_STEPS
+    if args.steps is None:
+        args.steps = expected_steps
+    if args.steps != expected_steps:
+        raise ValueError(
+            f"{args.run_kind} must run exactly {expected_steps} steps, got {args.steps}"
+        )
+    if args.batch_size != 32:
+        raise ValueError("the frozen protocol requires batch size 32")
+    if args.validation_windows != 64 or args.diagnostic_windows != 16:
+        raise ValueError("the frozen protocol requires 64 validation / 16 diagnostic windows")
+    return args
+
+
+def configure_determinism(seed: int) -> None:
+    if os.environ.get("CUBLAS_WORKSPACE_CONFIG") != ":4096:8":
+        raise RuntimeError("CUBLAS_WORKSPACE_CONFIG must be :4096:8 before Python starts")
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed_all(seed)
+    torch.use_deterministic_algorithms(True)
+    torch.backends.cudnn.benchmark = False
+    torch.backends.cudnn.deterministic = True
+    torch.backends.cuda.matmul.allow_tf32 = False
+    torch.backends.cudnn.allow_tf32 = False
+    torch.set_float32_matmul_precision("highest")
+
+
+def canonical_sha256(value: Any) -> str:
+    payload = json.dumps(
+        value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+    ).encode()
+    return hashlib.sha256(payload).hexdigest()
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def tensor_bytes(tensor: torch.Tensor) -> bytes:
+    value = tensor.detach().cpu().contiguous()
+    return (
+        f"{value.dtype}|{tuple(value.shape)}|".encode()
+        + value.reshape(-1).view(torch.uint8).numpy().tobytes()
+    )
+
+
+def named_state_hash(
+    model: nn.Module, *, include_mixers: bool | None
+) -> str:
+    digest = hashlib.sha256()
+    for name, tensor in sorted(model.state_dict().items()):
+        is_mixer = name.startswith("mixers.") or name.startswith("output_mixer.")
+        if include_mixers is not None and is_mixer != include_mixers:
+            continue
+        digest.update(name.encode())
+        digest.update(b"\0")
+        digest.update(tensor_bytes(tensor))
+    return digest.hexdigest()
+
+
+def state_structure_hash(
+    model: nn.Module, *, include_mixers: bool | None
+) -> tuple[str, int, int]:
+    digest = hashlib.sha256()
+    tensor_count = 0
+    element_count = 0
+    for name, tensor in sorted(model.state_dict().items()):
+        is_mixer = name.startswith("mixers.") or name.startswith("output_mixer.")
+        if include_mixers is not None and is_mixer != include_mixers:
+            continue
+        digest.update(
+            f"{name}|{tuple(tensor.shape)}|{tensor.dtype}|{tensor.numel()}\n".encode()
+        )
+        tensor_count += 1
+        element_count += tensor.numel()
+    return digest.hexdigest(), tensor_count, element_count
+
+
+def recursive_state_hash(value: Any) -> str:
+    digest = hashlib.sha256()
+
+    def visit(path: str, item: Any) -> None:
+        if torch.is_tensor(item):
+            digest.update(f"{path}|tensor|".encode())
+            digest.update(tensor_bytes(item))
+        elif isinstance(item, dict):
+            digest.update(f"{path}|dict|{len(item)}\n".encode())
+            for key in sorted(item, key=lambda candidate: str(candidate)):
+                visit(f"{path}/{key}", item[key])
+        elif isinstance(item, (list, tuple)):
+            digest.update(f"{path}|sequence|{len(item)}\n".encode())
+            for index, child in enumerate(item):
+                visit(f"{path}/{index}", child)
+        else:
+            digest.update(f"{path}|scalar|{repr(item)}\n".encode())
+
+    visit("root", value)
+    return digest.hexdigest()
+
+
+@dataclass
+class ActivationTrace:
+    block_outputs: list[torch.Tensor]
+    layer_input_rms: list[float]
+    branch_output_rms: list[float]
+    stream_state_rms: list[float]
+    depth_weights: list[dict[str, Any]]
+    output_weights: dict[str, Any] | None = None
+
+
+def rms(value: torch.Tensor) -> float:
+    return value.float().square().mean().sqrt().detach().cpu().item()
+
+
+class GradientLanguageModel(round04.ReducedLanguageModel):
+    """Round 04 trunk with aligned post-MLP activation capture."""
+
+    def forward(
+        self, input_ids: torch.Tensor, capture: bool = False
+    ) -> tuple[torch.Tensor, ActivationTrace | None]:
+        embedded = self.embed(input_ids)
+        trace = ActivationTrace([], [], [], [], []) if capture else None
+
+        if self.architecture == "baseline":
+            hidden = embedded
+            for block in self.blocks:
+                attention_input = hidden
+                attention_output = block.attention(block.attention_norm(attention_input))
+                hidden = hidden + attention_output
+                if trace is not None:
+                    trace.layer_input_rms.append(rms(attention_input))
+                    trace.branch_output_rms.append(rms(attention_output))
+                    trace.stream_state_rms.append(rms(hidden))
+                mlp_input = hidden
+                mlp_output = block.mlp(block.mlp_norm(mlp_input))
+                hidden = hidden + mlp_output
+                if trace is not None:
+                    hidden.retain_grad()
+                    trace.block_outputs.append(hidden)
+                    trace.layer_input_rms.append(rms(mlp_input))
+                    trace.branch_output_rms.append(rms(mlp_output))
+                    trace.stream_state_rms.append(rms(hidden))
+        else:
+            completed = [embedded]
+            partial: torch.Tensor | None = None
+            mixer_index = 0
+            for block in self.blocks:
+                for branch_index in range(2):
+                    sources = completed + ([] if partial is None else [partial])
+                    branch_input, weights = self.mixers[mixer_index](
+                        sources, capture
+                    )
+                    mixer_index += 1
+                    if branch_index == 0:
+                        branch_output = block.attention(
+                            block.attention_norm(branch_input)
+                        )
+                    else:
+                        branch_output = block.mlp(block.mlp_norm(branch_input))
+                    branch_for_residual = branch_output.float()
+                    partial = (
+                        branch_for_residual
+                        if partial is None
+                        else partial + branch_for_residual
+                    )
+                    if trace is not None:
+                        trace.layer_input_rms.append(rms(branch_input))
+                        trace.branch_output_rms.append(rms(branch_output))
+                        trace.stream_state_rms.append(rms(partial))
+                        trace.depth_weights.append(weights or {})
+                        if branch_index == 1:
+                            partial.retain_grad()
+                            trace.block_outputs.append(partial)
+                    if mixer_index % round04.SUBLAYERS_PER_BLOCK == 0:
+                        completed.append(partial)
+                        partial = None
+            if partial is not None or len(completed) != BLOCK_GROUPS + 1:
+                raise RuntimeError("Block AttnRes aggregation contract failed")
+            if self.output_mixer is None:
+                raise RuntimeError("Block AttnRes output mixer missing")
+            hidden, output_weights = self.output_mixer(completed, capture)
+            if trace is not None:
+                trace.output_weights = output_weights
+
+        normalized = self.final_norm(hidden)
+        logits = F.linear(normalized, self.token_embedding.weight)
+        return logits, trace
+
+
+def cross_entropy(logits: torch.Tensor, targets: torch.Tensor) -> torch.Tensor:
+    return F.cross_entropy(
+        logits.float().reshape(-1, VOCABULARY), targets.reshape(-1)
+    )
+
+
+def learning_rate(step: int, total_steps: int) -> float:
+    if step <= WARMUP_STEPS:
+        return PEAK_LR * step / WARMUP_STEPS
+    progress = (step - WARMUP_STEPS) / max(1, total_steps - WARMUP_STEPS)
+    cosine = 0.5 * (1 + math.cos(math.pi * progress))
+    return MIN_LR + (PEAK_LR - MIN_LR) * cosine
+
+
+@torch.no_grad()
+def evaluate(
+    model: GradientLanguageModel,
+    corpus: Any,
+    window_count: int,
+    eval_batch_size: int,
+) -> dict[str, float]:
+    model.eval()
+    loss_sum = 0.0
+    target_count = 0
+    for begin in range(0, window_count, eval_batch_size):
+        end = min(begin + eval_batch_size, window_count)
+        inputs, targets = corpus.fixed_batch(
+            corpus.validation_starts, begin, end
+        )
+        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+            logits, _ = model(inputs)
+        loss = F.cross_entropy(
+            logits.float().reshape(-1, VOCABULARY),
+            targets.reshape(-1),
+            reduction="sum",
+        )
+        loss_sum += loss.detach().cpu().item()
+        target_count += targets.numel()
+    nats = loss_sum / target_count
+    return {"cross_entropy_nats": nats, "bits_per_byte": nats / math.log(2)}
+
+
+def mean(values: Iterable[float]) -> float:
+    return statistics.fmean(values)
+
+
+def depth_statistics(values: list[float]) -> dict[str, Any]:
+    average = mean(values)
+    variance = mean((value - average) ** 2 for value in values)
+    quartile = len(values) // 4
+    first = mean(values[:quartile])
+    last = mean(values[-quartile:])
+    ratio = first / last
+    return {
+        "mean": average,
+        "population_cv": math.sqrt(variance) / average,
+        "normalized": [value / average for value in values],
+        "first_quartile_mean": first,
+        "last_quartile_mean": last,
+        "first_to_last_ratio": ratio,
+        "imbalance_abs_log_ratio": abs(math.log(ratio)),
+    }
+
+
+def core_parameter_gradient_rms(model: GradientLanguageModel) -> list[float]:
+    values = []
+    for block in model.blocks:
+        sum_square = 0.0
+        count = 0
+        for parameter in block.parameters():
+            if parameter.grad is None:
+                raise RuntimeError("missing core parameter gradient")
+            gradient = parameter.grad.detach().float()
+            if not torch.isfinite(gradient).all():
+                raise RuntimeError("non-finite core parameter gradient")
+            sum_square += gradient.square().sum().detach().cpu().item()
+            count += gradient.numel()
+        values.append(math.sqrt(sum_square / count))
+    return values
+
+
+def activation_storage_unique(outputs: list[torch.Tensor]) -> bool:
+    pointers = [output.untyped_storage().data_ptr() for output in outputs]
+    return len(pointers) == len(set(pointers))
+
+
+def diagnostic(
+    model: GradientLanguageModel,
+    corpus: Any,
+    window_count: int,
+    *,
+    loss_scale: float = 1.0,
+) -> dict[str, Any]:
+    model.eval()
+    model.zero_grad(set_to_none=True)
+    inputs, targets = corpus.fixed_batch(
+        corpus.diagnostic_starts, 0, window_count
+    )
+    with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+        logits, trace = model(inputs, capture=True)
+        unscaled_loss = cross_entropy(logits, targets)
+        loss = unscaled_loss * loss_scale
+    if trace is None or len(trace.block_outputs) != len(model.blocks):
+        raise RuntimeError("aligned activation capture count mismatch")
+    expected_shape = (window_count, CONTEXT, round04.D_MODEL)
+    if any(tuple(output.shape) != expected_shape for output in trace.block_outputs):
+        raise RuntimeError("aligned activation capture shape mismatch")
+    if any(output.dtype != torch.float32 for output in trace.block_outputs):
+        raise RuntimeError("aligned activation capture must use FP32 residual state")
+    if not activation_storage_unique(trace.block_outputs):
+        raise RuntimeError("captured block outputs alias storage")
+    loss.backward()
+
+    activation_grad_rms = []
+    activation_output_rms = []
+    activation_dtypes = []
+    for output in trace.block_outputs:
+        if output.grad is None:
+            raise RuntimeError("captured activation gradient is None")
+        gradient = output.grad.detach().float()
+        if not torch.isfinite(gradient).all():
+            raise RuntimeError("captured activation gradient is non-finite")
+        activation_grad_rms.append(
+            gradient.square().mean().sqrt().detach().cpu().item()
+        )
+        activation_output_rms.append(rms(output))
+        activation_dtypes.append(str(output.dtype))
+    parameter_grad_rms = core_parameter_gradient_rms(model)
+
+    return {
+        "loss_nats": unscaled_loss.detach().cpu().item(),
+        "bits_per_byte": unscaled_loss.detach().cpu().item() / math.log(2),
+        "loss_scale": loss_scale,
+        "capture": {
+            "count": len(trace.block_outputs),
+            "shape": list(expected_shape),
+            "dtypes": activation_dtypes,
+            "all_gradients_finite": True,
+            "all_gradients_present": True,
+            "storage_unique": True,
+            "position": (
+                "post-MLP Transformer-block output; Block AttnRes is captured "
+                "before aggregation-partial reset"
+            ),
+        },
+        "activation_grad_rms_by_block": activation_grad_rms,
+        "activation_grad_statistics": depth_statistics(activation_grad_rms),
+        "activation_output_rms_by_block": activation_output_rms,
+        "activation_output_statistics": depth_statistics(activation_output_rms),
+        "core_parameter_grad_rms_by_block": parameter_grad_rms,
+        "core_parameter_grad_statistics": depth_statistics(parameter_grad_rms),
+        "layer_input_rms_by_sublayer": trace.layer_input_rms,
+        "branch_output_rms_by_sublayer": trace.branch_output_rms,
+        "stream_state_rms_by_sublayer": trace.stream_state_rms,
+        "depth_weights": trace.depth_weights,
+        "output_weights": trace.output_weights,
+    }
+
+
+def loss_scale_gate(
+    model: GradientLanguageModel, corpus: Any, window_count: int
+) -> tuple[dict[str, Any], dict[str, Any]]:
+    base = diagnostic(model, corpus, window_count, loss_scale=1.0)
+    doubled = diagnostic(model, corpus, window_count, loss_scale=2.0)
+    base_values = base["activation_grad_rms_by_block"]
+    doubled_values = doubled["activation_grad_rms_by_block"]
+    ratios = [
+        doubled_value / base_value
+        for base_value, doubled_value in zip(base_values, doubled_values)
+    ]
+    base_stats = base["activation_grad_statistics"]
+    doubled_stats = doubled["activation_grad_statistics"]
+    cv_delta = abs(
+        doubled_stats["population_cv"] - base_stats["population_cv"]
+    )
+    ratio_delta = abs(
+        doubled_stats["first_to_last_ratio"]
+        - base_stats["first_to_last_ratio"]
+    )
+    normalized_max_delta = max(
+        abs(left - right)
+        for left, right in zip(
+            base_stats["normalized"], doubled_stats["normalized"]
+        )
+    )
+    passed = (
+        all(abs(ratio - 2.0) <= 1e-5 for ratio in ratios)
+        and cv_delta <= 1e-6
+        and ratio_delta <= 1e-6
+        and normalized_max_delta <= 1e-6
+    )
+    gate = {
+        "passed": passed,
+        "per_block_scale_ratios": ratios,
+        "max_abs_scale_ratio_error": max(abs(ratio - 2.0) for ratio in ratios),
+        "population_cv_abs_delta": cv_delta,
+        "first_to_last_ratio_abs_delta": ratio_delta,
+        "normalized_spectrum_max_abs_delta": normalized_max_delta,
+        "thresholds": {
+            "scale_ratio_abs": 1e-5,
+            "shape_abs": 1e-6,
+        },
+    }
+    if not passed:
+        raise RuntimeError(f"loss-scale diagnostic gate failed: {gate}")
+    return base, gate
+
+
+def percentile(values: list[float], quantile: float) -> float:
+    return float(np.quantile(np.asarray(values, dtype=np.float64), quantile))
+
+
+def parameter_inventory(model: GradientLanguageModel) -> dict[str, int]:
+    total = sum(parameter.numel() for parameter in model.parameters())
+    mixer = sum(
+        parameter.numel()
+        for name, parameter in model.named_parameters()
+        if name.startswith("mixers.") or name.startswith("output_mixer.")
+    )
+    return {
+        "total": total,
+        "core": total - mixer,
+        "mixer": mixer,
+        "embedding": (
+            model.token_embedding.weight.numel()
+            + model.position_embedding.weight.numel()
+        ),
+    }
+
+
+def model_input_gate_hashes(
+    corpus: Any, manifest: dict[str, Any], seed: int, batch_size: int
+) -> dict[str, str]:
+    values: dict[str, str] = {}
+    for step in manifest["windows"]["gate_steps"]:
+        raw_digest = hashlib.sha256()
+        for row in range(batch_size):
+            start = round04.window_start(seed, step, row, len(corpus.train))
+            raw_digest.update(
+                np.asarray(
+                    corpus.train[start : start + CONTEXT + 1], dtype=np.uint8
+                ).tobytes()
+            )
+        expected_raw_hash = manifest["windows"][
+            "gate_training_tensor_sha256"
+        ][str(seed)][str(step)]
+        if raw_digest.hexdigest() != expected_raw_hash:
+            raise RuntimeError(f"manifest gate tensor mismatch at step {step}")
+        inputs, targets = corpus.training_batch(seed, step, batch_size)
+        digest = hashlib.sha256()
+        digest.update(tensor_bytes(inputs))
+        digest.update(tensor_bytes(targets))
+        values[str(step)] = digest.hexdigest()
+    return values
+
+
+def main() -> None:
+    args = parse_args()
+    if not torch.cuda.is_available():
+        raise RuntimeError("CUDA is required by the frozen protocol")
+    if args.seed not in EXPECTED_SEEDS:
+        raise ValueError(f"seed is not preregistered: {args.seed}")
+    configure_round04_globals(args.depth)
+    configure_determinism(args.seed)
+    device = torch.device("cuda")
+
+    manifest = json.loads(args.manifest.read_text())
+    if manifest["protocol_id"] != PROTOCOL_ID:
+        raise ValueError("manifest protocol mismatch")
+    if manifest["windows"]["formal_steps"] != FORMAL_STEPS:
+        raise ValueError("manifest formal-step mismatch")
+    corpus = round04.ByteCorpus(args.cache_dir, manifest, device)
+
+    model = GradientLanguageModel(args.architecture).to(device)
+    public_structure_hash, public_tensors, public_elements = state_structure_hash(
+        model, include_mixers=False
+    )
+    initial_public_hash = named_state_hash(model, include_mixers=False)
+    initial_mixer_hash = (
+        named_state_hash(model, include_mixers=True)
+        if args.architecture == "block"
+        else None
+    )
+    input_gate_hashes = model_input_gate_hashes(
+        corpus, manifest, args.seed, args.batch_size
+    )
+
+    decay_parameters: list[nn.Parameter] = []
+    no_decay_parameters: list[nn.Parameter] = []
+    for parameter in model.parameters():
+        (decay_parameters if parameter.ndim >= 2 else no_decay_parameters).append(
+            parameter
+        )
+    optimizer = torch.optim.AdamW(
+        [
+            {"params": decay_parameters, "weight_decay": WEIGHT_DECAY},
+            {"params": no_decay_parameters, "weight_decay": 0.0},
+        ],
+        lr=PEAK_LR,
+        betas=BETAS,
+        eps=ADAM_EPS,
+    )
+
+    evaluation_steps = sorted(
+        set(step for step in DIAGNOSTIC_STEPS if step <= args.steps)
+        | {0, args.steps}
+    )
+    evaluations = [
+        {
+            "step": 0,
+            **evaluate(
+                model, corpus, args.validation_windows, args.eval_batch_size
+            ),
+        }
+    ]
+    if args.run_kind == "smoke":
+        initial_diagnostic, gradient_gate = loss_scale_gate(
+            model, corpus, args.diagnostic_windows
+        )
+    else:
+        initial_diagnostic = diagnostic(
+            model, corpus, args.diagnostic_windows
+        )
+        gradient_gate = None
+    diagnostics = [{"step": 0, **initial_diagnostic}]
+    print(
+        json.dumps(
+            {
+                "event": "diagnostic",
+                "step": 0,
+                "architecture": args.architecture,
+                "depth": args.depth,
+                "validation_bpc": evaluations[0]["bits_per_byte"],
+                "activation_gradient_cv": initial_diagnostic[
+                    "activation_grad_statistics"
+                ]["population_cv"],
+            },
+            sort_keys=True,
+        ),
+        flush=True,
+    )
+    model.zero_grad(set_to_none=True)
+
+    training_history: list[dict[str, float | int]] = []
+    step_times: list[float] = []
+    model.train()
+    for step in range(1, args.steps + 1):
+        lr = learning_rate(step, args.steps)
+        for group in optimizer.param_groups:
+            group["lr"] = lr
+        inputs, targets = corpus.training_batch(args.seed, step, args.batch_size)
+        optimizer.zero_grad(set_to_none=True)
+
+        torch.cuda.synchronize()
+        started = time.perf_counter()
+        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+            logits, _ = model(inputs)
+            loss = cross_entropy(logits, targets)
+        if not torch.isfinite(loss):
+            raise RuntimeError(f"non-finite loss at step {step}: {loss}")
+        loss.backward()
+        unclipped_norm = torch.nn.utils.clip_grad_norm_(
+            model.parameters(), GRAD_CLIP
+        )
+        optimizer.step()
+        torch.cuda.synchronize()
+        elapsed_ms = (time.perf_counter() - started) * 1000
+
+        if step == args.timing_warmup:
+            torch.cuda.reset_peak_memory_stats()
+        elif step > args.timing_warmup:
+            step_times.append(elapsed_ms)
+        if step == 1 or step % 10 == 0 or step == args.steps:
+            training_history.append(
+                {
+                    "step": step,
+                    "loss_nats": loss.detach().cpu().item(),
+                    "bits_per_byte": loss.detach().cpu().item() / math.log(2),
+                    "learning_rate": lr,
+                    "unclipped_grad_norm": float(unclipped_norm.detach().cpu()),
+                }
+            )
+
+        if step in evaluation_steps and step != 0:
+            evaluations.append(
+                {
+                    "step": step,
+                    **evaluate(
+                        model,
+                        corpus,
+                        args.validation_windows,
+                        args.eval_batch_size,
+                    ),
+                }
+            )
+            diagnostics.append(
+                {
+                    "step": step,
+                    **diagnostic(
+                        model, corpus, args.diagnostic_windows
+                    ),
+                }
+            )
+            print(
+                json.dumps(
+                    {
+                        "event": "diagnostic",
+                        "step": step,
+                        "architecture": args.architecture,
+                        "depth": args.depth,
+                        "validation_bpc": evaluations[-1]["bits_per_byte"],
+                        "activation_gradient_cv": diagnostics[-1][
+                            "activation_grad_statistics"
+                        ]["population_cv"],
+                    },
+                    sort_keys=True,
+                ),
+                flush=True,
+            )
+            model.zero_grad(set_to_none=True)
+            model.train()
+
+    training_peak_allocated = torch.cuda.max_memory_allocated()
+    training_peak_reserved = torch.cuda.max_memory_reserved()
+    final_public_hash = named_state_hash(model, include_mixers=False)
+    final_mixer_hash = (
+        named_state_hash(model, include_mixers=True)
+        if args.architecture == "block"
+        else None
+    )
+    final_full_hash = named_state_hash(model, include_mixers=None)
+    optimizer_hash = recursive_state_hash(optimizer.state_dict())
+    timing = {
+        "warmup_steps_excluded": args.timing_warmup,
+        "measured_steps": len(step_times),
+        "mean_ms": mean(step_times) if step_times else None,
+        "median_ms": statistics.median(step_times) if step_times else None,
+        "p95_ms": percentile(step_times, 0.95) if step_times else None,
+        "peak_allocated_bytes": training_peak_allocated,
+        "peak_reserved_bytes": training_peak_reserved,
+    }
+    result = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "run_kind": args.run_kind,
+        "architecture": args.architecture,
+        "depth": args.depth,
+        "seed": args.seed,
+        "steps": args.steps,
+        "batch_size": args.batch_size,
+        "target_bytes_seen": args.steps * args.batch_size * CONTEXT,
+        "manifest": {
+            "path": str(args.manifest),
+            "file_sha256": file_sha256(args.manifest),
+            "formal_schedule_sha256": manifest["windows"][
+                "formal_schedule_sha256"
+            ],
+            "validation_tensor_sha256": manifest["windows"][
+                "validation_tensor_sha256"
+            ],
+            "diagnostic_tensor_sha256": manifest["windows"][
+                "diagnostic_tensor_sha256"
+            ],
+            "input_gate_tensor_hashes": input_gate_hashes,
+        },
+        "model": {
+            "layers": args.depth,
+            "sublayers": args.depth * 2,
+            "attnres_aggregation_groups": BLOCK_GROUPS,
+            "sublayers_per_attnres_group": args.depth * 2 // BLOCK_GROUPS,
+            "transformer_blocks_per_attnres_group": args.depth // BLOCK_GROUPS,
+            "d_model": round04.D_MODEL,
+            "heads": round04.HEADS,
+            "d_head": round04.D_HEAD,
+            "d_ff": round04.D_FF,
+            "context": CONTEXT,
+            "vocabulary": VOCABULARY,
+            "parameters": parameter_inventory(model),
+        },
+        "optimizer": {
+            "name": "AdamW",
+            "betas": list(BETAS),
+            "epsilon": ADAM_EPS,
+            "weight_decay_ndim_ge_2": WEIGHT_DECAY,
+            "peak_lr": PEAK_LR,
+            "min_lr": MIN_LR,
+            "warmup_steps": WARMUP_STEPS,
+            "grad_clip": GRAD_CLIP,
+        },
+        "hashes": {
+            "initial_public_parameter_structure": public_structure_hash,
+            "initial_public_parameter_tensors": public_tensors,
+            "initial_public_parameter_elements": public_elements,
+            "initial_public_parameters": initial_public_hash,
+            "initial_mixer_parameters": initial_mixer_hash,
+            "final_public_parameters": final_public_hash,
+            "final_mixer_parameters": final_mixer_hash,
+            "final_model_state": final_full_hash,
+            "final_optimizer_state": optimizer_hash,
+        },
+        "evaluations": evaluations,
+        "diagnostics": diagnostics,
+        "training_history": training_history,
+        "gradient_gate": gradient_gate,
+        "timing": timing,
+        "environment": {
+            "python": platform.python_version(),
+            "torch": torch.__version__,
+            "cuda": torch.version.cuda,
+            "gpu": torch.cuda.get_device_name(0),
+            "compute_capability": list(torch.cuda.get_device_capability(0)),
+            "cublas_workspace_config": os.environ["CUBLAS_WORKSPACE_CONFIG"],
+            "deterministic_algorithms": torch.are_deterministic_algorithms_enabled(),
+            "autocast": "cuda-bfloat16-forward-fp32-cross-entropy",
+            "compile": False,
+        },
+    }
+    result["canonical_sha256_without_self"] = canonical_sha256(result)
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    temporary = args.output.with_suffix(args.output.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    os.replace(temporary, args.output)
+    print(
+        json.dumps(
+            {
+                "output": str(args.output),
+                "run_kind": args.run_kind,
+                "architecture": args.architecture,
+                "depth": args.depth,
+                "seed": args.seed,
+                "steps": args.steps,
+                "final_bpc": evaluations[-1]["bits_per_byte"],
+                "final_activation_gradient_cv": diagnostics[-1][
+                    "activation_grad_statistics"
+                ]["population_cv"],
+                "canonical_sha256": result["canonical_sha256_without_self"],
+                "timing": timing,
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_local_path/README.md

@@ -0,0 +1,50 @@
+# Attention Residuals local mixer-path diagnostics
+
+This directory implements preregistered protocol
+`llm-atlas-k3-attnres-local-path-v1`.
+
+It is a targeted follow-up to Round 06. It exact-replays the same depth-32
+Block training and keeps the learned forward unchanged while switching source
+value-gradient coefficients only at frozen mixer scopes. It is not a Kimi K3
+checkpoint run, a trainable variant, an additive attribution, or a reproduction
+of unpublished Figure 5 telemetry.
+
+## Frozen environment
+
+```text
+Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
+PyTorch 2.11.0+cu128
+GPU NVIDIA GeForce RTX 5090
+CUBLAS_WORKSPACE_CONFIG=:4096:8
+```
+
+The preregistration was committed as `6911efc` before the runner or any result
+file existed.
+
+## Step-0 smoke
+
+```bash
+CUBLAS_WORKSPACE_CONFIG=:4096:8 \
+/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
+  experiments/k3/attnres_local_path/train.py \
+  --run-kind smoke \
+  --seed 2026073001 \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
+  --data-manifest experiments/k3/attnres_gradient/manifest.json \
+  --parent-manifest experiments/k3/attnres_spike/manifest.json \
+  --manifest experiments/k3/attnres_local_path/manifest.json \
+  --output /home/wuyang/.cache/llm-atlas/k3-attnres-local-path-v1/smoke/seed-2026073001.json
+```
+
+The first smoke passed all 14-mode forward-identity, selector, Round 06 endpoint,
+initialization-negative-control, parent-learned, and loss-scale gates. Its
+canonical content hash is
+`f708200f4fb122f61f30b97393839382a2094a71a8ea5d48cf47fb7fa094e69b`.
+
+Formal cells use the same command with `--run-kind formal`, one of the three
+manifest seeds, and a new output path. The independent seed-2026073001 run uses
+`--run-kind replay`.
+
+Only `analyze.py` may calculate the global log gap, sufficiency/restoration
+scores, and preregistered gates. The site consumes its frozen aggregate rather
+than reimplementing thresholds in TypeScript.

experiments/k3/attnres_local_path/analyze.py

@@ -0,0 +1,498 @@
+#!/usr/bin/env python3
+"""Aggregate and gate preregistered Round 07 local-path results."""
+
+from __future__ import annotations
+
+import argparse
+import copy
+import hashlib
+import json
+import math
+import statistics
+from pathlib import Path
+from typing import Any
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-local-path-v1"
+METRICS = ("spike_contrast", "peak_normalized")
+SUFFICIENCY_MODES = (
+    "uniform_group_6_only",
+    "uniform_group_7_only",
+    "uniform_groups_6_7_only",
+    "uniform_group_6_attention_only",
+    "uniform_group_6_mlp_only",
+    "uniform_group_7_attention_only",
+    "uniform_group_7_mlp_only",
+    "uniform_output_only",
+    "uniform_depth_all",
+    "uniform_all",
+)
+RESTORATION_MODES = (
+    "uniform_except_group_6",
+    "uniform_except_group_7",
+    "uniform_except_groups_6_7",
+)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument(
+        "--formal", type=Path, action="append", required=True
+    )
+    parser.add_argument("--replay", type=Path, required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def canonical_sha256(value: Any) -> str:
+    payload = json.dumps(
+        value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+    ).encode()
+    return hashlib.sha256(payload).hexdigest()
+
+
+def read_result(path: Path) -> dict[str, Any]:
+    value = json.loads(path.read_text())
+    if value["protocol_id"] != PROTOCOL_ID:
+        raise RuntimeError(f"protocol mismatch: {path}")
+    expected = value["canonical_sha256_without_self"]
+    without_self = {
+        key: item
+        for key, item in value.items()
+        if key != "canonical_sha256_without_self"
+    }
+    if canonical_sha256(without_self) != expected:
+        raise RuntimeError(f"canonical self-hash mismatch: {path}")
+    return value
+
+
+def replay_payload(value: dict[str, Any]) -> dict[str, Any]:
+    cleaned = copy.deepcopy(value)
+    for key in ("run_kind", "timing", "canonical_sha256_without_self"):
+        cleaned.pop(key)
+    return cleaned
+
+
+def primary_metrics(mode: dict[str, Any]) -> dict[str, float]:
+    stats = mode["positions"]["post_mlp_state"]["reductions"][
+        "element_rms"
+    ]["statistics"]
+    return {name: float(stats[name]) for name in METRICS}
+
+
+def all_true(values: list[bool]) -> bool:
+    return len(values) > 0 and all(values)
+
+
+def mixed_status(
+    by_seed_metric: dict[str, dict[str, dict[str, Any]]],
+    threshold: float,
+) -> dict[str, Any]:
+    passed = []
+    signs = []
+    metric_passes = {metric: [] for metric in METRICS}
+    for seed_values in by_seed_metric.values():
+        for metric in METRICS:
+            score = seed_values[metric]["score"]
+            cell_passed = score is not None and score >= threshold
+            passed.append(cell_passed)
+            metric_passes[metric].append(cell_passed)
+            if score is not None:
+                signs.append(1 if score >= 0 else -1)
+    reasons = []
+    if any(passed) and not all(passed):
+        reasons.append("seed_or_metric_pass_split")
+    if metric_passes[METRICS[0]] != metric_passes[METRICS[1]]:
+        reasons.append("metric_direction_split")
+    if len(set(signs)) > 1:
+        reasons.append("score_sign_split")
+    return {
+        "passed": all_true(passed),
+        "threshold": threshold,
+        "required_cells": len(passed),
+        "passed_cells": sum(passed),
+        "mixed": bool(reasons),
+        "mixed_reasons": reasons,
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    manifest = json.loads(args.manifest.read_text())
+    if manifest["protocol_id"] != PROTOCOL_ID:
+        raise RuntimeError("manifest protocol mismatch")
+    if len(args.formal) != 3:
+        raise RuntimeError("exactly three formal result paths are required")
+
+    formal_pairs = [(path, read_result(path)) for path in args.formal]
+    formal_pairs.sort(key=lambda item: item[1]["seed"])
+    expected_seeds = manifest["formal_seeds"]
+    if [value["seed"] for _, value in formal_pairs] != expected_seeds:
+        raise RuntimeError("formal seeds do not match manifest")
+    for path, value in formal_pairs:
+        if value["run_kind"] != "formal" or value["steps"] != 8000:
+            raise RuntimeError(f"invalid formal cell: {path}")
+        if not value["round06_equivalence"]["passed"]:
+            raise RuntimeError(f"parent equivalence failed: {path}")
+        for diagnostic in value["diagnostics"]:
+            if not diagnostic["parent_learned_round06_exact"]:
+                raise RuntimeError(f"parent diagnostic mismatch: {path}")
+            if diagnostic["local_matrix"] is not None:
+                for gate in ("forward_identity_gate", "endpoint_exactness"):
+                    if not diagnostic[gate]["passed"]:
+                        raise RuntimeError(f"{gate} failed: {path}")
+                for mode in manifest["matrix_modes"]:
+                    if not diagnostic["local_matrix"][mode]["selector"][
+                        "passed"
+                    ]:
+                        raise RuntimeError(f"selector failed: {path}:{mode}")
+        step0 = value["diagnostics"][0]
+        if (
+            not step0["initialization_negative_control"]["passed"]
+            or not step0["loss_scale_gate"]["passed"]
+        ):
+            raise RuntimeError(f"step-0 control failed: {path}")
+
+    replay = read_result(args.replay)
+    if (
+        replay["run_kind"] != "replay"
+        or replay["seed"] != expected_seeds[0]
+        or replay["steps"] != 8000
+    ):
+        raise RuntimeError("invalid replay cell")
+    replay_exact = (
+        replay_payload(formal_pairs[0][1]) == replay_payload(replay)
+    )
+    if not replay_exact:
+        raise RuntimeError("formal seed1 and replay are not canonical exact")
+
+    thresholds = manifest["thresholds"]
+    cells = []
+    sufficiency_by_mode: dict[str, dict[str, dict[str, Any]]] = {
+        mode: {} for mode in SUFFICIENCY_MODES
+    }
+    restoration_by_mode: dict[str, dict[str, dict[str, Any]]] = {
+        mode: {} for mode in RESTORATION_MODES
+    }
+
+    for path, value in formal_pairs:
+        seed_key = str(value["seed"])
+        final = value["diagnostics"][-1]["local_matrix"]
+        mode_metrics = {
+            mode: primary_metrics(final[mode])
+            for mode in manifest["matrix_modes"]
+        }
+        global_metrics = {}
+        for metric in METRICS:
+            reference = mode_metrics["detached_learned"][metric]
+            uniform_all = mode_metrics["uniform_all"][metric]
+            log_gap = math.log(reference / uniform_all)
+            relative_drop = (reference - uniform_all) / reference
+            established = (
+                reference
+                > thresholds["positive_denominator_epsilon"]
+                and uniform_all
+                > thresholds["positive_denominator_epsilon"]
+                and log_gap > 0
+                and relative_drop
+                >= thresholds["global_relative_drop_minimum"]
+            )
+            global_metrics[metric] = {
+                "reference": reference,
+                "uniform_all": uniform_all,
+                "log_gap": log_gap,
+                "relative_drop": relative_drop,
+                "established": established,
+            }
+        for mode in SUFFICIENCY_MODES:
+            sufficiency_by_mode[mode][seed_key] = {}
+            for metric in METRICS:
+                established = global_metrics[metric]["established"]
+                score = (
+                    math.log(
+                        mode_metrics["detached_learned"][metric]
+                        / mode_metrics[mode][metric]
+                    )
+                    / global_metrics[metric]["log_gap"]
+                    if established
+                    else None
+                )
+                sufficiency_by_mode[mode][seed_key][metric] = {
+                    "score": score,
+                    "metric_value": mode_metrics[mode][metric],
+                    "global_gap_established": established,
+                }
+        for mode in RESTORATION_MODES:
+            restoration_by_mode[mode][seed_key] = {}
+            for metric in METRICS:
+                established = global_metrics[metric]["established"]
+                score = (
+                    math.log(
+                        mode_metrics[mode][metric]
+                        / mode_metrics["uniform_all"][metric]
+                    )
+                    / global_metrics[metric]["log_gap"]
+                    if established
+                    else None
+                )
+                restoration_by_mode[mode][seed_key][metric] = {
+                    "score": score,
+                    "metric_value": mode_metrics[mode][metric],
+                    "global_gap_established": established,
+                }
+        cells.append(
+            {
+                "seed": value["seed"],
+                "path": str(path),
+                "file_sha256": file_sha256(path),
+                "canonical_sha256": value[
+                    "canonical_sha256_without_self"
+                ],
+                "mode_metrics": mode_metrics,
+                "global": global_metrics,
+                "interaction_residual": {
+                    metric: (
+                        1.0
+                        - sufficiency_by_mode["uniform_output_only"][
+                            seed_key
+                        ][metric]["score"]
+                        - sufficiency_by_mode["uniform_depth_all"][
+                            seed_key
+                        ][metric]["score"]
+                    )
+                    for metric in METRICS
+                },
+            }
+        )
+
+    global_gap_passed = all_true(
+        [
+            cell["global"][metric]["established"]
+            for cell in cells
+            for metric in METRICS
+        ]
+    )
+    sufficiency_gates = {
+        "groups_6_7": mixed_status(
+            sufficiency_by_mode["uniform_groups_6_7_only"],
+            thresholds["groups_6_7_sufficiency_minimum"],
+        ),
+        "group_6": mixed_status(
+            sufficiency_by_mode["uniform_group_6_only"],
+            thresholds["single_group_material_minimum"],
+        ),
+        "group_7": mixed_status(
+            sufficiency_by_mode["uniform_group_7_only"],
+            thresholds["single_group_material_minimum"],
+        ),
+        "output_half_gap": mixed_status(
+            sufficiency_by_mode["uniform_output_only"],
+            thresholds["output_half_gap_minimum"],
+        ),
+    }
+    restoration_gates = {
+        "groups_6_7": mixed_status(
+            restoration_by_mode["uniform_except_groups_6_7"],
+            thresholds["groups_6_7_restoration_minimum"],
+        ),
+        "group_6": mixed_status(
+            restoration_by_mode["uniform_except_group_6"],
+            thresholds["single_group_material_minimum"],
+        ),
+        "group_7": mixed_status(
+            restoration_by_mode["uniform_except_group_7"],
+            thresholds["single_group_material_minimum"],
+        ),
+    }
+
+    branch_gates = {}
+    for group in (6, 7):
+        group_passed = sufficiency_gates[f"group_{group}"]["passed"]
+        candidates = {}
+        for branch, sibling in (("attention", "mlp"), ("mlp", "attention")):
+            branch_mode = f"uniform_group_{group}_{branch}_only"
+            sibling_mode = f"uniform_group_{group}_{sibling}_only"
+            checks = []
+            margins = []
+            for seed in expected_seeds:
+                seed_key = str(seed)
+                for metric in METRICS:
+                    left = sufficiency_by_mode[branch_mode][seed_key][
+                        metric
+                    ]["score"]
+                    right = sufficiency_by_mode[sibling_mode][seed_key][
+                        metric
+                    ]["score"]
+                    margin = (
+                        left - right
+                        if left is not None and right is not None
+                        else None
+                    )
+                    margins.append(margin)
+                    checks.append(
+                        left is not None
+                        and left >= thresholds["branch_material_minimum"]
+                        and margin is not None
+                        and margin
+                        >= thresholds["branch_dominance_margin"]
+                    )
+            candidates[branch] = {
+                "passed": group_passed and all_true(checks),
+                "group_gate_passed": group_passed,
+                "passed_cells": sum(checks),
+                "required_cells": len(checks),
+                "margins": margins,
+            }
+        dominant = [
+            branch
+            for branch, gate in candidates.items()
+            if gate["passed"]
+        ]
+        branch_gates[f"group_{group}"] = {
+            "passed": len(dominant) == 1,
+            "dominant_branch": dominant[0] if len(dominant) == 1 else None,
+            "exploratory_sufficiency_only": True,
+            "candidates": candidates,
+        }
+
+    localization_passed = (
+        global_gap_passed
+        and sufficiency_gates["groups_6_7"]["passed"]
+        and restoration_gates["groups_6_7"]["passed"]
+    )
+    localization_status = (
+        "established_at_preregistered_bidirectional_50pct_threshold"
+        if localization_passed
+        else (
+            "one_sided_evidence_localization_not_established"
+            if (
+                sufficiency_gates["groups_6_7"]["passed"]
+                != restoration_gates["groups_6_7"]["passed"]
+            )
+            else "not_established_at_preregistered_threshold"
+        )
+    )
+
+    mode_means = {}
+    for mode in manifest["matrix_modes"]:
+        mode_means[mode] = {
+            metric: statistics.fmean(
+                cell["mode_metrics"][mode][metric] for cell in cells
+            )
+            for metric in METRICS
+        }
+    sufficiency_means = {
+        mode: {
+            metric: statistics.fmean(
+                sufficiency_by_mode[mode][str(seed)][metric]["score"]
+                for seed in expected_seeds
+            )
+            for metric in METRICS
+        }
+        for mode in SUFFICIENCY_MODES
+    }
+    restoration_means = {
+        mode: {
+            metric: statistics.fmean(
+                restoration_by_mode[mode][str(seed)][metric]["score"]
+                for seed in expected_seeds
+            )
+            for metric in METRICS
+        }
+        for mode in RESTORATION_MODES
+    }
+
+    result = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "study_identity": manifest["study_identity"],
+        "manifest": {
+            "path": str(args.manifest),
+            "file_sha256": file_sha256(args.manifest),
+        },
+        "formal_cells": cells,
+        "replay": {
+            "path": str(args.replay),
+            "file_sha256": file_sha256(args.replay),
+            "canonical_sha256": replay[
+                "canonical_sha256_without_self"
+            ],
+            "formal_seed1_exact_excluding_run_kind_and_timing": replay_exact,
+        },
+        "scores": {
+            "sufficiency": sufficiency_by_mode,
+            "restoration": restoration_by_mode,
+        },
+        "means": {
+            "mode_metrics": mode_means,
+            "sufficiency": sufficiency_means,
+            "restoration": restoration_means,
+        },
+        "gates": {
+            "all_input_and_parent_gates_passed": True,
+            "global_gap": {
+                "passed": global_gap_passed,
+                "required_cells": 6,
+                "passed_cells": sum(
+                    cell["global"][metric]["established"]
+                    for cell in cells
+                    for metric in METRICS
+                ),
+            },
+            "sufficiency": sufficiency_gates,
+            "restoration": restoration_gates,
+            "localization": {
+                "passed": localization_passed,
+                "status": localization_status,
+                "requires": (
+                    "groups 6+7 sufficiency and restoration both >=0.50 "
+                    "for 3/3 seeds and both metrics"
+                ),
+            },
+            "branch_dominance": branch_gates,
+        },
+        "limitations": [
+            "same-forward diagnostic backward-rule sensitivity only",
+            "reduced byte-level language model, not the Kimi K3 checkpoint",
+            "effects are non-additive and are not contribution percentages",
+            "group 7 includes layers 26-28 outside fixed spike set 21-25",
+            "three-seed threshold gates are not population inference",
+        ],
+    }
+    result["canonical_sha256_without_self"] = canonical_sha256(result)
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    temporary = args.output.with_suffix(args.output.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True)
+        + "\n"
+    )
+    temporary.replace(args.output)
+    print(
+        json.dumps(
+            {
+                "output": str(args.output),
+                "formal_cells": len(cells),
+                "replay_exact": replay_exact,
+                "global_gap": global_gap_passed,
+                "localization": localization_status,
+                "canonical_sha256": result[
+                    "canonical_sha256_without_self"
+                ],
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_local_path/manifest.json

@@ -0,0 +1,540 @@
+{
+  "schema_version": 1,
+  "protocol_id": "llm-atlas-k3-attnres-local-path-v1",
+  "parent_protocol_id": "llm-atlas-k3-attnres-spike-path-v1",
+  "study_identity": "targeted local-path follow-up informed by Round 05 and Round 06; not blind discovery",
+  "architecture": "block",
+  "depth": 32,
+  "aggregation_groups": 8,
+  "blocks_per_group": 4,
+  "depth_mixers": 64,
+  "output_mixers": 1,
+  "formal_seeds": [
+    2026073001,
+    2026073002,
+    2026073003
+  ],
+  "replay": {
+    "architecture": "block",
+    "depth": 32,
+    "seed": 2026073001,
+    "environment_scope": "same host, GPU, Python, PyTorch, CUDA and CUBLAS_WORKSPACE_CONFIG"
+  },
+  "training": {
+    "steps": 8000,
+    "batch_size": 32,
+    "context": 256,
+    "target_bytes_per_cell": 65536000,
+    "parent_diagnostic_steps": [
+      0,
+      100,
+      500,
+      2000,
+      4000,
+      8000
+    ],
+    "local_matrix_steps": [
+      0,
+      8000
+    ],
+    "matrix_used_during_training": false
+  },
+  "primary_object": {
+    "position": "post_mlp_state",
+    "reduction": "element_rms",
+    "spike_layers_one_based": [
+      21,
+      22,
+      23,
+      24,
+      25
+    ],
+    "metrics": [
+      "spike_contrast",
+      "peak_normalized"
+    ]
+  },
+  "matrix_modes": [
+    "detached_learned",
+    "uniform_group_6_only",
+    "uniform_group_7_only",
+    "uniform_groups_6_7_only",
+    "uniform_group_6_attention_only",
+    "uniform_group_6_mlp_only",
+    "uniform_group_7_attention_only",
+    "uniform_group_7_mlp_only",
+    "uniform_output_only",
+    "uniform_depth_all",
+    "uniform_all",
+    "uniform_except_group_6",
+    "uniform_except_group_7",
+    "uniform_except_groups_6_7"
+  ],
+  "selector": {
+    "coefficient_choices": [
+      "detached_learned",
+      "uniform"
+    ],
+    "full_autograd_learned_in_matrix": false,
+    "depth_identity": "kind=depth,index=0..63; layer=floor(index/2)+1; attention iff index even",
+    "output_identity": "kind=output,index=64; parallel-schema alias only; parent mixer_index remains null",
+    "uniform_depth_indices": {
+      "detached_learned": [],
+      "uniform_group_6_only": [
+        40,
+        41,
+        42,
+        43,
+        44,
+        45,
+        46,
+        47
+      ],
+      "uniform_group_7_only": [
+        48,
+        49,
+        50,
+        51,
+        52,
+        53,
+        54,
+        55
+      ],
+      "uniform_groups_6_7_only": [
+        40,
+        41,
+        42,
+        43,
+        44,
+        45,
+        46,
+        47,
+        48,
+        49,
+        50,
+        51,
+        52,
+        53,
+        54,
+        55
+      ],
+      "uniform_group_6_attention_only": [
+        40,
+        42,
+        44,
+        46
+      ],
+      "uniform_group_6_mlp_only": [
+        41,
+        43,
+        45,
+        47
+      ],
+      "uniform_group_7_attention_only": [
+        48,
+        50,
+        52,
+        54
+      ],
+      "uniform_group_7_mlp_only": [
+        49,
+        51,
+        53,
+        55
+      ],
+      "uniform_output_only": [],
+      "uniform_depth_all": [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15,
+        16,
+        17,
+        18,
+        19,
+        20,
+        21,
+        22,
+        23,
+        24,
+        25,
+        26,
+        27,
+        28,
+        29,
+        30,
+        31,
+        32,
+        33,
+        34,
+        35,
+        36,
+        37,
+        38,
+        39,
+        40,
+        41,
+        42,
+        43,
+        44,
+        45,
+        46,
+        47,
+        48,
+        49,
+        50,
+        51,
+        52,
+        53,
+        54,
+        55,
+        56,
+        57,
+        58,
+        59,
+        60,
+        61,
+        62,
+        63
+      ],
+      "uniform_all": [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15,
+        16,
+        17,
+        18,
+        19,
+        20,
+        21,
+        22,
+        23,
+        24,
+        25,
+        26,
+        27,
+        28,
+        29,
+        30,
+        31,
+        32,
+        33,
+        34,
+        35,
+        36,
+        37,
+        38,
+        39,
+        40,
+        41,
+        42,
+        43,
+        44,
+        45,
+        46,
+        47,
+        48,
+        49,
+        50,
+        51,
+        52,
+        53,
+        54,
+        55,
+        56,
+        57,
+        58,
+        59,
+        60,
+        61,
+        62,
+        63
+      ],
+      "uniform_except_group_6": [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15,
+        16,
+        17,
+        18,
+        19,
+        20,
+        21,
+        22,
+        23,
+        24,
+        25,
+        26,
+        27,
+        28,
+        29,
+        30,
+        31,
+        32,
+        33,
+        34,
+        35,
+        36,
+        37,
+        38,
+        39,
+        48,
+        49,
+        50,
+        51,
+        52,
+        53,
+        54,
+        55,
+        56,
+        57,
+        58,
+        59,
+        60,
+        61,
+        62,
+        63
+      ],
+      "uniform_except_group_7": [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15,
+        16,
+        17,
+        18,
+        19,
+        20,
+        21,
+        22,
+        23,
+        24,
+        25,
+        26,
+        27,
+        28,
+        29,
+        30,
+        31,
+        32,
+        33,
+        34,
+        35,
+        36,
+        37,
+        38,
+        39,
+        40,
+        41,
+        42,
+        43,
+        44,
+        45,
+        46,
+        47,
+        56,
+        57,
+        58,
+        59,
+        60,
+        61,
+        62,
+        63
+      ],
+      "uniform_except_groups_6_7": [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15,
+        16,
+        17,
+        18,
+        19,
+        20,
+        21,
+        22,
+        23,
+        24,
+        25,
+        26,
+        27,
+        28,
+        29,
+        30,
+        31,
+        32,
+        33,
+        34,
+        35,
+        36,
+        37,
+        38,
+        39,
+        56,
+        57,
+        58,
+        59,
+        60,
+        61,
+        62,
+        63
+      ]
+    },
+    "uniform_output": {
+      "detached_learned": false,
+      "uniform_group_6_only": false,
+      "uniform_group_7_only": false,
+      "uniform_groups_6_7_only": false,
+      "uniform_group_6_attention_only": false,
+      "uniform_group_6_mlp_only": false,
+      "uniform_group_7_attention_only": false,
+      "uniform_group_7_mlp_only": false,
+      "uniform_output_only": true,
+      "uniform_depth_all": false,
+      "uniform_all": true,
+      "uniform_except_group_6": true,
+      "uniform_except_group_7": true,
+      "uniform_except_groups_6_7": true
+    },
+    "expected_uniform_counts": {
+      "detached_learned": 0,
+      "uniform_group_6_only": 8,
+      "uniform_group_7_only": 8,
+      "uniform_groups_6_7_only": 16,
+      "uniform_group_6_attention_only": 4,
+      "uniform_group_6_mlp_only": 4,
+      "uniform_group_7_attention_only": 4,
+      "uniform_group_7_mlp_only": 4,
+      "uniform_output_only": 1,
+      "uniform_depth_all": 64,
+      "uniform_all": 65,
+      "uniform_except_group_6": 57,
+      "uniform_except_group_7": 57,
+      "uniform_except_groups_6_7": 49
+    }
+  },
+  "thresholds": {
+    "positive_denominator_epsilon": 1e-30,
+    "step0_spectrum_tolerance": 1e-6,
+    "loss_scale_tolerance": 1e-5,
+    "global_relative_drop_minimum": 0.2,
+    "groups_6_7_sufficiency_minimum": 0.5,
+    "groups_6_7_restoration_minimum": 0.5,
+    "single_group_material_minimum": 0.2,
+    "branch_material_minimum": 0.2,
+    "branch_dominance_margin": 0.15,
+    "output_half_gap_minimum": 0.5,
+    "formal_seed_gate": "3/3 independently for both metrics; means are display-only"
+  },
+  "formulas": {
+    "global_log_gap": "G_X = ln(X_ref / X_uniform_all)",
+    "global_relative_drop": "(X_ref - X_uniform_all) / X_ref",
+    "sufficiency": "S_X(m) = ln(X_ref / X_m) / G_X",
+    "restoration": "R_X(r) = ln(X_r / X_uniform_all) / G_X",
+    "score_clipping": false
+  },
+  "parent_artifacts": {
+    "manifest_path": "experiments/k3/attnres_spike/manifest.json",
+    "manifest_sha256": "d5302a249249a07d362819134763d14e7d32307f22cff416c665ed9606142fef",
+    "runner_path": "experiments/k3/attnres_spike/train.py",
+    "runner_sha256": "77298081d3c491d2e88e4705995174b9879ef377f520eb5fe5ea107e7a1da084",
+    "protocol_path": "research/K3_ATTNRES_SPIKE_PROTOCOL.md",
+    "protocol_sha256": "6cb101b8760d9f1c81caeb2f16880b16152da103867224a06761a75a12984a16",
+    "scoping_path": "research/K3_ATTNRES_SPIKE_SCOPING.md",
+    "scoping_sha256": "590166bd62580bb8238293823cfcc39bc0a465fec4c697025343f3f1138abd27",
+    "formal_schedule_sha256": "5041e09b167f229248d2462324e8c254b8f5938975f135dcd8192b00a54a4f4e",
+    "validation_tensor_sha256": "f459316f13078a163b47c133511bb7181e05170ab89516e196490113893ce338",
+    "diagnostic_tensor_sha256": "21117e31db302b10d67b63f035665dc8f220b879d216ccd12b7d2ba86e7b1716"
+  },
+  "current_artifacts": {
+    "protocol_path": "research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md",
+    "protocol_sha256": "5ecc7ca92314ddb50aecf0cb50e115814c8983aa8bffb30e3634f7b3ce6dca1d",
+    "scoping_path": "research/K3_ATTNRES_LOCAL_PATH_SCOPING.md",
+    "scoping_sha256": "670ca4edf31a4be1f54937d9c7a760dba7a96e1e820c38c6b10405e22b078fc8",
+    "grok_review_path": "research/K3_ATTNRES_LOCAL_PATH_GROK_REVIEW.md",
+    "grok_review_sha256": "2da1b6bf1f455c4121a7a2c5cfe40e102327dabafc7e24dccc23ed0d00ac6d71",
+    "grok_session": "019fb151-9627-76c1-b7d7-53012874f85c"
+  },
+  "round06_expected": {
+    "2026073001": {
+      "raw_file_sha256": "e39e93b7a7fce3c56f5f14f95cfdc04afdce53628affee1202fe62bd1bdb7f71",
+      "canonical_sha256": "76b0ccfb55c38baef50c395788ac4b351cbe0d58de70064b50702acb5c93f515",
+      "final_model_state": "3f0b97ece3a15571ba3d656f589f512ca0bb9e20083c9f58a42ccaee14892f59",
+      "final_optimizer_state": "ed03e6fbd4a12d8b063dcb22e0437754285f54d585374cd52fbd534f05d24637"
+    },
+    "2026073002": {
+      "raw_file_sha256": "1c6f6c731030ec0adb2a8e7a4d586e0c4005cc3319568a7ac83c08c2a4b8eaf8",
+      "canonical_sha256": "5352c74eca853b375c0e85933dafd7c5916c39fc59052e742ca14ffd6d68bc78",
+      "final_model_state": "bd2556388aeaa211b798c283c7cbd8ccd29edf166a2922fa13d172e8dfdc38d1",
+      "final_optimizer_state": "0b101eab3bc7d8d654be2ea335c86fc25563ce19912d721844ee4e639c569e77"
+    },
+    "2026073003": {
+      "raw_file_sha256": "115f8245577ece6dfaaa8ada68445c186e6523a7f3b26efcc3eb4c0c4ce82406",
+      "canonical_sha256": "7e764c07e90b78c4cd0acc2e99600225f16428cbb25d5188a0a5a8fe797f8766",
+      "final_model_state": "638568aede21890773b6932a19ec4e112f5ac0a4770ba3402fcd82980a9ecf76",
+      "final_optimizer_state": "83947fd743ec8e3e31ca7788fd201981846f0f1c7e9e38c88afcef95cfc6ec4e"
+    }
+  }
+}

experiments/k3/attnres_local_path/package.py

@@ -0,0 +1,274 @@
+#!/usr/bin/env python3
+"""Package frozen Round 07 outputs without recomputing any result gate."""
+
+from __future__ import annotations
+
+import argparse
+import copy
+import hashlib
+import json
+from pathlib import Path
+from typing import Any
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-local-path-v1"
+SEEDS = (2026073001, 2026073002, 2026073003)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--raw-dir", type=Path, required=True)
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument("--aggregate", type=Path, required=True)
+    parser.add_argument("--reproduction-output", type=Path, required=True)
+    parser.add_argument("--compact-output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def canonical_sha256(value: Any) -> str:
+    payload = json.dumps(
+        value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+    ).encode()
+    return hashlib.sha256(payload).hexdigest()
+
+
+def load_canonical(path: Path) -> dict[str, Any]:
+    value = json.loads(path.read_text())
+    expected = value["canonical_sha256_without_self"]
+    payload = {
+        key: item
+        for key, item in value.items()
+        if key != "canonical_sha256_without_self"
+    }
+    if canonical_sha256(payload) != expected:
+        raise RuntimeError(f"canonical hash mismatch: {path}")
+    return value
+
+
+def write_canonical(path: Path, value: dict[str, Any]) -> None:
+    value["canonical_sha256_without_self"] = canonical_sha256(value)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(path.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True)
+        + "\n"
+    )
+    temporary.replace(path)
+
+
+def replay_payload(value: dict[str, Any]) -> dict[str, Any]:
+    cleaned = copy.deepcopy(value)
+    for key in ("run_kind", "timing", "canonical_sha256_without_self"):
+        cleaned.pop(key)
+    return cleaned
+
+
+def artifact_hashes(repo_root: Path) -> dict[str, str]:
+    paths = {
+        "runner": "experiments/k3/attnres_local_path/train.py",
+        "analyzer": "experiments/k3/attnres_local_path/analyze.py",
+        "packager": "experiments/k3/attnres_local_path/package.py",
+        "manifest": "experiments/k3/attnres_local_path/manifest.json",
+        "protocol": "research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md",
+        "scoping": "research/K3_ATTNRES_LOCAL_PATH_SCOPING.md",
+        "preresult_grok_review": (
+            "research/K3_ATTNRES_LOCAL_PATH_GROK_REVIEW.md"
+        ),
+    }
+    return {
+        name: file_sha256(repo_root / path) for name, path in paths.items()
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    repo_root = Path(__file__).resolve().parents[3]
+    manifest = json.loads(args.manifest.read_text())
+    aggregate = load_canonical(args.aggregate)
+    if (
+        manifest["protocol_id"] != PROTOCOL_ID
+        or aggregate["protocol_id"] != PROTOCOL_ID
+    ):
+        raise RuntimeError("protocol mismatch")
+
+    formal = {}
+    raw_files = {}
+    for seed in SEEDS:
+        name = f"formal-seed-{seed}.json"
+        path = args.raw_dir / name
+        run = load_canonical(path)
+        if (
+            run["run_kind"] != "formal"
+            or run["seed"] != seed
+            or not run["round06_equivalence"]["passed"]
+        ):
+            raise RuntimeError(f"invalid formal run: {name}")
+        formal[seed] = run
+        raw_files[name] = {
+            "file_sha256": file_sha256(path),
+            "canonical_sha256": run["canonical_sha256_without_self"],
+        }
+
+    replay_name = f"replay-seed-{SEEDS[0]}.json"
+    replay_path = args.raw_dir / replay_name
+    replay = load_canonical(replay_path)
+    if replay["run_kind"] != "replay" or replay["seed"] != SEEDS[0]:
+        raise RuntimeError("invalid replay")
+    raw_files[replay_name] = {
+        "file_sha256": file_sha256(replay_path),
+        "canonical_sha256": replay["canonical_sha256_without_self"],
+    }
+    compare_payload = replay_payload(formal[SEEDS[0]])
+    replay_exact = compare_payload == replay_payload(replay)
+    if not replay_exact or not aggregate["replay"][
+        "formal_seed1_exact_excluding_run_kind_and_timing"
+    ]:
+        raise RuntimeError("replay exactness failed")
+    replay_gate = {
+        "passed": True,
+        "excluded_fields": [
+            "run_kind",
+            "timing",
+            "canonical_sha256_without_self",
+        ],
+        "frozen_compare_sha256": canonical_sha256(compare_payload),
+    }
+
+    reproduction = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "raw_files": raw_files,
+        "replay_gate": replay_gate,
+        "artifacts": artifact_hashes(repo_root),
+        "aggregate": {
+            "file_sha256": file_sha256(args.aggregate),
+            "canonical_sha256": aggregate[
+                "canonical_sha256_without_self"
+            ],
+        },
+        "post_result_grok_review": {
+            "session": "019fb19d-94a3-7231-9a63-3a1ef33a9892",
+            "role": "read-only adversarial implementation audit; not an evidence source",
+            "blocking_errors": 0,
+            "localization_status_confirmed": True,
+        },
+    }
+    write_canonical(args.reproduction_output, reproduction)
+
+    final_spectra = []
+    for seed in SEEDS:
+        final = formal[seed]["diagnostics"][-1]["local_matrix"]
+        final_spectra.append(
+            {
+                "seed": seed,
+                "modes": {
+                    mode: {
+                        "normalized": final[mode]["positions"][
+                            "post_mlp_state"
+                        ]["reductions"]["element_rms"]["statistics"][
+                            "normalized"
+                        ],
+                        "spike_contrast": final[mode]["positions"][
+                            "post_mlp_state"
+                        ]["reductions"]["element_rms"]["statistics"][
+                            "spike_contrast"
+                        ],
+                        "peak_normalized": final[mode]["positions"][
+                            "post_mlp_state"
+                        ]["reductions"]["element_rms"]["statistics"][
+                            "peak_normalized"
+                        ],
+                        "peak_layer": final[mode]["positions"][
+                            "post_mlp_state"
+                        ]["reductions"]["element_rms"]["statistics"][
+                            "peak_layer"
+                        ],
+                        "uniform_count": final[mode]["selector"][
+                            "uniform_count"
+                        ],
+                    }
+                    for mode in manifest["matrix_modes"]
+                },
+            }
+        )
+
+    compact = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "study": {
+            "identity": manifest["study_identity"],
+            "seeds": list(SEEDS),
+            "steps": manifest["training"]["steps"],
+            "formal_target_bytes": (
+                len(SEEDS)
+                * manifest["training"]["target_bytes_per_cell"]
+            ),
+            "total_target_bytes_with_replay": (
+                (len(SEEDS) + 1)
+                * manifest["training"]["target_bytes_per_cell"]
+            ),
+            "modes": manifest["matrix_modes"],
+            "spike_layers": manifest["primary_object"][
+                "spike_layers_one_based"
+            ],
+            "metrics": manifest["primary_object"]["metrics"],
+        },
+        "thresholds": manifest["thresholds"],
+        "formulas": manifest["formulas"],
+        "formal_cells": aggregate["formal_cells"],
+        "scores": aggregate["scores"],
+        "means": aggregate["means"],
+        "gates": aggregate["gates"],
+        "replay": {
+            **aggregate["replay"],
+            "frozen_compare_sha256": replay_gate[
+                "frozen_compare_sha256"
+            ],
+        },
+        "final_spectra": final_spectra,
+        "limitations": aggregate["limitations"],
+        "hashes": {
+            "aggregate_file_sha256": file_sha256(args.aggregate),
+            "aggregate_canonical_sha256": aggregate[
+                "canonical_sha256_without_self"
+            ],
+            "reproduction_file_sha256": file_sha256(
+                args.reproduction_output
+            ),
+            "reproduction_canonical_sha256": reproduction[
+                "canonical_sha256_without_self"
+            ],
+            "manifest_file_sha256": file_sha256(args.manifest),
+        },
+    }
+    write_canonical(args.compact_output, compact)
+    print(
+        json.dumps(
+            {
+                "reproduction": str(args.reproduction_output),
+                "compact": str(args.compact_output),
+                "raw_files": len(raw_files),
+                "replay_exact": replay_exact,
+                "localization": aggregate["gates"]["localization"][
+                    "status"
+                ],
+                "compact_canonical_sha256": compact[
+                    "canonical_sha256_without_self"
+                ],
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_local_path/reproduction.json

@@ -0,0 +1,51 @@
+{
+  "aggregate": {
+    "canonical_sha256": "b86d119cd2f106e2cbee8a35760ed3244336a2fcfeb9178ea1e7dab13fc6f215",
+    "file_sha256": "bb0ec9fce5b30d50ad5c50c4b95af7a892d614f205a2c20cfc2662125e10160e"
+  },
+  "artifacts": {
+    "analyzer": "e0921562463e43d1ba6d47e4d23015107eb2df23579921088550b69afd47d02b",
+    "manifest": "db01e92ef2cf0896212fcd529429bd94a344de0e1195db7f87b9a56dc3449139",
+    "packager": "6a9ada0bc35b40475f45d7aca82877a93aeca667ed117c8e4417d924416c9196",
+    "preresult_grok_review": "2da1b6bf1f455c4121a7a2c5cfe40e102327dabafc7e24dccc23ed0d00ac6d71",
+    "protocol": "5ecc7ca92314ddb50aecf0cb50e115814c8983aa8bffb30e3634f7b3ce6dca1d",
+    "runner": "b42879e242a2f2d54aa6a87a718aeac4cf4509eae42da2b14403656675a8b03d",
+    "scoping": "670ca4edf31a4be1f54937d9c7a760dba7a96e1e820c38c6b10405e22b078fc8"
+  },
+  "canonical_sha256_without_self": "6f5d98fce6446fecc966dd2675f272f2c4f0c9a39a5741fabc4ffad6852ca7f4",
+  "post_result_grok_review": {
+    "blocking_errors": 0,
+    "localization_status_confirmed": true,
+    "role": "read-only adversarial implementation audit; not an evidence source",
+    "session": "019fb19d-94a3-7231-9a63-3a1ef33a9892"
+  },
+  "protocol_id": "llm-atlas-k3-attnres-local-path-v1",
+  "raw_files": {
+    "formal-seed-2026073001.json": {
+      "canonical_sha256": "f0a44f119836ed632c15880c3c2bb225173c0a05c50ea07abbe0e464ff407592",
+      "file_sha256": "73d46ae443d3e5ae3fe839c1656cda758f5f41aaee5f220c971c3c39b8a8cc3f"
+    },
+    "formal-seed-2026073002.json": {
+      "canonical_sha256": "b4629672b7d3b88a6be5525d2839e63e34fc9e4603ba1e7b98e957558da6da05",
+      "file_sha256": "bca4674c746e35a035acde7d2094a9c3bd59a988b052cd3feb30eb66eb0ca60a"
+    },
+    "formal-seed-2026073003.json": {
+      "canonical_sha256": "190b3deb06ae06caba287fce047b55cee613af6f1ebeb1661fcb53dd245abab0",
+      "file_sha256": "712f349e7715fee71f8e4678dcde0619d01b8d6c3b5c34825c88b9af0bf2abbb"
+    },
+    "replay-seed-2026073001.json": {
+      "canonical_sha256": "378df53ed9c89b2a4e0f3045b4c1e72754a7108d87fa9436e1db7aefa442e5eb",
+      "file_sha256": "872aabd9285ac346b4016c23de10769ab83c4dc29e62d8bc8b3156af98228d4e"
+    }
+  },
+  "replay_gate": {
+    "excluded_fields": [
+      "run_kind",
+      "timing",
+      "canonical_sha256_without_self"
+    ],
+    "frozen_compare_sha256": "7dbd15ad03fbd357c5d91e159706d63b24703722f76c492ed1dc733535d6b9cf",
+    "passed": true
+  },
+  "schema_version": 1
+}

experiments/k3/attnres_local_path/train.py

@@ -0,0 +1,933 @@
+#!/usr/bin/env python3
+"""Exact-replay Round 06 with preregistered local mixer-path interventions."""
+
+from __future__ import annotations
+
+import argparse
+import importlib.util
+import json
+import math
+import os
+import platform
+import statistics
+import sys
+import time
+from pathlib import Path
+from typing import Any
+
+import torch
+import torch.nn as nn
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-local-path-v1"
+PARENT_PROTOCOL_ID = "llm-atlas-k3-attnres-spike-path-v1"
+DATA_PROTOCOL_ID = "llm-atlas-k3-attnres-gradient-scale-v1"
+DEPTH = 32
+SEEDS = (2026073001, 2026073002, 2026073003)
+FORMAL_STEPS = 8000
+PARENT_DIAGNOSTIC_STEPS = (0, 100, 500, 2000, 4000, 8000)
+MATRIX_STEPS = (0, 8000)
+MATRIX_MODES = (
+    "detached_learned",
+    "uniform_group_6_only",
+    "uniform_group_7_only",
+    "uniform_groups_6_7_only",
+    "uniform_group_6_attention_only",
+    "uniform_group_6_mlp_only",
+    "uniform_group_7_attention_only",
+    "uniform_group_7_mlp_only",
+    "uniform_output_only",
+    "uniform_depth_all",
+    "uniform_all",
+    "uniform_except_group_6",
+    "uniform_except_group_7",
+    "uniform_except_groups_6_7",
+)
+TRAIN_BATCH_SIZE = 32
+VALIDATION_WINDOWS = 64
+EVAL_BATCH_SIZE = 8
+TIMING_WARMUP = 20
+
+
+def load_parent() -> Any:
+    path = Path(__file__).resolve().parents[1] / "attnres_spike" / "train.py"
+    spec = importlib.util.spec_from_file_location(
+        "k3_attnres_spike_parent", path
+    )
+    if spec is None or spec.loader is None:
+        raise RuntimeError(f"cannot import parent runner from {path}")
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[spec.name] = module
+    spec.loader.exec_module(module)
+    return module
+
+
+parent = load_parent()
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--run-kind", choices=("smoke", "formal", "replay"), required=True
+    )
+    parser.add_argument("--seed", type=int, required=True)
+    parser.add_argument("--cache-dir", type=Path, required=True)
+    parser.add_argument("--data-manifest", type=Path, required=True)
+    parser.add_argument("--parent-manifest", type=Path, required=True)
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    args = parser.parse_args()
+    if args.seed not in SEEDS:
+        parser.error(f"seed must be one of {SEEDS}")
+    if args.run_kind == "replay" and args.seed != SEEDS[0]:
+        parser.error(f"replay seed must be {SEEDS[0]}")
+    return args
+
+
+class LocalPathLanguageModel(parent.SpikeLanguageModel):
+    """Parent model with a parallel, audited per-mixer coefficient selector."""
+
+    def __init__(self, architecture: str, manifest: dict[str, Any]):
+        super().__init__(architecture)
+        selector = manifest["selector"]
+        self.uniform_depth = {
+            mode: frozenset(indices)
+            for mode, indices in selector["uniform_depth_indices"].items()
+        }
+        self.uniform_output = selector["uniform_output"]
+        self.expected_uniform_counts = selector["expected_uniform_counts"]
+        self._active_selector_visits: list[dict[str, Any]] | None = None
+        self.last_selector_visits: list[dict[str, Any]] | None = None
+
+    def forward(
+        self,
+        input_ids: torch.Tensor,
+        capture: bool = False,
+        mixer_backward_mode: str = "learned",
+    ) -> tuple[torch.Tensor, parent.SpikeTrace | None]:
+        if not capture:
+            if mixer_backward_mode != "learned":
+                raise RuntimeError("training/evaluation cannot use an intervention")
+            return super().forward(
+                input_ids, capture=False, mixer_backward_mode="learned"
+            )
+        if mixer_backward_mode == "learned":
+            self.last_selector_visits = None
+            return super().forward(
+                input_ids, capture=True, mixer_backward_mode="learned"
+            )
+        if mixer_backward_mode not in MATRIX_MODES:
+            raise ValueError(f"unknown local matrix mode: {mixer_backward_mode}")
+        self._active_selector_visits = []
+        logits, trace = self._diagnostic_forward(
+            input_ids, mixer_backward_mode
+        )
+        self.last_selector_visits = self._active_selector_visits
+        self._active_selector_visits = None
+        return logits, trace
+
+    def _mix(
+        self,
+        mixer: nn.Module,
+        sources: list[torch.Tensor],
+        labels: list[str],
+        *,
+        mode: str,
+        mixer_index: int | None,
+        layer: int | None,
+        branch: str,
+        group: int | None,
+        offset: int | None,
+    ) -> tuple[torch.Tensor, dict[str, Any]]:
+        if mode == "learned":
+            return super()._mix(
+                mixer,
+                sources,
+                labels,
+                mode=mode,
+                mixer_index=mixer_index,
+                layer=layer,
+                branch=branch,
+                group=group,
+                offset=offset,
+            )
+        if self._active_selector_visits is None:
+            raise RuntimeError("local selector visit log is not active")
+
+        if mixer_index is None:
+            if (
+                layer is not None
+                or group is not None
+                or branch != "output"
+                or offset is not None
+            ):
+                raise RuntimeError("invalid output mixer identity")
+            identity = {
+                "kind": "output",
+                "index": 64,
+                "layer": None,
+                "group": None,
+                "branch": "output",
+                "offset": None,
+            }
+            use_uniform = bool(self.uniform_output[mode])
+        else:
+            expected_layer = mixer_index // 2 + 1
+            expected_branch = "attention" if mixer_index % 2 == 0 else "mlp"
+            expected_group = (expected_layer - 1) // 4 + 1
+            expected_offset = (expected_layer - 1) % 4 + 1
+            if (
+                not 0 <= mixer_index < 64
+                or layer != expected_layer
+                or branch != expected_branch
+                or group != expected_group
+                or offset != expected_offset
+            ):
+                raise RuntimeError("invalid depth mixer identity")
+            identity = {
+                "kind": "depth",
+                "index": mixer_index,
+                "layer": layer,
+                "group": group,
+                "branch": branch,
+                "offset": offset,
+            }
+            use_uniform = mixer_index in self.uniform_depth[mode]
+
+        parent_output, _ = mixer(sources, False)
+        weights = parent.recompute_weights(mixer, sources)
+        summary = parent.weight_summary(
+            weights,
+            labels,
+            mixer_index=mixer_index,
+            layer=layer,
+            branch=branch,
+            group=group,
+            offset=offset,
+        )
+        backward_weights = (
+            torch.full_like(weights, 1.0 / len(sources))
+            if use_uniform
+            else weights
+        )
+        values = torch.stack(sources, dim=0)
+        routed = parent.RoutedSourceBackward.apply(
+            values, parent_output, backward_weights
+        )
+        self._active_selector_visits.append(
+            {
+                **identity,
+                "sources": len(sources),
+                "use_uniform": use_uniform,
+                "coefficient": (
+                    "uniform" if use_uniform else "detached_learned"
+                ),
+            }
+        )
+        return routed, summary
+
+
+def expected_visit_identities() -> list[dict[str, Any]]:
+    result = []
+    for index in range(64):
+        layer = index // 2 + 1
+        result.append(
+            {
+                "kind": "depth",
+                "index": index,
+                "layer": layer,
+                "group": (layer - 1) // 4 + 1,
+                "branch": "attention" if index % 2 == 0 else "mlp",
+                "offset": (layer - 1) % 4 + 1,
+            }
+        )
+    result.append(
+        {
+            "kind": "output",
+            "index": 64,
+            "layer": None,
+            "group": None,
+            "branch": "output",
+            "offset": None,
+        }
+    )
+    return result
+
+
+def validate_selector_visits(
+    mode: str,
+    visits: list[dict[str, Any]] | None,
+    manifest: dict[str, Any],
+) -> dict[str, Any]:
+    if visits is None or len(visits) != 65:
+        raise RuntimeError("selector visit count mismatch")
+    identity_keys = ("kind", "index", "layer", "group", "branch", "offset")
+    actual_identities = [
+        {key: visit[key] for key in identity_keys} for visit in visits
+    ]
+    expected_identities = expected_visit_identities()
+    if actual_identities != expected_identities:
+        raise RuntimeError("selector identity order mismatch")
+    if len({(item["kind"], item["index"]) for item in visits}) != 65:
+        raise RuntimeError("selector identities are not unique")
+
+    expected_depth = set(
+        manifest["selector"]["uniform_depth_indices"][mode]
+    )
+    expected_output = manifest["selector"]["uniform_output"][mode]
+    actual_depth = {
+        item["index"]
+        for item in visits
+        if item["kind"] == "depth" and item["use_uniform"]
+    }
+    actual_output = visits[-1]["use_uniform"]
+    if actual_depth != expected_depth or actual_output != expected_output:
+        raise RuntimeError("selector exact-set mismatch")
+    uniform_count = sum(int(item["use_uniform"]) for item in visits)
+    expected_count = manifest["selector"]["expected_uniform_counts"][mode]
+    if uniform_count != expected_count:
+        raise RuntimeError("selector uniform census mismatch")
+    return {
+        "passed": True,
+        "visit_count": len(visits),
+        "identities_unique": True,
+        "identity_order_sha256": parent.canonical_sha256(
+            actual_identities
+        ),
+        "uniform_indices": [
+            item["index"] for item in visits if item["use_uniform"]
+        ],
+        "uniform_count": uniform_count,
+        "expected_uniform_count": expected_count,
+        "visits_sha256": parent.canonical_sha256(visits),
+        "visits": visits,
+    }
+
+
+def run_local_diagnostic(
+    model: LocalPathLanguageModel,
+    corpus: Any,
+    optimizer: torch.optim.Optimizer,
+    manifest: dict[str, Any],
+    *,
+    mode: str,
+    loss_scale: float = 1.0,
+) -> dict[str, Any]:
+    result = parent.run_diagnostic(
+        model, corpus, optimizer, mode=mode, loss_scale=loss_scale
+    )
+    result["selector"] = validate_selector_visits(
+        mode, model.last_selector_visits, manifest
+    )
+    return result
+
+
+def forward_identity_gate(matrix: dict[str, Any]) -> dict[str, Any]:
+    reference = matrix["detached_learned"]["forward"]
+    comparisons = {}
+    for mode in MATRIX_MODES[1:]:
+        other = matrix[mode]["forward"]
+        comparisons[mode] = {
+            "logits_exact": (
+                other["logits_sha256"] == reference["logits_sha256"]
+            ),
+            "loss_exact": other["loss_nats"] == reference["loss_nats"],
+            "activations_exact": (
+                other["activation_sha256"]
+                == reference["activation_sha256"]
+            ),
+            "mixer_summaries_exact": (
+                other["mixer_summary_sha256"]
+                == reference["mixer_summary_sha256"]
+            ),
+        }
+    if not all(all(checks.values()) for checks in comparisons.values()):
+        raise RuntimeError("local matrix forward identity failed")
+    return {"passed": True, "comparisons": comparisons}
+
+
+def spectrum_agreement(
+    left: dict[str, Any], right: dict[str, Any]
+) -> dict[str, Any]:
+    checks = {}
+    passed = True
+    for position in parent.POSITIONS:
+        left_metric = left["positions"][position]["reductions"][
+            "element_rms"
+        ]
+        right_metric = right["positions"][position]["reductions"][
+            "element_rms"
+        ]
+        raw_errors = [
+            abs(a - b) / a
+            for a, b in zip(
+                left_metric["values"], right_metric["values"]
+            )
+        ]
+        normalized_errors = [
+            abs(a - b)
+            for a, b in zip(
+                left_metric["statistics"]["normalized"],
+                right_metric["statistics"]["normalized"],
+            )
+        ]
+        item_passed = (
+            all(
+                math.isfinite(value) and value > 0
+                for value in left_metric["values"]
+            )
+            and max(raw_errors) <= parent.SPECTRUM_TOLERANCE
+            and max(normalized_errors) <= parent.SPECTRUM_TOLERANCE
+        )
+        passed = passed and item_passed
+        checks[position] = {
+            "passed": item_passed,
+            "max_raw_relative_error": max(raw_errors),
+            "max_normalized_absolute_error": max(normalized_errors),
+        }
+    return {"passed": passed, "checks": checks}
+
+
+def initialization_negative_control(
+    parent_learned: dict[str, Any], matrix: dict[str, Any]
+) -> dict[str, Any]:
+    reference = matrix["detached_learned"]
+    comparisons = {
+        "parent_learned_vs_detached": spectrum_agreement(
+            parent_learned, reference
+        )
+    }
+    for mode in MATRIX_MODES[1:]:
+        comparisons[mode] = spectrum_agreement(reference, matrix[mode])
+    passed = all(item["passed"] for item in comparisons.values())
+    if not passed:
+        raise RuntimeError("initialization negative control failed")
+    return {"passed": True, "comparisons": comparisons}
+
+
+def without_selector(result: dict[str, Any], rename: str | None = None) -> dict[str, Any]:
+    cleaned = {key: value for key, value in result.items() if key != "selector"}
+    if rename is not None:
+        cleaned["mode"] = rename
+    return cleaned
+
+
+def endpoint_exactness(
+    matrix: dict[str, Any], parent_diagnostic: dict[str, Any]
+) -> dict[str, Any]:
+    reference_exact = (
+        without_selector(matrix["detached_learned"])
+        == parent_diagnostic["modes"]["detached_learned"]
+    )
+    uniform_exact = (
+        without_selector(
+            matrix["uniform_all"], rename="uniform_value_backward"
+        )
+        == parent_diagnostic["modes"]["uniform_value_backward"]
+    )
+    checks = {
+        "detached_learned_round06_exact": reference_exact,
+        "uniform_all_round06_exact": uniform_exact,
+    }
+    if not all(checks.values()):
+        raise RuntimeError(f"Round 06 endpoint exactness failed: {checks}")
+    return {"passed": True, "checks": checks}
+
+
+def run_diagnostic_bundle(
+    model: LocalPathLanguageModel,
+    corpus: Any,
+    optimizer: torch.optim.Optimizer,
+    manifest: dict[str, Any],
+    parent_diagnostic: dict[str, Any],
+    step: int,
+) -> dict[str, Any]:
+    parent_learned = parent.run_diagnostic(
+        model, corpus, optimizer, mode="learned"
+    )
+    if parent_learned != parent_diagnostic["modes"]["learned"]:
+        raise RuntimeError("parent learned diagnostic is not Round 06 exact")
+    result: dict[str, Any] = {
+        "step": step,
+        "parent_learned": parent_learned,
+        "parent_learned_round06_exact": True,
+        "local_matrix": None,
+    }
+    if step not in MATRIX_STEPS:
+        return result
+
+    matrix = {
+        mode: run_local_diagnostic(
+            model, corpus, optimizer, manifest, mode=mode
+        )
+        for mode in MATRIX_MODES
+    }
+    result["local_matrix"] = matrix
+    result["forward_identity_gate"] = forward_identity_gate(matrix)
+    result["endpoint_exactness"] = endpoint_exactness(
+        matrix, parent_diagnostic
+    )
+    if step == 0:
+        result["initialization_negative_control"] = (
+            initialization_negative_control(parent_learned, matrix)
+        )
+        doubled = run_local_diagnostic(
+            model,
+            corpus,
+            optimizer,
+            manifest,
+            mode="detached_learned",
+            loss_scale=2.0,
+        )
+        result["loss_scale_gate"] = parent.loss_scale_gate(
+            matrix["detached_learned"], doubled
+        )
+    model.zero_grad(set_to_none=True)
+    return result
+
+
+def load_and_verify_inputs(
+    args: argparse.Namespace,
+) -> tuple[dict[str, Any], dict[str, Any], dict[str, Any], Path]:
+    manifest = json.loads(args.manifest.read_text())
+    parent_manifest = json.loads(args.parent_manifest.read_text())
+    data_manifest = json.loads(args.data_manifest.read_text())
+    repo_root = Path(__file__).resolve().parents[3]
+
+    if manifest["protocol_id"] != PROTOCOL_ID:
+        raise RuntimeError("Round 07 manifest protocol mismatch")
+    if parent_manifest["protocol_id"] != PARENT_PROTOCOL_ID:
+        raise RuntimeError("Round 06 parent manifest protocol mismatch")
+    if data_manifest["protocol_id"] != DATA_PROTOCOL_ID:
+        raise RuntimeError("data manifest protocol mismatch")
+    if manifest["formal_seeds"] != list(SEEDS):
+        raise RuntimeError("formal seed mismatch")
+    if manifest["matrix_modes"] != list(MATRIX_MODES):
+        raise RuntimeError("local matrix mode mismatch")
+    if (
+        manifest["training"]["parent_diagnostic_steps"]
+        != list(PARENT_DIAGNOSTIC_STEPS)
+        or manifest["training"]["local_matrix_steps"] != list(MATRIX_STEPS)
+        or manifest["training"]["steps"] != FORMAL_STEPS
+    ):
+        raise RuntimeError("diagnostic/training schedule mismatch")
+
+    parent_artifacts = manifest["parent_artifacts"]
+    if parent.file_sha256(args.parent_manifest) != parent_artifacts[
+        "manifest_sha256"
+    ]:
+        raise RuntimeError("Round 06 manifest physical hash mismatch")
+    if parent.file_sha256(Path(parent.__file__)) != parent_artifacts[
+        "runner_sha256"
+    ]:
+        raise RuntimeError("Round 06 runner physical hash mismatch")
+    for name in ("protocol", "scoping"):
+        path = repo_root / parent_artifacts[f"{name}_path"]
+        if parent.file_sha256(path) != parent_artifacts[f"{name}_sha256"]:
+            raise RuntimeError(f"Round 06 {name} physical hash mismatch")
+    for name in ("protocol", "scoping", "grok_review"):
+        path = repo_root / manifest["current_artifacts"][f"{name}_path"]
+        if parent.file_sha256(path) != manifest["current_artifacts"][
+            f"{name}_sha256"
+        ]:
+            raise RuntimeError(f"Round 07 {name} physical hash mismatch")
+    for key in (
+        "formal_schedule_sha256",
+        "validation_tensor_sha256",
+        "diagnostic_tensor_sha256",
+    ):
+        if (
+            data_manifest["windows"][key]
+            != parent_artifacts[key]
+            or parent_manifest["parent_artifacts"][key]
+            != parent_artifacts[key]
+        ):
+            raise RuntimeError(f"frozen data hash mismatch: {key}")
+
+    parent_raw_path = (
+        repo_root
+        / "experiments"
+        / "k3"
+        / "attnres_spike"
+        / "results"
+        / "raw"
+        / f"formal-seed-{args.seed}.json"
+    )
+    expected = manifest["round06_expected"][str(args.seed)]
+    if parent.file_sha256(parent_raw_path) != expected["raw_file_sha256"]:
+        raise RuntimeError("Round 06 raw physical hash mismatch")
+    parent_raw = json.loads(parent_raw_path.read_text())
+    if (
+        parent_raw["canonical_sha256_without_self"]
+        != expected["canonical_sha256"]
+        or parent_raw["hashes"]["final_model_state"]
+        != expected["final_model_state"]
+        or parent_raw["hashes"]["final_optimizer_state"]
+        != expected["final_optimizer_state"]
+    ):
+        raise RuntimeError("Round 06 raw expected-state mismatch")
+    return manifest, data_manifest, parent_raw, repo_root
+
+
+def frozen_training_compare(
+    result: dict[str, Any], parent_raw: dict[str, Any]
+) -> dict[str, Any]:
+    checks = {
+        "final_model_state": (
+            result["hashes"]["final_model_state"]
+            == parent_raw["hashes"]["final_model_state"]
+        ),
+        "final_optimizer_state": (
+            result["hashes"]["final_optimizer_state"]
+            == parent_raw["hashes"]["final_optimizer_state"]
+        ),
+        "evaluations": result["evaluations"] == parent_raw["evaluations"],
+        "training_history": (
+            result["training_history"] == parent_raw["training_history"]
+        ),
+    }
+    parent_diagnostics_exact = []
+    endpoint_exact = []
+    for new, old in zip(result["diagnostics"], parent_raw["diagnostics"]):
+        parent_diagnostics_exact.append(
+            new["step"] == old["step"]
+            and new["parent_learned"] == old["modes"]["learned"]
+        )
+        if new["step"] in MATRIX_STEPS:
+            endpoint_exact.append(new["endpoint_exactness"]["passed"])
+    checks["parent_learned_diagnostics"] = all(parent_diagnostics_exact)
+    checks["round06_endpoints"] = len(endpoint_exact) == 2 and all(
+        endpoint_exact
+    )
+    if not all(checks.values()):
+        raise RuntimeError(f"Round 06 training equivalence failed: {checks}")
+    return {"passed": True, "checks": checks}
+
+
+def main() -> None:
+    args = parse_args()
+    if not torch.cuda.is_available():
+        raise RuntimeError("CUDA is required")
+    if os.environ.get("CUBLAS_WORKSPACE_CONFIG") != ":4096:8":
+        raise RuntimeError("CUBLAS_WORKSPACE_CONFIG must be :4096:8")
+    manifest, data_manifest, parent_raw, repo_root = load_and_verify_inputs(args)
+    parent.parent.configure_round04_globals(DEPTH)
+    parent.parent.configure_determinism(args.seed)
+    corpus = parent.parent.round04.ByteCorpus(
+        args.cache_dir, data_manifest, torch.device("cuda")
+    )
+    model = LocalPathLanguageModel("block", manifest).to(
+        torch.device("cuda")
+    )
+
+    initial_public_hash = parent.parent.named_state_hash(
+        model, include_mixers=False
+    )
+    initial_mixer_hash = parent.parent.named_state_hash(
+        model, include_mixers=True
+    )
+    public_structure_hash, public_tensors, public_elements = (
+        parent.parent.state_structure_hash(model, include_mixers=False)
+    )
+    input_gate_hashes = parent.parent.model_input_gate_hashes(
+        corpus, data_manifest, args.seed, TRAIN_BATCH_SIZE
+    )
+
+    decay_parameters: list[nn.Parameter] = []
+    no_decay_parameters: list[nn.Parameter] = []
+    for parameter in model.parameters():
+        target = decay_parameters if parameter.ndim >= 2 else no_decay_parameters
+        target.append(parameter)
+    optimizer = torch.optim.AdamW(
+        [
+            {
+                "params": decay_parameters,
+                "weight_decay": parent.parent.WEIGHT_DECAY,
+            },
+            {"params": no_decay_parameters, "weight_decay": 0.0},
+        ],
+        lr=parent.parent.PEAK_LR,
+        betas=parent.parent.BETAS,
+        eps=parent.parent.ADAM_EPS,
+    )
+
+    parent_by_step = {
+        item["step"]: item for item in parent_raw["diagnostics"]
+    }
+    evaluations = [
+        {
+            "step": 0,
+            **parent.parent.evaluate(
+                model, corpus, VALIDATION_WINDOWS, EVAL_BATCH_SIZE
+            ),
+        }
+    ]
+    diagnostics = [
+        run_diagnostic_bundle(
+            model,
+            corpus,
+            optimizer,
+            manifest,
+            parent_by_step[0],
+            0,
+        )
+    ]
+    print(
+        json.dumps(
+            {
+                "event": "local_matrix",
+                "step": 0,
+                "seed": args.seed,
+                "modes": len(MATRIX_MODES),
+                "endpoint_exact": diagnostics[0][
+                    "endpoint_exactness"
+                ]["passed"],
+            },
+            sort_keys=True,
+        ),
+        flush=True,
+    )
+
+    steps = 0 if args.run_kind == "smoke" else FORMAL_STEPS
+    training_history: list[dict[str, float | int]] = []
+    step_times: list[float] = []
+    if steps:
+        model.train()
+        for step in range(1, steps + 1):
+            lr = parent.parent.learning_rate(step, steps)
+            for group in optimizer.param_groups:
+                group["lr"] = lr
+            inputs, targets = corpus.training_batch(
+                args.seed, step, TRAIN_BATCH_SIZE
+            )
+            optimizer.zero_grad(set_to_none=True)
+            torch.cuda.synchronize()
+            started = time.perf_counter()
+            with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+                logits, trace = model(inputs)
+                if trace is not None:
+                    raise RuntimeError(
+                        "training unexpectedly captured a trace"
+                    )
+                loss = parent.parent.cross_entropy(logits, targets)
+            if not torch.isfinite(loss):
+                raise RuntimeError(f"non-finite loss at step {step}")
+            loss.backward()
+            unclipped_norm = torch.nn.utils.clip_grad_norm_(
+                model.parameters(), parent.parent.GRAD_CLIP
+            )
+            optimizer.step()
+            torch.cuda.synchronize()
+            elapsed_ms = (time.perf_counter() - started) * 1000
+            if step == TIMING_WARMUP:
+                torch.cuda.reset_peak_memory_stats()
+            elif step > TIMING_WARMUP:
+                step_times.append(elapsed_ms)
+            if step == 1 or step % 10 == 0 or step == steps:
+                training_history.append(
+                    {
+                        "step": step,
+                        "loss_nats": loss.detach().cpu().item(),
+                        "bits_per_byte": (
+                            loss.detach().cpu().item() / math.log(2)
+                        ),
+                        "learning_rate": lr,
+                        "unclipped_grad_norm": float(
+                            unclipped_norm.detach().cpu()
+                        ),
+                    }
+                )
+            if step in PARENT_DIAGNOSTIC_STEPS:
+                evaluations.append(
+                    {
+                        "step": step,
+                        **parent.parent.evaluate(
+                            model,
+                            corpus,
+                            VALIDATION_WINDOWS,
+                            EVAL_BATCH_SIZE,
+                        ),
+                    }
+                )
+                diagnostic = run_diagnostic_bundle(
+                    model,
+                    corpus,
+                    optimizer,
+                    manifest,
+                    parent_by_step[step],
+                    step,
+                )
+                diagnostics.append(diagnostic)
+                event = {
+                    "event": "diagnostic",
+                    "step": step,
+                    "seed": args.seed,
+                    "validation_bpc": evaluations[-1]["bits_per_byte"],
+                    "parent_exact": diagnostic[
+                        "parent_learned_round06_exact"
+                    ],
+                }
+                if diagnostic["local_matrix"] is not None:
+                    event["modes"] = len(MATRIX_MODES)
+                    event["endpoint_exact"] = diagnostic[
+                        "endpoint_exactness"
+                    ]["passed"]
+                print(json.dumps(event, sort_keys=True), flush=True)
+                model.train()
+
+    timing = {
+        "warmup_steps_excluded": TIMING_WARMUP,
+        "measured_steps": len(step_times),
+        "mean_ms": (
+            statistics.fmean(step_times) if step_times else None
+        ),
+        "median_ms": (
+            statistics.median(step_times) if step_times else None
+        ),
+        "p95_ms": (
+            parent.type7_quantile(torch.tensor(sorted(step_times)), 0.95)
+            if step_times
+            else None
+        ),
+        "peak_allocated_bytes": torch.cuda.max_memory_allocated(),
+        "peak_reserved_bytes": torch.cuda.max_memory_reserved(),
+    }
+    result = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "parent_protocol_id": PARENT_PROTOCOL_ID,
+        "run_kind": args.run_kind,
+        "architecture": "block",
+        "depth": DEPTH,
+        "seed": args.seed,
+        "steps": steps,
+        "batch_size": TRAIN_BATCH_SIZE,
+        "target_bytes_seen": (
+            steps * TRAIN_BATCH_SIZE * parent.CONTEXT
+        ),
+        "manifest": {
+            "path": str(args.manifest),
+            "file_sha256": parent.file_sha256(args.manifest),
+            "parent_path": str(args.parent_manifest),
+            "parent_file_sha256": parent.file_sha256(
+                args.parent_manifest
+            ),
+            "data_path": str(args.data_manifest),
+            "data_file_sha256": parent.file_sha256(args.data_manifest),
+            "formal_schedule_sha256": data_manifest["windows"][
+                "formal_schedule_sha256"
+            ],
+            "validation_tensor_sha256": data_manifest["windows"][
+                "validation_tensor_sha256"
+            ],
+            "diagnostic_tensor_sha256": data_manifest["windows"][
+                "diagnostic_tensor_sha256"
+            ],
+            "input_gate_tensor_hashes": input_gate_hashes,
+            "selector_contract_sha256": parent.canonical_sha256(
+                manifest["selector"]
+            ),
+        },
+        "model": {
+            "layers": DEPTH,
+            "aggregation_groups": 8,
+            "blocks_per_group": 4,
+            "d_model": parent.parent.round04.D_MODEL,
+            "heads": parent.parent.round04.HEADS,
+            "d_ff": parent.parent.round04.D_FF,
+            "parameters": parent.parent.parameter_inventory(model),
+        },
+        "hashes": {
+            "initial_public_parameter_structure": public_structure_hash,
+            "initial_public_parameter_tensors": public_tensors,
+            "initial_public_parameter_elements": public_elements,
+            "initial_public_parameters": initial_public_hash,
+            "initial_mixer_parameters": initial_mixer_hash,
+            "final_public_parameters": parent.parent.named_state_hash(
+                model, include_mixers=False
+            ),
+            "final_mixer_parameters": parent.parent.named_state_hash(
+                model, include_mixers=True
+            ),
+            "final_model_state": parent.parent.named_state_hash(
+                model, include_mixers=None
+            ),
+            "final_optimizer_state": parent.parent.recursive_state_hash(
+                optimizer.state_dict()
+            ),
+        },
+        "evaluations": evaluations,
+        "diagnostics": diagnostics,
+        "training_history": training_history,
+        "timing": timing,
+        "environment": {
+            "python": platform.python_version(),
+            "torch": torch.__version__,
+            "cuda": torch.version.cuda,
+            "gpu": torch.cuda.get_device_name(0),
+            "compute_capability": list(
+                torch.cuda.get_device_capability(0)
+            ),
+            "cublas_workspace_config": os.environ[
+                "CUBLAS_WORKSPACE_CONFIG"
+            ],
+            "deterministic_algorithms": (
+                torch.are_deterministic_algorithms_enabled()
+            ),
+            "autocast": "cuda-bfloat16-forward-fp32-cross-entropy",
+            "compile": False,
+        },
+        "artifacts": {
+            "runner_sha256": parent.file_sha256(Path(__file__)),
+            "protocol_sha256": parent.file_sha256(
+                repo_root
+                / "research"
+                / "K3_ATTNRES_LOCAL_PATH_PROTOCOL.md"
+            ),
+            "scoping_sha256": parent.file_sha256(
+                repo_root
+                / "research"
+                / "K3_ATTNRES_LOCAL_PATH_SCOPING.md"
+            ),
+            "grok_review_sha256": parent.file_sha256(
+                repo_root
+                / "research"
+                / "K3_ATTNRES_LOCAL_PATH_GROK_REVIEW.md"
+            ),
+        },
+    }
+    result["round06_equivalence"] = (
+        frozen_training_compare(result, parent_raw) if steps else None
+    )
+    result["canonical_sha256_without_self"] = parent.canonical_sha256(
+        result
+    )
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    temporary = args.output.with_suffix(args.output.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(result, ensure_ascii=False, indent=2, sort_keys=True)
+        + "\n"
+    )
+    os.replace(temporary, args.output)
+    print(
+        json.dumps(
+            {
+                "output": str(args.output),
+                "run_kind": args.run_kind,
+                "seed": args.seed,
+                "steps": steps,
+                "final_bpc": evaluations[-1]["bits_per_byte"],
+                "canonical_sha256": result[
+                    "canonical_sha256_without_self"
+                ],
+                "timing": timing,
+            },
+            ensure_ascii=False,
+            indent=2,
+        ),
+        flush=True,
+    )
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_spike/README.md

@@ -0,0 +1,40 @@
+# Attention Residuals spike-path diagnostics
+
+This directory implements preregistered protocol
+`llm-atlas-k3-attnres-spike-path-v1`.
+
+It is a targeted follow-up to Round 05. It replays the exact depth-32 Block
+training contract and adds diagnostic-only activation positions, gradient
+reductions, and same-forward backward-rule interventions. It is not a Kimi K3
+checkpoint run and does not recover the paper's unpublished Figure 5 telemetry.
+
+## Frozen environment
+
+```text
+Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
+PyTorch 2.11.0+cu128
+GPU NVIDIA GeForce RTX 5090
+CUBLAS_WORKSPACE_CONFIG=:4096:8
+```
+
+## Step-0 smoke
+
+```bash
+CUBLAS_WORKSPACE_CONFIG=:4096:8 \
+/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
+  experiments/k3/attnres_spike/train.py \
+  --run-kind smoke \
+  --seed 2026073001 \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
+  --parent-manifest experiments/k3/attnres_gradient/manifest.json \
+  --manifest experiments/k3/attnres_spike/manifest.json \
+  --output /home/wuyang/.cache/llm-atlas/k3-attnres-spike-path-v1/smoke/seed-2026073001.json
+```
+
+Formal cells use `--run-kind formal` and all three preregistered seeds. The
+independent replay uses `--run-kind replay --seed 2026073001`. Formal and replay
+runs are fixed to 8,000 steps; smoke performs the complete step-0 diagnostic
+gate without an optimizer step.
+
+Raw outputs are copied into `results/raw/` only after training equivalence,
+forward identity, loss-scale, reduction, and replay gates pass.

experiments/k3/attnres_spike/analyze.py

@@ -0,0 +1,751 @@
+#!/usr/bin/env python3
+"""Aggregate and gate the preregistered AttnRes spike-path study."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import math
+import statistics
+from pathlib import Path
+from typing import Any, Iterable
+
+
+PROTOCOL_ID = "llm-atlas-k3-attnres-spike-path-v1"
+SEEDS = (2026073001, 2026073002, 2026073003)
+POSITIONS = (
+    "pre_attention_input",
+    "attention_branch_output",
+    "post_attention_state",
+    "pre_mlp_input",
+    "mlp_branch_output",
+    "post_mlp_state",
+)
+MAIN_REDUCTIONS = (
+    "element_rms",
+    "token_rms_mean",
+    "token_rms_median",
+    "token_rms_p95",
+)
+ALL_REDUCTIONS = MAIN_REDUCTIONS + (
+    "batch_mean_rms",
+    "token_mean_rms",
+    "global_l2",
+)
+MODES = (
+    "learned",
+    "detached_learned",
+    "uniform_value_backward",
+)
+SPIKE_LAYERS = (21, 22, 23, 24, 25)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--raw-dir", type=Path, required=True)
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument("--aggregate-output", type=Path, required=True)
+    parser.add_argument("--compact-output", type=Path, required=True)
+    parser.add_argument("--reproduction-output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def file_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def canonical_sha256(value: Any) -> str:
+    return hashlib.sha256(
+        json.dumps(
+            value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+        ).encode()
+    ).hexdigest()
+
+
+def mean(values: Iterable[float]) -> float:
+    return statistics.fmean(values)
+
+
+def average_ranks(values: list[float]) -> list[float]:
+    ordered = sorted(range(len(values)), key=lambda index: (values[index], index))
+    ranks = [0.0] * len(values)
+    cursor = 0
+    while cursor < len(ordered):
+        end = cursor + 1
+        while end < len(ordered) and values[ordered[end]] == values[ordered[cursor]]:
+            end += 1
+        average_rank = (cursor + 1 + end) / 2
+        for offset in range(cursor, end):
+            ranks[ordered[offset]] = average_rank
+        cursor = end
+    return ranks
+
+
+def pearson(left: list[float], right: list[float]) -> float:
+    left_mean = mean(left)
+    right_mean = mean(right)
+    numerator = sum(
+        (x - left_mean) * (y - right_mean)
+        for x, y in zip(left, right)
+    )
+    left_square = sum((value - left_mean) ** 2 for value in left)
+    right_square = sum((value - right_mean) ** 2 for value in right)
+    if left_square == 0 or right_square == 0:
+        raise RuntimeError("correlation is undefined for a constant vector")
+    return numerator / math.sqrt(left_square * right_square)
+
+
+def spearman(left: list[float], right: list[float]) -> float:
+    return pearson(average_ranks(left), average_ranks(right))
+
+
+def load_run(path: Path, *, expected_kind: str, expected_seed: int) -> dict[str, Any]:
+    value = json.loads(path.read_text())
+    if value["protocol_id"] != PROTOCOL_ID:
+        raise RuntimeError(f"protocol mismatch: {path}")
+    if value["run_kind"] != expected_kind or value["seed"] != expected_seed:
+        raise RuntimeError(f"run identity mismatch: {path}")
+    canonical = value.pop("canonical_sha256_without_self")
+    if canonical_sha256(value) != canonical:
+        raise RuntimeError(f"canonical hash mismatch: {path}")
+    value["canonical_sha256_without_self"] = canonical
+    if expected_kind != "smoke":
+        if not value["round05_equivalence"]["passed"]:
+            raise RuntimeError(f"Round 05 equivalence failed: {path}")
+        if value["steps"] != 8000:
+            raise RuntimeError(f"formal step count mismatch: {path}")
+    return value
+
+
+def final_diagnostic(run: dict[str, Any]) -> dict[str, Any]:
+    matches = [item for item in run["diagnostics"] if item["step"] == 8000]
+    if len(matches) != 1:
+        raise RuntimeError("final diagnostic missing or duplicated")
+    return matches[0]
+
+
+def metric(
+    run: dict[str, Any],
+    *,
+    position: str,
+    reduction: str,
+    mode: str = "learned",
+) -> dict[str, Any]:
+    return final_diagnostic(run)["modes"][mode]["positions"][position][
+        "reductions"
+    ][reduction]
+
+
+def compare_replay(formal: dict[str, Any], replay: dict[str, Any]) -> dict[str, Any]:
+    fields = (
+        "architecture",
+        "depth",
+        "seed",
+        "steps",
+        "batch_size",
+        "target_bytes_seen",
+        "manifest",
+        "model",
+        "optimizer",
+        "hashes",
+        "evaluations",
+        "diagnostics",
+        "training_history",
+        "environment",
+        "artifacts",
+        "round05_equivalence",
+    )
+    checks = {field: formal[field] == replay[field] for field in fields}
+    passed = all(checks.values())
+    if not passed:
+        raise RuntimeError(f"Round 06 replay mismatch: {checks}")
+    compare_payload = {field: formal[field] for field in fields}
+    return {
+        "passed": True,
+        "checks": checks,
+        "frozen_compare_sha256": canonical_sha256(compare_payload),
+    }
+
+
+def reduction_robustness(runs: list[dict[str, Any]], manifest: dict[str, Any]) -> dict[str, Any]:
+    threshold = manifest["thresholds"]
+    cells = []
+    for run in runs:
+        reference = metric(
+            run,
+            position="post_mlp_state",
+            reduction="element_rms",
+        )
+        for reduction in MAIN_REDUCTIONS:
+            candidate = metric(
+                run,
+                position="post_mlp_state",
+                reduction=reduction,
+            )
+            stats = candidate["statistics"]
+            rho = spearman(reference["values"], candidate["values"])
+            checks = {
+                "spike_contrast": (
+                    stats["spike_contrast"]
+                    >= threshold["spike_contrast"]
+                ),
+                "top_five_overlap": (
+                    stats["top_five_spike_overlap"]
+                    >= threshold["top_five_min_overlap"]
+                ),
+                "spearman": rho >= threshold["spearman_minimum"],
+            }
+            cells.append(
+                {
+                    "seed": run["seed"],
+                    "reduction": reduction,
+                    "spike_contrast": stats["spike_contrast"],
+                    "top_five_layers": stats["top_five_layers"],
+                    "top_five_spike_overlap": stats[
+                        "top_five_spike_overlap"
+                    ],
+                    "spearman_vs_element_rms": rho,
+                    "checks": checks,
+                    "passed": all(checks.values()),
+                }
+            )
+    passed_count = sum(item["passed"] for item in cells)
+    if passed_count == len(cells):
+        verdict = "robust within the preregistered reduction family"
+    elif passed_count == 0:
+        verdict = "not robust at this threshold"
+    else:
+        verdict = "mixed"
+    return {
+        "verdict": verdict,
+        "passed_cells": passed_count,
+        "total_cells": len(cells),
+        "cells": cells,
+    }
+
+
+def visible_positions(runs: list[dict[str, Any]], manifest: dict[str, Any]) -> dict[str, Any]:
+    threshold = manifest["thresholds"]["spike_contrast"]
+    positions = []
+    for position in POSITIONS:
+        per_seed = []
+        for run in runs:
+            stats = metric(
+                run, position=position, reduction="element_rms"
+            )["statistics"]
+            per_seed.append(
+                {
+                    "seed": run["seed"],
+                    "spike_contrast": stats["spike_contrast"],
+                    "peak_layer": stats["peak_layer"],
+                    "peak_normalized": stats["peak_normalized"],
+                    "passed": stats["spike_contrast"] >= threshold,
+                }
+            )
+        positions.append(
+            {
+                "position": position,
+                "visible_3_of_3": all(item["passed"] for item in per_seed),
+                "per_seed": per_seed,
+                "mean_spike_contrast": mean(
+                    item["spike_contrast"] for item in per_seed
+                ),
+            }
+        )
+    visible = [item["position"] for item in positions if item["visible_3_of_3"]]
+    return {
+        "criterion": f"step 8000 element-RMS spike contrast >= {threshold} in 3/3 seeds",
+        "visible_positions": visible,
+        "earliest_observed_tensor": visible[0] if visible else None,
+        "verdict": "visible at one or more positions" if visible else "position-mixed",
+        "positions": positions,
+    }
+
+
+def intervention_effect(
+    runs: list[dict[str, Any]],
+    manifest: dict[str, Any],
+    *,
+    source_mode: str,
+    target_mode: str,
+    label: str,
+) -> dict[str, Any]:
+    threshold = manifest["thresholds"]["material_relative_drop"]
+    epsilon = manifest["thresholds"]["positive_denominator_epsilon"]
+    per_seed = []
+    for run in runs:
+        source = metric(
+            run,
+            position="post_mlp_state",
+            reduction="element_rms",
+            mode=source_mode,
+        )["statistics"]
+        target = metric(
+            run,
+            position="post_mlp_state",
+            reduction="element_rms",
+            mode=target_mode,
+        )["statistics"]
+        if (
+            not math.isfinite(source["spike_contrast"])
+            or not math.isfinite(source["peak_normalized"])
+            or source["spike_contrast"] <= epsilon
+            or source["peak_normalized"] <= epsilon
+        ):
+            raise RuntimeError("invalid intervention denominator")
+        contrast_drop = (
+            source["spike_contrast"] - target["spike_contrast"]
+        ) / source["spike_contrast"]
+        peak_drop = (
+            source["peak_normalized"] - target["peak_normalized"]
+        ) / source["peak_normalized"]
+        per_seed.append(
+            {
+                "seed": run["seed"],
+                "source_spike_contrast": source["spike_contrast"],
+                "target_spike_contrast": target["spike_contrast"],
+                "relative_drop_contrast": contrast_drop,
+                "source_peak_normalized": source["peak_normalized"],
+                "target_peak_normalized": target["peak_normalized"],
+                "relative_drop_peak": peak_drop,
+                "passed": (
+                    contrast_drop >= threshold and peak_drop >= threshold
+                ),
+            }
+        )
+    passed = all(item["passed"] for item in per_seed)
+    same_direction = all(
+        item["relative_drop_contrast"] > 0
+        and item["relative_drop_peak"] > 0
+        for item in per_seed
+    )
+    opposite_direction = all(
+        item["relative_drop_contrast"] < 0
+        and item["relative_drop_peak"] < 0
+        for item in per_seed
+    )
+    if passed:
+        verdict = "material sensitivity at the preregistered threshold"
+    elif same_direction:
+        verdict = "same-direction but below the joint threshold"
+    elif opposite_direction:
+        verdict = "opposite direction in 3/3 seeds; no material reduction"
+    else:
+        verdict = "mixed"
+    return {
+        "label": label,
+        "source_mode": source_mode,
+        "target_mode": target_mode,
+        "threshold": threshold,
+        "passed_3_of_3": passed,
+        "same_direction_3_of_3": same_direction,
+        "opposite_direction_3_of_3": opposite_direction,
+        "verdict": verdict,
+        "per_seed": per_seed,
+        "mean_relative_drop_contrast": mean(
+            item["relative_drop_contrast"] for item in per_seed
+        ),
+        "mean_relative_drop_peak": mean(
+            item["relative_drop_peak"] for item in per_seed
+        ),
+    }
+
+
+def intervention_summary(runs: list[dict[str, Any]], manifest: dict[str, Any]) -> dict[str, Any]:
+    forward_gates = []
+    for run in runs:
+        diagnostic = final_diagnostic(run)
+        forward_gates.append(
+            {
+                "seed": run["seed"],
+                "passed": diagnostic["forward_identity_gate"]["passed"],
+            }
+        )
+    if not all(item["passed"] for item in forward_gates):
+        raise RuntimeError("a final forward identity gate failed")
+    return {
+        "scope": manifest["interventions"]["scope"],
+        "forward_identity": forward_gates,
+        "softmax_key_path": intervention_effect(
+            runs,
+            manifest,
+            source_mode="learned",
+            target_mode="detached_learned",
+            label="global removal of all mixer softmax/query/key source-gradient paths",
+        ),
+        "value_coefficients": intervention_effect(
+            runs,
+            manifest,
+            source_mode="detached_learned",
+            target_mode="uniform_value_backward",
+            label="global replacement of learned value-backward coefficients with 1/N",
+        ),
+    }
+
+
+def trajectory(runs: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    result = []
+    for run in runs:
+        evaluations = {
+            item["step"]: item["bits_per_byte"]
+            for item in run["evaluations"]
+        }
+        points = []
+        for diagnostic in run["diagnostics"]:
+            stats = diagnostic["modes"]["learned"]["positions"][
+                "post_mlp_state"
+            ]["reductions"]["element_rms"]["statistics"]
+            points.append(
+                {
+                    "step": diagnostic["step"],
+                    "bits_per_byte": evaluations[diagnostic["step"]],
+                    "population_cv": stats["population_cv"],
+                    "spike_contrast": stats["spike_contrast"],
+                    "peak_layer": stats["peak_layer"],
+                    "peak_normalized": stats["peak_normalized"],
+                }
+            )
+        result.append({"seed": run["seed"], "points": points})
+    return result
+
+
+def mixer_associations(runs: list[dict[str, Any]]) -> dict[str, Any]:
+    rows = []
+    for run in runs:
+        diagnostic = final_diagnostic(run)
+        gradients = metric(
+            run,
+            position="post_mlp_state",
+            reduction="element_rms",
+        )["statistics"]["normalized"]
+        mixers = diagnostic["modes"]["learned"]["mixers"]
+        for layer in range(1, 33):
+            attention = mixers[2 * (layer - 1)]
+            mlp = mixers[2 * (layer - 1) + 1]
+            rows.append(
+                {
+                    "seed": run["seed"],
+                    "layer": layer,
+                    "group": (layer - 1) // 4 + 1,
+                    "offset": (layer - 1) % 4 + 1,
+                    "normalized_gradient": gradients[layer - 1],
+                    "attention_latest": attention["latest_source_mass"],
+                    "attention_entropy": attention["normalized_entropy"],
+                    "mlp_latest": mlp["latest_source_mass"],
+                    "mlp_entropy": mlp["normalized_entropy"],
+                    "attention_max": attention["max_source_mass"],
+                    "mlp_max": mlp["max_source_mass"],
+                }
+            )
+
+    def correlations(selected: list[dict[str, Any]]) -> dict[str, Any]:
+        gradient = [row["normalized_gradient"] for row in selected]
+        result = {"points": len(selected)}
+        for key in (
+            "attention_latest",
+            "attention_entropy",
+            "mlp_latest",
+            "mlp_entropy",
+            "attention_max",
+            "mlp_max",
+        ):
+            values = [row[key] for row in selected]
+            result[key] = {
+                "pearson": pearson(gradient, values),
+                "spearman": spearman(gradient, values),
+            }
+        return result
+
+    target_layers = []
+    for layer in range(19, 29):
+        selected = [row for row in rows if row["layer"] == layer]
+        target_layers.append(
+            {
+                "layer": layer,
+                "group": selected[0]["group"],
+                "offset": selected[0]["offset"],
+                **{
+                    key: mean(row[key] for row in selected)
+                    for key in (
+                        "normalized_gradient",
+                        "attention_latest",
+                        "attention_entropy",
+                        "mlp_latest",
+                        "mlp_entropy",
+                        "attention_max",
+                        "mlp_max",
+                    )
+                },
+                "per_seed_normalized_gradient": [
+                    row["normalized_gradient"] for row in selected
+                ],
+            }
+        )
+    return {
+        "identity": "observational association; not causal attribution",
+        "all_layers": correlations(rows),
+        "layers_19_28": correlations(
+            [row for row in rows if 19 <= row["layer"] <= 28]
+        ),
+        "target_layers": target_layers,
+    }
+
+
+def final_arrays(runs: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    values = []
+    for run in runs:
+        positions = {}
+        for position in POSITIONS:
+            positions[position] = {}
+            for reduction in ALL_REDUCTIONS:
+                item = metric(
+                    run, position=position, reduction=reduction
+                )
+                positions[position][reduction] = {
+                    "values": item["values"],
+                    "statistics": item["statistics"],
+                }
+        interventions = {}
+        for mode in MODES:
+            item = metric(
+                run,
+                position="post_mlp_state",
+                reduction="element_rms",
+                mode=mode,
+            )
+            interventions[mode] = {
+                "values": item["values"],
+                "statistics": item["statistics"],
+            }
+        values.append(
+            {
+                "seed": run["seed"],
+                "positions": positions,
+                "interventions": interventions,
+                "final_mixers": final_diagnostic(run)["modes"]["learned"][
+                    "mixers"
+                ],
+            }
+        )
+    return values
+
+
+def compact_final_arrays(runs: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    values = []
+    for run in runs:
+        position_element_rms = {
+            position: metric(
+                run,
+                position=position,
+                reduction="element_rms",
+            )
+            for position in POSITIONS
+        }
+        post_mlp_reductions = {
+            reduction: metric(
+                run,
+                position="post_mlp_state",
+                reduction=reduction,
+            )
+            for reduction in ALL_REDUCTIONS
+        }
+        interventions = {
+            mode: metric(
+                run,
+                position="post_mlp_state",
+                reduction="element_rms",
+                mode=mode,
+            )
+            for mode in MODES
+        }
+        values.append(
+            {
+                "seed": run["seed"],
+                "position_element_rms": position_element_rms,
+                "post_mlp_reductions": post_mlp_reductions,
+                "interventions": interventions,
+            }
+        )
+    return values
+
+
+def write_json(path: Path, value: dict[str, Any]) -> None:
+    value["canonical_sha256_without_self"] = canonical_sha256(value)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(path.suffix + ".tmp")
+    temporary.write_text(
+        json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
+    )
+    temporary.replace(path)
+
+
+def main() -> None:
+    args = parse_args()
+    manifest = json.loads(args.manifest.read_text())
+    if manifest["protocol_id"] != PROTOCOL_ID:
+        raise RuntimeError("manifest protocol mismatch")
+
+    formal_paths = {
+        seed: args.raw_dir / f"formal-seed-{seed}.json"
+        for seed in SEEDS
+    }
+    replay_path = args.raw_dir / f"replay-seed-{SEEDS[0]}.json"
+    formal = [
+        load_run(formal_paths[seed], expected_kind="formal", expected_seed=seed)
+        for seed in SEEDS
+    ]
+    replay = load_run(
+        replay_path, expected_kind="replay", expected_seed=SEEDS[0]
+    )
+    replay_gate = compare_replay(formal[0], replay)
+
+    reduction = reduction_robustness(formal, manifest)
+    positions = visible_positions(formal, manifest)
+    interventions = intervention_summary(formal, manifest)
+    trajectories = trajectory(formal)
+    mixers = mixer_associations(formal)
+    arrays = final_arrays(formal)
+
+    raw_files = {
+        path.name: {
+            "file_sha256": file_sha256(path),
+            "canonical_sha256": run["canonical_sha256_without_self"],
+        }
+        for path, run in [
+            *[(formal_paths[seed], run) for seed, run in zip(SEEDS, formal)],
+            (replay_path, replay),
+        ]
+    }
+    reproduction = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "raw_files": raw_files,
+        "replay_gate": replay_gate,
+        "artifacts": {
+            "manifest": file_sha256(args.manifest),
+            "runner": formal[0]["artifacts"]["runner_sha256"],
+            "protocol": formal[0]["artifacts"]["protocol_sha256"],
+            "scoping": formal[0]["artifacts"]["scoping_sha256"],
+            "analyzer": file_sha256(Path(__file__)),
+        },
+    }
+    write_json(args.reproduction_output, reproduction)
+
+    aggregate = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "study": {
+            "identity": manifest["study_identity"],
+            "architecture": "block",
+            "depth": 32,
+            "seeds": list(SEEDS),
+            "steps": 8000,
+            "spike_layers": list(SPIKE_LAYERS),
+        },
+        "gates": {
+            "round05_equivalence": [
+                {
+                    "seed": run["seed"],
+                    **run["round05_equivalence"],
+                }
+                for run in formal
+            ],
+            "replay": replay_gate,
+            "forward_identity": interventions["forward_identity"],
+            "initialization_negative_control": [
+                {
+                    "seed": run["seed"],
+                    "passed": run["diagnostics"][0][
+                        "initialization_negative_control"
+                    ]["passed"],
+                }
+                for run in formal
+            ],
+            "loss_scale": [
+                {
+                    "seed": run["seed"],
+                    "passed": run["diagnostics"][0][
+                        "loss_scale_gate"
+                    ]["passed"],
+                }
+                for run in formal
+            ],
+        },
+        "verdicts": {
+            "reduction_robustness": reduction,
+            "visible_positions": positions,
+            "interventions": interventions,
+        },
+        "trajectory": trajectories,
+        "mixer_associations": mixers,
+        "final_arrays": arrays,
+        "runs": [
+            {
+                "seed": run["seed"],
+                "canonical_sha256": run["canonical_sha256_without_self"],
+                "final_bpc": run["evaluations"][-1]["bits_per_byte"],
+                "final_model_state": run["hashes"]["final_model_state"],
+                "final_optimizer_state": run["hashes"][
+                    "final_optimizer_state"
+                ],
+                "timing": run["timing"],
+            }
+            for run in formal
+        ],
+        "reproduction_canonical_sha256": reproduction[
+            "canonical_sha256_without_self"
+        ],
+    }
+    write_json(args.aggregate_output, aggregate)
+
+    compact = {
+        "schema_version": 1,
+        "protocol_id": PROTOCOL_ID,
+        "study": aggregate["study"],
+        "gates": aggregate["gates"],
+        "verdicts": aggregate["verdicts"],
+        "trajectory": trajectories,
+        "mixer_associations": mixers,
+        "final_arrays": compact_final_arrays(formal),
+        "runs": aggregate["runs"],
+        "hashes": {
+            "aggregate_canonical_sha256": aggregate[
+                "canonical_sha256_without_self"
+            ],
+            "reproduction_canonical_sha256": reproduction[
+                "canonical_sha256_without_self"
+            ],
+        },
+    }
+    write_json(args.compact_output, compact)
+    print(
+        json.dumps(
+            {
+                "aggregate": str(args.aggregate_output),
+                "compact": str(args.compact_output),
+                "reproduction": str(args.reproduction_output),
+                "reduction_verdict": reduction["verdict"],
+                "earliest_observed_tensor": positions[
+                    "earliest_observed_tensor"
+                ],
+                "softmax_key_path": interventions["softmax_key_path"][
+                    "verdict"
+                ],
+                "value_coefficients": interventions["value_coefficients"][
+                    "verdict"
+                ],
+                "replay_exact": replay_gate["passed"],
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

experiments/k3/attnres_spike/manifest.json

@@ -0,0 +1,118 @@
+{
+  "schema_version": 1,
+  "protocol_id": "llm-atlas-k3-attnres-spike-path-v1",
+  "parent_protocol_id": "llm-atlas-k3-attnres-gradient-scale-v1",
+  "study_identity": "targeted follow-up informed by Round 05; not blind discovery",
+  "architecture": "block",
+  "depth": 32,
+  "aggregation_groups": 8,
+  "formal_seeds": [
+    2026073001,
+    2026073002,
+    2026073003
+  ],
+  "replay": {
+    "architecture": "block",
+    "depth": 32,
+    "seed": 2026073001
+  },
+  "training": {
+    "steps": 8000,
+    "batch_size": 32,
+    "context": 256,
+    "target_bytes_per_cell": 65536000,
+    "diagnostic_steps": [
+      0,
+      100,
+      500,
+      2000,
+      4000,
+      8000
+    ]
+  },
+  "positions": [
+    "pre_attention_input",
+    "attention_branch_output",
+    "post_attention_state",
+    "pre_mlp_input",
+    "mlp_branch_output",
+    "post_mlp_state"
+  ],
+  "reductions": {
+    "confirmatory": [
+      "element_rms",
+      "token_rms_mean",
+      "token_rms_median",
+      "token_rms_p95"
+    ],
+    "exploratory": [
+      "batch_mean_rms",
+      "token_mean_rms"
+    ],
+    "algebraic_control": [
+      "global_l2"
+    ],
+    "quantile_definition": "Hyndman-Fan Type 7 linear interpolation on explicitly sorted values"
+  },
+  "interventions": {
+    "modes": [
+      "learned",
+      "detached_learned",
+      "uniform_value_backward"
+    ],
+    "steps": [
+      0,
+      8000
+    ],
+    "scope": "all 64 depth mixers plus the output mixer",
+    "training_uses_custom_autograd": false,
+    "confirmatory_reduction": "element_rms"
+  },
+  "spike_layers_one_based": [
+    21,
+    22,
+    23,
+    24,
+    25
+  ],
+  "thresholds": {
+    "spike_contrast": 1.5,
+    "top_five_min_overlap": 3,
+    "spearman_minimum": 0.8,
+    "material_relative_drop": 0.2,
+    "spectrum_absolute_tolerance": 1e-06,
+    "raw_relative_tolerance": 1e-06,
+    "positive_denominator_epsilon": 1e-30
+  },
+  "parent_artifacts": {
+    "manifest_path": "experiments/k3/attnres_gradient/manifest.json",
+    "manifest_sha256": "080afb17d1e036c0bba0a799fdb8b98ee4ad652bd42dd1b3b67110dd2ede6371",
+    "runner_path": "experiments/k3/attnres_gradient/train.py",
+    "runner_sha256": "04ae69e10c58972c9193c2d31c7e09d924a0d0e834107afa4ba128c64ac5800f",
+    "protocol_path": "research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md",
+    "protocol_sha256": "f772629b3b82975b6756721c3a3bb57cc4171dfa26ba5b1e8043dc91c9dcce22",
+    "formal_schedule_sha256": "5041e09b167f229248d2462324e8c254b8f5938975f135dcd8192b00a54a4f4e",
+    "validation_tensor_sha256": "f459316f13078a163b47c133511bb7181e05170ab89516e196490113893ce338",
+    "diagnostic_tensor_sha256": "21117e31db302b10d67b63f035665dc8f220b879d216ccd12b7d2ba86e7b1716"
+  },
+  "round05_expected": {
+    "2026073001": {
+      "raw_file_sha256": "29e1d638b481619c7b32de402122523db8b17cc1fc67a8881fa1ba132a1d5c38",
+      "canonical_sha256": "c7554d9beea6fe9e63617aafd88403f7fc93b4b305a0c6d603a98a8572e5b5f1",
+      "final_model_state": "3f0b97ece3a15571ba3d656f589f512ca0bb9e20083c9f58a42ccaee14892f59",
+      "final_optimizer_state": "ed03e6fbd4a12d8b063dcb22e0437754285f54d585374cd52fbd534f05d24637"
+    },
+    "2026073002": {
+      "raw_file_sha256": "21199deb2199395061e51220fd8c7afd04a1135a6381e406da9b5795e3ad5032",
+      "canonical_sha256": "b4262e697e2269457fdebf31f75008383c6e8024ee1bbf96e5962fb2ba1dec18",
+      "final_model_state": "bd2556388aeaa211b798c283c7cbd8ccd29edf166a2922fa13d172e8dfdc38d1",
+      "final_optimizer_state": "0b101eab3bc7d8d654be2ea335c86fc25563ce19912d721844ee4e639c569e77"
+    },
+    "2026073003": {
+      "raw_file_sha256": "c0d7f1bcfa9fa7f8f3134ca4571bdf23a951182d03b7de9611a6b4b89667d4ac",
+      "canonical_sha256": "513b85666fcdbf75424596e968cd73733314ec188345e453998de08b18dcef65",
+      "final_model_state": "638568aede21890773b6932a19ec4e112f5ac0a4770ba3402fcd82980a9ecf76",
+      "final_optimizer_state": "83947fd743ec8e3e31ca7788fd201981846f0f1c7e9e38c88afcef95cfc6ec4e"
+    }
+  }
+}

experiments/k3/attnres_spike/reproduction.json

@@ -0,0 +1,52 @@
+{
+  "artifacts": {
+    "analyzer": "b9dd7f723e8c5765ecffd9ae3050ff7a95643f95853adaeeaa2c2c323d0ca057",
+    "manifest": "d5302a249249a07d362819134763d14e7d32307f22cff416c665ed9606142fef",
+    "protocol": "6cb101b8760d9f1c81caeb2f16880b16152da103867224a06761a75a12984a16",
+    "runner": "77298081d3c491d2e88e4705995174b9879ef377f520eb5fe5ea107e7a1da084",
+    "scoping": "590166bd62580bb8238293823cfcc39bc0a465fec4c697025343f3f1138abd27"
+  },
+  "canonical_sha256_without_self": "7c522314742920961be84b750d823ce41814f10fd6129f0531208116e2fe3515",
+  "protocol_id": "llm-atlas-k3-attnres-spike-path-v1",
+  "raw_files": {
+    "formal-seed-2026073001.json": {
+      "canonical_sha256": "76b0ccfb55c38baef50c395788ac4b351cbe0d58de70064b50702acb5c93f515",
+      "file_sha256": "e39e93b7a7fce3c56f5f14f95cfdc04afdce53628affee1202fe62bd1bdb7f71"
+    },
+    "formal-seed-2026073002.json": {
+      "canonical_sha256": "5352c74eca853b375c0e85933dafd7c5916c39fc59052e742ca14ffd6d68bc78",
+      "file_sha256": "1c6f6c731030ec0adb2a8e7a4d586e0c4005cc3319568a7ac83c08c2a4b8eaf8"
+    },
+    "formal-seed-2026073003.json": {
+      "canonical_sha256": "7e764c07e90b78c4cd0acc2e99600225f16428cbb25d5188a0a5a8fe797f8766",
+      "file_sha256": "115f8245577ece6dfaaa8ada68445c186e6523a7f3b26efcc3eb4c0c4ce82406"
+    },
+    "replay-seed-2026073001.json": {
+      "canonical_sha256": "c9ebccba0acf10b05fea65beade50e588fd9a83b1807f7ebba8ec1ab836b26fb",
+      "file_sha256": "4f91c7a2698662ea69af9aa3425e144107834fd9b6b717fb82b84f153b867dc5"
+    }
+  },
+  "replay_gate": {
+    "checks": {
+      "architecture": true,
+      "artifacts": true,
+      "batch_size": true,
+      "depth": true,
+      "diagnostics": true,
+      "environment": true,
+      "evaluations": true,
+      "hashes": true,
+      "manifest": true,
+      "model": true,
+      "optimizer": true,
+      "round05_equivalence": true,
+      "seed": true,
+      "steps": true,
+      "target_bytes_seen": true,
+      "training_history": true
+    },
+    "frozen_compare_sha256": "05396eaf56429ca8d87815ecf00a5f78be5a9b2ec7cc96014856c09dcedcbb87",
+    "passed": true
+  },
+  "schema_version": 1
+}

experiments/k3/attnres_spike/scope.py

@@ -0,0 +1,178 @@
+#!/usr/bin/env python3
+"""Rebuild the Round 06 scoping table from frozen Round 05 raw files."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import statistics
+from pathlib import Path
+from typing import Any, Callable
+
+
+SEEDS = (2026073001, 2026073002, 2026073003)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--raw-dir",
+        type=Path,
+        default=Path(__file__).resolve().parents[1]
+        / "attnres_gradient"
+        / "results"
+        / "raw",
+    )
+    return parser.parse_args()
+
+
+def pearson(left: list[float], right: list[float]) -> float:
+    left_mean = statistics.fmean(left)
+    right_mean = statistics.fmean(right)
+    numerator = sum(
+        (x - left_mean) * (y - right_mean)
+        for x, y in zip(left, right)
+    )
+    left_square = sum((value - left_mean) ** 2 for value in left)
+    right_square = sum((value - right_mean) ** 2 for value in right)
+    return numerator / math.sqrt(left_square * right_square)
+
+
+def mixer_metrics(value: dict[str, Any]) -> dict[str, float | int]:
+    weights = value["mean_weights"]
+    sources = value["sources"]
+    return {
+        "sources": sources,
+        "latest": weights[-1],
+        "maximum": max(weights),
+        "normalized_entropy": (
+            1.0
+            if sources == 1
+            else value["entropy_mean"] / math.log(sources)
+        ),
+    }
+
+
+def average(rows: list[dict[str, Any]], key: Callable[[dict[str, Any]], float]) -> float:
+    return statistics.fmean(key(row) for row in rows)
+
+
+def main() -> None:
+    args = parse_args()
+    rows = []
+    source_hashes = {}
+    for seed in SEEDS:
+        path = (
+            args.raw_dir
+            / f"formal-depth-32-block-seed-{seed}.json"
+        )
+        run = json.loads(path.read_text())
+        source_hashes[str(seed)] = run["canonical_sha256_without_self"]
+        diagnostic = next(
+            item for item in run["diagnostics"] if item["step"] == 8000
+        )
+        gradients = diagnostic["activation_grad_rms_by_block"]
+        gradient_mean = statistics.fmean(gradients)
+        for layer in range(32):
+            rows.append(
+                {
+                    "seed": seed,
+                    "layer": layer + 1,
+                    "group": layer // 4 + 1,
+                    "offset": layer % 4 + 1,
+                    "normalized_gradient": gradients[layer] / gradient_mean,
+                    "attention": mixer_metrics(
+                        diagnostic["depth_weights"][2 * layer]
+                    ),
+                    "mlp": mixer_metrics(
+                        diagnostic["depth_weights"][2 * layer + 1]
+                    ),
+                }
+            )
+
+    layer_means = []
+    for layer in range(1, 33):
+        selected = [row for row in rows if row["layer"] == layer]
+        layer_means.append(
+            {
+                "layer": layer,
+                "group": selected[0]["group"],
+                "offset": selected[0]["offset"],
+                "normalized_gradient": average(
+                    selected, lambda row: row["normalized_gradient"]
+                ),
+                "attention_latest": average(
+                    selected, lambda row: row["attention"]["latest"]
+                ),
+                "mlp_latest": average(
+                    selected, lambda row: row["mlp"]["latest"]
+                ),
+                "attention_normalized_entropy": average(
+                    selected,
+                    lambda row: row["attention"]["normalized_entropy"],
+                ),
+                "mlp_normalized_entropy": average(
+                    selected, lambda row: row["mlp"]["normalized_entropy"]
+                ),
+                "per_seed_normalized_gradient": [
+                    row["normalized_gradient"] for row in selected
+                ],
+            }
+        )
+
+    correlations = {}
+    for label, selected in (
+        ("all_layers", rows),
+        ("layers_19_28", [
+            row for row in rows if 19 <= row["layer"] <= 28
+        ]),
+    ):
+        gradients = [row["normalized_gradient"] for row in selected]
+        correlations[label] = {
+            "points": len(selected),
+            "attention_latest": pearson(
+                gradients,
+                [row["attention"]["latest"] for row in selected],
+            ),
+            "mlp_latest": pearson(
+                gradients, [row["mlp"]["latest"] for row in selected]
+            ),
+            "attention_normalized_entropy": pearson(
+                gradients,
+                [
+                    row["attention"]["normalized_entropy"]
+                    for row in selected
+                ],
+            ),
+            "mlp_normalized_entropy": pearson(
+                gradients,
+                [row["mlp"]["normalized_entropy"] for row in selected],
+            ),
+            "attention_maximum": pearson(
+                gradients,
+                [row["attention"]["maximum"] for row in selected],
+            ),
+            "mlp_maximum": pearson(
+                gradients, [row["mlp"]["maximum"] for row in selected]
+            ),
+        }
+
+    print(
+        json.dumps(
+            {
+                "schema_version": 1,
+                "identity": "exploratory Round 05 scoping; not confirmatory Round 06",
+                "source_canonical_sha256": source_hashes,
+                "layers_19_28": layer_means[18:28],
+                "correlations": correlations,
+            },
+            ensure_ascii=False,
+            indent=2,
+            sort_keys=True,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

package.json

@@ -19,6 +19,11 @@
     "build:data:deepseek-chat-cross-source-sampling": "node scripts/build-deepseek-chat-cross-source-sampling-compact.mjs",
     "build:data:deepseek-chat-task-bootstrap": "node scripts/build-deepseek-chat-task-bootstrap-crn-compact.mjs",
     "check:data:deepseek-chat-task-bootstrap": "node scripts/check-deepseek-chat-task-bootstrap-crn-data.mjs",
+    "check:data:k3-attnres": "node scripts/check-k3-attnres-data.mjs",
+    "check:data:k3-attnres-gradient": "node scripts/check-k3-attnres-gradient-data.mjs",
+    "check:data:k3-attnres-spike": "node scripts/check-k3-attnres-spike-data.mjs",
+    "check:data:k3-attnres-local-path": "node scripts/check-k3-attnres-local-path-data.mjs",
+    "check:data:k3-attnres-forward": "node scripts/check-k3-attnres-forward-data.mjs",
     "check:site": "node scripts/check-site.mjs",
     "check:moe-browser": "node scripts/check-moe-browser.mjs",
     "check:reasoning-browser": "node scripts/check-reasoning-browser.mjs",
@@ -38,6 +43,11 @@
     "check:deepseek-sampling-browser": "node scripts/check-deepseek-sampling-browser.mjs",
     "check:deepseek-cross-source-sampling-browser": "node scripts/check-deepseek-cross-source-sampling-browser.mjs",
     "check:deepseek-task-bootstrap-browser": "node scripts/check-deepseek-task-bootstrap-browser.mjs",
+    "check:k3-attnres-browser": "node scripts/check-k3-attnres-browser.mjs",
+    "check:k3-attnres-gradient-browser": "node scripts/check-k3-attnres-gradient-browser.mjs",
+    "check:k3-attnres-spike-browser": "node scripts/check-k3-attnres-spike-browser.mjs",
+    "check:k3-attnres-local-path-browser": "node scripts/check-k3-attnres-local-path-browser.mjs",
+    "check:k3-attnres-forward-browser": "node scripts/check-k3-attnres-forward-browser.mjs",
     "check:k3-browser": "node scripts/check-k3-browser.mjs"
   },
   "dependencies": {

research/K3_ARTIFACT_AUDIT.md

@@ -438,11 +438,19 @@ src/data/k3-flashkda-runtime.json
 - [x] 使用 CUDA 13.0 / glibc 2.39 隔离环境产出可校验 `sm_120a` wheel；
 - [x] RTX 5090 上六组 FlashKDA / official torch reference exact correctness；
 - [x] K3 fixed / varlen shape、三种 state mode 的 1,800 个 CUDA Event samples；
+- [x] 在不裁剪 `A_log` 的前提下完成独立缩小 AttnRes 机制实验；
+- [x] Baseline / Full / Block × 三 seed × 2,000 steps 与指定正式格全新进程 exact replay；
+- [x] BPC 主结果、Residual RMS、depth weights、gradient counterevidence 与执行成本公开入账；
 - [ ] 取得真实 hidden-state / router load trace；
 - [ ] 取得可加载的 reduced checkpoint、官方 trace 或多机资源；
 - [ ] 对 Figure 3 / 4 / 5 做真实数值重绘；
 - [ ] 对 AttnRes 读取分布做真实 token / layer trace。
 
+这里的“缩小 AttnRes 实验”只完成独立机制 probe，不满足最后一项真实 K3 trace。完整合同与
+审计见 `K3_ATTNRES_REDUCED_PROTOCOL.md` / `K3_ATTNRES_REDUCED_AUDIT.md`。下一轮优先
+对齐论文 gradient 观测对象并增加 depth / budget；真实 checkpoint forward 继续等待公开
+`A_log [128] → [96]` 转换或权重修订。
+
 ## 11. 发布与回归证据
 
 - 源提交：`2ef846f`；

research/K3_ATTNRES_FORWARD_TRAINING_AUDIT.md

@@ -0,0 +1,112 @@
+# Round 08 AttnRes 训练期前向干预：结果审计
+
+审计日期：2026-07-30  
+协议：`llm-atlas-k3-attnres-forward-training-v1`  
+结果后 Grok 会话：`019fb28b-a9e1-7643-8e43-06f5e16a2077`
+
+## 1. 一句话结论
+
+冻结 analyzer 的唯一主状态为：
+
+```text
+forward_training_attenuation_established_within_reduced_protocol
+```
+
+联合 `groups 6+7` 的训练期 uniform-forward 消融在三个预注册 seed 上同时通过
+`spike contrast` 与 `peak / mean` 的 20% attenuation 门；逐 seed 与三 seed 平均
+validation BPC 也全部通过质量护栏。结果后独立只读复算得到：
+
+```text
+blocking_errors = 0
+status_confirmed = true
+replay_confirmed = true
+```
+
+这只是在固定 depth-32 缩小 Block AttnRes、固定数据与 8,000-step 预算中的训练期
+架构消融；**不是真实 Kimi-K3 / 2.8T checkpoint 结果，也不是 Figure 5(c) 未公开
+telemetry 的复现。**
+
+## 2. 主门复算
+
+冻结定义：
+
+```text
+S = layers 21–25
+R = other 27 layers
+C = mean(g[S]) / mean(g[R])
+P = max(g) / mean(g)
+D = (X_reference - X_variant) / X_reference
+```
+
+主变体为 `uniform_groups_6_7_forward`，只读复算如下：
+
+| seed | C reference | C variant | C drop | P reference | P variant | P drop | ΔBPC |
+|---|---:|---:|---:|---:|---:|---:|---:|
+| 2026073001 | 3.046269 | 0.806011 | 73.54% | 3.219067 | 1.435618 | 55.40% | +0.005696 |
+| 2026073002 | 3.332848 | 0.758201 | 77.25% | 3.618924 | 1.375016 | 62.00% | +0.009598 |
+| 2026073003 | 1.881492 | 0.712882 | 62.11% | 2.288267 | 1.548141 | 32.34% | +0.004441 |
+
+因此 attenuation 为 `6/6`；最小 contrast drop 为 `62.11%`，最小 peak drop 为
+`32.34%`，都高于冻结的 `20%` 门槛。BPC 三格都低于 `+0.05`，平均
+`+0.006578` 低于 `+0.03`，质量门为 `4/4`。
+
+这里的 contrast 下降不等价于“尖峰层被关闭”：它可能由 spike-window 分子下降、
+27 层 reference 分母上升，或两者共同造成；被联合消融覆盖的 layers 26–28 仍属于
+这个分母。
+
+## 3. 完整性与复现
+
+- 13 个新 raw 文件完整：12 formal + 1 primary replay；
+- 每格 `65,536,000` target bytes，新处理总量 `851,968,000`；
+- 三个历史 paired reference 合计 `196,608,000` bytes，未在 Round 08 重跑；
+- 13/13 raw canonical self-hash、aggregate 与 reproduction self-hash 自洽；
+- architecture / depth / steps / batch 固定为 Block / 32 / 8,000 / 32；
+- initial public/mixer state、schedule、validation、diagnostic、input-gate、model、
+  optimizer 与 scientific environment 对同 seed historical reference 配对 exact；
+- primary seed 1 的 formal / replay scientific payload exact：
+  `b85563ca5cb53e60b39c3801d372376206105b8a089a8633b3e81973a7f0c051`。
+
+四个正式 selector 的 visit、source count、uniform arithmetic 与 reachability 均通过。
+被选 mixer 的 query / key norm 留在 optimizer param groups，但 forward 不再调用它们：
+gradient hook 为 0、Adam state 不存在、最终 tensor 与初始值 byte-exact。未选 mixer 的
+optimizer-state 检查是比科学协议更强的实现审计，不参与主 status。
+
+## 4. 描述性 non-additivity
+
+冻结的 bookkeeping residual 为：
+
+```text
+I67 = ln(Xref / X67) - ln(Xref / X6) - ln(Xref / X7)
+```
+
+step 8,000 的三 seed 平均为：
+
+- spike contrast：`-0.367038`
+- peak / mean：`-0.170448`
+
+它来自三套独立训练，只能描述 joint run 与两个 single runs 的 log-effect 残差；不能
+写成因果 interaction、Shapley contribution 或“group 6/7 互相抑制”的机制结论。
+
+## 5. 两阶段独立审阅
+
+结果前 Grok 实现审阅指出 smoke-only empty selector 的 visit census 可空真。正式
+4×3 路径全部是非空 selector，因此不影响 raw formal 数值；矩阵结束后已删除 early
+return、加入 `forward_calls > 0`，并重跑 step-zero、parent smoke、wrapper smoke
+与 equivalence。修补后的 learned wrapper 实际执行 39 次 forward，父/包装器 15 组
+科学字段仍全部 exact。
+
+结果后 Grok 在只读 sandbox 中从 13 个 raw 与三个 historical references 独立复算
+identity、自哈希、selector、pairing、两项主指标、BPC、replay 与 `I67`。它报告
+`0 mismatch`、`blocking_errors=0`，确认 analyzer status 与 claim boundary。
+
+## 6. 最终 claim boundary
+
+可以说：在这一固定缩小协议内，联合 group 6+7 的 train-time uniform-forward
+architecture ablation 相对历史同 seed reference 达到预注册 attenuation + BPC 门控。
+
+不能说：
+
+- 已定位真实 Kimi-K3 的训练尖峰；
+- 已复现 K3 报告 Figure 5(c)；
+- 已把 forward、natural backward 与 optimizer update 分离成纯因果效应；
+- 已证明下游能力等价、总体统计显著性、可加性或因果 interaction。

research/K3_ATTNRES_FORWARD_TRAINING_GROK_REVIEW.md

@@ -0,0 +1,69 @@
+# Round 08 Grok Headless 对抗审阅与处置
+
+审阅日期：2026-07-30  
+审阅会话：`019fb1ce-7ffa-70e3-b860-c4a31a4c6621`  
+身份：**外部模型的只读方法学审稿，不是论文证据源**
+
+## 1. 调用边界
+
+Grok CLI 使用 single/headless + plan permission 读取：
+
+- `research/K3_ATTNRES_FORWARD_TRAINING_SCOPING.md`
+- `research/K3_ATTNRES_FORWARD_TRAINING_PROTOCOL.md`
+- `experiments/k3/attnres_forward/manifest.json`
+- Round 04 / 05 父 runners
+
+关闭 web search、禁止 subagents；它没有修改文件，也没有运行训练。
+
+## 2. Blocking findings 与处置
+
+| finding | 风险 | 处置 |
+|---|---|---|
+| `mean` 与父 `softmax+einsum` 的 FP32 归约顺序不保证 byte-exact | step-0 会假失败 | **采纳**：选中路径改成参数无关 constant-zero logits，并复用同一 `softmax+einsum` kernel |
+| pairing 要求了父 JSON 不存在的 initial full hash，并把 GPU 名写成 exact | 科学身份正确却因 metadata 失败 | **采纳**：只 hard-gate 初始化、数据、拓扑、optimizer 与确定性/autocast 合同 |
+| replay 只排除三个字段，path 会制造假差异 | exact replay 假失败 | **采纳**：冻结 scientific canonical payload 与 path normalization |
+| wrapper 若把新 protocol ID 写入 `window_start` salt，会静默换数据 | 历史逐 seed 配对失效 | **采纳**：父 salt 只由父 manifest/runner 管，新 ID 只进 wrapper output |
+| 主判定没有在协议正文再写 final-only | analyzer 可能误读六 checkpoints | **采纳**：主 attenuation 与 BPC 只读 step 8,000 |
+| spike layer numbering 未 machine-readable 固定 | 可能整体平移一层 | **采纳**：manifest 新增 `spike_layers_1based`，协议钉死 Python index = layer−1 |
+
+## 3. Non-blocking findings 与处置
+
+全部采纳：
+
+- selected 参数留在 AdamW param groups，但因 graph 不可达而没有 state entry，不能写成
+  “训练了但没动”；
+- group 6 / 7 indices 与 source counts 经独立复算正确；
+- reference 统一称 historical paired reference；
+- layers 26–28 位于 contrast 分母 `R`，必须拆报 `mean(S)` / `mean(R)`；
+- `I67` 明确是三次独立训练之间的 log residual；
+- missing / non-finite / structural error 统一 `contract_failed`，不能冒充科学失败；
+- wrapper 禁止改变 FP32 residual、AdamW grouping、parent salt 与 empty-selector path。
+
+## 4. 算术复核
+
+```text
+8,000 × 32 × 256 = 65,536,000 bytes / cell
+12 formal cells   = 786,432,000 bytes
++ primary replay  = 851,968,000 newly processed bytes
+historical refs   = 196,608,000 bytes（不重跑）
+```
+
+selector：
+
+```text
+group 6       = 40..47; source N = 6,7,7,7,7,7,7,7
+group 7       = 48..55; source N = 7,8,8,8,8,8,8,8
+group 7 MLP   = 49,51,53,55
+groups 6+7    = 40..55
+```
+
+主 attenuation 为 `3 seeds × {contrast, peak} = 6` cells；quality 为三个
+per-seed BPC gates 加一个 mean gate，共 4 项。两者合取，且只读 final step。
+
+## 5. 复核结论
+
+Grok 判断研究身份、矩阵、source counts 与主公式骨架可以保留；主要风险来自浮点
+arithmetic identity、wrapper salt、over-exact metadata 与层号歧义。以上项目已在任何
+正式输出出现前全部修订，manifest 状态随后改为 `frozen-before-model-output`。
+
+审稿意见不会进入实验结果、论文事实或官网证据等级；它只用于结果前强化协议。

research/K3_ATTNRES_FORWARD_TRAINING_IMPLEMENTATION_REVIEW.md

@@ -0,0 +1,89 @@
+# Round 08 runner / analyzer 实现审阅与处置
+
+审阅日期：2026-07-30  
+Grok 会话：`019fb1f0-bb24-7202-8123-295edda7518f`  
+身份：**正式文件完成前的外部模型只读实现审计，不是结果或论文证据**
+
+## 1. 审阅边界
+
+Grok Headless 只读检查：
+
+- 冻结协议与 manifest；
+- `train.py` / `verify.py` / `analyze.py` / `run_matrix.py`；
+- 父 runner 的 salt、schema 与 DepthMixer 算术路径。
+
+明确禁止读取 Round 08 raw formal results、运行训练、修改文件、web search 与 subagents。
+
+## 2. 对正式 4×3 路径的确认
+
+审阅确认：
+
+- 新 protocol ID 没有进入父 `window_start` salt；
+- selected 路径用 constant-zero FP32 logits 和父 `softmax+einsum` kernel；
+- 非空 selector 的 query / key norm hook、AdamW state 与 final=initial gate 自洽；
+- historical pairing 比较的字段在父 JSON 中真实存在；
+- replay payload 正确排除 run kind、timing、path、parent self hash 与 GPU/version metadata，
+  同时保留模型、optimizer、diagnostics、history、selector 与 reachability；
+- analyzer 强制 4×3 identity、step 8,000 主判定、1-based layers 21–25、finite/positive、
+  `D=(ref-variant)/ref`、BPC `variant-ref` 与 descriptive `I67`；
+- 12 formal + 1 replay、65,536,000 bytes/cell 与最大两进程算术正确。
+
+没有发现会改变正在运行的四个非空 formal variants 数值语义的 blocking error。
+
+## 3. Blocking finding：smoke-only empty selector 的空审计
+
+`learned_reference` 直接调用父 `forward`，没有递增 wrapper 的 visit counters。因此：
+
+```text
+forward_calls = 0
+all depth/output visits = 0
+visit_gate = all(0 == 0) = true
+selected reachability = all([]) = true
+```
+
+这不会影响四个正式变体，它们全部是非空 selector；而 empty selector 另有 20-step
+parent-equivalence gate，模型/optimizer/evaluations/diagnostics/history/hash 已 exact。
+但 `forward_intervention.passed` 本身不能在修复前被当作 learned census 证据。
+
+处置：
+
+- formal matrix 完成后，删除 `selected_indices` 为空时的父路径 early return；
+- 让 empty selector 也走同一 copied forward，其中 `mix` 对 64 个节点逐个调用父
+  `DepthMixer.forward`；
+- 加入 `forward_calls > 0`；
+- 重跑 step-0、parent smoke、wrapper smoke 与 parent equivalence；
+- 只有 copied path 仍逐字段 exact 才保留。
+
+这项修复只强化 smoke audit，不更改任何正式 variant 的 selector 或 forward。
+
+## 4. Non-blocking findings 与处置
+
+| finding | 处置 |
+|---|---|
+| `smoke_compare` 整体比较 `manifest.path` | 规范化 path 后比较 scientific manifest fields |
+| unselected gate 额外要求 optimizer state | 保留为强实现 gate，但在文档中标成 protocol 之外的额外审计，不用于科学 status |
+| GPU/version 差异只有 `metadata_equal`，没有 warnings 数组 | aggregate 增加显式 metadata warnings |
+| `run_matrix` 失败后只 terminate、不 wait/kill，invalid file 会阻塞重跑 | 加入 terminate→wait→kill 清理，并在 cell failure 时标明 exact invalid target |
+| step-0 CE 只比较 Python float | 追加 scalar tensor SHA-256 |
+| `EXPECTED_SOURCE_COUNTS` 未使用 | 用于 runner↔manifest 交叉校验 |
+
+上述修订不读取结果、不改变冻结阈值或主公式。
+
+## 5. 矩阵结束后的处置结果
+
+13 个单元全部退出后才应用上述修订；四个正式非空 selector 的 raw 文件未被重写。
+修补后的前置闸门结果：
+
+- step-zero 五个 variant 的 logits、loss tensor、evaluation 与 diagnostic exact；
+- smoke-only learned wrapper 实际执行 `39` 次 forward，64 个 depth mixer 与
+  output mixer 的 visit census 全部非零且 exact；
+- parent 与 wrapper 的 architecture、seed、schedule、model、optimizer、hash、
+  evaluation、diagnostic、history、gradient gate 与 scientific environment 共
+  15 组字段全部 exact；
+- primary smoke 的 selected 参数仍为 0 hook、无 optimizer state、final=initial；
+- unselected optimizer-state 条件继续作为额外实现闸门，不进入科学 status。
+
+结果后 Grok 会话 `019fb28b-a9e1-7643-8e43-06f5e16a2077` 在只读 sandbox 中独立
+复算 13 个 raw、三个 historical references 与 aggregate，报告
+`blocking_errors=0`、`status_confirmed=true`、`replay_confirmed=true`。详细数字与
+claim boundary 见 `research/K3_ATTNRES_FORWARD_TRAINING_AUDIT.md`。

research/K3_ATTNRES_FORWARD_TRAINING_PROTOCOL.md

@@ -0,0 +1,389 @@
+# K3 Attention Residuals 训练期前向干预协议
+
+协议 ID：`llm-atlas-k3-attnres-forward-training-v1`  
+冻结日期：2026-07-30  
+协议状态：**结果前预注册 frozen；任何语义变更必须更换 protocol ID**  
+父协议：`llm-atlas-k3-attnres-gradient-scale-v1`
+
+## 0. 研究身份
+
+这是 Round 07 定向线索之后的训练期架构消融。选中 depth mixer 在每一次 train / eval /
+diagnostic forward 都用 source states 的算术平均，完全绕过该 mixer 的
+`query + key_norm + softmax` 路径。
+
+允许回答：
+
+1. 固定 groups 6+7 的 uniform forward 训练变体，能否在不触发预注册 BPC 失败护栏时，
+   material 地降低最终固定 activation-gradient spike？
+2. group 6、group 7 与 joint 的训练轨迹呈现什么非加性关系？
+3. Round 07 指向的 group 7 MLP-only 路径能否独立产生 material response？
+
+不允许回答：
+
+- 真实 Kimi K3 2.8T checkpoint 的梯度或训练动力学；
+- 论文 Figure 5(c) 未公开 telemetry 的复现；
+- “forward effect” 与 natural backward/update effect 的分离；
+- selected query/key 参数如果继续训练会怎样；
+- 三 seed 外的总体显著性、置信区间或 p-value；
+- 下游能力保持、通用质量等价或最优 AttnRes 设计；
+- 单组 effects 的可加性、Shapley value、方差贡献或因果交互；
+- 与 Round 07 value-coefficient intervention 同构的“纯 forward”因果复制；
+- K3 `A_log` 的官方修复裁决。
+
+## 1. 冻结训练与数据合同
+
+| 字段 | 固定值 |
+|---|---|
+| architecture | Block AttnRes |
+| Transformer depth | 32 |
+| aggregation groups | 8 |
+| blocks / group | 4 |
+| depth / output mixers | 64 / 1 |
+| width / heads / FFN | 192 / 6 / 768 |
+| context / vocabulary | 256 / byte-256 |
+| seeds | 2026073001 / 2026073002 / 2026073003 |
+| steps / batch | 8,000 / 32 |
+| target bytes / new formal cell | 65,536,000 |
+| optimizer | AdamW |
+| peak / min LR | 3e-4 / 3e-5 |
+| warmup | 400 |
+| weight decay | 0.1 for ndim ≥ 2 |
+| betas / epsilon | 0.9, 0.95 / 1e-8 |
+| clip | global norm 1.0 |
+| forward | CUDA BF16 autocast |
+| residual accumulation | explicit FP32 |
+| validation | fixed 64 × 256-byte windows |
+| diagnostic | fixed 16 × 256-byte windows |
+| checkpoints | 0 / 100 / 500 / 2,000 / 4,000 / 8,000 |
+| concurrency | at most two independent processes |
+
+训练输入 schedule **逐 step 复用父协议**。`window_start` 使用父
+`llm-atlas-k3-attnres-gradient-scale-v1` 的 salt；新 protocol ID 只写入 wrapper output
+和 study manifest，绝不能进入 `round04.PROTOCOL_ID` 或训练窗口散列。父 manifest
+负责 bytes / windows / schedule，新 manifest 只负责 variants / selector / thresholds。
+每个 variant / seed 的初始化、optimizer input、validation 与 diagnostic tensors 必须
+exact 相同。
+
+runner 必须继承 Round 05 `GradientLanguageModel` 的 explicit FP32 Block residual
+accumulation；不得退回 Round 04 的旧累加路径。
+
+新正式处理量：
+
+```text
+4 variants × 3 seeds × 65,536,000 = 786,432,000 target bytes
+1 primary replay                         65,536,000 target bytes
+total newly processed                   851,968,000 target bytes
+historical learned reference            196,608,000 target bytes（不重跑）
+```
+
+每格必须使用全新 Python process。最多并行两个；不能共享 model、optimizer、RNG、
+CUDA graph 或 output file。
+
+## 2. 冻结正式矩阵
+
+正式 variants：
+
+| variant | exact selected depth indices | layers / branch |
+|---|---|---|
+| `uniform_group_6_forward` | 40–47 | 21–24 / both |
+| `uniform_group_7_forward` | 48–55 | 25–28 / both |
+| `uniform_groups_6_7_forward` | 40–55 | 21–28 / both |
+| `uniform_group_7_mlp_forward` | 49, 51, 53, 55 | 25–28 / MLP |
+
+`learned_reference` 只允许 smoke，不进入新正式矩阵。output mixer index 64 永远 learned。
+
+正式运行 12 格；另从初始化 replay：
+
+```text
+replay / uniform_groups_6_7_forward / seed 2026073001
+```
+
+## 3. 唯一 selector 与 forward 语义
+
+runner 必须只有一个 machine-readable selector：
+
+```text
+selected(variant, depth_mixer_index) -> bool
+```
+
+不得把四个 variant 分叉成四份 model forward。
+
+对未选中 depth mixer 和 output mixer，逐调用父 `DepthMixer.forward`。对选中 mixer，
+为了让初始化负控制复用相同浮点归约顺序，使用参数无关的零 logits，但仍走父
+`softmax + einsum` 数值 kernel：
+
+```text
+values = stack(sources, dim=0)
+logits = zeros([N, batch, tokens], dtype=FP32)
+weights = softmax(logits, dim=0)
+output = einsum("nbt,nbtd->btd", weights, values.float()).to(values.dtype)
+```
+
+capture summary 必须仍使用父 schema：
+
+```text
+mean_weights = [1 / N] × N
+entropy_mean = ln(N)
+sources      = N
+```
+
+选中路径不得调用 `query`、`key_norm` 或 source-dependent logits，也不得用
+stop-gradient trick 让这些参数看似参与。这里保留的 softmax 只把常数零 logits 变成
+`1/N`，目的是与父路径保持同一 arithmetic kernel；它没有可训练参数。自然结果是选中
+mixer 的 `query` 和 `key_norm.weight`：
+
+- gradient hook call count 必须为 0；
+- optimizer state entry 必须不存在；
+- final tensor 必须与 initial tensor byte-exact。
+
+所有未选中 depth mixer 和 output mixer 的两个参数都必须有正的 gradient hook call
+count；这只证明图可达，不要求它们的梯度非零或最终 tensor 一定变化。hook census
+从 model 构造后开始，覆盖所有 training backward 与 diagnostic backward；eval
+forward 不计 hook。selected 参数允许保留在原 AdamW param groups，但 state entry 必须
+不存在，不能表述为“训练了但没有移动”。
+
+## 4. 结果前实现闸门
+
+### 4.1 empty-selector 父等价
+
+`learned_reference` smoke 必须直接走父 forward，不得走常数 uniform 分支；它与父
+Round 05 runner 在相同 seed / 20 steps 下必须：
+
+- initial/final model hashes exact；
+- final optimizer hash exact；
+- evaluations、diagnostics 与 training history exact；
+- gradient gate exact；
+- input tensor hashes exact。
+
+允许不同字段只限 protocol wrapper identity、study-manifest wrapper、timing、规范化
+后的 manifest path、output path 与 self-hash。AdamW 参数分组必须保持父语义；
+empty-selector 不能删除任何 mixer 参数。
+
+### 4.2 step-0 identity negative control
+
+父模型所有 depth mixer query 初始化为 0，因此 learned softmax 在 step 0 是 exact
+uniform。选中分支用同 dtype 的 constant-zero logits 和同一个
+`softmax + einsum` kernel。四个 variant 与 learned reference 在固定 input 上必须：
+
+- logits byte-exact；
+- CE byte-exact；
+- activation-gradient spectrum byte-exact；
+- validation metrics byte-exact；
+- selected 的**归约前 weight tensor** 等于 FP32 `1/N`，max absolute error
+  `≤ 1e-12`；capture 的 `mean_weights` 因 FP32 大规模 mean 可有约 `1e-8` 的归约舍入，
+  但必须与父 learned capture summary byte-exact；
+- 若任何跨 variant byte-exact 比较失败，hard-fail；不得在结果后改成容差 gate。
+
+这个负控制只约束初始化；训练开始后 forward 必须允许分化。
+
+### 4.3 selector census
+
+每个 forward 的 64 个 depth index 必须各访问一次，output 访问一次且保持 learned。
+每个 variant 的 selected set 必须与第 2 节 exact。group 内 source counts 必须满足：
+
+```text
+group 6: index 40 has N=6; indices 41..47 have N=7
+group 7: index 48 has N=7; indices 49..55 have N=8
+```
+
+推导前提是 `completed` 含 embedding，且每 8 个 depth mixer 才把 `partial` 聚合为一个
+completed group。
+
+正式 output 保存 exact selected indices、实际 visit census、source counts 与
+uniform-weight max error。任何漏访、重访、越界或 output 被选中都失败。
+
+### 4.4 数据、有限性与梯度尺度
+
+- 父 manifest、train/validation/diagnostic bytes 与 schedule hashes exact；
+- step 0 / 1 / 7,999 optimizer input gate hashes exact；
+- loss、logits、所有主 activation gradients 全部 finite；
+- global clip 后每一步都执行 optimizer update，不允许 skip；
+- smoke 的 diagnostic loss `×2` 时，每层 activation-gradient RMS 比值在
+  `2 ± 1e-5`，normalized spectrum max delta `≤ 1e-6`。
+
+## 5. 历史 reference 配对合同
+
+reference 固定为：
+
+```text
+experiments/k3/attnres_gradient/results/raw/
+  formal-depth-32-block-seed-{seed}.json
+```
+
+analyzer 的 pairing hard gates：
+
+- `protocol_id = llm-atlas-k3-attnres-gradient-scale-v1`；
+- formal / block / depth 32 / 8,000 steps / batch 32；
+- seed exact；
+- initial public 与 mixer hashes 跟对应新 variant exact；
+- formal schedule、validation tensor、diagnostic tensor 和三个 input gate hashes exact；
+- model topology、optimizer hyperparameters、CUBLAS workspace、deterministic flags 与
+  autocast 语义 exact。
+
+以下字段明确**不参与 pairing equality**：
+
+- 所有 final hashes、evaluations、diagnostics、training history 与 gradient gate；
+- timing、run kind、self-hash、output path 与 manifest path 字符串；
+- GPU 名称、driver / CUDA / torch version 的 minor 差异。
+
+环境完整记录；若数值栈变化，aggregate 给出 metadata warning，但只要上述确定性与
+autocast 合同相同就不将其误判为 pairing failure。父 JSON 没有 initial full-state 字段，
+不得假定它存在；public ∪ mixer 的完整性只用结构/元素 census 自洽。
+
+reference 是历史配对基线，不得写成同期随机对照。若任何合同不等，整轮 aggregate
+失败，而不是降级为“近似比较”。
+
+## 6. 固定主对象与公式
+
+每个 diagnostic checkpoint 从：
+
+```text
+activation_grad_rms_by_block = [g1, ..., g32]
+layer_id ∈ {1,...,32}
+g[layer_id] = activation_grad_rms_by_block[layer_id - 1]
+S = {21,22,23,24,25}  # 1-based
+R = {1,...,32} \ S
+```
+
+计算：
+
+```text
+C = mean(g[S]) / mean(g[R])  # spike contrast
+P = max(g) / mean(g)         # peak normalized
+```
+
+所有 `g`、`C`、`P` 必须 finite 且严格大于 `1e-30`。groups 6+7 还改写落在 `R`
+中的 layers 26–28，所以 analyzer 同时报告 `mean(g[S])` 与 `mean(g[R])`，但不把它们
+加入主 status。
+
+对同 seed reference `X_ref` 与 variant `X_v`：
+
+```text
+D_X(v) = (X_ref - X_v) / X_ref
+```
+
+`D>0` 表示 attenuation，`D<0` 表示 amplification。不得取绝对值，不得更换分母。
+
+## 7. 预注册判定
+
+### 7.1 主判定
+
+`uniform_groups_6_7_forward` 的主 attenuation gate **只读取 step=8,000**：
+
+```text
+D_C >= 0.20 AND D_P >= 0.20
+for all 3 seeds
+```
+
+质量 gate 也只读取 `evaluations[step=8000].bits_per_byte`：
+
+```text
+delta_bpc(seed) = final_bpc_variant - final_bpc_reference
+delta_bpc(seed) <= 0.05 for all 3 seeds
+mean(delta_bpc) <= 0.03
+```
+
+只有 attenuation 6/6 与 quality 4/4 同时通过，正式 status 才是：
+
+```text
+forward_training_attenuation_established_within_reduced_protocol
+```
+
+否则按失败位置使用：
+
+```text
+attenuation_not_established
+quality_guard_failed
+attenuation_and_quality_failed
+```
+
+不能用次级变体补救主判定。任何 step 8,000 缺失/重复、数组长度错误、hash 不配对、
+selector / reachability 失败、`g/C/P/BPC/D/log` 缺失或非 finite 都是
+`contract_failed` 并让 analyzer non-zero exit；不能把结构失败包装成上面的科学状态。
+
+### 7.2 次级 material response
+
+group 6、group 7、group 7 MLP-only 各自使用同一 `20% / 3-seed / 2-metric` attenuation
+threshold 和同一 quality guard，分别报告：
+
+```text
+secondary_material_response / secondary_response_not_established
+```
+
+它们不改变主 status，也不升级成 localization。
+
+### 7.3 BPC 护栏的解释
+
+`+0.05 per seed / +0.03 mean` 是预注册的 catastrophic-degradation screen：
+
+- 失败说明不能把 spike 下降当成健康训练的证据；
+- 通过不说明能力、校准或下游任务等价；
+- BPC 改善也不说明总体架构更优。
+
+## 8. 非加性交互与轨迹
+
+对 `X ∈ {C,P}`、每个 seed、每个 checkpoint：
+
+```text
+E6  = ln(X_ref / X_group6)
+E7  = ln(X_ref / X_group7)
+E67 = ln(X_ref / X_groups6+7)
+I67 = E67 - E6 - E7
+```
+
+保存 `E6/E7/E67/I67` 原值、对应 `D_C/D_P` 和三 seed mean/range。没有通过阈值、
+p-value 或 CI。它是三套独立训练在相同 checkpoint 的跨-run log residual；
+`I67` 不能写成可加贡献、独立作用、Shapley value 或因果 interaction estimate。
+
+同时全量保存：
+
+- 六 checkpoints 的 32-layer raw / normalized spectra；
+- peak layer、top-five layers；
+- validation BPC 与 train-loss trajectory；
+- selected/unselected mixer weight summaries；
+- selected-parameter reachability audit；
+- per-cell timing 与显存（不进入数值结论）。
+
+## 9. replay 与 analyzer 合同
+
+primary seed-1 replay 使用 analyzer 定义的 scientific canonical payload。先删除：
+
+```text
+run_kind
+timing
+canonical_sha256_without_self
+manifest.path
+study_manifest.path
+```
+
+再比较以下固定字段 exact：protocol / variant / architecture / depth / seed / steps / batch /
+target bytes、manifest scientific hashes、model、optimizer、initial/final hashes、
+evaluations、diagnostics、training history、selector 与 reachability audits，以及
+environment 中 deterministic / autocast scientific subset。GPU/版本 metadata 保留在
+两份文件中单独展示，不进入 canonical equality。
+
+所有主指标、阈值、status、interaction map 与 compact website artifact 只能由单一
+`experiments/k3/attnres_forward/analyze.py` 生成。网站不能在 TypeScript 中重新计算
+另一套结论。
+
+analyzer 在任何结构、hash、selector、reachability、finite、reference pairing、
+replay 或 threshold contract 失败时必须 non-zero exit，不得输出部分通过结论。
+
+## 10. 报告语言红线
+
+允许：
+
+- “在这个固定缩小模型与训练协议内，局部 uniform-forward 变体……”
+- “selected mixer 参数在此架构消融中结构性不可达……”
+- “joint log effect 呈现正/负 interaction residual……”
+
+禁止：
+
+- “证明 K3 的训练尖峰来自 group 6/7”
+- “只改变 forward，所以这是纯 forward 因果效应”
+- “BPC gate 通过，所以能力不受影响”
+- “interaction residual 是两个 group 的真实贡献”
+- “contrast 下降证明尖峰层本身下降”（未同时检查 `S` / `R` 分拆）
+- “step-0 exact 说明训练期始终与 learned forward 恒等”
+- “复现了 K3 Figure 5(c)”
+- “已经验证官方 2.8T checkpoint”

research/K3_ATTNRES_FORWARD_TRAINING_SCOPING.md

@@ -0,0 +1,195 @@
+# K3 Attention Residuals 训练期前向干预：Round 08 前置定位
+
+研究日期：2026-07-30  
+阶段身份：**Round 07 后的定向 scoping，不是 Round 08 结果**  
+上游协议：`llm-atlas-k3-attnres-local-path-v1`
+
+## 1. 为什么还需要一次训练期实验
+
+Round 06 / 07 都保持 learned forward 完全不变，只在固定 diagnostic 的 backward 中
+替换 source value coefficients。它们回答的是：
+
+- 全部 65 个 mixer 的 uniform value backward 能不能压低固定尖峰；
+- group 6 / 7 的 16 个 depth mixer 在 learned 背景上是否足以复现全局下降；
+- 从 all-uniform 背景恢复这些 mixer 是否能反向恢复尖峰。
+
+Round 07 的正式结论是：
+
+```text
+groups 6+7 sufficiency：6 / 6 seed×metric cells 通过
+groups 6+7 restoration：3 / 6 cells 通过
+formal status：one_sided_evidence_localization_not_established
+```
+
+这已经足以排除“局部 mask 完全没有反应”，但还不能回答：
+
+> 如果训练的每一次 forward 都真的把这段 depth routing 改成算术平均，模型会怎样适应？
+
+Round 08 把 intervention 放进 optimizer path。它不再追求 forward-identical，而是让
+选中 mixer 的输出在训练、验证与诊断中始终为所有 source states 的等权平均。
+
+## 2. 这不是“只改变 forward”
+
+选中 mixer 的 learned 路径原本是：
+
+```text
+keys    = RMSNorm(sources)
+logits  = query · keys
+weights = softmax(logits over source-depth)
+output  = Σ weights_i × source_i
+```
+
+Round 08 的选中路径是：
+
+```text
+output = (1 / N) × Σ source_i
+```
+
+因此 intervention 同时改变：
+
+1. forward 的 branch input；
+2. 由新 forward 自然产生的 source gradients；
+3. 下游 activation、loss 与所有后续 optimizer updates；
+4. 选中 mixer 的参数可达性：`query` 与 `key_norm.weight` 不参与图，不得到梯度。
+
+这是一项**训练期架构消融**，不是“只改变 forward、不改变 backward”的可分离因果实验。
+结果不能被翻译成 query/key 路径的纯因果效应。
+
+## 3. 为什么只选四个新变体
+
+固定 depth-32 / Block AttnRes 拓扑：
+
+| scope | layers | 0-based depth mixer indices | count |
+|---|---:|---:|---:|
+| group 6 | 21–24 | 40–47 | 8 |
+| group 7 | 25–28 | 48–55 | 8 |
+| groups 6+7 | 21–28 | 40–55 | 16 |
+| group 7 MLP | 25–28 | 49, 51, 53, 55 | 4 |
+
+四个新训练变体固定为：
+
+1. `uniform_group_6_forward`
+2. `uniform_group_7_forward`
+3. `uniform_groups_6_7_forward`（主变体）
+4. `uniform_group_7_mlp_forward`
+
+选择依据不是 Round 08 结果：
+
+- joint 6+7 是 Round 07 的固定主 scope；
+- 单 group 6 / 7 用来构成交互图；
+- group 7 MLP-only 是 Round 07 的 branch-level 次级线索；
+- output mixer 保持 learned，避免把 local depth intervention 扩成全局 readout 改写。
+
+不加入 attention-only、output-only、all-depth 或 all-65，是为了把本轮正式算力集中在
+“局部训练适应”而不是重新展开 14-mode backward census。
+
+## 4. 冻结参考，而不是重新挑基线
+
+paired learned reference 复用 Round 05 已公开的三个：
+
+```text
+formal / depth-32 / block / seeds 2026073001..3
+```
+
+它们与新变体使用相同：
+
+- 初始化过程；
+- 8,000-step deterministic byte schedule；
+- 64 validation windows；
+- 16 diagnostic windows；
+- optimizer、LR schedule、BF16 forward 和 FP32 residual accumulation；
+- 六个 diagnostic checkpoints。
+
+runner 的 `learned_reference` smoke mode 必须逐字段复现父 runner 数值。正式 analyzer
+还必须验证新旧 cell 的 initial public/mixer hashes、输入 gate tensor hashes、
+validation tensor hash、diagnostic tensor hash 与 deterministic / autocast 数值合同。
+GPU 名称、驱动 minor 与 timing 只记录，不作为配对 hard gate。
+
+复用参考的意义是避免为已冻结、可复现的三格再消耗 196,608,000 target bytes。报告必须
+把它写成**历史配对参考**，不能写成同期随机对照。
+
+## 5. 主问题与质量护栏
+
+主对象沿用 Round 05–07，并且主判定只读取最终 step 8,000：
+
+```text
+step      = 8,000
+position  = post-MLP Transformer-block output
+reduction = activation-gradient element RMS
+S         = layers 21–25
+R         = other 27 layers
+C         = mean(g[S]) / mean(g[R])
+P         = max(g) / mean(g)
+```
+
+对变体 `v` 和同 seed learned reference：
+
+```text
+D_X(v) = (X_ref - X_v) / X_ref
+```
+
+主变体 `uniform_groups_6_7_forward` 只有在：
+
+- `C` 与 `P` 的 `D_X ≥ 0.20`；
+- 三个 seed 的六个 cell 全部通过；
+- 每个 seed 的 final validation BPC 相对 reference 不劣化超过 `+0.05`；
+- 三 seed mean BPC delta 不超过 `+0.03`；
+
+时才得到 “forward-training attenuation established within reduced protocol”。
+
+质量阈值在结果前固定。它不是“能力保持”的通用标准，只用于阻止把明显训练失败造成的
+梯度塌缩计成正结果。即使通过，也不能推出下游能力等价。
+
+## 6. 非加性局部交互图
+
+对每个 seed、checkpoint 和指标：
+
+```text
+E6  = ln(X_ref / X_group6)
+E7  = ln(X_ref / X_group7)
+E67 = ln(X_ref / X_groups6+7)
+I67 = E67 - E6 - E7
+```
+
+- `I67 > 0`：joint log attenuation 大于两个 single effects 的和；
+- `I67 < 0`：joint log attenuation 小于两个 single effects 的和；
+- `I67 = 0`：只是在这个定义下恰好 log-additive。
+
+`I67` 没有预注册显著性阈值，不是 Shapley value、方差分解、独立性检验或因果交互估计。
+三个 effect 来自三套独立训练，它只是跨 run 的 log-attenuation residual。它的用途是把
+训练轨迹中的补偿/放大关系画清楚，而不是制造一个新的“通过/失败”结论。
+
+groups 6+7 覆盖 layers 21–28，而固定尖峰窗只到 layer 25；layers 26–28 落在 `R`。
+所以 `C` 的变化可能同时来自 `S` 下降与 `R` 上升。正式结果必须把两者拆开报告，不能把
+contrast 下降单独翻译成“尖峰层被关闭”。
+
+## 7. 真实 K3 checkpoint 的同期边界
+
+截至本轮预检，官方 Kimi-K3 Hugging Face main 仍停在 revision
+`9f62e4e9fffbd0a83ddd60e1c209d828994b3569`，remote code 仍按 96 heads 初始化
+`A_log`，公开 checkpoint header 仍为 `[128]`。社区 PR #144 / #150 仍是两个未合并、
+语义不同的候选修复；没有官方裁决。
+
+所以本轮不下载约 1.56 TB 权重，不声称对真实 K3 forward 做了验证。缩小实验只继承
+Block AttnRes 的拓扑动机，不是 K3 checkpoint 的数值替身。
+
+一手状态页：
+
+- [Kimi-K3 official main](https://huggingface.co/moonshotai/Kimi-K3/tree/main)
+- [main `modeling_kimi_linear.py`](https://huggingface.co/moonshotai/Kimi-K3/blob/main/modeling_kimi_linear.py)
+- [community PR #144](https://huggingface.co/moonshotai/Kimi-K3/discussions/144)
+- [community PR #150](https://huggingface.co/moonshotai/Kimi-K3/discussions/150)
+
+## 8. 本轮可证伪交付
+
+Round 08 将在查看正式结果前完成：
+
+1. 冻结协议与 machine-readable manifest；
+2. 实现一个 selector，而不是四份分叉 forward；
+3. 通过 empty-selector 父等价、step-0 uniform identity、selector census、参数不可达性、
+   loss-scale 与输入 hash 闸门；
+4. 运行 4 variants × 3 seeds × 8,000 steps；
+5. 从初始化 replay 主变体 seed 2026073001；
+6. 由单一 analyzer 生成主判定、质量闸门、轨迹与非加性交互；
+7. 独立审阅机器可读结果；
+8. 以五视图交互实验接入网站、开源并发布。

research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md

@@ -0,0 +1,198 @@
+# Kimi K3 第五轮前置审计：Attention Residuals 的“梯度更均匀”到底指什么
+
+> 审计日期：2026-07-30（Asia/Shanghai）  
+> 官方仓库：`MoonshotAI/Attention-Residuals@85e22310fe5ee860b4a023de312d791de8a5a5e6`  
+> 官方 PDF SHA-256：`e5831b0db1347606453b5176b0142115a18887b6a9c2e1d05a266d4805a26b2f`  
+> 结论性质：一手工件审计，不是新的实验结果
+
+## 0. 先说结论
+
+K3 Round 04 的反结果——Block AttnRes 的**核心参数梯度 RMS 跨层 CV 更高**——不能直接
+反驳 Attention Residuals 论文 Figure 5(c) 所说的“梯度分布更均匀”，因为两边很可能测的
+不是同一个对象：
+
+```text
+Round 04：
+每个 Transformer block 内所有核心参数梯度拼接后的 RMS
+∇θL，θ = attention + MLP + 两个输入 norm 的参数
+
+论文 Figure 5(c)：
+图题只写 “Each transformer block's gradient magnitude”
+结合 Figure 5(b) 的 block output magnitude 和正文，最自然的操作化是
+每个 block 输出 activation 的梯度 ∂L/∂h_l
+```
+
+但“最自然”不等于“官方已经明确定义”。论文和当前官方仓库都没有给出足以唯一重建
+Figure 5(c) 的测量合同，也没有发布训练代码。因此，下一轮不会把自己的 activation-gradient
+定义冒充成论文原始实现，而会把它命名为：
+
+> **与 Figure 5 叙述对齐的一种公开、冻结、可复现的 operationalization**
+
+这一区分很重要：参数梯度回答“这一层的权重此刻收到多大更新信号”，activation 梯度回答
+“损失对这一深度的表征有多敏感”。二者相关，但不会因为链式法则而自动同方向。
+
+---
+
+## 1. 官方工件实际提供了什么
+
+官方仓库在固定 revision 下只包含：
+
+- `README.md`；
+- `Attention_Residuals.pdf`；
+- 论文图片资产；
+- citation 与外部入口。
+
+仓库**不包含**：
+
+- 模型或 residual mixer 的可执行实现；
+- Figure 5 的统计脚本；
+- 训练配置、日志或 checkpoint；
+- gradient hook、norm 和 reduction 定义；
+- 用于复画 Figure 5 的原始数组。
+
+因此，本审计能固定论文的文字、公式、图与模型尺度，不能从官方代码恢复一个不存在的
+隐藏测量合同。
+
+## 2. Figure 5 能确认的事实
+
+官方 `training_dynamics.png` 和 PDF Figure 5 有三个并列面板：
+
+| 面板 | 图题 | 横轴 |
+|---|---|---|
+| (a) | Validation Loss | training step |
+| (b) | Output Magnitude | Layer / Transformer Block Index |
+| (c) | Gradient Magnitude ×10⁻⁵ | Layer / Transformer Block Index |
+
+Figure 5 caption 对 (b) 与 (c) 的完整对象描述分别是：
+
+```text
+Each transformer block's output magnitude at the end of training.
+Each transformer block's gradient magnitude.
+```
+
+正文明确表达了两个方向：
+
+1. Baseline 的 hidden-state magnitude 随深度单调增长；Block AttnRes 把增长约束在局部
+   residual block 内，形成周期性的深度图案。
+2. Baseline 的早期层梯度“不成比例地大”；Block AttnRes 的可学习 softmax 权重产生了
+   “明显更均匀”的梯度分布。
+
+图中最终大模型约有 27 个 Transformer blocks。论文的最终模型描述也是 27 个 Transformer
+blocks / 54 个 residual layers；Block AttnRes 每 6 个 residual layers 聚合一次，共 9 个
+聚合块，外加 embedding 形成 10 个跨块来源。
+
+## 3. Figure 5 不能确认的事项
+
+下面每一项都会改变曲线，却没有在论文或官方仓库中被唯一指定：
+
+| 未定义项 | 至少两种合理解释 |
+|---|---|
+| 梯度对象 | block 输出 activation 梯度；block 参数梯度；分支输出梯度 |
+| block 输出位置 | attention+MLP 后；只在 MLP 后；进入下一层 norm 前；聚合块边界后 |
+| norm | L2 norm；RMS；mean absolute value；每 token norm 后再平均 |
+| reduction | batch/token/channel 联合；先按 token 再按 batch；只取末 token |
+| loss | token mean；sample mean；未归一化 sum；带或不带 mask |
+| 采样 | 一个 batch；多 batch 平均；训练流中的 moving average |
+| 时间点 | “训练结束”单点；末段平均；某个 checkpoint |
+| 数值阶段 | AMP 缩放前/后；gradient clipping 前/后；BF16 或 FP32 |
+| 运行模式 | train 或 eval；dropout 是否开启 |
+| 归一化 | 绝对值；再除全层均值；再除 Baseline |
+
+Figure 5(c) 的纵轴是绝对 magnitude 标度，并不等价于 CV。只报告 CV 还会丢失两个信息：
+
+- 全部层梯度是否一起缩小或放大；
+- 不均匀来自“早层系统性偏大”，还是某一个中间/末端尖峰。
+
+所以 Round 05 必须同时公开绝对曲线、按层均值归一化曲线、CV 与前后深度分位比。
+
+## 4. 为什么 activation gradient 是合理推断，但仍只是推断
+
+把 Figure 5(c) 操作化为 `∂L/∂h_l` 有三条证据：
+
+1. 它与 Figure 5(b) 的 “transformer block output magnitude” 在横轴和叙述上成对；
+2. “早期层的梯度”在表示传播语境中通常可由对 block output 保留梯度直接比较；
+3. 参数张量的大小和类型在 attention 与 MLP 间差异很大，若把参数拼接，论文通常需要说明
+   聚合口径，否则 “each transformer block” 不是天然的单一标量。
+
+但也有无法排除的替代解释：
+
+- 论文作者可能测 block 参数梯度；
+- 可能测 residual branch output 而非完整 block output；
+- 可能先对每个 token 做 L2 norm，再跨 token 平均；
+- 可能在内部训练系统中有未公开的统一 telemetry 定义。
+
+因此，网站和审计只说“与论文叙述对齐的公开定义”，不说“论文就是这样算的”，也不把
+数值和 Figure 5 纵轴直接对齐。
+
+## 5. Round 04 与 Round 05 的对象对照
+
+| 维度 | Round 04 已测对象 | Round 05 主对象 |
+|---|---|---|
+| 数学对象 | `∇θ_l L` | `∂L/∂h_l` |
+| `l` 的单位 | Transformer block | Transformer block |
+| 张量内容 | block 的核心参数 | block 的 post-MLP output activation |
+| 聚合 | 参数元素联合 RMS | batch×time×channel 联合 RMS |
+| 是否含 AttnRes 参数 | 否 | 不适用；梯度穿过 mixer |
+| 时间 | final diagnostic batch | 全部预注册 diagnostic steps |
+| 目的 | 权重更新信号是否均匀 | 表征深度的反向信号是否均匀 |
+
+Round 04 的参数梯度结果不会被改名、删去或用新指标覆盖。它仍是一个有效反结果，只是不能
+代表论文未定义清楚的 Figure 5(c)。
+
+## 6. 两种结构怎样取得真正对齐的 16 / 32 个位置
+
+Grok Headless 被用作一次对抗式方法审阅，不作为事实来源。它正确指出了 non-leaf tensor、
+alias、AMP、clip 时点和只看 CV 的风险；但它也提出了一个不适用于本实现的担忧：
+“Block AttnRes 只有约 8 个 block 输出，无法与 Baseline 的 16 / 32 层对齐”。
+
+这里要区分两种 block：
+
+```text
+Transformer block：
+attention + MLP；depth=16 时始终有 16 个，depth=32 时始终有 32 个
+
+AttnRes aggregation group：
+把若干 residual sublayers 的 partial sum 保存为一个跨组 source；
+两种深度都约为 8 组
+```
+
+Round 05 在**每个 Transformer block 的 MLP 分支完成后**取 `h_l`。所以 Baseline 与 Block
+都有完全相同的 `l=1..depth`：
+
+| 架构 | Round 05 的 `h_l` | shape |
+|---|---|---|
+| Baseline | 第 `l` 个 attention residual 与 MLP residual 都完成后的 hidden state | `[B,T,C]` |
+| Block | 第 `l` 个 MLP branch 加入后、可能保存并清空 aggregation partial **之前**的 partial output | `[B,T,C]` |
+
+Block 的 `h_l` 是局部 residual partial，而不是跨组 source 列表；这正对应论文 Figure 5(b)
+所描述的“增长被限制在每个 block 内”的周期性图案。组边界前取值也避免把 reset 后的零张量
+错误当成 Transformer block output。
+
+## 7. 实验实现必须通过的梯度测量闸门
+
+正式训练前，四个结构格（2 个深度 × 2 个 residual graph）都必须证明：
+
+1. 所有 `h_l` 都是不同的、非别名的捕获对象，数量严格等于 Transformer depth；
+2. `retain_grad()` 或 hook 后所有梯度非 `None`、finite，shape 与 activation 完全一致；
+3. diagnostic loss 使用固定输入、固定 token-mean CE、`eval()`、无 optimizer step；
+4. backward 发生在 parameter gradient clip 之前，且不经过 `GradScaler`；
+5. 将同一 diagnostic loss 精确乘 2 后，每层 activation-gradient RMS 也乘 2；
+6. 乘 2 前后的 CV、归一化曲线与深度分位比在数值容差内不变；
+7. 同配置全新进程重复运行，冻结诊断字段 exact。
+
+若任一项失败，正式 8,000-step grid 不得开始。
+
+## 8. 本审计带来的研究决策
+
+下一轮不再把一个宽度 192、深度 16、训练 2,000 step 的参数梯度 CV 与论文最终模型图强行
+放在同一条结论线上，而是：
+
+- 增加 depth 32；
+- 把训练预算扩为 8,000 step；
+- Baseline / Block 使用相同的 Transformer block index；
+- 在 6 个固定时点测 post-MLP activation gradient；
+- 绝对标度、归一化形状、CV、前后四分位失衡一起公开；
+- 参数梯度作为次要指标保留；
+- 预注册“支持 / 混合 / 不支持”规则，并公开全部反结果。
+
+精确协议见 `research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md`。

research/K3_ATTNRES_GRADIENT_SCALE_AUDIT.md

@@ -0,0 +1,545 @@
+# Kimi K3 第五轮：Attention Residuals 梯度定义与深度扩展实验审计
+
+> 协议：`llm-atlas-k3-attnres-gradient-scale-v1`  
+> 前置定义审计：`research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md`  
+> 预注册协议：`research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md`  
+> 数据清单：`experiments/k3/attnres_gradient/manifest.json`  
+> 执行日期：2026-07-30  
+> 设备：NVIDIA GeForce RTX 5090；PyTorch `2.11.0+cu128`
+
+## 0. 先说结论
+
+这一轮本来想澄清一个看似矛盾的问题：
+
+```text
+论文 Figure 5：
+Block AttnRes 的梯度沿深度“明显更均匀”
+
+本站 Round 04：
+Block AttnRes 的核心参数梯度 RMS 跨层 CV 反而更高
+```
+
+一手工件审计先确认：论文没有公开 Figure 5(c) 的确切 gradient tensor、norm、reduction、
+diagnostic batch、AMP / clip 时点或统计代码。因此，Round 05 没有假装恢复作者的隐藏实现，
+而是冻结一个与 Figure 5 的 output/gradient 并列叙述对齐、可复现的定义：
+
+```text
+h_l：
+第 l 个 Transformer block 完成 attention + MLP 后的 FP32 residual output
+
+m_l：
+sqrt(mean((∂L / ∂h_l)² over batch × time × channel))
+
+L：
+固定 16 × 256 targets 的 token-mean cross entropy
+```
+
+结果不是一句“是”或“不是”，而是一个更有信息量的分解：
+
+1. **Block 确实大幅缓解了早层整体偏大。**  
+   Baseline 首四分位梯度平均是末四分位的 `3.01–3.90×`；Block 把它改到
+   `0.55–1.40×`。预注册的首尾失衡指标在 6 / 6 个 depth×seed 配对中都改善，
+   depth-16 平均改善 `61.0%`，depth-32 平均改善 `72.0%`。
+
+2. **但 Block 没有让整条深度谱更平。**  
+   它在中后段形成了局部尖峰，所以 population CV 在 6 / 6 个配对中都恶化：
+   depth-16 平均相对恶化 `10.3%`，depth-32 平均相对恶化 `60.0%`。
+
+3. **Block 的绝对 activation-gradient 平均尺度更小。**  
+   它只有 Baseline 的 `57.4%`（depth-16）和 `54.4%`（depth-32）。这意味着“首尾更接近”
+   不能自动解释为所有层都获得更强更新信号。
+
+4. **参数梯度仍与 Round 04 同方向。**  
+   核心参数梯度 CV 从 `0.416→0.683`（depth-16），从 `0.397→0.772`
+   （depth-32）；Block 更不均匀。
+
+5. **验证 BPC 在 6 / 6 配对中都更低，但不是同算力优势。**  
+   平均改善 `0.00894 BPC`（depth-16）与 `0.00987 BPC`（depth-32）；Block 实际 step
+   time 是 Baseline 的约 `2.55–2.60×`，peak allocated memory 约 `2.15–2.16×`。
+
+按看结果前冻结的联合判据，CV 与首尾失衡必须同时改善才算 support；必须同时恶化才算
+concern。这里二者方向相反，所以：
+
+| depth | 预注册判定 |
+|---:|---|
+| 16 | **mixed / inconclusive at this depth** |
+| 32 | **mixed / inconclusive at this depth** |
+| 总判定 | **depth-dependent or inconclusive** |
+
+最准确的中文总结是：
+
+> 在这套公开 operationalization 中，Block AttnRes 把“早层系统性偏大”变成了“首尾更接近、
+> 但中后段有局部尖峰”的另一种梯度分布。它修正了一类失衡，却没有降低全层离散度。
+
+这不复现论文 Figure 5 的数值，也不反驳一个没有公开测量合同的隐藏实现。
+
+---
+
+## 1. 为什么要另开一轮，而不是改写 Round 04
+
+Round 04 的梯度对象是：
+
+```text
+每个 Transformer block 的 attention、MLP 与两个 input norm
+所有核心参数梯度拼接后的 RMS
+```
+
+这回答“该层权重收到多大更新信号”。Round 05 的对象是 `∂L/∂h_l`，回答“损失对该深度
+表征有多敏感”。链式法则把两者联系起来，但不会保证跨层形状同方向。
+
+因此：
+
+- Round 04 参数梯度反结果继续有效；
+- Round 05 不把它改名为 activation gradient；
+- 两种对象在网站并排展示；
+- 任何方向冲突都保留，而不是选择更像论文的一种。
+
+官方工件边界见前置定义审计。固定 revision 为：
+
+```text
+MoonshotAI/Attention-Residuals
+85e22310fe5ee860b4a023de312d791de8a5a5e6
+
+Attention_Residuals.pdf SHA-256
+e5831b0db1347606453b5176b0142115a18887b6a9c2e1d05a266d4805a26b2f
+```
+
+官方仓库没有可执行训练代码或 Figure 5 原始数组。
+
+## 2. 实验规模与配对合同
+
+正式网格：
+
+```text
+2 depths
+× 2 residual graphs
+× 3 seeds
+× 8,000 steps
+× 32 windows
+× 256 target bytes
+= 12 independent runs
+= 786,432,000 formal target bytes
+```
+
+| 项 | depth-16 | depth-32 |
+|---|---:|---:|
+| Transformer blocks | 16 | 32 |
+| residual sublayers | 32 | 64 |
+| AttnRes aggregation groups | 8 | 8 |
+| sublayers / group | 4 | 8 |
+| Transformer blocks / group | 2 | 4 |
+| width / heads / FFN | 192 / 6 / 768 | 192 / 6 / 768 |
+
+每个 depth / seed 的 Baseline 与 Block：
+
+- 使用逐 tensor exact 的公共主干初始化；
+- 使用逐 step / row exact 的 byte windows；
+- 使用相同 optimizer、LR schedule、batch、context 与 target-byte budget；
+- 分支线性计算走 BF16 autocast；
+- residual accumulator 与被测 `h_l` 都为 FP32；
+- 每个结构在全新进程中从零训练。
+
+二者**不匹配**：
+
+- mixer 参数；
+- mixer FLOPs；
+- step wall time；
+- activation memory。
+
+所以 BPC 只能叫“同 token / step 预算对比”，不能叫“同算力优势”。
+
+## 3. 数据与日程
+
+| 对象 | 固定值 |
+|---|---|
+| dataset | `Salesforce/wikitext` |
+| revision | `b08601e04326c79dfdd32d625aee71d232d685c3` |
+| variant | `wikitext-2-raw-v1` |
+| train bytes | 10,951,563 |
+| train SHA-256 | `0ca7d3e7…e9b4` |
+| validation bytes | 1,148,008 |
+| validation SHA-256 | `a42356f6…e719` |
+| formal schedule cells | 768,000 |
+| schedule SHA-256 | `5041e09b…f4e` |
+| validation tensor SHA-256 | `f459316f…338` |
+| diagnostic tensor SHA-256 | `21117e31…716` |
+
+固定诊断时点：
+
+```text
+0, 100, 500, 2,000, 4,000, 8,000
+```
+
+六个时点的全部 activation gradient、output RMS、parameter gradient、mixer 权重与验证 BPC
+都进入 raw JSON；没有只挑“最好看”的 checkpoint。
+
+## 4. 正式训练前的故障与修订
+
+### 4.1 第一次 smoke 的标量序列化错误
+
+首个 depth-16 Baseline 20-step smoke 已完成数值计算，但在写 JSON 前失败：
+
+```text
+RuntimeError:
+self.dim() cannot be 0 to view Float as Byte
+```
+
+原因是 optimizer 的 step 是 0 维 tensor，hash helper 直接把它 `view(torch.uint8)`。修复为：
+
+```text
+tensor.reshape(-1).view(torch.uint8)
+```
+
+当时：
+
+- 没有 formal 运行；
+- 没有输出 JSON；
+- 没有可供选择的正式结果。
+
+### 4.2 smoke 发现被测 residual dtype 不一致
+
+第一版 smoke 通过了 finite / loss×2 / replay 闸门，但检查 capture metadata 时发现：
+
+```text
+Baseline post-MLP h_l：FP32
+Block aggregation partial：BF16
+```
+
+原因：
+
+- Baseline 把 BF16 branch 加到 FP32 embedding/residual stream；
+- Block 每组第一个 partial 直接引用 BF16 branch output。
+
+这会把数值精度差异混进结构比较。正式训练前，协议与实现补充为：
+
+```text
+BF16 branch output → 显式转 FP32 → residual partial 累加
+```
+
+随后 4 个 depth×architecture 格的 smoke 全部从头重跑两次。正式输出是在这次修订之后才开始。
+
+### 4.3 一次未启动模型的 zsh 调度错误
+
+首个正式 Baseline 完成后，批处理脚本用 Bash 式标量切分处理 zsh 字符串，第一行就退出：
+
+```text
+argument --architecture: invalid choice: ''
+```
+
+runner 没有启动，也没有创建新结果文件。调度改为显式 `:` 分隔数组后继续。这是 orchestration
+故障，不是模型运行失败，但仍在审计时间线中保留。
+
+## 5. 运行前与复现闸门
+
+### 5.1 activation-gradient 测量闸门
+
+4 / 4 个 depth×architecture 格都通过：
+
+- 捕获数量严格等于 16 / 32；
+- shape 严格为 `[16,256,192]`；
+- dtype 全部为 FP32；
+- gradient 全部 present、finite；
+- capture storage 全部不别名；
+- diagnostic loss×2 后每层 gradient RMS 精确×2；
+- CV、归一化谱、首尾比不变。
+
+### 5.2 两次独立 smoke
+
+每个格都在两个全新进程中训练 20 steps。排除 timing 后的冻结字段：
+
+| 格 | compare SHA-256 |
+|---|---|
+| depth-16 Baseline | `525cdcd9…bd2a` |
+| depth-16 Block | `e4330a0d…90cc` |
+| depth-32 Baseline | `8ee0dc37…8e56` |
+| depth-32 Block | `289073b1…34c` |
+
+4 / 4 exact。
+
+### 5.3 完整 formal replay
+
+预注册格：
+
+```text
+depth-32 / Block / seed-2026073001
+```
+
+从初始化重新训练完整 8,000 steps，不加载 formal checkpoint。排除 `run_kind`、timing、
+memory 与进程元数据后的全部冻结字段：
+
+```text
+formal compare SHA-256
+46300a452840a9dc6a5180efe7949cf3d81cc4471ed4a942da407e9343064817
+
+replay compare SHA-256
+46300a452840a9dc6a5180efe7949cf3d81cc4471ed4a942da407e9343064817
+```
+
+最终状态：
+
+| 对象 | formal | replay |
+|---|---|---|
+| model state | `3f0b97ec…2f59` | `3f0b97ec…2f59` |
+| optimizer state | `ed03e6fb…4637` | `ed03e6fb…4637` |
+
+字段级 exact。
+
+## 6. 主结果：CV 与首尾失衡为什么方向相反
+
+### 6.1 depth-16
+
+| seed | Base CV | Block CV | 相对 CV 改善 | Base imbalance | Block imbalance | 相对 imbalance 改善 |
+|---:|---:|---:|---:|---:|---:|---:|
+| 2026073001 | 0.41719 | 0.46768 | −12.1% | 1.29458 | 0.38821 | +70.0% |
+| 2026073002 | 0.43744 | 0.45175 | −3.3% | 1.35991 | 0.58958 | +56.6% |
+| 2026073003 | 0.41927 | 0.48391 | −15.4% | 1.29826 | 0.56693 | +56.3% |
+| 均值 | 0.42463 | 0.46778 | **−10.3%** | 1.31759 | 0.51491 | **+61.0%** |
+
+这里“相对 CV 改善”为负，表示恶化。
+
+原始首/末四分位比：
+
+```text
+Baseline：3.65×, 3.90×, 3.66×
+Block：   0.68×, 0.55×, 0.57×
+```
+
+Block 不只是把早层优势降到 1；它在三个 seed 中都略微“过冲”，变成末四分位平均更大。
+但 `abs(log(first/last))` 仍比 Baseline 更接近 0，所以失衡改善。
+
+三 seed 平均 normalized activation-gradient 的最高点：
+
+```text
+Baseline：layer 3 = 1.50× mean；layer 4 = 1.42×
+Block：   layer 11 = 2.09× mean；layer 13 = 1.95×
+```
+
+Baseline 是宽而平滑的早层隆起；Block 是更局部的中后段尖峰。CV 对尖峰敏感，所以升高。
+
+### 6.2 depth-32
+
+| seed | Base CV | Block CV | 相对 CV 改善 | Base imbalance | Block imbalance | 相对 imbalance 改善 |
+|---:|---:|---:|---:|---:|---:|---:|
+| 2026073001 | 0.36109 | 0.64203 | −77.8% | 1.10030 | 0.20781 | +81.1% |
+| 2026073002 | 0.37162 | 0.72997 | −96.4% | 1.14087 | 0.43810 | +61.6% |
+| 2026073003 | 0.40320 | 0.42685 | −5.9% | 1.25169 | 0.33417 | +73.3% |
+| 均值 | 0.37864 | 0.59962 | **−60.0%** | 1.16429 | 0.32669 | **+72.0%** |
+
+原始首/末四分位比：
+
+```text
+Baseline：3.01×, 3.13×, 3.50×
+Block：   0.81×, 0.65×, 1.40×
+```
+
+三 seed 平均 normalized spectrum 的 Block 峰值：
+
+```text
+layer 21 = 3.04× mean
+layer 22 = 2.41×
+layer 23 = 1.91×
+layer 25 = 1.77×
+```
+
+depth-32 每个 AttnRes aggregation group 含 4 个 Transformer blocks。21–24 是第 6 组，
+25–28 是第 7 组。尖峰集中在这两个中后段组附近，是数据中直接可见的结构；但仅凭本实验
+不能断言 pseudo-query、某个 source 或组边界是唯一因果。
+
+### 6.3 seed-3 的中期反例
+
+depth-32 seed-3 在 step 2,000：
+
+```text
+Baseline CV 0.39965
+Block CV    0.34914
+```
+
+此时 Block 更平；到 step 8,000 才变成：
+
+```text
+Baseline CV 0.40320
+Block CV    0.42685
+```
+
+前两个 seed 在 step 2,000 已明显恶化，seed-3 没有。网站必须保留 seed switch，不能用最终
+均值倒写成“三个 seed 从头到尾都一样”。
+
+## 7. 绝对梯度尺度：更平不等于更强
+
+最终 activation-gradient mean：
+
+| depth | Baseline | Block | Block / Baseline |
+|---:|---:|---:|---:|
+| 16 | 约 `2.02×10⁻⁴` | 约 `1.16×10⁻⁴` | **0.574×** |
+| 32 | 约 `1.44×10⁻⁴` | 约 `0.78×10⁻⁴` | **0.544×** |
+
+所以 Block 的首尾比更接近 1，并不是因为它把晚层全部抬高到 Baseline 早层的强度。更接近的
+描述是：
+
+> 整体尺度下降，早层系统性高值被削弱，同时某些中后段位置相对全层均值形成尖峰。
+
+这也是只看 normalized curve 或只看 CV 都不够的原因。
+
+## 8. 参数梯度没有翻转 Round 04
+
+最终核心参数梯度 CV：
+
+| depth | Baseline mean | Block mean | Block / Baseline |
+|---:|---:|---:|---:|
+| 16 | 0.41596 | 0.68287 | 1.64× |
+| 32 | 0.39661 | 0.77176 | 1.95× |
+
+6 / 6 个配对中 Block 都更高。Round 04 的反结果不是在把梯度对象改成 activation 后自动消失；
+两种梯度对象在本轮 final endpoint 都显示更高的跨层 CV。
+
+但 activation gradient 又显示首尾失衡大幅改善，这说明“均匀”至少要拆成：
+
+```text
+首尾是否平衡
+全层是否有尖峰
+绝对尺度多大
+参数更新信号是否平衡
+```
+
+一个标量不能代替全部。
+
+## 9. Output RMS：最接近论文叙述的正向结果
+
+三 seed 最终 post-MLP output RMS 的最后/第一层比：
+
+| depth | Baseline | Block |
+|---:|---:|---:|
+| 16 | 4.59× | 1.17× |
+| 32 | 6.08× | 1.89× |
+
+Baseline output magnitude 随深度明显累积；Block 把增长限制在 aggregation group 内并产生
+周期性 reset。这个缩小实验的 output-RMS 方向与论文 Figure 5(b) 的叙述一致。
+
+仍不能把数值直接叠到论文图上：
+
+- 模型宽度、深度和数据不同；
+- 训练 Token 相差巨大；
+- 论文图的确切 output norm / reduction 也未完整公开；
+- 本实验的 Block group 是 8 组固定设计。
+
+## 10. 验证 BPC 与真实成本
+
+最终 BPC：
+
+### depth-16
+
+| seed | Baseline | Block | Block − Base |
+|---:|---:|---:|---:|
+| 2026073001 | 1.74880 | 1.73739 | −0.01141 |
+| 2026073002 | 1.73683 | 1.73118 | −0.00565 |
+| 2026073003 | 1.73637 | 1.72661 | −0.00976 |
+| 均值 | 1.74067 | 1.73173 | **−0.00894** |
+
+### depth-32
+
+| seed | Baseline | Block | Block − Base |
+|---:|---:|---:|---:|
+| 2026073001 | 1.71790 | 1.71235 | −0.00554 |
+| 2026073002 | 1.72698 | 1.70932 | −0.01765 |
+| 2026073003 | 1.70951 | 1.70310 | −0.00642 |
+| 均值 | 1.71813 | 1.70826 | **−0.00987** |
+
+6 / 6 为负。这是有价值的次要方向，但 Round 05 没有为 BPC 再预注册一个新的 support 阈值，
+因此不追加事后显著性结论。
+
+真实成本：
+
+| depth | Base mean ms | Block mean ms | time ratio | Base peak alloc | Block peak alloc | memory ratio |
+|---:|---:|---:|---:|---:|---:|---:|
+| 16 | 21.47 | 54.71 | 2.55× | 3.04 GB | 6.54 GB | 2.15× |
+| 32 | 42.11 | 109.38 | 2.60× | 5.94 GB | 12.82 GB | 2.16× |
+
+这个简单 eager 实现没有论文训练系统的 kernel、并行或工程优化；成本数值不应外推到 K3。
+但它足以说明本站的 BPC 对比不是同 wall time / FLOPs。
+
+## 11. 预注册判定为何是 mixed
+
+支持需要：
+
+```text
+CV：3 / 3 seeds 改善，平均相对改善 ≥20%
+AND
+imbalance：3 / 3 seeds 改善，平均相对改善 ≥20%
+```
+
+concern 需要两项都以相同规则恶化。
+
+实际：
+
+```text
+depth-16：
+CV 3 / 3 恶化，平均 10.3%
+imbalance 3 / 3 改善，平均 61.0%
+
+depth-32：
+CV 3 / 3 恶化，平均 60.0%
+imbalance 3 / 3 改善，平均 72.0%
+```
+
+两个指标相反，所以两个 depth 都是 `mixed / inconclusive at this depth`。这不是“数据没规律”，
+而是预注册的“更均匀”概念被实验拆成了两个方向相反的组成部分。
+
+## 12. 开放工件与校验哈希
+
+| 工件 | SHA-256 |
+|---|---|
+| manifest | `080afb17…6371` |
+| protocol | `f772629b…ce22` |
+| definition audit | `79221c56…cc6` |
+| runner | `04ae69e1…800f` |
+| analyzer | `017d38d9…2fc8` |
+| aggregate canonical | `69be133c…7b51` |
+| compact canonical | `8cdb7180…044f` |
+| reproduction canonical | `addb2e59…f68a` |
+
+公开目录包含：
+
+- 12 个完整 formal raw JSON；
+- 8 个两套 smoke raw JSON；
+- 1 个完整 replay raw JSON；
+- 完整 aggregate；
+- 网站 compact payload；
+- manifest、runner、analyzer 与 reproduction 清单；
+- 前置定义审计、本协议和本结果审计。
+
+`experiments/k3/attnres_gradient/reproduction.json` 记录每个 raw 文件 SHA-256。
+
+## 13. 允许和禁止的结论
+
+允许：
+
+> 在本轮公开定义下，Block AttnRes 一致缓解了首/末深度四分位失衡，但在中后段形成局部
+> 梯度尖峰，导致全层 CV 一致升高；因此“梯度更均匀”必须拆成多个指标解释。
+
+> 同 token / step 预算下，Block 的最终验证 BPC 在 6 / 6 个配对中更低，但实际运行成本
+> 约为 Baseline 的 2.6× step time 与 2.2× peak allocated memory。
+
+禁止：
+
+- “复现了论文 Figure 5(c)”；
+- “论文的梯度结论是错的”；
+- “已测到 Kimi K3 checkpoint 的真实梯度”；
+- “Block 解决了梯度消失 / 爆炸”；
+- “CV 更高就代表训练一定更不稳定”；
+- “BPC 改善是同 FLOPs / wall time 优势”；
+- 从 3 seeds 推导总体显著性；
+- 从 depth 16 / 32 外推到 48B、1T+400B Token 或 K3 2.8T 参数。
+
+## 14. 下一步最值得问什么
+
+这轮已经把“梯度”从一个模糊词拆成了可复查对象。下一个有价值的问题不是再换一个漂亮
+汇总指标，而是追踪局部尖峰从哪里来：
+
+1. 分开捕获 pre-attention 与 pre-MLP residual positions；
+2. 把 layer 21–25 的 activation-gradient 与 mixer source weights 同步对齐；
+3. 比较 aggregation-group boundary 前后；
+4. 在不改变 formal 结果的前提下，对相同 raw gradient tensor 做多种公开 reduction
+   sensitivity analysis；
+5. 若官方之后发布 Figure 5 telemetry 代码，再按其定义单独开新 protocol。
+
+这些属于后续轮次，不能倒写进本轮预注册结论。

research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md

@@ -0,0 +1,459 @@
+# Kimi K3 第五轮：Attention Residuals 梯度定义与深度扩展实验协议
+
+> 协议 ID：`llm-atlas-k3-attnres-gradient-scale-v1`  
+> 冻结日期：2026-07-30（Asia/Shanghai）  
+> 状态：正式输出前预注册  
+> 前置定义审计：`research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md`
+
+## 0. 目标与一句话研究问题
+
+Round 04 在缩小模型中得到两个同时成立的结果：
+
+- Full / Block AttnRes 的 2,000-step 验证 BPC 都优于 Baseline；
+- 以“每个 Transformer block 的核心**参数**梯度 RMS”定义时，跨深度 CV 反而更高。
+
+论文 Figure 5(c) 没有公开足以唯一恢复的梯度测量合同。Round 05 不猜作者的隐藏代码，而是
+冻结一个可复现、与 Figure 5 的 output/gradient 并列叙述对齐的 activation-gradient 定义，
+再问：
+
+> 当深度从 16 增至 32、训练预算从 2,000 增至 8,000 step 时，Block AttnRes 是否比
+> PreNorm Baseline 更一致地降低 post-MLP block-output activation gradient 的跨深度失衡？
+
+这不是 K3 checkpoint forward，也不是论文 Figure 5 数值复画。
+
+## 1. 一手来源与不可补写的空白
+
+| 工件 | 固定 revision / checksum | 用途 |
+|---|---|---|
+| Attention Residuals GitHub | `85e22310fe5ee860b4a023de312d791de8a5a5e6` | 公式、Figure 5 / 8、模型尺度 |
+| `Attention_Residuals.pdf` | SHA-256 `e5831b0d…a26b2f` | 论文一手图文 |
+| WikiText-2 raw | `Salesforce/wikitext@b08601e04326c79dfdd32d625aee71d232d685c3` | 固定公开训练语料 |
+| Round 04 protocol | `llm-atlas-k3-attnres-reduced-v1` | 公共主干与数据合同来源 |
+
+官方仓库没有模型实现、训练脚本、checkpoint 或 Figure 5 原始数据。以下字段不能归因给论文：
+
+- Figure 5 的确切 gradient tensor；
+- norm / reduction；
+- diagnostic batch；
+- AMP / clipping 时点；
+- 单点还是时间平均。
+
+Grok Headless 只进行一次对抗式方法检查；其建议和错误都在前置审计中公开，不是事实来源。
+
+## 2. 设计总览
+
+```text
+2 个深度：16 / 32 Transformer blocks
+× 2 个 residual graph：PreNorm Baseline / Block AttnRes
+× 3 个冻结 seed
+× 8,000 training steps
+× 32 windows/step
+× 256 target bytes/window
+= 12 个正式训练格
+= 786,432,000 target bytes
+```
+
+只比较 Baseline 与 Block，因为论文 Figure 5 的训练动力学面板也是这两个结构的直接对照。
+Round 04 的 Full AttnRes 结果保持公开，但本轮不增加一个与主问题无关的 6-run 分支。
+
+## 3. 数据合同
+
+继承 Round 04 的语料和预处理：
+
+1. 按固定 parquet 行序读取 `text`；
+2. 每行追加一个 `\n`；
+3. UTF-8 编码，无 normalization、strip、去空行或大小写改写；
+4. byte vocabulary `0..255`；
+5. 每个窗口连续取 257 bytes，前 256 预测后 256。
+
+固定拼接后 split：
+
+| split | bytes | SHA-256 |
+|---|---:|---|
+| train | 10,951,563 | `0ca7d3e74dbe44564ea5942b85232f1bbcb525c9cd481cd5d28a87ee90e7e9b4` |
+| validation | 1,148,008 | `a42356f6a8ff1d25daf25ec9db49e10a537c265581b61c74604bb63231dee719` |
+| test | 1,292,014 | `bfe9eb16ab9987fb88bde4ea9a30a00f2a45db01dfc14bad78d05325789c4f12` |
+
+训练第 `step`、第 `row` 的 window 起点：
+
+```text
+z = first 8 bytes of SHA256(
+  protocol_id + "\0train-window\0" + seed + "\0" + step + "\0" + row
+)
+start = uint64_be(z) mod (len(train_bytes) - 257)
+```
+
+同一 seed 的 4 个结构格逐 step / row 使用完全相同的 token tensor。validation 64 windows、
+diagnostic 16 windows，分别由标签 `validation-window` / `diagnostic-window` 与固定 index
+生成，对全部结构与 seed 相同。
+
+正式运行前 manifest 必须记录 parquet hash、split bytes/hash、全部 `3×8,000×32=768,000`
+唯一训练窗口起点的 schedule hash、validation tensor hash 与 diagnostic tensor hash。
+
+## 4. 模型合同
+
+### 4.1 两个深度共享的结构
+
+| 项 | 固定值 |
+|---|---:|
+| vocabulary | 256 bytes |
+| context | 256 |
+| `d_model` | 192 |
+| heads / head dimension | 6 / 32 |
+| `d_ff` | 768 |
+| dropout | 0 |
+| positional embedding | learned absolute，256 × 192 |
+| norm | RMSNorm，`eps=1e-6` |
+| attention | causal MHA；score 以 FP32 softmax |
+| MLP | bias-free SwiGLU，`192→768`, `192→768`, `768→192` |
+| embedding / readout | tied；final RMSNorm 后乘 token embedding |
+
+全部 bias-free linear 与 embedding 初始化为 `N(0,0.02)`。attention output projection 与 MLP
+down projection 的标准差为 `0.02 / sqrt(2×depth)`；普通 RMSNorm 为 1。
+
+数值精度进一步固定为：attention / MLP 线性分支受 BF16 autocast；embedding、Baseline hidden
+residual stream 与 Block aggregation partial 都以 FP32 累加。也就是说，Block 每个 BF16
+branch output 在进入 `partial` 前显式转为 FP32。这样两种结构被捕获的 `h_l` 都是 FP32，
+不会把 residual accumulator 精度差异混进梯度形状对比。
+
+### 4.2 深度与 Block AttnRes 聚合
+
+| Transformer depth | residual sublayers | aggregation groups | sublayers/group | Transformer blocks/group |
+|---:|---:|---:|---:|---:|
+| 16 | 32 | 8 | 4 | 2 |
+| 32 | 64 | 8 | 8 | 4 |
+
+Baseline 子层为：
+
+```text
+h ← h + f(RMSNorm(h))
+```
+
+Block AttnRes：
+
+- embedding 永远是 source 0；
+- 对已经完成的 aggregation-group sums 做跨组 softmax mixture；
+- 组内 attention / MLP branch output 累加到 `partial`；
+- 达到组边界时，保存完整 `partial` 为新 source，再开始下一组；
+- output mixer 聚合 embedding + 8 个完整 group sums。
+
+每个子层的 pseudo-query 为 `d_model` 向量，严格 zero-init；每个 source 的 key RMSNorm weight
+严格 one-init。Baseline / Block 不要求总参数量或 residual-mixer FLOPs 相等，但同 seed /
+depth 的 token/position embedding、attention、MLP、input norm、final norm 和 tied readout
+必须逐 tensor SHA-256 exact。
+
+## 5. 训练合同
+
+| 项 | 固定值 |
+|---|---:|
+| seeds | `2026073001, 2026073002, 2026073003` |
+| formal steps | 8,000 |
+| batch | 32 |
+| context | 256 |
+| target bytes / run | 65,536,000 |
+| optimizer | AdamW |
+| betas / epsilon | `(0.9,0.95)` / `1e-8` |
+| peak / min LR | `3e-4` / `3e-5` |
+| warmup | 400 steps，linear |
+| decay | cosine，step 400→8,000 |
+| weight decay | `0.1` if `ndim>=2`，否则 `0` |
+| parameter grad clip | global norm `1.0` |
+| compute | BF16 autocast；FP32 optimizer state |
+| compile | off / eager |
+| device | one RTX 5090 |
+| deterministic | deterministic algorithms；`CUBLAS_WORKSPACE_CONFIG=:4096:8` |
+
+验证和诊断都发生在：
+
+```text
+step 0, 100, 500, 2,000, 4,000, 8,000
+```
+
+验证固定 64 windows，8 windows/eval batch，报告 token-mean CE nats 与
+`bits_per_byte = CE / ln(2)`。诊断固定 16 windows，一次性输入，不改变 optimizer state。
+
+计时合同：
+
+- 前 20 个 training step 不计入；
+- step 21–8,000 每步前后 CUDA synchronize；
+- step 20 后 reset peak memory；
+- 报告 mean / median / p95 step ms、peak allocated / reserved；
+- timing、wall clock、hostname、GPU temperature 不进入 exact replay 字段。
+
+## 6. 主梯度对象：精确到代码位置
+
+### 6.1 `h_l` 定义
+
+对 `l=1..depth`，统一在第 `l` 个 Transformer block 的 MLP branch 完成后捕获：
+
+| 架构 | 精确定义 |
+|---|---|
+| Baseline | attention residual 与 MLP residual 都完成后的 hidden state |
+| Block | MLP branch 已加入、本 aggregation partial 可能保存/reset **之前**的 partial |
+
+所有 `h_l.shape = [16,256,192]`、dtype 为 FP32。实现必须为 diagnostic forward 返回独立
+引用列表，不允许捕获 reset 后的零张量，不允许把 8 个 aggregation sources 当成 16 / 32 个
+Transformer outputs。
+
+### 6.2 diagnostic loss 与 gradient magnitude
+
+```text
+model.eval()
+logits, h[1..depth] = forward(fixed_diagnostic_x, capture=true)
+L = mean(cross_entropy(logits.float(), fixed_diagnostic_y))
+backward(L)          # 不使用 GradScaler，不执行 optimizer.step
+m_l = sqrt(mean(float32(h_l.grad)² over batch×time×channel))
+```
+
+规则：
+
+- forward 仍使用与训练一致的 BF16 autocast；
+- logits 在 FP32 中计算 CE；
+- loss 对全部 `16×256` targets 做算术平均，无 mask、无 label smoothing；
+- backward 前 model / optimizer gradients 清零；
+- activation gradient 在任何 parameter clipping 之前读取；
+- diagnostic 不消耗训练数据，不进入 optimizer，不改变学习率或模型状态。
+
+### 6.3 同位置 output magnitude
+
+同一批 `h_l` 计算：
+
+```text
+o_l = sqrt(mean(float32(h_l)² over batch×time×channel))
+```
+
+它用于显示 Baseline 单调累积与 Block 组内周期，而不作为主 confirmatory endpoint。
+
+## 7. 主指标与预注册判据
+
+每个 diagnostic step 保存完整 `m_1..m_depth`。以下统计由未四舍五入的 float64 数组计算。
+
+### 7.1 绝对尺度
+
+```text
+mean_grad = mean_l(m_l)
+```
+
+它防止“曲线更平只是全部梯度趋近于零”被 CV 隐藏。绝对尺度不设置优劣阈值，只公开。
+
+### 7.2 归一化谱与 CV
+
+```text
+n_l = m_l / mean_grad
+CV = population_std_l(m_l) / mean_grad
+```
+
+使用 population standard deviation（`ddof=0`）。网站必须同时显示 `m_l` 与 `n_l`，不能只显示
+CV 排名。
+
+### 7.3 前后四分位失衡
+
+```text
+q = depth / 4
+Q_first = mean(m_1 .. m_q)
+Q_last  = mean(m_(depth-q+1) .. m_depth)
+imbalance = abs(ln(Q_first / Q_last))
+```
+
+depth 16 时各取 4 层；depth 32 时各取 8 层。`imbalance=0` 才表示首尾一致；这个定义不会把
+“早层偏大”和“晚层偏大”错误地都解释为越小越好。原始有符号比 `Q_first/Q_last` 仍公开。
+
+### 7.4 seed 内配对对比
+
+只在最终 step 8,000 做 confirmatory verdict：
+
+```text
+relative_CV_reduction
+  = (CV_baseline - CV_block) / CV_baseline
+
+relative_imbalance_reduction
+  = (imbalance_baseline - imbalance_block) / imbalance_baseline
+```
+
+若 Baseline imbalance 精确为 0，则该 seed 的 relative imbalance reduction 定义为不可计算，
+该 depth 自动不能得到“联合支持”；仍公开绝对差。
+
+对每个 depth 分别判定：
+
+- **joint directional support at this depth**：三个 seed 的 CV reduction 都 `>0`，其均值
+  `>=20%`；同时三个 seed 的 imbalance reduction 都 `>0`，其均值 `>=20%`。
+- **joint directional concern at this depth**：三个 seed 的两项 reduction 都 `<0`，且两项
+  平均相对恶化都 `>=20%`。
+- 其他：**mixed / inconclusive at this depth**。
+
+总判定：
+
+- 两个 depth 都 support：**scale-consistent directional support in this operationalization**；
+- 两个 depth 都 concern：**scale-consistent directional concern in this operationalization**；
+- 其他：**depth-dependent or inconclusive**。
+
+不计算 p-value、population CI，不把 3 seeds 称为统计证明。
+
+## 8. 必须公开的次要指标
+
+### 8.1 全时间轨迹
+
+六个预注册时点的以下数据必须全部公开，不能选择“最好看”的 checkpoint：
+
+- validation BPC；
+- absolute activation-gradient spectrum；
+- normalized activation-gradient spectrum；
+- activation CV；
+- first/last quartile ratio 与 imbalance；
+- output RMS spectrum；
+- mean activation-gradient scale。
+
+### 8.2 参数梯度
+
+延续 Round 04 定义：每个 Transformer block 的 attention、MLP 与两个 input norm 的参数梯度
+拼接后计算：
+
+```text
+parameter_grad_rms[l] = sqrt(sum(g²) / total_parameter_elements)
+```
+
+不含 embedding、final norm、LM head 与 AttnRes mixer 参数；clip 前读取。报告完整谱、CV 和
+前后四分位失衡，但它们不进入 Round 05 主判定。
+
+### 8.3 mixer / 成本
+
+Block 同报：
+
+- 各子层 softmax mixture 的 source-depth 分布；
+- output mixer 分布；
+- entropy 与 embedding / latest-complete-group mass；
+- step time、peak allocated/reserved、参数量。
+
+这些用于解释机制与成本，不改变 confirmatory verdict。
+
+## 9. 运行前闸门
+
+### 9.1 数据闸门
+
+- split bytes/hash 与 Round 04 exact；
+- 新 protocol 的 768,000-window schedule hash 落盘；
+- validation / diagnostic tensor hash 落盘；
+- 同 seed 四结构的至少 step 0 / 1 / 7,999 输入 tensor hash exact。
+
+### 9.2 公共权重闸门
+
+每个 depth / seed 的 Baseline 与 Block 公共参数逐 tensor exact；输出：
+
+- 公共参数 tensor 数；
+- 公共参数 element 数；
+- name / shape / dtype / bytes 联合 hash；
+- value bytes 联合 hash。
+
+### 9.3 activation-gradient 闸门
+
+四个 depth×architecture 格都必须通过：
+
+1. 捕获数量严格等于 depth，shape 均为 `[16,256,192]`；
+2. 全部 gradient 非 `None`、finite、storage 不别名；
+3. 同输入把 loss 乘 2 后，每层 `m_l` 比值在 `2±1e-5`；
+4. loss×2 前后 CV、normalized spectrum、quartile ratio 在 `1e-6` 绝对容差内；
+5. 全新进程重复 smoke 的冻结字段 exact。
+
+### 9.4 smoke
+
+四个格都运行 20 training steps；冻结字段包括：
+
+- protocol、architecture、depth、seed、device、dtype；
+- data/schedule/tensor hashes；
+- public-weight hashes；
+- step 0 / 20 loss 与 validation；
+- 全部预注册 diagnostic 数组；
+- finite / alias / loss-scale checks。
+
+smoke 不能写入 formal 目录。
+
+## 10. 正式执行与独立 replay
+
+12 个 formal grid 必须各自在全新进程中执行。目录键为：
+
+```text
+depth-{16|32}/{baseline|block}/seed-{2026073001|2026073002|2026073003}
+```
+
+完成后预注册 replay：
+
+```text
+depth-32 / block / seed-2026073001
+```
+
+replay 再从初始化训练完整 8,000 steps，不加载 formal checkpoint。比较时排除：
+
+- wall time / step-time samples；
+- peak memory；
+- process ID / hostname；
+- GPU 温度与驱动层瞬时字段；
+- 文件路径和生成时间。
+
+必须 exact 的字段：
+
+- 数据与公共权重 hashes；
+- 全部 validation CE/BPC；
+- 全部 activation/output/parameter gradient 数组；
+- mixer 数组与 summary；
+- final model-state tensor hash；
+- optimizer-state tensor hash；
+- training-loss checkpoint 数组。
+
+若 replay 不 exact，停止聚合并公开失败，不挑选另一 seed 替代。
+
+## 11. 公开工件
+
+正式结果完成后仓库必须包含：
+
+```text
+experiments/k3/attnres_gradient/
+  README.md
+  build_dataset.py
+  manifest.json
+  train.py
+  analyze.py
+  results/raw/*.json
+  results/compact.json
+  reproduction.json
+
+research/
+  K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md
+  K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md
+  K3_ATTNRES_GRADIENT_SCALE_AUDIT.md
+```
+
+网站至少提供五个互相联动的视图：
+
+1. 论文 Figure 5 的“已知 / 未定义”拆解；
+2. activation gradient 绝对谱与 normalized spectrum；
+3. depth 16 / 32、三 seed、六时间点对比；
+4. output RMS 周期与 Block aggregation boundary；
+5. activation gradient / parameter gradient 并排，以及判定、成本、哈希和声明边界。
+
+图中必须能切换到所有负结果；不能只放均值、只放 final 或隐藏某个 seed。
+
+## 12. 允许与禁止的结论
+
+若达到支持条件，允许写：
+
+> 在这个公开定义、两种缩小深度和 8,000-step byte-LM 合同中，Block AttnRes 方向一致地
+> 降低了 post-MLP output activation gradient 的跨深度 CV 与首尾四分位失衡。
+
+无论结果怎样，都禁止写：
+
+- 复现了论文 Figure 5 的数值；
+- 证明了论文未公开实现采用相同梯度定义；
+- 证明了 Kimi K3 的真实梯度更健康；
+- 证明 AttnRes 解决梯度消失、梯度爆炸或训练稳定性的全部问题；
+- 从 3 seeds 推导总体显著性；
+- 从 depth 16 / 32 外推至 48B、1T+400B Token 或 K3 2.8T 参数；
+- 隐藏 activation 与 parameter gradient 方向不一致的结果。
+
+## 13. 变更纪律
+
+本文件提交后：
+
+- 允许修复使实现符合本协议的 bug；
+- 允许补充日志、注释、可视化和不改变数值的导出；
+- 不允许看过 formal 结果后修改主指标、阈值、diagnostic step、seed、训练预算或 replay 格；
+- 任何不得不改变实验合同的事项必须先停止、写入审计、升级 protocol ID，再重新执行全部 grid。

research/K3_ATTNRES_LOCAL_PATH_AUDIT.md

@@ -0,0 +1,302 @@
+# K3 Attention Residuals 局部 mixer 路径：Round 07 结果审计
+
+研究日期：2026-07-30  
+协议：`llm-atlas-k3-attnres-local-path-v1`  
+预注册 commit：`6911efc`  
+结果前 runner / analyzer commit：`39a9ad6`  
+研究身份：**受 Round 05 / 06 启发的定向 reduced-model mechanism probe**
+
+## 0. 一句话结论
+
+> group 6+7 的 16 个 depth mixers 在 learned 背景上的局部 uniform intervention，
+> 足以复现全局 log-gap reduction 的至少一半；但从 all-uniform 背景只恢复这 16 个
+> mixer 时，contrast 恢复超过一半，peak 只恢复约 35.5%–41.3%。因此本轮得到强的
+> **one-sided evidence**，但没有通过预注册的双向 localization 门。
+
+这不是一句保守套话，而是协议第 13 节的直接判定：
+
+```text
+groups 6+7 sufficiency  = PASS 6 / 6
+groups 6+7 restoration = FAIL 3 / 6
+localization            = NOT ESTABLISHED
+```
+
+## 1. 运行与输入闸门
+
+正式网格：
+
+| run | seed | steps | target bytes | final validation BPC |
+|---|---:|---:|---:|---:|
+| formal | 2026073001 | 8,000 | 65,536,000 | 1.7123525941 |
+| formal | 2026073002 | 8,000 | 65,536,000 | 1.7093240656 |
+| formal | 2026073003 | 8,000 | 65,536,000 | 1.7030966813 |
+| replay | 2026073001 | 8,000 | 65,536,000 | 1.7123525941 |
+
+正式三格合计 196,608,000 target bytes，含 replay 为 262,144,000。
+
+全部通过：
+
+- 三 seed final model-state 与 Round 06 exact；
+- 三 seed final optimizer-state 与 Round 06 exact；
+- 六个 validation BPC、training history、六个 parent `learned` diagnostics exact；
+- step 0 / 8,000 的 `detached_learned` 与 Round 06 同名 endpoint exact；
+- step 0 / 8,000 的 `uniform_all` 与 Round 06
+  `uniform_value_backward` endpoint exact；
+- 14 modes 的 logits、loss、六位置 activations、父 mixer summaries exact；
+- 65 次 selector visits 的 identity、顺序、唯一性、exact mask 与 census 全部通过；
+- step-0 14-mode negative control、parent learned-vs-detached control、
+  `loss ×1 / ×2` scale gate 全部通过；
+- seed-1 从初始化完整 replay exact。
+
+因此后续差异来自同一 forward state 上的预注册 backward coefficient masks，不是不同训练
+状态、batch、loss、activation 或 selector 漂移。
+
+## 2. 全局端点先复现
+
+最终 `post_mlp_state / element_rms`：
+
+| metric | detached learned（3-seed mean） | uniform all（3-seed mean） | Round 06 per-seed mean relative drop |
+|---|---:|---:|---:|
+| spike contrast | 2.8065 | 0.7859 | 70.2% |
+| peak / layer mean | 3.0651 | 1.8964 | 37.0% |
+
+每个 seed、两个指标的 `G_X = ln(X_ref / X_uniform_all)` 都严格为正，raw relative
+drop 6 / 6 超过 20%。global gap gate 完整成立。
+
+这一步重要，因为局部 score 的分母不是任意“改善空间”，而是同一 seed、同一指标的
+实测 global log gap。若任一 global gap 不成立，本轮主判定就必须停止；实际没有触发该
+停止规则。
+
+## 3. groups 6+7：充分性很强
+
+只把 mixer indices 40–55 改成 uniform，其他 49 个 mixer 保持 detached learned：
+
+| seed | `S_contrast` | `S_peak` | 50% × two metrics |
+|---:|---:|---:|---:|
+| 2026073001 | 0.697 | 1.817 | PASS |
+| 2026073002 | 0.676 | 1.783 | PASS |
+| 2026073003 | 0.658 | 1.501 | PASS |
+| **mean** | **0.677** | **1.700** | **6 / 6** |
+
+raw metric 的三 seed mean 也从 reference 的 `2.8065 / 3.0651` 变为
+`1.1737 / 1.3450`。
+
+`S_peak > 1` 不是 170% 因果贡献。它只表示在 log ratio 上，局部 uniform groups 6+7
+把 peak 推得比 all-65 uniform endpoint 还低。这是很直接的 non-additivity / interaction
+信号，也是协议坚持“不裁剪 score 到 [0,1]”的原因。
+
+允许结论：
+
+> groups 6+7 在本 diagnostic 中足以复现至少一半 global log-gap reduction。
+
+禁止结论：
+
+- “这 16 个 mixer 解释了 67.7% / 170.0% 的尖峰”；
+- “剩下 49 个 mixer 只贡献 32.3% / −70.0%”；
+- “group 6+7 是唯一原因”。
+
+## 4. 反向 restoration 没有给出同样答案
+
+从 all-uniform 背景出发，只把 groups 6+7 恢复为 detached-learned coefficients；
+其他 49 个 mixer 仍为 uniform：
+
+| seed | `R_contrast` | `R_peak` | 50% × two metrics |
+|---:|---:|---:|---:|
+| 2026073001 | 0.649 PASS | 0.372 FAIL | FAIL |
+| 2026073002 | 0.621 PASS | 0.355 FAIL | FAIL |
+| 2026073003 | 0.680 PASS | 0.413 FAIL | FAIL |
+| **mean** | **0.650** | **0.380** | **3 / 6** |
+
+raw metric 的三 seed mean 从 all-uniform 的 `0.7859 / 1.8964` 恢复为
+`1.7693 / 2.2658`。contrast 明显朝 reference 回升，但 peak 的 log-gap recovery
+没有一个 seed 达到 50%。
+
+这说明同一个 scope 的作用强烈依赖其他 mixer 处于 learned 还是 uniform 背景：
+
+- learned 背景中 uniformize groups 6+7，足以大幅压低 contrast 与 peak；
+- uniform 背景中 restore groups 6+7，足以恢复 contrast，却不足以恢复 peak；
+- 两个方向不对称，不能用单侧 sufficiency 替代双向 localization。
+
+因此正式 verdict 是：
+
+```text
+one_sided_evidence_localization_not_established
+```
+
+不是 “almost passed”，也不因 `R_peak` mean 约 0.38 而软化 0.50 阈值。
+
+## 5. 单 group 结果：同样显示交互
+
+### 5.1 sufficiency
+
+| scope | mean `S_contrast` | mean `S_peak` | 20% gate |
+|---|---:|---:|---:|
+| group 6 | 0.281 | 1.044 | PASS 6 / 6 |
+| group 7 | 0.438 | 0.843 | PASS 6 / 6 |
+
+两个单 group 都在两个指标、三个 seed 通过 material local sufficiency。
+
+但：
+
+```text
+S(group6) + S(group7) ≠ S(groups6+7)
+```
+
+尤其 peak 上，两个单 group 与联合 scope 都可能超过 global endpoint，不能按 mixer
+数量或 score 相加做贡献账。
+
+### 5.2 restoration
+
+| restored scope | mean `R_contrast` | mean `R_peak` | 20% gate |
+|---|---:|---:|---:|
+| group 6 | +0.248 | −0.146 | MIXED / FAIL 3 / 6 |
+| group 7 | +0.409 | −0.191 | MIXED / FAIL 3 / 6 |
+
+恢复单个 group 时，contrast 在三 seed 都超过 20%，peak 却在三 seed 全为负：相对
+all-uniform，恢复一个 group 的 learned coefficients 反而让最高层 / 均值更低。
+
+这不是 “group 没作用”，而是 effect direction 随 metric 与背景改变。它进一步反对
+简单、可加的局部归因故事。
+
+## 6. attention vs MLP：只有 group 7 过闸
+
+### group 6
+
+MLP-only 在 6 个 branch cells 中赢 5 个；seed 2026073003 的 contrast
+`S=0.177 < 0.20`。因此：
+
+```text
+group 6 branch dominance = NOT ESTABLISHED
+```
+
+不能因为 margin 大、均值高，忽略 material threshold 的单格失败。
+
+### group 7
+
+| branch | mean `S_contrast` | mean `S_peak` |
+|---|---:|---:|
+| attention-only | 0.015 | 0.026 |
+| MLP-only | 0.426 | 0.823 |
+
+MLP-only 自身 material，且在两个指标、三个 seed 都比 attention-only 高至少
+15 percentage points，因此：
+
+```text
+group 7 MLP branch-dominant at the preregistered margin
+```
+
+这是协议第 14 节的**次级、sufficiency-only、探索性**判定；没有 branch-level
+restoration，不得升级为第 13 节的双向 localization。
+
+## 7. output / depth controls
+
+| control | mean `S_contrast` | mean `S_peak` | verdict |
+|---|---:|---:|---|
+| output-only（1 mixer） | 0.131 | 0.187 | 0 / 6 at 50% |
+| all-depth（64 mixers） | 0.949 | 1.145 | near / beyond global endpoint |
+
+output mixer 单独无法解释 global gap 的一半。all-depth 已复现绝大多数 contrast gap，
+peak 甚至超过 all-65 endpoint；把 output 与 depth scores 相加会产生负
+`interaction_residual`。该 residual 只是 bookkeeping，不预期为 0，不是统计交互检验。
+
+## 8. 32-layer 谱的直观变化
+
+三个 seed 的 reference peak 都在 layer 21；all-uniform peak 都迁到 layer 2。
+
+groups 6+7 only：
+
+- seed 1 peak → layer 5；
+- seed 2 peak → layer 25；
+- seed 3 peak → layer 6。
+
+restore groups 6+7 on uniform background：
+
+- 三 seed peak 都回到 layer 21；
+- 但 peak / mean 的恢复比例仍只有 0.355–0.413。
+
+“peak layer 回来了”与“peak 强度恢复超过一半”不是同一判据。网站会同时展示谱与
+预注册 score，避免只凭最高点位置讲故事。
+
+## 9. replay 与 artifact 链
+
+seed 2026073001 从初始化完整重跑。排除 `run_kind`、timing 与 self canonical hash 后：
+
+```text
+formal seed1 == replay
+compare SHA-256 = 7dbd15ad03fbd357c5d91e159706d63b24703722f76c492ed1dc733535d6b9cf
+```
+
+它覆盖训练状态、六 checkpoints、14-mode 两端矩阵、六位置 gradient reductions、
+selector visits 与所有 gates，不只是 final BPC。
+
+冻结物理 hashes：
+
+| artifact | SHA-256 |
+|---|---|
+| manifest | `db01e92e…9139` |
+| runner | `b42879e2…b03d` |
+| analyzer | `e0921562…d02b` |
+| packager | `6a9ada0b…9196` |
+| aggregate | `bb0ec9fc…160e` |
+| compact | `3bb6c158…9c08` |
+| reproduction | `524a6883…5db` |
+
+canonical hashes：
+
+```text
+aggregate    b86d119cd2f106e2cbee8a35760ed3244336a2fcfeb9178ea1e7dab13fc6f215
+compact      2aff9288f52d3d41bb1f59c64d9a07518ad3e2120c61615478087b24aaabd835
+reproduction 6f5d98fce6446fecc966dd2675f272f2c4f0c9a39a5741fabc4ffad6852ca7f4
+```
+
+## 10. 两次 Grok Headless 审阅
+
+### 结果前
+
+session `019fb151-9627-76c1-b7d7-53012874f85c` 找出 selector API、output identity、
+“restore to learned”歧义、parent learned 调度、20% 分母和 negative control 六个硬问题。
+全部在预注册 freeze 前修正并留档。
+
+### 结果后
+
+session `019fb19d-94a3-7231-9a63-3a1ef33a9892` 只读对照 protocol、manifest、
+analyzer 与 aggregate，独立复算代表性 `G/S/R` cells：
+
+- 阻断实现错误：0；
+- one-sided verdict：确认正确；
+- 不允许修改阈值；
+- 指出单 group restoration 的 `score_sign_split` reason 是跨指标汇总，因此 reason
+  wording 略宽；`mixed / fail` 本身仍由 C/P pass split 独立成立，主 verdict 不受影响。
+
+Grok 是方法学审稿人，不是论文或实验事实来源；正式证据仍是冻结代码与 raw outputs。
+
+## 11. `A_log` 工件边界同步更新
+
+本轮仍不是 K3 2.8T checkpoint forward。截止 2026-07-30 12:35 CST：
+
+- official main 仍是 `9f62e4e9fffbd0a83ddd60e1c209d828994b3569`，96 vs 128
+  mismatch 未修；
+- community PR #144 把 parameter 改成 128，但没有独立 forward 验证；
+- community PR #150 保留 96，并在加载时验证 / 裁掉 32 个全零尾项；提交者报告检查
+  69 层并完成 disk-offloaded end-to-end generation；
+- 两个 PR 都未合并，Moonshot 尚未给出官方裁决。
+
+因此“没有任何公开候选解释”已经过时；“官方 contract 已解决”同样不成立。
+
+## 12. 最强允许结论
+
+可以说：
+
+> 在本缩小 Block AttnRes 模型的同前向 diagnostic backward 中，global learned-value
+> coefficient sensitivity 对 groups 6+7 的局部 uniformization 具有强 sufficiency；
+> restoration 只在 contrast 上超过一半，在 peak 上稳定不足一半，因此预注册的双向
+> localization 未建立。group 7 的 MLP mixer 在次级 sufficiency-only branch 判定中占优。
+
+不能说：
+
+- “证明 K3 的尖峰来自 group 6 / 7”；
+- “groups 6+7 解释了 67.7% / 170.0%”；
+- “MLP 是唯一原因”；
+- “只要把这些 mixer 训练成 uniform 就会更稳定”；
+- “复现了 K3 Figure 5(c)”；
+- “community PR #150 已经是官方 `A_log` 修复”。

research/K3_ATTNRES_LOCAL_PATH_GROK_REVIEW.md

@@ -0,0 +1,64 @@
+# Round 07 Grok Headless 对抗审阅与处置
+
+审阅日期：2026-07-30  
+审阅会话：`019fb151-9627-76c1-b7d7-53012874f85c`  
+身份：**外部模型的只读方法学审稿，不是论文证据源**
+
+## 1. 调用边界
+
+Grok CLI 使用 single/headless 方式读取：
+
+- `research/K3_ATTNRES_LOCAL_PATH_SCOPING.md`
+- `research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md`
+- `experiments/k3/attnres_spike/train.py`
+
+关闭 web search、禁止 subagents、使用 plan permission；它没有修改文件。审稿任务是找
+selector、公式、endpoint exactness、negative control、replay 与父 runner API 的冲突。
+
+## 2. Blocking findings 与处置
+
+| finding | 风险 | 处置 |
+|---|---|---|
+| 父 runner 只有 global mode，局部 selector 尚不可实现 | 不同实现可能选错 indices 或混入 full autograd | **采纳**：协议冻结 14 个 exact identity sets；所有矩阵 mode 只用同一个 `RoutedSourceBackward` |
+| 新 output `index=64` 与父 summary 的 `mixer_index=null` 冲突 | 改 schema 会破坏 Round 06 summary hash | **采纳**：父 `trace.mixers` 不动；新建平行 `selector_visits`，64 只作为新 schema alias |
+| “restore to learned” 会被误解成恢复 query/key/softmax autograd | 分母是 detached learned，实验会回答不同问题 | **采纳**：统一改为 “restore to detached-learned value coefficients” |
+| 14-mode 合同没明确保留六个 parent `learned` diagnostics | 可能削弱或破坏 Round 06 exactness | **采纳**：六 checkpoints 跑 parent learned；0/8000 追加 14-mode matrix |
+| 20% raw drop 没写明确分母 | gap gate 可能用不同公式 | **采纳**：冻结 `(X_ref-X_all)/X_ref`，并定义缺失格如何使 3×2 主 gate 失败 |
+| step-0 只做 custom-mode 互比不够 | 同一种 surrogate 错误可全体一致 | **采纳**：双 endpoint exact、learned-vs-detached 负控制、detached `×1/×2` loss-scale gate |
+
+## 3. Non-blocking findings 与处置
+
+全部采纳：
+
+- 统一符号为 `X_ref = X_detached_learned`；
+- 在 protocol 正文列出 exact indices，而不只依赖 scoping 公式；
+- branch dominance 标为 sufficiency-only 次级探索；
+- interaction residual 明确“不预期为 0、不是检验”；
+- 明示 group 7 包含固定 `S` 外的 layers 26–28；
+- 精确定义 `mixed`；
+- replay 限定 same-host environment；
+- 所有 score / gate 只由单一 `analyze.py` 生成。
+
+“单 group 20% 与 joint 50% 不按 mixer 数量成比例”保留为预注册决策；协议已禁止把它解释为
+per-mixer rate。
+
+## 4. 复核结论
+
+Grok 复核 14-mode arithmetic：
+
+```text
+1 reference + 10 sufficiency + 3 restoration = 14
+```
+
+uniform census `0 / 1 / 4 / 8 / 16 / 49 / 57 / 64 / 65` 与 64 depth + 1 output
+拓扑一致。真正风险是 identity/schema，不是计数；本轮修订已把两者写成独立 exact gate。
+
+最终公式被审阅为内部自洽：
+
+```text
+G_X    = ln(X_ref / X_all)
+S_X(m) = ln(X_ref / X_m) / G_X
+R_X(r) = ln(X_r / X_all) / G_X
+```
+
+审稿意见不会进入实验结果、论文事实或官网证据等级；它只用于在结果出现前强化协议。

research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md

@@ -0,0 +1,539 @@
+# K3 Attention Residuals 局部 mixer 路径干预协议
+
+协议 ID：`llm-atlas-k3-attnres-local-path-v1`  
+冻结日期：2026-07-30  
+协议状态：**结果前预注册 frozen；任何语义变更必须更换 protocol ID**  
+父协议：`llm-atlas-k3-attnres-spike-path-v1`
+
+## 0. 研究身份
+
+本轮是 Round 06 结果后的**定向机制追踪**，不是盲发现。
+
+已知：
+
+- depth-32 / Block 的固定尖峰集合是 layers 21–25；
+- 全部 65 个 mixer 的 source value backward coefficients 从 learned 改成 uniform，
+  在三 seed 上 material 地降低最终 `post_mlp_state` 的 spike contrast 和 peak；
+- 该全局干预保持 forward exact；
+- group 6 覆盖 layers 21–24，group 7 覆盖 layers 25–28。
+
+未知：
+
+- 全局下降是否主要集中在 group 6 / 7 的 16 个 depth mixers；
+- group 6 与 group 7 是否各自有稳定 effect；
+- attention 与 MLP mixer 是否能在预注册阈值下区分；
+- output mixer 是否解释了大量全局 effect；
+- 局部 sufficiency 与反向 restoration 是否给出一致证据。
+
+前置证据、拓扑和同期 artifact audit 固定在
+`research/K3_ATTNRES_LOCAL_PATH_SCOPING.md`。任何结果不得倒写成事前未知。
+
+## 1. 允许回答的问题
+
+1. 在相同模型、batch、loss、activation 与 learned forward weights 下，只改变某个固定
+   mixer scope 的 source-gradient coefficients，能复现多少全局 log gap？
+2. 从 all-uniform 背景只恢复 group 6 / 7 的 learned coefficients，能恢复多少
+   global log gap？
+3. groups 6+7 是否在 sufficiency 与 restoration 两个方向、两个 spike 指标、三 seed
+   同时通过 50% 阈值？
+4. 单独 group 6 或 group 7 是否在两个指标、三 seed 通过 20% 阈值？
+5. attention-only 与 MLP-only 是否达到预注册的 branch dominance 规则？
+6. output-only 与 all-depth 控制是否显示 effect 主要来自最终 output mixer？
+
+## 2. 明确不回答的问题
+
+- Kimi K3 2.8T checkpoint 的真实训练梯度；
+- 论文 Figure 5(c) 未公开 telemetry 的精确定义；
+- 哪个 layer、source 或 operator “产生”尖峰；
+- learned source weight 的语义归因；
+- 重新训练局部 uniform variant 的最终能力；
+- intervention effect 的可加性、Shapley value 或方差分解；
+- 三 seed 外的总体显著性、置信区间或 p-value；
+- K3 `A_log` 两个社区修复中哪个已经得到官方认可；
+- checkpoint conversion、推理正确性或部署可用性。
+
+## 3. 冻结训练与数据合同
+
+完整复用 Round 06：
+
+| 字段 | 固定值 |
+|---|---|
+| architecture | Block AttnRes |
+| Transformer depth | 32 |
+| aggregation groups | 8 |
+| blocks / group | 4 |
+| depth mixers / output mixers | 64 / 1 |
+| width / heads / FFN | 192 / 6 / 768 |
+| context / vocabulary | 256 / byte-256 |
+| seeds | 2026073001 / 2026073002 / 2026073003 |
+| steps / batch | 8,000 / 32 |
+| target bytes / formal cell | 65,536,000 |
+| optimizer | AdamW |
+| peak / min LR | 3e-4 / 3e-5 |
+| warmup | 400 |
+| weight decay | 0.1 for ndim ≥ 2 |
+| betas / epsilon | 0.9, 0.95 / 1e-8 |
+| clip | global norm 1.0 |
+| forward | CUDA BF16 autocast |
+| residual accumulation | explicit FP32 |
+| diagnostic CE | fixed 16 × 256 token-mean FP32 CE |
+| parent diagnostic steps | 0 / 100 / 500 / 2,000 / 4,000 / 8,000 |
+| local matrix steps | 0 / 8,000 |
+
+训练路径必须逐调用父 runner 原始 forward；局部 custom autograd 只能存在于 optimizer
+step 外的 diagnostic。数据 bytes、schedule、validation tensor、diagnostic tensor 与三
+seed 的每个 optimizer input 都必须与 Round 06 exact。
+
+诊断调度明确分成两条：
+
+- 六个 parent diagnostic steps 都运行原始 `learned` mode，用于 Round 06 等价；
+- step 0 / 8,000 另外运行下述 14-mode local matrix；
+- localization 公式只读取 14-mode matrix，绝不把 full-autograd `learned` 混入分母。
+
+## 4. 正式矩阵与 replay
+
+正式运行：
+
+```text
+depth-32 / block / seed-2026073001
+depth-32 / block / seed-2026073002
+depth-32 / block / seed-2026073003
+```
+
+另从初始化完整重跑：
+
+```text
+replay / depth-32 / block / seed-2026073001
+```
+
+正式三格处理 196,608,000 target bytes；含 replay 共 262,144,000 bytes。每格使用
+全新 Python 进程。最多并行两个进程；不能共享 model、optimizer、RNG 或 CUDA graph。
+wall-time 不进入数值复现合同。
+
+## 5. 固定主对象与指标
+
+主对象固定为最终 step 8,000：
+
+```text
+position  = post_mlp_state
+reduction = element_rms
+S         = layers 21, 22, 23, 24, 25
+R         = other 27 layers
+```
+
+对 mode `m`：
+
+```text
+C_m = mean(metric[S]) / mean(metric[R])   # spike contrast
+P_m = max(metric) / mean(metric)          # peak normalized
+```
+
+`C_m` 与 `P_m` 必须 finite 且严格大于 `1e-30`。后文统一用 `X_ref` 表示
+`X_detached_learned`；不用 `detached_reference` 等其他别名。不允许用其他位置、reduction、layer
+集合或 metric 替换主对象。32-layer raw spectrum、normalized spectrum、peak layer 和
+top-five layers 全量报告，但不参与主阈值。
+
+## 6. 14 种冻结模式
+
+所有模式调用同一个 parent learned forward。custom Function 的 forward 直接返回
+parent output，只有 backward 对 source tensors 使用选定 coefficients。
+
+14-mode matrix **全部**使用 `RoutedSourceBackward`：每个 mixer 只在
+`stopgrad(w)` 与 `1/N` 两种 source value coefficients 中选择。完整
+query / key / softmax autograd 的 `learned` 不属于这 14 种模式，只用于训练和父诊断
+等价。实现不得把某个 restoration scope 切回 full-autograd `learned`。
+
+### 6.1 reference
+
+`detached_learned`
+
+- 65 个 mixer 全部使用 learned `w` 作为 source value backward coefficients；
+- `w` detach，不走 query / key / softmax derivative path；
+- 必须 exact reproduce Round 06 的同名 mode。
+
+### 6.2 learned 背景上的局部 uniform：sufficiency family
+
+未选 mixer 使用 detached learned coefficients；选中 mixer 使用 `1/N`：
+
+1. `uniform_group_6_only`
+2. `uniform_group_7_only`
+3. `uniform_groups_6_7_only`
+4. `uniform_group_6_attention_only`
+5. `uniform_group_6_mlp_only`
+6. `uniform_group_7_attention_only`
+7. `uniform_group_7_mlp_only`
+8. `uniform_output_only`
+9. `uniform_depth_all`
+10. `uniform_all`
+
+`uniform_all` 必须 exact reproduce Round 06 的 `uniform_value_backward`。
+
+### 6.3 all-uniform 背景上的 detached-learned restoration family
+
+选中 scope 恢复 detached learned coefficients，其余保持 uniform：
+
+1. `uniform_except_group_6`
+2. `uniform_except_group_7`
+3. `uniform_except_groups_6_7`
+
+名字中的 `except` 表示该 scope **不是 uniform**。报告和网站必须同时展示人话标签
+“restore ... to detached-learned value coefficients”，避免误读。
+
+## 7. selector 的唯一合同
+
+为了同时满足新 selector audit 和 Round 06 endpoint exactness，保留两个互不混写的
+schema：
+
+1. 父 `trace.mixers` summary **逐字段不变**；output 仍使用父 schema 的
+   `mixer_index=null`，并继续参与父 `mixer_summary_sha256`；
+2. 新增平行 `selector_visits`，只用于 local mask audit，不写入父 summary。
+
+`selector_visits` 的 depth mixer identity 用：
+
+```text
+(kind="depth", index=0..63, layer=1..32,
+ group=1..8, branch in {"attention","mlp"})
+```
+
+`selector_visits` 的 output mixer identity 用：
+
+```text
+(kind="output", index=64, layer=null, group=null, branch="output")
+```
+
+这里 `index=64` 只是新 selector schema 的稳定别名，不得回写父 summary。
+
+令 `D_i` 表示 `kind=depth,index=i`，`O` 表示 output。14 种 mode 的 uniform identity
+集合冻结如下：
+
+| mode | exact uniform set |
+|---|---|
+| `detached_learned` | `∅` |
+| `uniform_group_6_only` | `{D40,…,D47}` |
+| `uniform_group_7_only` | `{D48,…,D55}` |
+| `uniform_groups_6_7_only` | `{D40,…,D55}` |
+| `uniform_group_6_attention_only` | `{D40,D42,D44,D46}` |
+| `uniform_group_6_mlp_only` | `{D41,D43,D45,D47}` |
+| `uniform_group_7_attention_only` | `{D48,D50,D52,D54}` |
+| `uniform_group_7_mlp_only` | `{D49,D51,D53,D55}` |
+| `uniform_output_only` | `{O}` |
+| `uniform_depth_all` | `{D0,…,D63}` |
+| `uniform_all` | `{D0,…,D63,O}` |
+| `uniform_except_group_6` | `{D0,…,D39,D48,…,D63,O}` |
+| `uniform_except_group_7` | `{D0,…,D47,D56,…,D63,O}` |
+| `uniform_except_groups_6_7` | `{D0,…,D39,D56,…,D63,O}` |
+
+runner 必须把这些 set 编码为一个 frozen selector 函数；不能散落在 mode-specific
+if/else 中。manifest 同时保存 machine-readable exact index lists。runner 通过
+override `_mix` 或等价 hook 做 set lookup，并替换父 runner 中只接受三种 global mode
+的 mode validation、bundle loop 和相关 gate；训练 forward 继续直接调用父路径。
+
+每次 forward 必须验证：
+
+- exactly 65 个 mixer visits；
+- identity 不重复；
+- identity 顺序与 reference exact；
+- 父 `trace.mixers` schema 与 hash 路径没有新字段；
+- uniform census 与 manifest exact；
+- selected identity list 与 selector rule exact；
+- reference 的 uniform count 为 0；
+- branch-only 4，group-only 8，groups 6+7 为 16；
+- output-only 1，all-depth 64，all 65；
+- except-one-group 57，except-two-groups 49。
+
+任一 gate 失败，cell invalid；不得只改结果 JSON。
+
+## 8. forward identity 与 parent exactness
+
+### 8.1 所有 14 模式的 forward identity
+
+同 seed / step 相对 `detached_learned` 必须满足：
+
+- logits tensor SHA-256 exact；
+- loss FP32 value exact；
+- 六位置 activation tensor hashes exact；
+- 65 个 mixer forward summaries exact。
+
+任一 mode 失败，整格 invalid。
+
+### 8.2 Round 06 endpoint exactness
+
+对 step 0 / 8,000：
+
+- `detached_learned` 的 logits、loss、六位置 activation、未改 schema 的 mixer
+  summaries 和六位置 gradient reductions 必须与对应 Round 06
+  raw output exact；
+- `uniform_all` 的同一组字段和六位置 gradient reductions必须与对应 Round 06
+  `uniform_value_backward` exact；
+- 正式训练的 final model hash、optimizer hash、六个 validation BPC、training
+  history 与六个 parent `learned` diagnostics 必须与 Round 06 exact；
+- 新 `selector_visits` 不参与旧 `mixer_summary_sha256`，而由独立 canonical hash
+  和 exact-set gate 管理。
+
+runner / protocol / scoping / manifest 物理 hash 在运行前冻结。父 raw 文件同时检查 physical
+SHA-256、canonical SHA-256、final model hash 和 final optimizer hash。
+
+## 9. 初始化负控制
+
+step 0 的 mixer query 为零，learned `w` 是 uniform。14 模式在六个位置的
+`element_rms` 必须：
+
+- 32 个 raw values 全部 finite、strictly positive；
+- 相对 reference 的逐层 raw relative error `≤1e-6`；
+- normalized absolute error `≤1e-6`。
+
+此外：
+
+- `detached_learned` 与 `uniform_all` 必须分别与 Round 06 step-0 endpoint exact；
+- parent full-autograd `learned` 与 `detached_learned` 必须按 Round 06 负控制在
+  `1e-6` tolerance 内一致；
+- `detached_learned` 另执行同一 loss 的 `×1 / ×2` backward，六位置、七 reductions
+  都必须通过父协议相同的 scale 与 normalized-spectrum gate。
+
+失败表示 selector 或 surrogate 没有隔离预期路径；正式结果无效。
+
+## 10. global log gap
+
+对每个 seed 和每个指标 `X ∈ {C,P}`：
+
+```text
+G_X = ln(X_ref / X_uniform_all)
+relative_drop_X = (X_ref - X_uniform_all) / X_ref
+```
+
+只有同时满足以下条件才允许解释局部比例：
+
+1. `G_C > 0` 且 `G_P > 0`；
+2. 上式 `relative_drop_X ≥0.20`；
+3. Round 06 endpoint exactness 通过。
+
+seed `s` 的 metric `X` 任一条件不满足，则该 `(s,X)` 称为
+`global gap not established`，不计算该格 `S_X / R_X`。groups 6+7 的主 gate 要求
+3 seed × 2 metrics 全部存在，因此任一 required cell 缺失都会使主 localization
+判定失败；仍公开 raw matrix，不使用事后替代分母。
+
+log ratio 用于让相同的乘法变化在两个方向可比。所有归一化值按原值报告，**不裁剪到
+[0,1]**；负值表示反方向，超过 1 表示局部 intervention 超过 all-uniform endpoint。
+
+## 11. sufficiency score
+
+对 sufficiency mode `m`：
+
+```text
+S_X(m) = ln(X_ref / X_m) / G_X
+```
+
+### 11.1 groups 6+7 主判定
+
+只有 `uniform_groups_6_7_only` 对 `C` 和 `P` 都满足：
+
+```text
+S_X(m) ≥ 0.50
+```
+
+且三个 formal seed 6 / 6 全部达标，才记为：
+
+> groups 6+7 的 16 个 depth mixers 在本 diagnostic 中，足以复现至少一半
+> all-65 uniform intervention 的预注册 log-gap reduction。
+
+任一失败记为 `not sufficient at the preregistered 50% threshold`。`mixed` 精确定义为：
+seed 通过/失败不一致、`C/P` 通过/失败不一致，或 score 的正负号跨 seed 不一致；可同时
+附加多个原因，不得降低阈值。
+
+### 11.2 单 group
+
+group 6 / group 7 分别对 `C` 和 `P`、三 seed 全部满足：
+
+```text
+S_X(m) ≥ 0.20
+```
+
+才称为 `material local sufficiency at the 20% threshold`。没过阈值不等于 effect 为零。
+
+## 12. restoration score
+
+对 restoration mode `r`：
+
+```text
+R_X(r) = ln(X_r / X_uniform_all) / G_X
+```
+
+### 12.1 groups 6+7 主判定
+
+只有 `uniform_except_groups_6_7` 对 `C` 和 `P`、三 seed全部满足：
+
+```text
+R_X(r) ≥ 0.50
+```
+
+才称为：
+
+> 从 all-uniform 背景只恢复 groups 6+7 的 learned coefficients，恢复了至少一半
+> 预注册 global log gap。
+
+这仍是同前向 backward-rule restoration sensitivity，不是严格 causal necessity。
+
+### 12.2 单 group
+
+`uniform_except_group_6` / `uniform_except_group_7` 分别以 `≥0.20`、两个指标、三 seed
+作为 material restoration threshold。
+
+## 13. localization 总闸门
+
+只有以下两项同时通过：
+
+1. groups 6+7 sufficiency：`S_C,S_P ≥0.50`，3 / 3 seeds；
+2. groups 6+7 restoration：`R_C,R_P ≥0.50`，3 / 3 seeds；
+
+才允许写：
+
+> 在本缩小模型、固定训练状态和 diagnostic backward 下，全局 value-coefficient
+> sensitivity 的主要部分 localization 到 group 6 / 7 mixer path。
+
+即使通过，也必须紧邻注明：
+
+- “主要部分”由 50% 双向阈值定义；
+- effect non-additive；
+- 不是唯一来源或 layer-origin；
+- 不是真实 K3 checkpoint 结论。
+
+一侧通过一侧失败，统一写成 `one-sided evidence, localization not established`。
+
+## 14. attention vs MLP branch 判定
+
+每个 group 独立比较 attention-only 与 MLP-only sufficiency score。只有某 branch：
+
+1. `S_C ≥0.20` 且 `S_P ≥0.20`；
+2. 在 `C` 与 `P` 上都比 sibling 高至少 `0.15`；
+3. 三 seed 全部满足前两项；
+
+才称为 `branch-dominant at the preregistered margin`。
+
+若 group-level sufficiency 未通过 20% 阈值，不允许宣称其内部 branch dominance。
+branch-only scores 可能交互、超加或相互抵消，不能相加成 group score。
+本节只有 sufficiency 方向，没有 branch-level restoration，属于预注册的次级探索性
+判定，证据层级低于 §13 双向 localization。
+
+## 15. output 与 depth 控制
+
+`uniform_output_only` 和 `uniform_depth_all` 不进入 group localization 主判定。
+
+探索性报告：
+
+```text
+S_X(output)
+S_X(depth_all)
+interaction_residual_X =
+  1 - S_X(output) - S_X(depth_all)
+```
+
+`interaction_residual` 只是 log-gap bookkeeping，不是统计交互估计或贡献分解。
+它不预期接近 0，也不是 hypothesis test。
+
+只有 output-only 对两个指标、三 seed 都 `≥0.50`，才标记
+`output mixer alone captures at least half the global gap`。即使如此，也不否定
+groups 6+7；两者可能重叠、串联或超加。
+
+## 16. 报告顺序与反 cherry-picking
+
+固定报告顺序：
+
+1. input / parent / endpoint exactness；
+2. step-0 negative control；
+3. 每 seed 的 raw `C` / `P` 矩阵；
+4. global gaps；
+5. groups 6+7 sufficiency；
+6. groups 6+7 restoration；
+7. localization gate；
+8. single-group scores；
+9. branch scores；
+10. output / depth controls；
+11. full 32-layer spectra；
+12. replay；
+13. limitations。
+
+所有 14 modes、两个指标、三个 seed 都公开。不得只展示通过阈值的 scope。不得用跨 seed
+均值替代 3 / 3 gate；均值只用于视觉摘要。
+
+## 17. replay 与复现闸门
+
+seed 2026073001 从初始化独立 replay，比较去除以下字段后的 canonical content：
+
+- `run_kind`；
+- wall-clock timing；
+- output path；
+- self canonical hash。
+
+至少以下字段必须 exact：
+
+- input tensor hashes；
+- initial/final model 与 optimizer hashes；
+- evaluations / training history；
+- parent diagnostics；
+- 14-mode step-0 / step-8,000 forward hashes；
+- selector census / identities；
+- 六位置 raw gradient reductions；
+- global gaps / local scores / gates。
+
+若正式 seed1 与 replay 不 exact，Round 07 数值结论无效。
+replay 固定在与 formal 相同 host、GPU、Python、PyTorch、CUDA 和
+`CUBLAS_WORKSPACE_CONFIG` 环境；本协议不声称跨硬件 bit exact。
+
+## 18. 预期失败与停止规则
+
+以下任一项使 cell invalid：
+
+- CUDA deterministic contract 未开启；
+- parent manifest / runner / protocol / scoping / raw hash 不匹配；
+- 训练等价失败；
+- diagnostic 改变 optimizer state；
+- forward identity 失败；
+- selector identity / census 失败；
+- Round 06 endpoint exactness 失败；
+- step-0 negative control 失败；
+- raw gradient missing、non-finite 或 non-positive；
+- global gap denominator 不成立。
+
+程序错误修复必须：
+
+1. 保存失败日志；
+2. 修改 runner；
+3. 更新 runner hash；
+4. 明确判断协议语义是否改变；
+5. 若改变 selector、mode、metric、threshold 或 aggregation，创建新 protocol ID；
+6. 全部受影响 cell 从初始化重跑。
+
+## 19. 结果语言边界
+
+允许：
+
+- “在同前向 diagnostic backward 下，uniformizing scope X 改变了固定尖峰指标”；
+- “groups 6+7 在预注册 50% 双向阈值下建立 / 未建立 localization”；
+- “branch effect mixed / below threshold”；
+- “这是一项 reduced-model mechanism probe”。
+
+禁止：
+
+- “证明 K3 的尖峰来自第 6 组”；
+- “这些 mixer 贡献了 X% 梯度”；
+- “group effect 加总为 100%”；
+- “uniform mixer 更适合训练”；
+- “复现了 Figure 5(c)”；
+- “验证了 K3 2.8T checkpoint”；
+- “社区 PR #144 或 #150 已成为官方修复”。
+
+## 20. 冻结清单
+
+在任何 formal 结果产生前必须完成：
+
+- [x] scoping 文件完成；
+- [x] protocol 状态改为 frozen；
+- [ ] 14 modes 与 selector census 写入 manifest；
+- [ ] thresholds / formulas 写入 manifest；
+- [ ] Round 06 父 artifact physical / canonical hashes 写入 manifest；
+- [ ] runner、protocol、scoping、manifest hashes 固定；
+- [x] Grok Headless 对抗审阅完成，采纳/拒绝理由留档；
+- [ ] step-0 smoke 全门通过；
+- [ ] formal 命令与环境写入 README；
+- [ ] 单一 `analyze.py` 实现所有 score / gate，网站只消费其冻结输出；
+- [ ] protocol commit 早于 formal result commit。

research/K3_ATTNRES_LOCAL_PATH_SCOPING.md

@@ -0,0 +1,197 @@
+# K3 Attention Residuals 局部 mixer 路径：Round 07 前置定位
+
+研究日期：2026-07-30  
+阶段身份：**定向 scoping，不是 Round 07 预注册结果**  
+上游协议：`llm-atlas-k3-attnres-spike-path-v1`
+
+## 1. 已知到什么程度
+
+Round 06 在同一个 depth-32 / Block AttnRes 缩小模型上，把前向保持为 learned
+weights，只改写 mixer 的反向规则。三 seed 的最终 `post_mlp_state / element_rms`
+结果为：
+
+| backward rule | spike contrast（3-seed mean） | 相对 detached learned | peak normalized（3-seed mean） | 相对 detached learned |
+|---|---:|---:|---:|---:|
+| learned | 2.754 | — | 4.803 | — |
+| detached learned | 2.812 | reference | 4.956 | reference |
+| uniform value / all 65 mixers | 0.837 | **−70.2%** | 3.122 | **−37.0%** |
+
+其中：
+
+- `learned → detached learned` 没有降低尖峰，contrast 反而平均增加约 2.0%；
+- `detached learned → uniform value backward` 在 contrast 和 peak 上都 3 / 3 seed
+  超过预注册的 20% material threshold；
+- 三种模式的 logits、loss、六位置 activation 和 mixer forward 摘要全部 exact；
+- 这证明的是**全局 backward-rule sensitivity**，不是训练变体，也不是局部归因。
+
+因此 Round 07 不再重复问“learned value coefficients 是否重要”，而是问：
+
+> 65 个 mixer 全局改写带来的下降，主要能否由尖峰邻近的 group 6 / 7
+> depth mixers 复现，并能否从反方向恢复？
+
+## 2. 固定拓扑，而不是结果后挑层
+
+depth 32 的 Block AttnRes 有 8 个 aggregation groups，每组 4 个 Transformer
+blocks。每层有 attention 和 MLP 两个 depth mixers，合计 64 个；模型末尾还有一个
+独立 output mixer，合计 65 个 intervention nodes。
+
+对 1-based layer `l`：
+
+```text
+group                 = floor((l - 1) / 4) + 1
+attention mixer index = 2 × (l - 1)       # 0-based
+MLP mixer index       = 2 × (l - 1) + 1   # 0-based
+```
+
+所以：
+
+| scope | layers | 0-based depth mixer indices | mixer count |
+|---|---:|---:|---:|
+| group 6 | 21–24 | 40–47 | 8 |
+| group 7 | 25–28 | 48–55 | 8 |
+| groups 6+7 | 21–28 | 40–55 | 16 |
+| output | — | separate node | 1 |
+
+Round 05 已在看过数据后冻结尖峰集合 `S = layers 21–25`。它覆盖完整 group 6 和
+group 7 的首层。因此 Round 07 明确是**定向邻域追踪**，不能称为盲发现；group 6 / 7
+也不能结果后替换成更好看的范围。
+
+## 3. 为什么需要两个方向
+
+只在 learned 背景把 group 6 / 7 改成 uniform，回答的是：
+
+> 只改这段是否足以复现全局干预的一大部分下降？
+
+但 mixer 路径有串联、分流和 nonlinear interaction，单侧结果可能被其他 learned
+路径补偿。反过来，在 all-uniform 背景只把 group 6 / 7 恢复为 learned，回答的是：
+
+> 只恢复这段是否足以让尖峰朝 reference 回升？
+
+两种值都不是“贡献百分比”，也不要求相加为 100%。Round 07 用相同的 global log gap
+归一化两种方向，只把双向、跨 seed 稳定的结果称为 localization evidence。
+
+## 4. 冻结候选范围
+
+从 `detached_learned` 背景出发的 sufficiency scopes：
+
+1. group 6；
+2. group 7；
+3. groups 6+7；
+4. group 6 attention-only；
+5. group 6 MLP-only；
+6. group 7 attention-only；
+7. group 7 MLP-only；
+8. output-only；
+9. all 64 depth mixers；
+10. all 65 mixers。
+
+从 all-uniform 背景出发的 restoration scopes：
+
+1. restore group 6 to detached-learned value coefficients；
+2. restore group 7 to detached-learned value coefficients；
+3. restore groups 6+7 to detached-learned value coefficients。
+
+加上 `detached_learned` reference，共 14 种模式。预期 uniform selector census 为：
+
+| mode | uniform mixers |
+|---|---:|
+| detached reference | 0 |
+| group branch only | 4 |
+| group only | 8 |
+| groups 6+7 | 16 |
+| output only | 1 |
+| all depth | 64 |
+| all | 65 |
+| uniform except group 6 / 7 | 57 |
+| uniform except groups 6+7 | 49 |
+
+每次 diagnostic 都必须保存实际选中的 mixer identity；不能只信 mode 名称。
+group 7 的 scope 包含 layers 26–28，它们不在固定尖峰集合 `S=21–25` 中，所以
+group 7 是预先定义的**完整邻接 group intervention**，不是 spike-layer-only
+intervention。
+
+## 5. 不把局部 intervention 误译成什么
+
+即使 groups 6+7 双向通过，结论也只限于：
+
+- 固定训练状态；
+- 固定 diagnostic batch 与 loss；
+- 固定 `S = 21–25` 指标；
+- 同前向、替代 source-gradient coefficient 的 diagnostic backward。
+
+它不等于：
+
+- 这些层“产生”了尖峰；
+- group 6 / 7 是唯一原因；
+- 真实 K3 checkpoint 有相同梯度路径；
+- 把 mixer 训练成 uniform 会有同样结果；
+- 局部 effect 可加，或可解释成方差分解；
+- 论文 Figure 5(c) 的未公开 telemetry 已被复现。
+
+保留 output-only 和 all-depth 两个控制，是为了看清最终 readout 与 depth path 的关系；
+它们不进入 group 6 / 7 localization 的主判定。
+
+## 6. 同期 artifact 状态审计：`A_log`
+
+这一问题与缩小实验的局部梯度机制**相互独立**，但会限制任何真实 K3 checkpoint
+验证，因此在冻结 Round 07 前重新检查官方模型仓库。
+
+截至 **2026-07-30 12:35 CST**：
+
+- 官方 Hugging Face main commit 仍为
+  `9f62e4e9fffbd0a83ddd60e1c209d828994b3569`；
+- main 的 `modeling_kimi_linear.py` 仍以 `num_heads=96` 初始化 `A_log`；
+- 已发布 checkpoint 中该张量的公开 shape 是 `[128]`，与 main 存在加载不匹配；
+- 官方 main 尚未合并修复或给出 conversion contract。
+
+同时出现了两个**未合并、互相竞争的社区 PR**：
+
+### PR #144：把参数改成 128
+
+- 一行把初始化从 `self.num_heads` 改为 `self.head_dim`；
+- 提交者报告所有 shards 能加载；
+- 提交者明确说没有独立验证 forward；
+- 它把 checkpoint shape 当作权威语义。
+
+### PR #150：保留 96，加载时验证并裁零尾
+
+- 保持模型参数为 `[num_heads]=[96]`；
+- `_load_from_state_dict` 检查 `[96:128]` 全为零后再裁掉；
+- 提交者报告检查了 69 个 KDA 层，所有 32 项尾部都 exact zero；
+- 提交者还报告经过 disk-offloaded MoE 的完整生成；
+- 这些 checkpoint 全量扫描和生成是**提交者报告**，本项目没有下载约 1.56 TB
+  权重独立复核；本项目只核对了 PR diff、main 代码路径和 PR 状态。
+
+PR #150 进一步指出，forward 中 `v` 被 reshape 为 96 heads，kernel 随后接收
+`A_log`；若直接采用 #144 的 128 元素参数，现有 `view(H, 1)` 路径会在 96 heads 下
+失败。这个论证比单看 checkpoint shape 更完整，但在官方合并或独立复核前，仍必须标成
+高可信社区解释，而不是 Kimi 官方结论。
+
+当前准确状态应写成：
+
+> official main 仍然不匹配；社区已有两个竞争性候选修复，其中 #150 提供了更完整的
+> checkpoint-tail 与 end-to-end 证据，但尚无官方裁决。
+
+来源：
+
+- [Kimi-K3 official main](https://huggingface.co/moonshotai/Kimi-K3/tree/main)
+- [main `modeling_kimi_linear.py`](https://huggingface.co/moonshotai/Kimi-K3/blob/main/modeling_kimi_linear.py)
+- [community PR #144](https://huggingface.co/moonshotai/Kimi-K3/discussions/144)
+- [community PR #150](https://huggingface.co/moonshotai/Kimi-K3/discussions/150)
+
+## 7. Round 07 的可证伪问题
+
+Round 07 将：
+
+1. exact replay Round 06 的三 seed 训练；
+2. exact reproduce Round 06 的 `detached_learned` 与 `uniform_all` 两个端点；
+3. 在 step 0 对 14 种模式做负控制，在 step 8,000 做正式矩阵；
+4. 同时测 `spike_contrast` 与 `peak_normalized`；
+5. 用 groups 6+7 的 sufficiency 与 restoration 两个方向预注册 50% log-gap
+   localization threshold；
+6. 用单 group 的 20% 阈值和 attention-vs-MLP 的 15 percentage-point margin
+   作更细分的层级判定；
+7. 从初始化完整 replay seed 2026073001。
+
+完整模式、公式、失败规则和复现合同见
+`research/K3_ATTNRES_LOCAL_PATH_PROTOCOL.md`。

research/K3_ATTNRES_REDUCED_AUDIT.md

@@ -0,0 +1,584 @@
+# Kimi K3 第四轮：Attention Residuals 缩小版独立机制实验审计
+
+> 协议：`llm-atlas-k3-attnres-reduced-v1`
+>
+> 预注册：`research/K3_ATTNRES_REDUCED_PROTOCOL.md`
+>
+> 数据清单：`experiments/k3/attnres/manifest.json`
+>
+> 执行日期：2026-07-30
+>
+> 执行设备：NVIDIA GeForce RTX 5090；PyTorch `2.11.0+cu128`
+
+## 0. 先说结论
+
+这一轮没有伪装成“跑通了 K3”。它做的是一个刻意缩小、从零训练、可公开复查的
+Attention Residuals（AttnRes）机制实验：
+
+```text
+3 个结构
+× 3 个预先冻结的初始化 seed
+× 每格 2,000 steps
+× 每步 32 × 256 target bytes
+= 9 个正式训练格，147,456,000 target bytes
+```
+
+三个结构共享完全相同的 16 个 Transformer blocks、32 个残差子层、初始公共参数、
+训练窗口、优化器与验证集。唯一设计变量是“前面产生的 residual states 怎样供下一层读取”：
+
+```text
+Baseline：只读上一残差状态
+Full AttnRes：对全部历史残差状态做按维 softmax 混合
+Block AttnRes：每 4 个残差子层形成一个局部块，块间再混合
+```
+
+在冻结的 64 个验证窗口上，最终 bits per byte（BPC，越低越好）为：
+
+| seed | Baseline | Full | Block | Full − Base | Block − Base |
+|---:|---:|---:|---:|---:|---:|
+| 2026073001 | 2.00054 | 1.98454 | 1.94788 | −0.01600 | −0.05266 |
+| 2026073002 | 1.99844 | 1.98523 | 1.95709 | −0.01321 | −0.04135 |
+| 2026073003 | 1.99842 | 1.98392 | 1.96503 | −0.01450 | −0.03339 |
+| 三 seed 均值 | 1.99913 | 1.98457 | 1.95667 | **−0.01457** | **−0.04247** |
+
+按照看结果前冻结的判据——三个 seed 同为负，并且均值不高于 `−0.010 BPC`——Full 与
+Block 都得到：
+
+> **directional support in this reduced protocol**
+
+中文应该读成：
+
+> 在这套缩小训练合同中，允许子层重新读取更早的残差状态，方向一致地改善了验证 BPC。
+
+它**不应该**读成：
+
+- 已复现 AttnRes 论文的大模型收益；
+- 已运行 Kimi K3 checkpoint；
+- 已证明 Block 一般优于 Full；
+- 已得到同参数、同 FLOPs 或同 wall time 的优势；
+- 三个 seed 可以支持总体显著性、置信区间或 scaling-law 外推。
+
+本轮还有一个同样重要的反结果：预注册的“16 个 Transformer blocks 的核心参数梯度 RMS
+变异系数”在本实验中，Baseline 为 `0.3447`，Full 为 `0.5087`，Block 为 `0.6306`。
+在这个定义和尺度下，AttnRes **没有**表现出更均匀的跨深度梯度。这个结果与论文的大模型
+梯度叙述不能直接对齐，网站必须把它作为边界而不是藏起来。
+
+---
+
+## 1. 为什么不是直接跑 K3
+
+### 1.1 公开 checkpoint 的未决形状冲突
+
+第三轮工件审计在 K3 第一层 KDA 中发现：
+
+| 官方工件 | `A_log` 所要求或实际给出的形状 |
+|---|---|
+| `config.json` | `num_heads = 96` |
+| Hugging Face remote code | 按 `num_heads` 构造，即 `[96]` |
+| FlashKDA kernel API | `[H]`，K3 中应为 `[96]` |
+| checkpoint safetensors header | `[128]` |
+
+本轮再次检查了当前官方 Hugging Face / GitHub 工件，并额外检查当前 vLLM 与 SGLang
+Kimi K3 loader。两者都按 local head 维构造和切分 `A_log`，没有公开 `128 → 96` 的转换规则。
+
+因此，以下做法都会越过证据：
+
+```text
+裁掉最后 32 个值
+把 128 强行 reshape 成别的语义
+把 128 解释成 head_dim
+绕过 loader 后把输出叫作“K3 forward”
+```
+
+在 Moonshot 给出转换合同、修订权重，或一个官方 loader 明确处理这 32 个额外值之前，
+本站不制造“真实 K3 前向结果”。
+
+### 1.2 为什么缩小版仍有价值
+
+不能诚实执行 1.56 TB checkpoint，不等于只能停在架构示意图。AttnRes 的核心问题可以被
+缩成一个更小、但仍可被证伪的问题：
+
+> 在相同 Transformer 主干、相同输入窗口与相同初始化下，把“固定单位 residual
+> connection”替换成“学习的历史 residual 混合”，短预算训练是否出现一致方向？
+
+这个问题不依赖 KDA、MLA、MoE、MXFP4、视觉塔或完整 K3 参数。它只检验 AttnRes
+的局部机制方向，并且可以把全部代码、清单、聚合指标和复现哈希开源。
+
+---
+
+## 2. 先冻结什么
+
+首版协议在正式结果产生前冻结以下项目：
+
+- 数据仓库、revision、config、split 拼接规则与 byte tokenizer；
+- 模型层数、宽度、head 数、FFN 宽度、位置编码、RMSNorm 与激活函数；
+- Baseline / Full / Block 的精确定义；
+- 三个初始化 seed；
+- 每格训练步数、batch、context 与总 target bytes；
+- AdamW、学习率曲线、weight decay、gradient clipping；
+- 验证步、固定验证窗口与主指标；
+- 主对比、方向判据和“不能宣称什么”；
+- 深度 RMS、混合权重与梯度诊断；
+- smoke 与正式独立进程 replay 合同。
+
+正式协议不是从本轮曲线倒推的。关键提交顺序为：
+
+```text
+9039de1  research: preregister reduced AttnRes study
+e361753  research: lock reduced AttnRes model contract
+f998720  research: add reduced AttnRes runner
+1f20f81  research: freeze reduced AttnRes corpus
+```
+
+### 2.1 一次被公开保留的预训练故障
+
+第一次 smoke 在 step 0 的验证 forward 后停止：
+
+```text
+RuntimeError:
+view size is not compatible with input tensor's size and stride
+```
+
+原因是切片后的 target tensor 不连续，而 loss 路径用了 `.view()`。当时：
+
+- 尚未执行一个 optimizer step；
+- 没有正式结果文件；
+- 没有任何条件的训练或最终 BPC 可供选择。
+
+修复仅把 `.view()` 改为语义等价且支持非连续输入的 `.reshape()`：
+
+```text
+5f49906  fix: flatten noncontiguous AttnRes targets
+```
+
+随后三个结构分别完成两次独立 smoke，冻结字段逐字段 exact。这个故障不改变实验设计，
+但应留在审计链中，避免“第一次就完美运行”的虚假叙事。
+
+---
+
+## 3. 数据合同：文本怎样变成 byte 任务
+
+### 3.1 固定数据源
+
+| 项目 | 固定值 |
+|---|---|
+| repository | `Salesforce/wikitext` |
+| revision | `b08601e04326c79dfdd32d625aee71d232d685c3` |
+| config | `wikitext-2-raw-v1` |
+| 行处理 | `(text or "") + "\n"` |
+| 编码 | UTF-8 |
+| tokenizer | byte ID `0..255` |
+| vocabulary | 256 |
+
+拼接后的 split：
+
+| split | bytes | SHA-256 |
+|---|---:|---|
+| train | 10,951,563 | `0ca7d3e74dbe44564ea5942b85232f1bbcb525c9cd481cd5d28a87ee90e7e9b4` |
+| validation | 1,148,008 | `a42356f6a8ff1d25daf25ec9db49e10a537c265581b61c74604bb63231dee719` |
+| test | 1,292,014 | `bfe9eb16ab9987fb88bde4ea9a30a00f2a45db01dfc14bad78d05325789c4f12` |
+
+这里的 byte tokenizer 不是为了追求最佳语言模型性能，而是为了移除另一个潜在变量：
+不同 BPE 模型、词表和 normalization。BPC 也因此可以直接比较，而不受 tokenization
+长度变化影响。
+
+### 3.2 无状态窗口计划
+
+训练窗口不靠进程内 RNG 顺序产生。对每个：
+
+```text
+architecture + seed + step + batch row
+```
+
+协议用 SHA-256 派生 train start offset。三种结构在同一 seed 下使用相同窗口；结构名不进入
+窗口选择的有效随机盐。冻结计划包含 192,000 个起点，SHA-256 为：
+
+```text
+81521a70ec61f3717968f160cb711e50c5f52a665a6961538d339360cb695f48
+```
+
+固定 64 个验证窗口 tensor hash：
+
+```text
+5f71fda757fc75010ed16e7636bc394c69f55b34a3713b3b5a7ef8e03eae3c20
+```
+
+固定 16 个诊断窗口 hash：
+
+```text
+d970af9b0c656c9826f369b5fe6e3869a6f6cfeccfa5a922fe94ed1d24b86818
+```
+
+---
+
+## 4. 模型合同：只改变 residual 读取拓扑
+
+### 4.1 公共主干
+
+| 项目 | 值 |
+|---|---:|
+| Transformer blocks | 16 |
+| residual sublayers | 32（每 block attention + FFN） |
+| model width | 192 |
+| attention heads | 6 |
+| head dimension | 32 |
+| SwiGLU hidden | 768 |
+| context | 256 bytes |
+| dropout | 0 |
+| position | learned absolute embedding |
+| norm | pre-RMSNorm + final RMSNorm |
+| attention softmax | causal, float32 |
+| embedding / LM head | tied |
+
+Attention projection 与 SwiGLU 都不使用 bias。三个结构的公共 core 参数均为
+`9,541,824`，每个 seed 的公共参数初始化哈希在三种结构间 exact。
+
+### 4.2 Baseline
+
+普通 residual 子层：
+
+```text
+x_(l+1) = x_l + F_l(RMSNorm(x_l))
+```
+
+它只保留一个随深度持续累积的 residual stream。
+
+### 4.3 Full AttnRes
+
+第 `l` 个子层先对从 embedding 到当前深度的全部 residual states 做学习混合：
+
+```text
+α_l = softmax(q_l · RMSNorm(states))
+x̃_l = Σ_i α_(l,i) state_i
+state_(l+1) = F_l(RMSNorm(x̃_l))
+```
+
+`q_l` 是按 hidden dimension 学习的 pseudoquery，初始化为 0，所以初始 softmax 为均匀
+读取。最后还有一个 output mixer，把 33 个可见 sources 混成 LM head 的输入。
+
+### 4.4 Block AttnRes
+
+32 个 residual sublayers 被分成 8 块，每块 4 层：
+
+```text
+块内：新 branch state 做普通局部累加
+块边界：对历史块状态做学习混合，产生下一块输入
+```
+
+它保留“可以回读历史”的机制，同时把 Full 随深度增长的状态集合限制在块级。
+
+### 4.5 参数公平与计算不公平
+
+| 结构 | core | mixer | total | 相对 Baseline mixer overhead |
+|---|---:|---:|---:|---:|
+| Baseline | 9,541,824 | 0 | 9,541,824 | 0 |
+| Full | 9,541,824 | 12,672 | 9,554,496 | 0.1328% |
+| Block | 9,541,824 | 12,672 | 9,554,496 | 0.1328% |
+
+这是近似同参数，不是同 FLOPs。当前教学实现用 PyTorch eager 保存和混合历史 states，
+没有使用论文的大模型优化 kernel，所以它适合机制观察，不适合推断生产吞吐。
+
+---
+
+## 5. 训练与主指标
+
+### 5.1 每格预算
+
+```text
+2,000 steps
+× batch 32
+× context 256 target bytes
+= 16,384,000 target bytes / run
+```
+
+优化器：
+
+```text
+AdamW β=(0.9, 0.95), ε=1e-8
+peak LR=3e-4, min LR=3e-5
+100-step warmup + cosine decay
+weight decay=0.1 for ndim>=2
+global grad clip=1.0
+BF16 autocast
+```
+
+验证发生在 `0, 100, 250, 500, 1000, 1500, 2000` steps。主指标只使用 step 2000
+的固定验证 BPC；曲线用于帮助理解，不用于重新选择终点。
+
+### 5.2 判据为什么这么保守
+
+只有三个 seed，不能可靠估计总体方差或给出有意义的 population confidence interval。
+所以预注册不用 p-value，而只问两个简单问题：
+
+```text
+三个 paired deltas 是否同方向？
+mean delta 是否至少达到 0.010 BPC？
+```
+
+如果答案都是“是”，只写作本协议内的 directional support / concern。它是一道防止
+夸大结论的阈值，不是一个通用显著性标准。
+
+### 5.3 正式结果
+
+Full 的三组配对差：
+
+```text
+−0.015996
+−0.013207
+−0.014498
+mean = −0.014567 BPC
+```
+
+Block 的三组配对差：
+
+```text
+−0.052660
+−0.041349
+−0.033388
+mean = −0.042466 BPC
+```
+
+两组都满足预注册的方向支持规则。Block 相对 Full 的均值差为 `−0.027898 BPC`，三个
+seed 也同为负；但“Block − Full”不是预注册主判据，而且本实现的计算图与优化效率不同，
+所以它只适合描述，不升级成一般性排名。
+
+---
+
+## 6. 它花了多少计算与显存
+
+排除每次前 20 个计时 warmup steps 后，三个 seed 的均值：
+
+| 结构 | mean step | 相对 Baseline | peak allocated | 相对 Baseline |
+|---|---:|---:|---:|---:|
+| Baseline | 21.62 ms | 1.00× | 3.04 GB | 1.00× |
+| Full | 146.81 ms | 6.79× | 14.26 GB | 4.70× |
+| Block | 54.63 ms | 2.53× | 6.52 GB | 2.15× |
+
+这张表应该怎样读：
+
+- Full 在教学实现中为每层保留并读取更多历史 states，因此最贵；
+- Block 把可见历史限制在块级，成本明显下降；
+- 时间与显存是“这份 PyTorch eager 实现 + RTX 5090”的观测；
+- 不能把 6.79× / 2.53× 外推到论文 kernel、K3 训练系统或生产推理；
+- BPC 改善不能被写成同 FLOPs 改善。
+
+---
+
+## 7. 看 residual stream：Block 为什么呈现锯齿
+
+Baseline 的 stream-state RMS 从平均 `0.0661` 增到 `0.2110`：普通 residual
+connection 把分支输出一路累积。
+
+Block 的前 12 个 partial-state RMS 是：
+
+```text
+0.0334  0.0598  0.0967  0.1181
+0.0509  0.0569  0.0745  0.0815
+0.0532  0.0581  0.1079  0.1158
+```
+
+每四个值形成一个局部块：
+
+```text
+块内：逐层累积，RMS 通常上升
+块间：重新从历史块状态混合，partial stream 被重置
+```
+
+所以曲线不是训练不稳定造成的随机锯齿，而是 Block 拓扑的直接几何痕迹。完整 32 点向量、
+三个 seed 的 min / mean / max 均进入公开 JSON。
+
+Full 的 layer-input RMS 则从 `0.0544` 降到 `0.00866`。这不等于信息“消失”：
+每层输入是多个经过 RMSNorm 的历史 states 的学习加权和，混合可以通过方向抵消改变
+合成向量的 RMS。只看单个标量不能判断信息保留量。
+
+---
+
+## 8. mixer 权重：一个事后但有解释力的观察
+
+Full 在第 31 个 residual sublayer 的 branch-output RMS 平均达到 `1.4964`，是 32 层中
+最大值。但最终 output mixer 给这个 source 的平均权重只有：
+
+```text
+observed = 0.002794
+uniform  = 1 / 33 = 0.030303
+ratio    = 0.0922× uniform
+```
+
+一个直观解释是：输出混合器学会了压低这个幅值突增的 source，而不是被迫把它以单位
+residual 权重传到输出。
+
+必须同时保留两个限制：
+
+1. “最大 spike 对应低权重”是看完完整 trace 后挑出的描述；
+2. 它不是预注册 endpoint，不能作为独立确认性证据。
+
+网站会明确标注 **post-hoc descriptive callout**，并展示完整深度权重图，让读者看到它
+不是从被隐藏的其他 source 中挑出的孤立数字。
+
+---
+
+## 9. 梯度结果没有复现论文叙述
+
+预注册诊断对每个 Transformer block 的公共 core 参数计算 gradient RMS，再求 16 个
+block 间的 coefficient of variation：
+
+| 结构 | seed 1 | seed 2 | seed 3 | mean CV |
+|---|---:|---:|---:|---:|
+| Baseline | 0.3537 | 0.3409 | 0.3396 | **0.3447** |
+| Full | 0.4890 | 0.5100 | 0.5272 | **0.5087** |
+| Block | 0.6622 | 0.6849 | 0.5448 | **0.6306** |
+
+CV 越低，按这个特定定义才越均匀。因此本轮观察是：
+
+```text
+Baseline < Full < Block
+```
+
+这和 AttnRes 论文在大模型训练中报告的、更平坦的跨深度梯度幅值叙述不是同一个结果。
+合理边界包括：
+
+- 本实验只有 width 192、16 blocks、2,000 steps；
+- 本指标是“按 block 汇总的核心参数 gradient RMS”；
+- 论文图可能观察 activation / residual-output gradients，聚合对象并不相同；
+- byte-level WikiText-2 与论文的大规模训练数据、优化器状态和训练阶段不同。
+
+正确表述是：
+
+> 本缩小实验的主 BPC 对比支持 AttnRes 的方向，但预注册的核心参数梯度均匀性指标不支持
+> 论文式叙述；这提示该解释可能依赖尺度、指标定义或训练阶段，需要后续专门实验。
+
+不正确的做法是改换一个看起来更漂亮的梯度统计后，只展示新指标。
+
+---
+
+## 10. 复现链
+
+### 10.1 smoke
+
+三个结构各执行两次独立的 20-step smoke。以下字段对每个结构都 exact：
+
+```text
+manifest
+model
+optimizer
+hashes
+evaluations
+training_history
+diagnostic
+environment
+```
+
+### 10.2 正式独立进程 replay
+
+预先指定：
+
+```text
+architecture = block
+seed = 2026073001
+steps = 2000
+```
+
+正式格与 fresh-process replay 的最终 BPC 都是：
+
+```text
+1.947877975922777
+```
+
+上面的八组冻结字段全部 exact。计时不要求 exact，实际也不相等，因为 wall time 受系统
+调度影响。
+
+| 文件 | SHA-256 |
+|---|---|
+| formal block / seed 1 | `5df870369d9a86ccb4ba4191fbd1d6f3642893dd47a60f8f6d1143006bdfbdaf` |
+| fresh replay | `e74d3323e5fe31378bb8aad7a8efa2fb91995cd7224c158cf03006466cdea2a7` |
+
+### 10.3 公开产物
+
+| 产物 | 内容 | SHA-256 |
+|---|---|---|
+| `src/data/k3-attnres-reduced.json` | 9 个完整 run + 聚合 + 复现记录 | `44f8622654d32485f8d6e698c02ba1365ddffb10cbd73db0294136d0bd93ce88` |
+| `src/data/k3-attnres-reduced-compact.json` | 网站所需完整曲线与诊断 | `44864d48eddb2ae5887fba4b74f63b5a3d6a23497886ee307decf5b4f45d9faf` |
+| `experiments/k3/attnres/reproduction.json` | smoke、初始化与 replay audit | `545543b7e4a970ca3bc0e6246610a32fb53ec3d9546a98e0f17f38d7121918a2` |
+
+聚合器在同一批只读 run 文件上再次运行后，三个文件 SHA-256 全部不变。
+
+---
+
+## 11. 证据等级
+
+### A. 本轮可直接主张
+
+- 冻结协议下 9 个训练格的最终 BPC 与完整验证曲线；
+- Full / Block 相对 Baseline 的三 seed 配对方向；
+- 当前实现的参数量、实测 step time 与 peak allocated memory；
+- 固定诊断窗口上的 residual RMS、mixer 权重与参数梯度统计；
+- smoke 与指定正式格的独立进程 exact replay；
+- 数据、窗口、初始化和产物 SHA-256。
+
+### B. 只能作为机制解释
+
+- Block 的四层锯齿与块边界重混合一致；
+- Full mixer 可能通过降低权重抑制高 RMS source；
+- Full / Block 的短预算优势可能来自更灵活的深度路由。
+
+这些解释与观测相容，但不是唯一因果解释。
+
+### C. 本轮明确不主张
+
+- K3 checkpoint 已成功 forward 或训练；
+- 论文表格、Figure 4–8 或 paper-scale scaling 已复现；
+- AttnRes 在任意模型、数据和预算上都降低 loss；
+- Block 一般优于 Full；
+- 梯度在 AttnRes 中更均匀；
+- 同 FLOPs、同 wall time 或生产系统的性价比优势；
+- 三个 seed 支持总体统计显著性。
+
+---
+
+## 12. 下一步
+
+这个实验把“能运行的独立机制 probe”完成了，但真实 K3 仍有两道门：
+
+1. `A_log [128]` 的官方转换或权重修订；
+2. 能加载完整或官方切分 K3 的受支持执行环境。
+
+AttnRes 本身的下一轮也不应只增加 seed。优先级更高的是：
+
+- 对齐论文实际使用的 activation / output-gradient 诊断定义；
+- 增加 depth 与训练预算，检验梯度结论是否随尺度翻转；
+- 做 mixer 计算的优化实现，再讨论同 wall-time 或近似同-FLOP 对比；
+- 冻结一个更强 tokenizer / corpus 后检查 byte-level 结论是否保持；
+- 将 Block size 作为预注册变量，而不是看完结果后挑 4。
+
+---
+
+## 13. 一手来源
+
+- [Kimi K3 Technical Report](https://arxiv.org/abs/2607.24653)
+- [Kimi K3 official checkpoint](https://huggingface.co/moonshotai/Kimi-K3)
+- [Kimi K3 official code repository](https://github.com/MoonshotAI/Kimi-K3)
+- [Attention Residuals](https://arxiv.org/abs/2603.15031)
+- [Official Attention Residuals implementation](https://github.com/MoonshotAI/Attention-Residuals)
+- [WikiText dataset repository](https://huggingface.co/datasets/Salesforce/wikitext)
+- [vLLM Kimi K3 implementation](https://github.com/vllm-project/vllm)
+- [SGLang Kimi K3 implementation](https://github.com/sgl-project/sglang)
+
+Grok CLI 在协议冻结前只承担一次对抗式方法审阅：它提出锁定参数容量、残差拓扑、数据顺序、
+指标定义和复现合同的检查项。所有论文事实和实验结论仍由一手来源、冻结代码与本地运行产物
+支持；Grok 输出不作为证据来源。
+
+---
+
+## 14. 发布与回归证据
+
+- 功能源提交：`4ce780d`；
+- 不可变镜像：`docker.k1412.top/wuyang/llm-atlas:20260729T233142Z-4ce780d`；
+- OCI index digest：`sha256:6e89f802f07fd8dacb55a734f58b5e6d8cbe143d03b9b11cde87cc3c4f25f582`；
+- NAS：Compose Manager project `llm-atlas`，`12010 → 8080`，容器 `healthy`、0 次重启；
+- 代理：NPM host `31`，certificate `41`，forward `100.94.16.3:12010`；
+- 公网：21 / 21 页面路由返回 200，HTTP → HTTPS 301，HTTPS/2 health 200；
+- 静态资产：gzip 与 `public, max-age=604800, immutable` 保持生效；
+- 门户：`LLM ATLAS` / `projects` / order `180`；
+- 回归：冻结数据检查、Astro 零诊断、站内引用检查、AttnRes 专项与 K3 全量生产 Chrome
+  全部通过；桌面与 390px 移动端零文档级溢出、零运行时异常；
+- 回滚镜像：`20260729T221654Z-975ed3d`。

research/K3_ATTNRES_REDUCED_PROTOCOL.md

@@ -0,0 +1,276 @@
+# Kimi K3 第四轮：Attention Residuals 独立小模型复现协议
+
+> 协议 ID：`llm-atlas-k3-attnres-reduced-v1`  
+> 冻结日期：2026-07-30（Asia/Shanghai）  
+> 状态：正式输出前预注册  
+> 目标：用可在单张 RTX 5090 上完整训练、复跑和检查的小模型，验证 Attention
+> Residuals 的运算合同与早期训练动力学；不冒充 K3 checkpoint forward 或原论文规模复现。
+
+## 1. 为什么本轮不直接“跑 K3 第一层”
+
+截至本协议冻结时，官方开放工件仍是：
+
+- `moonshotai/Kimi-K3@9f62e4e9fffbd0a83ddd60e1c209d828994b3569`；
+- K3 GitHub `MoonshotAI/Kimi-K3@7c5be9599120d7993748de66a76128614f15f210`；
+- Hugging Face remote code 把 KDA `A_log` 初始化为 `[num_heads]=[96]`；
+- checkpoint header 中每个 KDA `A_log` 仍为 `[128]`；
+- 当前 vLLM `fa2a2589bd2a1fce0851df7fd42ffb54b6195f04` 与 SGLang
+  `3c1717d9b6355e48855be7cc079db3ef9958f36d` 都把参数构造成 head 轴 96，并沿该轴
+  sharding；没有公开的 128→96 转换合同。
+
+因此，裁剪、重复、平均或把 `[128]` 改解释成 channel 参数都会引入未公开假设。本轮不做。
+官方也没有发布可加载的 K3 reduced checkpoint；Attention Residuals 论文所述 48B/3B 模型权重
+仍未公开。
+
+## 2. 一手来源
+
+| 工件 | 固定 revision / checksum | 本轮用途 |
+|---|---|---|
+| Kimi K3 HF | `9f62e4e9…b3569` | 说明完整 checkpoint 的执行边界 |
+| Kimi K3 GitHub | `7c5be959…f210` | 报告与部署入口 |
+| Attention Residuals GitHub | `85e22310fe5ee860b4a023de312d791de8a5a5e6` | 官方论文与公式 |
+| `Attention_Residuals.pdf` | SHA-256 `e5831b0d…a26b2f` | Full / Block AttnRes 运算合同 |
+| WikiText-2 raw | `Salesforce/wikitext@b08601e04326c79dfdd32d625aee71d232d685c3` | 固定公开语料 |
+
+Grok Headless 只承担正式输出前的对抗式方法检查，不是事实来源，也不参与结果解释。
+
+## 3. 研究问题与声明边界
+
+本轮只问：
+
+1. 在核心 Transformer 权重、训练 Token、输入窗口与 optimizer 合同相同的缩小系统中，
+   Full / Block AttnRes 的验证 bits-per-byte 相对 PreNorm 方向是否跨三个 seed 一致？
+2. 三种残差图的输入、分支输出和累计状态 RMS 怎样随深度变化？
+3. 固定诊断 batch 上，每个 Transformer block 的参数梯度 RMS 怎样分布？
+4. zero-init pseudo-query 经过训练后，是否出现邻近来源、embedding 和远层来源的可见分工？
+5. 额外残差运算带来多少实际 step time 与 peak allocated memory？
+
+本轮不能回答：
+
+- K3 2.8T checkpoint 的真实 hidden state、router load 或 cache trace；
+- 原论文 194M–528M activated / 38.7B–119B Token scaling law；
+- AttnRes 的 benchmark 能力、长上下文质量或生产吞吐；
+- KDA、MLA、MoE 与 AttnRes 的联合因果贡献；
+- 任意总体显著性、p-value 或跨数据集普适性。
+
+## 4. 数据合同
+
+下载并校验：
+
+```text
+wikitext-2-raw-v1/train-00000-of-00001.parquet
+wikitext-2-raw-v1/validation-00000-of-00001.parquet
+wikitext-2-raw-v1/test-00000-of-00001.parquet
+```
+
+预处理固定为：
+
+1. 按 parquet 行序读取 `text`；
+2. 每行原文后追加一个 `\n`；
+3. UTF-8 编码，不做 Unicode normalization、去空行、大小写或空白改写；
+4. 词表固定为 256 个原始 byte；
+5. 每个样本取连续 `context+1=257` bytes，前 256 预测后 256。
+
+训练第 `step`、第 `row` 的起点：
+
+```text
+z = first 8 bytes of SHA256(
+  protocol_id + "\0train-window\0" + seed + "\0" + step + "\0" + row
+)
+start = uint64_be(z) mod (len(train_bytes) - 257)
+```
+
+因此同一 seed 的三种架构逐 step、逐 row 使用完全相同的 token tensor，不依赖 Python、NumPy
+或 CUDA RNG 的消费顺序。
+
+验证窗口固定 64 条，诊断窗口固定 16 条，分别用标签 `validation-window` 与
+`diagnostic-window`、固定 index 取 SHA-256 起点；它们对全部 seed / 架构相同。
+
+manifest 必须记录：
+
+- dataset revision、三个 parquet SHA-256；
+- 拼接后 split byte length 与 SHA-256；
+- 训练 seed×step×row 的总 window-schedule hash；
+- validation / diagnostic tensor hash。
+
+## 5. 模型合同
+
+所有模型：
+
+| 项 | 固定值 |
+|---|---:|
+| vocabulary | 256 bytes |
+| context | 256 |
+| Transformer blocks | 16 |
+| residual sublayers | 32（每块 attention + MLP） |
+| `d_model` | 192 |
+| heads | 6 |
+| head dimension | 32 |
+| `d_ff` | 768 |
+| dropout | 0 |
+| positional embedding | learned absolute, 256 × 192 |
+| norm | RMSNorm, `eps=1e-6` |
+| attention | causal multi-head softmax，score 用 FP32 softmax |
+| MLP | bias-free SwiGLU，two `192→768` projections + one `768→192` projection |
+| input/output embedding | tied |
+| final readout | RMSNorm 后乘 tied token embedding |
+
+三种 residual graph：
+
+1. **PreNorm baseline**：`h ← h + f(RMSNorm(h))`。
+2. **Full AttnRes**：每个 attention / MLP 子层从 embedding 与所有早期子层输出中，
+   以 `softmax(q_l^T RMSNorm(source))` 选出输入；当前分支输出成为下一个独立 source；
+   32 个子层结束后用同型 output mixer 聚合 embedding + 32 个输出。
+3. **Block AttnRes**：32 个子层按顺序冻结为 8 块、每块 4 个子层；跨块保存 8 个 block
+   sums，块内使用 partial sum，embedding 永远是 source 0；最后用同型 output mixer 聚合
+   embedding + 8 个完整 block sums。
+
+AttnRes 每个子层增加：
+
+- 一个 `d_model` pseudo-query，初始化严格为 0；
+- 一个 `d_model` RMSNorm key weight，初始化严格为 1。
+
+Full / Block 各自再增加一组相同结构的 output mixer。所有 bias-free linear 与 embedding 使用
+`N(0, 0.02)`；attention output projection 与 MLP down projection 按
+`0.02 / sqrt(2 × 16)` 缩放初始化；普通 RMSNorm 为 1。
+
+核心 token / position embedding、attention、MLP、输入 norm 与 tied output weights 在同一 seed
+的三种架构中必须逐 tensor SHA-256 exact。AttnRes 额外参数与 mixer 运算如实报告，不通过
+无作用 dummy 参数伪造“完全等容量”。
+
+本实验匹配：
+
+- 核心 Transformer 参数；
+- optimizer steps；
+- 每步训练 Token；
+- 逐 Token 数据顺序。
+
+本实验**不匹配**：
+
+- residual mixer FLOPs；
+- step wall time；
+- peak activation memory。
+
+后二者正是次要观测指标，不能被用于支持“同算力下更好”。
+
+## 6. 优化与运行合同
+
+| 项 | 固定值 |
+|---|---:|
+| seeds | `2026073001, 2026073002, 2026073003` |
+| formal steps | 2,000 |
+| batch | 32 |
+| tokens / run | 16,384,000 target bytes |
+| optimizer | AdamW |
+| betas | `(0.9, 0.95)` |
+| epsilon | `1e-8` |
+| peak LR | `3e-4` |
+| min LR | `3e-5` |
+| warmup | 100 steps，linear |
+| decay | cosine，step 100→2,000 |
+| weight decay | `0.1` for parameters with `ndim >= 2`; otherwise `0` |
+| grad clip | global norm `1.0` |
+| compute | BF16 autocast，FP32 optimizer state |
+| device | one RTX 5090 |
+| compile | off / eager |
+| RNG | deterministic algorithms，`CUBLAS_WORKSPACE_CONFIG=:4096:8` |
+
+验证发生在 step `0, 100, 250, 500, 1000, 1500, 2000`。每次使用固定 64 条窗口；
+以 8 条一个 eval batch，报告 token-mean cross entropy（nats）与：
+
+```text
+bits_per_byte = cross_entropy_nats / ln(2)
+```
+
+计时：
+
+- 前 20 个训练 step 不进入 wall-time 统计；
+- step 21–2,000 每步前后 CUDA synchronize；
+- step 20 后 reset peak memory stats；
+- 同报 mean / median / p95 step ms、peak allocated 与 peak reserved。
+
+## 7. 预注册指标
+
+### 7.1 主指标
+
+每个 seed 在 step 2,000 的验证 `bits_per_byte`：
+
+```text
+Δ_full  = Full AttnRes − PreNorm
+Δ_block = Block AttnRes − PreNorm
+```
+
+只报告三个 paired delta、mean、min–max。
+
+预注册解释规则：
+
+- 三个 seed 全同为负，且 mean delta ≤ `−0.010 BPC`：**directional support in this reduced protocol**；
+- 三个 seed 全同为正，且 mean delta ≥ `+0.010 BPC`：**directional concern in this reduced protocol**；
+- 其他情况：**inconclusive at this budget**。
+
+不报告 p-value、population CI 或“复现了论文提升”。
+
+### 7.2 次要指标与精确定义
+
+- `branch_output_rms[l]`：第 l 个 attention / MLP 分支输出在 batch×time×channel 上的 RMS；
+- `layer_input_rms[l]`：进入第 l 个分支 norm 之前的 mixture / residual state RMS；
+- `stream_state_rms[l]`：baseline 加法后的 state；Full 为新 source 输出集合的整体 RMS；
+  Block 为当前 partial sum 的 RMS；
+- `parameter_grad_rms[block]`：固定 diagnostic batch 上、clip 前，该 block 所有核心参数梯度
+  拼接后的 `sqrt(sum(g²)/numel)`；不含 embedding、LM head 与 AttnRes mixer 参数；
+- `depth_weights[l, source]`：固定 diagnostic batch 上 softmax 权重对 batch×time 求均值；
+- `source_entropy[l]`：同一权重先逐 token 计算 entropy，再对 batch×time 求均值；
+- `step_ms` 与 CUDA memory：按 §6 固定协议。
+
+这些指标是描述性机制账，不进入主判定。
+
+## 8. Smoke、正式运行与复现
+
+正式输出前先跑：
+
+- 三种架构 × seed 1；
+- 20 steps；
+- batch 4；
+- 同一 window schedule；
+- 断言无 NaN、loss finite、common-weight hashes exact；
+- 在全新进程重跑相同 smoke，比较数据 hash、初始 common hash、逐 eval loss 和最终 common
+  checkpoint hash。
+
+smoke 只决定实现是否可运行，不用于改变 formal 超参数或挑选架构。
+
+正式运行顺序冻结为：
+
+```text
+seed 2026073001: baseline → full → block
+seed 2026073002: baseline → full → block
+seed 2026073003: baseline → full → block
+```
+
+全部 9 个 run 完成后，在全新进程复跑 `seed 2026073001 / block / 2,000 steps`，比较：
+
+- manifest / dataset / schedule hashes；
+- common initial weight hash；
+- step 0–2,000 eval history；
+- final common parameter hash；
+- final mixer parameter hash；
+- diagnostic tensors与 depth-weight hashes。
+
+计时不要求 exact；数值字段要求 exact，若底层确定性限制导致非 exact，必须报告最大差异，
+不能降低字段范围后宣称复现。
+
+## 9. 冻结产物
+
+```text
+research/K3_ATTNRES_REDUCED_PROTOCOL.md
+research/K3_ATTNRES_REDUCED_AUDIT.md
+experiments/k3/attnres/build_dataset.py
+experiments/k3/attnres/train.py
+experiments/k3/attnres/analyze.py
+experiments/k3/attnres/README.md
+experiments/k3/attnres/manifest.json
+src/data/k3-attnres-reduced.json
+src/data/k3-attnres-reduced-compact.json
+```
+
+原始 parquet 与训练 checkpoint 留在本机 cache，不进入公开仓库；manifest、原始指标 JSON、
+分析结果、代码、环境、checksum 和确定性复现账进入开源树。

research/K3_ATTNRES_SPIKE_AUDIT.md

@@ -0,0 +1,503 @@
+# K3 Attention Residuals 局部梯度尖峰与反向路由审计
+
+研究日期：2026-07-30  
+协议 ID：`llm-atlas-k3-attnres-spike-path-v1`  
+父协议：`llm-atlas-k3-attnres-gradient-scale-v1`  
+研究身份：**Round 05 已知异常驱动的定向机制追踪，不是盲发现**
+
+## 0. 一句话结果
+
+在与 Round 05 **model / optimizer / BPC / post-MLP gradient 全部 exact** 的
+depth-32 Block AttnRes 三 seed 重放中：
+
+1. layer 21–25 的局部峰在六个捕获位置最终都 3 / 3 可见；
+2. 四种预注册 Token-gradient reductions 的 12 / 12 单元全部过闸；
+3. 全局切断 64 个 depth mixers 与 output mixer 的 softmax / query / key
+   source-gradient path，峰没有下降，反而在 3 / 3 seed 略增；
+4. 保持 learned forward 完全不变、只把所有 mixer 的 value backward coefficients
+   从 learned `w` 改为 `1/N` 后，spike contrast 平均下降 **70.2%**、peak normalized
+   平均下降 **37.0%**，两项都在 3 / 3 seed 超过预注册 20% 阈值；
+5. seed-1 从初始化完整重训 8,000 steps 的 replay，连六位置、七 reductions、三模式和
+   65 个 mixer quantile summaries 都 exact。
+
+最窄、最准确的结论是：
+
+> 在这个已训练的缩小 Block AttnRes 图上，最终 layer 21–25 峰对 learned mixer 的
+> **value-backward routing coefficients** 有预注册阈值下的强全局敏感性；它不是由
+> softmax/key source-gradient derivative path 单独解释的。
+
+这不等于：
+
+- uniform backward 是更好的训练方法；
+- learned value weights 让模型训练变差；
+- 某一个 layer 或同层 MLP mixer 是唯一原因；
+- Kimi K3 真实 checkpoint 存在同样尖峰。
+
+## 1. 研究问题从哪里来
+
+Round 05 已知：
+
+- Block 的首/末四分位失衡在 6 / 6 配对中改善；
+- 全层 CV 却在 6 / 6 配对中恶化；
+- depth-32 三 seed 平均的相对高值集中在 layer 21–25；
+- post-MLP gradient 与 MLP latest-source mean weight 在 96 个
+  `seed × layer` 点上 Pearson `r≈0.651`，在 layer 19–28 的 30 点上 `r≈0.690`。
+
+这些 observational correlations 不能回答：
+
+- 是 softmax / key 的导数路径重要，还是 value coefficients 重要？
+- 峰是否只是 element RMS 的 reduction 产物？
+- 峰在一个 block 内到哪个位置才出现？
+
+因此 Round 06 在看见新结果前固定：
+
+- 六个 activation 位置；
+- 四个主 reductions、两个 cancellation-sensitive reductions 和一个代数控制；
+- 三种 same-forward backward rules；
+- fixed spike set `S={21,22,23,24,25}`；
+- `contrast≥1.5`、top-5 overlap、Spearman、20% material drop；
+- 三 formal seeds 和一个完整 replay。
+
+前置 scoping、协议与 runner 分别在提交 `204f599`、`54c80fb` 前后进入公开仓库，
+formal 训练随后才开始。
+
+## 2. 对抗审查如何改变协议
+
+本机 Grok Headless 只用于一次只读、对抗性方法审查，不提供事实来源，也没有修改文件。
+主代理采纳并在结果前修正的关键问题包括：
+
+- 把“最早生成位置”降级为“最早可见 tensor”；
+- 主判定时点固定为 step 8,000；
+- intervention 明确作用于全部 64 个 depth mixers 加 output mixer；
+- 结论从局部因果分解降级为全局 backward-rule sensitivity；
+- material 判定固定只用 `element_rms`；
+- median / p95 固定 Hyndman–Fan Type 7；
+- top-5 ties、Spearman ties、正分母与 invalid 条件全部写死；
+- 增加六位置 loss×2、optimizer isolation 和训练不得进入 custom autograd 的闸门。
+
+这次审查的价值是发现协议漏洞，不是为结果背书。
+
+## 3. 第一次 smoke 为什么被拦住
+
+最初 intervention custom backward 直接用 elementwise multiplication 分配
+`weights × grad_output`。forward identity 已经 exact，但 step-0 uniform negative
+control 失败：
+
+```text
+六位置 max raw relative difference：约 0.0037–0.0046
+六位置 max normalized difference：约 0.0011–0.0020
+```
+
+这说明虽然代数公式一致，手写 backward 没有复用父 runner autocast einsum 的实际数值
+路径，不能满足预注册的“query=0 时三模式等价”。
+
+正式训练尚未开始。实现随后改为：
+
+1. forward 直接返回父 `DepthMixer.forward` 已算出的 `y_parent`；
+2. backward 在 `torch.enable_grad()` 下，用冻结的 learned / uniform weights
+   重新走同一个 BF16-autocast einsum surrogate；
+3. 只把 surrogate 对 source values 的梯度返回；
+4. 不向 `y_parent`、query、key path 回传。
+
+修正后：
+
+```text
+三模式 logits / loss / 六位置 activations：byte-exact
+65 个 mixer summaries：exact
+step-0 六位置 raw gradient spectra：max difference 0
+step-0 normalized spectra：max difference 0
+六位置 × 七 reductions loss×2：scale / shape error 0
+两个全新进程非计时字段：exact
+```
+
+这个失败不是需要隐藏的“调试噪声”；它正是负控制在正式结果前发挥作用的证据。
+
+## 4. 训练合同与工作量
+
+Round 06 没有创建新的训练分布：
+
+| 字段 | 值 |
+|---|---:|
+| architecture | Block AttnRes |
+| depth | 32 Transformer blocks |
+| groups | 8，每组 4 blocks |
+| width / heads / FFN | 192 / 6 / 768 |
+| context / batch | 256 / 32 |
+| steps | 8,000 |
+| formal seeds | 2026073001 / 2 / 3 |
+| replay | seed-1，从初始化完整 8,000 steps |
+| formal target bytes | 196,608,000 |
+| 含 replay target bytes | 262,144,000 |
+| diagnostic tensor | 固定 16 × 256 |
+| diagnostic steps | 0 / 100 / 500 / 2,000 / 4,000 / 8,000 |
+| intervention steps | 0 / 8,000 |
+
+数据与父协议 exact，完整值为：
+
+```text
+formal schedule:
+5041e09b167f229248d2462324e8c254b8f5938975f135dcd8192b00a54a4f4e
+
+validation:
+f459316f13078a163b47c133511bb7181e05170ab89516e196490113893ce338
+
+diagnostic:
+21117e31db302b10d67b63f035665dc8f220b879d216ccd12b7d2ba86e7b1716
+```
+
+## 5. 父协议训练等价闸门
+
+三个 formal seed 全部满足：
+
+| 字段 | seed-1 | seed-2 | seed-3 |
+|---|---:|---:|---:|
+| final model-state hash | exact | exact | exact |
+| final optimizer-state hash | exact | exact | exact |
+| 六个 validation BPC | exact | exact | exact |
+| 六条 post-MLP element-RMS arrays | exact | exact | exact |
+| training history | exact | exact | exact |
+
+最终 BPC：
+
+| seed | Round 06 | Round 05 |
+|---:|---:|---:|
+| 2026073001 | 1.7123525940760473 | exact |
+| 2026073002 | 1.7093240655856785 | exact |
+| 2026073003 | 1.7030966812870931 | exact |
+
+因此新增 capture / diagnostics 没有改变 optimizer trajectory。Round 06 不是一组“看起来
+相近”的新模型，而是同一冻结训练的独立数值重放。
+
+## 6. 峰在训练中形成，不是初始化常数
+
+最终主指标是 post-MLP element-RMS：
+
+```text
+spike_contrast =
+mean(layers 21–25) / mean(other 27 layers)
+```
+
+时间轨迹：
+
+| step | seed-1 | seed-2 | seed-3 |
+|---:|---:|---:|---:|
+| 0 | 0.782 | 0.691 | 0.797 |
+| 100 | 0.560 | 0.556 | 0.575 |
+| 500 | 0.522 | 0.573 | 0.478 |
+| 2,000 | 2.497 | 2.740 | **1.461** |
+| 4,000 | 2.851 | 3.157 | 1.773 |
+| 8,000 | **3.046** | **3.333** | **1.881** |
+
+step 500 时三个 seed 的目标层都低于其他层均值；到 step 2,000，seed-1/2 已形成明显峰，
+seed-3 仍略低于预注册 1.5 阈值；seed-3 到 step 4,000 才跨过。
+
+所以允许说：
+
+> 尖峰在这个训练过程中发展出来，且 onset 有 seed 差异。
+
+不允许说：
+
+> aggregation boundary 本身在初始化就必然产生 layer 21 峰。
+
+## 7. 六个位置：处处可见，不等于找到 origin
+
+最终 element-RMS spike contrast：
+
+| 位置 | seed-1 | seed-2 | seed-3 | 三 seed mean | 3/3 ≥1.5 |
+|---|---:|---:|---:|---:|---:|
+| pre-attention input | 3.724 | 3.189 | 2.040 | 2.984 | 是 |
+| attention branch output | 3.200 | 3.662 | 2.005 | 2.955 | 是 |
+| post-attention state | 3.200 | 3.662 | 2.005 | 2.955 | 是 |
+| pre-MLP input | 3.829 | 4.527 | 2.978 | **3.778** | 是 |
+| MLP branch output | 3.046 | 3.333 | 1.881 | 2.754 | 是 |
+| post-MLP state | 3.046 | 3.333 | 1.881 | 2.754 | 是 |
+
+协议顺序中的第一个可见 tensor 是 `pre_attention_input`。但 activation gradient
+本来就汇总全部下游路径，所以这只说明：
+
+> 在进入该层 attention 之前，固定深度模式已经能在损失对该 tensor 的敏感性中看到。
+
+它不说明峰在 pre-attention mixer “生成”。六个位置全部可见，反而证明单靠位置 trace
+无法定位唯一 origin。
+
+值得注意的是，`pre_mlp_input` 的平均 contrast 最高，但这仍是读数，不是“MLP 是唯一
+原因”的消融证据。
+
+## 8. reduction robustness：12 / 12 过闸
+
+主 family：
+
+1. element RMS；
+2. mean token RMS；
+3. median token RMS；
+4. p95 token RMS。
+
+每个 seed / reduction 必须同时满足：
+
+- contrast ≥1.5；
+- top-5 与固定集合至少重合 3；
+- 与 element-RMS 32-layer Spearman ≥0.8。
+
+结果：
+
+| reduction | seed contrasts | 最低 top-5 overlap | 最低 Spearman | verdict |
+|---|---|---:|---:|---:|
+| element RMS | 3.046 / 3.333 / 1.881 | 3 | 1.000 | pass |
+| token RMS mean | 2.869 / 3.182 / 1.808 | 3 | 0.958 | pass |
+| token RMS median | 2.646 / 2.905 / 1.666 | 3 | **0.895** | pass |
+| token RMS p95 | 3.143 / 3.532 / 2.012 | 3 | 0.993 | pass |
+
+总判定：
+
+```text
+12 / 12 cells pass
+robust within the preregistered reduction family
+```
+
+`global_l2` 与 element RMS 的固定 shape 代数控制也通过。batch-mean / token-mean
+reduction 同样保存，但它们允许正负梯度先抵消，按协议只作探索性读数，不进入 robustness
+判定。
+
+这个结果支持：
+
+> layer 21–25 峰不是只由“先平方再平均”的单一 reduction 选择制造的。
+
+它不支持：
+
+> 论文所有可能的 gradient magnitude 定义都会给出同一曲线。
+
+## 9. same-forward interventions 到底改了什么
+
+三模式：
+
+### learned
+
+原计算图：
+
+```text
+w = softmax(qᵀ RMSNorm(sources))
+y = Σ w_i source_i
+```
+
+### detached learned
+
+forward 的 `w`、`y`、logits、loss 完全相同；backward 对 sources 保留 learned
+value coefficients，但切断 softmax / query / key 的 source-gradient path。
+
+### uniform value backward
+
+forward 仍逐字节返回 learned `y`；backward 对每个 source 返回 `grad_y/N`，不向
+softmax / query / key 回传。
+
+作用域是全部 64 个 depth mixers 加 output mixer。它不是只改 layer 21 附近，也不是一次
+训练消融。
+
+三个 seed 的最终 forward identity 均满足：
+
+```text
+logits hash exact
+loss exact
+六位置 activation hashes exact
+65 mixer summary hash exact
+```
+
+所以三模式的梯度差不能归因于不同 forward activation 或不同预测。
+
+## 10. softmax / key path：没有降低峰
+
+learned → detached learned：
+
+| seed | contrast | detached | relative drop | peak relative drop |
+|---:|---:|---:|---:|---:|
+| 2026073001 | 3.046 | 3.093 | −1.54% | −0.67% |
+| 2026073002 | 3.333 | 3.397 | −1.93% | −0.38% |
+| 2026073003 | 1.881 | 1.929 | −2.53% | −1.47% |
+| mean | — | — | **−2.00%** | **−0.84%** |
+
+“relative drop”为负表示 detaching 后峰略高。预注册规则要求 contrast 与 peak 都下降至少
+20%、3 / 3 seed 同向；实际是两项在 3 / 3 seed 都朝相反方向。
+
+结论：
+
+> 全局移除 softmax / query / key 的 source-gradient derivative path，没有在本阈值下
+> 降低固定峰；它在三个 seed 中都使峰略增。
+
+这不证明该路径处处为零，也不证明它对训练没有作用。它只否定本轮预注册的
+“material peak reduction”方向。
+
+## 11. learned value coefficients：3 / 3 过 material 闸门
+
+detached learned → uniform value backward：
+
+| seed | detached contrast | uniform contrast | contrast drop | peak drop |
+|---:|---:|---:|---:|---:|
+| 2026073001 | 3.093 | 0.780 | **74.8%** | **39.3%** |
+| 2026073002 | 3.397 | 0.786 | **76.9%** | **43.2%** |
+| 2026073003 | 1.929 | 0.792 | **58.9%** | **28.6%** |
+| mean | — | — | **70.2%** | **37.0%** |
+
+六个相对下降全部超过预注册 20%。uniform value backward 下：
+
+- 固定 target-layer contrast 都低于 0.8；
+- peak layer 从 learned / detached 的 layer 21 移到 layer 2；
+- forward logits、loss 与 activation 完全不变。
+
+预注册判定：
+
+```text
+material sensitivity at the preregistered threshold
+3 / 3 seeds
+```
+
+允许说：
+
+> 在最终已训练计算图上，learned source value-backward coefficients 是该局部梯度形状的
+> material carrier / amplifier。
+
+禁止说：
+
+- “把训练改成 uniform backward 一定更好”；
+- “learned mixer 导致 BPC 下降或上升”；
+- “峰完全由同层 MLP latest weight造成”；
+- “70.2% 是因果贡献率”。
+
+相对下降是对一个**全局反向规则干预**的指标变化，不是可加的路径归因百分比。
+
+## 12. mixer association 与 intervention 如何一起读
+
+最终 observational association：
+
+| scope | metric | Pearson | Spearman |
+|---|---|---:|---:|
+| 96 seed×layer | MLP latest mass | 0.651 | 0.692 |
+| 96 seed×layer | MLP normalized entropy | −0.274 | −0.417 |
+| layer 19–28 | MLP latest mass | 0.690 | 0.693 |
+| layer 19–28 | MLP max mass | 0.702 | 0.705 |
+| layer 19–28 | MLP normalized entropy | −0.636 | −0.640 |
+| layer 19–28 | attention latest mass | 0.158 | 0.339 |
+
+Round 06 的 intervention 让“value coefficients 值得关注”从单纯相关线索升级为
+same-forward backward sensitivity。但它仍然是**全部 mixers 同时改写**，所以不能把
+全局结果定向归因给表中同层 MLP latest mass。
+
+更窄的下一步需要：
+
+- 逐 group / 逐 mixer intervention；
+- output mixer 与 64 个 depth mixers 分开；
+- current partial、completed group、embedding source 分开；
+- 保持 forward exact 的局部 backward path ablation。
+
+这些没有在本轮结果后追加，以免事后挑选 layer。
+
+## 13. replay 与完整复现
+
+seed-1 replay 从初始化完整训练 8,000 steps，不加载 formal checkpoint。
+
+exact 字段：
+
+```text
+architecture / depth / seed / steps / batch / target bytes
+manifest / model / optimizer contracts
+model and optimizer state hashes
+six validation evaluations
+all training-history checkpoints
+six diagnostic steps
+six positions
+seven reductions
+three intervention modes at step 0 / 8,000
+65 mixer summaries with quantiles
+forward identity / negative control / loss-scale gates
+environment / artifact identities
+Round 05 equivalence record
+```
+
+冻结 compare SHA-256：
+
+```text
+05396eaf56429ca8d87815ecf00a5f78be5a9b2ec7cc96014856c09dcedcbb87
+```
+
+排除计时、run kind 和输出路径。replay verdict：`exact`。
+
+## 14. 成本数字的正确解释
+
+formal 三个进程的 mean step time 都约 224 ms；peak allocated 约 12.84 GB，
+reserved 约 13.92 GB。
+
+它们与 Round 05 单进程约 109 ms 不可直接比较，因为本轮两进程并行竞争同一 RTX 5090，
+GPU 利用率 100%。训练 step 本身走父 runner exact 路径，新增诊断只发生在冻结时点。
+
+因此：
+
+- 计时证明实验真实执行；
+- 计时不用于评价新诊断 overhead；
+- 更不能外推到 K3 训练系统。
+
+## 15. 工件与哈希
+
+核心源码：
+
+| 工件 | SHA-256 |
+|---|---|
+| manifest | `d5302a249249a07d362819134763d14e7d32307f22cff416c665ed9606142fef` |
+| protocol | `6cb101b8760d9f1c81caeb2f16880b16152da103867224a06761a75a12984a16` |
+| scoping | `590166bd62580bb8238293823cfcc39bc0a465fec4c697025343f3f1138abd27` |
+| runner | `77298081d3c491d2e88e4705995174b9879ef377f520eb5fe5ea107e7a1da084` |
+| analyzer | `b9dd7f723e8c5765ecffd9ae3050ff7a95643f95853adaeeaa2c2c323d0ca057` |
+
+raw physical hashes：
+
+| 文件 | SHA-256 |
+|---|---|
+| formal seed-1 | `e39e93b7a7fce3c56f5f14f95cfdc04afdce53628affee1202fe62bd1bdb7f71` |
+| formal seed-2 | `1c6f6c731030ec0adb2a8e7a4d586e0c4005cc3319568a7ac83c08c2a4b8eaf8` |
+| formal seed-3 | `115f8245577ece6dfaaa8ada68445c186e6523a7f3b26efcc3eb4c0c4ce82406` |
+| replay seed-1 | `4f91c7a2698662ea69af9aa3425e144107834fd9b6b717fb82b84f153b867dc5` |
+
+聚合工件：
+
+| 工件 | physical SHA-256 | canonical SHA-256 |
+|---|---|---|
+| full aggregate | `75058f9e…c1e90` | `1d0faef2…fc8b4` |
+| compact | `7b3a3749…8ef6c` | `a430dd26…26cd` |
+| reproduction | `86ac20ee…879a` | `7c522314…3515` |
+
+完整值保存在文件自身和 `reproduction.json`，缩写只用于阅读。
+
+## 16. 允许与禁止的最终表述
+
+### 允许
+
+> 固定 layer 21–25 峰在四种预注册 Token-gradient reductions 中 12 / 12 过闸，
+> 所以它在本轮 reduction family 内稳健。
+
+> 在同一个 learned forward 上，全局 value-backward coefficient uniformization 使
+> contrast / peak 在 3 / 3 seed 超过 20% 下降阈值。
+
+> 全局移除 softmax/key source-gradient path 没有降低峰，反而在 3 / 3 seed 略增。
+
+> 峰从训练中发展，且 seed-3 的形成更晚、更弱。
+
+### 禁止
+
+- “找到了梯度尖峰的唯一来源”；
+- “pre-attention 是尖峰生成点”；
+- “论文 Figure 5(c) 已复现”；
+- “Kimi K3 checkpoint 有 layer 21 尖峰”；
+- “uniform backward 会提高训练质量”；
+- “learned mixer 的 70.2% 梯度是坏的”；
+- “softmax/key path 没有任何作用”；
+- 从 3 seeds 推断总体显著性；
+- 从 width 192 / depth 32 外推到 2.8T 参数。
+
+## 17. 下一轮最有价值的实验
+
+1. 把全局 intervention 拆成 output mixer、group 6、group 7 和其他 mixers 四组；
+2. 对 completed-group value path 与 current-partial value path 分开做 same-forward backward
+   ablation；
+3. 预先固定 local intervention grid，不根据本轮最大的单层事后选择；
+4. 比较“只改诊断 backward”与“从初始化按替代 backward 训练”的身份差异；
+5. 若 Attention Residuals 官方发布 Figure 5 telemetry，再开独立 protocol 对齐其 tensor、
+   norm、reduction、batch 与 AMP 时点。
+
+真实 K3 forward 仍受 `A_log [128]→[96]` 官方转换合同缺失阻断；本轮不绕过该边界。

research/K3_ATTNRES_SPIKE_PROTOCOL.md

@@ -0,0 +1,406 @@
+# K3 Attention Residuals 局部梯度尖峰与归约敏感性协议
+
+协议 ID：`llm-atlas-k3-attnres-spike-path-v1`  
+冻结日期：2026-07-30  
+协议状态：**结果前预注册**  
+父协议：`llm-atlas-k3-attnres-gradient-scale-v1`
+
+## 0. 研究身份
+
+本轮是 Round 05 的**定向机制追踪**，不是盲发现：
+
+- 已知 depth-32 / Block 的三 seed 平均 layer 21–25 normalized post-MLP
+  activation-gradient RMS 较高；
+- 已知 observationally，MLP mixer latest-source mass 与该梯度谱相关；
+- 未知尖峰最早在哪个 block position 出现；
+- 未知 mixer 的 softmax/key derivative path 与 learned value coefficients 分别贡献多少；
+- 未知更换 raw gradient tensor 的公开 reduction 后，layer 21–25 是否仍构成稳定局部峰。
+
+前置已知结果和相关分析固定在
+`research/K3_ATTNRES_SPIKE_SCOPING.md`。任何 Round 06 输出不得被倒写成“事前未知”。
+
+## 1. 允许回答的问题
+
+1. 在同一个缩小 Block AttnRes 模型里，layer 21–25 的相对高值在哪些
+   attention / MLP 位置已经可见？
+2. 在前向完全相同的条件下，对**全部 mixer** 切断 weight / key 导数路径是否降低尖峰？
+3. 在前向完全相同的条件下，把**全部 mixer** 的 source value 反向系数改为均匀权重，
+   是否降低尖峰？
+4. element RMS、token-RMS mean / median / p95 是否给出一致的 spike 结论？
+5. Round 06 的新增观测是否保持 Round 05 训练状态、BPC 和 post-MLP global RMS exact？
+
+## 2. 明确不回答的问题
+
+- 论文 Figure 5(c) 的未公开 telemetry 定义；
+- Kimi K3 2.8T checkpoint 的真实梯度；
+- learned mixer 对大模型最终能力的因果收益；
+- 改写 backward rule 后重新训练会发生什么；
+- 哪个 source 具有可命名的语义；
+- 三个 seed 之外的总体显著性或置信区间；
+- 同 FLOPs、wall time 或参数量公平性。
+
+## 3. 冻结训练合同
+
+Round 06 不创建新训练任务分布，完整复用 Round 05：
+
+| 字段 | 固定值 |
+|---|---|
+| architecture | Block AttnRes |
+| Transformer depth | 32 |
+| aggregation groups | 8 |
+| blocks / group | 4 |
+| width / heads / FFN | 192 / 6 / 768 |
+| context / vocabulary | 256 / byte-256 |
+| formal seeds | 2026073001 / 2026073002 / 2026073003 |
+| steps / batch | 8,000 / 32 |
+| target bytes / formal cell | 65,536,000 |
+| optimizer | AdamW |
+| peak / min LR | 3e-4 / 3e-5 |
+| warmup | 400 |
+| weight decay | 0.1 for ndim ≥ 2 |
+| betas / epsilon | 0.9, 0.95 / 1e-8 |
+| clip | global norm 1.0 |
+| forward | CUDA BF16 autocast |
+| residual accumulation | explicit FP32 |
+| diagnostic CE | fixed 16 × 256 token-mean FP32 CE |
+| diagnostic steps | 0 / 100 / 500 / 2,000 / 4,000 / 8,000 |
+
+数据 bytes、training schedule、validation tensor 和 diagnostic tensor hashes 必须与父协议
+manifest exact。训练窗口仍由父协议 ID 派生；Round 06 ID 只标识新增 telemetry，不能改变
+任一 optimizer input。
+
+## 4. 正式网格与 replay
+
+正式运行：
+
+```text
+depth-32 / block / seed-2026073001
+depth-32 / block / seed-2026073002
+depth-32 / block / seed-2026073003
+```
+
+另从初始化完整重跑：
+
+```text
+replay / depth-32 / block / seed-2026073001
+```
+
+正式三格共处理 196,608,000 target bytes；含 replay 共 262,144,000 bytes。
+
+每格都必须在全新 Python 进程中运行。可并行两个进程，但不能共享 model、optimizer、RNG
+或 CUDA graph。性能计时不进入数值复现合同。
+
+## 5. 六个 activation 位置
+
+对每个 Transformer block `l` 捕获：
+
+1. `pre_attention_input`：attention RMSNorm 的输入；Block 中是 attention mixer 输出；
+2. `attention_branch_output`：attention projection 输出；
+3. `post_attention_state`：attention branch 加入后的 FP32 partial state；
+4. `pre_mlp_input`：MLP RMSNorm 的输入；Block 中是独立 MLP mixer 输出；
+5. `mlp_branch_output`：MLP down projection 输出；
+6. `post_mlp_state`：MLP branch 加入后的 FP32 partial state，即 Round 05 主对象。
+
+每个位置必须有 32 个不同计算节点，shape 为 `[16,256,192]`，梯度全部 present / finite。
+跨位置允许计算图语义上的共享来源，但同一位置的 32 个条目不得意外复用同一 storage。
+
+捕获在一次 learned-mode diagnostic forward / backward 中完成；不能把六个位置拆成六次
+不同 loss 的 backward：
+
+- `attention_branch_output` / `mlp_branch_output` 在原 module 输出、转成 FP32 residual
+  之前捕获，保留其实际 autocast dtype；
+- `post_attention_state` / `post_mlp_state` 在显式 FP32 residual accumulation 后捕获；
+- 所有 `retain_grad()` 只允许出现在 diagnostic `capture=True` 路径；
+- 8,000 个 optimizer training steps 必须走父 runner 的原始 `DepthMixer.forward`，
+  `capture=False`，不得进入 intervention custom autograd；
+- 每次 diagnostic / intervention 前后都 `zero_grad(set_to_none=True)`；
+- diagnostic 前后 optimizer-state tensor hash 必须 exact。
+
+### 5.1 尖峰可见位置
+
+固定目标集合：
+
+```text
+S = layers 21, 22, 23, 24, 25
+R = other 27 layers
+```
+
+对每个位置和 seed：
+
+```text
+spike_contrast = mean(metric[S]) / mean(metric[R])
+peak_normalized = max(metric) / mean(metric)
+```
+
+主判定**只使用 step 8,000**。若某个位置 `spike_contrast ≥ 1.5` 在 3 / 3 seed
+成立，则称“该位置已可见局部峰”。六个位置按上述 forward 顺序报告；第一个满足者只标为
+**earliest tensor where the pattern is observed**，禁止写成 origin、injection point 或
+“在该算子生成”。其他 diagnostic steps 只展示轨迹，不参与位置判定。若没有位置 3 / 3
+达标，结论为 position-mixed。
+
+## 6. 同一 raw gradient tensor 的 reductions
+
+令某层某位置梯度为 `g ∈ R[B,T,D]`，其中 `B=16,T=256,D=192`。
+
+### 6.1 主 sensitivity family
+
+1. `element_rms = sqrt(mean_btd(g²))`
+2. `token_rms_mean = mean_bt(sqrt(mean_d(g²)))`
+3. `token_rms_median = median_bt(sqrt(mean_d(g²)))`
+4. `token_rms_p95 = p95_bt(sqrt(mean_d(g²)))`
+
+这四项都测每 Token 梯度长度的分布，只改变平方根与 Token reduction 的次序/统计量。
+
+median / p95 统一使用排序后的 **Hyndman–Fan Type 7 linear interpolation**：
+
+```text
+h = (N - 1) × p
+j = floor(h)
+q_p = x_sorted[j] + (h - j) × (x_sorted[j + 1] - x_sorted[j])
+```
+
+`p=.5/.95`，零基下标；若 `h` 为整数则直接取 `x_sorted[h]`。实现不得依赖 numpy /
+torch 版本相关的默认 quantile 方法。
+
+### 6.2 cancellation-sensitive diagnostics
+
+5. `batch_mean_rms = sqrt(mean_td((mean_b g)²))`
+6. `token_mean_rms = sqrt(mean_bd((mean_t g)²))`
+
+它们允许正负梯度先抵消，测的是更相干的方向信号，只作探索性诊断，不进入主 robustness
+判定。
+
+### 6.3 代数控制
+
+7. `global_l2 = sqrt(sum_btd(g²))`
+
+固定 shape 下它应满足：
+
+```text
+global_l2 = element_rms × sqrt(B×T×D)
+```
+
+逐层 normalized spectrum、CV、spike contrast 应与 element RMS 在 `1e-6` 内相同。
+
+### 6.4 reduction robustness 闸门
+
+对最终 `post_mlp_state`，每个主 family reduction、每个 seed 必须同时满足：
+
+1. `spike_contrast ≥ 1.5`；
+2. 该 reduction 的 top-5 layers 与固定集合 `S` 至少重合 3 层；
+3. 与 element RMS 的 32-layer Spearman `ρ ≥ 0.8`。
+
+四种 reductions、三个 seed 全部满足才记为
+`robust within the preregistered reduction family`。任何一格失败即为 `mixed`；
+全数不满足才记为 `not robust at this threshold`。不添加事后替代阈值。
+
+top-5 固定按 `metric descending, layer index ascending` 排序；Spearman 对并列值使用
+average ranks。cancellation-sensitive diagnostics 不得进入本节判定。
+
+## 7. mixer backward-path interventions
+
+三种模式使用完全相同的 learned forward weights 与 model state。intervention 的作用域固定
+为**全部 64 个 depth mixers 加最终 output mixer**；不允许结果后只改 layer 21–25 邻域。
+因此本节回答的是“全局改写 mixer backward rule 后，固定尖峰指标是否变化”，不是把某个
+局部 mixer 宣布为唯一原因。
+
+### 7.1 `learned`
+
+原始 mixer：
+
+```text
+w = softmax(qᵀ RMSNorm(sources))
+y = Σ w_i source_i
+```
+
+反向同时经过 value coefficients 和 softmax / query / key 路径。
+
+### 7.2 `detached_learned`
+
+前向仍用同一 `w`，但 `w` 在反向中 detach：
+
+```text
+y = Σ stopgrad(w_i) source_i
+```
+
+它保留 learned value coefficients，切断 softmax / query / key 对 source gradient 的路径。
+
+### 7.3 `uniform_value_backward`
+
+使用自定义 autograd：
+
+- 先调用父 runner 原始 `DepthMixer.forward` 得到 `y_parent`；
+- custom Function 的 forward 直接返回已经计算出的 `y_parent`，不重算或做
+  `y + z - z` 式浮点抵消；
+- backward 对每个 source 返回 `grad_y / N`；
+- 不向 learned weights / query / key 回传。
+
+因此 forward logits、loss、所有 activation 值必须与 `learned` **逐元素 exact**，但 source
+value 的 backward coefficient 变为均匀。
+
+`detached_learned` 同样以 custom Function 原样返回 `y_parent`，backward 才按 learned
+`w_i` 把 `grad_y` 分配给 source。两种 custom Function 都只允许在 step 0 / 8,000
+diagnostic 使用。
+
+这些是**全局 backward-rule sensitivity diagnostics**，不是可训练模型变体，也不声称是
+合理部署方案。
+
+## 8. intervention 判定
+
+只在 step 0 和 8,000 运行三模式。
+
+### 8.1 forward identity gate
+
+同 seed / step 三模式必须满足：
+
+- logits tensor SHA-256 exact；
+- loss FP32 value exact；
+- 六位置 activation tensor hashes exact；
+- mixer forward mean / quantile summaries exact。
+
+任一失败，正式结果无效。
+
+learned mode 还必须证明：训练和普通 diagnostic 的每个 mixer output 使用父 runner 数值
+路径；custom Function 不得被训练 step 调用。
+
+### 8.2 初始化负控制
+
+step 0 的 mixer query 全为零，learned weights 是均匀分布。三模式的六位置
+`element_rms`：
+
+- 32 个 raw values 必须 finite 且严格大于 0；
+- raw spectra 的逐层相对差必须 ≤ `1e-6`；
+- normalized spectra 最大绝对差必须 ≤ `1e-6`。
+
+失败则说明 intervention 实现没有隔离预期路径。
+
+### 8.3 softmax / key derivative path
+
+最终 `post_mlp_state`，且 reduction 固定为 `element_rms`：
+
+```text
+relative_drop_contrast =
+  (contrast_learned - contrast_detached) / contrast_learned
+
+relative_drop_peak =
+  (peak_learned - peak_detached) / peak_learned
+```
+
+`contrast_learned` 与 `peak_learned` 必须 finite 且 `>1e-30`，否则本格 invalid 并停止
+聚合。若两项都 `≥20%` 且 3 / 3 seed 同向，记为：
+
+> 全局切断所有 mixer 的 softmax / query / key source-gradient path，在本阈值下对固定
+> 尖峰指标有 material sensitivity。
+
+禁止缩写成“layer 21–25 由 softmax/key path 造成”。
+
+### 8.4 learned value coefficients
+
+用 `detached_learned → uniform_value_backward` 的同一公式，分母也必须 finite 且
+`>1e-30`。若 contrast 和 peak 都下降 `≥20%` 且 3 / 3 seed 同向，记为：
+
+> 全部 mixer 的 value backward coefficients 从 learned `w` 改成 `1/N`，在本阈值下
+> 对固定尖峰指标有 material sensitivity。
+
+`N` 随 source count 变化；均匀系数不等于各 source 的数值贡献均匀。禁止写成“learned
+value weights 是尖峰唯一原因”。
+
+未过阈值只表示本协议没有达到“material”规则，不证明路径贡献为零。seed 或两个指标方向
+分裂时统一记为 `mixed`。
+
+## 9. mixer 分布摘要
+
+每个子层 mixer 保存：
+
+- source count；
+- source labels；
+- 每 source mean / p05 / median / p95；
+- entropy mean；
+- normalized entropy；
+- max source mass；
+- latest-source mass；
+- group / layer / attention-or-MLP 身份。
+
+相关分析报告 Pearson 与 Spearman，但永远标为 association；只有第 7–8 节的同前向
+backward intervention 可以支持路径贡献判断。
+
+不保存可还原语料内容的逐 Token weight arrays。
+
+## 10. 训练等价与复现闸门
+
+每个 formal seed 必须与对应 Round 05 depth-32 / Block raw 结果满足：
+
+- final model-state hash exact；
+- final optimizer-state hash exact；
+- 六个 validation BPC exact；
+- 六个 Round 05 post-MLP element-RMS arrays exact；
+- training-history 冻结字段 exact。
+
+此外，step-0 smoke 对六个位置逐一执行同一 loss 的 `×1 / ×2` backward：
+
+- 每层每 reduction 中线性尺度应乘 2 的指标，其比值在 `2±1e-5`；
+- normalized spectrum、CV、spike contrast 的绝对差 ≤ `1e-6`；
+- shape、dtype、gradient presence / finite 与同位置 storage uniqueness 全部过闸。
+
+普通 diagnostic 与三种 intervention 前后 optimizer-state hash 必须 exact；训练 step
+禁止 `capture=True`。
+
+seed-1 replay 还必须与 Round 06 formal seed-1 的以下字段 exact：
+
+- 全部训练等价字段；
+- 六位置、七 reductions、三 intervention 的全部数值；
+- mixer quantile summaries；
+- logits / activation tensor hashes；
+- final model / optimizer hashes。
+
+排除：
+
+- run kind / output path；
+- wall time / step time；
+- peak allocated / reserved memory；
+- process / host 瞬时字段。
+
+任一训练等价字段失败，不能聚合机制结果；replay 失败则公开失败并停止网站结论。
+
+## 11. 正式工件
+
+```text
+experiments/k3/attnres_spike/
+  README.md
+  manifest.json
+  scope.py
+  train.py
+  analyze.py
+  reproduction.json
+  results/raw/*.json
+
+research/
+  K3_ATTNRES_SPIKE_SCOPING.md
+  K3_ATTNRES_SPIKE_PROTOCOL.md
+  K3_ATTNRES_SPIKE_AUDIT.md
+
+src/data/
+  k3-attnres-spike.json
+  k3-attnres-spike-compact.json
+```
+
+网站必须把四种证据身份分开：
+
+- Round 05 已知峰；
+- observational mixer association；
+- same-forward backward intervention；
+- reduction robustness / failure。
+
+## 12. 停止规则
+
+出现以下任一情况立即停止正式聚合：
+
+- 训练 hash 不匹配 Round 05；
+- forward identity gate 失败；
+- step-0 uniform negative control 失败；
+- 六位置 loss×2 或 optimizer isolation gate 失败；
+- activation gradient 缺失、非 finite 或 shape 错误；
+- replay 数值不 exact；
+- 正式 raw 文件缺失或 canonical hash 不闭合。
+
+遇到反结果时不修改集合 `S`、20% 阈值、1.5 contrast、top-5 overlap 或 Spearman 阈值。

research/K3_ATTNRES_SPIKE_SCOPING.md

@@ -0,0 +1,120 @@
+# K3 Attention Residuals 局部梯度尖峰：Round 06 前置定位
+
+研究日期：2026-07-30  
+阶段身份：**探索性 scoping，不是 Round 06 预注册结果**  
+上游协议：`llm-atlas-k3-attnres-gradient-scale-v1`
+
+## 1. 为什么先做 scoping
+
+Round 05 已经公开了一个方向分裂：
+
+- Block AttnRes 在 depth 16 / 32、三个 seed 的 6 / 6 配对中都改善首/末四分位失衡；
+- 但它也在 6 / 6 配对中提高全层 activation-gradient CV；
+- depth 32 的三 seed 平均峰值集中在 layer 21–25。
+
+这意味着下一步不该再问一个笼统的“梯度是否均匀”，而应该问：
+
+1. 尖峰在 attention / MLP block 的哪个位置已经出现？
+2. 它是否与 learned mixer 的 source 权重有关？
+3. 相关来自 mixer softmax / key 的导数路径，还是来自 learned value coefficients？
+4. 换一种公开、合理的 gradient reduction 后，尖峰是否还存在？
+
+Round 05 已保存每个诊断点的逐层 post-MLP gradient RMS 和 64 个子层 mixer
+分布。这里先用那些**已经见过的数据**定位假设，再冻结新协议。Round 06 的因果式
+backward-path intervention 不能被描述成盲验证。
+
+## 2. 对齐规则
+
+depth 32 的 Block AttnRes 使用 8 个 aggregation groups，每组 4 个 Transformer
+blocks、8 个 attention / MLP residual sublayers。
+
+对 1-based Transformer layer `l`：
+
+```text
+attention mixer index = 2 × (l - 1)
+MLP mixer index       = 2 × (l - 1) + 1
+group                 = floor((l - 1) / 4) + 1
+offset in group       = ((l - 1) mod 4) + 1
+```
+
+`latest source mass` 定义为对应 mixer `mean_weights` 的最后一个元素。它在不同
+group offset 的语义并不完全相同：
+
+- group 首层 attention mixer 没有 current partial，最后源是最近完成的 group；
+- 其他 attention mixer 的最后源是 current partial；
+- 每层 MLP mixer 的最后源都是刚加入 attention branch 后的 current partial。
+
+因此不能把所有 `last` 自动命名为同一种“最近层贡献”。
+
+## 3. layer 19–28 的三 seed 均值
+
+下表的 gradient 是每个 seed 先按该 seed 的 32 层均值归一化，再跨 seed 平均。
+权重和 normalized entropy 也跨三个 seed 平均。
+
+| layer | group / offset | gradient / layer mean | attn last | MLP last | attn H/logN | MLP H/logN |
+|---:|---:|---:|---:|---:|---:|---:|
+| 19 | 5 / 3 | 1.063 | 0.349 | 0.452 | 0.929 | 0.842 |
+| 20 | 5 / 4 | 1.014 | 0.316 | 0.419 | 0.946 | 0.863 |
+| 21 | 6 / 1 | **3.042** | 0.250 | **0.513** | 0.969 | 0.763 |
+| 22 | 6 / 2 | **2.405** | 0.436 | **0.563** | 0.851 | 0.721 |
+| 23 | 6 / 3 | **1.910** | 0.400 | **0.551** | 0.879 | 0.737 |
+| 24 | 6 / 4 | **1.548** | 0.384 | **0.515** | 0.889 | 0.771 |
+| 25 | 7 / 1 | **1.769** | 0.317 | 0.358 | 0.934 | 0.766 |
+| 26 | 7 / 2 | 1.369 | 0.343 | 0.420 | 0.876 | 0.764 |
+| 27 | 7 / 3 | 0.986 | 0.325 | 0.402 | 0.891 | 0.775 |
+| 28 | 7 / 4 | 0.802 | 0.296 | 0.304 | 0.902 | 0.822 |
+
+layer 21–24 正好是第 6 个 group，layer 25 是第 7 个 group 的首层。梯度峰并不
+只是一个 group boundary 单点；它在第 6 组内部递减，并在下一组首层出现较小的第二峰。
+
+## 4. 相关线索
+
+把三个 seed 的 32 层合成 96 个点，post-MLP normalized gradient 与 mixer 摘要的
+Pearson 相关为：
+
+| 变量 | 96 点 `r` | layer 19–28 的 30 点 `r` |
+|---|---:|---:|
+| attention latest-source mass | 0.066 | 0.158 |
+| MLP latest-source mass | **0.651** | **0.690** |
+| attention normalized entropy | −0.296 | −0.057 |
+| MLP normalized entropy | −0.274 | **−0.636** |
+| attention max source mass | −0.024 | 0.158 |
+| MLP max source mass | 0.230 | **0.702** |
+
+这些数值只支持：
+
+> 尖峰层与更集中的 MLP mixer、尤其较大的 latest-partial mean weight 同时出现。
+
+它们不支持：
+
+- “MLP latest weight 导致梯度尖峰”；
+- “第 6 group 是唯一原因”；
+- “降低 entropy 就一定增加梯度”；
+- “K3 真实 checkpoint 也有相同模式”。
+
+同一个 learned mixer 同时改变 forward activation、value-path gradient coefficient
+和 softmax/key derivative path。单看 observational correlation 无法分解这三者。
+
+## 5. Round 06 要冻结的可证伪问题
+
+Round 06 将复跑与 Round 05 完全相同的 depth-32 / Block 三 seed 训练，并要求最终
+model / optimizer hash 与 Round 05 exact。新增诊断只在 optimizer step 之外执行。
+
+它将同时测量：
+
+- pre-attention input；
+- attention branch output；
+- post-attention partial state；
+- pre-MLP input；
+- MLP branch output；
+- post-MLP partial state；
+- 同一 raw gradient tensor 的多种 reduction；
+- learned backward；
+- learned weights detached backward；
+- learned forward + uniform value backward。
+
+其中后两项保持**前向 logits、loss 与 activation 完全相同**，只改变反向路径。这样才能
+区分“相关”与“哪条 backward path 对尖峰有实质贡献”。
+
+完整阈值、失败规则和复现合同见
+`research/K3_ATTNRES_SPIKE_PROTOCOL.md`。

scripts/check-agents-browser.mjs

@@ -228,7 +228,7 @@ if (numeric(reliability.initial.passAt) <= numeric(reliability.k2.passAt) || num
 if (reliability.nonIdempotent.sideRisk === "LOW") failures.push("非幂等写操作风险没有提升");
 if (numeric(rl.wait.utilization) >= numeric(rl.full.utilization) || numeric(rl.wait.lostWork) <= numeric(rl.full.lostWork)) failures.push("wait-all 长尾/重算方向异常");
 if (!rl.wait.takeaway.includes("wait-all") || rl.keyboardSelected !== "rl" || rl.keyboardVisible !== "rl") failures.push("长程 RL 解释或键盘导航异常");
-if (home.releaseCards !== 17 || !home.firstRelease.includes("47 页不再压成摘要") || home.firstHref !== "/k3/") failures.push("首页评测首发入口异常");
+if (home.releaseCards !== 17 || !home.firstRelease.includes("局部 uniform routing 进入完整训练后") || home.firstHref !== "/k3/") failures.push("首页 K3 首发入口异常");
 if (home.paperCount !== "486" || papers.total !== 486 || !papers.hasAgentFilter || papers.agentVisible < 52) failures.push("论文库 Agent 标签或论文总数异常");
 if (!mobile.menuVisible || mobile.menuOpen !== "true" || mobile.tabs !== 4) failures.push("移动端导航或实验异常");
 if (exceptions.length) failures.push(`浏览器异常：${exceptions.join(" | ")}`);