wuyang/llm-atlas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

site: explain AttnRes spike path study

Browse Source
This commit is contained in:
wuyang
2026-07-30 12:20:47 +08:00
parent 3cbb163715
commit a1a52d4280
8 changed files with 1190 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/components/K3AttnResSpikeLab.astro
+892
View File
@@ -0,0 +1,892 @@
---
import rawLab from "@/data/k3-attnres-spike-compact.json";
const lab = rawLab as any;
const json = JSON.stringify(lab).replaceAll("<", "\\u003c");
const reductions = lab.verdicts.reduction_robustness.cells;
const replayHash = lab.gates.replay.frozen_compare_sha256;
const shortHash = (value: string) => `${value.slice(0, 10)}…${value.slice(-8)}`;
const reductionLabels: Record<string, string> = {
element_rms: "全元素 RMS",
token_rms_mean: "Token RMS 均值",
token_rms_median: "Token RMS 中位数",
token_rms_p95: "Token RMS P95",
};
---
<figure class="spike-lab" data-spike-lab>
<header class="spike-head">
<div>
<p>ROUND 06 / SPIKE TRAJECTORY × BACKWARD PATH</p>
<h3>尖峰不是出生时就有；它在训练中形成，并对 value 路径敏感</h3>
</div>
<p>
固定 Round 05 的 depth-32 Block 训练格，只在 optimizer 之外增加诊断。
3 个正式 seed、1 个完整重放；所有前向值完全相同，只改变同一次反向传播的局部导数规则。
</p>
</header>
<div class="spike-ledger">
<article><span>FORMAL TARGET BYTES</span><b>196.608M</b><p>3 × 8,000 steps</p></article>
<article><span>FULL REPLAY</span><b>exact</b><p>16 组冻结字段</p></article>
<article><span>VISIBLE POSITIONS</span><b>6 / 6</b><p>每个位置 3 / 3 seed</p></article>
<article class="pass"><span>REDUCTIONS</span><b>12 / 12</b><p>预注册家族内稳健</p></article>
<article class="counter"><span>KEY / SOFTMAX PATH</span><b>−2.0%</b><p>移除后反而略升</p></article>
<article class="pass"><span>VALUE COEFFICIENTS</span><b>−70.2%</b><p>平均 contrast 降幅</p></article>
</div>
<div class="spike-tabs" role="tablist" aria-label="选择 AttnRes 尖峰路径实验视图">
<button type="button" role="tab" data-spike-tab="trajectory" aria-selected="true">
<span>01</span><b>尖峰何时形成</b><small>step 0 → 8,000</small>
</button>
<button type="button" role="tab" data-spike-tab="positions" aria-selected="false">
<span>02</span><b>六个位置都看见什么</b><small>visible ≠ origin</small>
</button>
<button type="button" role="tab" data-spike-tab="reductions" aria-selected="false">
<span>03</span><b>换一种算法还在吗</b><small>12 / 12 cells</small>
</button>
<button type="button" role="tab" data-spike-tab="intervention" aria-selected="false">
<span>04</span><b>哪条反向路径敏感</b><small>same forward · 3 rules</small>
</button>
<button type="button" role="tab" data-spike-tab="mixer" aria-selected="false">
<span>05</span><b>Mixer 线索与证据边界</b><small>association ≠ attribution</small>
</button>
</div>
<section class="spike-panel" data-spike-panel="trajectory">
<div class="panel-lead">
<div><span>I / DEVELOPMENT</span><h4>先问“训练到什么时候出现”，再问“由哪条路径放大”</h4></div>
<p>
尖峰集合 S 固定为 layer 21–25。纵轴 contrast = S 内均值 ÷ 其余层均值：
1 表示没有集中，越大表示梯度越集中在这五层。
</p>
</div>
<div class="lab-controls">
<label>SEED
<select data-spike-time-seed>
<option value="mean">3-SEED MEAN</option>
<option value="2026073001">2026073001</option>
<option value="2026073002">2026073002</option>
<option value="2026073003">2026073003</option>
</select>
</label>
<div>
<button type="button" data-spike-time-metric="spike_contrast" aria-pressed="true">SPIKE CONTRAST</button>
<button type="button" data-spike-time-metric="population_cv" aria-pressed="false">LAYER CV</button>
<button type="button" data-spike-time-metric="peak_normalized" aria-pressed="false">PEAK / MEAN</button>
</div>
</div>
<div class="chart-shell">
<header><b data-spike-time-title>SPIKE CONTRAST · S / REST</b><span data-spike-time-state>3-SEED MEAN</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="六个训练检查点上的尖峰轨迹" data-spike-time-chart>
<g data-spike-time-grid></g>
<line class="threshold" x1="58" x2="900" y1="0" y2="0" data-spike-time-threshold></line>
<polyline class="series copper" points="" data-spike-time-line></polyline>
<g data-spike-time-points></g>
</svg>
<div class="chart-legend"><span><i class="copper"></i>post-MLP activation-gradient</span><span><i class="threshold-key"></i>预注册可见阈值 1.5</span></div>
</div>
<div class="trajectory-story">
<article><span>STEP 0</span><b>0.757×</b><p>三 seed 均低于 1；layer 21–25 并不特殊。</p></article>
<i>→</i>
<article><span>STEP 500</span><b>0.524×</b><p>学习已经发生，但尖峰尚未出现。</p></article>
<i>→</i>
<article class="accent"><span>STEP 2,000</span><b>2.233×</b><p>峰值层同时迁移到 layer 21，结构开始显形。</p></article>
<i>→</i>
<article><span>STEP 8,000</span><b>2.754×</b><p>seed 3 只有 1.881×：方向一致，强度并不整齐。</p></article>
</div>
<div class="plain-rule">
<b>通俗读法</b>
<p>这不像一个由初始化直接写死的“坏层”；它更像训练动力学与 Block mixer 共同塑出的深度模式。</p>
</div>
</section>
<section class="spike-panel" data-spike-panel="positions" hidden>
<div class="panel-lead">
<div><span>II / LOCALIZATION</span><h4>从一个 Block 向前追：尖峰在六个候选张量上都已经可见</h4></div>
<p>
“最早观测到”是沿本站采样点的相对顺序；两个采样点之间仍有 mixer、归一化与残差运算，
所以它不是尖峰的物理出生点。
</p>
</div>
<div class="tensor-path" aria-label="一个 Transformer block 的六个梯度采样位置">
<button type="button" data-spike-position="pre_attention_input" aria-pressed="true"><span>01</span><b>Attn 前输入</b><small>3.724 / 3.189 / 2.040</small></button>
<i>→</i>
<button type="button" data-spike-position="attention_branch_output" aria-pressed="false"><span>02</span><b>Attn 分支输出</b><small>3.200 / 3.662 / 2.005</small></button>
<i>→</i>
<button type="button" data-spike-position="post_attention_state" aria-pressed="false"><span>03</span><b>Attn 后状态</b><small>3.200 / 3.662 / 2.005</small></button>
<i>→</i>
<button type="button" data-spike-position="pre_mlp_input" aria-pressed="false"><span>04</span><b>MLP 前输入</b><small>3.829 / 4.527 / 2.978</small></button>
<i>→</i>
<button type="button" data-spike-position="mlp_branch_output" aria-pressed="false"><span>05</span><b>MLP 分支输出</b><small>3.046 / 3.333 / 1.881</small></button>
<i>→</i>
<button type="button" data-spike-position="post_mlp_state" aria-pressed="false"><span>06</span><b>MLP 后状态</b><small>3.046 / 3.333 / 1.881</small></button>
</div>
<div class="lab-controls position-controls">
<label>SEED
<select data-spike-position-seed>
<option value="2026073001">2026073001</option>
<option value="2026073002">2026073002</option>
<option value="2026073003">2026073003</option>
</select>
</label>
<span data-spike-position-state>ATTENTION 前输入 · SEED 2026073001</span>
</div>
<div class="position-layout">
<div class="chart-shell">
<header><b>NORMALIZED GRADIENT BY LAYER</b><span>layer mean = 1</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="所选张量位置的 32 层归一化梯度谱" data-spike-position-chart>
<rect class="spike-zone" x="0" y="24" width="0" height="280" data-spike-position-zone></rect>
<g data-spike-position-grid></g>
<polyline class="series green" points="" data-spike-position-line></polyline>
<g data-spike-position-points></g>
</svg>
<div class="chart-legend"><span><i class="green"></i>所选位置</span><span><i class="zone-key"></i>S = layer 21–25</span></div>
</div>
<aside>
<span>SELECTED READOUT</span>
<b data-spike-position-label>Attention 前输入</b>
<dl>
<div><dt>SPIKE CONTRAST</dt><dd data-spike-position-contrast>3.724×</dd></div>
<div><dt>PEAK LAYER</dt><dd data-spike-position-peak>22</dd></div>
<div><dt>PEAK / MEAN</dt><dd data-spike-position-peak-value>4.233×</dd></div>
</dl>
<p>六个位置都满足 contrast ≥ 1.5 的 3 / 3 seed 规则。</p>
</aside>
</div>
<div class="boundary-pair">
<article class="yes"><span>可以说</span><b>最早采样点已经看见尖峰</b><p><code>pre_attention_input</code> 是本站六点链中最早的可见位置。</p></article>
<article class="no"><span>不能说</span><b>尖峰起源于 Attention 输入</b><p>观测链不是干预链；更早 mixer 与跨层回传已经作用于该张量。</p></article>
</div>
</section>
<section class="spike-panel" data-spike-panel="reductions" hidden>
<div class="panel-lead">
<div><span>III / ROBUSTNESS</span><h4>把 “RMS” 拆成四种合理算法，layer 21–25 仍然突出</h4></div>
<p>
预注册主家族只包含四种 reduction。每个 seed × reduction 必须同时满足：
contrast ≥ 1.5、top-5 与 S 至少重叠 3 层、相对 element RMS 的 Spearman ≥ .8。
</p>
</div>
<div class="lab-controls">
<label>SEED
<select data-spike-reduction-seed>
<option value="2026073001">2026073001</option>
<option value="2026073002">2026073002</option>
<option value="2026073003">2026073003</option>
</select>
</label>
<label>REDUCTION
<select data-spike-reduction>
<option value="element_rms">全元素 RMS</option>
<option value="token_rms_mean">Token RMS 均值</option>
<option value="token_rms_median">Token RMS 中位数</option>
<option value="token_rms_p95">Token RMS P95</option>
</select>
</label>
<span data-spike-reduction-state>SEED 2026073001 · 全元素 RMS</span>
</div>
<div class="position-layout reduction-layout">
<div class="chart-shell">
<header><b>POST-MLP · NORMALIZED BY LAYER MEAN</b><span>32 layers</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="所选 reduction 下的归一化层梯度谱" data-spike-reduction-chart>
<rect class="spike-zone" x="0" y="24" width="0" height="280" data-spike-reduction-zone></rect>
<g data-spike-reduction-grid></g>
<polyline class="series copper" points="" data-spike-reduction-line></polyline>
<g data-spike-reduction-points></g>
</svg>
</div>
<aside>
<span>THREE GATES</span>
<dl>
<div><dt>CONTRAST ≥ 1.5</dt><dd data-spike-reduction-contrast>3.046×</dd></div>
<div><dt>TOP-5 OVERLAP ≥ 3</dt><dd data-spike-reduction-overlap>4 / 5</dd></div>
<div><dt>SPEARMAN ≥ .8</dt><dd data-spike-reduction-rho>1.000</dd></div>
</dl>
<p class="pass-copy">PASS · 三个条件必须同时成立</p>
</aside>
</div>
<div class="reduction-table-wrap">
<table class="reduction-table">
<thead><tr><th>SEED</th><th>REDUCTION</th><th>CONTRAST</th><th>TOP-5 ∩ S</th><th>SPEARMAN</th><th>VERDICT</th></tr></thead>
<tbody>
{reductions.map((cell: any) => (
<tr>
<th>{cell.seed}</th>
<td>{reductionLabels[cell.reduction]}</td>
<td>{cell.spike_contrast.toFixed(3)}×</td>
<td>{cell.top_five_spike_overlap} / 5</td>
<td>{cell.spearman_vs_element_rms.toFixed(3)}</td>
<td class="good">PASS</td>
</tr>
))}
</tbody>
</table>
</div>
<div class="plain-rule">
<b>判定原文</b><p><code>robust within the preregistered reduction family</code>——只对这四种、这三个 seed、这个诊断 batch 成立。</p>
</div>
</section>
<section class="spike-panel" data-spike-panel="intervention" hidden>
<div class="panel-lead">
<div><span>IV / SAME-FORWARD INTERVENTION</span><h4>前向数值一字不动，只问梯度经过 mixer 时沿哪条边回去</h4></div>
<p>
干预覆盖全部 64 个 depth mixer 与 output mixer。它能定位“反向路径敏感性”，
不能替代重新训练，更不能把降幅解释成因果贡献百分比。
</p>
</div>
<div class="backward-diagram">
<div class="forward"><span>FORWARD / ALL MODES EXACT</span><b>α = softmax(q·k)</b><i>→</i><b>y = Σ αᵢvᵢ</b><i>→</i><b>same logits · same loss</b></div>
<div class="paths">
<article><span>LEARNED</span><b>∂q, ∂k, ∂v</b><p>正常回传：权重路径和值路径都保留。</p></article>
<article><span>DETACHED LEARNED</span><b><s>∂q, ∂k</s> · ∂v</b><p>α 数值不变，但切断 softmax / query / key 源梯度。</p></article>
<article class="accent"><span>UNIFORM VALUE BACKWARD</span><b><s>α</s> → 1/N</b><p>前向仍用 learned α；仅 value-backward 系数改成均匀。</p></article>
</div>
</div>
<div class="lab-controls">
<label>SEED
<select data-spike-intervention-seed>
<option value="2026073001">2026073001</option>
<option value="2026073002">2026073002</option>
<option value="2026073003">2026073003</option>
</select>
</label>
<span data-spike-intervention-state>SEED 2026073001 · THREE BACKWARD RULES</span>
</div>
<div class="chart-shell">
<header><b>POST-MLP NORMALIZED GRADIENT</b><span>same forward · different backward</span></header>
<svg viewBox="0 0 940 350" role="img" aria-label="三种反向规则下的 32 层梯度谱" data-spike-intervention-chart>
<rect class="spike-zone" x="0" y="24" width="0" height="280" data-spike-intervention-zone></rect>
<g data-spike-intervention-grid></g>
<polyline class="series gray" points="" data-spike-intervention-line="learned"></polyline>
<polyline class="series copper dash" points="" data-spike-intervention-line="detached_learned"></polyline>
<polyline class="series green" points="" data-spike-intervention-line="uniform_value_backward"></polyline>
</svg>
<div class="chart-legend">
<span><i class="gray"></i>learned</span>
<span><i class="copper dashed"></i>detached learned</span>
<span><i class="green"></i>uniform value backward</span>
</div>
</div>
<div class="intervention-results">
<article class="counter">
<span>LEARNED → DETACHED</span>
<b data-spike-key-drop>−1.54%</b>
<p>contrast “降幅”为负；移除 key / softmax 路径后，峰值在 3 / 3 seed 略微上升。</p>
</article>
<article class="pass">
<span>DETACHED → UNIFORM VALUE</span>
<b data-spike-value-drop>74.79%</b>
<p>contrast 在 3 / 3 seed 至少下降 20%，达到预注册 material sensitivity 阈值。</p>
</article>
<article>
<span>PEAK LAYER</span>
<b data-spike-peak-shift>21 → 2</b>
<p>均匀 value-backward 后，三个 seed 的最高层都从中后段移到 layer 2。</p>
</article>
</div>
<div class="causal-warning">
<b>最强允许结论</b>
<p>该尖峰对“全局 learned value-backward coefficients”具有材料级敏感性；softmax / query / key 源梯度路径没有显示同方向材料级作用。</p>
<span>不是训练出的 uniform 模型 · 不是 70.2% 因果贡献 · 不是某个局部 mixer 的归因</span>
</div>
</section>
<section class="spike-panel" data-spike-panel="mixer" hidden>
<div class="panel-lead">
<div><span>V / ASSOCIATION × AUDIT</span><h4>mixer 权重给出一条可追的线索；干预才把它推进到“路径敏感”</h4></div>
<p>
每个点是 layer 19–28 的一个 layer × seed。横轴是 MLP mixer 的统计量，
纵轴是归一化梯度；相关性仍然只是观察关系。
</p>
</div>
<div class="lab-controls">
<div>
<button type="button" data-spike-mixer-metric="mlp_latest" aria-pressed="true">LATEST-SOURCE WEIGHT</button>
<button type="button" data-spike-mixer-metric="mlp_entropy" aria-pressed="false">NORMALIZED ENTROPY</button>
</div>
<span data-spike-mixer-state>MLP LATEST · LAYERS 19–28 · N=30</span>
</div>
<div class="mixer-layout">
<div class="chart-shell">
<header><b data-spike-mixer-title>MLP LATEST-SOURCE WEIGHT × NORMALIZED GRADIENT</b><span>3 seeds</span></header>
<svg viewBox="0 0 940 390" role="img" aria-label="Mixer 统计量和层梯度的散点关系" data-spike-mixer-chart>
<g data-spike-mixer-grid></g>
<line class="fit" x1="0" x2="0" y1="0" y2="0" data-spike-mixer-fit></line>
<g data-spike-mixer-points></g>
</svg>
</div>
<aside>
<span>LAYERS 19–28</span>
<dl>
<div><dt>PEARSON</dt><dd data-spike-mixer-pearson>+.690</dd></div>
<div><dt>SPEARMAN</dt><dd data-spike-mixer-spearman>+.693</dd></div>
<div><dt>POINTS</dt><dd>30</dd></div>
</dl>
<p data-spike-mixer-copy>越依赖最新 residual source 的 MLP mixer，局部梯度往往越高。</p>
</aside>
</div>
<div class="evidence-ladder">
<article><span>1 / OBSERVE</span><b>layer 21–25 集中</b><p>六位置、四 reduction、三 seed 都能看见。</p></article>
<i>→</i>
<article><span>2 / ASSOCIATE</span><b>MLP latest r≈.69</b><p>相关性提供候选机制，不承担因果结论。</p></article>
<i>→</i>
<article class="accent"><span>3 / INTERVENE</span><b>value route −70.2%</b><p>全局反向规则干预支持路径敏感性。</p></article>
<i>→</i>
<article><span>4 / NEXT GATE</span><b>局部 group 6 / 7</b><p>下一轮冻结局部 mixer 的干预矩阵。</p></article>
</div>
<div class="audit-grid">
<article><span>ROUND 05 EQUIVALENCE</span><b>3 / 3 exact</b><p>model、optimizer、history、六个 BPC 与全部 post-MLP 数组一致。</p></article>
<article><span>FULL REPLAY</span><b>16 / 16 exact</b><p>compare hash <code>{shortHash(replayHash)}</code></p></article>
<article><span>NEGATIVE CONTROL</span><b>step 0 exact</b><p>query-zero 与 loss×2 gate 在三 seed 全部通过。</p></article>
<article class="boundary"><span>EVIDENCE IDENTITY</span><b>targeted follow-up</b><p>尖峰集合来自 Round 05；本轮不是盲发现研究。</p></article>
</div>
<div class="hash-strip">
<span>PROTOCOL <code>{lab.protocol_id}</code></span>
<span>REPLAY <code>{shortHash(replayHash)}</code></span>
<span>COMPACT <code>{shortHash(lab.canonical_sha256_without_self)}</code></span>
</div>
</section>
<script is:inline type="application/json" data-spike-payload set:html={json}></script>
</figure>
<script>
const initializeSpikeLab = (root: HTMLElement) => {
if (root.dataset.ready === "true") return;
root.dataset.ready = "true";
const payload = root.querySelector<HTMLScriptElement>("[data-spike-payload]");
if (!payload) return;
const data = JSON.parse(payload.textContent || "{}");
const ns = "http://www.w3.org/2000/svg";
const seeds = data.study.seeds;
const steps = data.trajectory[0].points.map((point: any) => point.step);
const labels: Record<string, string> = {
pre_attention_input: "Attention 前输入",
attention_branch_output: "Attention 分支输出",
post_attention_state: "Attention 后状态",
pre_mlp_input: "MLP 前输入",
mlp_branch_output: "MLP 分支输出",
post_mlp_state: "MLP 后状态",
element_rms: "全元素 RMS",
token_rms_mean: "Token RMS 均值",
token_rms_median: "Token RMS 中位数",
token_rms_p95: "Token RMS P95",
};
const colors = ["#ba603b", "#163f3b", "#77746b"];
const average = (values: number[]) => values.reduce((sum, value) => sum + value, 0) / values.length;
const svgNode = (name: string, attrs: Record<string, string | number>) => {
const node = document.createElementNS(ns, name);
Object.entries(attrs).forEach(([key, value]) => node.setAttribute(key, String(value)));
return node;
};
const seedFinal = (seed: number) => data.final_arrays.find((row: any) => row.seed === seed);
const formatSignedPct = (value: number) => `${value < 0 ? "−" : "+"}${Math.abs(value * 100).toFixed(2)}%`;
const tabs = Array.from(root.querySelectorAll<HTMLButtonElement>("[data-spike-tab]"));
const selectTab = (name: string) => {
tabs.forEach((tab) => tab.setAttribute("aria-selected", String(tab.dataset.spikeTab === name)));
root.querySelectorAll<HTMLElement>("[data-spike-panel]").forEach((panel) => {
panel.hidden = panel.dataset.spikePanel !== name;
});
};
tabs.forEach((tab, index) => {
tab.addEventListener("click", () => selectTab(tab.dataset.spikeTab || "trajectory"));
tab.addEventListener("keydown", (event) => {
if (!["ArrowLeft", "ArrowRight", "Home", "End"].includes(event.key)) return;
event.preventDefault();
const next = event.key === "Home" ? 0
: event.key === "End" ? tabs.length - 1
: (index + (event.key === "ArrowRight" ? 1 : -1) + tabs.length) % tabs.length;
tabs[next].focus();
selectTab(tabs[next].dataset.spikeTab || "trajectory");
});
});
const drawLine = (
svg: SVGSVGElement,
grid: SVGGElement,
series: Record<string, number[]>,
xLabels: string[],
options: {
formatter?: (value: number) => string;
zero?: boolean;
pointGroup?: SVGGElement | null;
threshold?: number;
thresholdLine?: SVGLineElement | null;
zone?: SVGRectElement | null;
} = {},
) => {
grid.replaceChildren();
const allValues = Object.values(series).flat();
const rawMin = options.zero === false ? Math.min(...allValues) : 0;
const rawMax = Math.max(...allValues, options.threshold || 0);
const padding = Math.max(1e-8, (rawMax - rawMin) * .08);
const min = options.zero === false ? rawMin - padding : 0;
const max = rawMax + padding;
const left = 58;
const right = 900;
const top = 24;
const bottom = 304;
const xAt = (index: number) => left + index / Math.max(1, xLabels.length - 1) * (right - left);
const yAt = (value: number) => top + (max - value) / Math.max(1e-30, max - min) * (bottom - top);
const formatter = options.formatter || ((value: number) => value.toFixed(2));
for (let index = 0; index < 5; index += 1) {
const value = max - index / 4 * (max - min);
const y = top + index / 4 * (bottom - top);
grid.append(svgNode("line", { x1: left, x2: right, y1: y, y2: y, class: "grid-line" }));
const text = svgNode("text", { x: left - 10, y: y + 4, class: "axis-label y" });
text.textContent = formatter(value);
grid.append(text);
}
xLabels.forEach((label, index) => {
if (xLabels.length > 12 && ![0, 3, 7, 11, 15, 19, 20, 24, 27, 31].includes(index)) return;
const text = svgNode("text", { x: xAt(index), y: 328, class: "axis-label x" });
text.textContent = label;
grid.append(text);
});
if (options.thresholdLine && options.threshold !== undefined) {
const y = yAt(options.threshold);
options.thresholdLine.setAttribute("y1", String(y));
options.thresholdLine.setAttribute("y2", String(y));
}
if (options.zone && xLabels.length === 32) {
const start = xAt(20) - (right - left) / 62;
const end = xAt(24) + (right - left) / 62;
options.zone.setAttribute("x", String(start));
options.zone.setAttribute("width", String(end - start));
}
Object.entries(series).forEach(([key, values], seriesIndex) => {
const line = svg.querySelector<SVGPolylineElement>(`[data-line-key="${key}"]`)
|| svg.querySelectorAll<SVGPolylineElement>("polyline")[seriesIndex];
const points = values.map((value, index) => `${xAt(index)},${yAt(value)}`).join(" ");
line?.setAttribute("points", points);
if (seriesIndex === 0 && options.pointGroup) {
options.pointGroup.replaceChildren();
values.forEach((value, index) => {
const circle = svgNode("circle", { cx: xAt(index), cy: yAt(value), r: 4, fill: colors[seriesIndex] });
const title = svgNode("title", {});
title.textContent = `${xLabels[index]} · ${formatter(value)}`;
circle.append(title);
options.pointGroup?.append(circle);
});
}
});
return { min, max, xAt, yAt };
};
let timeMetric = "spike_contrast";
const timeSeed = root.querySelector<HTMLSelectElement>("[data-spike-time-seed]")!;
const timeChart = root.querySelector<SVGSVGElement>("[data-spike-time-chart]")!;
const updateTime = () => {
const selected = timeSeed.value === "mean"
? data.trajectory
: data.trajectory.filter((row: any) => String(row.seed) === timeSeed.value);
const values = steps.map((_: number, index: number) =>
average(selected.map((row: any) => row.points[index][timeMetric])));
const names: Record<string, string> = {
spike_contrast: "SPIKE CONTRAST · S / REST",
population_cv: "POPULATION CV ACROSS 32 LAYERS",
peak_normalized: "PEAK / LAYER MEAN",
};
root.querySelector<HTMLElement>("[data-spike-time-title]")!.textContent = names[timeMetric];
root.querySelector<HTMLElement>("[data-spike-time-state]")!.textContent =
timeSeed.value === "mean" ? "3-SEED MEAN" : `SEED ${timeSeed.value}`;
drawLine(
timeChart,
timeChart.querySelector("[data-spike-time-grid]")!,
{ trajectory: values },
steps.map((step: number) => step >= 1000 ? `${step / 1000}K` : String(step)),
{
formatter: (value) => value.toFixed(2),
pointGroup: timeChart.querySelector<SVGGElement>("[data-spike-time-points]"),
threshold: timeMetric === "spike_contrast" ? 1.5 : undefined,
thresholdLine: timeChart.querySelector<SVGLineElement>("[data-spike-time-threshold]"),
},
);
const threshold = timeChart.querySelector<SVGLineElement>("[data-spike-time-threshold]");
if (threshold) threshold.style.display = timeMetric === "spike_contrast" ? "" : "none";
};
root.querySelectorAll<HTMLButtonElement>("[data-spike-time-metric]").forEach((button) => button.addEventListener("click", () => {
timeMetric = button.dataset.spikeTimeMetric || "spike_contrast";
root.querySelectorAll<HTMLButtonElement>("[data-spike-time-metric]").forEach((peer) =>
peer.setAttribute("aria-pressed", String(peer === button)));
updateTime();
}));
timeSeed.addEventListener("change", updateTime);
updateTime();
let positionName = "pre_attention_input";
const positionSeed = root.querySelector<HTMLSelectElement>("[data-spike-position-seed]")!;
const positionChart = root.querySelector<SVGSVGElement>("[data-spike-position-chart]")!;
const updatePosition = () => {
const seed = Number(positionSeed.value);
const cell = seedFinal(seed).position_element_rms[positionName];
drawLine(
positionChart,
positionChart.querySelector("[data-spike-position-grid]")!,
{ position: cell.statistics.normalized },
Array.from({ length: 32 }, (_, index) => String(index + 1)),
{
formatter: (value) => `${value.toFixed(1)}×`,
pointGroup: positionChart.querySelector<SVGGElement>("[data-spike-position-points]"),
zone: positionChart.querySelector<SVGRectElement>("[data-spike-position-zone]"),
},
);
root.querySelector<HTMLElement>("[data-spike-position-state]")!.textContent = `${labels[positionName].toUpperCase()} · SEED ${seed}`;
root.querySelector<HTMLElement>("[data-spike-position-label]")!.textContent = labels[positionName];
root.querySelector<HTMLElement>("[data-spike-position-contrast]")!.textContent = `${cell.statistics.spike_contrast.toFixed(3)}×`;
root.querySelector<HTMLElement>("[data-spike-position-peak]")!.textContent = String(cell.statistics.peak_layer);
root.querySelector<HTMLElement>("[data-spike-position-peak-value]")!.textContent = `${cell.statistics.peak_normalized.toFixed(3)}×`;
};
root.querySelectorAll<HTMLButtonElement>("[data-spike-position]").forEach((button) => button.addEventListener("click", () => {
positionName = button.dataset.spikePosition || "pre_attention_input";
root.querySelectorAll<HTMLButtonElement>("[data-spike-position]").forEach((peer) =>
peer.setAttribute("aria-pressed", String(peer === button)));
updatePosition();
}));
positionSeed.addEventListener("change", updatePosition);
updatePosition();
const reductionSeed = root.querySelector<HTMLSelectElement>("[data-spike-reduction-seed]")!;
const reductionSelect = root.querySelector<HTMLSelectElement>("[data-spike-reduction]")!;
const reductionChart = root.querySelector<SVGSVGElement>("[data-spike-reduction-chart]")!;
const updateReduction = () => {
const seed = Number(reductionSeed.value);
const name = reductionSelect.value;
const cell = seedFinal(seed).post_mlp_reductions[name];
const verdict = data.verdicts.reduction_robustness.cells.find((row: any) => row.seed === seed && row.reduction === name);
drawLine(
reductionChart,
reductionChart.querySelector("[data-spike-reduction-grid]")!,
{ reduction: cell.statistics.normalized },
Array.from({ length: 32 }, (_, index) => String(index + 1)),
{
formatter: (value) => `${value.toFixed(1)}×`,
pointGroup: reductionChart.querySelector<SVGGElement>("[data-spike-reduction-points]"),
zone: reductionChart.querySelector<SVGRectElement>("[data-spike-reduction-zone]"),
},
);
root.querySelector<HTMLElement>("[data-spike-reduction-state]")!.textContent = `SEED ${seed} · ${labels[name]}`;
root.querySelector<HTMLElement>("[data-spike-reduction-contrast]")!.textContent = `${verdict.spike_contrast.toFixed(3)}×`;
root.querySelector<HTMLElement>("[data-spike-reduction-overlap]")!.textContent = `${verdict.top_five_spike_overlap} / 5`;
root.querySelector<HTMLElement>("[data-spike-reduction-rho]")!.textContent = verdict.spearman_vs_element_rms.toFixed(3);
};
reductionSeed.addEventListener("change", updateReduction);
reductionSelect.addEventListener("change", updateReduction);
updateReduction();
const interventionSeed = root.querySelector<HTMLSelectElement>("[data-spike-intervention-seed]")!;
const interventionChart = root.querySelector<SVGSVGElement>("[data-spike-intervention-chart]")!;
const updateIntervention = () => {
const seed = Number(interventionSeed.value);
const cell = seedFinal(seed);
const modes = ["learned", "detached_learned", "uniform_value_backward"];
const series = Object.fromEntries(modes.map((mode) => [mode, cell.interventions[mode].statistics.normalized]));
modes.forEach((mode) => interventionChart.querySelector(`[data-spike-intervention-line="${mode}"]`)?.setAttribute("data-line-key", mode));
drawLine(
interventionChart,
interventionChart.querySelector("[data-spike-intervention-grid]")!,
series,
Array.from({ length: 32 }, (_, index) => String(index + 1)),
{
formatter: (value) => `${value.toFixed(1)}×`,
zone: interventionChart.querySelector<SVGRectElement>("[data-spike-intervention-zone]"),
},
);
const keyRow = data.verdicts.interventions.softmax_key_path.per_seed.find((row: any) => row.seed === seed);
const valueRow = data.verdicts.interventions.value_coefficients.per_seed.find((row: any) => row.seed === seed);
root.querySelector<HTMLElement>("[data-spike-intervention-state]")!.textContent = `SEED ${seed} · THREE BACKWARD RULES`;
root.querySelector<HTMLElement>("[data-spike-key-drop]")!.textContent = formatSignedPct(keyRow.relative_drop_contrast);
root.querySelector<HTMLElement>("[data-spike-value-drop]")!.textContent = `${(valueRow.relative_drop_contrast * 100).toFixed(2)}%`;
root.querySelector<HTMLElement>("[data-spike-peak-shift]")!.textContent =
`${cell.interventions.detached_learned.statistics.peak_layer} → ${cell.interventions.uniform_value_backward.statistics.peak_layer}`;
};
interventionSeed.addEventListener("change", updateIntervention);
updateIntervention();
let mixerMetric = "mlp_latest";
const mixerChart = root.querySelector<SVGSVGElement>("[data-spike-mixer-chart]")!;
const mixerPoints = data.mixer_associations.target_layers.flatMap((row: any) =>
row.per_seed_normalized_gradient.map((gradient: number, index: number) => ({
x: row[mixerMetric],
y: gradient,
seed: seeds[index],
layer: row.layer,
})));
const updateMixer = () => {
const points = data.mixer_associations.target_layers.flatMap((row: any) =>
row.per_seed_normalized_gradient.map((gradient: number, index: number) => ({
x: row[mixerMetric],
y: gradient,
seed: seeds[index],
layer: row.layer,
})));
const group = mixerChart.querySelector<SVGGElement>("[data-spike-mixer-points]")!;
const grid = mixerChart.querySelector<SVGGElement>("[data-spike-mixer-grid]")!;
const fit = mixerChart.querySelector<SVGLineElement>("[data-spike-mixer-fit]")!;
group.replaceChildren();
grid.replaceChildren();
const left = 72, right = 900, top = 24, bottom = 330;
const xs = points.map((point: any) => point.x);
const ys = points.map((point: any) => point.y);
const minX = Math.min(...xs) * .94;
const maxX = Math.max(...xs) * 1.04;
const minY = Math.max(0, Math.min(...ys) * .88);
const maxY = Math.max(...ys) * 1.08;
const xAt = (value: number) => left + (value - minX) / (maxX - minX) * (right - left);
const yAt = (value: number) => top + (maxY - value) / (maxY - minY) * (bottom - top);
for (let index = 0; index < 5; index += 1) {
const yValue = maxY - index / 4 * (maxY - minY);
const y = top + index / 4 * (bottom - top);
grid.append(svgNode("line", { x1: left, x2: right, y1: y, y2: y, class: "grid-line" }));
const yText = svgNode("text", { x: left - 10, y: y + 4, class: "axis-label y" });
yText.textContent = `${yValue.toFixed(1)}×`;
grid.append(yText);
const xValue = minX + index / 4 * (maxX - minX);
const x = left + index / 4 * (right - left);
const xText = svgNode("text", { x, y: 355, class: "axis-label x" });
xText.textContent = xValue.toFixed(2);
grid.append(xText);
}
points.forEach((point: any, index: number) => {
const circle = svgNode("circle", {
cx: xAt(point.x), cy: yAt(point.y), r: point.layer >= 21 && point.layer <= 25 ? 6 : 4,
fill: colors[index % 3], opacity: point.layer >= 21 && point.layer <= 25 ? .95 : .55,
});
const title = svgNode("title", {});
title.textContent = `L${point.layer} · ${point.seed} · x=${point.x.toFixed(3)} · gradient=${point.y.toFixed(3)}×`;
circle.append(title);
group.append(circle);
});
const meanX = average(xs);
const meanY = average(ys);
const slope = xs.reduce((sum: number, x: number, index: number) => sum + (x - meanX) * (ys[index] - meanY), 0)
/ xs.reduce((sum: number, x: number) => sum + (x - meanX) ** 2, 0);
const intercept = meanY - slope * meanX;
fit.setAttribute("x1", String(xAt(minX)));
fit.setAttribute("x2", String(xAt(maxX)));
fit.setAttribute("y1", String(yAt(intercept + slope * minX)));
fit.setAttribute("y2", String(yAt(intercept + slope * maxX)));
const stats = data.mixer_associations.layers_19_28[mixerMetric];
root.querySelector<HTMLElement>("[data-spike-mixer-pearson]")!.textContent = `${stats.pearson >= 0 ? "+" : "−"}${Math.abs(stats.pearson).toFixed(3)}`;
root.querySelector<HTMLElement>("[data-spike-mixer-spearman]")!.textContent = `${stats.spearman >= 0 ? "+" : "−"}${Math.abs(stats.spearman).toFixed(3)}`;
const latest = mixerMetric === "mlp_latest";
root.querySelector<HTMLElement>("[data-spike-mixer-state]")!.textContent =
`${latest ? "MLP LATEST" : "MLP ENTROPY"} · LAYERS 19–28 · N=30`;
root.querySelector<HTMLElement>("[data-spike-mixer-title]")!.textContent =
`${latest ? "MLP LATEST-SOURCE WEIGHT" : "MLP NORMALIZED ENTROPY"} × NORMALIZED GRADIENT`;
root.querySelector<HTMLElement>("[data-spike-mixer-copy]")!.textContent = latest
? "越依赖最新 residual source 的 MLP mixer，局部梯度往往越高。"
: "Mixer 越分散、熵越高，局部梯度往往越低；这仍不是局部因果归因。";
};
root.querySelectorAll<HTMLButtonElement>("[data-spike-mixer-metric]").forEach((button) => button.addEventListener("click", () => {
mixerMetric = button.dataset.spikeMixerMetric || "mlp_latest";
root.querySelectorAll<HTMLButtonElement>("[data-spike-mixer-metric]").forEach((peer) =>
peer.setAttribute("aria-pressed", String(peer === button)));
updateMixer();
}));
void mixerPoints;
updateMixer();
};
document.querySelectorAll<HTMLElement>("[data-spike-lab]").forEach(initializeSpikeLab);
document.addEventListener("astro:page-load", () => {
document.querySelectorAll<HTMLElement>("[data-spike-lab]").forEach(initializeSpikeLab);
});
</script>
<style>
.spike-lab {
--s-ink: #1c201e;
--s-muted: #747168;
--s-line: rgba(28, 32, 30, .16);
--s-paper: #f4f0e7;
--s-raised: #faf7ef;
--s-copper: #ba603b;
--s-green: #163f3b;
width: min(1120px, 100%);
margin: 42px 0;
color: var(--s-ink);
border: 1px solid var(--s-line);
background: var(--s-paper);
box-shadow: 0 30px 80px rgba(28, 32, 30, .09);
}
.spike-head {
display: grid;
grid-template-columns: minmax(0, 1.45fr) minmax(260px, .7fr);
gap: 44px;
padding: 30px;
color: #f5efe4;
background: var(--s-green);
}
.spike-head p { margin: 0; color: rgba(245,239,228,.7); font: .65rem/1.7 var(--mono); }
.spike-head div > p { color: #d58a68; letter-spacing: .08em; }
.spike-head h3 { max-width: 760px; margin: 14px 0 0; color: inherit; font-size: clamp(1.15rem, 2.2vw, 1.75rem); line-height: 1.35; }
.spike-ledger { display: grid; grid-template-columns: repeat(6, 1fr); border-bottom: 1px solid var(--s-line); }
.spike-ledger article { min-height: 126px; padding: 18px 15px; border-right: 1px solid var(--s-line); }
.spike-ledger article:last-child { border-right: 0; }
.spike-ledger span, .panel-lead span { color: var(--s-muted); font: .56rem/1.2 var(--mono); letter-spacing: .08em; }
.spike-ledger b { display: block; margin-top: 23px; font: 700 .95rem/1 var(--mono); }
.spike-ledger p { margin: 8px 0 0; color: var(--s-muted); font-size: .6rem; line-height: 1.45; }
.spike-ledger .pass { color: #f7f0e6; background: var(--s-green); }
.spike-ledger .counter { color: #f7f0e6; background: var(--s-copper); }
.spike-ledger .pass span, .spike-ledger .pass p, .spike-ledger .counter span, .spike-ledger .counter p { color: rgba(247,240,230,.72); }
.spike-tabs { display: grid; grid-template-columns: repeat(5, 1fr); border-bottom: 1px solid var(--s-line); background: #e9e4da; }
.spike-tabs button { min-height: 116px; padding: 16px; text-align: left; color: inherit; border: 0; border-right: 1px solid var(--s-line); background: transparent; cursor: pointer; }
.spike-tabs button:last-child { border-right: 0; }
.spike-tabs button[aria-selected="true"] { color: #f7f0e6; background: var(--s-copper); }
.spike-tabs span, .spike-tabs small { display: block; color: var(--s-muted); font: .54rem/1.25 var(--mono); }
.spike-tabs b { display: block; margin: 15px 0 8px; font-size: .69rem; line-height: 1.35; }
.spike-tabs button[aria-selected="true"] span, .spike-tabs button[aria-selected="true"] small { color: rgba(247,240,230,.72); }
.spike-panel { padding: 30px; }
.panel-lead { display: grid; grid-template-columns: 1.05fr .95fr; gap: 48px; align-items: end; margin-bottom: 28px; }
.panel-lead h4 { max-width: 680px; margin: 10px 0 0; font-size: 1.2rem; line-height: 1.4; }
.panel-lead p { margin: 0; color: var(--s-muted); font-size: .7rem; line-height: 1.7; }
.lab-controls { display: flex; flex-wrap: wrap; gap: 10px 18px; align-items: end; margin-bottom: 20px; }
.lab-controls > div { display: flex; }
.lab-controls button, .lab-controls select { min-height: 38px; padding: 10px 12px; color: var(--s-muted); font: 700 .56rem/1 var(--mono); border: 1px solid var(--s-line); background: var(--s-raised); }
.lab-controls button { cursor: pointer; }
.lab-controls button + button { border-left: 0; }
.lab-controls button[aria-pressed="true"] { color: #fff9ef; background: var(--s-green); }
.lab-controls label { display: grid; gap: 6px; color: var(--s-muted); font: .52rem/1 var(--mono); }
.lab-controls > span { margin: 0 0 11px auto; color: var(--s-copper); font: .55rem/1 var(--mono); }
.chart-shell { min-width: 0; padding: 18px; border: 1px solid var(--s-line); background: var(--s-raised); }
.chart-shell header { display: flex; justify-content: space-between; gap: 12px; color: var(--s-muted); font: .55rem/1 var(--mono); }
.chart-shell svg { display: block; width: 100%; height: auto; margin-top: 10px; overflow: visible; }
.series { fill: none; stroke-width: 3; stroke-linejoin: round; stroke-linecap: round; }
.series.copper { stroke: var(--s-copper); }
.series.green { stroke: var(--s-green); }
.series.gray { stroke: #77746b; }
.series.dash { stroke-dasharray: 8 7; }
.grid-line { stroke: rgba(28,32,30,.1); stroke-width: 1; }
.threshold { stroke: var(--s-copper); stroke-width: 1.5; stroke-dasharray: 6 5; }
.spike-zone { fill: rgba(186,96,59,.11); }
.axis-label { fill: #8b867c; font: 11px var(--mono); }
.axis-label.y { text-anchor: end; }
.axis-label.x { text-anchor: middle; }
.chart-legend { display: flex; flex-wrap: wrap; gap: 18px; margin-top: 4px; color: var(--s-muted); font: .56rem/1 var(--mono); }
.chart-legend span { display: inline-flex; gap: 7px; align-items: center; }
.chart-legend i { width: 22px; height: 3px; }
.chart-legend i.copper { background: var(--s-copper); }
.chart-legend i.green { background: var(--s-green); }
.chart-legend i.gray { background: #77746b; }
.chart-legend i.dashed { height: 0; border-top: 3px dashed var(--s-copper); background: transparent; }
.chart-legend i.threshold-key { height: 0; border-top: 2px dashed var(--s-copper); }
.chart-legend i.zone-key { width: 18px; height: 12px; background: rgba(186,96,59,.16); }
.trajectory-story, .evidence-ladder { display: grid; grid-template-columns: 1fr 26px 1fr 26px 1fr 26px 1fr; gap: 7px; align-items: center; margin-top: 20px; }
.trajectory-story article, .evidence-ladder article { min-height: 150px; padding: 18px; border: 1px solid var(--s-line); background: var(--s-raised); }
.trajectory-story article.accent, .evidence-ladder article.accent { color: #f7f0e6; background: var(--s-green); }
.trajectory-story span, .evidence-ladder span, .position-layout aside > span, .backward-diagram span, .intervention-results span, .causal-warning span, .audit-grid span {
color: var(--s-copper); font: .56rem/1 var(--mono); letter-spacing: .06em;
}
.trajectory-story .accent span, .trajectory-story .accent p, .evidence-ladder .accent span, .evidence-ladder .accent p { color: rgba(247,240,230,.7); }
.trajectory-story b, .evidence-ladder b { display: block; margin-top: 25px; font-size: .78rem; }
.trajectory-story p, .evidence-ladder p { color: var(--s-muted); font-size: .61rem; line-height: 1.5; }
.trajectory-story > i, .evidence-ladder > i, .tensor-path > i { color: var(--s-copper); font-style: normal; text-align: center; }
.plain-rule { display: grid; grid-template-columns: 120px 1fr; margin-top: 20px; color: #f7f0e6; background: var(--s-green); }
.plain-rule > * { margin: 0; padding: 17px; }
.plain-rule b { color: #d58a68; font: .6rem/1.6 var(--mono); }
.plain-rule p { font-size: .67rem; line-height: 1.6; }
.tensor-path { display: grid; grid-template-columns: repeat(11, auto); align-items: stretch; margin-bottom: 22px; overflow-x: auto; }
.tensor-path button { min-width: 130px; min-height: 145px; padding: 16px; text-align: left; color: inherit; border: 1px solid var(--s-line); background: var(--s-raised); cursor: pointer; }
.tensor-path button[aria-pressed="true"] { color: #f7f0e6; background: var(--s-green); }
.tensor-path span, .tensor-path small { display: block; color: var(--s-muted); font: .5rem/1.3 var(--mono); }
.tensor-path b { display: block; margin: 24px 0 10px; font-size: .68rem; }
.tensor-path button[aria-pressed="true"] span, .tensor-path button[aria-pressed="true"] small { color: rgba(247,240,230,.68); }
.tensor-path > i { align-self: center; padding: 0 6px; }
.position-layout, .mixer-layout { display: grid; grid-template-columns: minmax(0, 1fr) 235px; border: 1px solid var(--s-line); background: var(--s-raised); }
.position-layout .chart-shell, .mixer-layout .chart-shell { border: 0; border-right: 1px solid var(--s-line); }
.position-layout aside, .mixer-layout aside { padding: 20px 18px; }
.position-layout aside > b { display: block; margin-top: 18px; font-size: .86rem; }
.position-layout dl, .mixer-layout dl { margin: 24px 0 0; }
.position-layout dl div, .mixer-layout dl div { display: flex; justify-content: space-between; gap: 12px; padding: 15px 0; border-bottom: 1px solid var(--s-line); }
.position-layout dt, .mixer-layout dt { color: var(--s-muted); font: .5rem/1.2 var(--mono); }
.position-layout dd, .mixer-layout dd { margin: 0; font: 700 .68rem/1 var(--mono); }
.position-layout aside p, .mixer-layout aside p { color: var(--s-muted); font-size: .61rem; line-height: 1.55; }
.pass-copy { color: var(--s-green) !important; font: 700 .55rem/1.5 var(--mono) !important; }
.boundary-pair { display: grid; grid-template-columns: repeat(2, 1fr); margin-top: 20px; }
.boundary-pair article { min-height: 170px; padding: 22px; }
.boundary-pair .yes { color: #f7f0e6; background: var(--s-green); }
.boundary-pair .no { background: #e7d8ca; }
.boundary-pair span { color: var(--s-copper); font: .56rem/1 var(--mono); }
.boundary-pair .yes span, .boundary-pair .yes p { color: rgba(247,240,230,.72); }
.boundary-pair b { display: block; margin-top: 25px; font-size: .83rem; }
.boundary-pair p { color: var(--s-muted); font-size: .63rem; line-height: 1.55; }
.reduction-table-wrap { margin-top: 20px; overflow-x: auto; }
.reduction-table { width: 100%; min-width: 790px; border-collapse: collapse; font-size: .62rem; }
.reduction-table th, .reduction-table td { padding: 13px 11px; border-bottom: 1px solid var(--s-line); text-align: left; }
.reduction-table thead th { color: var(--s-muted); font: .52rem/1.3 var(--mono); }
.reduction-table tbody th, .reduction-table .good { font: 700 .61rem/1 var(--mono); }
.reduction-table .good { color: var(--s-green); }
.backward-diagram { border: 1px solid var(--s-line); background: var(--s-raised); }
.backward-diagram .forward { display: flex; flex-wrap: wrap; gap: 14px; align-items: center; padding: 18px; color: #f7f0e6; background: var(--s-green); }
.backward-diagram .forward span { margin-right: auto; color: #d58a68; }
.backward-diagram .forward b { font: 700 .65rem/1 var(--mono); }
.backward-diagram .forward i { color: #d58a68; font-style: normal; }
.backward-diagram .paths { display: grid; grid-template-columns: repeat(3, 1fr); }
.backward-diagram article { min-height: 180px; padding: 20px; border-right: 1px solid var(--s-line); }
.backward-diagram article:last-child { border-right: 0; }
.backward-diagram article.accent { background: #e7d8ca; }
.backward-diagram article b { display: block; margin-top: 30px; font-size: .85rem; }
.backward-diagram article p { color: var(--s-muted); font-size: .62rem; line-height: 1.55; }
.intervention-results { display: grid; grid-template-columns: repeat(3, 1fr); margin-top: 20px; }
.intervention-results article { min-height: 185px; padding: 20px; border: 1px solid var(--s-line); background: var(--s-raised); }
.intervention-results .counter { color: #f7f0e6; background: var(--s-copper); }
.intervention-results .pass { color: #f7f0e6; background: var(--s-green); }
.intervention-results .counter span, .intervention-results .counter p, .intervention-results .pass span, .intervention-results .pass p { color: rgba(247,240,230,.72); }
.intervention-results b { display: block; margin-top: 28px; font: 700 1.1rem/1 var(--mono); }
.intervention-results p { color: var(--s-muted); font-size: .62rem; line-height: 1.55; }
.causal-warning { display: grid; grid-template-columns: 145px 1fr; margin-top: 20px; padding: 20px; border-left: 5px solid var(--s-copper); background: #e7d8ca; }
.causal-warning b { font-size: .78rem; }
.causal-warning p { margin: 0; font-size: .67rem; line-height: 1.6; }
.causal-warning span { grid-column: 2; margin-top: 13px; line-height: 1.5; }
.fit { stroke: var(--s-copper); stroke-width: 2; stroke-dasharray: 8 6; }
.audit-grid { display: grid; grid-template-columns: repeat(4, 1fr); margin-top: 20px; }
.audit-grid article { min-height: 175px; padding: 18px; border: 1px solid var(--s-line); background: var(--s-raised); }
.audit-grid .boundary { color: #f7f0e6; background: var(--s-green); }
.audit-grid .boundary span, .audit-grid .boundary p { color: rgba(247,240,230,.7); }
.audit-grid b { display: block; margin-top: 25px; font-size: .75rem; }
.audit-grid p { color: var(--s-muted); font-size: .59rem; line-height: 1.55; overflow-wrap: anywhere; }
.audit-grid code { font-size: .52rem; }
.hash-strip { display: flex; flex-wrap: wrap; gap: 12px 28px; margin-top: 20px; padding: 16px; background: #e9e4da; }
.hash-strip span { color: var(--s-muted); font: .5rem/1.5 var(--mono); }
.hash-strip code { color: var(--s-green); }
@media (max-width: 920px) {
.spike-ledger { grid-template-columns: repeat(3, 1fr); }
.spike-ledger article:nth-child(3) { border-right: 0; }
.spike-tabs { grid-template-columns: repeat(3, 1fr); }
.trajectory-story, .evidence-ladder { grid-template-columns: 1fr 24px 1fr; }
.trajectory-story > i:nth-of-type(n+3), .evidence-ladder > i:nth-of-type(n+3) { display: none; }
.audit-grid { grid-template-columns: repeat(2, 1fr); }
}
@media (max-width: 680px) {
.spike-head, .panel-lead { grid-template-columns: 1fr; gap: 20px; }
.spike-head, .spike-panel { padding: 20px; }
.spike-ledger { grid-template-columns: repeat(2, 1fr); }
.spike-ledger article:nth-child(3) { border-right: 1px solid var(--s-line); }
.spike-ledger article:nth-child(even) { border-right: 0; }
.spike-tabs { display: flex; overflow-x: auto; }
.spike-tabs button { flex: 0 0 190px; }
.lab-controls { align-items: stretch; }
.lab-controls > div, .lab-controls label { flex: 1 0 100%; }
.lab-controls button { flex: 1; }
.lab-controls select { width: 100%; }
.lab-controls > span { margin: 5px 0 0; line-height: 1.4; }
.chart-shell { padding: 12px 8px; }
.chart-shell header { padding: 0 8px; }
.axis-label { font-size: 9px; }
.trajectory-story, .evidence-ladder { grid-template-columns: 1fr; }
.trajectory-story > i, .evidence-ladder > i { display: block !important; transform: rotate(90deg); }
.plain-rule, .causal-warning { grid-template-columns: 1fr; }
.plain-rule p, .causal-warning span { grid-column: 1; }
.position-layout, .mixer-layout { grid-template-columns: 1fr; }
.position-layout .chart-shell, .mixer-layout .chart-shell { border-right: 0; border-bottom: 1px solid var(--s-line); }
.boundary-pair, .backward-diagram .paths, .intervention-results, .audit-grid { grid-template-columns: 1fr; }
.backward-diagram article { border-right: 0; border-bottom: 1px solid var(--s-line); }
.tensor-path { padding-bottom: 8px; }
.chart-shell svg { min-width: 620px; }
.chart-shell { overflow-x: auto; }
}
</style>
src/pages/k3/index.astro
+33 -6
View File
@@ -3,6 +3,7 @@ import BaseLayout from "@/layouts/BaseLayout.astro";
import ArchitectureExplorer from "@/components/ArchitectureExplorer.astro";
import K3ArtifactLab from "@/components/K3ArtifactLab.astro";
import K3AttnResGradientLab from "@/components/K3AttnResGradientLab.astro";
import K3AttnResSpikeLab from "@/components/K3AttnResSpikeLab.astro";
import K3AttnResTraceLab from "@/components/K3AttnResTraceLab.astro";
import K3ReportLab from "@/components/K3ReportLab.astro";
import { k3FigureAtlas, k3Ledgers, k3PaperChain, k3ReportMap } from "@/data/k3";
@@ -40,7 +41,8 @@ const toc = [
["29", "artifacts", "开放权重工件审计"],
["30", "attnres-reduced", "AttnRes 缩小机制实验"],
["31", "attnres-gradient", "梯度定义与深度扩展"],
["32", "audit", "21 张图表审计"],
["32", "attnres-spike", "尖峰轨迹与反向路径"],
["33", "audit", "21 张图表审计"],
["↳", "papers", "100 节点阅读链"],
];
@@ -109,13 +111,13 @@ const paperGroups = [
<BaseLayout
title="Kimi K3 技术报告完整深读：架构、训练、RL、系统与评测"
description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图、两轮十个 AttnRes 独立实验视图与一百个一手阅读节点，逐节读懂 Kimi K3。"
description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图、三轮十五个 AttnRes 独立实验视图与一百个一手阅读节点，逐节读懂 Kimi K3。"
section="k3"
>
<header class="page-hero k3-hero">
<div class="page-hero-inner">
<div>
<p class="eyebrow"><span>ANCHOR REPORT / ROUND 05</span> KIMI K3 · REPORT → ARTIFACTS → INDEPENDENT PROBE</p>
<p class="eyebrow"><span>ANCHOR REPORT / ROUND 06</span> KIMI K3 · REPORT → ARTIFACTS → INDEPENDENT PROBE</p>
<h1>不把报告压成摘要<br />把每个因果环节<br />重新展开</h1>
<p class="lead">
K3 同时扩展序列、深度、宽度、视觉与 Agent 轨迹。真正值得读的不是 2.8T 这个最大数字，
@@ -125,11 +127,11 @@ const paperGroups = [
<dl class="page-facts">
<div><dt>QUESTIONS</dt><dd>32 张问题账</dd></div>
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div>
<div><dt>LABS</dt><dd>8 + 4 + 5 + 5 个交互视图</dd></div>
<div><dt>LABS</dt><dd>8 + 4 + 5 + 5 + 5 个交互视图</dd></div>
<div><dt>READING</dt><dd>100 个一手 / 官方节点</dd></div>
<div><dt>MODEL</dt><dd>2.78T total / 104.2B active</dd></div>
<div><dt>ARTIFACTS</dt><dd>96 shards · 497,220 tensors</dd></div>
<div><dt>STATUS</dt><dd>K3 五轮 · 梯度定义闭环</dd></div>
<div><dt>STATUS</dt><dd>K3 六轮 · 尖峰路径审计</dd></div>
</dl>
</div>
</header>
@@ -922,8 +924,33 @@ const paperGroups = [
</div>
</section>
<section class="article-section" id="attnres-spike">
<p class="eyebrow"><span>32</span> SPIKE TRAJECTORY × BACKWARD PATH</p>
<h2>layer 21–25 的尖峰从什么时候出现，又对 mixer 的哪条反向路径敏感？</h2>
<p class="lede">
第六轮不把上一轮的局部尖峰直接解释成机制。本站预注册六个训练时点、六个张量位置、
四种 confirmatory reduction 与三种 same-forward backward rule；正式训练严格复用
Round 05 depth-32 Block 的模型、数据、optimizer 与 schedule，诊断全部位于 optimizer 之外。
结果显示尖峰在 step 500 后才形成、六个位置均可见；切断 softmax / query / key 源梯度没有降低尖峰，
而把全局 learned value-backward coefficients 改成均匀后，contrast 平均下降 70.2%。
</p>
<div class="artifact-callout">
<article><span>F / FROZEN</span><b>3 formal + 1 replay</b><p>196,608,000 formal target bytes；完整重放再加 65,536,000。</p></article>
<article><span>X / DEVELOPMENT</span><b>step 500 → 2,000</b><p>三 seed 的尖峰 contrast 从 0.524× 跃到平均 2.233×。</p></article>
<article><span>X / ROBUSTNESS</span><b>12 / 12 PASS</b><p>四种预注册 reduction 都保留 layer 21–25 集中。</p></article>
<article class="warning"><span>B / BOUNDARY</span><b>global sensitivity</b><p>不是训练变体、因果贡献百分比或局部 mixer 归因。</p></article>
</div>
<K3AttnResSpikeLab />
<div class="hero-actions">
<a class="button primary" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_SPIKE_AUDIT.md">阅读完整结果审计</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_SPIKE_PROTOCOL.md">核对预注册协议</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_SPIKE_SCOPING.md">查看范围冻结</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/attnres_spike">复跑实验与分析</a>
</div>
</section>
<section class="article-section" id="audit">
<p class="eyebrow"><span>32</span> FIGURE & TABLE AUDIT</p>
<p class="eyebrow"><span>33</span> FIGURE & TABLE AUDIT</p>
<h2>Figure 1–16、Table 1–5：每张图究竟支持什么，不能支持什么</h2>
<div class="figure-atlas">
{k3FigureAtlas.map(([id, report, title, contract]) => (
src/pages/progress/index.astro
+9 -4
View File
@@ -9,7 +9,7 @@ const researching = chapters.filter((chapter) => ["researching", "drafting"].inc
const workstreams = [
{ label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" },
{ label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" },
{ label: "Kimi K3 深读", value: 98, next: "对齐 layer 21–25 梯度尖峰与 mixer weights；等待 A_log 官方转换合同" },
{ label: "Kimi K3 深读", value: 99, next: "对 group 6 / 7 做局部 mixer backward 干预；等待 A_log 官方转换合同" },
{ label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau，并加入真实小语料复现" },
{ label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" },
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
@@ -50,7 +50,7 @@ const workstreams = [
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-30 10:05 CST</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-30 12:20 CST</dd></div>
<div><dt>MODE</dt><dd>持续迭代，不锁死版本</dd></div>
</dl>
</div>
@@ -97,7 +97,7 @@ const workstreams = [
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全，包含先修依赖和三条贯穿案例。</p></article>
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
<article><span>✓</span><h3>九十九个原创交互视图</h3><p>K3 三轴图、八联报告实验、四联开放工件实验与两轮十联 AttnRes 独立实验，DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>一百零四个原创交互视图</h3><p>K3 三轴图、八联报告实验、四联开放工件实验与三轮十五联 AttnRes 独立实验，DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
@@ -107,6 +107,7 @@ const workstreams = [
<article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions，审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes；四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
<article><span>✓</span><h3>Kimi K3 四轮 AttnRes 独立实验</h3><p>冻结三结构 × 三 seed 的 9 个 2,000-step 格；Full / Block 相对 Baseline 的平均 paired delta 为 −0.01457 / −0.04247 BPC，但核心参数梯度 CV 没有复现论文叙述。指定正式格全新进程八字段 exact，五视图同时展示结果、反证、成本与 claim boundary。</p></article>
<article><span>✓</span><h3>Kimi K3 五轮梯度定义与深度扩展</h3><p>先确认 Figure 5 没有公开唯一 gradient telemetry 合同，再冻结 16/32 blocks × Baseline/Block × 3 seeds 的 12 个 8,000-step 格。Block 的首尾失衡 6/6 改善但全层 CV 6/6 恶化，两个深度都判为 mixed；指定 32 层格完整重训的模型、优化器与全部冻结字段 exact。</p></article>
<article><span>✓</span><h3>Kimi K3 六轮尖峰轨迹与反向路径</h3><p>严格复用 Round 05 depth-32 Block 的三个正式格：尖峰在 step 500 后形成，六个位置 3/3 seed 可见，四种 reduction 12/12 格稳健。切断 key/softmax 源梯度没有降低尖峰；uniform value-backward 让 contrast 平均下降 70.2%，只判为全局 backward-rule sensitivity。完整 replay 的 16 组冻结字段 exact。</p></article>
<article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel；6/6 官方参考逐元素相等，并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
<article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article>
@@ -135,7 +136,7 @@ const workstreams = [
</div>
<div class="queue-table">
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
<div><span>P0</span><strong>K3 五轮后续</strong><p>对齐 layer 21–25 尖峰、pre-attention / pre-MLP 与 mixer source weights → 等待 A_log 官方合同后进入真实 checkpoint forward</p><em>局部机制 + 工件边界</em></div>
<div><span>P0</span><strong>K3 六轮后续</strong><p>group 6 / 7 局部 mixer intervention matrix → 前向训练变体 → 等待 A_log 官方合同后进入真实 checkpoint forward</p><em>局部机制 + 工件边界</em></div>
<div><span>P0</span><strong>DeepSeek 八轮后续</strong><p>干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
@@ -225,6 +226,10 @@ const workstreams = [
<div><time>2026-07-30</time><b>首尾平衡与全层 CV 永久分账</b><p>Block 在 6/6 配对中改善 first/last，却因中后段局部尖峰让 CV 在 6/6 配对中恶化；联合判定保持 mixed。</p></div>
<div><time>2026-07-30</time><b>绝对梯度尺度必须与归一化谱同屏</b><p>Block mean gradient 约为 Baseline 的 54%–57%；更接近 1 的首尾比不能偷换成各层信号更强。</p></div>
<div><time>2026-07-30</time><b>32 层完整重放扩到状态哈希</b><p>8,000-step fresh replay 的全部冻结字段以及 model / optimizer state hashes exact；额外 replay bytes 单列，不混入 formal 预算。</p></div>
<div><time>2026-07-30</time><b>尖峰是定向复查，不是盲发现</b><p>layer 21–25 来自 Round 05；Round 06 先固定目标集合，再检查训练时点、张量位置、reduction 与反向路径。</p></div>
<div><time>2026-07-30</time><b>最早可见不等于物理起源</b><p>pre-attention input 是六个采样点中最早可见位置；更早 mixer 与跨层回传已经作用，不能写成尖峰从这里注入。</p></div>
<div><time>2026-07-30</time><b>同一前向只识别反向规则敏感性</b><p>三模式的 logits、loss、activations 与 mixer summaries exact；uniform value-backward 的 70.2% contrast 降幅不是训练变体或因果贡献百分比。</p></div>
<div><time>2026-07-30</time><b>相关性、全局干预与局部归因分三层</b><p>MLP latest weight 的局部 r≈.69 只提供候选；全局 value-route 干预支持路径敏感性，下一轮才做 group 6 / 7 局部归因矩阵。</p></div>
<div><time>2026-07-29</time><b>32-token 对照改为同源 16→24</b><p>TNEWS 只有 105/10,000 条达到 32 tokens，强行统一会落入约 1% 极端长尾；24-token eligibility 仍保留 1,609 条中文候选。</p></div>
<div><time>2026-07-29</time><b>长度敏感性必须成对重采样</b><p>16-token 输入严格是 24-token 输入前缀，2,000 次 bootstrap 共用 prompt indices；结果只描述固定 cohort 的长度敏感性。</p></div>
<div><time>2026-07-29</time><b>三类 cohort 永久分身份</b><p>自然长度回答本批样本如何路由；matched-16 / 24 回答同一 prompt 多看 8 tokens 后如何变化，不把二者混成内容因果。</p></div>