feat: add reduced AttnRes trace lab

This commit is contained in:
wuyang
2026-07-30 07:30:01 +08:00
parent d1d9d22bf3
commit 4ce780dcc9
11 changed files with 1070 additions and 26 deletions
+717
View File
@@ -0,0 +1,717 @@
---
import rawLab from "@/data/k3-attnres-reduced-compact.json";
const lab = rawLab as any;
const json = JSON.stringify(lab).replaceAll("<", "\\u003c");
const architectures = ["baseline", "full", "block"];
const labels: Record<string, string> = {
baseline: "Baseline",
full: "Full AttnRes",
block: "Block AttnRes",
};
const colors: Record<string, string> = {
baseline: "#79746b",
full: "#ba603b",
block: "#163f3b",
};
const final = lab.final_validation;
const reproduction = lab.reproduction;
const formatSigned = (value: number, digits = 5) =>
`${value < 0 ? "−" : value > 0 ? "+" : ""}${Math.abs(value).toFixed(digits)}`;
const gib = (value: number) => value / 2 ** 30;
const heatOpacity = (weight: number, sources: number) =>
Math.min(1, 0.12 + Math.min(2.2, weight * sources) / 2.2 * 0.88);
const gradientMax = Math.max(
...architectures.flatMap((architecture) => lab.gradients[architecture].by_block.mean),
);
---
<figure class="attnres-trace" data-attnres-lab>
<header class="trace-head">
<div>
<p>ROUND 04 / REDUCED INDEPENDENT MECHANISM PROBE</p>
<h3>不裁剪 K3 的冲突权重:从零训练一个可以完整审计的深度路由实验</h3>
</div>
<p>
WikiText-2 byte LM · 16 blocks / 32 residual sublayers · 3 structures × 3 seeds。
这是缩小机制探针,不是 K3 checkpoint forward,也不是论文规模复现。
</p>
</header>
<div class="trace-ledger">
<article><span>FORMAL GRID</span><b>3 × 3</b><p>9 个 2,000-step runs</p></article>
<article><span>TARGET BYTES</span><b>147.456M</b><p>每格 16,384,000</p></article>
<article><span>SHARED CORE</span><b>9.542M</b><p>同 seed 公共初始化 exact</p></article>
<article class="pass"><span>FULL Δ BPC</span><b>{formatSigned(final.full_contrast.mean_delta_bpc)}</b><p>3 / 3 paired negative</p></article>
<article class="pass"><span>BLOCK Δ BPC</span><b>{formatSigned(final.block_contrast.mean_delta_bpc)}</b><p>3 / 3 paired negative</p></article>
<article class="pass"><span>FRESH REPLAY</span><b>8 / 8 exact</b><p>timing 明确不要求 exact</p></article>
</div>
<div class="trace-tabs" role="tablist" aria-label="选择 Attention Residuals 缩小实验视图">
<button type="button" role="tab" data-attnres-tab="outcome" aria-selected="true">
<span>01</span><b>训练曲线与配对结果</b><small>three seeds · final BPC</small>
</button>
<button type="button" role="tab" data-attnres-tab="rms" aria-selected="false" tabindex="-1">
<span>02</span><b>残差流怎样改变</b><small>RMS · block sawtooth</small>
</button>
<button type="button" role="tab" data-attnres-tab="mixer" aria-selected="false" tabindex="-1">
<span>03</span><b>深度权重读了谁</b><small>mixer heatmap · output</small>
</button>
<button type="button" role="tab" data-attnres-tab="gradient" aria-selected="false" tabindex="-1">
<span>04</span><b>没有复现的梯度故事</b><small>counterevidence · metric boundary</small>
</button>
<button type="button" role="tab" data-attnres-tab="audit" aria-selected="false" tabindex="-1">
<span>05</span><b>成本、重放与边界</b><small>compute · hashes · claims</small>
</button>
</div>
<section class="trace-panel" data-attnres-panel="outcome">
<div class="panel-lead">
<div><span>I / PREREGISTERED PRIMARY ENDPOINT</span><h4>先看三条完整曲线,再放大最后一个配对点</h4></div>
<p>
BPC 越低越好。主判据只读 step 2000:三个 seed 必须同方向,且 mean paired
delta 至少达到 −0.010;中途曲线不用于改终点。
</p>
</div>
<div class="curve-controls" role="group" aria-label="选择训练曲线 seed">
<button type="button" data-curve-seed="mean" aria-pressed="true">3-SEED MEAN</button>
{lab.grid.seeds.map((seed: number, index: number) => (
<button type="button" data-curve-seed={String(index)} aria-pressed="false">SEED {String(seed).slice(-2)}</button>
))}
</div>
<div class="curve-layout">
<div class="curve-chart">
<header><span>VALIDATION BPC</span><b>0 → 2,000 TRAINING STEPS</b></header>
<svg viewBox="0 0 840 330" role="img" aria-label="三种结构的验证 BPC 训练曲线">
<g class="chart-grid">
{[2, 3, 4, 6, 8].map((tick) => {
const y = 20 + (8.2 - tick) / 6.4 * 260;
return <g><line x1="62" x2="818" y1={y} y2={y}></line><text x="50" y={y + 4}>{tick}</text></g>;
})}
{[0, 500, 1000, 1500, 2000].map((tick) => {
const x = 62 + tick / 2000 * 756;
return <g><line x1={x} x2={x} y1="20" y2="280"></line><text x={x} y="308">{tick}</text></g>;
})}
</g>
{architectures.map((architecture) => (
<g data-curve-series={architecture}>
<polyline
data-curve-line
fill="none"
stroke={colors[architecture]}
stroke-width={architecture === "block" ? "4" : "3"}
stroke-linecap="round"
stroke-linejoin="round"
></polyline>
<g data-curve-points></g>
</g>
))}
</svg>
<div class="chart-legend">
{architectures.map((architecture) => <span style={`--legend:${colors[architecture]}`}><i></i>{labels[architecture]}</span>)}
</div>
</div>
<div class="curve-readout">
<span>STEP 2000 / SELECTED VIEW</span>
{architectures.map((architecture) => (
<article>
<i style={`--series:${colors[architecture]}`}></i>
<b>{labels[architecture]}</b>
<strong data-curve-final={architecture}>{final.means[architecture].toFixed(5)}</strong>
</article>
))}
<p data-curve-copy>三 seed 均值;正式判据使用逐 seed paired delta,不把三次运行当成 benchmark 样本总体。</p>
</div>
</div>
<div class="seed-pairs">
{final.by_seed.map((row: any) => (
<article>
<header><span>SEED {String(row.seed).slice(-2)}</span><b>BASE {row.final_bpc.baseline.toFixed(5)}</b></header>
<div><span>FULL − BASE</span><strong>{formatSigned(row.full_minus_baseline, 5)}</strong></div>
<div><span>BLOCK − BASE</span><strong>{formatSigned(row.block_minus_baseline, 5)}</strong></div>
</article>
))}
<article class="verdict">
<header><span>FROZEN RULE</span><b>3 / 3 + |MEAN| ≥ .010</b></header>
<strong>DIRECTIONAL SUPPORT</strong>
<p>只在这个 reduced protocol 内;不是总体显著性,也不外推 paper scale。</p>
</article>
</div>
</section>
<section class="trace-panel" data-attnres-panel="rms" hidden>
<div class="panel-lead">
<div><span>II / FIXED DIAGNOSTIC WINDOWS</span><h4>普通 residual 一路累加;Block 每四层重新混合</h4></div>
<p>
下面是三个 seed 在固定 16 个诊断窗口上的均值。RMS 只描述幅值,不等于信息量、有效秩或因果重要性。
</p>
</div>
<div class="rms-controls" role="group" aria-label="选择 residual RMS 指标">
<button type="button" data-rms-metric="stream_state_rms" aria-pressed="true">STREAM / PARTIAL STATE</button>
<button type="button" data-rms-metric="layer_input_rms" aria-pressed="false">LAYER INPUT</button>
<button type="button" data-rms-metric="branch_output_rms" aria-pressed="false">BRANCH OUTPUT</button>
</div>
<div class="rms-chart">
<header><span data-rms-title>STREAM / PARTIAL STATE RMS</span><b>32 RESIDUAL SUBLAYERS</b></header>
<svg viewBox="0 0 840 330" role="img" aria-label="三种结构沿 32 个残差子层的 RMS 曲线">
<g class="chart-grid">
{[0, 8, 16, 24, 32].map((tick) => {
const x = 62 + tick / 32 * 756;
return <g><line x1={x} x2={x} y1="20" y2="280"></line><text x={x} y="308">{tick}</text></g>;
})}
{[0, 0.25, 0.5, 0.75, 1].map((part) => {
const y = 20 + (1 - part) * 260;
return <g><line x1="62" x2="818" y1={y} y2={y}></line><text data-rms-tick={String(part)} x="50" y={y + 4}>—</text></g>;
})}
</g>
{architectures.map((architecture) => (
<polyline
data-rms-series={architecture}
fill="none"
stroke={colors[architecture]}
stroke-width={architecture === "block" ? "4" : "3"}
stroke-linecap="round"
stroke-linejoin="round"
></polyline>
))}
</svg>
<div class="chart-legend">
{architectures.map((architecture) => <span style={`--legend:${colors[architecture]}`}><i></i>{labels[architecture]}</span>)}
</div>
</div>
<div class="block-rhythm">
<header><span>BLOCK / STREAM STATE</span><b>每四格是一个局部 residual block</b></header>
<div>
{lab.traces.block.stream_state_rms.mean.map((value: number, index: number) => (
<i
class:list={{ boundary: (index + 1) % 4 === 0 }}
style={`--height:${Math.max(8, value / 0.12 * 100)}%`}
title={`sublayer ${index + 1}: ${value.toFixed(5)}`}
><span>{index + 1}</span></i>
))}
</div>
<p>块内通常向上累积;下一块重新读取历史 block sources 后,partial state 出现重置。锯齿是拓扑痕迹,不是自动等价于“更稳定”。</p>
</div>
</section>
<section class="trace-panel" data-attnres-panel="mixer" hidden>
<div class="panel-lead">
<div><span>III / LEARNED DEPTH ROUTING</span><h4>横轴是历史 source,纵轴是当前 residual sublayer</h4></div>
<p>
每格颜色按“实际权重 ÷ 均匀权重”归一:深色代表比该行均匀读取更强。灰格表示那一层尚看不到该 source。
</p>
</div>
<div class="mixer-controls" role="group" aria-label="选择 depth mixer 结构">
<button type="button" data-mixer-arch="full" aria-pressed="true">FULL · 32 SUBLAYER SOURCES</button>
<button type="button" data-mixer-arch="block" aria-pressed="false">BLOCK · 8 BLOCK SOURCES</button>
</div>
{(["full", "block"] as const).map((architecture, architectureIndex) => (
<div class="mixer-view" data-mixer-view={architecture} hidden={architectureIndex !== 0}>
<div class="heatmap-scroll">
<div
class="depth-heatmap"
style={`--columns:${lab.mixers[architecture].max_sources}`}
aria-label={`${labels[architecture]} depth weight heatmap`}
>
{lab.mixers[architecture].rows.map((row: any) => (
<div class="heat-row">
<span>{String(row.sublayer).padStart(2, "0")}</span>
<div style={`--columns:${lab.mixers[architecture].max_sources}`}>
{Array.from({ length: lab.mixers[architecture].max_sources }, (_, source) =>
source < row.sources
? <i
style={`--heat:${heatOpacity(row.mean_weights[source], row.sources)}`}
title={`L${row.sublayer} ← source ${source}: ${row.mean_weights[source].toFixed(6)}`}
></i>
: <i class="empty"></i>
)}
</div>
</div>
))}
</div>
</div>
<div class="output-weights">
<header><span>FINAL OUTPUT MIXER</span><b>{lab.mixers[architecture].output.sources} SOURCES · 3-SEED MEAN</b></header>
<div>
{lab.mixers[architecture].output.mean_weights.map((weight: number, source: number) => (
<i
class:list={{ spikeSource: architecture === "full" && source === 31 }}
style={`--weight:${weight / Math.max(...lab.mixers[architecture].output.mean_weights) * 100}%`}
title={`source ${source}: ${weight.toFixed(6)}`}
><span>{source}</span></i>
))}
</div>
</div>
</div>
))}
<aside class="posthoc-callout">
<span>POST-HOC DESCRIPTIVE CALLOUT</span>
<div><b>SUBLAYER 31 BRANCH RMS</b><strong>{lab.posthoc.largest_full_branch_rms.toFixed(4)}</strong></div>
<i>→</i>
<div><b>FINAL MIXER WEIGHT</b><strong>{lab.posthoc.corresponding_final_output_weight.toFixed(6)}</strong></div>
<i>→</i>
<div><b>RELATIVE TO UNIFORM</b><strong>{lab.posthoc.weight_over_uniform.toFixed(3)}×</strong></div>
<p>高幅值 source 同时被最终 mixer 强烈降权,与“学习抑制异常 source”相容;这是事后观察,不是预注册证据。</p>
</aside>
</section>
<section class="trace-panel" data-attnres-panel="gradient" hidden>
<div class="panel-lead">
<div><span>IV / PREREGISTERED COUNTEREVIDENCE</span><h4>BPC 支持,不代表每一种机制解释都同时得到支持</h4></div>
<p>
指标是每个 Transformer block 的公共核心参数 gradient RMS,再看 16 个 block 的变异系数。
CV 越低,按这一定义才越均匀。
</p>
</div>
<div class="gradient-cv">
{architectures.map((architecture) => (
<article class:list={{ warning: architecture !== "baseline" }}>
<span>{labels[architecture]}</span>
<b>{lab.gradients[architecture].mean_cv.toFixed(4)}</b>
<p>seed CV · {lab.gradients[architecture].cv_by_seed.map((value: number) => value.toFixed(3)).join(" / ")}</p>
</article>
))}
<article class="verdict">
<span>OBSERVED ORDER</span>
<b>BASE &lt; FULL &lt; BLOCK</b>
<p>本指标下 AttnRes 更不均匀,不能写成论文梯度结果复现。</p>
</article>
</div>
<div class="gradient-depth">
<header><span>CORE-PARAMETER GRADIENT RMS</span><b>3-SEED MEAN BY TRANSFORMER BLOCK</b></header>
<div class="gradient-grid">
{Array.from({ length: 16 }, (_, block) => (
<article>
<span>B{String(block + 1).padStart(2, "0")}</span>
<div>
{architectures.map((architecture) => {
const value = lab.gradients[architecture].by_block.mean[block];
return <i
style={`--bar:${value / gradientMax * 100}%;--series:${colors[architecture]}`}
title={`${labels[architecture]}: ${value.toExponential(4)}`}
></i>;
})}
</div>
</article>
))}
</div>
</div>
<div class="metric-boundary">
<article><span>THIS PROBE</span><b>core parameter gradients</b><p>16 blocks · fixed diagnostic batch · reduced model</p></article>
<i>≠</i>
<article><span>PAPER NARRATIVE</span><b>large-model depth gradients</b><p>尺度、训练阶段与聚合对象都可能不同</p></article>
<p>下一步应先对齐论文实际 activation / residual-output gradient 定义,再增加 depth 与训练预算;不能先换指标再只展示好看的图。</p>
</div>
</section>
<section class="trace-panel" data-attnres-panel="audit" hidden>
<div class="panel-lead">
<div><span>V / COST + REPRODUCTION + CLAIM BOUNDARY</span><h4>近似同参数,不是同计算;数值 exact,不要求计时 exact</h4></div>
<p>
Full / Block 只增加 12,672 个 mixer 参数,但教学实现必须保存、归一化并混合历史 states,
所以参数开销小不等于执行开销小。
</p>
</div>
<div class="cost-grid">
{architectures.map((architecture) => (
<article>
<header><span>{labels[architecture]}</span><b>{lab.parameters[architecture].total.toLocaleString("en-US")} PARAMS</b></header>
<div><span>MEAN STEP</span><strong>{lab.timing[architecture].mean_step_ms.toFixed(2)} ms</strong></div>
<div><span>PEAK ALLOCATED</span><strong>{gib(lab.timing[architecture].mean_peak_allocated_bytes).toFixed(2)} GiB</strong></div>
<p>{architecture === "baseline"
? "1.00× time · 1.00× memory"
: `${lab.timing.relative_to_baseline[architecture].step_time_ratio.toFixed(2)}× time · ${lab.timing.relative_to_baseline[architecture].allocated_memory_ratio.toFixed(2)}× memory`}</p>
</article>
))}
</div>
<div class="replay-chain">
<article><span>FREEZE</span><b>protocol + manifest</b><p>{lab.dataset.schedule_sha256.slice(0, 12)}… schedule</p></article>
<i>→</i>
<article><span>SMOKE</span><b>3 × two processes</b><p>eight frozen fields exact</p></article>
<i>→</i>
<article><span>FORMAL</span><b>9 / 9 complete</b><p>budget · init · finite checks</p></article>
<i>→</i>
<article class="pass"><span>REPLAY</span><b>Block / seed 01</b><p>2,000 steps · 8 / 8 exact</p></article>
</div>
<div class="hash-ledger">
<article><span>MANIFEST</span><code>{reproduction.manifest_sha256}</code></article>
<article><span>FORMAL REPLAY SOURCE</span><code>{reproduction.formal_replay.formal_file_sha256}</code></article>
<article><span>FRESH REPLAY</span><code>{reproduction.formal_replay.replay_file_sha256}</code></article>
<article><span>COMPACT PAYLOAD</span><code>{lab.canonical_sha256_without_self}</code></article>
</div>
<div class="claim-grid">
<article class="yes">
<span>可以说</span>
<ul>
<li>冻结的缩小协议中,两种 AttnRes 的 BPC 配对方向都为负。</li>
<li>Block partial-state RMS 每四层出现与拓扑一致的重置。</li>
<li>指定 2,000-step run 在全新进程中数值与哈希字段 exact。</li>
</ul>
</article>
<article class="no">
<span>不可以说</span>
<ul>
<li>K3 checkpoint 已 forward,或论文规模收益已经复现。</li>
<li>这是同 FLOPs / 同 wall-time 优势,或 Block 普遍优于 Full。</li>
<li>AttnRes 梯度更均匀;本轮预注册指标恰好给出相反结果。</li>
</ul>
</article>
</div>
</section>
<script is:inline type="application/json" data-attnres-data set:html={json}></script>
</figure>
<script>
type AttnResArchitecture = "baseline" | "full" | "block";
type AttnResRmsMetric = "stream_state_rms" | "layer_input_rms" | "branch_output_rms";
const initializeAttnRes = (root: HTMLElement) => {
if (root.dataset.ready === "true") return;
root.dataset.ready = "true";
const dataNode = root.querySelector("[data-attnres-data]");
const data: any = JSON.parse(dataNode?.textContent || "{}");
const architectures: AttnResArchitecture[] = ["baseline", "full", "block"];
const labels: Record<AttnResArchitecture, string> = { baseline: "Baseline", full: "Full AttnRes", block: "Block AttnRes" };
const colors: Record<AttnResArchitecture, string> = { baseline: "#79746b", full: "#ba603b", block: "#163f3b" };
const tabs = [...root.querySelectorAll<HTMLButtonElement>("[data-attnres-tab]")];
const panels = [...root.querySelectorAll<HTMLElement>("[data-attnres-panel]")];
const selectTab = (id: string | undefined) => {
tabs.forEach((tab) => {
const selected = tab.dataset.attnresTab === id;
tab.setAttribute("aria-selected", String(selected));
tab.tabIndex = selected ? 0 : -1;
});
panels.forEach((panel) => { panel.hidden = panel.dataset.attnresPanel !== id; });
};
tabs.forEach((tab, index) => {
tab.addEventListener("click", () => selectTab(tab.dataset.attnresTab));
tab.addEventListener("keydown", (event: KeyboardEvent) => {
if (!["ArrowRight", "ArrowLeft", "Home", "End"].includes(event.key)) return;
event.preventDefault();
let next = index;
if (event.key === "ArrowRight") next = (index + 1) % tabs.length;
if (event.key === "ArrowLeft") next = (index - 1 + tabs.length) % tabs.length;
if (event.key === "Home") next = 0;
if (event.key === "End") next = tabs.length - 1;
selectTab(tabs[next].dataset.attnresTab);
tabs[next].focus();
});
});
const curveButtons = [...root.querySelectorAll<HTMLButtonElement>("[data-curve-seed]")];
const curvePoint = (step: number, value: number) => {
const x = 62 + step / 2000 * 756;
const y = 20 + (8.2 - value) / 6.4 * 260;
return [x, y];
};
const updateCurve = (seedKey: string | undefined) => {
const seedIndex = seedKey === "mean" ? -1 : Number(seedKey);
architectures.forEach((architecture) => {
const values: { step: number; value: number }[] = data.evaluation_curves[architecture].map((row: any) => ({
step: row.step,
value: seedIndex < 0 ? row.mean_bpc : row.by_seed[seedIndex],
}));
const group = root.querySelector(`[data-curve-series="${architecture}"]`);
const polyline = group?.querySelector("[data-curve-line]");
polyline?.setAttribute("points", values.map((row) => curvePoint(row.step, row.value).join(",")).join(" "));
const points = group?.querySelector("[data-curve-points]");
if (points) {
points.innerHTML = values.map((row: { step: number; value: number }) => {
const [x, y] = curvePoint(row.step, row.value);
return `<circle cx="${x}" cy="${y}" r="4.2" fill="${colors[architecture]}"><title>${labels[architecture]} · step ${row.step}: ${row.value.toFixed(5)} BPC</title></circle>`;
}).join("");
}
const finalNode = root.querySelector<HTMLElement>(`[data-curve-final="${architecture}"]`);
const finalValue = values[values.length - 1]?.value;
if (finalNode && Number.isFinite(finalValue)) finalNode.textContent = finalValue.toFixed(5);
});
const copy = root.querySelector<HTMLElement>("[data-curve-copy]");
if (copy) copy.textContent = seedIndex < 0
? "三 seed 均值;正式判据使用逐 seed paired delta,不把三次运行当成 benchmark 样本总体。"
: `初始化 seed ${data.grid.seeds[seedIndex]};三种结构共享这一 seed 的公共参数与全部训练窗口。`;
curveButtons.forEach((button) => button.setAttribute("aria-pressed", String(button.dataset.curveSeed === seedKey)));
};
curveButtons.forEach((button) => button.addEventListener("click", () => updateCurve(button.dataset.curveSeed)));
updateCurve("mean");
const rmsButtons = [...root.querySelectorAll<HTMLButtonElement>("[data-rms-metric]")];
const rmsTitles: Record<AttnResRmsMetric, string> = {
stream_state_rms: "STREAM / PARTIAL STATE RMS",
layer_input_rms: "LAYER INPUT RMS",
branch_output_rms: "BRANCH OUTPUT RMS",
};
const updateRms = (metric: AttnResRmsMetric) => {
const all: number[] = architectures.flatMap((architecture) => data.traces[architecture][metric].mean);
const maximum = Math.max(...all) * 1.06;
architectures.forEach((architecture) => {
const values: number[] = data.traces[architecture][metric].mean;
const points = values.map((value: number, index: number) => {
const x = 62 + (index + 1) / 32 * 756;
const y = 20 + (1 - value / maximum) * 260;
return `${x},${y}`;
}).join(" ");
root.querySelector(`[data-rms-series="${architecture}"]`)?.setAttribute("points", points);
});
root.querySelectorAll<HTMLElement>("[data-rms-tick]").forEach((node) => {
node.textContent = (Number(node.dataset.rmsTick) * maximum).toFixed(maximum >= 1 ? 2 : 3);
});
const title = root.querySelector<HTMLElement>("[data-rms-title]");
if (title) title.textContent = rmsTitles[metric];
rmsButtons.forEach((button) => button.setAttribute("aria-pressed", String(button.dataset.rmsMetric === metric)));
};
rmsButtons.forEach((button) => button.addEventListener("click", () => updateRms(button.dataset.rmsMetric as AttnResRmsMetric)));
updateRms("stream_state_rms");
const mixerButtons = [...root.querySelectorAll<HTMLButtonElement>("[data-mixer-arch]")];
const mixerViews = [...root.querySelectorAll<HTMLElement>("[data-mixer-view]")];
const updateMixer = (architecture: "full" | "block") => {
mixerButtons.forEach((button) => button.setAttribute("aria-pressed", String(button.dataset.mixerArch === architecture)));
mixerViews.forEach((view) => { view.hidden = view.dataset.mixerView !== architecture; });
};
mixerButtons.forEach((button) => button.addEventListener("click", () => updateMixer(button.dataset.mixerArch as "full" | "block")));
};
document.querySelectorAll<HTMLElement>("[data-attnres-lab]").forEach(initializeAttnRes);
document.addEventListener("astro:page-load", () => {
document.querySelectorAll<HTMLElement>("[data-attnres-lab]").forEach(initializeAttnRes);
});
</script>
<style>
.attnres-trace {
--trace-ink: #1c201e;
--trace-muted: #77746b;
--trace-line: rgba(28, 32, 30, .16);
--trace-paper: #f4f0e7;
--trace-raised: #faf7ef;
--trace-copper: #ba603b;
--trace-green: #163f3b;
width: min(1120px, 100%);
margin: 42px 0;
color: var(--trace-ink);
border: 1px solid var(--trace-line);
background: var(--trace-paper);
box-shadow: 0 30px 80px rgba(28, 32, 30, .09);
}
.trace-head {
display: grid;
grid-template-columns: minmax(0, 1.45fr) minmax(260px, .7fr);
gap: 44px;
padding: 30px;
color: #f5efe4;
background: var(--trace-green);
}
.trace-head p { margin: 0; color: rgba(245, 239, 228, .7); font: .65rem/1.7 var(--mono); }
.trace-head div > p { color: #d58a68; letter-spacing: .08em; }
.trace-head h3 { max-width: 720px; margin: 14px 0 0; color: inherit; font-size: clamp(1.15rem, 2.2vw, 1.75rem); line-height: 1.35; }
.trace-ledger {
display: grid;
grid-template-columns: repeat(6, 1fr);
border-bottom: 1px solid var(--trace-line);
}
.trace-ledger article { min-height: 126px; padding: 18px 15px; border-right: 1px solid var(--trace-line); }
.trace-ledger article:last-child { border-right: 0; }
.trace-ledger span, .panel-lead span, .output-weights span, .gradient-depth header span {
color: var(--trace-muted); font: .56rem/1.2 var(--mono); letter-spacing: .08em;
}
.trace-ledger b { display: block; margin-top: 23px; font: 700 1rem/1 var(--mono); }
.trace-ledger p { margin: 8px 0 0; color: var(--trace-muted); font-size: .6rem; line-height: 1.45; }
.trace-ledger .pass { color: #f7f0e6; background: var(--trace-copper); }
.trace-ledger .pass span, .trace-ledger .pass p { color: rgba(247, 240, 230, .74); }
.trace-tabs {
display: grid;
grid-template-columns: repeat(5, 1fr);
border-bottom: 1px solid var(--trace-line);
background: #e9e4da;
}
.trace-tabs button {
min-height: 112px;
padding: 16px;
text-align: left;
color: inherit;
border: 0;
border-right: 1px solid var(--trace-line);
background: transparent;
cursor: pointer;
}
.trace-tabs button:last-child { border-right: 0; }
.trace-tabs button[aria-selected="true"] { color: #f7f0e6; background: var(--trace-copper); }
.trace-tabs span, .trace-tabs small { display: block; color: var(--trace-muted); font: .55rem/1.2 var(--mono); }
.trace-tabs b { display: block; margin: 15px 0 8px; font-size: .71rem; }
.trace-tabs button[aria-selected="true"] span, .trace-tabs button[aria-selected="true"] small { color: rgba(247, 240, 230, .72); }
.trace-panel { padding: 30px; }
.panel-lead { display: grid; grid-template-columns: 1.05fr .95fr; gap: 48px; align-items: end; margin-bottom: 28px; }
.panel-lead h4 { max-width: 680px; margin: 10px 0 0; font-size: 1.2rem; line-height: 1.4; }
.panel-lead p { margin: 0; color: var(--trace-muted); font-size: .7rem; line-height: 1.7; }
.curve-controls, .rms-controls, .mixer-controls {
display: flex; flex-wrap: wrap; gap: 0; margin: 12px 0 20px;
}
.curve-controls button, .rms-controls button, .mixer-controls button {
padding: 11px 14px;
color: var(--trace-muted);
font: 700 .58rem/1 var(--mono);
border: 1px solid var(--trace-line);
background: var(--trace-raised);
cursor: pointer;
}
.curve-controls button + button, .rms-controls button + button, .mixer-controls button + button { border-left: 0; }
.curve-controls button[aria-pressed="true"], .rms-controls button[aria-pressed="true"], .mixer-controls button[aria-pressed="true"] {
color: #fff9ef; background: var(--trace-green);
}
.curve-layout { display: grid; grid-template-columns: minmax(0, 1fr) 235px; border: 1px solid var(--trace-line); background: var(--trace-raised); }
.curve-chart { min-width: 0; padding: 18px; border-right: 1px solid var(--trace-line); }
.curve-chart header, .rms-chart header, .output-weights header, .gradient-depth header {
display: flex; justify-content: space-between; color: var(--trace-muted); font: .56rem/1 var(--mono);
}
.curve-chart svg, .rms-chart svg { display: block; width: 100%; height: auto; margin-top: 12px; overflow: visible; }
.chart-grid line { stroke: rgba(28, 32, 30, .1); stroke-width: 1; }
.chart-grid text { fill: #8b867c; font: 12px var(--mono); text-anchor: end; }
.chart-grid g:has(line[x1="62"]) text { text-anchor: end; }
.chart-grid g:not(:has(line[x1="62"])) text { text-anchor: middle; }
.chart-legend { display: flex; flex-wrap: wrap; gap: 18px; padding: 7px 4px 2px; color: var(--trace-muted); font: .58rem/1 var(--mono); }
.chart-legend span { display: inline-flex; gap: 7px; align-items: center; }
.chart-legend i { width: 22px; height: 3px; background: var(--legend); }
.curve-readout { padding: 22px 18px; }
.curve-readout > span { color: var(--trace-copper); font: .56rem/1 var(--mono); }
.curve-readout article { display: grid; grid-template-columns: 6px 1fr auto; gap: 10px; align-items: center; padding: 18px 0; border-bottom: 1px solid var(--trace-line); }
.curve-readout article i { width: 4px; height: 34px; background: var(--series); }
.curve-readout article b { font-size: .68rem; }
.curve-readout article strong { font: 700 .9rem/1 var(--mono); }
.curve-readout p { color: var(--trace-muted); font-size: .62rem; line-height: 1.55; }
.seed-pairs { display: grid; grid-template-columns: repeat(4, 1fr); margin-top: 20px; border-top: 1px solid var(--trace-line); border-left: 1px solid var(--trace-line); }
.seed-pairs article { min-height: 165px; padding: 17px; border-right: 1px solid var(--trace-line); border-bottom: 1px solid var(--trace-line); background: var(--trace-raised); }
.seed-pairs header { display: flex; justify-content: space-between; color: var(--trace-muted); font: .53rem/1 var(--mono); }
.seed-pairs article > div { display: flex; justify-content: space-between; margin-top: 24px; }
.seed-pairs article > div span { color: var(--trace-muted); font: .54rem/1 var(--mono); }
.seed-pairs article > div strong { color: var(--trace-green); font: 700 .72rem/1 var(--mono); }
.seed-pairs .verdict { color: #f7f0e6; background: var(--trace-green); }
.seed-pairs .verdict header { color: rgba(247, 240, 230, .65); }
.seed-pairs .verdict > strong { display: block; margin-top: 26px; font: 700 .8rem/1.2 var(--mono); }
.seed-pairs .verdict p { color: rgba(247, 240, 230, .7); font-size: .61rem; line-height: 1.5; }
.rms-chart { padding: 18px; border: 1px solid var(--trace-line); background: var(--trace-raised); }
.block-rhythm { margin-top: 20px; padding: 20px; color: #f7f0e6; background: var(--trace-green); }
.block-rhythm header { display: flex; justify-content: space-between; color: rgba(247, 240, 230, .68); font: .56rem/1 var(--mono); }
.block-rhythm > div { display: grid; grid-template-columns: repeat(32, 1fr); align-items: end; height: 160px; margin-top: 18px; border-bottom: 1px solid rgba(255,255,255,.3); }
.block-rhythm i { position: relative; display: block; min-height: 8px; height: min(100%, var(--height)); margin-right: 2px; background: #d4835d; }
.block-rhythm i.boundary { margin-right: 7px; }
.block-rhythm i span { position: absolute; bottom: -18px; left: 50%; color: rgba(255,255,255,.55); font: .46rem/1 var(--mono); transform: translateX(-50%); }
.block-rhythm p { margin: 35px 0 0; color: rgba(247, 240, 230, .72); font-size: .65rem; line-height: 1.6; }
.heatmap-scroll { overflow-x: auto; padding-bottom: 8px; }
.depth-heatmap { min-width: 760px; padding: 15px; border: 1px solid var(--trace-line); background: var(--trace-raised); }
.heat-row { display: grid; grid-template-columns: 28px 1fr; gap: 7px; margin-bottom: 3px; }
.heat-row > span { color: var(--trace-muted); font: .48rem/11px var(--mono); }
.heat-row > div { display: grid; grid-template-columns: repeat(var(--columns), 1fr); gap: 2px; }
.heat-row i { display: block; height: 11px; background: rgba(186, 96, 59, var(--heat)); }
.heat-row i.empty { background: rgba(28, 32, 30, .055); }
.output-weights { margin-top: 18px; padding: 18px; border: 1px solid var(--trace-line); background: var(--trace-raised); }
.output-weights > div { display: grid; grid-template-columns: repeat(auto-fit, minmax(10px, 1fr)); align-items: end; height: 170px; gap: 3px; margin-top: 15px; border-bottom: 1px solid var(--trace-line); }
.output-weights i { position: relative; display: block; height: max(3px, var(--weight)); background: var(--trace-green); }
.output-weights i.spikeSource { background: #d24e3f; }
.output-weights i span { position: absolute; bottom: -17px; left: 50%; color: var(--trace-muted); font: .43rem/1 var(--mono); transform: translateX(-50%); }
.posthoc-callout { display: grid; grid-template-columns: 1fr auto 1fr auto 1fr; gap: 16px; align-items: center; margin: 35px 0 0; padding: 22px; border-left: 5px solid var(--trace-copper); background: #ebe2d5; }
.posthoc-callout > span { grid-column: 1 / -1; color: var(--trace-copper); font: .57rem/1 var(--mono); }
.posthoc-callout div b { display: block; color: var(--trace-muted); font: .53rem/1 var(--mono); }
.posthoc-callout div strong { display: block; margin-top: 12px; font: 700 .95rem/1 var(--mono); }
.posthoc-callout > i { color: var(--trace-copper); }
.posthoc-callout > p { grid-column: 1 / -1; margin: 3px 0 0; color: var(--trace-muted); font-size: .64rem; line-height: 1.6; }
.gradient-cv { display: grid; grid-template-columns: repeat(4, 1fr); border-top: 1px solid var(--trace-line); border-left: 1px solid var(--trace-line); }
.gradient-cv article { min-height: 145px; padding: 18px; border-right: 1px solid var(--trace-line); border-bottom: 1px solid var(--trace-line); background: var(--trace-raised); }
.gradient-cv span { color: var(--trace-muted); font: .55rem/1 var(--mono); }
.gradient-cv b { display: block; margin-top: 25px; font: 700 1.1rem/1 var(--mono); }
.gradient-cv p { color: var(--trace-muted); font-size: .59rem; line-height: 1.5; }
.gradient-cv .warning { border-top: 4px solid var(--trace-copper); }
.gradient-cv .verdict { color: #f7f0e6; background: var(--trace-green); }
.gradient-cv .verdict span, .gradient-cv .verdict p { color: rgba(247, 240, 230, .68); }
.gradient-depth { margin-top: 20px; padding: 18px; border: 1px solid var(--trace-line); background: var(--trace-raised); }
.gradient-grid { display: grid; grid-template-columns: repeat(16, 1fr); gap: 5px; align-items: end; height: 250px; margin-top: 17px; }
.gradient-grid article { display: grid; grid-template-rows: 1fr auto; height: 100%; }
.gradient-grid article > span { grid-row: 2; margin-top: 6px; color: var(--trace-muted); font: .45rem/1 var(--mono); text-align: center; }
.gradient-grid article > div { display: grid; grid-template-columns: repeat(3, 1fr); gap: 1px; align-items: end; }
.gradient-grid i { display: block; height: max(2px, var(--bar)); background: var(--series); }
.metric-boundary { display: grid; grid-template-columns: 1fr 40px 1fr; gap: 12px; align-items: center; margin-top: 20px; padding: 20px; background: #ebe2d5; }
.metric-boundary article { padding: 12px; }
.metric-boundary article span { color: var(--trace-copper); font: .55rem/1 var(--mono); }
.metric-boundary article b { display: block; margin-top: 13px; font-size: .8rem; }
.metric-boundary article p { color: var(--trace-muted); font-size: .61rem; }
.metric-boundary > i { color: var(--trace-copper); font-size: 1.2rem; text-align: center; }
.metric-boundary > p { grid-column: 1 / -1; margin: 0; padding-top: 16px; border-top: 1px solid var(--trace-line); color: var(--trace-muted); font-size: .65rem; line-height: 1.6; }
.cost-grid { display: grid; grid-template-columns: repeat(3, 1fr); border-top: 1px solid var(--trace-line); border-left: 1px solid var(--trace-line); }
.cost-grid article { padding: 20px; border-right: 1px solid var(--trace-line); border-bottom: 1px solid var(--trace-line); background: var(--trace-raised); }
.cost-grid header { display: flex; justify-content: space-between; color: var(--trace-muted); font: .54rem/1 var(--mono); }
.cost-grid article > div { display: flex; justify-content: space-between; margin-top: 28px; }
.cost-grid article > div span { color: var(--trace-muted); font: .53rem/1 var(--mono); }
.cost-grid article > div strong { font: 700 .7rem/1 var(--mono); }
.cost-grid article > p { margin: 20px 0 0; padding-top: 13px; border-top: 1px solid var(--trace-line); color: var(--trace-copper); font: .58rem/1 var(--mono); }
.replay-chain { display: flex; gap: 10px; align-items: stretch; margin-top: 20px; }
.replay-chain article { flex: 1; padding: 18px; border: 1px solid var(--trace-line); background: var(--trace-raised); }
.replay-chain article.pass { color: #f7f0e6; background: var(--trace-green); }
.replay-chain span { color: var(--trace-muted); font: .54rem/1 var(--mono); }
.replay-chain article.pass span, .replay-chain article.pass p { color: rgba(247, 240, 230, .66); }
.replay-chain b { display: block; margin-top: 20px; font-size: .7rem; }
.replay-chain p { color: var(--trace-muted); font-size: .57rem; line-height: 1.5; }
.replay-chain > i { align-self: center; color: var(--trace-copper); }
.hash-ledger { display: grid; grid-template-columns: repeat(2, 1fr); margin-top: 20px; border-top: 1px solid var(--trace-line); border-left: 1px solid var(--trace-line); }
.hash-ledger article { min-width: 0; padding: 16px; border-right: 1px solid var(--trace-line); border-bottom: 1px solid var(--trace-line); background: #e9e4da; }
.hash-ledger span { display: block; color: var(--trace-muted); font: .52rem/1 var(--mono); }
.hash-ledger code { display: block; margin-top: 11px; overflow: hidden; color: var(--trace-green); font: .55rem/1.3 var(--mono); text-overflow: ellipsis; }
.claim-grid { display: grid; grid-template-columns: repeat(2, 1fr); margin-top: 20px; }
.claim-grid article { padding: 22px; }
.claim-grid .yes { color: #f7f0e6; background: var(--trace-green); }
.claim-grid .no { background: #e7d8ca; }
.claim-grid span { font: .58rem/1 var(--mono); letter-spacing: .07em; }
.claim-grid ul { margin: 18px 0 0; padding-left: 17px; }
.claim-grid li { margin-top: 10px; font-size: .65rem; line-height: 1.55; }
.claim-grid .yes span, .claim-grid .yes li { color: rgba(247, 240, 230, .78); }
@media (max-width: 920px) {
.trace-ledger { grid-template-columns: repeat(3, 1fr); }
.trace-ledger article:nth-child(3) { border-right: 0; }
.trace-tabs { grid-template-columns: repeat(3, 1fr); }
.curve-layout { grid-template-columns: 1fr; }
.curve-chart { border-right: 0; border-bottom: 1px solid var(--trace-line); }
.seed-pairs, .gradient-cv { grid-template-columns: repeat(2, 1fr); }
.gradient-grid { grid-template-columns: repeat(8, 1fr); height: 440px; }
.gradient-grid article { height: 210px; }
}
@media (max-width: 680px) {
.trace-head, .panel-lead { grid-template-columns: 1fr; gap: 20px; }
.trace-head, .trace-panel { padding: 20px; }
.trace-ledger { grid-template-columns: repeat(2, 1fr); }
.trace-ledger article:nth-child(3) { border-right: 1px solid var(--trace-line); }
.trace-ledger article:nth-child(even) { border-right: 0; }
.trace-tabs { display: flex; overflow-x: auto; }
.trace-tabs button { flex: 0 0 190px; }
.seed-pairs, .gradient-cv, .cost-grid, .hash-ledger, .claim-grid { grid-template-columns: 1fr; }
.rms-controls button, .mixer-controls button { flex: 1 0 100%; border-left: 1px solid var(--trace-line) !important; }
.block-rhythm > div { min-width: 680px; }
.block-rhythm { overflow-x: auto; }
.posthoc-callout { grid-template-columns: 1fr; }
.posthoc-callout > i { transform: rotate(90deg); text-align: center; }
.posthoc-callout > p { grid-column: 1; }
.gradient-grid { grid-template-columns: repeat(4, 1fr); height: 850px; }
.metric-boundary { grid-template-columns: 1fr; }
.metric-boundary > i { transform: rotate(90deg); }
.metric-boundary > p { grid-column: 1; }
.replay-chain { flex-direction: column; }
.replay-chain > i { transform: rotate(90deg); }
}
</style>
+9 -9
View File
@@ -107,7 +107,7 @@ const paths = [
</div>
<aside class="hero-aside" aria-label="项目统计">
<span>RESEARCH CUTOFF</span>
<strong>29 · 07 · 2026</strong>
<strong>30 · 07 · 2026</strong>
<p>首版持续建设中,所有动态结论带日期</p>
<div class="hero-stats">
<div><b>17</b><span>核心专题</span></div>
@@ -128,20 +128,20 @@ const paths = [
<div class="release-grid">
<a class="release-card k3-release" href="/k3/">
<div>
<p class="eyebrow"><span>NEW / K3 ROUND 03</span> REPORT · CHECKPOINT · KERNEL · BOUNDARY</p>
<h2>47 页不再压成摘要:再把 1.56 TB 开放工件接回报告</h2>
<p class="eyebrow"><span>NEW / K3 ROUND 04</span> ATTENTION RESIDUALS · PREREGISTERED PROBE</p>
<h2>真实 K3 权重仍有冲突:先把一个可证伪的深度路由问题完整做完</h2>
<p>
在三十二张报告问题账之外,继续审计 96 个 safetensors 分片、497,220 个 tensor entries、
真实 KDA / MLA / MoE / MoonViT shape、小范围权重统计,并把 FlashKDA 推进到 RTX 5090
6/6 exact-match、K3 fixed / varlen 计时与未决 checkpoint 形状矛盾。
不裁剪 <code>A_log [128]</code> 冒充 96-head K3 forward;冻结相同主干、初始化、数据窗口和预算,
从零训练 Baseline / Full / Block 共 9 个 2,000-step 格。两种 AttnRes 的三 seed BPC
配对方向都为负,但梯度均匀性指标没有复现论文叙述;支持与反证在同一实验室展示。
</p>
</div>
<dl>
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div>
<div><dt>ARTIFACTS</dt><dd>96 shards · 497,220 entries</dd></div>
<div><dt>KERNEL</dt><dd>sm_120a · exact 6/6</dd></div>
<div><dt>GRID</dt><dd>3 structures × 3 seeds</dd></div>
<div><dt>REPLAY</dt><dd>2,000 steps · 8/8 exact</dd></div>
</dl>
<span class="release-arrow" aria-hidden="true">从报告目录进入开放工件证据链 →</span>
<span class="release-arrow" aria-hidden="true">进入训练曲线、深度权重与梯度反证 →</span>
</a>
<a class="release-card deepseek-release" href="/deepseek/">
<div>
+31 -6
View File
@@ -2,6 +2,7 @@
import BaseLayout from "@/layouts/BaseLayout.astro";
import ArchitectureExplorer from "@/components/ArchitectureExplorer.astro";
import K3ArtifactLab from "@/components/K3ArtifactLab.astro";
import K3AttnResTraceLab from "@/components/K3AttnResTraceLab.astro";
import K3ReportLab from "@/components/K3ReportLab.astro";
import { k3FigureAtlas, k3Ledgers, k3PaperChain, k3ReportMap } from "@/data/k3";
@@ -36,7 +37,8 @@ const toc = [
["27", "xtml", "XTML 协议"],
["28", "lab", "八联交互实验"],
["29", "artifacts", "开放权重工件审计"],
["30", "audit", "21 张图表审计"],
["30", "attnres-reduced", "AttnRes 缩小机制实验"],
["31", "audit", "21 张图表审计"],
["↳", "papers", "100 节点阅读链"],
];
@@ -105,13 +107,13 @@ const paperGroups = [
<BaseLayout
title="Kimi K3 技术报告完整深读:架构、训练、RL、系统与评测"
description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图与一百个一手阅读节点,逐节读懂 Kimi K3。"
description="用三十二张问题账、二十一张图表审计、八个机制实验、四个开放工件视图、五个 AttnRes 独立实验视图与一百个一手阅读节点,逐节读懂 Kimi K3。"
section="k3"
>
<header class="page-hero k3-hero">
<div class="page-hero-inner">
<div>
<p class="eyebrow"><span>ANCHOR REPORT / ROUND 03</span> KIMI K3 · REPORT → OPEN ARTIFACTS</p>
<p class="eyebrow"><span>ANCHOR REPORT / ROUND 04</span> KIMI K3 · REPORT → ARTIFACTS → INDEPENDENT PROBE</p>
<h1>不把报告压成摘要<br />把每个因果环节<br />重新展开</h1>
<p class="lead">
K3 同时扩展序列、深度、宽度、视觉与 Agent 轨迹。真正值得读的不是 2.8T 这个最大数字,
@@ -121,11 +123,11 @@ const paperGroups = [
<dl class="page-facts">
<div><dt>QUESTIONS</dt><dd>32 张问题账</dd></div>
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div>
<div><dt>LABS</dt><dd>8 个机制实验 + 4 个工件视图</dd></div>
<div><dt>LABS</dt><dd>8 + 4 + 5 个交互视图</dd></div>
<div><dt>READING</dt><dd>100 个一手 / 官方节点</dd></div>
<div><dt>MODEL</dt><dd>2.78T total / 104.2B active</dd></div>
<div><dt>ARTIFACTS</dt><dd>96 shards · 497,220 tensors</dd></div>
<div><dt>STATUS</dt><dd>K3 三轮进行中</dd></div>
<div><dt>STATUS</dt><dd>K3 四轮 · AttnRes 实验</dd></div>
</dl>
</div>
</header>
@@ -871,8 +873,31 @@ const paperGroups = [
</div>
</section>
<section class="article-section" id="attnres-reduced">
<p class="eyebrow"><span>30</span> REDUCED ATTENTION RESIDUALS STUDY</p>
<h2>真实 K3 权重还不能诚实 forward;先把一个可证伪的 AttnRes 问题完整做完</h2>
<p class="lede">
checkpoint 的 <code>A_log [128]</code> 与 config、remote code、FlashKDA、vLLM 和 SGLang
期望的 96 heads 仍没有公开转换合同。本轮不裁剪权重冒充 K3,而是预注册一个从零训练的缩小实验:
相同 16-block Transformer、相同数据窗口与相同初始化,只改变 residual source 的读取拓扑。
</p>
<div class="artifact-callout">
<article><span>F / FROZEN</span><b>3 structures × 3 seeds</b><p>9 格各 2,000 steps;每格 16,384,000 target bytes。</p></article>
<article><span>X / OBSERVED</span><b>Full −0.01457 BPC</b><p>三个 paired seed 同为负,达到预注册 −0.010 判据。</p></article>
<article><span>X / OBSERVED</span><b>Block −0.04247 BPC</b><p>同样满足本 reduced protocol 内的方向支持规则。</p></article>
<article class="warning"><span>B / BOUNDARY</span><b>gradient CV 未复现</b><p>Baseline 0.3447;Full 0.5087;Block 0.6306。</p></article>
</div>
<K3AttnResTraceLab />
<div class="hero-actions">
<a class="button primary" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/research/K3_ATTNRES_REDUCED_AUDIT.md">阅读完整研究审计</a>
<a class="button" href="https://git.k1412.top/wuyang/llm-atlas/src/branch/main/experiments/k3/attnres">复跑公开实验代码</a>
<a class="button" href="https://arxiv.org/abs/2603.15031">Attention Residuals 原论文</a>
<a class="button" href="https://github.com/MoonshotAI/Attention-Residuals">官方实现</a>
</div>
</section>
<section class="article-section" id="audit">
<p class="eyebrow"><span>30</span> FIGURE & TABLE AUDIT</p>
<p class="eyebrow"><span>31</span> FIGURE & TABLE AUDIT</p>
<h2>Figure 1–16、Table 1–5:每张图究竟支持什么,不能支持什么</h2>
<div class="figure-atlas">
{k3FigureAtlas.map(([id, report, title, contract]) => (
+9 -4
View File
@@ -9,7 +9,7 @@ const researching = chapters.filter((chapter) => ["researching", "drafting"].inc
const workstreams = [
{ label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" },
{ label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" },
{ label: "Kimi K3 深读", value: 94, next: "接入真实 hidden-state / expert-load / cache traces,并重绘报告数值图" },
{ label: "Kimi K3 深读", value: 96, next: "对齐 AttnRes 梯度定义并扩展深度/预算;等待 A_log 官方转换合同" },
{ label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau,并加入真实小语料复现" },
{ label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" },
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
@@ -50,7 +50,7 @@ const workstreams = [
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-30 04:00 CST</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-30 07:30 CST</dd></div>
<div><dt>MODE</dt><dd>持续迭代,不锁死版本</dd></div>
</dl>
</div>
@@ -97,7 +97,7 @@ const workstreams = [
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全,包含先修依赖和三条贯穿案例。</p></article>
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
<article><span>✓</span><h3>八十九个原创交互视图</h3><p>K3 三轴图、八联报告实验与四联开放工件实验,DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验,以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>九十四个原创交互视图</h3><p>K3 三轴图、八联报告实验、四联开放工件实验与五联 AttnRes 独立实验,DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed、cross-source 与 task-bootstrap CRN 五轮实验,以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
@@ -105,6 +105,7 @@ const workstreams = [
<article><span>✓</span><h3>DeepSeek 八轮真实权重里程碑</h3><p>把覆盖扩到 HumanEval / GSM8K 各 32 条冻结任务,用显式 SHA-256 uniform tape 驱动四个条件的共同随机数采样:352 条正式输出中 343 条 natural EOS、320 个 unique trajectories;10,000 次选定任务配对 bootstrap 的正确性区间均跨零,长度则揭示 Code 与 Math 的相反方向。新进程十二字段重放 64 / 64 exact。</p></article>
<article><span>✓</span><h3>Kimi K3 技术报告二轮深读</h3><p>三十二张问题账、Figure 1–16 / Table 1–5 审计、100 节点阅读链,以及 Delta—Decay—AttnRes—LatentMoE—SiTU—QB—MOPD—Cache 八联实验。</p></article>
<article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions,审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes;四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
<article><span>✓</span><h3>Kimi K3 四轮 AttnRes 独立实验</h3><p>冻结三结构 × 三 seed 的 9 个 2,000-step 格;Full / Block 相对 Baseline 的平均 paired delta 为 −0.01457 / −0.04247 BPC,但核心参数梯度 CV 没有复现论文叙述。指定正式格全新进程八字段 exact,五视图同时展示结果、反证、成本与 claim boundary。</p></article>
<article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel;6/6 官方参考逐元素相等,并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
<article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article>
@@ -133,7 +134,7 @@ const workstreams = [
</div>
<div class="queue-table">
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
<div><span>P0</span><strong>K3 三轮</strong><p>开放权重 traces → FlashKDA / AttnRes / MoE 真实行为 → Figure 1–16 数值重绘与独立复现</p><em>运行证据 + 逐图复现</em></div>
<div><span>P0</span><strong>K3 四轮后续</strong><p>对齐论文梯度定义 → 增加 depth / budget → 等待 A_log 官方合同后进入真实 checkpoint forward</p><em>尺度复查 + 工件边界</em></div>
<div><span>P0</span><strong>DeepSeek 八轮后续</strong><p>干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
@@ -215,6 +216,10 @@ const workstreams = [
<div><time>2026-07-29</time><b>A_log 形状冲突保持未决</b><p>checkpoint 的 [128] 与 config / remote code / FlashKDA API 期待的 [96] 并列展示;不宣布权重损坏,也不把 channel-wise 假设写成真实 forward。</p></div>
<div><time>2026-07-29</time><b>FlashKDA 编译与执行永久分两道闸门</b><p>容器产出 sm_120a wheel 只证明可编译;RTX 5090 的 6/6 official-reference exact suite 通过后,才把证据升级为本机执行 X。</p></div>
<div><time>2026-07-29</time><b>作者表与 RTX 5090 表永久分账</b><p>H20 / GB200 保持 O;本站只报告独立环境、协议、300 samples/mode 和延迟分布,未跑本机 FLA 就不写本机 speedup。</p></div>
<div><time>2026-07-30</time><b>K3 权重冲突不靠裁剪“解决”</b><p>HF / FlashKDA / vLLM / SGLang 仍没有公开 A_log 128→96 转换;真实 K3 forward 继续标为未决。</p></div>
<div><time>2026-07-30</time><b>AttnRes 缩小实验先冻结、后运行</b><p>三结构共享公共主干、初始化、窗口与优化器;只按三个 paired seed 和预注册 −0.010 BPC 阈值给出本协议内方向判断。</p></div>
<div><time>2026-07-30</time><b>支持结果与梯度反结果同时进入主视区</b><p>Full / Block 的最终 BPC 同向改善;核心参数 gradient RMS CV 却高于 Baseline,不换指标掩盖。</p></div>
<div><time>2026-07-30</time><b>正式重放不把 wall time 纳入 exact</b><p>Block / seed-1 的模型、优化器、曲线、历史、诊断和环境八字段 exact;计时受调度影响,单独报告。</p></div>
<div><time>2026-07-29</time><b>32-token 对照改为同源 16→24</b><p>TNEWS 只有 105/10,000 条达到 32 tokens,强行统一会落入约 1% 极端长尾;24-token eligibility 仍保留 1,609 条中文候选。</p></div>
<div><time>2026-07-29</time><b>长度敏感性必须成对重采样</b><p>16-token 输入严格是 24-token 输入前缀,2,000 次 bootstrap 共用 prompt indices;结果只描述固定 cohort 的长度敏感性。</p></div>
<div><time>2026-07-29</time><b>三类 cohort 永久分身份</b><p>自然长度回答本批样本如何路由;matched-16 / 24 回答同一 prompt 多看 8 tokens 后如何变化,不把二者混成内容因果。</p></div>