wuyang/llm-atlas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: deepen Kimi K3 report guide

Browse Source
This commit is contained in:
wuyang
2026-07-29 12:02:04 +08:00
parent 6560e97737
commit b669615225
27 changed files with 2730 additions and 566 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/components/K3ReportLab.astro
+725
View File
@@ -0,0 +1,725 @@
<figure class="k3-lab" data-k3-lab>
<div class="lab-head">
<div>
<p>INTERACTIVE / EIGHT REPORT WORKBENCHES</p>
<h3>把八个容易误读的机制，拆成可以动手验算的对象</h3>
</div>
<p>
数学输出按页面公式实时计算；风险、通信与吞吐标签只是显式 toy model。
本实验没有复跑 K3 checkpoint、FlashKDA kernel、MoonEP 集群或线上 cache。
</p>
</div>
<div class="lab-tabs" role="tablist" aria-label="选择 K3 技术实验">
{[
["memory", "01", "DELTA MEMORY", "纠错写入"],
["decay", "02", "BOUNDED DECAY", "BF16 范围"],
["depth", "03", "ATTNRES", "深度来源"],
["width", "04", "LATENTMOE", "通信 payload"],
["situ", "05", "SITU-GLU", "有界激活"],
["qb", "06", "QB ROUTER", "分位均衡"],
["rl", "07", "MOPD / RL", "预算与长尾"],
["cache", "08", "PREFIX CACHE", "混合状态命中"],
].map(([id, number, title, subtitle], index) => (
<button
type="button"
role="tab"
data-k3-tab={id}
aria-selected={index === 0 ? "true" : "false"}
tabindex={index === 0 ? "0" : "-1"}
>
<span>{number}</span><b>{title}</b><small>{subtitle}</small>
</button>
))}
</div>
<section class="lab-panel" data-k3-panel="memory">
<div class="panel-intro">
<div><span>WORKBENCH 01 / DELTA RULE</span><h4>同一个 key 反复出现时，累加和纠错写入会走向完全不同的 state</h4></div>
<p>为便于看懂，这里把矩阵 state 压成“当前 key 对应的标量预测”。真实 KDA 是每个 head 的矩阵状态。</p>
</div>
<div class="controls four">
<label><span>重复写入次数 <output data-memory-writes-label>6</output></span><input data-memory-writes type="range" min="1" max="20" value="6" /></label>
<label><span>写入强度 β <output data-memory-beta-label>0.50</output></span><input data-memory-beta type="range" min="5" max="100" value="50" /></label>
<label><span>旧预测 <output data-memory-old-label>0.00</output></span><input data-memory-old type="range" min="-100" max="100" value="0" /></label>
<label><span>目标 value <output data-memory-target-label>1.00</output></span><input data-memory-target type="range" min="-100" max="200" value="100" /></label>
</div>
<div class="memory-stage">
<article>
<span>NAIVE ADDITIVE</span>
<code>s ← s + βv</code>
<div class="trace" data-additive-trace></div>
<b data-additive-result>3.00</b>
<p>每次都继续追加；重复 key 会把同一 value 越写越大。</p>
</article>
<article class="accent">
<span>DELTA UPDATE</span>
<code>s ← s + β(v − s)</code>
<div class="trace" data-delta-trace></div>
<b data-delta-result>0.98</b>
<p>只写“目标与旧预测的差”；在这个 toy key 上逐步逼近目标。</p>
</article>
</div>
<div class="metrics three">
<article><span>ADDITIVE ERROR</span><b data-additive-error>2.00</b><p>|state − target|</p></article>
<article><span>DELTA ERROR</span><b data-delta-error>0.02</b><p>|state − target|</p></article>
<article class="dark"><span>REAL KDA</span><b>matrix state</b><p>S ∈ R<sup>dₖ×dᵥ</sup>，另有 α 与 key geometry</p></article>
</div>
<div class="boundary"><b>精确标量递推 / 教学降维</b><p>这个实验只解释“纠错写入”直觉，不证明固定状态能无损回忆任意历史。</p></div>
</section>
<section class="lab-panel" data-k3-panel="decay" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 02 / LOWER-BOUNDED DECAY</span><h4>关键不是“衰减得慢”，而是让 16-token tile 的最坏 rescale 有有限上界</h4></div>
<p>K3 报告固定 g<sub>min</sub>=−5；此时 α&gt;e⁻⁵，16 步累计 log-decay&gt;−80，倒数小于 e⁸⁰。</p>
</div>
<div class="controls three">
<label><span>log-decay 下界 gmin <output data-decay-g-label>−5.0</output></span><input data-decay-g type="range" min="-120" max="-10" value="-50" /></label>
<label><span>secondary tile <output data-decay-tile-label>16</output></span><input data-decay-tile type="range" min="1" max="32" value="16" /></label>
<label><span>BF16 最大量级</span><input value="≈ 3.39 × 10³⁸" disabled /></label>
</div>
<div class="decay-plot">
<article><span>MIN RETENTION / STEP</span><b data-decay-alpha>6.74 × 10⁻³</b><div><i data-decay-alpha-bar></i></div><p>αmin = exp(gmin)</p></article>
<article><span>WORST TILE LOG-DECAY</span><b data-decay-log>−80.0</b><div><i data-decay-log-bar></i></div><p>tile × gmin</p></article>
<article class="accent"><span>RECIPROCAL RESCALE</span><b data-decay-recip>≈ 5.54 × 10³⁴</b><div><i data-decay-recip-bar></i></div><p>exp(−tile × gmin)</p></article>
</div>
<div class="verdict" data-decay-verdict><span>BF16 RANGE</span><b>WITHIN RANGE</b><p>报告默认 −5 × 16 = −80，理论倒数仍低于 BF16 最大有限值。</p></div>
<div class="boundary"><b>确定性范围计算 / 非 kernel benchmark</b><p>能落进动态范围不等于精度误差为零；报告的系统收益还来自把 diagonal tile 改成 Tensor Core 稠密矩阵乘。</p></div>
</section>
<section class="lab-panel" data-k3-panel="depth" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 03 / BLOCK ATTENTION RESIDUALS</span><h4>把“保存每一层”改成“保存每一块”，再看深度来源数量怎样变化</h4></div>
<p>计数是确定性的；内存只用“每 Token 每 hidden element 的来源份数”表示，不冒充真实运行时 GiB。</p>
</div>
<div class="controls three">
<label><span>backbone layers L <output data-depth-layers-label>93</output></span><input data-depth-layers type="range" min="24" max="120" value="93" /></label>
<label><span>block size S <output data-depth-block-label>12</output></span><input data-depth-block type="range" min="2" max="24" value="12" /></label>
<label><span>hidden d <output data-depth-hidden-label>7168</output></span><input data-depth-hidden type="range" min="1024" max="8192" step="512" value="7168" /></label>
</div>
<div class="depth-track" data-depth-track aria-label="Block AttnRes 教学层条带"></div>
<div class="metrics four">
<article><span>FULL SOURCES</span><b data-full-sources>94</b><p>L 层输出 + embedding</p></article>
<article><span>BLOCK SOURCES</span><b data-block-sources>9</b><p>ceil(L/S) + embedding</p></article>
<article><span>SOURCE REDUCTION</span><b data-depth-reduction>90.4%</b><p>来源份数的确定性比较</p></article>
<article class="dark"><span>K3 CONTRACT</span><b>12 / 8 / 9</b><p>12 层/块 · 8 个 layer blocks · 含 embedding 共 9 来源</p></article>
</div>
<div class="boundary"><b>精确计数 / 非吞吐预测</b><p>报告称 Full 的算术因 L&lt;100 尚可，真正代价在存活 activation 和 pipeline 跨 stage 通信。</p></div>
</section>
<section class="lab-panel" data-k3-panel="width" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 04 / LATENTMOE PAYLOAD</span><h4>激活更多专家时，先缩窄 routed representation 能省下多少 dispatch payload？</h4></div>
<p>元素数与字节数按选择精确计算；真实成本还含 router、all-to-all、权重读取、shared experts 与拓扑。</p>
</div>
<div class="controls five">
<label><span>full width d <output data-width-full-label>7168</output></span><input data-width-full type="range" min="1024" max="8192" step="512" value="7168" /></label>
<label><span>latent width ℓ <output data-width-latent-label>3584</output></span><input data-width-latent type="range" min="512" max="7168" step="256" value="3584" /></label>
<label><span>active experts k <output data-width-topk-label>16</output></span><input data-width-topk type="range" min="1" max="32" value="16" /></label>
<label><span>tokens <output data-width-tokens-label>4096</output></span><input data-width-tokens type="range" min="256" max="8192" step="256" value="4096" /></label>
<label><span>payload dtype</span><select data-width-bytes><option value="1">1 byte</option><option value="2" selected>2 bytes</option><option value="4">4 bytes</option></select></label>
</div>
<div class="payload-compare">
<article><span>CONVENTIONAL FULL-WIDTH ROUTE</span><b data-width-conventional>896.0 MiB</b><div><i></i></div><p>tokens × k × d × bytes</p></article>
<article class="accent"><span>LATENT ROUTE</span><b data-width-latent>448.0 MiB</b><div><i data-width-latent-bar></i></div><p>tokens × k × ℓ × bytes</p></article>
</div>
<div class="metrics three">
<article><span>PAYLOAD REDUCTION</span><b data-width-reduction>50.0%</b><p>只比较 routed token elements</p></article>
<article><span>SPARSITY</span><b>896 / 16 = 56×</b><p>K3 expert pool / active routed</p></article>
<article class="dark"><span>SHARED PATH</span><b>2 × full width</b><p>公共变换仍要付激活计算</p></article>
</div>
<div class="boundary"><b>精确 payload 算术 / 非网络流量实测</b><p>实际系统由 MoonEP 静态 shape、zero-copy communication 与专家执行共同决定。</p></div>
</section>
<section class="lab-panel" data-k3-panel="situ" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 05 / SITU-GLU</span><h4>在原点附近保留 SwiGLU 的形状，在大正输入处给乘积明确上界</h4></div>
<p>报告配置 β₁=4、β₂=25，因此标量切片的 |f(x)|≤100。曲线是函数，不是 validation loss。</p>
</div>
<div class="controls three">
<label><span>输入 x <output data-situ-x-label>20.0</output></span><input data-situ-x type="range" min="-100" max="1000" value="200" /></label>
<label><span>gate cap β₁ <output data-situ-b1-label>4</output></span><input data-situ-b1 type="range" min="1" max="12" value="4" /></label>
<label><span>up cap β₂ <output data-situ-b2-label>25</output></span><input data-situ-b2 type="range" min="5" max="50" value="25" /></label>
</div>
<div class="curve-stage">
<svg viewBox="0 0 720 260" role="img" aria-label="GLU、SwiGLU 与 SiTU-GLU 教学曲线">
<line x1="36" y1="224" x2="700" y2="224"></line>
<line x1="94" y1="20" x2="94" y2="240"></line>
<path data-curve-glu class="curve glu"></path>
<path data-curve-swiglu class="curve swiglu"></path>
<path data-curve-situ class="curve situ"></path>
</svg>
<div class="curve-legend"><span class="glu">GLU</span><span class="swiglu">SwiGLU</span><span class="situ">SiTU-GLU</span></div>
</div>
<div class="metrics four">
<article><span>GLU(x)</span><b data-situ-glu>20.00</b><p>σ(x) · x</p></article>
<article><span>SWIGLU(x)</span><b data-situ-swiglu>400.00</b><p>[xσ(x)] · x</p></article>
<article><span>SITU-GLU(x)</span><b data-situ-value>65.63</b><p>[β₁tanh(x/β₁)σ(x)]·β₂tanh(x/β₂)</p></article>
<article class="dark"><span>FORMAL BOUND</span><b data-situ-bound>100</b><p>β₁ × β₂</p></article>
</div>
<div class="boundary"><b>精确标量函数 / 非完整 FFN</b><p>真实模型的 Wg x 与 Wu x 并不相同；本图复现报告 Figure 4 的共同标量切片。</p></div>
</section>
<section class="lab-panel" data-k3-panel="qb" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 06 / QUANTILE BALANCING</span><h4>一次 forward 里先照常路由，再用“进入 Top-k 还差多少”决定下一步 bias</h4></div>
<p>固定使用报告 Figure 5 的 m=8、n=4、k=1 规模；score 是本站构造，不是 K3 trace。</p>
</div>
<div class="controls two">
<label><span>bias 更新强度 <output data-qb-strength-label>1.00</output></span><input data-qb-strength type="range" min="0" max="150" value="100" /></label>
<label><span>histogram bins <output data-qb-bins-label>64</output></span><input data-qb-bins type="range" min="4" max="128" step="4" value="64" /></label>
</div>
<div class="qb-stage">
<article><span>CURRENT TOP-1</span><div class="load-bars" data-qb-before></div><p data-qb-before-copy>loads = (4, 3, 1, 0)</p></article>
<i>→</i>
<article class="quantile-box"><span>TOP-2 CUTOFF</span><b>margin = score − α</b><p>每个 expert 取目标 q=mk/n=2 对应的分位阈值，再移除公共 offset。</p></article>
<i>→</i>
<article class="accent"><span>NEXT-STEP TOP-1</span><div class="load-bars" data-qb-after></div><p data-qb-after-copy>loads = (2, 2, 2, 2)</p></article>
</div>
<div class="metrics four">
<article><span>TARGET / EXPERT</span><b>q = 2</b><p>8 tokens × top-1 / 4 experts</p></article>
<article><span>BIAS VECTOR</span><b data-qb-bias>—</b><p>居中后用于 next step</p></article>
<article><span>MAX LOAD GAP</span><b data-qb-gap>0</b><p>after max − min</p></article>
<article class="dark"><span>HIST ERROR</span><b data-qb-error>≤ 1/64 range</b><p>通信近似只保证 bin-width 级</p></article>
</div>
<div class="boundary"><b>公式级 toy router / 非训练轨迹</b><p>bias 只影响 Top-k selection，不进入最终 mixture weight；当前 batch 不能使用由自身算出的新 bias。</p></div>
</section>
<section class="lab-panel" data-k3-panel="rl" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 07 / EFFORT × PARTIAL ROLLOUT × MOPD</span><h4>长程 RL 同时要记三本账：每题预算、完成阈值、教师密集信号</h4></div>
<p>三块标量计算互不替代：budget 管过度思考，λ 管 straggler，MOPD reward 管学生当前 prefix。</p>
</div>
<div class="controls five">
<label><span>prompts N <output data-rl-n-label>32</output></span><input data-rl-n type="range" min="4" max="64" step="4" value="32" /></label>
<label><span>rollouts K <output data-rl-k-label>8</output></span><input data-rl-k type="range" min="1" max="16" value="8" /></label>
<label><span>pause fraction λ <output data-rl-lambda-label>0.75</output></span><input data-rl-lambda type="range" min="10" max="100" value="75" /></label>
<label><span>base budget b₀ <output data-rl-base-label>8K</output></span><input data-rl-base type="range" min="2" max="32" value="8" /></label>
<label><span>effort multiplier τ <output data-rl-tau-label>2.0</output></span><input data-rl-tau type="range" min="5" max="50" value="20" /></label>
</div>
<div class="controls four">
<label><span>trajectory tokens T(y) <output data-rl-tokens-label>12K</output></span><input data-rl-tokens type="range" min="1" max="96" value="12" /></label>
<label><span>teacher p(yt) <output data-rl-teacher-label>0.60</output></span><input data-rl-teacher type="range" min="1" max="99" value="60" /></label>
<label><span>student p(yt) <output data-rl-student-label>0.30</output></span><input data-rl-student type="range" min="1" max="99" value="30" /></label>
<label><span>clip Rmax <output data-rl-clip-label>1.0</output></span><input data-rl-clip type="range" min="1" max="30" value="10" /></label>
</div>
<div class="rl-flow">
<article><span>EFFORT BUDGET</span><b data-rl-budget>16K</b><p data-rl-budget-copy>12K ≤ 2.0 × 8K：保留任务 reward</p></article>
<i>→</i>
<article><span>PARTIAL ROLLOUT</span><b data-rl-complete>192 complete</b><p data-rl-paused>64 paused / resume next iteration</p></article>
<i>→</i>
<article class="accent"><span>MOPD TOKEN REWARD</span><b data-rl-reward>+0.693</b><p>clip(log pteacher/pstudent, ±Rmax)</p></article>
</div>
<div class="boundary"><b>报告公式的标量例子 / 非 policy optimizer</b><p>实际系统还有 per-token regularization、domain/effort teacher selection、sandbox state 与 group completion dispatch。</p></div>
</section>
<section class="lab-panel" data-k3-panel="cache" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 08 / HYBRID PREFIX CACHE</span><h4>MLA prefix 对上了还不够：同一边界必须有每个 KDA group 的 state checkpoint</h4></div>
<p>默认复现报告 Figure 12 的 6144-token physical block 与 512-token hash block；checkpoint 周期是教学变量。</p>
</div>
<div class="controls four">
<label><span>matched prefix <output data-cache-matched-label>2800</output></span><input data-cache-matched type="range" min="0" max="12288" step="128" value="2816" /></label>
<label><span>physical block <output data-cache-physical-label>6144</output></span><input data-cache-physical type="range" min="2048" max="8192" step="512" value="6144" /></label>
<label><span>hash block <output data-cache-hash-label>512</output></span><input data-cache-hash type="range" min="128" max="1024" step="128" value="512" /></label>
<label><span>KDA checkpoint / hash blocks <output data-cache-checkpoint-label>1</output></span><input data-cache-checkpoint type="range" min="1" max="12" value="1" /></label>
</div>
<div class="cache-strip" data-cache-strip aria-label="混合 KDA MLA prefix cache 条带"></div>
<div class="metrics four">
<article><span>MLA HASH MATCH</span><b data-cache-mla>2560</b><p>floor(match/hash) × hash</p></article>
<article><span>KDA CHECKPOINT</span><b data-cache-kda>2560</b><p>最长同时存在的 state 边界</p></article>
<article><span>JOINT HIT</span><b data-cache-hit>2560</b><p>两类 cache 的共同最长前缀</p></article>
<article class="dark"><span>RECOMPUTE</span><b data-cache-recompute>256 tokens</b><p>matched − joint hit</p></article>
</div>
<div class="boundary"><b>精确边界算术 / 非线上 hit rate</b><p>真实系统还需 pin、copy-on-write、跨 KDA group 原子失效、调度 step 可见性和 prefill/decode TP re-layout。</p></div>
</section>
</figure>
<script>
const roots = document.querySelectorAll<HTMLElement>("[data-k3-lab]");
const fmt = new Intl.NumberFormat("en-US", { maximumFractionDigits: 2 });
roots.forEach((root) => {
const $ = <T extends HTMLElement = HTMLElement>(selector: string) => root.querySelector<T>(selector)!;
const $$ = <T extends HTMLElement = HTMLElement>(selector: string) => [...root.querySelectorAll<T>(selector)];
const value = (selector: string) => Number(root.querySelector<HTMLInputElement | HTMLSelectElement>(selector)?.value ?? 0);
const put = (selector: string, content: string | number) => {
const node = $(selector);
if (node) node.textContent = String(content);
};
const tabs = $$<HTMLButtonElement>("[data-k3-tab]");
const panels = $$<HTMLElement>("[data-k3-panel]");
const selectTab = (id: string) => {
tabs.forEach((tab) => {
const active = tab.dataset.k3Tab === id;
tab.setAttribute("aria-selected", String(active));
tab.tabIndex = active ? 0 : -1;
});
panels.forEach((panel) => {
panel.hidden = panel.dataset.k3Panel !== id;
});
};
tabs.forEach((tab, index) => {
tab.addEventListener("click", () => selectTab(tab.dataset.k3Tab ?? "memory"));
tab.addEventListener("keydown", (event: KeyboardEvent) => {
if (!["ArrowLeft", "ArrowRight", "Home", "End"].includes(event.key)) return;
event.preventDefault();
let next = index;
if (event.key === "ArrowLeft") next = (index - 1 + tabs.length) % tabs.length;
if (event.key === "ArrowRight") next = (index + 1) % tabs.length;
if (event.key === "Home") next = 0;
if (event.key === "End") next = tabs.length - 1;
selectTab(tabs[next].dataset.k3Tab ?? "memory");
tabs[next].focus();
});
});
const formatSigned = (n: number, digits = 2) => `${n >= 0 ? "+" : ""}${n.toFixed(digits)}`;
const formatBytes = (bytes: number) => {
if (bytes >= 2 ** 30) return `${(bytes / 2 ** 30).toFixed(2)} GiB`;
if (bytes >= 2 ** 20) return `${(bytes / 2 ** 20).toFixed(1)} MiB`;
if (bytes >= 2 ** 10) return `${(bytes / 2 ** 10).toFixed(1)} KiB`;
return `${fmt.format(bytes)} B`;
};
const sigmoid = (x: number) => 1 / (1 + Math.exp(-x));
const renderMemory = () => {
const writes = value("[data-memory-writes]");
const beta = value("[data-memory-beta]") / 100;
const initial = value("[data-memory-old]") / 100;
const target = value("[data-memory-target]") / 100;
let additive = initial;
let delta = initial;
const additiveTrace = [additive];
const deltaTrace = [delta];
for (let i = 0; i < writes; i += 1) {
additive += beta * target;
delta += beta * (target - delta);
additiveTrace.push(additive);
deltaTrace.push(delta);
}
put("[data-memory-writes-label]", writes);
put("[data-memory-beta-label]", beta.toFixed(2));
put("[data-memory-old-label]", initial.toFixed(2));
put("[data-memory-target-label]", target.toFixed(2));
put("[data-additive-result]", additive.toFixed(3));
put("[data-delta-result]", delta.toFixed(3));
put("[data-additive-error]", Math.abs(additive - target).toFixed(3));
put("[data-delta-error]", Math.abs(delta - target).toFixed(3));
const draw = (selector: string, trace: number[], colorClass: string) => {
const maxAbs = Math.max(1, ...trace.map((v) => Math.abs(v)));
$(selector).innerHTML = trace.map((v, i) =>
`<i class="${colorClass}" style="height:${Math.max(4, Math.abs(v) / maxAbs * 62)}px" title="step ${i}: ${v.toFixed(3)}"></i>`
).join("");
};
draw("[data-additive-trace]", additiveTrace, "plain");
draw("[data-delta-trace]", deltaTrace, "accent");
};
const renderDecay = () => {
const g = value("[data-decay-g]") / 10;
const tile = value("[data-decay-tile]");
const alpha = Math.exp(g);
const logDecay = g * tile;
const log10Recip = -logDecay / Math.LN10;
const bf16Log10 = Math.log10(3.3895313892515355e38);
const within = log10Recip <= bf16Log10;
put("[data-decay-g-label]", g.toFixed(1).replace("-", "−"));
put("[data-decay-tile-label]", tile);
put("[data-decay-alpha]", alpha < .001 ? alpha.toExponential(2) : alpha.toPrecision(3));
put("[data-decay-log]", logDecay.toFixed(1).replace("-", "−"));
put("[data-decay-recip]", `≈ 10^${log10Recip.toFixed(2)}`);
$("[data-decay-alpha-bar]").style.width = `${Math.max(2, Math.min(100, alpha * 100))}%`;
$("[data-decay-log-bar]").style.width = `${Math.min(100, Math.abs(logDecay) / 120 * 100)}%`;
$("[data-decay-recip-bar]").style.width = `${Math.min(100, log10Recip / bf16Log10 * 100)}%`;
const verdict = $("[data-decay-verdict]");
verdict.classList.toggle("danger", !within);
verdict.querySelector<HTMLElement>("b")!.textContent = within ? "WITHIN RANGE" : "OVERFLOW RISK";
verdict.querySelector<HTMLElement>("p")!.textContent = within
? `最坏倒数约 10^${log10Recip.toFixed(2)}，低于 BF16 最大有限量级。`
: `最坏倒数约 10^${log10Recip.toFixed(2)}，超过 BF16 最大有限量级。`;
};
const renderDepth = () => {
const layers = value("[data-depth-layers]");
const size = value("[data-depth-block]");
const hidden = value("[data-depth-hidden]");
const blocks = Math.ceil(layers / size);
const full = layers + 1;
const grouped = blocks + 1;
put("[data-depth-layers-label]", layers);
put("[data-depth-block-label]", size);
put("[data-depth-hidden-label]", hidden);
put("[data-full-sources]", full);
put("[data-block-sources]", grouped);
put("[data-depth-reduction]", `${((1 - grouped / full) * 100).toFixed(1)}%`);
const track = $("[data-depth-track]");
track.innerHTML = Array.from({ length: blocks }, (_, index) => {
const count = Math.min(size, layers - index * size);
return `<div style="flex:${count}" title="block ${index + 1}: ${count} layers"><span>B${index + 1}</span><b>${count}</b></div>`;
}).join("");
};
const renderWidth = () => {
const full = value("[data-width-full]");
const latent = Math.min(value("[data-width-latent]"), full);
const topk = value("[data-width-topk]");
const tokens = value("[data-width-tokens]");
const bytes = value("[data-width-bytes]");
const conventional = tokens * topk * full * bytes;
const compressed = tokens * topk * latent * bytes;
put("[data-width-full-label]", full);
put("[data-width-latent-label]", latent);
put("[data-width-topk-label]", topk);
put("[data-width-tokens-label]", tokens);
put("[data-width-conventional]", formatBytes(conventional));
put("[data-width-latent]", formatBytes(compressed));
put("[data-width-reduction]", `${((1 - compressed / conventional) * 100).toFixed(1)}%`);
$("[data-width-latent-bar]").style.width = `${latent / full * 100}%`;
};
const situ = (x: number, b1: number, b2: number) => b1 * Math.tanh(x / b1) * sigmoid(x) * b2 * Math.tanh(x / b2);
const renderSitu = () => {
const x = value("[data-situ-x]") / 10;
const b1 = value("[data-situ-b1]");
const b2 = value("[data-situ-b2]");
const glu = sigmoid(x) * x;
const swiglu = x * sigmoid(x) * x;
put("[data-situ-x-label]", x.toFixed(1));
put("[data-situ-b1-label]", b1);
put("[data-situ-b2-label]", b2);
put("[data-situ-glu]", glu.toFixed(2));
put("[data-situ-swiglu]", swiglu.toFixed(2));
put("[data-situ-value]", situ(x, b1, b2).toFixed(2));
put("[data-situ-bound]", b1 * b2);
const ranges = Array.from({ length: 121 }, (_, i) => -10 + i * (110 / 120));
const mapX = (v: number) => 36 + (v + 10) / 110 * 664;
const mapY = (v: number) => 224 - Math.max(-20, Math.min(120, v)) / 140 * 190;
const path = (fn: (input: number) => number) => ranges.map((v, i) => `${i ? "L" : "M"}${mapX(v).toFixed(1)},${mapY(fn(v)).toFixed(1)}`).join(" ");
$("[data-curve-glu]").setAttribute("d", path((v) => sigmoid(v) * v));
$("[data-curve-swiglu]").setAttribute("d", path((v) => v * sigmoid(v) * v));
$("[data-curve-situ]").setAttribute("d", path((v) => situ(v, b1, b2)));
};
const scores = [
[.58, .46, .77, .21],
[.61, .91, .71, .90],
[.81, .59, .13, .43],
[.71, .30, .64, .30],
[.83, .23, .52, .70],
[.44, .74, .64, .33],
[.81, .69, .42, .15],
[.75, .84, .56, .64],
];
const topIndex = (row: number[]) => row.reduce((best, score, index) => score > row[best] ? index : best, 0);
const loadsFor = (bias: number[]) => {
const loads = [0, 0, 0, 0];
scores.forEach((row) => {
const adjusted = row.map((score, index) => score + bias[index]);
loads[topIndex(adjusted)] += 1;
});
return loads;
};
const renderLoadBars = (selector: string, loads: number[]) => {
const max = Math.max(1, ...loads);
$(selector).innerHTML = loads.map((load, index) =>
`<div><i style="height:${load / max * 80}px"></i><b>E${index + 1}</b><span>${load}</span></div>`
).join("");
};
const renderQB = () => {
const strength = value("[data-qb-strength]") / 100;
const bins = value("[data-qb-bins]");
const before = loadsFor([0, 0, 0, 0]);
const margins: number[][] = [[], [], [], []];
scores.forEach((row) => {
const sorted = [...row].sort((a, b) => b - a);
const cutoff = sorted[1];
row.forEach((score, expert) => margins[expert].push(score - cutoff));
});
const rawBias = margins.map((values) => {
const ordered = [...values].sort((a, b) => b - a);
return -ordered[2];
});
const mean = rawBias.reduce((sum, n) => sum + n, 0) / rawBias.length;
const centered = rawBias.map((n) => (n - mean) * strength);
const after = loadsFor(centered);
put("[data-qb-strength-label]", strength.toFixed(2));
put("[data-qb-bins-label]", bins);
put("[data-qb-before-copy]", `loads = (${before.join(", ")})`);
put("[data-qb-after-copy]", `loads = (${after.join(", ")})`);
put("[data-qb-bias]", centered.map((n) => formatSigned(n, 2)).join(" · "));
put("[data-qb-gap]", Math.max(...after) - Math.min(...after));
put("[data-qb-error]", `≤ 1/${bins} range`);
renderLoadBars("[data-qb-before]", before);
renderLoadBars("[data-qb-after]", after);
};
const renderRL = () => {
const n = value("[data-rl-n]");
const k = value("[data-rl-k]");
const lambda = value("[data-rl-lambda]") / 100;
const base = value("[data-rl-base]");
const tau = value("[data-rl-tau]") / 10;
const tokens = value("[data-rl-tokens]");
const teacher = value("[data-rl-teacher]") / 100;
const student = value("[data-rl-student]") / 100;
const clip = value("[data-rl-clip]") / 10;
const total = n * k;
const complete = Math.ceil(total * lambda);
const budget = base * tau;
const reward = Math.max(-clip, Math.min(clip, Math.log(teacher / student)));
const pass = tokens <= budget;
put("[data-rl-n-label]", n);
put("[data-rl-k-label]", k);
put("[data-rl-lambda-label]", lambda.toFixed(2));
put("[data-rl-base-label]", `${base}K`);
put("[data-rl-tau-label]", tau.toFixed(1));
put("[data-rl-tokens-label]", `${tokens}K`);
put("[data-rl-teacher-label]", teacher.toFixed(2));
put("[data-rl-student-label]", student.toFixed(2));
put("[data-rl-clip-label]", clip.toFixed(1));
put("[data-rl-budget]", `${budget.toFixed(1).replace(".0", "")}K`);
put("[data-rl-budget-copy]", `${tokens}K ${pass ? "≤" : ">"} ${tau.toFixed(1)} × ${base}K：${pass ? "保留任务 reward" : "reward 改为 −1"}`);
put("[data-rl-complete]", `${complete} complete`);
put("[data-rl-paused]", `${total - complete} paused / resume next iteration`);
put("[data-rl-reward]", formatSigned(reward, 3));
};
const renderCache = () => {
const matched = value("[data-cache-matched]");
const physical = value("[data-cache-physical]");
const hash = Math.min(value("[data-cache-hash]"), physical);
const checkpointEvery = value("[data-cache-checkpoint]");
const mla = Math.floor(matched / hash) * hash;
const checkpointSpan = hash * checkpointEvery;
const kda = Math.floor(mla / checkpointSpan) * checkpointSpan;
const hit = Math.min(mla, kda);
put("[data-cache-matched-label]", matched);
put("[data-cache-physical-label]", physical);
put("[data-cache-hash-label]", hash);
put("[data-cache-checkpoint-label]", checkpointEvery);
put("[data-cache-mla]", mla);
put("[data-cache-kda]", kda);
put("[data-cache-hit]", hit);
put("[data-cache-recompute]", `${matched - hit} tokens`);
const slots = Math.max(4, Math.ceil(Math.min(physical * 2, Math.max(matched, physical)) / hash));
$("[data-cache-strip]").innerHTML = Array.from({ length: slots }, (_, index) => {
const end = (index + 1) * hash;
const matchedBlock = end <= mla;
const checkpoint = end % checkpointSpan === 0;
const joint = end <= hit;
return `<div class="${joint ? "joint" : matchedBlock ? "mla" : ""}">
<span>${index + 1}</span><i></i><b>${end}</b>${checkpoint ? "<em title=\"KDA checkpoint\">●</em>" : "<em>○</em>"}
</div>`;
}).join("");
};
const renders = [renderMemory, renderDecay, renderDepth, renderWidth, renderSitu, renderQB, renderRL, renderCache];
$$<HTMLInputElement | HTMLSelectElement>("input, select").forEach((control) => control.addEventListener("input", () => renders.forEach((render) => render())));
renders.forEach((render) => render());
});
</script>
<style>
.k3-lab {
max-width: 1080px;
margin: 42px 0;
border: 1px solid var(--ink);
background: var(--paper-raised);
}
.lab-head, .panel-intro {
display: grid;
grid-template-columns: minmax(0, 1.1fr) minmax(260px, .9fr);
gap: 30px;
padding: 28px;
border-bottom: 1px solid var(--line);
}
.lab-head { color: var(--paper); background: var(--ink); }
.lab-head p, .panel-intro p { margin: 0; font-size: .72rem; line-height: 1.75; }
.lab-head > div > p, .panel-intro span {
color: var(--copper);
font: .6rem/1.2 var(--mono);
letter-spacing: .12em;
}
.lab-head h3, .panel-intro h4 { margin: 10px 0 0; font-size: 1.25rem; line-height: 1.35; }
.lab-head h3 { color: var(--paper); }
.lab-head > p { color: rgba(255,255,255,.65); }
.lab-tabs {
display: grid;
grid-template-columns: repeat(4, 1fr);
border-bottom: 1px solid var(--line);
}
.lab-tabs button {
min-height: 78px;
padding: 13px 15px;
border: 0;
border-right: 1px solid var(--line);
border-bottom: 1px solid var(--line);
color: var(--ink);
background: transparent;
text-align: left;
cursor: pointer;
}
.lab-tabs button:nth-child(4n) { border-right: 0; }
.lab-tabs button:nth-child(n+5) { border-bottom: 0; }
.lab-tabs button[aria-selected="true"] { color: var(--paper); background: var(--copper); }
.lab-tabs span, .lab-tabs b, .lab-tabs small { display: block; }
.lab-tabs span { opacity: .7; font: .55rem/1 var(--mono); }
.lab-tabs b { margin-top: 8px; font: 700 .67rem/1 var(--mono); letter-spacing: .06em; }
.lab-tabs small { margin-top: 5px; opacity: .72; font-size: .61rem; }
.lab-panel { padding-bottom: 26px; }
.panel-intro { padding: 24px 28px; }
.panel-intro > p { color: var(--muted); }
.controls {
display: grid;
gap: 0;
margin: 0 28px 26px;
border: 1px solid var(--line);
}
.controls.two { grid-template-columns: repeat(2, 1fr); }
.controls.three { grid-template-columns: repeat(3, 1fr); }
.controls.four { grid-template-columns: repeat(4, 1fr); }
.controls.five { grid-template-columns: repeat(5, 1fr); }
.controls label {
min-width: 0;
padding: 14px;
border-right: 1px solid var(--line);
color: var(--muted);
font: .62rem/1.35 var(--mono);
}
.controls label:last-child { border-right: 0; }
.controls span { display: flex; justify-content: space-between; gap: 8px; min-height: 28px; }
.controls output { color: var(--copper); }
.controls input, .controls select { width: 100%; margin-top: 11px; accent-color: var(--copper); }
.controls input[disabled] { padding: 5px; border: 1px solid var(--line); color: var(--ink); background: var(--paper); font: .65rem var(--mono); }
.memory-stage, .decay-plot, .payload-compare, .qb-stage, .rl-flow {
display: grid;
gap: 14px;
margin: 0 28px 26px;
}
.memory-stage, .payload-compare { grid-template-columns: repeat(2, 1fr); }
.decay-plot { grid-template-columns: repeat(3, 1fr); }
.qb-stage, .rl-flow { grid-template-columns: 1fr 24px 1fr 24px 1fr; align-items: center; }
.memory-stage article, .decay-plot article, .payload-compare article, .qb-stage article, .rl-flow article {
min-width: 0;
padding: 20px;
border: 1px solid var(--line);
background: var(--paper);
}
.accent { border-color: rgba(173,100,69,.55) !important; background: var(--copper-pale) !important; }
.memory-stage span, .decay-plot span, .payload-compare span, .qb-stage span, .rl-flow span {
color: var(--muted);
font: .58rem/1 var(--mono);
letter-spacing: .08em;
}
.memory-stage code { display: block; margin: 13px 0; font-size: .72rem; }
.memory-stage b, .decay-plot b, .payload-compare b, .rl-flow b { display: block; margin: 14px 0 7px; font: 700 1.05rem/1 var(--mono); }
.memory-stage p, .decay-plot p, .payload-compare p, .qb-stage p, .rl-flow p { margin: 7px 0 0; color: var(--muted); font-size: .68rem; line-height: 1.55; }
.trace { display: flex; height: 68px; gap: 3px; align-items: flex-end; border-bottom: 1px solid var(--line); }
.trace i { flex: 1; min-width: 2px; background: var(--sage); }
.trace i.accent { background: var(--copper) !important; }
.metrics {
display: grid;
margin: 0 28px 24px;
border: 1px solid var(--line);
}
.metrics.three { grid-template-columns: repeat(3, 1fr); }
.metrics.four { grid-template-columns: repeat(4, 1fr); }
.metrics article { min-width: 0; min-height: 112px; padding: 17px; border-right: 1px solid var(--line); }
.metrics article:last-child { border-right: 0; }
.metrics span { color: var(--muted); font: .55rem/1 var(--mono); letter-spacing: .08em; }
.metrics b { display: block; margin-top: 18px; overflow-wrap: anywhere; color: var(--copper); font: 700 .92rem/1.2 var(--mono); }
.metrics p { margin: 8px 0 0; color: var(--muted); font-size: .62rem; line-height: 1.5; }
.metrics .dark { color: var(--paper); background: var(--ink); }
.metrics .dark span, .metrics .dark p { color: rgba(255,255,255,.55); }
.metrics .dark b { color: var(--paper); }
.boundary {
margin: 0 28px;
padding: 15px 17px;
border-left: 3px solid var(--sage);
background: var(--sage-pale);
}
.boundary b { font: 700 .62rem/1 var(--mono); letter-spacing: .06em; }
.boundary p { margin: 7px 0 0; color: var(--muted); font-size: .67rem; line-height: 1.55; }
.decay-plot article > div, .payload-compare article > div { height: 7px; margin-top: 18px; overflow: hidden; background: var(--line); }
.decay-plot i, .payload-compare i { display: block; width: 100%; height: 100%; background: var(--sage); }
.decay-plot article.accent i, .payload-compare article.accent i { background: var(--copper); }
.verdict {
display: grid;
grid-template-columns: .6fr .8fr 1.6fr;
gap: 20px;
align-items: center;
margin: 0 28px 24px;
padding: 18px;
color: var(--paper);
background: var(--sage);
}
.verdict.danger { background: #8d3f35; }
.verdict span { font: .58rem var(--mono); opacity: .7; }
.verdict b { font: 700 .9rem var(--mono); }
.verdict p { margin: 0; font-size: .67rem; line-height: 1.55; }
.depth-track {
display: flex;
gap: 4px;
min-height: 88px;
margin: 0 28px 26px;
align-items: stretch;
}
.depth-track div { display: flex; min-width: 24px; padding: 10px 4px; flex-direction: column; justify-content: space-between; border: 1px solid var(--line); background: var(--sage-pale); text-align: center; }
.depth-track span { color: var(--muted); font: .51rem var(--mono); }
.depth-track b { color: var(--sage); font: 700 .75rem var(--mono); }
.curve-stage { position: relative; margin: 0 28px 24px; border: 1px solid var(--line); background: var(--paper); }
.curve-stage svg { display: block; width: 100%; }
.curve-stage line { stroke: var(--line); stroke-width: 1; }
.curve { fill: none; stroke-width: 2.4; }
.curve.glu { stroke: var(--sage); }
.curve.swiglu { stroke: var(--ink); }
.curve.situ { stroke: var(--copper); stroke-width: 3.2; }
.curve-legend { position: absolute; top: 13px; right: 14px; display: flex; gap: 12px; padding: 7px 9px; background: rgba(255,255,255,.84); }
.curve-legend span { font: .55rem var(--mono); }
.curve-legend .glu { color: var(--sage); }
.curve-legend .situ { color: var(--copper); }
.qb-stage > i, .rl-flow > i { color: var(--muted-light); font-style: normal; text-align: center; }
.quantile-box b { display: block; margin: 14px 0; font: .7rem var(--mono); }
.load-bars { display: flex; height: 104px; gap: 10px; align-items: flex-end; margin-top: 13px; }
.load-bars div { display: flex; flex: 1; min-width: 0; height: 100%; flex-direction: column; align-items: center; justify-content: flex-end; }
.load-bars i { width: 70%; min-height: 3px; background: var(--sage); }
.accent .load-bars i { background: var(--copper); }
.load-bars b, .load-bars span { margin-top: 5px; font: .55rem var(--mono); }
.cache-strip {
display: flex;
gap: 3px;
margin: 0 28px 26px;
overflow-x: auto;
padding-bottom: 8px;
}
.cache-strip div { position: relative; flex: 1 0 48px; min-width: 48px; padding: 9px 4px 7px; border: 1px solid var(--line); text-align: center; background: var(--paper); }
.cache-strip div.mla { background: #e8efed; }
.cache-strip div.joint { border-color: var(--copper); background: var(--copper-pale); }
.cache-strip span, .cache-strip b { display: block; color: var(--muted); font: .49rem var(--mono); }
.cache-strip i { display: block; height: 20px; margin: 6px 0; background: var(--line); }
.cache-strip .mla i { background: var(--sage); }
.cache-strip .joint i { background: var(--copper); }
.cache-strip em { position: absolute; right: 3px; bottom: 2px; color: var(--copper); font-style: normal; font-size: .55rem; }
@media (max-width: 900px) {
.lab-head, .panel-intro { grid-template-columns: 1fr; }
.controls.five { grid-template-columns: repeat(3, 1fr); }
.metrics.four { grid-template-columns: repeat(2, 1fr); }
.qb-stage, .rl-flow { grid-template-columns: 1fr; }
.qb-stage > i, .rl-flow > i { transform: rotate(90deg); }
}
@media (max-width: 640px) {
.lab-tabs { grid-template-columns: repeat(2, 1fr); }
.lab-tabs button:nth-child(2n) { border-right: 0; }
.lab-tabs button:nth-child(n+5) { border-bottom: 1px solid var(--line); }
.lab-tabs button:nth-child(n+7) { border-bottom: 0; }
.controls.two, .controls.three, .controls.four, .controls.five,
.memory-stage, .decay-plot, .payload-compare, .metrics.three, .metrics.four {
grid-template-columns: 1fr;
}
.controls label, .metrics article { border-right: 0; border-bottom: 1px solid var(--line); }
.controls label:last-child, .metrics article:last-child { border-bottom: 0; }
.verdict { grid-template-columns: 1fr; }
}
</style>
src/data/k3.ts
+173
View File
@@ -0,0 +1,173 @@
export const k3Ledgers = [
["Q01", "参数角色", "2.78T / 104.2B 各表示什么？", "total 是总容量，active 是一条 Token 路径经过的参数规模；二者都不等于端到端 FLOPs。"],
["Q02", "效率口径", "约 2.5× 究竟是什么？", "它是架构、数据与 recipe 共同形成的相对 K2 scaling efficiency，不是推理速度或 KDA 单项收益。"],
["Q03", "混合节奏", "3 KDA + 1 MLA 如何落到 93 层？", "Table 1 给出 69 KDA + 24 MLA，最后一层也是 MLA；另有一层 dense。"],
["Q04", "序列状态", "KDA 保存了什么？", "它维护固定形状 recurrent state，而不是所有历史 Token 的两两配对。"],
["Q05", "纠错写入", "Delta Rule 为什么不是简单累加？", "先减掉 state 对当前 key 的已有预测，再写真实 value 与预测的差。"],
["Q06", "数值下界", "gmin = −5 在解决什么？", "它限制 chunk 内累计衰减倒数，令关键 tile 可落在 BF16 Tensor Core 范围。"],
["Q07", "并行算法", "递归状态如何训练并行？", "chunk 间传状态，chunk 内改写成并行矩阵乘；decode 仍逐步更新固定状态。"],
["Q08", "读出门", "KDA / MLA 的 output gate 是 router 吗？", "不是；它按输入控制 attention 读出通道，MoE router 则选择专家。"],
["Q09", "位置来源", "NoPE 是否意味着没有顺序？", "没有显式位置 embedding；KDA 的门控与衰减隐式传递顺序，MLA 负责全局内容匹配。"],
["Q10", "舍入合同", "为什么 attention output 保留 FP32？", "报告用它修正 FlashAttention 有偏舍入，并为片上 tile buffer 重新排布。"],
["Q11", "深度路由", "AttnRes 怎样改变 residual？", "每层用 pseudo-query 选择早层来源；Block 版用块内累加换取更低内存与跨 stage 通信。"],
["Q12", "主模型边界", "12 层一块还是 2 层一块？", "K3 主模型为 12；§7 芯片 nano case 的 2 不能写回主规格。"],
["Q13", "路由宽度", "LatentMoE 为什么先压到 3584？", "shared path 保留 7168，routed path 在较窄空间执行，以降低多专家激活的 payload 与权重流。"],
["Q14", "专家角色", "896、16、2 怎样读？", "896 routed 中每 Token 选 16，另有 2 shared；稀疏不等于消费级硬件轻松运行。"],
["Q15", "有界激活", "SiTU-GLU 在稳定什么？", "它限制 routed path 大正输入的幅值，同时近似 SwiGLU 在原点附近的形状。"],
["Q16", "路由分位", "Quantile Balancing 怎样更新 bias？", "用 Top-(k+1) score cutoff 的全局分位量做下一步更新；最终 bias 冻结用于推理。"],
["Q17", "平衡层次", "QB 与 MoonEP 是同一件事吗？", "QB 调模型路由偏置；MoonEP 调系统执行与通信形状；二者互补但不能合并。"],
["Q18", "原生视觉", "视觉塔是否后接到预训练 LLM？", "不是。MoonViT-V2 从头训练，视觉与文本从训练开始就在同一 NTP objective 联合优化。"],
["Q19", "视觉流", "图像和视频怎样进入主干？", "401M / 27 层编码器共享图像视频参数，经时空分解、pooling、pixel shuffle 与 projector 进入主干。"],
["Q20", "优化器", "Per-Head Muon 为什么按 head 分组？", "多头投影按相对独立的 head 组织矩阵更新；它只是完整训练 recipe 的一部分。"],
["Q21", "数据披露", "我们知道和不知道哪些语料事实？", "知道四类文本、视觉种类与清洗/重写流程；不知道完整来源、精确配比和总训练 Token。"],
["Q22", "长度课程", "1M 是怎样训练出来的？", "pretrain 8K→64K，cooldown 256K→1M，并配长数据清洗、上采样和跨全局证据的合成任务。"],
["Q23", "SFT 冷启动", "Agent 轨迹从哪里来？", "前代 Kimi 专家合成，经多阶段验证与 HITL 标注，再用 XTML 序列化；SFT 起即做 QAT。"],
["Q24", "九位老师", "3 × 3 专家是否在线投票？", "不是。它们是三领域 × 三 effort 的训练策略，最终经 MOPD 整合进一个学生。"],
["Q25", "思考预算", "low / high / max 如何定义？", "每题从 cold-start 估计 b₀(x)，按 domain 退火 τ；超过 τb₀ 的轨迹 reward 改为 −1。"],
["Q26", "长尾轨迹", "Partial rollout 暂停了什么？", "N×K 轨迹中 λ 比例完成就更新，未完轨迹下轮优先恢复；代价是 stale/off-policy。"],
["Q27", "蒸馏信号", "MOPD 为什么是 on-policy？", "学生生成自己会访问的 prefix，匹配的 teacher 在该 prefix 上给 clipped token log-ratio reward。"],
["Q28", "部署约束", "量化与 speculative draft 何时进入？", "SFT/RL 全程 experts MXFP4/activations MXFP8 QAT；MTP 后调为 EAGLE-3 draft 并优化 LK loss。"],
["Q29", "Harness", "Agent 能力属于模型还是脚手架？", "系统结果来自模型与 tool、prompt、context、skills、memory、subagent、verifier 的组合。"],
["Q30", "状态寿命", "训练、RL、服务分别搬运什么？", "参数/优化器、KDA/MLA 序列状态、KV/轨迹、sandbox 与 prefix 的寿命完全不同。"],
["Q31", "混合缓存", "KDA state 与 MLA KV 怎样共同命中？", "统一页池；细 hash boundary 与粗 physical block 解耦，命中点必须两类 cache 同时有效。"],
["Q32", "评测协议", "一个分数至少需要哪些脚注？", "effort、tool、harness、fallback/guard、日期；报告也明确总体仍落后最强闭源模型。"],
] as const;
export const k3ReportMap = [
["§1", "Introduction", "四条扩展轴与总体结论", "Figure 1"],
["§2", "Architecture", "KDA、AttnRes、Stable LatentMoE、MoonViT、Muon", "Figures 2–6 · Table 1"],
["§3", "Pre-Training", "数据、scaling law、联合训练、1M curriculum", "Figure 7"],
["§4", "Post-Training", "SFT、9 experts、partial rollout、MOPD、QAT、环境", "Figures 8–10"],
["§5", "Infrastructure", "FlashKDA/KCP、MoonEP、RL 系统、AgentENV、serving", "Figures 11–12"],
["§6", "Evaluations", "四能力轴、第三方结果、成本与协议", "Figure 13 · Tables 2–5"],
["§7", "Case Studies", "kernel、MiniTriton、chip、research、knowledge、video", "Figures 14–15"],
["§8", "Conclusion", "开放权重与剩余差距", "—"],
["App.", "A–F", "KDA/AttnRes 细节、评测补充、XTML", "Figure 16"],
] as const;
export const k3FigureAtlas = [
["F01", "Figure 1", "主结果", "四能力轴必须与总体仍落后 Claude Fable 5 / GPT-5.6 Sol 的结论同屏。"],
["F02", "Figure 2", "完整架构", "把视觉入口、3:1 hybrid stack、Block AttnRes 与 Stable LatentMoE 分成不同信息流。"],
["F03", "Figure 3", "Bounded decay", "展示下界、累计倒数与 BF16 风险，不承诺任意配置绝对稳定。"],
["F04", "Figure 4", "SiTU-GLU", "对比 GLU / SwiGLU / SiTU 的函数形状，不从曲线直接推训练 loss。"],
["F05", "Figure 5", "Quantile Balancing", "Top-(k+1) cutoff → global quantile → expert bias → next step。"],
["F06", "Figure 6", "原生视觉", "只在作者消融配置内说明 from-scratch MoonViT-V2 梯度更平稳。"],
["F07", "Figure 7", "Scaling law", "约 2.5× 是相对 K2 的整体 scaling efficiency，不是 inference speedup。"],
["F08", "Figure 8", "RL scaling", "RL FLOPs、平均 steps 与能力共同增长；相关性不自动成为跨域因果律。"],
["F09", "Figure 9", "任务合成", "知识图谱 → 材料检索 → knowledge/coding/vision task → verification。"],
["F10", "Figure 10", "AET", "初始状态、动作、独立 verifier 与 curriculum，奖励落在环境结果。"],
["F11", "Figure 11", "训练系统", "画状态的放置与移动，不补写报告未公开的总集群规模。"],
["F12", "Figure 12", "Prefix cache", "6144 physical / 512 hash / sparse KDA checkpoint 是报告示例。"],
["F13", "Figure 13", "分数与成本", "score 与 output token / cost 同图，并保留比较协议。"],
["F14", "Figure 14", "Kernel agent", "283.6→114.4 ms 等数字限定在作者 24h/task 个案。"],
["F15", "Figure 15", "MiniTriton", "L20 实测点对 roofline；不推广到其他硬件和工作负载。"],
["F16", "Figure 16", "XTML", "global / one-shot / input options、channel 和 message boundary 分层。"],
["T01", "Table 1", "K2 → K3", "精确逐字段规格，hero 的 2.8T / 104B 只作易读四舍五入。"],
["T02", "Table 2", "Reasoning / Coding", "把 effort、harness、tool 与 fallback 变成表头的一部分。"],
["T03", "Table 3", "Agentic", "分数是 model × harness × environment × verifier 的系统结果。"],
["T04", "Table 4", "Vision", "有/无 Python tool 必须分列。"],
["T05", "Table 5", "第三方 / 成本", "价格点只描述当时服务条件，不作为架构因果证据。"],
] as const;
export const k3PaperChain = [
["001", "2014", "Neural Machine Translation by Jointly Learning to Align and Translate", "https://arxiv.org/abs/1409.0473", "内容寻址注意力前史", "FOUNDATION"],
["002", "2015", "Deep Residual Learning for Image Recognition", "https://arxiv.org/abs/1512.03385", "标准 residual 起点", "DEPTH"],
["003", "2017", "Attention Is All You Need", "https://arxiv.org/abs/1706.03762", "全局 self-attention 坐标", "FOUNDATION"],
["004", "2017", "Language Modeling with Gated Convolutional Networks", "https://proceedings.mlr.press/v70/dauphin17a.html", "GLU 前史", "NUMERICS"],
["005", "2018", "Parallelizing Linear Recurrent Neural Nets Over Sequence Length", "https://openreview.net/forum?id=HyUNwulC-", "递归的并行扫描", "STATE"],
["006", "2018", "Quantization and Training of Neural Networks", "https://arxiv.org/abs/1712.05877", "QAT 前史", "DEPLOY"],
["007", "2018", "Online normalizer calculation for softmax", "https://arxiv.org/abs/1805.02867", "online-softmax merge", "SYSTEM"],
["008", "2019", "RMSNorm", "https://arxiv.org/abs/1910.07467", "尺度控制", "NUMERICS"],
["009", "2019", "GPipe", "https://arxiv.org/abs/1811.06965", "pipeline schedule 前史", "SYSTEM"],
["010", "2019", "Triton", "https://dl.acm.org/doi/10.1145/3315508.3329973", "tile compiler 前史", "SYSTEM"],
["011", "2020", "Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention", "https://proceedings.mlr.press/v119/katharopoulos20a.html", "线性状态视角", "STATE"],
["012", "2020", "Scaling Laws for Neural Language Models", "https://arxiv.org/abs/2001.08361", "经验缩放坐标", "SCALING"],
["013", "2020", "GLU Variants Improve Transformer", "https://arxiv.org/abs/2002.05202", "SwiGLU 坐标", "NUMERICS"],
["014", "2020", "GShard", "https://arxiv.org/abs/2006.16668", "大规模 expert parallel", "MOE"],
["015", "2020", "ZeRO", "https://arxiv.org/abs/1910.02054", "训练状态分片", "SYSTEM"],
["016", "2020", "Firecracker", "https://www.usenix.org/conference/nsdi20/presentation/agache", "AgentENV 隔离前史", "AGENT"],
["017", "2021", "Linear Transformers Are Secretly Fast Weight Programmers", "https://proceedings.mlr.press/v139/schlag21a.html", "Delta memory 直觉", "STATE"],
["018", "2021", "Switch Transformers", "https://arxiv.org/abs/2101.03961", "稀疏容量坐标", "MOE"],
["019", "2021", "BASE Layers", "https://proceedings.mlr.press/v139/lewis21a.html", "专家负载分配", "MOE"],
["020", "2021", "MLIR", "https://ieeexplore.ieee.org/document/9370308", "MiniTriton 编译基础", "SYSTEM"],
["021", "2022", "Training Compute-Optimal Large Language Models", "https://arxiv.org/abs/2203.15556", "参数/数据预算", "SCALING"],
["022", "2022", "FlashAttention", "https://arxiv.org/abs/2205.14135", "IO-aware exact attention", "SYSTEM"],
["023", "2023", "Ring Attention", "https://arxiv.org/abs/2310.01889", "长序列块式并行", "CONTEXT"],
["024", "2023", "DeepSpeed Ulysses", "https://arxiv.org/abs/2309.14509", "sequence parallel 对照", "CONTEXT"],
["025", "2023", "YaRN", "https://arxiv.org/abs/2309.00071", "RoPE 外推对照", "CONTEXT"],
["026", "2023", "Microscaling Data Formats for Deep Learning", "https://arxiv.org/abs/2310.10537", "MX formats", "DEPLOY"],
["027", "2024", "DeepSeekMoE", "https://arxiv.org/abs/2401.06066", "shared + fine-grained experts", "DEEPSEEK"],
["028", "2024", "DeepSeek-V2", "https://arxiv.org/abs/2405.04434", "MLA 与 MoE 直接祖先", "DEEPSEEK"],
["029", "2024", "DeepSeek-V3", "https://arxiv.org/abs/2412.19437", "aux-loss-free、FP8、MTP 对照", "DEEPSEEK"],
["030", "2024", "Griffin", "https://arxiv.org/abs/2402.19427", "recurrent + attention hybrid", "STATE"],
["031", "2024", "Mamba-2 / Transformers are SSMs", "https://arxiv.org/abs/2405.21060", "状态空间二元性", "STATE"],
["032", "2024", "HGRN2", "https://arxiv.org/abs/2404.07904", "gated recurrence 与 state expansion", "STATE"],
["033", "2024", "Linear Attention Sequence Parallelism", "https://arxiv.org/abs/2404.02882", "linear attention CP", "CONTEXT"],
["034", "2024", "Gated Linear Attention Transformers", "https://proceedings.mlr.press/v235/yang24ab.html", "硬件友好 GLA", "STATE"],
["035", "2024", "Parallelizing Linear Transformers with the Delta Rule", "https://arxiv.org/abs/2406.06484", "chunkwise delta rule", "STATE"],
["036", "2024", "Mooncake", "https://arxiv.org/abs/2407.00079", "KV-centric serving", "SYSTEM"],
["037", "2025", "Kimi k1.5", "https://arxiv.org/abs/2501.12599", "长 CoT 与 RL 前代", "KIMI"],
["038", "2025", "Gated Delta Networks", "https://openreview.net/forum?id=r8H7xhYPwz", "KDA 机制近邻", "STATE"],
["039", "2025", "Kimi-VL", "https://arxiv.org/abs/2504.07491", "Kimi 视觉前代", "KIMI"],
["040", "2025", "Kimi K2", "https://arxiv.org/abs/2507.20534", "主干、Muon、Agent 前代", "KIMI"],
["041", "2025", "Kimi Linear", "https://arxiv.org/abs/2510.26692", "KDA 与 hybrid 直接前身", "KIMI"],
["042", "2025", "Muon", "https://kellerjordan.github.io/posts/muon/", "矩阵正交化更新", "NUMERICS"],
["043", "2025", "Muon is Scalable for LLM Training", "https://arxiv.org/abs/2502.16982", "Muon 扩展证据", "NUMERICS"],
["044", "2025", "LASP-2", "https://arxiv.org/abs/2502.07563", "hybrid linear attention SP", "CONTEXT"],
["045", "2025", "EAGLE-3", "https://arxiv.org/abs/2503.01840", "draft model 直接来源", "DEPLOY"],
["046", "2025", "Gated Attention for Large Language Models", "https://arxiv.org/abs/2505.06708", "attention output gating", "STATE"],
["047", "2025", "Optimus", "https://www.usenix.org/conference/atc25/presentation/feng", "多模态训练 bubble", "SYSTEM"],
["048", "2025", "On-policy distillation", "https://thinkingmachines.ai/blog/on-policy-distillation/", "学生分布上的蒸馏", "RL"],
["049", "2025", "DeepEP", "https://github.com/deepseek-ai/DeepEP", "专家并行对照实现", "DEEPSEEK"],
["050", "2025", "Flash Linear Attention", "https://github.com/fla-org/flash-linear-attention", "线性注意力实现生态", "SYSTEM"],
["051", "2025", "Video-MME", "https://arxiv.org/abs/2405.21075", "视频评测", "EVAL"],
["052", "2025", "MMMU-Pro", "https://arxiv.org/abs/2409.02813", "稳健多学科视觉评测", "EVAL"],
["053", "2025", "CharXiv", "https://arxiv.org/abs/2406.18521", "图表理解评测", "EVAL"],
["054", "2025", "Math-Vision", "https://arxiv.org/abs/2402.14804", "视觉数学评测", "EVAL"],
["055", "2025", "OmniDocBench", "https://arxiv.org/abs/2412.07626", "文档解析评测", "EVAL"],
["056", "2025", "Humanity's Last Exam", "https://arxiv.org/abs/2501.14249", "高难知识推理评测", "EVAL"],
["057", "2025", "BrowseComp", "https://arxiv.org/abs/2504.12516", "浏览与 context management", "EVAL"],
["058", "2025", "SciCode", "https://arxiv.org/abs/2407.13168", "科研编码评测", "EVAL"],
["059", "2025", "Tool Decathlon", "https://arxiv.org/abs/2510.25726", "长程工具使用评测", "EVAL"],
["060", "2025", "DeepSearchQA", "https://storage.googleapis.com/deepmind-media/DeepSearchQA/DeepSearchQA_benchmark_paper.pdf", "深度研究覆盖度", "EVAL"],
["061", "2025", "OSWorld-Verified", "https://xlang.ai/blog/osworld-verified", "电脑使用验证", "EVAL"],
["062", "2025", "Terminal-Bench", "https://arxiv.org/abs/2601.11868", "终端 Agent 评测", "EVAL"],
["063", "2025", "GDPval", "https://arxiv.org/abs/2510.04374", "经济价值任务", "EVAL"],
["064", "2025", "ZeroBench", "https://arxiv.org/abs/2502.09696", "高难视觉基准", "EVAL"],
["065", "2025", "SonicMoE", "https://arxiv.org/abs/2512.14080", "IO / tile-aware MoE", "SYSTEM"],
["066", "2025", "TileLang", "https://arxiv.org/abs/2504.17577", "tile programming 对照", "SYSTEM"],
["067", "2025", "ThunderKittens", "https://openreview.net/forum?id=0fJfVOSUra", "GPU kernel DSL 对照", "SYSTEM"],
["068", "2026", "Kimi K2.5", "https://arxiv.org/abs/2602.02276", "视觉 Agent 与 RL 直接前代", "KIMI"],
["069", "2026", "LatentMoE", "https://arxiv.org/abs/2601.18089", "routed latent width", "MOE"],
["070", "2026", "Attention Residuals", "https://arxiv.org/abs/2603.15031", "跨深度选择", "DEPTH"],
["071", "2026", "Kimi K3", "https://arxiv.org/abs/2607.24653", "本课程锚点报告", "KIMI"],
["072", "2026", "Kimi K3 official repository", "https://github.com/MoonshotAI/Kimi-K3", "权重、报告与开放边界", "KIMI"],
["073", "2026", "FlashKDA", "https://github.com/MoonshotAI/FlashKDA", "KDA 官方 kernel", "SYSTEM"],
["074", "2026", "AgentENV", "https://github.com/kvcache-ai/AgentENV", "可恢复 microVM 环境", "AGENT"],
["075", "2026", "ReplaySSM", "https://tridao.me/blog/2026/replayssm/", "缓存输入而非每步 state", "DEPLOY"],
["076", "2026", "LK Losses", "https://arxiv.org/abs/2602.23881", "直接优化 speculative acceptance", "DEPLOY"],
["077", "2026", "PowLU", "https://arxiv.org/abs/2605.25704", "有界/稳定激活对照", "NUMERICS"],
["078", "2026", "Why Low-Precision Transformer Training Fails", "https://arxiv.org/abs/2510.04212", "FlashAttention 舍入风险", "NUMERICS"],
["079", "2026", "Expert Threshold Routing", "https://arxiv.org/abs/2603.11535", "动态专家与均衡对照", "MOE"],
["080", "2026", "BIP Expert Load Balancing", "https://arxiv.org/abs/2502.15451", "专家分配对照", "MOE"],
["081", "2026", "UltraEP", "https://arxiv.org/abs/2606.04101", "rack-scale expert parallel", "SYSTEM"],
["082", "2026", "Megatron Core MoE", "https://arxiv.org/abs/2603.07685", "MoE 系统复现坐标", "SYSTEM"],
["083", "2026", "Warp Decode", "https://cursor.com/blog/warp-decode", "token-centric MoE decode", "SYSTEM"],
["084", "2026", "DeepSeek-V4", "https://arxiv.org/abs/2606.19348", "百万上下文同期对照", "DEEPSEEK"],
["085", "2025", "DeepSeek-R1", "https://arxiv.org/abs/2501.12948", "reasoning RL 重点对照", "DEEPSEEK"],
["086", "2026", "MCP-Atlas", "https://arxiv.org/abs/2602.00933", "MCP 工具能力评测", "EVAL"],
["087", "2026", "MCPMark", "https://arxiv.org/abs/2509.24002", "真实 MCP 压力测试", "EVAL"],
["088", "2026", "ResearchRubrics", "https://openreview.net/forum?id=ErnvfmSX0P", "深度研究 rubric", "EVAL"],
["089", "2026", "AutomationBench", "https://arxiv.org/abs/2604.18934", "自动化 Agent 评测", "EVAL"],
["090", "2026", "SaaS-Bench", "https://arxiv.org/abs/2605.15777", "专业 SaaS workflow", "EVAL"],
["091", "2026", "Agents' Last Exam", "https://arxiv.org/abs/2606.05405", "通用 Agent 压力测试", "EVAL"],
["092", "2026", "APEX-Agents", "https://arxiv.org/abs/2601.14242", "跨域 Agent 评测", "EVAL"],
["093", "2026", "OSWorld 2.0", "https://arxiv.org/abs/2606.29537", "长程电脑使用", "EVAL"],
["094", "2026", "OfficeQA Pro", "https://arxiv.org/abs/2603.08655", "企业知识工作", "EVAL"],
["095", "2026", "SpreadsheetBench 2", "https://arxiv.org/abs/2606.29955", "端到端表格工作", "EVAL"],
["096", "2026", "BabyVision", "https://arxiv.org/abs/2601.06521", "超越语言先验的视觉推理", "EVAL"],
["097", "2026", "MMVU", "https://openaccess.thecvf.com/content/CVPR2025/html/Zhao_MMVU_Measuring_Expert-Level_Multi-Discipline_Video_Understanding_CVPR_2025_paper.html", "专家级视频理解", "EVAL"],
["098", "2026", "WorldVQA", "https://arxiv.org/abs/2602.02537", "原子视觉世界知识", "EVAL"],
["099", "2026", "PerceptionBench", "https://www.kimi.com/blog/perception-bench", "Kimi 原子视觉感知", "KIMI"],
["100", "2026", "Preliminary Assessment of Kimi K3's Cyber Capabilities", "https://www.aisi.gov.uk/blog/preliminary-assessment-of-kimi-k3s-cyber-capabilities", "独立安全评估边界", "EVAL"],
] as const;
src/pages/index.astro
+28 -2
View File
@@ -40,8 +40,8 @@ const paths = [
label: "K3 反向拆解",
title: "先看全貌，再沿组件回到每条技术祖先",
text: "适合已经用过大模型、想迅速读懂 K3 报告的人。每个组件都能跳回其历史专题。",
steps: ["K3 三维信息流", "KDA 与 MLA", "Stable LatentMoE", "1M Agentic RL 与系统"],
time: "6–10 小时",
steps: ["32 张问题账", "KDA / MLA / AttnRes / LatentMoE", "预训练与 MOPD", "1M RL / 服务 / 评测"],
time: "10–16 小时",
target: "能逐节解释 K3 技术报告",
},
{
@@ -126,6 +126,22 @@ const paths = [
<section class="section compact release-section" id="new-chapters">
<div class="release-grid">
<a class="release-card k3-release" href="/k3/">
<div>
<p class="eyebrow"><span>NEW / K3 ROUND 02</span> REPORT · FORMULA · SYSTEM · EVIDENCE</p>
<h2>47 页不再压成摘要：把 K3 的每个因果环节重新展开</h2>
<p>
用三十二张问题账逐节读完 KDA、Gated MLA、AttnRes、Stable LatentMoE、原生视觉、
预训练、九专家 MOPD、Agent 环境、FlashKDA / MoonEP、混合 prefix cache、评测与案例边界。
</p>
</div>
<dl>
<div><dt>REPORT</dt><dd>16 Figures · 5 Tables</dd></div>
<div><dt>NODES</dt><dd>100 个一手 / 官方节点</dd></div>
<div><dt>LAB</dt><dd>Delta · Decay · AttnRes · MoE · QB · RL · Cache</dd></div>
</dl>
<span class="release-arrow" aria-hidden="true">从报告目录进入完整因果链 →</span>
</a>
<a class="release-card deepseek-release" href="/deepseek/">
<div>
<p class="eyebrow"><span>NEW / DEEPSEEK ROUND 02</span> CAPACITY · STATE · SYSTEM · REASONING</p>
@@ -615,6 +631,7 @@ const paths = [
transition: transform 180ms ease, border-color 180ms ease;
}
.k3-release,
.deepseek-release,
.representation-release,
.inference-release,
@@ -639,6 +656,14 @@ const paths = [
var(--paper-raised);
}
.k3-release {
background:
radial-gradient(circle at 82% 16%, rgba(159, 91, 52, .28), transparent 31%),
radial-gradient(circle at 58% 74%, rgba(76, 118, 112, .22), transparent 30%),
repeating-linear-gradient(135deg, transparent 0 62px, rgba(159, 91, 52, .045) 62px 63px),
var(--paper-raised);
}
.representation-release {
background:
radial-gradient(circle at 82% 18%, rgba(159, 91, 52, 0.22), transparent 31%),
@@ -785,6 +810,7 @@ const paths = [
padding-bottom: 76px;
}
.k3-release,
.deepseek-release,
.representation-release,
.inference-release,
src/pages/k3/index.astro
+1054 -536
View File
File diff suppressed because it is too large Load Diff
src/pages/progress/index.astro
+8 -3
View File
@@ -9,7 +9,7 @@ const researching = chapters.filter((chapter) => ["researching", "drafting"].inc
const workstreams = [
{ label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" },
{ label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" },
{ label: "Kimi K3 深读", value: 66, next: "扩写 pre-training / infra 逐图笔记" },
{ label: "Kimi K3 深读", value: 88, next: "第三轮加入官方权重 traces、独立复现与逐图数值重绘" },
{ label: "语言模型前史", value: 78, next: "逐图精读 Kneser–Ney、LSTM 与 Bahdanau，并加入真实小语料复现" },
{ label: "Transformer 基础", value: 79, next: "逐图精读多头电路、Pre/Post-LN 与真实 kernel / KV 配置" },
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
@@ -50,7 +50,7 @@ const workstreams = [
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-29 11:10 CST</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-29 11:54 CST</dd></div>
<div><dt>MODE</dt><dd>持续迭代，不锁死版本</dd></div>
</dl>
</div>
@@ -97,12 +97,13 @@ const workstreams = [
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全，包含先修依赖和三条贯穿案例。</p></article>
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
<article><span>✓</span><h3>五十九个原创交互视图</h3><p>K3、语言模型前史、Transformer、表示深度、DeepSeek 四联实验、长上下文、MoE、推理、Agent、多模态，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>六十七个原创交互视图</h3><p>K3 三轴图与八联实验、语言模型前史、Transformer、表示深度、DeepSeek 四联实验、长上下文、MoE、推理、Agent、多模态，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
<article><span>✓</span><h3>表示、位置与残差高速公路深度专题</h3><p>二十张问题账、66 个一手节点、DeepSeek/Kimi 双谱系，以及 Token—位置—Norm—Residual/FFN 四联实验。</p></article>
<article><span>✓</span><h3>DeepSeek 技术谱系二轮深读</h3><p>二十四张问题账、十次技术转向、60 个一手/官方节点，以及稀疏容量—MLA 缓存—V3 协同—RL 偏差四联实验。</p></article>
<article><span>✓</span><h3>Kimi K3 技术报告二轮深读</h3><p>三十二张问题账、Figure 1–16 / Table 1–5 审计、100 节点阅读链，以及 Delta—Decay—AttnRes—LatentMoE—SiTU—QB—MOPD—Cache 八联实验。</p></article>
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
<article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article>
<article><span>✓</span><h3>长上下文深度专题</h3><p>五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。</p></article>
@@ -130,6 +131,7 @@ const workstreams = [
</div>
<div class="queue-table">
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
<div><span>P0</span><strong>K3 三轮</strong><p>开放权重 traces → FlashKDA / AttnRes / MoE 真实行为 → Figure 1–16 数值重绘与独立复现</p><em>运行证据 + 逐图复现</em></div>
<div><span>P0</span><strong>DeepSeek 三轮</strong><p>真实 expert load / MLA kernel → FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
@@ -204,6 +206,9 @@ const workstreams = [
<div><time>2026-07-29</time><b>DeepSeek 与 Kimi 服务谱系按状态对象重建</b><p>MLA→V4 异构状态与 Mooncake→KDA→K3 混合缓存分开说明；作者报告、精确公式和教学估算使用不同标签。</p></div>
<div><time>2026-07-29</time><b>表示与深度按二十张账组织</b><p>计算单位、词表接口、上下文化、位置、外推、Norm 对象、拓扑、残差路由与非线性极值不再混成一个 hidden-state 名词。</p></div>
<div><time>2026-07-29</time><b>K3 Block AttnRes 来源数按原报告重算</b><p>93 层按 12 层形成 8 个 layer blocks（7 个完整块加 1 个尾块）；再加 embedding，共 9 个 block-level 来源，废弃早期错误的“2 层一块”读法。</p></div>
<div><time>2026-07-29</time><b>K3 二轮按三十二张账重建</b><p>从架构组件摘要升级为覆盖预训练、后训练、环境、系统、评测、案例与附录的完整报告因果链。</p></div>
<div><time>2026-07-29</time><b>K3 原生视觉事实纠错</b><p>MoonViT-V2 从头训练；视觉与文本从训练开始在同一个 NTP objective 中联合优化，不再沿用冻结/解冻式 post-hoc 叙述。</p></div>
<div><time>2026-07-29</time><b>K3 图表与实验永久分级</b><p>Figure 1–16 / Table 1–5 建立视觉契约；报告事实、原论文、确定性推导与教学模型使用 R/P/D/T 四种身份。</p></div>
</div>
</section>
src/pages/roadmap/index.astro
+6 -6
View File
@@ -101,13 +101,13 @@ const stages = [
</article>
<article>
<span>ROUTE B · K3 REVERSE</span>
<h3>从 K3 反向拆组件</h3>
<p>K3 → 07 → 06 → 03 → 11 → 08/14</p>
<h3>从 47 页 K3 报告反向拆组件</h3>
<p>K3 架构 → 07/06/03 → 05/04 → 11/12 → 08/14/15</p>
<ol>
<li>解释三维信息流</li>
<li>比较 KDA 与 MLA</li>
<li>解释 2.8T / 104B</li>
<li>读懂 1M Agentic RL 系统</li>
<li>沿 32 张问题账解释三维信息流</li>
<li>用 8 个实验比较 KDA、MLA、AttnRes 与 LatentMoE</li>
<li>分清 2.78T / 104.2B、2.5× 与 1M 的证据口径</li>
<li>读懂九专家 MOPD、AgentENV、混合缓存与评测协议</li>
</ol>
</article>
<article>