wuyang/llm-atlas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: deepen DeepSeek technical lineage

Browse Source
This commit is contained in:
wuyang
2026-07-29 11:17:53 +08:00
parent 5c8a2e44bd
commit cd96dab22d
27 changed files with 2905 additions and 102 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/components/DeepSeekLab.astro
+623
View File
@@ -0,0 +1,623 @@
<figure class="ds-lab" data-deepseek-lab>
<div class="ds-lab-head">
<div>
<p>INTERACTIVE / DEEPSEEK SYSTEM ATLAS</p>
<h3>四本账，把“模型创新”拆回可计算对象</h3>
</div>
<p>
这里混合精确计数与显式 toy model：参数组合和 KV 元素按公式计算；通信、bubble 与梯度权重只展示方向。
每个面板都标出证据边界，不能拿来替代真实 checkpoint 或集群复跑。
</p>
</div>
<div class="ds-tabs" role="tablist" aria-label="选择 DeepSeek 技术实验">
<button type="button" role="tab" data-ds-tab="capacity" aria-selected="true">
<span>01</span><b>稀疏容量账</b><small>MoE · shared · communication</small>
</button>
<button type="button" role="tab" data-ds-tab="cache" aria-selected="false" tabindex="-1">
<span>02</span><b>MLA 缓存账</b><small>MHA · GQA · latent · RoPE</small>
</button>
<button type="button" role="tab" data-ds-tab="codesign" aria-selected="false" tabindex="-1">
<span>03</span><b>V3 协同账</b><small>FP8 · DualPipe · MTP</small>
</button>
<button type="button" role="tab" data-ds-tab="rl" aria-selected="false" tabindex="-1">
<span>04</span><b>RL 偏差镜</b><small>GRPO · DAPO · Dr.GRPO</small>
</button>
</div>
<section class="ds-panel" data-ds-panel="capacity">
<div class="panel-intro">
<div><span>WORKBENCH 01 / SPARSE CAPACITY</span><h4>总参数很大，不代表每个 Token 都经过全部专家</h4></div>
<p>切换架构或自定义专家配置，分开观察总容量、激活计算和跨设备通信；组合数只是可选路径，不是能力分数。</p>
</div>
<div class="preset-row" role="group" aria-label="选择容量预设">
<button type="button" data-capacity-preset="dense"><b>DENSE</b><small>一条完整 FFN</small></button>
<button type="button" data-capacity-preset="coarse"><b>COARSE MOE</b><small>8 选 2</small></button>
<button type="button" data-capacity-preset="deepseekmoe"><b>DEEPSEEKMOE</b><small>细粒度 + shared</small></button>
<button type="button" data-capacity-preset="v3" class="active"><b>V3</b><small>256 routed · 8 active</small></button>
</div>
<div class="control-grid five">
<label><span>routed experts E <output data-experts-label>256</output></span><input data-experts type="range" min="1" max="384" value="256" /></label>
<label><span>routed top-k <output data-topk-label>8</output></span><input data-topk type="range" min="0" max="16" value="8" /></label>
<label><span>shared experts <output data-shared-label>1</output></span><input data-shared type="range" min="0" max="4" value="1" /></label>
<label><span>expert width <output data-expert-width-label>0.125×</output></span><input data-expert-width type="range" min="1" max="100" value="12.5" step=".5" /></label>
<label><span>EP nodes <output data-ep-label>4</output></span><input data-ep-nodes type="range" min="1" max="16" value="4" /></label>
</div>
<div class="capacity-stage">
<div class="router-box">
<span>TOKEN</span><b>h<sub>t</sub></b><small>router scores</small>
</div>
<i>→</i>
<div class="expert-map" data-expert-map aria-label="教学专家池"></div>
<i>→</i>
<div class="router-box output"><span>COMBINE</span><b>Σ gᵢEᵢ(h)</b><small>shared + routed</small></div>
</div>
<div class="metric-grid four">
<article><span>TOTAL EXPERT CAPACITY</span><b data-total-capacity>33.0× FFN</b><p>按 expert width 折算，不含 attention</p></article>
<article><span>ACTIVE EXPERT COMPUTE</span><b data-active-compute>2.0× FFN</b><p>每 Token 的教学 FFN 单位</p></article>
<article><span>ROUTE COMBINATIONS</span><b data-combinations>≈ 10¹⁵</b><p>只表示可组合路径，不代表专长质量</p></article>
<article class="dark"><span>COMMUNICATION PRESSURE</span><b data-communication>HIGH</b><p>教学指标；不是 GB/s 实测</p></article>
</div>
<div class="evidence-boundary">
<b data-capacity-name>DEEPSEEK-V3 / CAPACITY CONTRACT</b>
<p data-capacity-explain>V3 每层含 1 个 shared 与 256 个 routed experts，每 Token 激活 8 个 routed；理论稀疏计算仍需要 expert dispatch/combine 与负载均衡。</p>
</div>
</section>
<section class="ds-panel" data-ds-panel="cache" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 02 / INFERENCE STATE</span><h4>MLA 省的不是模型权重，而是每个请求不断增长的历史状态</h4></div>
<p>所有方案都与当前 MHA 基线比较；MLA 的缓存必须包含 joint latent 和 decoupled RoPE key，不能只报 latent。</p>
</div>
<div class="control-grid four">
<label><span>layers L <output data-cache-layers-label>60</output></span><input data-cache-layers type="range" min="12" max="96" step="12" value="60" /></label>
<label><span>context T</span><select data-context-length><option value="4096">4K</option><option value="32768">32K</option><option value="131072" selected>128K</option><option value="1048576">1M</option></select></label>
<label><span>batch B <output data-batch-label>4</output></span><input data-batch type="range" min="1" max="32" value="4" /></label>
<label><span>storage</span><select data-cache-bytes><option value="1">FP8 / 1 byte</option><option value="2" selected>BF16 / 2 bytes</option><option value="4">FP32 / 4 bytes</option></select></label>
</div>
<div class="control-grid five">
<label><span>MHA heads <output data-heads-label>128</output></span><input data-heads type="range" min="8" max="160" step="8" value="128" /></label>
<label><span>head dim dₕ <output data-head-dim-label>128</output></span><input data-head-dim type="range" min="32" max="256" step="32" value="128" /></label>
<label><span>GQA KV groups <output data-kv-groups-label>8</output></span><input data-kv-groups type="range" min="1" max="32" value="8" /></label>
<label><span>MLA latent d꜀ <output data-latent-label>512</output></span><input data-latent type="range" min="128" max="2048" step="128" value="512" /></label>
<label><span>RoPE key dᴿ <output data-rope-dim-label>64</output></span><input data-rope-dim type="range" min="0" max="128" step="16" value="64" /></label>
</div>
<div class="cache-comparison">
<article data-cache-card="mha">
<span>MHA</span><b data-mha-elements>32,768</b><small>elements / token / layer</small>
<div><i data-mha-bar></i></div><p>完整 K 与 V，所有 heads 分开缓存。</p>
</article>
<article data-cache-card="gqa">
<span>GQA</span><b data-gqa-elements>2,048</b><small>elements / token / layer</small>
<div><i data-gqa-bar></i></div><p>Query heads 分组共享 K/V。</p>
</article>
<article class="selected" data-cache-card="mla">
<span>MLA / V2-LIKE</span><b data-mla-elements>576</b><small>d꜀ + dᴿ elements</small>
<div><i data-mla-bar></i></div><p>joint latent + decoupled RoPE key。</p>
</article>
</div>
<div class="absorption-grid">
<div class="absorb-flow">
<span>CONTENT PATH / ASSOCIATIVITY</span>
<code>qᵀ(W<sub>UK</sub>c) = (W<sub>UK</sub>ᵀq)ᵀc</code>
<div><b>q</b><i>× absorbed W</i><b>c<sup>KV</sup></b><i>→</i><b>score</b></div>
<p>固定线性上投影可吸收到 query/output 权重，不必先物化完整多头 content K/V。</p>
</div>
<div class="rope-flow">
<span>POSITION PATH / DECOUPLED</span>
<code>q<sup>R</sup>ᵀ R(j−i) k<sup>R</sup></code>
<div><b>qᴿ</b><i>rotate</i><b>kᴿ</b><i>cache</i><b data-rope-cache>64</b></div>
<p>RoPE 是位置相关运算，无法作为固定矩阵吸收；因此 key 的小位置分支仍要缓存。</p>
</div>
</div>
<div class="metric-grid four">
<article><span>SELECTED MLA CACHE</span><b data-selected-cache>32.96 GiB</b><p>当前 L × T × B × bytes</p></article>
<article><span>MHA BASELINE</span><b data-mha-cache>1.88 TiB</b><p>同一配置，不是报告基线</p></article>
<article><span>REDUCTION VS MHA</span><b data-cache-reduction>98.2%</b><p>由当前滑条计算</p></article>
<article class="dark"><span>REPORT ≠ TOY</span><b>93.3%</b><p>V2 报告相对 DeepSeek 67B</p></article>
</div>
<div class="evidence-boundary">
<b>EXACT ELEMENT COUNT / CONFIGURATION-DEPENDENT BYTES</b>
<p data-cache-boundary>元素公式是精确账；GiB 只按所选 dtype 计算，没有加入 allocator、page、quantization metadata 或 kernel workspace。</p>
</div>
</section>
<section class="ds-panel" data-ds-panel="codesign" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 03 / V3 CO-DESIGN</span><h4>FP8、DualPipe 与 MTP 改的是三种不同成本</h4></div>
<p>调度台只显示方向：bubble 与通信是 toy schedule；精度角色和 MTP 生命周期来自 V3 报告。</p>
</div>
<div class="codesign-grid">
<div class="schedule-workbench">
<div class="subhead"><span>PIPELINE</span><h5>依赖等待怎样暴露设备空闲？</h5></div>
<div class="choice-row" role="group" aria-label="选择教学 pipeline schedule">
<button type="button" data-schedule="onef1b" class="active"><b>1F1B-LIKE</b><small>单向填充</small></button>
<button type="button" data-schedule="dual"><b>DUAL-ENDED TOY</b><small>两端注入 + overlap</small></button>
</div>
<div class="control-grid three compact">
<label><span>PP stages <output data-stages-label>8</output></span><input data-stages type="range" min="2" max="16" step="2" value="8" /></label>
<label><span>micro-batches <output data-micro-label>20</output></span><input data-micro type="range" min="4" max="64" step="2" value="20" /></label>
<label><span>comm / compute <output data-comm-label>0.60×</output></span><input data-comm-ratio type="range" min="0" max="150" value="60" /></label>
</div>
<div class="pipeline-strip" data-pipeline-strip aria-label="教学流水线时隙"></div>
<div class="metric-grid two">
<article><span>TOY BUBBLE</span><b data-bubble>25.9%</b><p>只由 stages / micro-batches 构造</p></article>
<article class="dark"><span>EXPOSED COMM</span><b data-exposed-comm>45.0%</b><p>重叠假设后的教学比例</p></article>
</div>
</div>
<div class="precision-workbench">
<div class="subhead"><span>PRECISION CONTRACT</span><h5>一个“FP8”标签不够描述训练</h5></div>
<div class="choice-row" role="group" aria-label="选择精度配方">
<button type="button" data-precision="bf16"><b>BF16</b><small>统一高精度基线</small></button>
<button type="button" data-precision="naive"><b>NAIVE FP8</b><small>错误的一键强转</small></button>
<button type="button" data-precision="mixed" class="active"><b>V3 MIXED</b><small>角色分治</small></button>
</div>
<div class="dtype-contract">
<article><span>GEMM INPUTS</span><b data-gemm-dtype>FP8 / tiled scale</b></article>
<article><span>ACCUMULATION</span><b data-accum-dtype>FP32-assisted</b></article>
<article><span>MASTER / OPTIMIZER</span><b data-master-dtype>FP32 / BF16 roles</b></article>
<article><span>SENSITIVE OPS</span><b data-sensitive-dtype>BF16 / FP32</b></article>
</div>
<div class="risk-meter"><span>NUMERICAL RISK</span><div><i data-risk-bar></i></div><b data-risk-label>MANAGED</b></div>
<p data-precision-explain>V3 把主要 GEMM、缩放、累加、master state、敏感算子和通信分别设计；“混合精度”才是完整对象。</p>
</div>
</div>
<div class="mtp-workbench">
<div class="subhead"><span>MULTI-TOKEN PREDICTION</span><h5>同一模块在生命周期中有三种角色</h5></div>
<div class="choice-row mtp" role="group" aria-label="选择 MTP 角色">
<button type="button" data-mtp-role="off"><b>OFF</b><small>只做 NTP</small></button>
<button type="button" data-mtp-role="train" class="active"><b>TRAIN</b><small>额外未来监督</small></button>
<button type="button" data-mtp-role="draft"><b>SPECULATIVE</b><small>候选草稿</small></button>
</div>
<div class="mtp-flow" data-mtp-flow></div>
<div class="metric-grid three">
<article><span>SUPERVISION DEPTH</span><b data-mtp-supervision>2 token targets</b><p>主 NTP + 一层 V3-style MTP</p></article>
<article><span>MAIN MODEL COST</span><b data-mtp-main-cost>TRAIN + MODULE</b><p>推理可丢弃额外 module</p></article>
<article class="dark"><span>AUTOREGRESSIVE?</span><b data-mtp-ar>YES</b><p>草稿仍需验证，不取代主模型</p></article>
</div>
<p class="boundary-copy" data-mtp-explain>V3 的顺序 MTP 首先增加训练信号；报告明确推理可直接丢弃，也可复用于 speculative decoding。</p>
</div>
<div class="evidence-boundary">
<b>TOY SCHEDULE / SOURCE-GROUNDED ROLE CONTRACT</b>
<p data-codesign-boundary>本台没有复跑 DualPipe 或 FP8 kernel；它只阻止把三个独立机制压成“V3 训练便宜”一句话。</p>
</div>
</section>
<section class="ds-panel" data-ds-panel="rl" hidden>
<div class="panel-intro">
<div><span>WORKBENCH 04 / POLICY GRADIENT</span><h4>奖励相同、回答长度和归一方式都会改变“谁被学得更多”</h4></div>
<p>DAPO 与 Dr.GRPO 是 R1 之后的公开研究，不是 DeepSeek 已披露的 R1 内部配方；这里用 toy weight 暴露目标函数偏差。</p>
</div>
<div class="preset-row rl-presets" role="group" aria-label="选择 rollout 奖励预设">
<button type="button" data-reward-preset="mixed" class="active"><b>MIXED</b><small>0 · 1 · .7 · .2</small></button>
<button type="button" data-reward-preset="same"><b>ALL SAME</b><small>零组内信号</small></button>
<button type="button" data-reward-preset="longwrong"><b>LONG WRONG</b><small>长度偏差探针</small></button>
</div>
<div class="rollout-controls">
{[
["1", "0", "512"],
["2", "1", "768"],
["3", ".7", "1536"],
["4", ".2", "3072"],
].map(([index, reward, length]) => (
<label>
<span>y{index}</span>
<small>reward</small><input data-rollout-reward type="number" min="-1" max="1" step=".1" value={reward} />
<small>tokens</small><input data-rollout-length type="number" min="64" max="8192" step="64" value={length} />
</label>
))}
</div>
<div class="choice-row algorithm-row" role="group" aria-label="选择策略梯度教学算法">
<button type="button" data-rl-algorithm="grpo" class="active"><b>GRPO</b><small>std norm · response avg</small></button>
<button type="button" data-rl-algorithm="dapo"><b>DAPO-STYLE</b><small>token loss · dynamic filter</small></button>
<button type="button" data-rl-algorithm="dr"><b>Dr.GRPO</b><small>no std · fixed denominator</small></button>
</div>
<div class="control-grid three compact">
<label><span>clip low <output data-clip-low-label>0.20</output></span><input data-clip-low type="range" min="5" max="40" value="20" /></label>
<label><span>clip high <output data-clip-high-label>0.20</output></span><input data-clip-high type="range" min="5" max="80" value="20" /></label>
<label><span>overlong threshold <output data-overlong-label>4096</output></span><input data-overlong type="range" min="512" max="8192" step="512" value="4096" /></label>
</div>
<div class="rl-stage">
<div class="advantage-table">
<div class="head"><span>ROLLOUT</span><span>R</span><span>LENGTH</span><span>Â</span><span>TOY WEIGHT</span></div>
<div data-advantage-rows></div>
</div>
<div class="rl-diagnosis">
<span data-algorithm-name>GRPO / ORIGINAL FAMILY</span>
<h5 data-signal-state>GROUP-RELATIVE SIGNAL</h5>
<p data-algorithm-explain>奖励按组均值和标准差归一；response-level loss 再按各自长度平均，可能改变长短回答的 Token 权重。</p>
<dl>
<div><dt>reward mean</dt><dd data-reward-mean>0.475</dd></div>
<div><dt>reward std</dt><dd data-reward-std>0.396</dd></div>
<div><dt>effective samples</dt><dd data-effective>4 / 4</dd></div>
<div><dt>provenance</dt><dd data-provenance>DeepSeekMath / R1</dd></div>
</dl>
</div>
</div>
<div class="pipeline-switch">
<div class="subhead"><span>TRAINING IDENTITY</span><h5>R1-Zero、R1 与 Distill 不是三个名字相近的同一步</h5></div>
<div class="choice-row" role="group" aria-label="选择 R1 训练身份">
<button type="button" data-r1-mode="zero" class="active"><b>R1-ZERO</b><small>base → rule RL</small></button>
<button type="button" data-r1-mode="r1"><b>R1</b><small>cold start → multi-stage</small></button>
<button type="button" data-r1-mode="distill"><b>DISTILL</b><small>teacher traces → SFT</small></button>
</div>
<div class="r1-mode-flow" data-r1-mode-flow></div>
<p data-r1-mode-explain>R1-Zero 从 V3 Base 直接进行规则奖励 GRPO；“无 reasoning SFT”不等于“无预训练知识”。</p>
</div>
<div class="evidence-boundary">
<b data-rl-boundary-title>TOY GRADIENT WEIGHT / NOT A TRAINING REPLAY</b>
<p data-rl-boundary>优势公式与论文定义对齐；权重只用来显示归一和长度方向，没有 policy ratio、真实 token probability、KL 或 optimizer。</p>
</div>
</section>
<figcaption>
<span>证据分层</span>
KV 元素与组合参数按公式；FP8/MTP/R1 角色来自官方报告；bubble、通信、数值风险和梯度 weight 为本站教学模型。
</figcaption>
</figure>
<script>
const roots = document.querySelectorAll<HTMLElement>("[data-deepseek-lab]");
roots.forEach((root) => {
const one = <T extends Element>(selector: string) => root.querySelector<T>(selector);
const all = <T extends Element>(selector: string) => [...root.querySelectorAll<T>(selector)];
const value = (selector: string) => Number(one<HTMLInputElement | HTMLSelectElement>(selector)?.value ?? 0);
const set = (selector: string, text: string) => {
const node = one<HTMLElement>(selector);
if (node) node.textContent = text;
};
const compact = (number: number) => {
if (number >= 1024 ** 4) return `${(number / 1024 ** 4).toFixed(2)} TiB`;
if (number >= 1024 ** 3) return `${(number / 1024 ** 3).toFixed(2)} GiB`;
if (number >= 1024 ** 2) return `${(number / 1024 ** 2).toFixed(2)} MiB`;
return `${number.toLocaleString()} B`;
};
const tabs = all<HTMLButtonElement>("[data-ds-tab]");
const panels = all<HTMLElement>("[data-ds-panel]");
const selectTab = (tab: HTMLButtonElement) => {
tabs.forEach((candidate) => {
const selected = candidate === tab;
candidate.setAttribute("aria-selected", String(selected));
candidate.tabIndex = selected ? 0 : -1;
});
panels.forEach((panel) => panel.hidden = panel.dataset.dsPanel !== tab.dataset.dsTab);
};
tabs.forEach((tab, index) => {
tab.addEventListener("click", () => selectTab(tab));
tab.addEventListener("keydown", (event) => {
if (!["ArrowLeft", "ArrowRight", "Home", "End"].includes(event.key)) return;
event.preventDefault();
let target = index;
if (event.key === "ArrowRight") target = (index + 1) % tabs.length;
if (event.key === "ArrowLeft") target = (index - 1 + tabs.length) % tabs.length;
if (event.key === "Home") target = 0;
if (event.key === "End") target = tabs.length - 1;
tabs[target].focus();
selectTab(tabs[target]);
});
});
const capacityPresets: Record<string, [number, number, number, number, number, string, string]> = {
dense: [1, 1, 0, 100, 1, "DENSE / ONE ACTIVE PATH", "Dense FFN 让总容量与每 Token 激活计算绑定；没有 expert dispatch，但容量增长会直接增加计算。"],
coarse: [8, 2, 0, 100, 2, "COARSE MOE / 8 CHOOSE 2", "少数大专家提供条件计算；每个专家仍覆盖较宽知识，跨节点路由开始产生通信。"],
deepseekmoe: [63, 7, 1, 25, 4, "DEEPSEEKMOE / FINE-GRAINED", "论文代表配置把标准专家切为 0.25×，使用 1 shared + 63 routed、激活 7 routed；具体规模随实验变化。"],
v3: [256, 8, 1, 12.5, 4, "DEEPSEEK-V3 / CAPACITY CONTRACT", "V3 每层含 1 个 shared 与 256 个 routed experts，每 Token 激活 8 个 routed；理论稀疏计算仍需要 expert dispatch/combine 与负载均衡。"],
};
const logChoose = (n: number, k: number) => {
const safeK = Math.min(k, n - k);
if (safeK <= 0) return 0;
let result = 0;
for (let index = 1; index <= safeK; index += 1) result += Math.log10(n - safeK + index) - Math.log10(index);
return result;
};
const renderCapacity = () => {
const experts = Math.max(1, Math.round(value("[data-experts]")));
const topk = Math.min(experts, Math.round(value("[data-topk]")));
const shared = Math.round(value("[data-shared]"));
const width = value("[data-expert-width]") / 100;
const nodes = Math.round(value("[data-ep-nodes]"));
const total = (experts + shared) * width;
const active = (topk + shared) * width;
const logComb = logChoose(experts, topk);
const remoteShare = nodes <= 1 ? 0 : (nodes - 1) / nodes;
const communication = topk * remoteShare;
const level = communication === 0 ? "NONE" : communication < .8 ? "LOW" : communication < 2 ? "MEDIUM" : "HIGH";
set("[data-experts-label]", String(experts));
set("[data-topk-label]", String(topk));
set("[data-shared-label]", String(shared));
set("[data-expert-width-label]", `${width.toFixed(3).replace(/0+$/, "").replace(/\.$/, "")}×`);
set("[data-ep-label]", String(nodes));
set("[data-total-capacity]", `${total.toFixed(1)}× FFN`);
set("[data-active-compute]", `${active.toFixed(2)}× FFN`);
set("[data-combinations]", topk === 0 ? "1 route" : logComb < 6 ? Math.round(10 ** logComb).toLocaleString() : `≈ 10^${Math.floor(logComb)}`);
set("[data-communication]", level);
const map = one<HTMLElement>("[data-expert-map]");
if (map) {
const visible = Math.min(24, experts);
map.innerHTML = [
...Array.from({ length: shared }, (_, index) => `<i class="shared active"><b>S${index + 1}</b><small>shared</small></i>`),
...Array.from({ length: visible }, (_, index) => `<i class="${index < topk ? "active" : ""}"><b>E${index + 1}</b><small>${index < topk ? "route" : "idle"}</small></i>`),
].join("");
}
};
all<HTMLButtonElement>("[data-capacity-preset]").forEach((button) => button.addEventListener("click", () => {
const preset = capacityPresets[button.dataset.capacityPreset ?? "v3"];
const selectors = ["[data-experts]", "[data-topk]", "[data-shared]", "[data-expert-width]", "[data-ep-nodes]"];
selectors.forEach((selector, index) => {
const input = one<HTMLInputElement>(selector);
if (input) input.value = String(preset[index]);
});
set("[data-capacity-name]", preset[5]);
set("[data-capacity-explain]", preset[6]);
all("[data-capacity-preset]").forEach((candidate) => candidate.classList.toggle("active", candidate === button));
renderCapacity();
}));
all<HTMLInputElement>("[data-experts],[data-topk],[data-shared],[data-expert-width],[data-ep-nodes]").forEach((input) => input.addEventListener("input", renderCapacity));
const renderCache = () => {
const layers = value("[data-cache-layers]");
const context = value("[data-context-length]");
const batch = value("[data-batch]");
const bytes = value("[data-cache-bytes]");
const heads = value("[data-heads]");
const headDim = value("[data-head-dim]");
const groups = Math.min(value("[data-kv-groups]"), heads);
const latent = value("[data-latent]");
const rope = value("[data-rope-dim]");
const mha = 2 * heads * headDim;
const gqa = 2 * groups * headDim;
const mla = latent + rope;
const total = (elements: number) => elements * layers * context * batch * bytes;
const max = Math.max(mha, gqa, mla);
set("[data-cache-layers-label]", String(layers));
set("[data-batch-label]", String(batch));
set("[data-heads-label]", String(heads));
set("[data-head-dim-label]", String(headDim));
set("[data-kv-groups-label]", String(groups));
set("[data-latent-label]", String(latent));
set("[data-rope-dim-label]", String(rope));
set("[data-mha-elements]", mha.toLocaleString());
set("[data-gqa-elements]", gqa.toLocaleString());
set("[data-mla-elements]", mla.toLocaleString());
set("[data-rope-cache]", String(rope));
set("[data-selected-cache]", compact(total(mla)));
set("[data-mha-cache]", compact(total(mha)));
set("[data-cache-reduction]", `${((1 - mla / mha) * 100).toFixed(1)}%`);
const bars: [string, number][] = [["[data-mha-bar]", mha], ["[data-gqa-bar]", gqa], ["[data-mla-bar]", mla]];
bars.forEach(([selector, number]) => {
const node = one<HTMLElement>(selector);
if (node) node.style.width = `${Math.max(2, number / max * 100)}%`;
});
set("[data-cache-boundary]", `当前是 ${layers} 层 × ${context.toLocaleString()} Token × batch ${batch} × ${bytes} byte；GiB 未加入 allocator、page、quantization metadata 或 kernel workspace。`);
};
all<HTMLInputElement | HTMLSelectElement>("[data-cache-layers],[data-context-length],[data-batch],[data-cache-bytes],[data-heads],[data-head-dim],[data-kv-groups],[data-latent],[data-rope-dim]").forEach((control) => control.addEventListener("input", renderCache));
let schedule = "onef1b";
let precision = "mixed";
let mtpRole = "train";
const renderSchedule = () => {
const stages = value("[data-stages]");
const micro = value("[data-micro]");
const comm = value("[data-comm-ratio]") / 100;
const effectiveStages = schedule === "dual" ? Math.max(1, stages / 2) : stages;
const bubble = (effectiveStages - 1) / (micro + effectiveStages - 1);
const overlap = schedule === "dual" ? .78 : .25;
const exposed = Math.max(0, comm * (1 - overlap));
set("[data-stages-label]", String(stages));
set("[data-micro-label]", String(micro));
set("[data-comm-label]", `${comm.toFixed(2)}×`);
set("[data-bubble]", `${(bubble * 100).toFixed(1)}%`);
set("[data-exposed-comm]", `${(exposed * 100).toFixed(1)}%`);
const strip = one<HTMLElement>("[data-pipeline-strip]");
if (strip) strip.innerHTML = Array.from({ length: 30 }, (_, index) => {
const warm = index < effectiveStages - 1 || index >= 30 - (effectiveStages - 1);
const commSlot = !warm && index % (schedule === "dual" ? 7 : 4) === 0;
return `<i class="${warm ? "bubble" : commSlot ? "comm" : "compute"}"><small>${warm ? "idle" : commSlot ? "a2a" : index % 2 ? "B" : "F"}</small></i>`;
}).join("");
};
const precisionDetails: Record<string, [string, string, string, string, number, string, string]> = {
bf16: ["BF16", "BF16/FP32", "FP32", "BF16/FP32", 28, "LOWER", "BF16 基线保留更宽动态范围，但增加存储、带宽与高密度算术成本。"],
naive: ["FP8 / one scale", "FP8", "FP8", "FP8", 96, "CRITICAL", "一键把输入、累加、master state 和敏感算子全转 FP8 会暴露溢出、舍入与更新失真；这不是 V3 配方。"],
mixed: ["FP8 / tiled scale", "FP32-assisted", "FP32 / BF16 roles", "BF16 / FP32", 46, "MANAGED", "V3 把主要 GEMM、缩放、累加、master state、敏感算子和通信分别设计；“混合精度”才是完整对象。"],
};
const renderPrecision = () => {
const detail = precisionDetails[precision];
set("[data-gemm-dtype]", detail[0]);
set("[data-accum-dtype]", detail[1]);
set("[data-master-dtype]", detail[2]);
set("[data-sensitive-dtype]", detail[3]);
set("[data-risk-label]", detail[5]);
set("[data-precision-explain]", detail[6]);
const bar = one<HTMLElement>("[data-risk-bar]");
if (bar) bar.style.width = `${detail[4]}%`;
};
const renderMtp = () => {
const flow = one<HTMLElement>("[data-mtp-flow]");
const details: Record<string, [string, string, string, string, string]> = {
off: ["1 token target", "MAIN ONLY", "YES", "hₜ → next token", "关闭 MTP 后只有主 next-token loss；这不是 V3 报告采用的预训练目标。"],
train: ["2 token targets", "TRAIN + MODULE", "YES", "hₜ → t+1 · MTP₁(hₜ,t+1) → t+2", "V3 的顺序 MTP 首先增加训练信号；报告明确推理可直接丢弃，也可复用于 speculative decoding。"],
draft: ["draft + verify", "EXTRA DRAFT COST", "YES", "MTP draft → main model verify → accept/reject", "把 MTP module 当草稿器仍需主模型验证；验收率与 kernel 决定是否真实加速。"],
};
const detail = details[mtpRole];
set("[data-mtp-supervision]", detail[0]);
set("[data-mtp-main-cost]", detail[1]);
set("[data-mtp-ar]", detail[2]);
set("[data-mtp-explain]", detail[4]);
if (flow) flow.innerHTML = detail[3].split("→").map((item, index, array) => `<b>${item.trim()}</b>${index < array.length - 1 ? "<i>→</i>" : ""}`).join("");
};
all<HTMLButtonElement>("[data-schedule]").forEach((button) => button.addEventListener("click", () => {
schedule = button.dataset.schedule ?? "onef1b";
all("[data-schedule]").forEach((candidate) => candidate.classList.toggle("active", candidate === button));
renderSchedule();
}));
all<HTMLInputElement>("[data-stages],[data-micro],[data-comm-ratio]").forEach((control) => control.addEventListener("input", renderSchedule));
all<HTMLButtonElement>("[data-precision]").forEach((button) => button.addEventListener("click", () => {
precision = button.dataset.precision ?? "mixed";
all("[data-precision]").forEach((candidate) => candidate.classList.toggle("active", candidate === button));
renderPrecision();
}));
all<HTMLButtonElement>("[data-mtp-role]").forEach((button) => button.addEventListener("click", () => {
mtpRole = button.dataset.mtpRole ?? "train";
all("[data-mtp-role]").forEach((candidate) => candidate.classList.toggle("active", candidate === button));
renderMtp();
}));
let algorithm = "grpo";
let r1Mode = "zero";
const rewardPresets: Record<string, [number[], number[]]> = {
mixed: [[0, 1, .7, .2], [512, 768, 1536, 3072]],
same: [[1, 1, 1, 1], [512, 768, 1536, 3072]],
longwrong: [[1, .7, .2, 0], [384, 768, 2048, 6144]],
};
const renderRl = () => {
const rewards = all<HTMLInputElement>("[data-rollout-reward]").map((input) => Number(input.value));
const lengths = all<HTMLInputElement>("[data-rollout-length]").map((input) => Math.max(1, Number(input.value)));
const mean = rewards.reduce((sum, number) => sum + number, 0) / rewards.length;
const variance = rewards.reduce((sum, number) => sum + (number - mean) ** 2, 0) / rewards.length;
const std = Math.sqrt(variance);
const threshold = value("[data-overlong]");
const centered = rewards.map((reward) => reward - mean);
const advantages = centered.map((number) => algorithm === "dr" ? number : std > 1e-8 ? number / std : 0);
const weights = advantages.map((advantage, index) => {
if (algorithm === "dr") return advantage / threshold * 1024;
if (algorithm === "dapo") return lengths[index] > threshold ? 0 : advantage * lengths[index] / Math.max(...lengths);
return advantage / lengths[index] * 1024;
});
const effective = algorithm === "dapo" ? rewards.filter((reward, index) => Math.abs(reward - mean) > 1e-8 && lengths[index] <= threshold).length : rewards.length;
const rows = one<HTMLElement>("[data-advantage-rows]");
if (rows) rows.innerHTML = rewards.map((reward, index) => {
const width = Math.min(100, Math.abs(weights[index]) / Math.max(...weights.map(Math.abs), .001) * 100);
return `<div><b>y${index + 1}</b><span>${reward.toFixed(2)}</span><span>${lengths[index].toLocaleString()}</span><span>${advantages[index].toFixed(2)}</span><span class="${weights[index] >= 0 ? "positive" : "negative"}"><i style="width:${width}%"></i>${weights[index].toFixed(2)}</span></div>`;
}).join("");
const details: Record<string, [string, string, string]> = {
grpo: ["GRPO / ORIGINAL FAMILY", "奖励按组均值和标准差归一；response-level loss 再按各自长度平均，可能改变长短回答的 Token 权重。", "DeepSeekMath / R1"],
dapo: ["DAPO-STYLE / FOLLOW-UP", "这里用 token-level 聚合方向和 overlong filter 展示 DAPO 的两个修正；真实 DAPO 还包含 Clip-Higher 与 Dynamic Sampling。", "DAPO · arXiv:2503.14476"],
dr: ["Dr.GRPO / FOLLOW-UP", "去掉组 std normalization，并用固定全局长度分母，暴露原 GRPO 的 response-length 与 question-difficulty 尺度问题。", "Dr.GRPO · arXiv:2503.20783"],
};
const detail = details[algorithm];
set("[data-algorithm-name]", detail[0]);
set("[data-algorithm-explain]", detail[1]);
set("[data-provenance]", detail[2]);
set("[data-reward-mean]", mean.toFixed(3));
set("[data-reward-std]", std.toFixed(3));
set("[data-effective]", `${effective} / ${rewards.length}`);
set("[data-signal-state]", std < 1e-8 ? "ZERO GROUP SIGNAL" : algorithm === "dapo" && effective < rewards.length ? "FILTER / RESAMPLE" : "GROUP-RELATIVE SIGNAL");
set("[data-clip-low-label]", (value("[data-clip-low]") / 100).toFixed(2));
set("[data-clip-high-label]", (value("[data-clip-high]") / 100).toFixed(2));
set("[data-overlong-label]", String(threshold));
set("[data-rl-boundary]", std < 1e-8
? "四条奖励完全相同：组内中心化后优势为零。DAPO Dynamic Sampling 会过滤此类无梯度组并补采，但 rollout 成本不会消失。"
: "优势公式与论文定义对齐；weight 只显示归一和长度方向，没有 policy ratio、真实 token probability、KL 或 optimizer。");
};
all<HTMLButtonElement>("[data-reward-preset]").forEach((button) => button.addEventListener("click", () => {
const preset = rewardPresets[button.dataset.rewardPreset ?? "mixed"];
all<HTMLInputElement>("[data-rollout-reward]").forEach((input, index) => { input.value = String(preset[0][index]); });
all<HTMLInputElement>("[data-rollout-length]").forEach((input, index) => { input.value = String(preset[1][index]); });
all("[data-reward-preset]").forEach((candidate) => candidate.classList.toggle("active", candidate === button));
renderRl();
}));
all<HTMLInputElement>("[data-rollout-reward],[data-rollout-length],[data-clip-low],[data-clip-high],[data-overlong]").forEach((control) => control.addEventListener("input", renderRl));
all<HTMLButtonElement>("[data-rl-algorithm]").forEach((button) => button.addEventListener("click", () => {
algorithm = button.dataset.rlAlgorithm ?? "grpo";
all("[data-rl-algorithm]").forEach((candidate) => candidate.classList.toggle("active", candidate === button));
renderRl();
}));
const r1Details: Record<string, [string[], string]> = {
zero: [["V3 BASE", "RULE REWARD", "GRPO", "R1-ZERO"], "R1-Zero 从 V3 Base 直接进行规则奖励 GRPO；“无 reasoning SFT”不等于“无预训练知识”。"],
r1: [["V3 BASE", "COLD START", "REASONING RL", "SFT MIX", "GENERAL RL"], "正式 R1 用 cold start 修可读性与语言，再经 reasoning RL、rejection/SFT mix 和通用 RL；它不是纯 RL 单阶段。"],
distill: [["R1 TEACHER", "≈800K FILTERED TRACES", "SFT", "1.5B–70B STUDENTS"], "报告中的蒸馏学生主要学习 R1 生成/筛选轨迹，没有重演同一大规模 RL 探索过程。"],
};
const renderR1 = () => {
const detail = r1Details[r1Mode];
const flow = one<HTMLElement>("[data-r1-mode-flow]");
if (flow) flow.innerHTML = detail[0].map((item, index) => `<b>${item}</b>${index < detail[0].length - 1 ? "<i>→</i>" : ""}`).join("");
set("[data-r1-mode-explain]", detail[1]);
};
all<HTMLButtonElement>("[data-r1-mode]").forEach((button) => button.addEventListener("click", () => {
r1Mode = button.dataset.r1Mode ?? "zero";
all("[data-r1-mode]").forEach((candidate) => candidate.classList.toggle("active", candidate === button));
renderR1();
}));
renderCapacity();
renderCache();
renderSchedule();
renderPrecision();
renderMtp();
renderRl();
renderR1();
});
</script>
<style>
.ds-lab { max-width:100%; margin:36px 0; overflow:hidden; box-sizing:border-box; border:1px solid var(--line-strong); background:var(--paper); box-shadow:0 24px 70px rgba(8,18,30,.16); }
.ds-lab-head { display:grid; grid-template-columns:1fr 1fr; gap:38px; align-items:end; padding:30px; color:#f4efe7; background:linear-gradient(135deg,#172437,#263b52); }
.ds-lab-head p:first-child,.panel-intro span,.subhead span { margin:0 0 9px; color:#d49a68; font:600 .55rem var(--mono); letter-spacing:.13em; }
.ds-lab-head h3 { margin:0; color:#fff; font:650 clamp(1.3rem,2.6vw,2.05rem) var(--serif); }
.ds-lab-head > p { margin:0; color:#b7c1cc; font-size:.65rem; line-height:1.8; }
.ds-tabs { display:grid; grid-template-columns:repeat(4,1fr); border-bottom:1px solid var(--line-strong); }
.ds-tabs button { min-height:94px; display:grid; grid-template-columns:32px 1fr; grid-template-rows:auto auto; gap:4px 8px; padding:18px; border:0; border-right:1px solid var(--line); color:var(--ink); background:var(--paper-raised); text-align:left; cursor:pointer; }
.ds-tabs button:last-child { border-right:0; }.ds-tabs button[aria-selected="true"] { color:#fff; background:var(--navy); }
.ds-tabs span { grid-row:1/3; color:var(--copper); font:600 .6rem var(--mono); }.ds-tabs b { font:650 .84rem var(--serif); }.ds-tabs small { color:var(--muted); font:.49rem var(--mono); }
.ds-tabs button[aria-selected="true"] small { color:#aab7c5; }
.ds-panel { padding:30px; }.panel-intro { display:grid; grid-template-columns:1.1fr .9fr; gap:34px; align-items:end; margin-bottom:24px; }
.panel-intro h4 { margin:0; font:650 clamp(1.2rem,2.5vw,1.85rem) var(--serif); }.panel-intro > p { margin:0; color:var(--muted); font-size:.65rem; line-height:1.8; }
.preset-row,.choice-row { display:grid; grid-template-columns:repeat(4,1fr); gap:8px; margin:18px 0; }
.preset-row button,.choice-row button { min-height:63px; padding:12px; border:1px solid var(--line); color:var(--ink); background:var(--paper-raised); text-align:left; cursor:pointer; }
.preset-row button.active,.choice-row button.active { color:#fff; border-color:var(--navy); background:var(--navy); }
.preset-row b,.choice-row b { display:block; font:.59rem var(--mono); }.preset-row small,.choice-row small { display:block; margin-top:7px; color:var(--muted); font:.48rem var(--mono); }
.preset-row button.active small,.choice-row button.active small { color:#b6c2ce; }
.control-grid { display:grid; gap:9px; margin:16px 0 22px; }.control-grid.five { grid-template-columns:repeat(5,1fr); }.control-grid.four { grid-template-columns:repeat(4,1fr); }.control-grid.three { grid-template-columns:repeat(3,1fr); }
.control-grid label { display:flex; flex-direction:column; justify-content:space-between; gap:9px; min-height:72px; padding:12px; border:1px solid var(--line); background:var(--paper-raised); color:var(--muted); font:.52rem var(--mono); }
.control-grid label > span { display:flex; justify-content:space-between; gap:6px; }.control-grid input[type="range"] { width:100%; accent-color:var(--copper); }.control-grid select { min-height:31px; border:1px solid var(--line); color:var(--ink); background:var(--paper); font:.55rem var(--mono); }
.capacity-stage { display:grid; grid-template-columns:130px 25px 1fr 25px 145px; gap:12px; align-items:center; margin:24px 0; padding:22px; background:#172437; color:#fff; }
.capacity-stage > i { color:#d49a68; text-align:center; font-style:normal; }.router-box { padding:18px; border:1px solid #405269; }.router-box span { color:#d49a68; font:.5rem var(--mono); }.router-box b { display:block; margin:10px 0; font:650 1.2rem var(--serif); }.router-box small { color:#98a9ba; font:.48rem var(--mono); }
.expert-map { display:grid; grid-template-columns:repeat(8,1fr); gap:5px; }.expert-map i { min-height:43px; display:flex; flex-direction:column; justify-content:center; align-items:center; border:1px solid #405269; color:#7d8fa3; font-style:normal; }.expert-map i.active { color:#fff; border-color:#d49a68; background:rgba(212,154,104,.18); }.expert-map i.shared { background:#8c5538; }
.expert-map b { font:.5rem var(--mono); }.expert-map small { margin-top:4px; font:.4rem var(--mono); }
.metric-grid { display:grid; border-top:1px solid var(--line); border-left:1px solid var(--line); margin:20px 0; }.metric-grid.four { grid-template-columns:repeat(4,1fr); }.metric-grid.three { grid-template-columns:repeat(3,1fr); }.metric-grid.two { grid-template-columns:repeat(2,1fr); }
.metric-grid article { min-height:125px; padding:18px; border-right:1px solid var(--line); border-bottom:1px solid var(--line); }.metric-grid span { color:var(--copper); font:.49rem var(--mono); letter-spacing:.08em; }.metric-grid b { display:block; margin:14px 0 9px; font:650 1.12rem var(--serif); }.metric-grid p { margin:0; color:var(--muted); font-size:.53rem; line-height:1.55; }.metric-grid .dark { color:#fff; background:var(--navy); }.metric-grid .dark p { color:#9fb0c2; }
.evidence-boundary { margin-top:18px; padding:18px 20px; border-left:3px solid var(--copper); background:rgba(193,124,68,.08); }.evidence-boundary b { font:.57rem var(--mono); letter-spacing:.08em; }.evidence-boundary p { margin:8px 0 0; color:var(--muted); font-size:.61rem; line-height:1.7; }
.cache-comparison { display:grid; grid-template-columns:repeat(3,1fr); border:1px solid var(--line); }.cache-comparison article { padding:20px; border-right:1px solid var(--line); }.cache-comparison article:last-child { border-right:0; }.cache-comparison article.selected { background:rgba(193,124,68,.08); }.cache-comparison span { color:var(--copper); font:.54rem var(--mono); }.cache-comparison b { display:block; margin:12px 0 4px; font:650 1.2rem var(--serif); }.cache-comparison small { color:var(--muted-light); font:.48rem var(--mono); }.cache-comparison article > div { height:9px; margin:16px 0; background:var(--line); }.cache-comparison i { display:block; height:100%; background:var(--copper); }.cache-comparison p { color:var(--muted); font-size:.57rem; line-height:1.6; }
.absorption-grid { display:grid; grid-template-columns:1fr 1fr; margin-top:18px; border:1px solid var(--line); }.absorption-grid > div { padding:22px; }.absorption-grid > div:first-child { border-right:1px solid var(--line); }.absorption-grid span { color:var(--copper); font:.5rem var(--mono); }.absorption-grid code { display:block; margin:14px 0; color:var(--ink); font:.73rem var(--mono); }.absorption-grid div > div { display:flex; align-items:center; gap:8px; }.absorption-grid div > div b { padding:9px; border:1px solid var(--line); font:.58rem var(--mono); }.absorption-grid div > div i { color:var(--copper); font:.48rem var(--mono); font-style:normal; }.absorption-grid p { color:var(--muted); font-size:.57rem; line-height:1.65; }
.codesign-grid { display:grid; grid-template-columns:1.15fr .85fr; gap:16px; }.schedule-workbench,.precision-workbench,.mtp-workbench { padding:22px; border:1px solid var(--line); background:var(--paper-raised); }.subhead h5 { margin:0; font:650 1.05rem var(--serif); }.choice-row { grid-template-columns:repeat(2,1fr); }.choice-row.mtp { grid-template-columns:repeat(3,1fr); }
.control-grid.compact label { min-height:62px; padding:10px; }.pipeline-strip { display:grid; grid-template-columns:repeat(15,1fr); gap:3px; min-height:92px; margin:18px 0; }.pipeline-strip i { display:flex; justify-content:center; align-items:center; min-height:36px; font-style:normal; }.pipeline-strip i.compute { color:#fff; background:#314a62; }.pipeline-strip i.comm { color:#fff; background:#b36c45; }.pipeline-strip i.bubble { color:#8a96a3; border:1px dashed var(--line-strong); }.pipeline-strip small { font:.4rem var(--mono); }
.dtype-contract { display:grid; grid-template-columns:1fr 1fr; border-top:1px solid var(--line); border-left:1px solid var(--line); }.dtype-contract article { min-height:85px; padding:13px; border-right:1px solid var(--line); border-bottom:1px solid var(--line); }.dtype-contract span { color:var(--copper); font:.45rem var(--mono); }.dtype-contract b { display:block; margin-top:13px; font:.61rem var(--mono); }.risk-meter { margin:18px 0; }.risk-meter > span { color:var(--muted); font:.48rem var(--mono); }.risk-meter > div { height:9px; margin:8px 0; background:var(--line); }.risk-meter i { display:block; height:100%; background:var(--copper); }.risk-meter > b { font:.56rem var(--mono); }.precision-workbench > p,.boundary-copy { color:var(--muted); font-size:.57rem; line-height:1.65; }
.mtp-workbench { margin-top:16px; }.mtp-flow,.r1-mode-flow { display:flex; align-items:center; flex-wrap:wrap; gap:8px; padding:18px; background:#172437; color:#fff; }.mtp-flow b,.r1-mode-flow b { padding:10px; border:1px solid #405269; font:.53rem var(--mono); }.mtp-flow i,.r1-mode-flow i { color:#d49a68; font-style:normal; }
.rollout-controls { display:grid; grid-template-columns:repeat(4,1fr); gap:8px; }.rollout-controls label { display:grid; grid-template-columns:38px 1fr; gap:7px; padding:12px; border:1px solid var(--line); background:var(--paper-raised); }.rollout-controls span { grid-row:1/5; color:var(--copper); font:650 .75rem var(--serif); }.rollout-controls small { color:var(--muted); font:.45rem var(--mono); }.rollout-controls input { min-width:0; width:100%; border:1px solid var(--line); background:var(--paper); color:var(--ink); font:.54rem var(--mono); }.algorithm-row { grid-template-columns:repeat(3,1fr); }
.rl-stage { display:grid; grid-template-columns:1.15fr .85fr; border:1px solid var(--line); }.advantage-table { overflow:auto; }.advantage-table .head,.advantage-table [data-advantage-rows] > div { min-width:590px; display:grid; grid-template-columns:70px 60px 80px 60px 1fr; gap:8px; align-items:center; padding:12px 15px; border-bottom:1px solid var(--line); }.advantage-table .head { color:#fff; background:#172437; font:.45rem var(--mono); }.advantage-table [data-advantage-rows] > div { font:.55rem var(--mono); }.advantage-table [data-advantage-rows] > div:last-child { border-bottom:0; }.advantage-table [data-advantage-rows] span:last-child { position:relative; min-height:25px; display:flex; align-items:center; padding-left:7px; overflow:hidden; }.advantage-table [data-advantage-rows] span:last-child i { position:absolute; inset:0 auto 0 0; opacity:.18; }.advantage-table .positive i { background:#2e876a; }.advantage-table .negative i { background:#b1504c; }
.rl-diagnosis { padding:22px; color:#fff; background:#172437; }.rl-diagnosis > span { color:#d49a68; font:.5rem var(--mono); }.rl-diagnosis h5 { margin:14px 0; font:650 1.15rem var(--serif); }.rl-diagnosis > p { color:#b7c1cc; font-size:.58rem; line-height:1.7; }.rl-diagnosis dl { margin:18px 0 0; }.rl-diagnosis dl div { display:flex; justify-content:space-between; gap:12px; padding:8px 0; border-top:1px solid #405269; }.rl-diagnosis dt { color:#8fa2b5; font:.45rem var(--mono); }.rl-diagnosis dd { margin:0; font:.52rem var(--mono); text-align:right; }
.pipeline-switch { margin-top:18px; padding:22px; border:1px solid var(--line); }.pipeline-switch > p { color:var(--muted); font-size:.6rem; line-height:1.7; }
.ds-lab figcaption { display:flex; gap:18px; padding:16px 22px; border-top:1px solid var(--line); color:var(--muted); font-size:.54rem; line-height:1.6; }.ds-lab figcaption span { color:var(--copper); font:.49rem var(--mono); }
@media (max-width:1000px) {
.control-grid.five { grid-template-columns:repeat(3,1fr); }.codesign-grid { grid-template-columns:1fr; }.capacity-stage { grid-template-columns:110px 20px 1fr; }.capacity-stage > i:nth-of-type(2),.capacity-stage .output { display:none; }
}
@media (max-width:720px) {
.ds-lab-head,.panel-intro { grid-template-columns:1fr; }.ds-tabs { grid-template-columns:1fr; }.ds-tabs button { min-height:74px; border-right:0; border-bottom:1px solid var(--line); }
.ds-panel { padding:16px; }.preset-row,.control-grid.five,.control-grid.four,.control-grid.three,.cache-comparison,.absorption-grid,.metric-grid.four,.metric-grid.three,.metric-grid.two,.rollout-controls,.algorithm-row,.rl-stage { grid-template-columns:1fr; }
.capacity-stage { grid-template-columns:1fr; }.capacity-stage > i { transform:rotate(90deg); }.capacity-stage > i:nth-of-type(2),.capacity-stage .output { display:block; }.expert-map { grid-template-columns:repeat(6,1fr); }
.cache-comparison article,.absorption-grid > div:first-child { border-right:0; border-bottom:1px solid var(--line); }.pipeline-strip { grid-template-columns:repeat(10,1fr); }.choice-row { grid-template-columns:1fr 1fr; }
.rl-stage { border:0; gap:12px; }.advantage-table,.rl-diagnosis { border:1px solid var(--line); }.ds-lab figcaption { flex-direction:column; }
}
</style>
src/components/DeepSeekLineage.astro
+24 -4
View File
@@ -14,6 +14,13 @@ const milestones = [
bridge: "把容量扩张和每 Token 计算量分开。",
url: "https://arxiv.org/abs/2401.06066",
},
{
year: "2024.02",
model: "DeepSeekMath",
idea: "从大规模数学数据工程走到 GRPO：用同题多条回答的相对奖励，省去独立 critic。",
bridge: "R1 的推理 RL 不是突然出现；算法与可验证数据的预演在这里发生。",
url: "https://arxiv.org/abs/2402.03300",
},
{
year: "2024.05",
model: "DeepSeek-V2",
@@ -31,10 +38,18 @@ const milestones = [
{
year: "2025.01",
model: "DeepSeek-R1",
idea: "R1-Zero 展示纯大规模 RL 可涌现推理；R1 用冷启动数据修复可读性与稳定性。",
idea: "R1-Zero 从强 V3 Base 直接做规则奖励 RL、没有 reasoning SFT；R1 再用冷启动与多阶段训练修复可读性和广度。",
bridge: "从“模仿答案”转向用可验证奖励塑造推理策略。",
url: "https://arxiv.org/abs/2501.12948",
},
{
year: "2025.03",
model: "DAPO / Dr.GRPO",
idea: "公开后续研究分别暴露 clipping、采样、截断、响应长度与题目难度归一偏差。",
bridge: "复现不是 R1 的内部 recipe，而是一台看清 RL 优化对象的显微镜。",
url: "https://arxiv.org/abs/2503.14476",
followup: true,
},
{
year: "2025.12",
model: "DeepSeek-V3.2",
@@ -45,7 +60,7 @@ const milestones = [
{
year: "2026.06",
model: "DeepSeek-V4",
idea: "围绕百万 Token 上下文效率继续扩展，成为 K3 报告直接比较的开放前沿之一。",
idea: "CSA/HCA 构成异构长状态，mHC 约束深层残差，Muon 与数值边界共同支撑百万 Token。",
bridge: "长上下文不再只是位置外推，而是注意力、训练与服务的全系统问题。",
url: "https://arxiv.org/abs/2606.19348",
},
@@ -54,13 +69,13 @@ const milestones = [
<div class="deepseek-lineage">
{milestones.map((item, index) => (
<a href={item.url} class="lineage-row" rel="noreferrer">
<a href={item.url} class:list={["lineage-row", { followup: item.followup }]} rel="noreferrer">
<div class="lineage-time">
<span>{item.year}</span>
<i aria-hidden="true"></i>
</div>
<div class="lineage-main">
<span>DS / {String(index + 1).padStart(2, "0")}</span>
<span>{item.followup ? "PUBLIC FOLLOW-UP" : `DS / ${String(index + 1).padStart(2, "0")}`}</span>
<h3>{item.model}</h3>
<p>{item.idea}</p>
</div>
@@ -93,6 +108,11 @@ const milestones = [
background: var(--paper-raised);
}
.lineage-row.followup {
border-right: 3px solid var(--copper);
background: var(--copper-pale);
}
.lineage-time {
display: grid;
grid-template-columns: 1fr 12px;
src/data/deepseek.ts
+110
View File
@@ -0,0 +1,110 @@
export const deepseekLedgers = [
["Q01", "Dense 坐标系", "为什么 DeepSeek LLM 不是可跳过的序章？", "它固定 tokenizer、数据、架构和 scaling 试验的起点；并不单独证明后续所有设计。"],
["Q02", "参数角色", "671B / 37B 各表示什么？", "total 是装下的容量，activated 是每 Token 经过的专家参数子集；都不等于端到端 FLOPs。"],
["Q03", "专家粒度", "为什么切小专家还要多选？", "DeepSeekMoE 把每个专家缩成 1/m，总数和激活数同乘 m，近似保持专家计算。"],
["Q04", "Shared expert", "为什么把公共知识单独隔离？", "始终激活的 shared experts 减少 routed experts 重复；它们仍然要付激活计算。"],
["Q05", "通信税", "为什么稀疏 FLOPs 不等于便宜？", "路由会产生 dispatch/combine、跨节点 all-to-all、负载长尾和权重访问。"],
["Q06", "均衡", "aux-loss-free 到底去掉了什么？", "V3 的 expert bias 影响选择、不进入最终 gate weight；仍有 sequence-wise auxiliary loss 防极端失衡。"],
["Q07", "KV 状态", "为什么 V2 把服务状态当架构问题？", "权重只装一次，KV 随请求、层、Token 增长，直接限制并发和长上下文。"],
["Q08", "Attention 压缩", "MQA、GQA、MLA 的差别是什么？", "MQA/GQA 共享 K/V 头；MLA 联合低秩压缩 K/V 内容并在计算中恢复。"],
["Q09", "矩阵吸收", "MLA 为什么不必恢复完整 content K/V？", "无位置项时可利用矩阵乘结合律，把 K/V 上投影吸收到 query/output 投影。"],
["Q10", "位置分叉", "为什么要 decoupled RoPE？", "RoPE 会阻断固定权重吸收，所以 V2 另设小 RoPE query/key 分支，并缓存 key。"],
["Q11", "FP8 合同", "“FP8 训练”包含哪些角色？", "主要 GEMM 用 FP8，并配细粒度缩放、较高精度累加和高精度敏感算子；不是全路径 FP8。"],
["Q12", "Pipeline", "DualPipe 隐藏了什么？", "从两端注入 micro-batch，让成对前后向 chunk 与通信重叠；它减少而非清零 bubble。"],
["Q13", "MTP", "训练和推理各怎样使用 MTP？", "顺序模块增加未来 Token 监督；推理可丢弃，也可复用于 speculative draft。"],
["Q14", "GRPO", "去掉 critic 后还剩什么？", "policy/reference、同题多 rollout、reward/verifier、clip 和 KL；主要省掉 value model。"],
["Q15", "可验证奖励", "R1-Zero 的奖励能覆盖哪些任务？", "论文用数学、代码、逻辑等规则可验证域和格式奖励；开放任务仍是限制。"],
["Q16", "纯 RL 实验", "R1-Zero 究竟证明了什么？", "强 V3 Base 在无 reasoning SFT 时可被规则奖励继续塑造；不等于没有预训练先验。"],
["Q17", "R1 pipeline", "正式 R1 为什么不是纯 RL？", "cold start → reasoning RL → rejection/SFT mix → general RL，分别修可读性、广度和对齐。"],
["Q18", "蒸馏", "学生为什么不是“小号 R1-Zero”？", "1.5B–70B 学生主要对约 800K 教师样本做 SFT，没有重演同一 RL 探索。"],
["Q19", "复现反查", "DAPO / Dr.GRPO 修的是哪类问题？", "它们处理 clip、采样、聚合、截断、长度和难度偏差；是后续研究，不是已披露 R1 内部配方。"],
["Q20", "DSA", "可学习 indexer 为什么不是固定稀疏？", "indexer 对历史内容评分，主 attention 只读 top-k；它需要专门训练，也可能漏检。"],
["Q21", "Agent 数据", "V3.2 怎样把 reasoning 放进环境？", "specialist distillation + mixed RL；环境、工具、任务、解法和 verifier 构成数据闭环。"],
["Q22", "V4 Attention", "CSA 与 HCA 各压什么？", "CSA 先压缩再稀疏 top-k；HCA 更强压缩后保留全部 compressed entries。"],
["Q23", "V4 稳定化", "mHC、Muon、QK/RMSNorm、clamp 各管什么？", "它们分别管残差混合、矩阵更新、attention 尺度和 FFN 极值，不能合成一个技巧。"],
["Q24", "K3 对照", "哪些是祖先，哪些只是同题新解？", "DeepSeekMoE/MLA 有明确继承；QB、KDA、AttnRes、SiTU、MOPD 多是新解或同期路线。"],
] as const;
export const deepseekWaves = [
["W1", "2024.01", "Dense 坐标", "先固定数据、tokenizer、训练与 scaling 对照，后面的结构收益才有可比起点。"],
["W2", "2024.01", "稀疏容量", "细粒度 routed experts 加 shared experts，把总容量与单 Token 激活计算第一次清楚分开。"],
["W3", "2024.05", "服务状态", "MLA 不再只优化训练 FLOPs，而是直接改写随请求增长的 KV Cache。"],
["W4", "2024.12", "协同训练", "V3 把路由、FP8、MTP、pipeline 与通信写成同一套训练合同。"],
["W5", "2024.02 → 2025.01", "推理 RL", "DeepSeekMath 先减掉 critic；R1-Zero 再隔离规则奖励，R1 恢复可读性与通用性。"],
["W6", "2025.03", "复现显微镜", "DAPO 与 Dr.GRPO 暴露 clipping、采样、截断、长度归一和题目难度偏差。"],
["W7", "2025.12", "稀疏检索", "V3.2 用学习型 indexer 选历史，再让主 attention 读取 top-k。"],
["W8", "2025.12", "Agent 环境", "推理从静态题目进入含工具、状态转移和 verifier 的交互数据闭环。"],
["W9", "2026.06", "异构长状态", "V4 用 CSA 与 HCA 处理不同时间尺度，并联合 mHC、Muon 和数值约束。"],
["W10", "2026.07", "K3 对照", "继承图必须允许没有箭头：相同的百万上下文目标，可以有完全不同的状态机器。"],
] as const;
export const deepseekBranches = [
["代码与专家", "DeepSeek-Coder → Coder-V2 → ESFT", "代码数据配方、continued pretraining 与只微调相关专家，说明 MoE 的价值不只在通用模型参数量。", "https://arxiv.org/abs/2406.11931"],
["数学与证明", "DeepSeekMath → Prover-V1.5 → Prover-V2", "从数学语料、GRPO 走到 formal proof feedback、subgoal decomposition 与可验证证明搜索。", "https://arxiv.org/abs/2504.21801"],
["视觉与压缩", "DeepSeek-VL/VL2 → Janus → OCR", "理解、生成与光学压缩形成另一条主干；它们不应被挤进纯文本 V2→V4 时间线。", "https://arxiv.org/abs/2412.10302"],
["系统实现", "DeepEP → DualPipe → DeepGEMM / FlashMLA", "论文里的稀疏计算、流水线、FP8 与 MLA 最终必须落到可调用的通信和 kernel 实现。", "https://github.com/deepseek-ai/DeepEP"],
["条件记忆", "Engram", "把可查表的静态模式从动态网络中分离，增加一条不同于 MoE 与 attention 的稀疏轴。", "https://arxiv.org/abs/2601.07372"],
] as const;
export const deepseekPaperChain = [
["01", "1991", "Adaptive Mixtures of Local Experts", "https://proceedings.neurips.cc/paper/1991/hash/59b90e1005a220e2ebc542eb9d950b1e-Abstract.html", "专家门控前史"],
["02", "2000", "Conditional Computation", "https://arxiv.org/abs/cs/0008102", "条件计算"],
["03", "2003", "A Neural Probabilistic Language Model", "https://www.jmlr.org/papers/v3/bengio03a.html", "Dense LM 坐标"],
["04", "2017", "Attention Is All You Need", "https://arxiv.org/abs/1706.03762", "Transformer 主干"],
["05", "2017", "Outrageously Large Neural Networks", "https://arxiv.org/abs/1701.06538", "稀疏 MoE"],
["06", "2017", "Proximal Policy Optimization", "https://arxiv.org/abs/1707.06347", "GRPO 对照"],
["07", "2018", "GPipe", "https://arxiv.org/abs/1811.06965", "Pipeline 前史"],
["08", "2018", "PipeDream", "https://arxiv.org/abs/1806.03377", "Pipeline schedule"],
["09", "2019", "Fast Transformer Decoding / MQA", "https://arxiv.org/abs/1911.02150", "KV 共享"],
["10", "2019", "Megatron-LM", "https://arxiv.org/abs/1909.08053", "模型并行"],
["11", "2019", "ZeRO", "https://arxiv.org/abs/1910.02054", "状态分片"],
["12", "2019", "RMSNorm", "https://arxiv.org/abs/1910.07467", "尺度控制"],
["13", "2020", "GShard", "https://arxiv.org/abs/2006.16668", "大规模 MoE"],
["14", "2020", "QK-Normalization", "https://arxiv.org/abs/2010.04245", "attention logit 稳定"],
["15", "2021", "Switch Transformers", "https://arxiv.org/abs/2101.03961", "coarse top-1 MoE"],
["16", "2021", "RoFormer / RoPE", "https://arxiv.org/abs/2104.09864", "MLA 位置分叉"],
["17", "2022", "ST-MoE", "https://arxiv.org/abs/2202.08906", "MoE 稳定性"],
["18", "2022", "DeepNet", "https://arxiv.org/abs/2203.00555", "深层残差"],
["19", "2022", "InstructGPT", "https://arxiv.org/abs/2203.02155", "SFT/RM/PPO 合同"],
["20", "2022", "FlashAttention", "https://arxiv.org/abs/2205.14135", "IO-aware exact attention"],
["21", "2022", "Process and Outcome Feedback", "https://arxiv.org/abs/2211.14275", "reasoning reward 前史"],
["22", "2022", "Self-Consistency", "https://arxiv.org/abs/2203.11171", "多采样聚合"],
["23", "2023", "GQA", "https://arxiv.org/abs/2305.13245", "KV 分组"],
["24", "2023", "Let's Verify Step by Step", "https://arxiv.org/abs/2305.20050", "verifier / PRM"],
["25", "2023", "Direct Preference Optimization", "https://arxiv.org/abs/2305.18290", "RL 外偏好路线"],
["26", "2023", "FlashAttention-2", "https://arxiv.org/abs/2307.08691", "attention kernel"],
["27", "2023", "PagedAttention / vLLM", "https://arxiv.org/abs/2309.06180", "KV 服务状态"],
["28", "2024", "DeepSeek LLM", "https://arxiv.org/abs/2401.02954", "Dense / scaling 基线"],
["29", "2024", "DeepSeek-Coder", "https://arxiv.org/abs/2401.14196", "代码数据旁支"],
["30", "2024", "DeepSeekMoE", "https://arxiv.org/abs/2401.06066", "细粒度 + shared"],
["31", "2024", "DeepSeekMath", "https://arxiv.org/abs/2402.03300", "数学数据 + GRPO"],
["32", "2024", "RLOO", "https://arxiv.org/abs/2402.14740", "critic-free 对照"],
["33", "2024", "DeepSeek-V2", "https://arxiv.org/abs/2405.04434", "MLA + MoE"],
["34", "2024", "Better & Faster LLMs via MTP", "https://arxiv.org/abs/2404.19737", "MTP 祖先"],
["35", "2024", "DeepSeek-Coder-V2", "https://arxiv.org/abs/2406.11931", "V2 continued pretrain"],
["36", "2024", "ESFT", "https://arxiv.org/abs/2407.01906", "专家特化微调"],
["37", "2024", "DeepSeek-Prover-V1.5", "https://arxiv.org/abs/2408.08152", "proof feedback RL"],
["38", "2024", "Hyper-Connections", "https://arxiv.org/abs/2409.19606", "mHC 前身"],
["39", "2024", "DeepSeek-V3", "https://arxiv.org/abs/2412.19437", "FP8 / DualPipe / MTP"],
["40", "2025", "DeepSeek-R1", "https://arxiv.org/abs/2501.12948", "R1-Zero / R1 / 蒸馏"],
["41", "2025", "Muon is Scalable for LLM Training", "https://arxiv.org/abs/2502.16982", "V4 optimizer 前史"],
["42", "2025", "DAPO", "https://arxiv.org/abs/2503.14476", "GRPO 工程修正"],
["43", "2025", "Understanding R1-Zero-Like Training", "https://arxiv.org/abs/2503.20783", "Dr.GRPO / 偏差"],
["44", "2025", "DeepSeek-Prover-V2", "https://arxiv.org/abs/2504.21801", "subgoal + RL"],
["45", "2025", "DeepEP", "https://github.com/deepseek-ai/DeepEP", "Expert Parallel kernel"],
["46", "2025", "DualPipe", "https://github.com/deepseek-ai/DualPipe", "V3/R1 pipeline 实现"],
["47", "2025", "DeepGEMM", "https://github.com/deepseek-ai/DeepGEMM", "FP8 GEMM 实现"],
["48", "2025", "DeepSeek-VL2", "https://arxiv.org/abs/2412.10302", "多模态理解旁支"],
["49", "2025", "Janus-Pro", "https://arxiv.org/abs/2501.17811", "统一理解/生成旁支"],
["50", "2025", "Kimi k1.5", "https://arxiv.org/abs/2501.12599", "同期 reasoning RL"],
["51", "2025", "Kimi K2", "https://arxiv.org/abs/2507.20534", "MLA / MoE / Muon 对照"],
["52", "2025", "Kimi Linear", "https://arxiv.org/abs/2510.26692", "KDA 前身"],
["53", "2025", "DeepSeek-V3.2", "https://arxiv.org/abs/2512.02556", "DSA + Agent"],
["54", "2025", "mHC", "https://arxiv.org/abs/2512.24880", "受约束 residual"],
["55", "2026", "Engram", "https://arxiv.org/abs/2601.07372", "条件记忆新稀疏轴"],
["56", "2026", "LatentMoE", "https://arxiv.org/abs/2601.18089", "K3 routed latent 前身"],
["57", "2026", "Attention Residuals", "https://arxiv.org/abs/2603.15031", "K3 深度路由"],
["58", "2026", "DeepSeek-V4", "https://arxiv.org/abs/2606.19348", "CSA / HCA / mHC / Muon"],
["59", "2026", "Kimi K3", "https://arxiv.org/abs/2607.24653", "对照锚点"],
["60", "2026", "Kimi K3 official repository", "https://github.com/MoonshotAI/Kimi-K3", "开放实现边界"],
] as const;
src/data/papers.ts
+54
View File
@@ -3896,6 +3896,60 @@ export const papers: Paper[] = [
contribution: "公开 Pre-LN decoder 训练配置与模型族，为深度、归一化和复现提供对照。",
verified: true,
},
{
year: 2024,
title: "DeepSeek-Coder: When the Large Language Model Meets Programming — The Rise of Code Intelligence",
url: "https://arxiv.org/abs/2401.14196",
topics: ["数据", "推理", "评测"],
contribution: "公开代码语料构造、fill-in-the-blank 目标与 1B–33B 模型族，建立 DeepSeek 代码能力旁支。",
spotlight: "DeepSeek",
verified: true,
},
{
year: 2024,
title: "DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence",
url: "https://arxiv.org/abs/2406.11931",
topics: ["数据", "MoE", "推理", "评测"],
contribution: "从 DeepSeek-V2 checkpoint continued pretrain，扩展代码语言、上下文和数学/代码能力。",
spotlight: "DeepSeek",
verified: true,
},
{
year: 2024,
title: "Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models",
url: "https://arxiv.org/abs/2407.01906",
topics: ["MoE", "后训练"],
contribution: "ESFT 依据专家相关性只更新任务相关 routed experts，研究 MoE 专长怎样进入低成本微调。",
spotlight: "DeepSeek",
verified: true,
},
{
year: 2024,
title: "DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search",
url: "https://arxiv.org/abs/2408.08152",
topics: ["推理", "后训练", "评测"],
contribution: "把 Lean proof assistant 的可验证反馈用于 formal proof RL，并结合 MCTS 扩展证明搜索。",
spotlight: "DeepSeek",
verified: true,
},
{
year: 2025,
title: "DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition",
url: "https://arxiv.org/abs/2504.21801",
topics: ["推理", "后训练", "评测"],
contribution: "用递归子目标分解构造 cold-start 数据，再以可验证证明奖励强化 formal reasoning。",
spotlight: "DeepSeek",
verified: true,
},
{
year: 2026,
title: "Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models",
url: "https://arxiv.org/abs/2601.07372",
topics: ["基础", "MoE", "数据", "推理服务"],
contribution: "Engram 把可查表的静态模式与动态计算分离，提出不同于 MoE 与 attention 的条件记忆稀疏轴。",
spotlight: "DeepSeek",
verified: true,
},
];
export const paperTopics: PaperTopic[] = [
src/pages/deepseek/index.astro
+953 -61
View File
File diff suppressed because it is too large Load Diff
src/pages/index.astro
+27 -1
View File
@@ -112,7 +112,7 @@ const paths = [
<div class="hero-stats">
<div><b>17</b><span>核心专题</span></div>
<div><b>151</b><span>K3 报告来源</span></div>
<div><b>480</b><span>关键论文索引</span></div>
<div><b>486</b><span>关键论文索引</span></div>
<div><b>47p</b><span>K3 技术报告</span></div>
</div>
</aside>
@@ -126,6 +126,22 @@ const paths = [
<section class="section compact release-section" id="new-chapters">
<div class="release-grid">
<a class="release-card deepseek-release" href="/deepseek/">
<div>
<p class="eyebrow"><span>NEW / DEEPSEEK ROUND 02</span> CAPACITY · STATE · SYSTEM · REASONING</p>
<h2>从 Dense 到百万上下文：每次创新都在偿还上一代最贵的一张账</h2>
<p>
用二十四张问题账和十次技术转向，从 DeepSeek LLM、MoE、V2 的 MLA 权重吸收，
走到 V3 的 FP8 / DualPipe / MTP、R1 与 DAPO / Dr.GRPO 反查、V3.2 Agent 环境和 V4 异构长状态。
</p>
</div>
<dl>
<div><dt>LINEAGE</dt><dd>1991 → 2026 · 10 次转向</dd></div>
<div><dt>NODES</dt><dd>60 个一手 / 官方节点</dd></div>
<div><dt>LAB</dt><dd>MoE · MLA · V3 协同 · RL 偏差</dd></div>
</dl>
<span class="release-arrow" aria-hidden="true">进入 DeepSeek 完整技术谱系 →</span>
</a>
<a class="release-card representation-release" href="/architecture/representation/">
<div>
<p class="eyebrow"><span>NEW / CHAPTER 03</span> TOKEN · POSITION · DEPTH · FFN</p>
@@ -599,6 +615,7 @@ const paths = [
transition: transform 180ms ease, border-color 180ms ease;
}
.deepseek-release,
.representation-release,
.inference-release,
.agent-release,
@@ -614,6 +631,14 @@ const paths = [
min-height: 510px;
}
.deepseek-release {
background:
radial-gradient(circle at 80% 18%, rgba(159, 91, 52, .24), transparent 30%),
radial-gradient(circle at 61% 72%, rgba(35, 86, 84, .2), transparent 28%),
repeating-linear-gradient(90deg, transparent 0 48px, rgba(159, 91, 52, .04) 48px 49px),
var(--paper-raised);
}
.representation-release {
background:
radial-gradient(circle at 82% 18%, rgba(159, 91, 52, 0.22), transparent 31%),
@@ -760,6 +785,7 @@ const paths = [
padding-bottom: 76px;
}
.deepseek-release,
.representation-release,
.inference-release,
.alignment-release,
src/pages/progress/index.astro
+6 -4
View File
@@ -15,7 +15,7 @@ const workstreams = [
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
{ label: "Scaling Laws", value: 74, next: "加入真实拟合复现、置信区间与更多模型族对照" },
{ label: "数据工程与预训练配方", value: 73, next: "逐图精读 FineWeb / DCLM，加入真实去重与 mixture traces" },
{ label: "DeepSeek 专题", value: 71, next: "补 R1 / DAPO 的逐图训练轨迹与复现对照" },
{ label: "DeepSeek 专题", value: 83, next: "加入真实专家负载、MLA kernel、RL 训练 traces 与独立复现" },
{ label: "指令微调与人类偏好", value: 75, next: "加入真实偏好分歧样本、RM 长度偏置与 PPO/DPO 小模型复现" },
{ label: "推理与测试时扩展", value: 76, next: "真实模型采样曲线、PRM 案例与逐篇图表精读" },
{ label: "工具使用与长程 Agent", value: 74, next: "补真实环境 traces、cross-harness 对照、Agent RL 训练曲线与安全案例" },
@@ -50,7 +50,7 @@ const workstreams = [
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-29 10:26 CST</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-29 11:10 CST</dd></div>
<div><dt>MODE</dt><dd>持续迭代，不锁死版本</dd></div>
</dl>
</div>
@@ -97,11 +97,12 @@ const workstreams = [
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全，包含先修依赖和三条贯穿案例。</p></article>
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
<article><span>✓</span><h3>五十五个原创交互视图</h3><p>K3、语言模型前史、Transformer、表示深度、DeepSeek、长上下文、MoE、推理、Agent、多模态，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>五十九个原创交互视图</h3><p>K3、语言模型前史、Transformer、表示深度、DeepSeek 四联实验、长上下文、MoE、推理、Agent、多模态，以及训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
<article><span>✓</span><h3>表示、位置与残差高速公路深度专题</h3><p>二十张问题账、66 个一手节点、DeepSeek/Kimi 双谱系，以及 Token—位置—Norm—Residual/FFN 四联实验。</p></article>
<article><span>✓</span><h3>DeepSeek 技术谱系二轮深读</h3><p>二十四张问题账、十次技术转向、60 个一手/官方节点，以及稀疏容量—MLA 缓存—V3 协同—RL 偏差四联实验。</p></article>
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
<article><span>✓</span><h3>数据工程深度专题</h3><p>十二张账、31 个一手节点、DeepSeek/Kimi 双谱系与流水线—去重—混合—改写四联实验。</p></article>
<article><span>✓</span><h3>长上下文深度专题</h3><p>五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。</p></article>
@@ -114,7 +115,7 @@ const workstreams = [
<article><span>✓</span><h3>原生多模态深度专题</h3><p>十六张账、55 个一手节点、DeepSeek 三分支、Kimi 三代 MoonViT，以及 Token—连接器—光学压缩—视觉闭环四联实验。</p></article>
<article><span>✓</span><h3>推理服务与低成本部署深度专题</h3><p>十八本账、62 个一手节点、DeepSeek V2→V4 与 Mooncake→K3 双谱系，以及显存—阶段—推测—集群四联实验。</p></article>
<article><span>✓</span><h3>评测、安全与“到底强不强”深度专题</h3><p>二十二张账、80 个一手节点、DeepSeek/K3 评测协议谱系，以及指标—Judge—污染—系统安全四联实验。</p></article>
<article><span>✓</span><h3>480 篇关键论文索引</h3><p>新增 output embedding、ELMo、BLT、Fixup、ReZero、Hyper-Connections、mHC、xPos、FIRE、LongRoPE 等 30 个表示与深度节点。</p></article>
<article><span>✓</span><h3>486 篇关键论文索引</h3><p>新增 DeepSeek-Coder/Coder-V2、ESFT、Prover-V1.5/V2 与 Engram 6 个 DeepSeek 旁支节点。</p></article>
<article><span>✓</span><h3>公开仓库与自托管发布</h3><p>源码公开到 git.k1412.top，网站由不可变镜像、Compose Manager 与 HTTPS 交付。</p></article>
</div>
</section>
@@ -129,6 +130,7 @@ const workstreams = [
</div>
<div class="queue-table">
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
<div><span>P0</span><strong>DeepSeek 三轮</strong><p>真实 expert load / MLA kernel → FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
<div><span>P0</span><strong>语言模型前史二轮</strong><p>Kneser–Ney / LSTM / Bahdanau 逐图 → 真实小语料复现 → tokenizer 公平性</p><em>可复现实验 + 逐图笔记</em></div>