feat: publish training systems deep dive

PROGRESS.md

@@ -14,7 +14,7 @@
 | 推理与测试时扩展 | 完成首版 | 76% | 真实模型采样曲线、PRM 案例与逐篇图表精读 |
 | 稀疏计算与 MoE | 完成首版 | 74% | 真实负载 traces 与专家特化案例 |
 | 长上下文专题 | 完成首版 | 72% | 真实模型配置、内核细节与失败案例 |
-| 大规模训练系统 | 研究完成首轮 | 38% | 实现内存—并行—流水线—通信四页签实验室 |
+| 大规模训练系统 | 完成首版 | 71% | 真实集群 traces、故障案例与精确 topology 配置 |
 | 引用与事实检查 | 进行中 | 54% | 自动化外链复查与来源等级扩展 |
 | 开源仓库 | 已完成首版 | 100% | 持续提交研究与网站迭代 |
 | k1412 部署 | 已完成首版 | 100% | 每轮发布保留不可变镜像与回滚点 |
@@ -28,10 +28,10 @@
 - [x] 提炼参考网站的编辑设计语言。
 - [x] 确认 `git.k1412.top` 为 Gitea/Forgejo 兼容服务且本机 HTTPS 凭据可用于既有仓库。
 - [x] 使用 Grok CLI 检索并形成约 95 篇一手论文的补充路线，主代理已回查关键来源。
-- [x] 完成 146 篇关键论文索引，覆盖 12 个专题与 Kimi/DeepSeek 聚光主线。
+- [x] 完成 166 篇关键论文索引，覆盖 12 个专题与 Kimi/DeepSeek 聚光主线。
 - [x] 完成可检索、可按专题筛选的论文库页面。
-- [x] 完成 K3、Transformer 基础、DeepSeek 谱系、长上下文、MoE 与推理六篇首版长文。
-- [x] 完成 K3 三轴架构、Self-Attention、DeepSeek 谱系、长上下文、MoE 路由与推理专题三页签等八个原创交互视图。
+- [x] 完成 K3、Transformer 基础、DeepSeek 谱系、长上下文、MoE、推理与训练系统七篇首版长文。
+- [x] 完成 K3 三轴架构、Self-Attention、DeepSeek 谱系、长上下文、MoE 路由、推理三页签与训练系统四页签等十二个原创交互视图。
 - [x] 完成长上下文首版：五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
 - [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文，并建立长上下文研究账本。
 - [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文，并建立 MoE 研究账本。
@@ -44,14 +44,16 @@
 - [x] 训练系统专题完成首轮证据账本：九张资源账、37 个一手节点、DeepSeek-V2/V3/V4 与 Kimi K2/K2.5/K3 系统谱系。
 - [x] 缓存并核验 18 篇训练系统核心论文，逐项校正 ZeRO 字节账、pipeline bubble、collective 口径、context parallel 与 MoonEP 上界。
 - [x] 将 Grok 产物降级为明确标注的未核验 leads；正式账本只采用回查一手论文后的结论。
-- [x] Astro 类型检查、生产构建、10 个内部路由和桌面/移动端视觉检查通过。
+- [x] 完成训练系统首版：15 节长文、37 个一手节点、12 类机制图与显存—网格—气泡—通信四页签实验室。
+- [x] 训练系统真实 Chrome 断言通过：OOM、无效 mesh、DualPipe 参数副本、overlap 与 MoonEP buffer 均正确响应。
+- [x] Astro 类型检查、生产构建、11 个内部路由和桌面/移动端视觉检查通过。
 - [x] 创建 `wuyang/llm-atlas` 公开仓库，匿名 API 确认 `private: false`。
-- [x] 本地生产镜像通过健康检查与全部 9 个页面路由烟雾测试。
+- [x] 上一生产镜像通过健康检查与当时全部页面路由烟雾测试。
 - [x] 通过 Unraid Compose Manager、Nginx Proxy Manager 与 HTTPS 发布首版。
 
 ## 正在进行
 
-- [ ] 大规模训练系统：实现四页签实验室与 ZeRO / Megatron → Expert/Context Parallel → DualPipe / MoonEP 长文。
+- [ ] 大规模训练系统二轮：真实集群 traces、故障恢复案例与精确 topology / kernel 配置。
 - [ ] 推理专题二轮：真实 pass@k 曲线、PRM 失败案例与逐篇图表精读。
 - [ ] 长上下文专题的真实模型配置对比、内核细节与失败案例二轮深化。
 - [ ] MoE 专题的真实集群 traces、专家特化案例与二轮外部证据。
@@ -79,6 +81,8 @@
 | 2026-07-29 | 并行度不再机械写成 `DP×TP×PP×EP×CP` | 先定义 device mesh、group overlap 与作用范围，再核算 world size |
 | 2026-07-29 | DeepSeek “近零通信”统一解释为特定 overlap 下的 exposed critical-path time | 防止把隐藏通信误写成没有传输字节 |
 | 2026-07-29 | Grok 训练系统包只保留为 `TRAINING_SYSTEMS_GROK_LEADS.md` | 候选论文、数字和公式不得越过一手证据账本直接进入站点 |
+| 2026-07-29 | 训练系统首版用四个独立实验闭环 | 显存、device mesh、pipeline 与通信不合成一个伪“系统总分” |
+| 2026-07-29 | 论文库随训练系统一手链扩充到 166 篇 | 新增 checkpoint、mixed precision、自动并行、FSDP、MoE/CP 与大集群系统节点 |
 
 ## 未决问题
 

README.md

@@ -17,8 +17,9 @@
 - 持续进度：[PROGRESS.md](./PROGRESS.md)
 - 证据与写作规范：[research/METHODOLOGY.md](./research/METHODOLOGY.md)
 
-当前里程碑包含 16 专题学习地图、146 篇关键论文索引、Kimi K3 完整导读、
-Transformer 基础、DeepSeek 技术谱系、长上下文与 MoE 深度专题，以及五张原创交互可视化。
+当前里程碑包含 16 专题学习地图、166 篇关键论文索引、Kimi K3 完整导读、
+Transformer 基础、DeepSeek 技术谱系、长上下文、MoE、推理与训练系统深度专题，
+以及覆盖核心机制的原创交互实验室。
 其余专题按进度账本持续扩建。
 
 ## 本地开发

package.json

@@ -11,7 +11,8 @@
     "check": "astro check",
     "check:site": "node scripts/check-site.mjs",
     "check:moe-browser": "node scripts/check-moe-browser.mjs",
-    "check:reasoning-browser": "node scripts/check-reasoning-browser.mjs"
+    "check:reasoning-browser": "node scripts/check-reasoning-browser.mjs",
+    "check:training-systems-browser": "node scripts/check-training-systems-browser.mjs"
   },
   "dependencies": {
     "@astrojs/sitemap": "3.7.3",

research/TRAINING_SYSTEMS_RESEARCH.md

@@ -1128,21 +1128,20 @@ K2 明确没有沿用 DualPipe：
 - [x] AgentENV 数字标为 K3 报告统计，不外推。
 - [ ] 所有 30+ 论文链接发布前做可达性检查。
 - [ ] 页面每个数字回链到邻近的一手来源。
-- [ ] 互动公式写单元测试与浏览器断言。
-- [ ] desktop / mobile / reduced-motion / keyboard 全部验证。
+- [x] 互动公式写单元测试与浏览器断言。
+- [x] desktop / mobile / reduced-motion / keyboard 全部验证。
 - [ ] immutable image、NAS、VPS、NPM、HTTPS、公开 Forgejo 全链验证。
 
 ## 首版页面完成闸门
 
-- [ ] 初学者能在不懂分布式系统的情况下读懂“为什么单卡放不下”。
-- [ ] 能用 byte ledger 解释参数、梯度、master weights 与 optimizer state。
-- [ ] 能区分 DP / TP / PP / EP / SP / CP 的切分对象。
-- [ ] 能解释为什么同样通信字节数仍可能有不同 wall time。
-- [ ] 能从时间轴上读出 GPipe、1F1B、ZeroBubble 与 DualPipe 差异。
-- [ ] 能解释 Ulysses、Ring 与 USP 的数据布局变化。
-- [ ] 能解释 DeepSeek-V3 的系统协同，而不是只背 DualPipe 名字。
-- [ ] 能解释为什么 K2 主动不采用 DualPipe。
-- [ ] 能沿 K2 → K2.5 → K3 读出内存与调度演进。
-- [ ] 能操作四个互动实验并触发至少一个 invalid / OOM / imbalance 情景。
+- [x] 初学者能在不懂分布式系统的情况下读懂“为什么单卡放不下”。
+- [x] 能用 byte ledger 解释参数、梯度、master weights 与 optimizer state。
+- [x] 能区分 DP / TP / PP / EP / SP / CP 的切分对象。
+- [x] 能解释为什么同样通信字节数仍可能有不同 wall time。
+- [x] 能从时间轴上读出 GPipe、1F1B、ZeroBubble 与 DualPipe 差异。
+- [x] 能解释 Ulysses、Ring 与 USP 的数据布局变化。
+- [x] 能解释 DeepSeek-V3 的系统协同，而不是只背 DualPipe 名字。
+- [x] 能解释为什么 K2 主动不采用 DualPipe。
+- [x] 能沿 K2 → K2.5 → K3 读出内存与调度演进。
+- [x] 能操作四个互动实验并触发至少一个 invalid / OOM / imbalance 情景。
 - [ ] 论文阅读链、类型检查、链接、浏览器与容器验证全部通过。
-

scripts/check-training-systems-browser.mjs

@@ -0,0 +1,243 @@
+import { writeFileSync } from "node:fs";
+
+const cdpPort = process.env.CDP_PORT ?? "9225";
+const baseUrl = process.env.SITE_URL ?? "http://127.0.0.1:4324";
+const pages = await fetch(`http://127.0.0.1:${cdpPort}/json/list`).then((response) => response.json());
+const page = pages.find((entry) => entry.type === "page");
+if (!page) throw new Error(`CDP ${cdpPort} 没有可用页面`);
+
+const socket = new WebSocket(page.webSocketDebuggerUrl);
+await new Promise((resolve, reject) => {
+  socket.addEventListener("open", resolve, { once: true });
+  socket.addEventListener("error", reject, { once: true });
+});
+
+let nextId = 0;
+const pending = new Map();
+const exceptions = [];
+socket.addEventListener("message", (event) => {
+  const message = JSON.parse(event.data);
+  if (message.id && pending.has(message.id)) {
+    const { resolve, reject } = pending.get(message.id);
+    pending.delete(message.id);
+    if (message.error) reject(new Error(message.error.message));
+    else resolve(message.result);
+  }
+  if (message.method === "Runtime.exceptionThrown") exceptions.push(message.params.exceptionDetails.text);
+});
+
+const command = (method, params = {}) => new Promise((resolve, reject) => {
+  const id = ++nextId;
+  pending.set(id, { resolve, reject });
+  socket.send(JSON.stringify({ id, method, params }));
+});
+const pause = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
+const evaluate = async (expression) => {
+  const result = await command("Runtime.evaluate", { expression, returnByValue: true, awaitPromise: true });
+  if (result.exceptionDetails) throw new Error(result.exceptionDetails.text);
+  return result.result.value;
+};
+const navigate = async (path) => {
+  await command("Page.navigate", { url: `${baseUrl}${path}` });
+  for (let attempt = 0; attempt < 50; attempt += 1) {
+    await pause(100);
+    if (await evaluate("document.readyState === 'complete'")) return;
+  }
+  throw new Error(`${path} 加载超时`);
+};
+const screenshot = async (path, full = false) => {
+  const params = { format: "png", captureBeyondViewport: full };
+  if (full) {
+    const metrics = await command("Page.getLayoutMetrics");
+    params.clip = {
+      x: 0,
+      y: 0,
+      width: metrics.cssContentSize.width,
+      height: metrics.cssContentSize.height,
+      scale: 1,
+    };
+  }
+  const result = await command("Page.captureScreenshot", params);
+  writeFileSync(path, Buffer.from(result.data, "base64"));
+};
+
+await command("Page.enable");
+await command("Runtime.enable");
+await command("Emulation.setDeviceMetricsOverride", {
+  width: 1440,
+  height: 1100,
+  deviceScaleFactor: 1,
+  mobile: false,
+});
+await navigate("/training-systems/");
+await screenshot("/tmp/llm-atlas-training-systems-desktop.png");
+
+const memory = await evaluate(`(() => {
+  const root = document.querySelector("[data-systems-lab]");
+  root.querySelector('[data-memory-preset="single"]').click();
+  const single = {
+    total: root.querySelector("[data-memory-total]").textContent,
+    verdict: root.querySelector("[data-memory-verdict]").textContent,
+    overflow: root.querySelector("[data-hbm-card]").classList.contains("is-overflow"),
+  };
+  root.querySelector('[data-memory-preset="k3"]').click();
+  return {
+    single,
+    k3: {
+      total: root.querySelector("[data-memory-total]").textContent,
+      verdict: root.querySelector("[data-memory-verdict]").textContent,
+      offload: root.querySelector('[data-memory-metric="offload"]').textContent,
+      formula: root.querySelector("[data-memory-formula]").textContent,
+    },
+  };
+})()`);
+
+const mesh = await evaluate(`(() => {
+  const root = document.querySelector("[data-systems-lab]");
+  root.querySelector('[data-system-tab="mesh"]').click();
+  root.querySelector('[data-mesh-preset="v3"]').click();
+  const v3 = {
+    equation: root.querySelector("[data-mesh-equation]").textContent,
+    status: root.querySelector("[data-mesh-capacity]").textContent,
+    invalid: root.querySelector("[data-mesh-status]").classList.contains("is-invalid"),
+  };
+  root.querySelector('[data-mesh-preset="invalid"]').click();
+  return {
+    v3,
+    invalid: {
+      equation: root.querySelector("[data-mesh-equation]").textContent,
+      status: root.querySelector("[data-mesh-capacity]").textContent,
+      invalid: root.querySelector("[data-mesh-status]").classList.contains("is-invalid"),
+    },
+  };
+})()`);
+
+const pipeline = await evaluate(`(() => {
+  const root = document.querySelector("[data-systems-lab]");
+  root.querySelector('[data-system-tab="pipeline"]').click();
+  root.querySelector('[data-schedule="1f1b"]').click();
+  const one = {
+    copies: root.querySelector('[data-pipeline-metric="copies"]').textContent,
+    reading: root.querySelector("[data-pipeline-reading]").textContent,
+    rows: root.querySelectorAll(".timeline-row").length,
+  };
+  root.querySelector('[data-schedule="dual"]').click();
+  return {
+    one,
+    dual: {
+      copies: root.querySelector('[data-pipeline-metric="copies"]').textContent,
+      reading: root.querySelector("[data-pipeline-reading]").textContent,
+      note: root.querySelector("[data-pipeline-note]").textContent,
+    },
+  };
+})()`);
+
+const communication = await evaluate(`(() => {
+  const root = document.querySelector("[data-systems-lab]");
+  root.querySelector('[data-system-tab="communication"]').click();
+  const overlap = root.querySelector('[data-comm-input="overlap"]');
+  overlap.value = "0";
+  overlap.dispatchEvent(new Event("input", { bubbles: true }));
+  const exposed = {
+    bytes: root.querySelector('[data-comm-metric="bytes"]').textContent,
+    time: root.querySelector('[data-comm-metric="exposed"]').textContent,
+  };
+  overlap.value = "95";
+  overlap.dispatchEvent(new Event("input", { bubbles: true }));
+  return {
+    exposed,
+    hidden: {
+      bytes: root.querySelector('[data-comm-metric="bytes"]').textContent,
+      time: root.querySelector('[data-comm-metric="exposed"]').textContent,
+    },
+    moe: {
+      deep: root.querySelector('[data-moe-metric="deep-buffer"]').textContent,
+      moon: root.querySelector('[data-moe-metric="moon-buffer"]').textContent,
+      slots: root.querySelector('[data-moe-metric="slots"]').textContent,
+      bars: root.querySelectorAll("[data-expert-rank]").length,
+    },
+  };
+})()`);
+
+const layout = await evaluate(`(() => {
+  const nav = document.querySelector(".top-nav");
+  const meta = document.querySelector(".header-meta");
+  return {
+    documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
+    navGap: Number((meta.getBoundingClientRect().left - nav.getBoundingClientRect().right).toFixed(1)),
+    navLinks: document.querySelectorAll(".top-nav a").length,
+    articleSections: document.querySelectorAll(".article-section").length,
+    paperLinks: document.querySelectorAll(".paper-chain a").length,
+    labTabs: document.querySelectorAll("[data-system-tab]").length,
+  };
+})()`);
+
+await evaluate(`(() => {
+  document.documentElement.style.scrollBehavior = "auto";
+  document.querySelector("[data-systems-lab]").scrollIntoView({ block: "start", behavior: "instant" });
+})()`);
+await pause(150);
+await screenshot("/tmp/llm-atlas-training-systems-lab-desktop.png");
+
+await command("Emulation.setDeviceMetricsOverride", {
+  width: 390,
+  height: 844,
+  deviceScaleFactor: 1,
+  mobile: true,
+});
+await navigate("/training-systems/");
+await screenshot("/tmp/llm-atlas-training-systems-mobile-closed.png");
+const mobile = await evaluate(`(() => {
+  const toggle = document.querySelector("#menu-toggle");
+  toggle.click();
+  return {
+    documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
+    menuVisible: getComputedStyle(toggle).display !== "none",
+    menuOpen: toggle.getAttribute("aria-expanded"),
+    title: document.querySelector("h1").innerText,
+  };
+})()`);
+
+await command("Emulation.setDeviceMetricsOverride", {
+  width: 1440,
+  height: 1100,
+  deviceScaleFactor: 1,
+  mobile: false,
+});
+await navigate("/");
+const home = await evaluate(`({
+  documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
+  releaseCards: document.querySelectorAll(".release-card").length,
+  firstRelease: document.querySelector(".release-card h2").textContent,
+  navLinks: document.querySelectorAll(".top-nav a").length,
+})`);
+await evaluate(`document.querySelector("#new-chapters").scrollIntoView({ block: "start", behavior: "instant" })`);
+await pause(100);
+await screenshot("/tmp/llm-atlas-home-training-systems-release.png");
+
+const report = { memory, mesh, pipeline, communication, layout, mobile, home, exceptions };
+console.log(JSON.stringify(report, null, 2));
+
+const numeric = (value) => Number.parseFloat(value);
+const failures = [];
+if (!memory.single.overflow || !memory.single.verdict.includes("OOM")) failures.push("单卡 OOM 场景未触发");
+if (!memory.k3.formula.includes("14P") || !memory.k3.verdict.includes("可以放入")) failures.push("K3 分层存储预设异常");
+if (mesh.v3.invalid || !mesh.v3.equation.includes("2,048")) failures.push("V3 设备网格异常");
+if (!mesh.invalid.invalid || !mesh.invalid.status.includes("相差")) failures.push("无效设备网格没有被拒绝");
+if (pipeline.one.rows !== 4 || pipeline.one.copies !== "1×") failures.push("1F1B 时间轴异常");
+if (pipeline.dual.copies !== "2×" || !pipeline.dual.note.includes("near-zero")) failures.push("DualPipe 边界解释缺失");
+if (communication.exposed.bytes !== communication.hidden.bytes) failures.push("overlap 错误改变了通信字节");
+if (numeric(communication.exposed.time) <= numeric(communication.hidden.time)) failures.push("overlap 没有降低 exposed time");
+if (communication.moe.bars !== 12 || !communication.moe.slots.includes("E/R")) failures.push("MoonEP 对照异常");
+if (layout.articleSections !== 16 || layout.paperLinks !== 37 || layout.labTabs !== 4) failures.push("章节、论文或实验数量异常");
+if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
+if (layout.navGap < 0) failures.push(`桌面导航碰撞：${layout.navGap}px`);
+if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
+if (home.releaseCards !== 4 || !home.firstRelease.includes("一万张 GPU")) failures.push("首页训练系统新章入口异常");
+if (exceptions.length) failures.push(`浏览器脚本异常：${exceptions.join("; ")}`);
+
+socket.close();
+if (failures.length) {
+  failures.forEach((failure) => console.error(`- ${failure}`));
+  process.exit(1);
+}

src/components/SiteHeader.astro

@@ -13,6 +13,7 @@ const items = [
   { id: "moe", href: "/moe/", label: "MoE" },
   { id: "long-context", href: "/long-context/", label: "长上下文" },
   { id: "reasoning", href: "/reasoning/", label: "推理" },
+  { id: "training-systems", href: "/training-systems/", label: "训练系统" },
   { id: "papers", href: "/papers/", label: "论文库" },
   { id: "progress", href: "/progress/", label: "进度" },
 ];

src/data/chapters.ts

@@ -121,16 +121,16 @@ export const chapters: Chapter[] = [
   },
   {
     number: "08",
-    slug: "systems/training",
+    slug: "training-systems",
     title: "大规模训练系统",
     kicker: "DISTRIBUTED TRAINING",
     question: "一个 2.8T 模型如何摊到成千上万张卡上？",
-    summary: "从 ZeRO、Megatron 和五类并行，走到 DeepSeek DualPipe/DeepEP 与 K3 MoonEP。",
-    status: "queued",
-    progress: 12,
-    papers: 18,
+    summary: "用九张资源账串起 ZeRO、Megatron、Expert/Context Parallel、DeepSeek DualPipe 与 K3 MoonEP/百万 Token RL。",
+    status: "published",
+    progress: 71,
+    papers: 37,
     prerequisites: ["04", "06"],
-    highlights: ["并行维度", "通信重叠", "MoonEP"],
+    highlights: ["显存字节账", "流水线与通信", "MoonEP"],
   },
   {
     number: "09",

src/data/papers.ts

@@ -590,6 +590,54 @@ export const papers: Paper[] = [
     contribution: "AdamW 将权重衰减与梯度更新正确解耦。",
     verified: true,
   },
+  {
+    year: 2012,
+    title: "Large Scale Distributed Deep Networks",
+    url: "https://arxiv.org/abs/1206.5533",
+    topics: ["训练系统"],
+    contribution: "DistBelief 与参数服务器奠定早期大规模分布式深度学习系统路线。",
+    verified: true,
+  },
+  {
+    year: 2014,
+    title: "One Weird Trick for Parallelizing Convolutional Neural Networks",
+    url: "https://arxiv.org/abs/1404.5997",
+    topics: ["训练系统"],
+    contribution: "系统比较数据并行、模型并行及其通信边界。",
+    verified: true,
+  },
+  {
+    year: 2016,
+    title: "Training Deep Nets with Sublinear Memory Cost",
+    url: "https://arxiv.org/abs/1604.06174",
+    topics: ["训练系统"],
+    contribution: "用 activation checkpointing 以额外计算换取 O(√n) feature-map memory。",
+    verified: true,
+  },
+  {
+    year: 2017,
+    title: "Mixed Precision Training",
+    url: "https://arxiv.org/abs/1710.03740",
+    topics: ["低精度", "训练系统"],
+    contribution: "建立 FP32 master weights、loss scaling 与 FP32 accumulation 配方。",
+    verified: true,
+  },
+  {
+    year: 2018,
+    title: "PipeDream: Fast and Efficient Pipeline Parallel DNN Training",
+    url: "https://arxiv.org/abs/1806.03377",
+    topics: ["训练系统"],
+    contribution: "以 1F1B、异步 pipeline 与 weight stashing 提高 stage 利用率。",
+    verified: true,
+  },
+  {
+    year: 2018,
+    title: "Mesh-TensorFlow: Deep Learning for Supercomputers",
+    url: "https://arxiv.org/abs/1811.02084",
+    topics: ["训练系统"],
+    contribution: "把 tensor dimension 的布局声明映射到逻辑 device mesh。",
+    verified: true,
+  },
   {
     year: 2019,
     title: "GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism",
@@ -614,6 +662,86 @@ export const papers: Paper[] = [
     contribution: "分片优化器、梯度与参数，消除数据并行状态冗余。",
     verified: true,
   },
+  {
+    year: 2021,
+    title: "Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM",
+    url: "https://arxiv.org/abs/2104.04473",
+    topics: ["训练系统"],
+    contribution: "组合 TP、PP、DP，并提出 interleaved pipeline schedule。",
+    verified: true,
+  },
+  {
+    year: 2021,
+    title: "ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning",
+    url: "https://arxiv.org/abs/2104.07857",
+    topics: ["训练系统"],
+    contribution: "把 GPU、CPU 与 NVMe 组成可带宽感知的异构训练内存层。",
+    verified: true,
+  },
+  {
+    year: 2021,
+    title: "GSPMD: General and Scalable Parallelization for ML Computation Graphs",
+    url: "https://arxiv.org/abs/2105.04663",
+    topics: ["训练系统"],
+    contribution: "用统一 SPMD 表达和 sharding propagation 扩展模型并行。",
+    verified: true,
+  },
+  {
+    year: 2021,
+    title: "Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines",
+    url: "https://arxiv.org/abs/2107.06925",
+    topics: ["训练系统"],
+    contribution: "从 pipeline 两端注入 micro-batches，降低同步流水线气泡。",
+    verified: true,
+  },
+  {
+    year: 2022,
+    title: "DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training",
+    url: "https://arxiv.org/abs/2201.05596",
+    topics: ["MoE", "训练系统"],
+    contribution: "组合 data、expert 与 tensor parallel，系统优化 MoE 训练和推理。",
+    verified: true,
+  },
+  {
+    year: 2022,
+    title: "Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning",
+    url: "https://arxiv.org/abs/2201.12023",
+    topics: ["训练系统"],
+    contribution: "用代价模型联合搜索 operator 内和 operator 间并行。",
+    verified: true,
+  },
+  {
+    year: 2022,
+    title: "Reducing Activation Recomputation in Large Transformer Models",
+    url: "https://arxiv.org/abs/2205.05198",
+    topics: ["训练系统"],
+    contribution: "以 Megatron sequence parallel 与 selective recomputation 降低激活内存和重算开销。",
+    verified: true,
+  },
+  {
+    year: 2022,
+    title: "Tutel: Adaptive Mixture-of-Experts at Scale",
+    url: "https://arxiv.org/abs/2206.03382",
+    topics: ["MoE", "训练系统"],
+    contribution: "让 MoE parallelism、dispatch 和 kernels 自适应模型与硬件配置。",
+    verified: true,
+  },
+  {
+    year: 2022,
+    title: "MegaBlocks: Efficient Sparse Training with Mixture-of-Experts",
+    url: "https://arxiv.org/abs/2211.15841",
+    topics: ["MoE", "训练系统"],
+    contribution: "用 block-sparse kernels 支持 dropless 动态 expert workloads。",
+    verified: true,
+  },
+  {
+    year: 2023,
+    title: "PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel",
+    url: "https://arxiv.org/abs/2304.11277",
+    topics: ["训练系统"],
+    contribution: "总结 fully sharded 参数、梯度、状态与 prefetch 的工程实践。",
+    verified: true,
+  },
   {
     year: 2023,
     title: "DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models",
@@ -622,6 +750,39 @@ export const papers: Paper[] = [
     contribution: "沿 attention head 做 sequence parallel，扩展极长序列训练。",
     verified: true,
   },
+  {
+    year: 2024,
+    title: "Zero Bubble Pipeline Parallelism",
+    url: "https://arxiv.org/abs/2401.10241",
+    topics: ["训练系统"],
+    contribution: "拆分 input-gradient 与 weight-gradient，并在内存约束下搜索同步零气泡调度。",
+    verified: true,
+  },
+  {
+    year: 2024,
+    title: "MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs",
+    url: "https://arxiv.org/abs/2402.15627",
+    topics: ["训练系统"],
+    contribution: "系统讨论 10K+ GPU 集群的并行、网络优化与可靠性。",
+    verified: true,
+  },
+  {
+    year: 2024,
+    title: "USP: A Unified Sequence Parallelism Approach for Long Context Generative AI",
+    url: "https://arxiv.org/abs/2405.07719",
+    topics: ["长上下文", "训练系统"],
+    contribution: "在二维 mesh 上组合 Ulysses All-to-All 与 Ring P2P。",
+    verified: true,
+  },
+  {
+    year: 2025,
+    title: "DeepEP: An Efficient Expert-Parallel Communication Library",
+    url: "https://github.com/deepseek-ai/DeepEP",
+    topics: ["MoE", "训练系统"],
+    contribution: "提供面向 NVLink/RDMA 的高吞吐与低延迟 expert dispatch/combine kernels。",
+    spotlight: "DeepSeek",
+    verified: true,
+  },
   {
     year: 2024,
     title: "Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving",
@@ -1055,7 +1216,7 @@ export const papers: Paper[] = [
     year: 2025,
     title: "Kimi K2: Open Agentic Intelligence",
     url: "https://arxiv.org/abs/2507.20534",
-    topics: ["MoE", "Agent", "后训练", "推理"],
+    topics: ["MoE", "训练系统", "Agent", "后训练", "推理"],
     contribution: "开放 1T MoE Agent 模型，以 verifiable gym 与 self-critique rubric 做 joint RL。",
     spotlight: "Kimi",
     verified: true,
@@ -1064,7 +1225,7 @@ export const papers: Paper[] = [
     year: 2026,
     title: "Kimi K2.5: Visual Agentic Intelligence",
     url: "https://arxiv.org/abs/2602.02276",
-    topics: ["Agent", "多模态", "推理"],
+    topics: ["训练系统", "Agent", "多模态", "推理"],
     contribution: "把视觉、工具使用和并行 Agent Swarm 纳入统一后训练。",
     spotlight: "Kimi",
     verified: true,

src/pages/index.astro

@@ -10,6 +10,7 @@ const routes: Record<string, string> = {
   moe: "/moe/",
   "long-context": "/long-context/",
   reasoning: "/reasoning/",
+  "training-systems": "/training-systems/",
 };
 
 const paths = [
@@ -80,16 +81,17 @@ const paths = [
           <a class="button" href="/moe/">MoE 专题</a>
           <a class="button" href="/long-context/">长上下文专题</a>
           <a class="button" href="/reasoning/">推理专题</a>
+          <a class="button" href="/training-systems/">训练系统专题</a>
         </div>
       </div>
       <aside class="hero-aside" aria-label="项目统计">
         <span>RESEARCH CUTOFF</span>
-        <strong>28 · 07 · 2026</strong>
+        <strong>29 · 07 · 2026</strong>
         <p>首版持续建设中，所有动态结论带日期</p>
         <div class="hero-stats">
           <div><b>16</b><span>核心专题</span></div>
           <div><b>151</b><span>K3 报告来源</span></div>
-          <div><b>146</b><span>关键论文索引</span></div>
+          <div><b>166</b><span>关键论文索引</span></div>
           <div><b>47p</b><span>K3 技术报告</span></div>
         </div>
       </aside>
@@ -103,6 +105,22 @@ const paths = [
 
   <section class="section compact release-section" id="new-chapters">
     <div class="release-grid">
+      <a class="release-card training-release" href="/training-systems/">
+        <div>
+          <p class="eyebrow"><span>NEW / CHAPTER 08</span> LARGE-SCALE TRAINING SYSTEMS</p>
+          <h2>一万张 GPU，为什么仍可能有一半在等？</h2>
+          <p>
+            用模型状态、激活、计算划分、气泡、collective、专家、上下文、数值与可靠性九张账，
+            从 ZeRO、Megatron 一路走到 DeepSeek DualPipe、Kimi MoonEP 与百万 Token Agentic RL。
+          </p>
+        </div>
+        <dl>
+          <div><dt>LINEAGE</dt><dd>2012 → 2026</dd></div>
+          <div><dt>PAPERS</dt><dd>37 个一手节点</dd></div>
+          <div><dt>LAB</dt><dd>显存 · 网格 · 气泡 · 通信</dd></div>
+        </dl>
+        <span class="release-arrow" aria-hidden="true">进入训练系统专题 →</span>
+      </a>
       <a class="release-card reasoning-release" href="/reasoning/">
         <div>
           <p class="eyebrow"><span>NEW / CHAPTER 11</span> REASONING & TEST-TIME SCALING</p>
@@ -384,9 +402,20 @@ const paths = [
       transition: transform 180ms ease, border-color 180ms ease;
     }
 
+    .training-release,
     .reasoning-release {
       grid-column: 1 / -1;
       min-height: 510px;
+    }
+
+    .training-release {
+      background:
+        radial-gradient(circle at 82% 18%, rgba(66, 111, 101, 0.19), transparent 30%),
+        repeating-linear-gradient(90deg, transparent 0 88px, rgba(94, 125, 117, 0.055) 88px 89px),
+        var(--paper-raised);
+    }
+
+    .reasoning-release {
       background:
         radial-gradient(circle at 82% 18%, rgba(56, 91, 128, 0.17), transparent 30%),
         radial-gradient(circle at 63% 72%, rgba(150, 93, 58, 0.11), transparent 28%),
@@ -455,6 +484,7 @@ const paths = [
         padding-bottom: 76px;
       }
 
+      .training-release,
       .reasoning-release {
         grid-column: auto;
       }

src/pages/moe/index.astro

@@ -683,7 +683,7 @@ const paperChain = [
         <div class="next-links">
           <a class="button primary" href="/k3/#moe">回到 K3：在整机架构中定位 Stable LatentMoE →</a>
           <a class="button" href="/deepseek/">DeepSeek 完整论文谱系</a>
-          <a class="button" href="/papers/">搜索全部 146 篇论文</a>
+          <a class="button" href="/papers/">搜索全部 166 篇论文</a>
         </div>
       </section>
     </article>

src/pages/progress/index.astro

@@ -15,6 +15,7 @@ const workstreams = [
   { label: "推理与测试时扩展", value: 76, next: "真实模型采样曲线、PRM 案例与逐篇图表精读" },
   { label: "稀疏计算与 MoE", value: 74, next: "补充真实集群 traces 与专家特化案例" },
   { label: "长上下文专题", value: 72, next: "加入更多论文逐图笔记与真实模型配置对比" },
+  { label: "大规模训练系统", value: 71, next: "补真实集群 traces、故障案例与精确 topology 配置" },
   { label: "引用与事实检查", value: 54, next: "自动化外链复查与来源等级扩展" },
   { label: "开源与部署", value: 100, next: "每轮保留不可变镜像、提交与回滚点" },
 ];
@@ -39,7 +40,7 @@ const workstreams = [
         <div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
         <div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
         <div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
-        <div><dt>UPDATED</dt><dd>2026-07-29 00:38 CST</dd></div>
+        <div><dt>UPDATED</dt><dd>2026-07-29 01:19 CST</dd></div>
         <div><dt>MODE</dt><dd>持续迭代，不锁死版本</dd></div>
       </dl>
     </div>
@@ -49,7 +50,7 @@ const workstreams = [
     <div class="section-heading">
       <div>
         <p class="eyebrow"><span>01</span> WORKSTREAMS</p>
-        <h2>十条工作流同时推进，但不混淆“有页面”和“已核验”</h2>
+        <h2>十一条工作流同时推进，但不混淆“有页面”和“已核验”</h2>
       </div>
       <p class="section-lead">
         内容首版优先打通全局脉络；随后每轮迭代选择一个专题推进到论文/工程层，并做独立事实复核。
@@ -86,12 +87,13 @@ const workstreams = [
       <article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
       <article><span>✓</span><h3>16 专题知识图</h3><p>从语言模型基础到评测安全，包含先修依赖和三条贯穿案例。</p></article>
       <article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
-      <article><span>✓</span><h3>八个原创交互视图</h3><p>K3、注意力、DeepSeek、长上下文、MoE，以及推理预算/GRPO/MOPD 三页签实验。</p></article>
-      <article><span>✓</span><h3>六篇首版长文</h3><p>K3 导读、Transformer 基础、DeepSeek 谱系、长上下文、MoE 与推理专题。</p></article>
+      <article><span>✓</span><h3>十二个原创交互视图</h3><p>K3、注意力、DeepSeek、长上下文、MoE、推理三页签，以及训练系统四页签实验。</p></article>
+      <article><span>✓</span><h3>七篇首版长文</h3><p>K3 导读、Transformer 基础、DeepSeek 谱系、长上下文、MoE、推理与训练系统专题。</p></article>
       <article><span>✓</span><h3>长上下文深度专题</h3><p>五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。</p></article>
       <article><span>✓</span><h3>MoE 深度专题</h3><p>六张账、19 篇一手论文、DeepSeek/K3 主线与路由—容量—通信交互实验室。</p></article>
       <article><span>✓</span><h3>推理深度专题</h3><p>八张账、30 篇一手论文链、DeepSeek/Kimi 双主线与三页签互动实验室。</p></article>
-      <article><span>✓</span><h3>146 篇关键论文索引</h3><p>覆盖 12 个专题，支持全文搜索、标签筛选与 Kimi/DeepSeek 聚光主线。</p></article>
+      <article><span>✓</span><h3>训练系统深度专题</h3><p>九张账、37 个一手节点、DeepSeek/Kimi 双谱系与显存—网格—气泡—通信实验室。</p></article>
+      <article><span>✓</span><h3>166 篇关键论文索引</h3><p>覆盖 12 个专题，支持全文搜索、标签筛选与 Kimi/DeepSeek 聚光主线。</p></article>
       <article><span>✓</span><h3>公开仓库与自托管发布</h3><p>源码公开到 git.k1412.top，网站由不可变镜像、Compose Manager 与 HTTPS 交付。</p></article>
     </div>
   </section>
@@ -106,7 +108,7 @@ const workstreams = [
     </div>
     <div class="queue-table">
       <div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
-      <div><span>P0</span><strong>大规模训练系统</strong><p>ZeRO / Megatron → Expert/Context Parallel → DualPipe / MoonEP</p><em>显存与通信计算器</em></div>
+      <div><span>P0</span><strong>大规模训练系统二轮</strong><p>真实集群 traces → 故障恢复 → 精确 topology / kernel 配置</p><em>案例库 + 实测边界</em></div>
       <div><span>P1</span><strong>长上下文二轮深化</strong><p>真实模型配置 → 内核细节 → 长上下文评测与失败案例</p><em>配置比较器 + 逐图论文笔记</em></div>
       <div><span>P1</span><strong>MoE 二轮深化</strong><p>真实负载 traces → 专家特化可解释性 → 共享专家语义</p><em>案例库 + 集群证据</em></div>
       <div><span>P1</span><strong>推理二轮深化</strong><p>真实 pass@k 曲线 → PRM 失败案例 → 逐篇图表精读</p><em>案例库 + 真实 traces</em></div>
@@ -150,6 +152,7 @@ const workstreams = [
       <div><time>2026-07-28</time><b>双重开放许可</b><p>代码 MIT，原创文字与图 CC BY-SA 4.0。</p></div>
       <div><time>2026-07-28</time><b>自托管交付</b><p>源码公开到 git.k1412.top，网站部署到 k1412 私有基础设施。</p></div>
       <div><time>2026-07-29</time><b>推理按八张账组织</b><p>把答案、覆盖、选择、过程、预算、优化、分布与系统证据分开核算。</p></div>
+      <div><time>2026-07-29</time><b>训练系统按九张账组织</b><p>把模型状态、激活、并行、气泡、通信、专家、上下文、数值与可靠性分开核算。</p></div>
     </div>
   </section>
 

src/pages/reasoning/index.astro

@@ -699,7 +699,7 @@ const paperChain = [
         <h2>不要按模型榜单读，按“旧方法留下了哪堵墙”读</h2>
         <p>
           下面 30 篇构成本专题首版主干。课程中的机制、公式与数字优先回到这些一手来源；
-          更完整的 146 篇跨专题索引位于论文库。
+          更完整的 166 篇跨专题索引位于论文库。
         </p>
         <div class="paper-chain">
           {paperChain.map(([year, title, href, note], index) => (