feat: publish scaling laws chapter

This commit is contained in:
wuyang
2026-07-29 02:17:31 +08:00
parent 4c4b5b0c7c
commit eff0e4c942
17 changed files with 4228 additions and 33 deletions
+14 -9
View File
@@ -6,17 +6,17 @@
| 工作流 | 状态 | 完成度 | 下一检查点 |
|---|---:|---:|---|
| 研究框架与规范 | 进行中 | 83% | 完成 Scaling Laws 逐篇图表/实验精读 |
| 研究框架与规范 | 进行中 | 83% | Scaling Laws 二轮拟合复现与逐图精读 |
| 网站设计系统 | 进行中 | 89% | 打印样式与更多通用可视化组件 |
| Kimi K3 深读 | 进行中 | 64% | 扩写 scaling / pre-training / infra 逐图笔记 |
| Kimi K3 深读 | 进行中 | 66% | 扩写 pre-training / infra 逐图笔记 |
| Transformer 基础 | 进行中 | 52% | 矩阵形状动画与手算练习 |
| Scaling Laws | 研究中 | 31% | 逐篇图表精读、拟合复现与四页签实验 |
| Scaling Laws | 完成首版 | 74% | 真实拟合复现、置信区间与更多模型族对照 |
| DeepSeek 专题 | 进行中 | 71% | 补 R1 / DAPO 的逐图训练轨迹与复现对照 |
| 推理与测试时扩展 | 完成首版 | 76% | 真实模型采样曲线、PRM 案例与逐篇图表精读 |
| 稀疏计算与 MoE | 完成首版 | 74% | 真实负载 traces 与专家特化案例 |
| 长上下文专题 | 完成首版 | 72% | 真实模型配置、内核细节与失败案例 |
| 大规模训练系统 | 完成首版 | 71% | 真实集群 traces、故障案例与精确 topology 配置 |
| 引用与事实检查 | 进行中 | 55% | Scaling Laws 数字、版本与外推边界复核 |
| 引用与事实检查 | 进行中 | 57% | 自动化外链复查与来源等级扩展 |
| 开源仓库 | 已完成首版 | 100% | 持续提交研究与网站迭代 |
| k1412 部署 | 已完成首版 | 100% | 每轮发布保留不可变镜像与回滚点 |
@@ -29,10 +29,10 @@
- [x] 提炼参考网站的编辑设计语言。
- [x] 确认 `git.k1412.top` 为 Gitea/Forgejo 兼容服务且本机 HTTPS 凭据可用于既有仓库。
- [x] 使用 Grok CLI 检索并形成约 95 篇一手论文的补充路线,主代理已回查关键来源。
- [x] 完成 166 篇关键论文索引,覆盖 12 个专题与 Kimi/DeepSeek 聚光主线。
- [x] 完成 182 篇关键论文索引,覆盖 12 个专题与 Kimi/DeepSeek 聚光主线。
- [x] 完成可检索、可按专题筛选的论文库页面。
- [x] 完成 K3、Transformer 基础、DeepSeek 谱系、长上下文、MoE、推理与训练系统篇首版长文。
- [x] 完成 K3 三轴架构、Self-Attention、DeepSeek 谱系、长上下文、MoE 路由、推理三页签训练系统四页签等十个原创交互视图。
- [x] 完成 K3、Transformer 基础、DeepSeek 谱系、Scaling Laws、长上下文、MoE、推理与训练系统篇首版长文。
- [x] 完成 K3 三轴架构、Self-Attention、DeepSeek 谱系、长上下文、MoE 路由、推理三页签,以及训练系统与 Scaling 各四页签等十个原创交互视图。
- [x] 完成长上下文首版:五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。
- [x] 核验 FlashAttention、DeepSeek-V2/V3.2/V4、Kimi Linear/K3 等六份论文原文,并建立长上下文研究账本。
- [x] 核验 Switch、ST-MoE、DeepSeekMoE、Loss-Free、V3、LatentMoE 与 K3 原文,并建立 MoE 研究账本。
@@ -47,17 +47,20 @@
- [x] 将 Grok 产物降级为明确标注的未核验 leads;正式账本只采用回查一手论文后的结论。
- [x] 完成训练系统首版:15 节长文、37 个一手节点、12 类机制图与显存—网格—气泡—通信四页签实验室。
- [x] 训练系统真实 Chrome 断言通过:OOM、无效 mesh、DualPipe 参数副本、overlap 与 MoonEP buffer 均正确响应。
- [x] Astro 类型检查、生产构建、11 个内部路由和桌面/移动端视觉检查通过。
- [x] Astro 类型检查、生产构建、12 个内部路由、464 个站内引用和桌面/移动端视觉检查通过。
- [x] 训练系统首版以提交 `c6306fe`、不可变镜像 `20260728T172750Z-c6306fe` 发布;NAS、VPS/Tailscale、NPM、DNS、HTTPS、证书与生产 Chrome 全链路通过。
- [x] 创建 `wuyang/llm-atlas` 公开仓库,匿名 API 确认 `private: false`
- [x] 当前生产镜像通过健康检查、11 个页面路由与关键资源烟雾测试。
- [x] 通过 Unraid Compose Manager、Nginx Proxy Manager 与 HTTPS 发布首版。
- [x] 启动 Scaling Laws 专题:缓存 22 篇新的一手论文,建立九张账、DeepSeek / Kimi 双谱系与四实验首轮证据框架。
- [x] 使用 Grok Headless 扩展 Scaling 候选路线,并纠正其对 K3 正式报告存在性的错误;线索与正式账本分离。
- [x] 完成 Scaling Laws 首版:16 节长文、29 个一手节点、九张账与曲面—部署—复用—涌现四联实验。
- [x] Scaling 真实 Chrome 断言通过:IsoFLOP 谷底、生命周期最优点、重复数据衰减和指标阶跃均正确响应。
- [x] 对 Scaling、训练系统、推理与 MoE 执行全量浏览器回归;12 项桌面导航、移动菜单和首页新章入口无回归。
## 正在进行
- [ ] Scaling Laws 二轮精读:逐篇图表、拟合公式、置信区间、外推失败与 DeepSeek / K3 对照。
- [ ] Scaling Laws 二轮精读:真实拟合复现、逐篇图表、置信区间、外推失败与更多模型族对照。
- [ ] 大规模训练系统二轮:真实集群 traces、故障恢复案例与精确 topology / kernel 配置。
- [ ] 推理专题二轮:真实 pass@k 曲线、PRM 失败案例与逐篇图表精读。
- [ ] 长上下文专题的真实模型配置对比、内核细节与失败案例二轮深化。
@@ -91,6 +94,8 @@
| 2026-07-29 | 训练系统首版用不可变镜像 `20260728T172750Z-c6306fe` 发布 | 保留 `12010→8080`、NPM host 31 / cert 41 与公开 Forgejo;11 路由、关键资源和四页签生产交互全部复核 |
| 2026-07-29 | Scaling Laws 拆成观测量、模型、数据、算术、配比、配方、外推、经济目标与能力阶段九张账 | 防止把 `20 TPP``6ND`、MoE 总参、部署成本和“涌现”混成一条万能曲线 |
| 2026-07-29 | Grok 对 K3 报告存在性的错误保留在未核验 leads | 正式账本回到 arXiv `2607.24653` 与本地 47 页报告,展示来源分级为何必要 |
| 2026-07-29 | Scaling 首版用 29 个一手节点和四个独立实验闭环 | IsoFLOP、训练—部署、数据复用与指标涌现分开建模,不合成伪“规模总分” |
| 2026-07-29 | 论文库随 Scaling 一手链扩充到 182 篇 | 新增早期跨任务幂律、Gopher、数据受限、过训练、部署经济、复现和 task ladder 节点 |
## 未决问题
+2 -2
View File
@@ -17,8 +17,8 @@
- 持续进度:[PROGRESS.md](./PROGRESS.md)
- 证据与写作规范:[research/METHODOLOGY.md](./research/METHODOLOGY.md)
当前里程碑包含 16 专题学习地图、166 篇关键论文索引、Kimi K3 完整导读、
Transformer 基础、DeepSeek 技术谱系、长上下文、MoE、推理与训练系统深度专题,
当前里程碑包含 16 专题学习地图、182 篇关键论文索引、Kimi K3 完整导读、
Transformer 基础、DeepSeek 技术谱系、Scaling Laws、长上下文、MoE、推理与训练系统深度专题,
以及覆盖核心机制的原创交互实验室。
其余专题按进度账本持续扩建。
+2 -1
View File
@@ -12,7 +12,8 @@
"check:site": "node scripts/check-site.mjs",
"check:moe-browser": "node scripts/check-moe-browser.mjs",
"check:reasoning-browser": "node scripts/check-reasoning-browser.mjs",
"check:training-systems-browser": "node scripts/check-training-systems-browser.mjs"
"check:training-systems-browser": "node scripts/check-training-systems-browser.mjs",
"check:scaling-browser": "node scripts/check-scaling-browser.mjs"
},
"dependencies": {
"@astrojs/sitemap": "3.7.3",
+1 -1
View File
@@ -177,7 +177,7 @@ if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentO
}
if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`);
if (!mobile.menuVisible) failures.push("移动端菜单按钮未显示");
if (home.releaseCards !== 3) failures.push(`首页新章卡数量异常:${home.releaseCards}`);
if (home.releaseCards !== 5) failures.push(`首页新章卡数量异常:${home.releaseCards}`);
if (exceptions.length) failures.push(`浏览器脚本异常:${exceptions.join("; ")}`);
socket.close();
+1 -1
View File
@@ -289,7 +289,7 @@ if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentO
}
if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`);
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
if (home.releaseCards !== 3 || !home.firstRelease.includes("多想一会儿")) failures.push("首页推理新章入口异常");
if (home.releaseCards !== 5 || !home.firstRelease.includes("大一点")) failures.push("首页新章入口异常");
if (exceptions.length) failures.push(`浏览器脚本异常:${exceptions.join("; ")}`);
socket.close();
+286
View File
@@ -0,0 +1,286 @@
import { writeFileSync } from "node:fs";
const cdpPort = process.env.CDP_PORT ?? "9225";
const baseUrl = process.env.SITE_URL ?? "http://127.0.0.1:4324";
const pages = await fetch(`http://127.0.0.1:${cdpPort}/json/list`).then((response) => response.json());
const page = pages.find((entry) => entry.type === "page");
if (!page) throw new Error(`CDP ${cdpPort} 没有可用页面`);
const socket = new WebSocket(page.webSocketDebuggerUrl);
await new Promise((resolve, reject) => {
socket.addEventListener("open", resolve, { once: true });
socket.addEventListener("error", reject, { once: true });
});
let nextId = 0;
const pending = new Map();
const exceptions = [];
socket.addEventListener("message", (event) => {
const message = JSON.parse(event.data);
if (message.id && pending.has(message.id)) {
const { resolve, reject } = pending.get(message.id);
pending.delete(message.id);
if (message.error) reject(new Error(message.error.message));
else resolve(message.result);
}
if (message.method === "Runtime.exceptionThrown") {
exceptions.push(message.params.exceptionDetails.exception?.description ?? message.params.exceptionDetails.text);
}
});
const command = (method, params = {}) => new Promise((resolve, reject) => {
const id = ++nextId;
pending.set(id, { resolve, reject });
socket.send(JSON.stringify({ id, method, params }));
});
const pause = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
const evaluate = async (expression) => {
const result = await command("Runtime.evaluate", { expression, returnByValue: true, awaitPromise: true });
if (result.exceptionDetails) throw new Error(result.exceptionDetails.exception?.description ?? result.exceptionDetails.text);
return result.result.value;
};
const navigate = async (path) => {
await command("Page.navigate", { url: `${baseUrl}${path}` });
for (let attempt = 0; attempt < 50; attempt += 1) {
await pause(100);
if (await evaluate("document.readyState === 'complete'")) return;
}
throw new Error(`${path} 加载超时`);
};
const screenshot = async (path, full = false) => {
const params = { format: "png", captureBeyondViewport: full };
if (full) {
const metrics = await command("Page.getLayoutMetrics");
params.clip = {
x: 0,
y: 0,
width: metrics.cssContentSize.width,
height: metrics.cssContentSize.height,
scale: 1,
};
}
const result = await command("Page.captureScreenshot", params);
writeFileSync(path, Buffer.from(result.data, "base64"));
};
await command("Page.enable");
await command("Runtime.enable");
await command("Emulation.setDeviceMetricsOverride", {
width: 1440,
height: 1100,
deviceScaleFactor: 1,
mobile: false,
});
await navigate("/scaling/");
await screenshot("/tmp/llm-atlas-scaling-desktop.png");
const surface = await evaluate(`(() => {
const root = document.querySelector("[data-scaling-lab]");
const initial = {
model: root.querySelector('[data-surface-metric="model"]').textContent,
data: root.querySelector('[data-surface-metric="data"]').textContent,
gap: root.querySelector('[data-surface-metric="gap"]').textContent,
path: root.querySelector("[data-surface-path]").getAttribute("d").length,
};
const tpp = root.querySelector('[data-surface-input="tpp"]');
tpp.value = "500";
tpp.dispatchEvent(new Event("input", { bubbles: true }));
return {
initial,
extreme: {
gap: root.querySelector('[data-surface-metric="gap"]').textContent,
reading: root.querySelector("[data-surface-reading]").textContent,
label: root.querySelector("[data-surface-current] text").textContent,
},
};
})()`);
const serve = await evaluate(`(() => {
const root = document.querySelector("[data-scaling-lab]");
root.querySelector('[data-scaling-tab="serve"]').click();
root.querySelector('[data-serve-preset="lab"]').click();
const lab = {
train: root.querySelector('[data-serve-metric="train"]').textContent,
total: root.querySelector('[data-serve-metric="total"]').textContent,
share: root.querySelector('[data-serve-metric="share"]').textContent,
};
root.querySelector('[data-serve-preset="mass"]').click();
return {
lab,
mass: {
train: root.querySelector('[data-serve-metric="train"]').textContent,
total: root.querySelector('[data-serve-metric="total"]').textContent,
share: root.querySelector('[data-serve-metric="share"]').textContent,
reading: root.querySelector("[data-serve-reading]").textContent,
candidates: root.querySelectorAll(".candidate-row").length,
},
};
})()`);
const data = await evaluate(`(() => {
const root = document.querySelector("[data-scaling-lab]");
root.querySelector('[data-scaling-tab="data"]').click();
root.querySelector('[data-data-preset="unique"]').click();
const unique = {
seen: root.querySelector('[data-data-metric="seen"]').textContent,
effective: root.querySelector('[data-data-metric="effective"]').textContent,
next: root.querySelector('[data-data-metric="next"]').textContent,
};
root.querySelector('[data-data-preset="twenty"]').click();
return {
unique,
twenty: {
seen: root.querySelector('[data-data-metric="seen"]').textContent,
effective: root.querySelector('[data-data-metric="effective"]').textContent,
next: root.querySelector('[data-data-metric="next"]').textContent,
reading: root.querySelector("[data-data-reading]").textContent,
bars: root.querySelectorAll(".epoch-bars > i").length,
more: root.querySelector(".epoch-more")?.textContent ?? "",
},
};
})()`);
const emergence = await evaluate(`(() => {
const root = document.querySelector("[data-scaling-lab]");
root.querySelector('[data-scaling-tab="emergence"]').click();
const scale = root.querySelector('[data-emergence-input="scale"]');
const measure = () => ({
token: root.querySelector('[data-emergence-metric="token"]').textContent,
threshold: root.querySelector('[data-emergence-metric="threshold"]').textContent,
exact: root.querySelector('[data-emergence-metric="exact"]').textContent,
});
scale.value = "59";
scale.dispatchEvent(new Event("input", { bubbles: true }));
const below = measure();
scale.value = "60";
scale.dispatchEvent(new Event("input", { bubbles: true }));
const above = measure();
const firstTab = root.querySelector('[data-scaling-tab="surface"]');
firstTab.focus();
firstTab.dispatchEvent(new KeyboardEvent("keydown", { key: "End", bubbles: true }));
return {
below,
above,
keyboard: {
selected: root.querySelector('[data-scaling-tab][aria-selected="true"]').dataset.scalingTab,
visible: root.querySelector("[data-scaling-view]:not([hidden])").dataset.scalingView,
},
paths: [...root.querySelectorAll("[data-emergence-line]")].map((path) => path.getAttribute("d").length),
};
})()`);
const layout = await evaluate(`(() => {
const nav = document.querySelector(".top-nav");
const meta = document.querySelector(".header-meta");
return {
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
navGap: Number((meta.getBoundingClientRect().left - nav.getBoundingClientRect().right).toFixed(1)),
navLinks: document.querySelectorAll(".top-nav a").length,
articleSections: document.querySelectorAll(".article-section").length,
paperLinks: document.querySelectorAll(".paper-chain a").length,
labTabs: document.querySelectorAll("[data-scaling-tab]").length,
views: document.querySelectorAll("[data-scaling-view]").length,
};
})()`);
await evaluate(`(() => {
document.documentElement.style.scrollBehavior = "auto";
document.querySelector("[data-scaling-lab]").scrollIntoView({ block: "start", behavior: "instant" });
})()`);
await pause(150);
await screenshot("/tmp/llm-atlas-scaling-lab-desktop.png");
await command("Emulation.setDeviceMetricsOverride", {
width: 390,
height: 844,
deviceScaleFactor: 1,
mobile: true,
});
await navigate("/scaling/");
await evaluate(`document.querySelector("[data-scaling-lab]").scrollIntoView({ block: "start", behavior: "instant" })`);
await pause(100);
await screenshot("/tmp/llm-atlas-scaling-lab-mobile.png");
const mobile = await evaluate(`(() => {
const toggle = document.querySelector("#menu-toggle");
toggle.click();
return {
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
menuVisible: getComputedStyle(toggle).display !== "none",
menuOpen: toggle.getAttribute("aria-expanded"),
mobileLinks: document.querySelectorAll("#mobile-nav a").length,
selectedTab: document.querySelector('[data-scaling-tab][aria-selected="true"]').dataset.scalingTab,
};
})()`);
await command("Emulation.setDeviceMetricsOverride", {
width: 1440,
height: 1100,
deviceScaleFactor: 1,
mobile: false,
});
await navigate("/");
const home = await evaluate(`({
documentOverflow: document.documentElement.scrollWidth - document.documentElement.clientWidth,
releaseCards: document.querySelectorAll(".release-card").length,
firstRelease: document.querySelector(".release-card h2").textContent,
firstHref: document.querySelector(".release-card").getAttribute("href"),
paperCount: document.querySelector(".hero-stats div:nth-child(3) b").textContent,
navLinks: document.querySelectorAll(".top-nav a").length,
})`);
await evaluate(`document.querySelector("#new-chapters").scrollIntoView({ block: "start", behavior: "instant" })`);
await pause(100);
await screenshot("/tmp/llm-atlas-home-scaling-release.png");
const report = { surface, serve, data, emergence, layout, mobile, home, exceptions };
console.log(JSON.stringify(report, null, 2));
const numeric = (value) => Number.parseFloat(value);
const modelSize = (value) => Number.parseFloat(value);
const scaledCount = (value) => {
const number = Number.parseFloat(value);
if (value.includes("T")) return number * 1000;
if (value.includes("M")) return number / 1000;
return number;
};
const failures = [];
if (!surface.initial.model || !surface.initial.data || surface.initial.path < 1000) failures.push("IsoFLOP 曲线或最优点未渲染");
if (numeric(surface.extreme.gap) <= numeric(surface.initial.gap) || !surface.extreme.reading.includes("模型偏小")) {
failures.push("极端 TPP 没有产生预期的 compute-allocation loss gap");
}
if (serve.mass.candidates !== 8) failures.push("训练—部署候选模型数量异常");
if (modelSize(serve.mass.total) >= modelSize(serve.mass.train) || !serve.mass.reading.includes("更小模型")) {
failures.push("海量调用没有把生命周期最优点推向更小模型");
}
if (data.unique.seen !== data.unique.effective) failures.push("单遍数据的 seen 与 effective 不相等");
if (scaledCount(data.twenty.effective) >= scaledCount(data.twenty.seen) || numeric(data.twenty.next) >= 20) {
failures.push("数据枯竭预设没有体现复用边际衰减");
}
if (data.twenty.bars !== 16 || !data.twenty.more.includes("+4")) failures.push("20 epoch 可视化截断异常");
if (!emergence.below.threshold.includes("FAIL") || !emergence.above.threshold.includes("PASS")) {
failures.push("连续能力跨过二值评分线时没有出现离散阶跃");
}
if (Math.abs(numeric(emergence.above.token) - numeric(emergence.below.token)) > 2) {
failures.push("相邻规模的底层连续能力变化异常过大");
}
if (emergence.keyboard.selected !== "emergence" || emergence.keyboard.visible !== "emergence") {
failures.push("实验页签键盘导航异常");
}
if (emergence.paths.some((length) => length < 500)) failures.push("涌现多指标曲线未完整渲染");
if (layout.articleSections !== 16 || layout.paperLinks !== 29 || layout.labTabs !== 4 || layout.views !== 4) {
failures.push("章节、论文或实验数量异常");
}
if (layout.navLinks !== 12 || mobile.mobileLinks !== 12 || home.navLinks !== 12) failures.push("全站导航未同步 Scaling");
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`);
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
if (home.releaseCards !== 5 || !home.firstRelease.includes("大一点") || home.firstHref !== "/scaling/") {
failures.push("首页 Scaling 新章入口异常");
}
if (home.paperCount !== "182") failures.push(`首页论文总数异常:${home.paperCount}`);
if (exceptions.length) failures.push(`浏览器脚本异常:${exceptions.join("; ")}`);
socket.close();
if (failures.length) {
failures.forEach((failure) => console.error(`- ${failure}`));
process.exit(1);
}
+1 -1
View File
@@ -233,7 +233,7 @@ if (layout.articleSections !== 16 || layout.paperLinks !== 37 || layout.labTabs
if (layout.documentOverflow > 0 || mobile.documentOverflow > 0 || home.documentOverflow > 0) failures.push("页面存在横向溢出");
if (layout.navGap < 0) failures.push(`桌面导航碰撞:${layout.navGap}px`);
if (!mobile.menuVisible || mobile.menuOpen !== "true") failures.push("移动端菜单不可用");
if (home.releaseCards !== 4 || !home.firstRelease.includes("一万张 GPU")) failures.push("首页训练系统新章入口异常");
if (home.releaseCards !== 5 || !home.firstRelease.includes("大一点")) failures.push("首页新章入口异常");
if (exceptions.length) failures.push(`浏览器脚本异常:${exceptions.join("; ")}`);
socket.close();
File diff suppressed because it is too large Load Diff
+1
View File
@@ -10,6 +10,7 @@ const items = [
{ id: "k3", href: "/k3/", label: "K3 解剖" },
{ id: "deepseek", href: "/deepseek/", label: "DeepSeek" },
{ id: "foundations", href: "/foundations/", label: "基础原理" },
{ id: "scaling", href: "/scaling/", label: "Scaling" },
{ id: "moe", href: "/moe/", label: "MoE" },
{ id: "long-context", href: "/long-context/", label: "长上下文" },
{ id: "reasoning", href: "/reasoning/", label: "推理" },
+5 -5
View File
@@ -73,12 +73,12 @@ export const chapters: Chapter[] = [
title: "Scaling Laws:规模为什么有效",
kicker: "SCALING",
question: "多大模型、多少数据、多少计算才是划算的?",
summary: "区分参数量、激活参数、训练计算和推理预算;理解 Kaplan、Chinchilla 与 K3 的双轴扩展。",
status: "researching",
progress: 22,
papers: 11,
summary: "用九张账区分参数、数据、计算、配方、外推与部署经济;理解 Kaplan、Chinchilla、DeepSeek 与 K3。",
status: "published",
progress: 74,
papers: 29,
prerequisites: ["01", "02"],
highlights: ["幂律", "计算最优", "测试时扩展"],
highlights: ["幂律曲面", "计算最优", "四联实验"],
},
{
number: "05",
+128
View File
@@ -574,6 +574,134 @@ export const papers: Paper[] = [
contribution: "开放 3T Token 语料与完整数据处理工具。",
verified: true,
},
{
year: 2017,
title: "Deep Learning Scaling is Predictable, Empirically",
url: "https://arxiv.org/abs/1712.00409",
topics: ["Scaling"],
contribution: "跨语言、视觉、语音与翻译任务观察数据规模和误差之间的可预测幂律。",
verified: true,
},
{
year: 2018,
title: "An Empirical Model of Large-Batch Training",
url: "https://arxiv.org/abs/1812.06162",
topics: ["Scaling", "训练系统"],
contribution: "以 gradient noise scale 连接关键 batch size、优化步数和训练计算效率。",
verified: true,
},
{
year: 2020,
title: "Scaling Laws for Autoregressive Generative Modeling",
url: "https://arxiv.org/abs/2010.14701",
topics: ["Scaling", "多模态"],
contribution: "把自回归幂律扩展到图像、视频、数学和语言,显示指数依赖数据模态。",
verified: true,
},
{
year: 2021,
title: "Scaling Language Models: Methods, Analysis & Insights from Training Gopher",
url: "https://arxiv.org/abs/2112.11446",
topics: ["Scaling", "评测"],
contribution: "以 280B Gopher 系列研究规模、任务表现、训练实践与能力风险的分化。",
verified: true,
},
{
year: 2022,
title: "Broken Neural Scaling Laws",
url: "https://arxiv.org/abs/2210.14891",
topics: ["Scaling"],
contribution: "用平滑折断幂律刻画平台、拐点、double descent 与斜率改变。",
verified: true,
},
{
year: 2023,
title: "Are Emergent Abilities of Large Language Models a Mirage?",
url: "https://arxiv.org/abs/2304.15004",
topics: ["Scaling", "评测"],
contribution: "展示不连续指标可把平滑的底层改进显示成突然涌现,要求同时检查评分函数。",
verified: true,
},
{
year: 2023,
title: "Scaling Data-Constrained Language Models",
url: "https://arxiv.org/abs/2305.16264",
topics: ["Scaling"],
contribution: "区分唯一数据量与重复看到的 Token,系统研究数据受限时的多轮训练。",
verified: true,
},
{
year: 2024,
title: "Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws",
url: "https://arxiv.org/abs/2401.00448",
topics: ["Scaling", "训练系统"],
contribution: "把推理需求加入训练目标,说明生命周期最优模型可能小于纯训练计算最优模型。",
verified: true,
},
{
year: 2024,
title: "Language Models Scale Reliably With Over-Training and on Downstream Tasks",
url: "https://arxiv.org/abs/2403.08540",
topics: ["Scaling", "评测"],
contribution: "检验超过 Chinchilla 配比后的 loss 与下游趋势,并给出多项可预测缩放结果。",
verified: true,
},
{
year: 2024,
title: "Chinchilla Scaling: A Replication Attempt",
url: "https://arxiv.org/abs/2404.10102",
topics: ["Scaling"],
contribution: "复查原论文三种估计方法,质疑部分精确参数与置信区间但未推翻近似等比分配结论。",
verified: true,
},
{
year: 2024,
title: "More Compute Is What You Need",
url: "https://arxiv.org/abs/2404.19484",
topics: ["Scaling"],
contribution: "提出训练计算分配会改变可迁移性的争议性假说;在课程中仅作为待检验观点。",
verified: true,
},
{
year: 2024,
title: "Resolving Discrepancies in Compute-Optimal Scaling of Language Models",
url: "https://arxiv.org/abs/2406.19146",
topics: ["Scaling"],
contribution: "从 FLOPs 口径、warmup 与随规模调参解释 Kaplan—Chinchilla 配比差异。",
verified: true,
},
{
year: 2024,
title: "Optimization Hyper-parameter Laws for Large Language Models",
url: "https://arxiv.org/abs/2409.04777",
topics: ["Scaling", "训练系统"],
contribution: "研究最优学习率等优化超参数怎样随模型规模和训练长度变化。",
verified: true,
},
{
year: 2024,
title: "Establishing Task Scaling Laws via Compute-Efficient Model Ladders",
url: "https://arxiv.org/abs/2412.04403",
topics: ["Scaling", "评测"],
contribution: "先由模型/数据预测任务 loss,再由任务 loss 预测能力,以小模型梯队降低外推成本。",
verified: true,
},
{
year: 2025,
title: "Overtrained Language Models Are Harder to Fine-Tune",
url: "https://arxiv.org/abs/2503.19206",
topics: ["Scaling", "后训练"],
contribution: "显示更长预训练可能降低后续微调可塑性,分开 base loss 与最终产品目标。",
verified: true,
},
{
year: 2026,
title: "Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws",
url: "https://arxiv.org/abs/2606.06888",
topics: ["Scaling"],
contribution: "以 SoftQ 和 masked-input regularization 更新有限唯一数据下的重复训练研究。",
verified: true,
},
{
year: 2014,
title: "Adam: A Method for Stochastic Optimization",
+28 -1
View File
@@ -7,6 +7,7 @@ import { chapters, statusLabel } from "@/data/chapters";
const routes: Record<string, string> = {
roadmap: "/roadmap/",
foundations: "/foundations/",
scaling: "/scaling/",
moe: "/moe/",
"long-context": "/long-context/",
reasoning: "/reasoning/",
@@ -78,6 +79,7 @@ const paths = [
<a class="button primary" href="/roadmap/">选择学习路径 <span aria-hidden="true">↓</span></a>
<a class="button" href="/k3/">直接解剖 K3</a>
<a class="button" href="/deepseek/">DeepSeek 专题</a>
<a class="button" href="/scaling/">Scaling Laws 专题</a>
<a class="button" href="/moe/">MoE 专题</a>
<a class="button" href="/long-context/">长上下文专题</a>
<a class="button" href="/reasoning/">推理专题</a>
@@ -91,7 +93,7 @@ const paths = [
<div class="hero-stats">
<div><b>16</b><span>核心专题</span></div>
<div><b>151</b><span>K3 报告来源</span></div>
<div><b>166</b><span>关键论文索引</span></div>
<div><b>182</b><span>关键论文索引</span></div>
<div><b>47p</b><span>K3 技术报告</span></div>
</div>
</aside>
@@ -105,6 +107,22 @@ const paths = [
<section class="section compact release-section" id="new-chapters">
<div class="release-grid">
<a class="release-card scaling-release" href="/scaling/">
<div>
<p class="eyebrow"><span>NEW / CHAPTER 04</span> SCALING LAWS</p>
<h2>“大一点就会更强”,远远不够指导一次训练</h2>
<p>
用观测量、模型、数据、算术、配比、配方、外推、经济目标与能力阶段九张账,
从早期经验幂律、Kaplan、Chinchilla 与复现争议,一路走到 DeepSeek scaling study 和 Kimi K3。
</p>
</div>
<dl>
<div><dt>LINEAGE</dt><dd>2017 → 2026</dd></div>
<div><dt>PAPERS</dt><dd>29 个一手节点</dd></div>
<div><dt>LAB</dt><dd>曲面 · 部署 · 重复 · 涌现</dd></div>
</dl>
<span class="release-arrow" aria-hidden="true">进入 Scaling Laws 专题 →</span>
</a>
<a class="release-card training-release" href="/training-systems/">
<div>
<p class="eyebrow"><span>NEW / CHAPTER 08</span> LARGE-SCALE TRAINING SYSTEMS</p>
@@ -402,12 +420,20 @@ const paths = [
transition: transform 180ms ease, border-color 180ms ease;
}
.scaling-release,
.training-release,
.reasoning-release {
grid-column: 1 / -1;
min-height: 510px;
}
.scaling-release {
background:
radial-gradient(circle at 82% 18%, rgba(159, 91, 52, 0.18), transparent 30%),
repeating-linear-gradient(135deg, transparent 0 84px, rgba(159, 91, 52, 0.045) 84px 85px),
var(--paper-raised);
}
.training-release {
background:
radial-gradient(circle at 82% 18%, rgba(66, 111, 101, 0.19), transparent 30%),
@@ -484,6 +510,7 @@ const paths = [
padding-bottom: 76px;
}
.scaling-release,
.training-release,
.reasoning-release {
grid-column: auto;
+1 -1
View File
@@ -683,7 +683,7 @@ const paperChain = [
<div class="next-links">
<a class="button primary" href="/k3/#moe">回到 K3:在整机架构中定位 Stable LatentMoE →</a>
<a class="button" href="/deepseek/">DeepSeek 完整论文谱系</a>
<a class="button" href="/papers/">搜索全部 166 篇论文</a>
<a class="button" href="/papers/">搜索全部 182 篇论文</a>
</div>
</section>
</article>
+13 -9
View File
@@ -7,16 +7,17 @@ const published = chapters.filter((chapter) => chapter.status === "published").l
const researching = chapters.filter((chapter) => ["researching", "drafting"].includes(chapter.status)).length;
const workstreams = [
{ label: "研究框架与规范", value: 82, next: "给推理专题补逐篇图表/实验精读层级" },
{ label: "研究框架与规范", value: 83, next: "给 Scaling 与推理专题补逐篇图表/实验精读层级" },
{ label: "网站设计系统", value: 89, next: "打印样式与更多通用可视化组件" },
{ label: "Kimi K3 深读", value: 64, next: "扩写 scaling / pre-training / infra 逐图笔记" },
{ label: "Kimi K3 深读", value: 66, next: "扩写 pre-training / infra 逐图笔记" },
{ label: "Transformer 基础", value: 52, next: "加入矩阵形状动画与手算练习" },
{ label: "Scaling Laws", value: 74, next: "加入真实拟合复现、置信区间与更多模型族对照" },
{ label: "DeepSeek 专题", value: 71, next: "补 R1 / DAPO 的逐图训练轨迹与复现对照" },
{ label: "推理与测试时扩展", value: 76, next: "真实模型采样曲线、PRM 案例与逐篇图表精读" },
{ label: "稀疏计算与 MoE", value: 74, next: "补充真实集群 traces 与专家特化案例" },
{ label: "长上下文专题", value: 72, next: "加入更多论文逐图笔记与真实模型配置对比" },
{ label: "大规模训练系统", value: 71, next: "补真实集群 traces、故障案例与精确 topology 配置" },
{ label: "引用与事实检查", value: 54, next: "自动化外链复查与来源等级扩展" },
{ label: "引用与事实检查", value: 57, next: "自动化外链复查与来源等级扩展" },
{ label: "开源与部署", value: 100, next: "每轮保留不可变镜像、提交与回滚点" },
];
---
@@ -40,7 +41,7 @@ const workstreams = [
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-29 01:19 CST</dd></div>
<div><dt>UPDATED</dt><dd>2026-07-29 02:08 CST</dd></div>
<div><dt>MODE</dt><dd>持续迭代,不锁死版本</dd></div>
</dl>
</div>
@@ -50,7 +51,7 @@ const workstreams = [
<div class="section-heading">
<div>
<p class="eyebrow"><span>01</span> WORKSTREAMS</p>
<h2>十条工作流同时推进,但不混淆“有页面”和“已核验”</h2>
<h2>十条工作流同时推进,但不混淆“有页面”和“已核验”</h2>
</div>
<p class="section-lead">
内容首版优先打通全局脉络;随后每轮迭代选择一个专题推进到论文/工程层,并做独立事实复核。
@@ -87,13 +88,14 @@ const workstreams = [
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
<article><span>✓</span><h3>16 专题知识图</h3><p>从语言模型基础到评测安全,包含先修依赖和三条贯穿案例。</p></article>
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
<article><span>✓</span><h3>十个原创交互视图</h3><p>K3、注意力、DeepSeek、长上下文、MoE、推理三页签,以及训练系统四页签实验。</p></article>
<article><span>✓</span><h3>篇首版长文</h3><p>K3 导读、Transformer 基础、DeepSeek 谱系、长上下文、MoE、推理与训练系统专题。</p></article>
<article><span>✓</span><h3>十个原创交互视图</h3><p>K3、注意力、DeepSeek、长上下文、MoE、推理三页签,以及训练系统与 Scaling 各四页签实验。</p></article>
<article><span>✓</span><h3>篇首版长文</h3><p>K3 导读、Transformer 基础、DeepSeek 谱系、Scaling、长上下文、MoE、推理与训练系统专题。</p></article>
<article><span>✓</span><h3>Scaling Laws 深度专题</h3><p>九张账、29 个一手节点、DeepSeek/Kimi 双谱系与曲面—部署—复用—涌现四联实验。</p></article>
<article><span>✓</span><h3>长上下文深度专题</h3><p>五张成本账、26 篇一手论文、10+ 机制图与 8 策略交互实验室。</p></article>
<article><span>✓</span><h3>MoE 深度专题</h3><p>六张账、19 篇一手论文、DeepSeek/K3 主线与路由—容量—通信交互实验室。</p></article>
<article><span>✓</span><h3>推理深度专题</h3><p>八张账、30 篇一手论文链、DeepSeek/Kimi 双主线与三页签互动实验室。</p></article>
<article><span>✓</span><h3>训练系统深度专题</h3><p>九张账、37 个一手节点、DeepSeek/Kimi 双谱系与显存—网格—气泡—通信实验室。</p></article>
<article><span>✓</span><h3>166 篇关键论文索引</h3><p>覆盖 12 个专题,支持全文搜索、标签筛选与 Kimi/DeepSeek 聚光主线。</p></article>
<article><span>✓</span><h3>182 篇关键论文索引</h3><p>覆盖 12 个专题,支持全文搜索、标签筛选与 Kimi/DeepSeek 聚光主线。</p></article>
<article><span>✓</span><h3>公开仓库与自托管发布</h3><p>源码公开到 git.k1412.top,网站由不可变镜像、Compose Manager 与 HTTPS 交付。</p></article>
</div>
</section>
@@ -108,7 +110,8 @@ const workstreams = [
</div>
<div class="queue-table">
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
<div><span>P0</span><strong>大规模训练系统二轮</strong><p>真实集群 traces → 故障恢复 → 精确 topology / kernel 配置</p><em>案例库 + 实测边界</em></div>
<div><span>P0</span><strong>Scaling Laws 二轮</strong><p>真实拟合复现 → 置信区间 → 更多模型族与下游任务外推</p><em>可复现实验 + 逐图笔记</em></div>
<div><span>P1</span><strong>大规模训练系统二轮</strong><p>真实集群 traces → 故障恢复 → 精确 topology / kernel 配置</p><em>案例库 + 实测边界</em></div>
<div><span>P1</span><strong>长上下文二轮深化</strong><p>真实模型配置 → 内核细节 → 长上下文评测与失败案例</p><em>配置比较器 + 逐图论文笔记</em></div>
<div><span>P1</span><strong>MoE 二轮深化</strong><p>真实负载 traces → 专家特化可解释性 → 共享专家语义</p><em>案例库 + 集群证据</em></div>
<div><span>P1</span><strong>推理二轮深化</strong><p>真实 pass@k 曲线 → PRM 失败案例 → 逐篇图表精读</p><em>案例库 + 真实 traces</em></div>
@@ -153,6 +156,7 @@ const workstreams = [
<div><time>2026-07-28</time><b>自托管交付</b><p>源码公开到 git.k1412.top,网站部署到 k1412 私有基础设施。</p></div>
<div><time>2026-07-29</time><b>推理按八张账组织</b><p>把答案、覆盖、选择、过程、预算、优化、分布与系统证据分开核算。</p></div>
<div><time>2026-07-29</time><b>训练系统按九张账组织</b><p>把模型状态、激活、并行、气泡、通信、专家、上下文、数值与可靠性分开核算。</p></div>
<div><time>2026-07-29</time><b>Scaling Laws 按九张账组织</b><p>把观测量、模型、数据、算术、配比、配方、外推、经济目标与能力阶段分开核算。</p></div>
</div>
</section>
+1 -1
View File
@@ -699,7 +699,7 @@ const paperChain = [
<h2>不要按模型榜单读,按“旧方法留下了哪堵墙”读</h2>
<p>
下面 30 篇构成本专题首版主干。课程中的机制、公式与数字优先回到这些一手来源;
更完整的 166 篇跨专题索引位于论文库。
更完整的 182 篇跨专题索引位于论文库。
</p>
<div class="paper-chain">
{paperChain.map(([year, title, href, note], index) => (
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -201,7 +201,7 @@
.top-nav {
display: flex;
align-items: center;
gap: 34px;
gap: clamp(12px, 1.55vw, 26px);
}
.top-nav a {