feat: audit DeepSeek Chat across sources
This commit is contained in:
wuyang
@@ -0,0 +1,670 @@
|
||||
---
|
||||
import rawLab from "@/data/deepseek-v2-lite-chat-cross-source-sampling-compact.json";
|
||||
|
||||
const lab = rawLab as any;
|
||||
const json = JSON.stringify(lab).replaceAll("<", "\\u003c");
|
||||
const domains = lab.contract.domains as string[];
|
||||
const conditions = lab.contract.conditions as string[];
|
||||
---
|
||||
|
||||
<figure class="cross-source-lab" data-cross-source-lab>
|
||||
<header class="cs-head">
|
||||
<div>
|
||||
<p>ROUND 07 / SOURCE-BLOCKED FOLLOW-UP</p>
|
||||
<h3>seed 不是题目：把同题重复与跨题覆盖拆成两层</h3>
|
||||
</div>
|
||||
<p>
|
||||
总预算仍是 256 条，但从上一轮的 <code>4 sources × 8 seeds × 8 cells</code>
|
||||
改成 <code>16 sources × 4 seeds × 4 cells</code>。source 是覆盖单位，seed
|
||||
只是题内重复；所有方向都先逐 source 计算，再描述四条 source 是否同向。
|
||||
</p>
|
||||
</header>
|
||||
|
||||
<div class="cs-ledger">
|
||||
<article><span>SOURCES</span><b>16</b><p>四域各 4 条，结果前冻结</p></article>
|
||||
<article><span>SAMPLED OUTPUTS</span><b>256</b><p>16 × 4 seeds × 4 cells</p></article>
|
||||
<article class="pass"><span>NATURAL EOS</span><b>250 / 256</b><p>6 条在 512-token 触顶</p></article>
|
||||
<article><span>MATH · STRICT</span><b>47 / 64</b><p>四道 GSM8K，不是同题 64 次</p></article>
|
||||
<article><span>CODE · TESTS</span><b>52 / 64</b><p>四道 HumanEval，沙箱执行</p></article>
|
||||
<article class="pass"><span>FRESH PROCESS</span><b>64 / 64</b><p>八项合同逐字段 exact</p></article>
|
||||
</div>
|
||||
|
||||
<div class="cs-tabs" role="tablist" aria-label="选择跨来源采样实验视图">
|
||||
<button type="button" role="tab" data-cs-tab="hierarchy" aria-selected="true">
|
||||
<span>01</span><b>先分清 source 与 seed</b><small>coverage hierarchy</small>
|
||||
</button>
|
||||
<button type="button" role="tab" data-cs-tab="tasks" aria-selected="false" tabindex="-1">
|
||||
<span>02</span><b>总分怎样藏住题目差异</b><small>task × condition matrix</small>
|
||||
</button>
|
||||
<button type="button" role="tab" data-cs-tab="directions" aria-selected="false" tabindex="-1">
|
||||
<span>03</span><b>平均值为什么会误导</b><small>source-level contrasts</small>
|
||||
</button>
|
||||
<button type="button" role="tab" data-cs-tab="reproduction" aria-selected="false" tabindex="-1">
|
||||
<span>04</span><b>随机轨迹如何被审计</b><small>hash chain + replay</small>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<section class="cs-panel" data-cs-panel="hierarchy">
|
||||
<div class="cs-panel-lead">
|
||||
<div><span>I / COVERAGE HIERARCHY</span><h4>64 个 seed 输出，不会自动变成 64 道题</h4></div>
|
||||
<p>
|
||||
先选择一个域与条件。每张卡是一条独立 source，四个圆点才是该题内的四次采样。
|
||||
圆点高度表示生成长度，颜色表示自然 EOS 或预算截断。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="unit-equation" aria-label="实验单位层级">
|
||||
<article><span>DOMAIN</span><b>4 类语料</b><p>English · 中文 · Code · Math</p></article>
|
||||
<i>→</i>
|
||||
<article class="primary"><span>PRIMARY COVERAGE UNIT</span><b>4 sources / 域</b><p>跨题方向只在这一层数</p></article>
|
||||
<i>→</i>
|
||||
<article><span>WITHIN-SOURCE REPEAT</span><b>4 seeds / 格</b><p>观察同题采样离散，不扩充题数</p></article>
|
||||
</div>
|
||||
|
||||
<div class="cs-controls">
|
||||
<label><span>DOMAIN</span>
|
||||
<select data-cs-domain aria-label="选择跨来源域">
|
||||
{domains.map((domain) => (
|
||||
<option value={domain}>{lab.contract.domainLabels[domain]}</option>
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
<label><span>CONDITION</span>
|
||||
<select data-cs-condition aria-label="选择跨来源 condition">
|
||||
{conditions.map((condition) => (
|
||||
<option value={condition}>{lab.contract.conditionLabels[condition]}</option>
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
<div><span>FIXED DECODE</span><b>T .3 · P .95 · K 0 · CAP 512</b></div>
|
||||
</div>
|
||||
|
||||
<div class="source-summary">
|
||||
<article><span>OUTPUTS</span><b data-cs-output-count>—</b><p>4 sources × 4 seeds</p></article>
|
||||
<article><span>NATURAL EOS</span><b data-cs-natural-eos>—</b><p>micro count；不等于正确</p></article>
|
||||
<article><span>BETWEEN-SOURCE RANGE</span><b data-cs-between-range>—</b><p>四条 source 的平均长度跨度</p></article>
|
||||
<article><span>WITHIN-SOURCE RANGE</span><b data-cs-within-range>—</b><p>每题 seed 长度跨度的均值</p></article>
|
||||
</div>
|
||||
|
||||
<div class="source-card-grid" data-cs-source-cards></div>
|
||||
|
||||
<aside class="cs-note">
|
||||
<b>读法：先横向比较四张卡，再纵向看每张卡里的四个 seed</b>
|
||||
<p>
|
||||
如果四张卡差很多，增加同一题的 seed 只能更精细地描出那一道题，不能弥补 source
|
||||
覆盖不足。这里保留 micro 总账，但主结论使用 source-blocked 描述。
|
||||
</p>
|
||||
</aside>
|
||||
</section>
|
||||
|
||||
<section class="cs-panel" data-cs-panel="tasks" hidden>
|
||||
<div class="cs-panel-lead">
|
||||
<div><span>II / TASK MATRIX</span><h4>47 / 64 与 52 / 64 背后，是八条完全不同的题目轨迹</h4></div>
|
||||
<p>
|
||||
每格分母固定为四个 seed。绿色越深表示通过越多；自然结束、数值抽取、AST
|
||||
解析与官方测试各自记账，不用“看起来完成了”替代任务正确。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="task-switch">
|
||||
<div role="group" aria-label="选择任务域">
|
||||
<button type="button" data-cs-task-domain="math" aria-pressed="true">MATH · GSM8K</button>
|
||||
<button type="button" data-cs-task-domain="code" aria-pressed="false">CODE · HUMANEVAL</button>
|
||||
</div>
|
||||
<p><span data-cs-task-total-label>MATH · STRICT</span><b data-cs-task-total>47 / 64</b></p>
|
||||
</div>
|
||||
|
||||
<div class="task-matrix" data-cs-task-matrix></div>
|
||||
|
||||
<div class="matrix-reading">
|
||||
<article>
|
||||
<span>题目差异</span>
|
||||
<b data-cs-task-range>—</b>
|
||||
<p data-cs-task-range-copy>—</p>
|
||||
</article>
|
||||
<article>
|
||||
<span>条件交互</span>
|
||||
<b data-cs-task-interaction>—</b>
|
||||
<p data-cs-task-interaction-copy>—</p>
|
||||
</article>
|
||||
<article>
|
||||
<span>失败身份</span>
|
||||
<b data-cs-task-failure>—</b>
|
||||
<p data-cs-task-failure-copy>—</p>
|
||||
</article>
|
||||
</div>
|
||||
|
||||
<aside class="cs-note dark">
|
||||
<b>最关键的反例：Code 的总体 interaction 恰好是 0</b>
|
||||
<p>
|
||||
但逐题看，HumanEval/44 是 +1.0，HumanEval/133 是 −1.0，另外两题为 0。
|
||||
“总体没有交互”在这里不是“每题都没有”，而是两道题正负抵消。
|
||||
</p>
|
||||
</aside>
|
||||
</section>
|
||||
|
||||
<section class="cs-panel" data-cs-panel="directions" hidden>
|
||||
<div class="cs-panel-lead">
|
||||
<div><span>III / SOURCE-LEVEL DIRECTIONS</span><h4>一个极端 source，可以让均值与多数方向相反</h4></div>
|
||||
<p>
|
||||
下图是每条 source 的“句点 − EOS”平均生成长度差。负数代表句点更短；
|
||||
四个点才是四个覆盖单位，域均值只放在旁边，不拿它替代点的方向。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="direction-controls">
|
||||
<label><span>DOMAIN</span>
|
||||
<select data-cs-direction-domain aria-label="选择方向审计域">
|
||||
{domains.map((domain) => (
|
||||
<option value={domain}>{lab.contract.domainLabels[domain]}</option>
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
<article><span>PERIOD SHORTER</span><b data-cs-shorter>—</b><p>四条 source 中负方向数量</p></article>
|
||||
<article><span>DOMAIN MEAN</span><b data-cs-domain-mean>—</b><p>tokens · period − EOS</p></article>
|
||||
<article><span>SOURCE MEDIAN</span><b data-cs-domain-median>—</b><p>比均值更不易被单点拖动</p></article>
|
||||
</div>
|
||||
|
||||
<div class="zero-axis">
|
||||
<header><span>PERIOD SHORTER</span><b>0 · NO LENGTH CHANGE</b><span>PERIOD LONGER</span></header>
|
||||
<div data-cs-direction-rows></div>
|
||||
</div>
|
||||
|
||||
<div class="round-correction">
|
||||
<article>
|
||||
<span>ROUND 06 · SINGLE ENGLISH SOURCE</span>
|
||||
<b>WikiText/0443：−121.0 tokens</b>
|
||||
<p>如果只看这一条，会得到“句点大幅缩短 English”的印象。</p>
|
||||
</article>
|
||||
<i>→</i>
|
||||
<article class="result">
|
||||
<span>ROUND 07 · FOUR ENGLISH SOURCES</span>
|
||||
<b>1 shorter · 3 longer</b>
|
||||
<p>另三条分别 +44.5、+41.0、+1.4；均值仍为 −8.5，只因首题权重大。</p>
|
||||
</article>
|
||||
</div>
|
||||
|
||||
<div class="direction-all">
|
||||
{domains.map((domain) => (
|
||||
<article>
|
||||
<span>{lab.contract.domainLabels[domain]}</span>
|
||||
<b>{lab.periodDirection[domain].shorter} / 4 shorter</b>
|
||||
<p>
|
||||
mean {
|
||||
lab.domainContrasts[domain].mean_generated_tokens
|
||||
.boundary_main_period_minus_eos.mean.toFixed(1)
|
||||
} · median {
|
||||
lab.domainContrasts[domain].mean_generated_tokens
|
||||
.boundary_main_period_minus_eos.median.toFixed(1)
|
||||
}
|
||||
</p>
|
||||
</article>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<aside class="cs-note">
|
||||
<b>11 / 16 source 的句点格平均更短，但这不是总体显著性</b>
|
||||
<p>
|
||||
句点是受 Round 06 结果启发的定向复查，不是未见前序结果的盲确认；16 条 source
|
||||
也没有随机代表任何总体。因此不报告 p-value 或 population confidence interval。
|
||||
</p>
|
||||
</aside>
|
||||
</section>
|
||||
|
||||
<section class="cs-panel" data-cs-panel="reproduction" hidden>
|
||||
<div class="cs-panel-lead">
|
||||
<div><span>IV / REPLAY + HASH CHAIN</span><h4>正式生成、独立评测、分析与复跑彼此锁定</h4></div>
|
||||
<p>
|
||||
R0 的 16 sources × 4 conditions 在全新进程重跑。比较不只看 headline，
|
||||
而是逐格检查 seed、prompt、完整 token IDs、文本、停止状态与 RNG pre-state。
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="repro-equation">
|
||||
<article><span>DIFFERENT SEEDS</span><b>63 / 64 分叉</b><p>R0 与 R1 的同 source-condition 轨迹</p></article>
|
||||
<i>+</i>
|
||||
<article><span>SAME FULL CONTRACT</span><b>64 / 64 exact</b><p>R0 与全新进程 R0′</p></article>
|
||||
<i>=</i>
|
||||
<article class="result"><span>AUDITABLE SAMPLING</span><b>随机 ≠ 漂移</b><p>分叉与复现同时过闸</p></article>
|
||||
</div>
|
||||
|
||||
<div class="repro-fields">
|
||||
{Object.entries(lab.reproduction.by_field).map(([field, count]) => (
|
||||
<article>
|
||||
<span>{String(field).replaceAll("_", " ").toUpperCase()}</span>
|
||||
<b>{String(count)} / 64</b><i>EXACT</i>
|
||||
</article>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<div class="hash-chain">
|
||||
{Object.entries(lab.artifacts).map(([name, artifact]: [string, any], index) => (
|
||||
<>
|
||||
<article>
|
||||
<span>{String(index + 1).padStart(2, "0")} / {name.toUpperCase()}</span>
|
||||
<b>{artifact.sha256.slice(0, 12)}…{artifact.sha256.slice(-8)}</b>
|
||||
<p>{artifact.bytes.toLocaleString()} bytes</p>
|
||||
</article>
|
||||
{index < Object.keys(lab.artifacts).length - 1 && <i>→</i>}
|
||||
</>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<div class="execution-contract">
|
||||
<article><span>CHECKPOINT</span><b>85864749…f64c7</b><p>官方 DeepSeek-V2-Lite-Chat revision</p></article>
|
||||
<article><span>SOFTWARE</span><b>Torch 2.11 · TF 4.41.2</b><p>BF16 · eager attention</p></article>
|
||||
<article><span>PLACEMENT</span><b>CUDA L0–23 · CPU L24–26</b><p>norm 与 LM head 也在 CPU</p></article>
|
||||
<article><span>CODE SANDBOX</span><b>network none · read-only</b><p>47 个唯一候选；17 次 cache hit</p></article>
|
||||
</div>
|
||||
|
||||
<aside class="cs-note">
|
||||
<b>exact replay 只属于这份执行合同</b>
|
||||
<p>
|
||||
更换软件、kernel、硬件、batch 行数或顺序都可能改变采样轨迹。这里证明的是固定合同内
|
||||
可复现，不承诺跨版本 bit-exact。
|
||||
</p>
|
||||
</aside>
|
||||
</section>
|
||||
|
||||
<figcaption>
|
||||
<b>证据边界</b>
|
||||
<span>
|
||||
16 sources 仍不是 benchmark；四题方向不推断总体。seed 是题内重复；句点不是官方聊天
|
||||
边界；source-blocked contrast 是固定网格描述量，没有 p-value 与总体置信区间。
|
||||
</span>
|
||||
<code>FORMAL f013132…d7f7c · EVAL e88b274…b8975 · REPLAY 143dc9d…02a6</code>
|
||||
</figcaption>
|
||||
|
||||
<script is:inline type="application/json" data-cs-data set:html={json}></script>
|
||||
</figure>
|
||||
|
||||
<script>
|
||||
document.querySelectorAll<HTMLElement>("[data-cross-source-lab]").forEach((root) => {
|
||||
const payload = root.querySelector<HTMLScriptElement>("[data-cs-data]");
|
||||
if (!payload) return;
|
||||
const data = JSON.parse(payload.textContent ?? "{}");
|
||||
const one = <T extends Element>(selector: string) => root.querySelector<T>(selector);
|
||||
const set = (selector: string, value: string) => {
|
||||
const node = one<HTMLElement>(selector);
|
||||
if (node) node.textContent = value;
|
||||
};
|
||||
const signed = (value: number, digits = 1) => (
|
||||
`${value > 0 ? "+" : ""}${value.toFixed(digits)}`
|
||||
);
|
||||
|
||||
const tabs = [...root.querySelectorAll<HTMLButtonElement>("[data-cs-tab]")];
|
||||
const panels = [...root.querySelectorAll<HTMLElement>("[data-cs-panel]")];
|
||||
tabs.forEach((button, index) => {
|
||||
button.addEventListener("click", () => {
|
||||
const target = button.dataset.csTab;
|
||||
tabs.forEach((candidate) => {
|
||||
const active = candidate === button;
|
||||
candidate.setAttribute("aria-selected", String(active));
|
||||
candidate.tabIndex = active ? 0 : -1;
|
||||
});
|
||||
panels.forEach((panel) => {
|
||||
panel.hidden = panel.dataset.csPanel !== target;
|
||||
});
|
||||
});
|
||||
button.addEventListener("keydown", (event) => {
|
||||
if (!["ArrowLeft", "ArrowRight"].includes(event.key)) return;
|
||||
event.preventDefault();
|
||||
const delta = event.key === "ArrowRight" ? 1 : -1;
|
||||
const target = tabs[(index + delta + tabs.length) % tabs.length];
|
||||
target.click();
|
||||
target.focus();
|
||||
});
|
||||
});
|
||||
|
||||
const domainSelect = one<HTMLSelectElement>("[data-cs-domain]");
|
||||
const conditionSelect = one<HTMLSelectElement>("[data-cs-condition]");
|
||||
const sourceCards = one<HTMLElement>("[data-cs-source-cards]");
|
||||
const renderHierarchy = () => {
|
||||
const domain = domainSelect?.value ?? "english";
|
||||
const condition = conditionSelect?.value ?? "s0_eos";
|
||||
const summary = data.domainConditions[domain][condition];
|
||||
set("[data-cs-output-count]", `${summary.outputs} / 16`);
|
||||
set("[data-cs-natural-eos]", `${summary.micro.natural_eos} / 16`);
|
||||
set(
|
||||
"[data-cs-between-range]",
|
||||
`${summary.variability.between_source_mean_length_range.toFixed(1)} tokens`,
|
||||
);
|
||||
set(
|
||||
"[data-cs-within-range]",
|
||||
`${summary.variability.within_source_seed_length_range.mean.toFixed(1)} tokens`,
|
||||
);
|
||||
if (!sourceCards) return;
|
||||
sourceCards.replaceChildren();
|
||||
data.sourcesByDomain[domain].forEach((sourceId: string) => {
|
||||
const cell = data.sourceCells[sourceId][condition];
|
||||
const samples = data.evalRows.filter((row: any) => (
|
||||
row.sourceId === sourceId && row.condition === condition
|
||||
));
|
||||
const article = document.createElement("article");
|
||||
article.dataset.sourceId = sourceId;
|
||||
const header = document.createElement("header");
|
||||
const heading = document.createElement("div");
|
||||
const index = document.createElement("span");
|
||||
index.textContent = `SOURCE ${String(data.sourceMeta[sourceId].within_domain_index + 1).padStart(2, "0")}`;
|
||||
const title = document.createElement("b");
|
||||
title.textContent = data.sourceMeta[sourceId].label;
|
||||
heading.append(index, title);
|
||||
const eos = document.createElement("em");
|
||||
eos.textContent = `${cell.natural_eos} / 4 EOS`;
|
||||
header.append(heading, eos);
|
||||
const dots = document.createElement("div");
|
||||
dots.className = "seed-dots";
|
||||
samples.forEach((sample: any) => {
|
||||
const dot = document.createElement("i");
|
||||
dot.className = sample.hitEos ? "eos" : "truncated";
|
||||
dot.style.setProperty("--seed-height", `${Math.max(12, sample.generatedTokens / 512 * 100)}%`);
|
||||
dot.title = `${sample.replicate}: ${sample.generatedTokens} tokens`;
|
||||
const label = document.createElement("span");
|
||||
label.textContent = sample.replicate;
|
||||
const value = document.createElement("b");
|
||||
value.textContent = String(sample.generatedTokens);
|
||||
dot.append(label, value);
|
||||
dots.append(dot);
|
||||
});
|
||||
const footer = document.createElement("p");
|
||||
const task = cell.strict_complete_success === null
|
||||
? `${cell.unique_trajectory_hashes} / 4 unique token trajectories`
|
||||
: `${cell.strict_complete_success} / 4 strict task pass`;
|
||||
footer.textContent = `mean ${cell.generated_tokens.mean.toFixed(1)} · range ${cell.generated_tokens.min}–${cell.generated_tokens.max} · ${task}`;
|
||||
article.append(header, dots, footer);
|
||||
sourceCards.append(article);
|
||||
});
|
||||
};
|
||||
domainSelect?.addEventListener("change", renderHierarchy);
|
||||
conditionSelect?.addEventListener("change", renderHierarchy);
|
||||
renderHierarchy();
|
||||
|
||||
const taskButtons = [
|
||||
...root.querySelectorAll<HTMLButtonElement>("[data-cs-task-domain]"),
|
||||
];
|
||||
const taskMatrix = one<HTMLElement>("[data-cs-task-matrix]");
|
||||
const renderTasks = (domain: "math" | "code") => {
|
||||
taskButtons.forEach((button) => button.setAttribute(
|
||||
"aria-pressed",
|
||||
String(button.dataset.csTaskDomain === domain),
|
||||
));
|
||||
const tasks = data.taskTotals[domain];
|
||||
const total = tasks.reduce((sum: number, task: any) => sum + task.totalPass, 0);
|
||||
set("[data-cs-task-total-label]", domain === "math" ? "MATH · STRICT" : "CODE · OFFICIAL TESTS");
|
||||
set("[data-cs-task-total]", `${total} / 64`);
|
||||
if (taskMatrix) {
|
||||
taskMatrix.replaceChildren();
|
||||
const header = document.createElement("header");
|
||||
const blank = document.createElement("b");
|
||||
blank.textContent = "SOURCE / TASK";
|
||||
header.append(blank);
|
||||
data.contract.conditions.forEach((condition: string) => {
|
||||
const label = document.createElement("b");
|
||||
label.textContent = data.contract.conditionLabels[condition];
|
||||
header.append(label);
|
||||
});
|
||||
const sum = document.createElement("b");
|
||||
sum.textContent = "TOTAL";
|
||||
header.append(sum);
|
||||
taskMatrix.append(header);
|
||||
tasks.forEach((task: any) => {
|
||||
const row = document.createElement("div");
|
||||
row.dataset.taskId = task.sourceId;
|
||||
const label = document.createElement("span");
|
||||
label.textContent = task.label;
|
||||
row.append(label);
|
||||
data.contract.conditions.forEach((condition: string) => {
|
||||
const cell = task.conditions[condition];
|
||||
const item = document.createElement("b");
|
||||
item.textContent = `${cell.pass} / 4`;
|
||||
item.dataset.pass = String(cell.pass);
|
||||
item.style.setProperty("--pass", String(cell.pass));
|
||||
item.title = `${cell.naturalEos}/4 EOS · mean ${cell.meanGeneratedTokens.toFixed(1)} tokens`;
|
||||
row.append(item);
|
||||
});
|
||||
const totalCell = document.createElement("strong");
|
||||
totalCell.textContent = `${task.totalPass} / 16`;
|
||||
row.append(totalCell);
|
||||
taskMatrix.append(row);
|
||||
});
|
||||
}
|
||||
const totals = tasks.map((task: any) => task.totalPass);
|
||||
set("[data-cs-task-range]", `${Math.min(...totals)} → ${Math.max(...totals)} / 16`);
|
||||
set(
|
||||
"[data-cs-task-range-copy]",
|
||||
domain === "math"
|
||||
? "GSM8K/0144 只有 8/16；GSM8K/1228 是 16/16。"
|
||||
: "HumanEval/44 只有 8/16；HumanEval/31 与 /23 都是 16/16。",
|
||||
);
|
||||
const interaction = data.domainContrasts[domain]
|
||||
.strict_complete_success_rate.interaction;
|
||||
set(
|
||||
"[data-cs-task-interaction]",
|
||||
`${interaction.directions.positive}↑ · ${interaction.directions.zero}＝ · ${interaction.directions.negative}↓`,
|
||||
);
|
||||
set(
|
||||
"[data-cs-task-interaction-copy]",
|
||||
domain === "math"
|
||||
? "两题为负、两题为零；domain mean −0.188。"
|
||||
: "一题正、一题负、两题为零；domain mean 恰好 0。",
|
||||
);
|
||||
const failures = data.taskFailures.filter((row: any) => row.domain === domain);
|
||||
set("[data-cs-task-failure]", `${failures.length} / 64`);
|
||||
set(
|
||||
"[data-cs-task-failure-copy]",
|
||||
domain === "math"
|
||||
? "17 条都是数值答案错误；它们仍全部自然 EOS。"
|
||||
: "5 次 runtime error、7 次 assertion failed；其中 1 条输出截断。",
|
||||
);
|
||||
};
|
||||
taskButtons.forEach((button) => button.addEventListener("click", () => (
|
||||
renderTasks((button.dataset.csTaskDomain ?? "math") as "math" | "code")
|
||||
)));
|
||||
renderTasks("math");
|
||||
|
||||
const directionSelect = one<HTMLSelectElement>("[data-cs-direction-domain]");
|
||||
const directionRows = one<HTMLElement>("[data-cs-direction-rows]");
|
||||
const renderDirections = () => {
|
||||
const domain = directionSelect?.value ?? "english";
|
||||
const contrast = data.domainContrasts[domain].mean_generated_tokens
|
||||
.boundary_main_period_minus_eos;
|
||||
set(
|
||||
"[data-cs-shorter]",
|
||||
`${data.periodDirection[domain].shorter} / ${data.periodDirection[domain].sources}`,
|
||||
);
|
||||
set("[data-cs-domain-mean]", `${signed(contrast.mean)} tokens`);
|
||||
set("[data-cs-domain-median]", `${signed(contrast.median)} tokens`);
|
||||
if (!directionRows) return;
|
||||
directionRows.replaceChildren();
|
||||
data.sourcesByDomain[domain].forEach((sourceId: string) => {
|
||||
const value = contrast.by_source[sourceId];
|
||||
const row = document.createElement("article");
|
||||
row.dataset.directionSource = sourceId;
|
||||
const label = document.createElement("span");
|
||||
label.textContent = data.sourceMeta[sourceId].label;
|
||||
const track = document.createElement("i");
|
||||
const dot = document.createElement("u");
|
||||
const position = Math.max(2, Math.min(98, 50 + value / 5));
|
||||
dot.style.setProperty("--dot-position", `${position}%`);
|
||||
dot.className = value < 0 ? "negative" : "positive";
|
||||
track.append(dot);
|
||||
const output = document.createElement("b");
|
||||
output.textContent = `${signed(value)} tokens`;
|
||||
row.append(label, track, output);
|
||||
directionRows.append(row);
|
||||
});
|
||||
};
|
||||
directionSelect?.addEventListener("change", renderDirections);
|
||||
renderDirections();
|
||||
});
|
||||
</script>
|
||||
|
||||
<style>
|
||||
.cross-source-lab { margin: 1.6rem 0 0; overflow: hidden; border: 1px solid var(--line); background: #f7f3ea; }
|
||||
.cs-head { display: grid; grid-template-columns: 1.08fr .92fr; gap: 1.5rem; padding: 1.55rem; color: #edf3f1; background: #263f47; }
|
||||
.cs-head p { margin: 0; font-size: .62rem; line-height: 1.65; }
|
||||
.cs-head > div > p { color: #8fcbbf; font: 690 .48rem/1.2 var(--font-mono); letter-spacing: .08em; }
|
||||
.cs-head h3 { margin: .6rem 0 0; max-width: 25ch; color: #edf3f1; font: 760 1.22rem/1.14 var(--font-display); }
|
||||
.cs-head code { color: #f0c7aa; font-size: .53rem; }
|
||||
.cs-ledger { display: grid; grid-template-columns: repeat(6,1fr); border-bottom: 1px solid var(--line); background: var(--line); gap: 1px; }
|
||||
.cs-ledger article { min-width: 0; padding: .82rem; background: #eee9df; }
|
||||
.cs-ledger article.pass { background: #dcebe5; }
|
||||
.cs-ledger span, .source-summary span, .direction-controls span { display: block; color: #687772; font: .44rem/1.2 var(--font-mono); }
|
||||
.cs-ledger b { display: block; margin-top: .38rem; font: 760 .72rem/1.1 var(--font-mono); }
|
||||
.cs-ledger p { margin: .32rem 0 0; color: #717975; font-size: .46rem; line-height: 1.4; }
|
||||
.cs-tabs { display: grid; grid-template-columns: repeat(4,1fr); gap: 1px; background: var(--line); border-bottom: 1px solid var(--line); }
|
||||
.cs-tabs button { position: relative; min-width: 0; padding: .88rem; text-align: left; color: #52615e; border: 0; background: #e5e0d5; cursor: pointer; }
|
||||
.cs-tabs button[aria-selected="true"] { color: #f0f5f3; background: #2e776c; }
|
||||
.cs-tabs span, .cs-tabs b, .cs-tabs small { display: block; }
|
||||
.cs-tabs span { color: var(--orange); font: 730 .44rem/1 var(--font-mono); }
|
||||
.cs-tabs button[aria-selected="true"] span { color: #f2c6a8; }
|
||||
.cs-tabs b { margin-top: .38rem; font-size: .62rem; }
|
||||
.cs-tabs small { margin-top: .25rem; opacity: .65; font: .42rem/1 var(--font-mono); }
|
||||
.cs-panel { padding: 1.25rem; }
|
||||
.cs-panel-lead { display: grid; grid-template-columns: 1.1fr .9fr; gap: 1.4rem; margin-bottom: 1rem; }
|
||||
.cs-panel-lead span { color: var(--orange); font: 700 .46rem/1.2 var(--font-mono); }
|
||||
.cs-panel-lead h4 { margin: .42rem 0 0; font: 750 .96rem/1.15 var(--font-display); }
|
||||
.cs-panel-lead p { margin: 0; color: #63706c; font-size: .57rem; line-height: 1.65; }
|
||||
.unit-equation { display: grid; grid-template-columns: 1fr auto 1.2fr auto 1fr; gap: .6rem; align-items: center; }
|
||||
.unit-equation article { padding: .82rem; border: 1px solid var(--line); background: #eee9df; }
|
||||
.unit-equation article.primary { color: #eef4f2; border: 0; background: #2f776c; }
|
||||
.unit-equation span { font: .43rem/1.2 var(--font-mono); opacity: .72; }
|
||||
.unit-equation b { display: block; margin-top: .4rem; font: 720 .67rem/1.15 var(--font-mono); }
|
||||
.unit-equation p { margin: .35rem 0 0; opacity: .72; font-size: .48rem; }
|
||||
.unit-equation > i { color: var(--orange); font: 740 .72rem/1 var(--font-mono); }
|
||||
.cs-controls { display: grid; grid-template-columns: 1fr 1fr 1.15fr; gap: 1px; margin-top: 1rem; border: 1px solid var(--line); background: var(--line); }
|
||||
.cs-controls label, .cs-controls > div { padding: .72rem; background: #eee9df; }
|
||||
.cs-controls span { display: block; color: #6e7975; font: .43rem/1 var(--font-mono); }
|
||||
.cs-controls select { width: 100%; margin-top: .38rem; padding: .42rem; font: 650 .54rem/1.2 var(--font-mono); border: 1px solid var(--line); background: #fffaf2; }
|
||||
.cs-controls b { display: block; margin-top: .48rem; font: 680 .54rem/1.2 var(--font-mono); }
|
||||
.source-summary { display: grid; grid-template-columns: repeat(4,1fr); gap: 1px; margin-top: 1rem; border: 1px solid var(--line); background: var(--line); }
|
||||
.source-summary article { padding: .72rem; background: #f9f5ec; }
|
||||
.source-summary b { display: block; margin-top: .38rem; color: #2f6f65; font: 740 .65rem/1.1 var(--font-mono); }
|
||||
.source-summary p { margin: .32rem 0 0; color: #737d79; font-size: .46rem; }
|
||||
.source-card-grid { display: grid; grid-template-columns: repeat(4,1fr); gap: .7rem; margin-top: 1rem; }
|
||||
.cs-note { margin-top: 1rem; padding: .85rem 1rem; border-left: .24rem solid var(--orange); background: #eee9df; }
|
||||
.cs-note.dark { color: #e8efed; border-left-color: #e0a17c; background: #29434a; }
|
||||
.cs-note b { font: 710 .6rem/1.3 var(--font-mono); }
|
||||
.cs-note p { margin: .42rem 0 0; opacity: .76; font-size: .53rem; line-height: 1.6; }
|
||||
.task-switch { display: flex; justify-content: space-between; gap: 1rem; align-items: stretch; padding: .65rem; border: 1px solid var(--line); background: #eee9df; }
|
||||
.task-switch > div { display: flex; gap: .35rem; }
|
||||
.task-switch button { padding: .55rem .75rem; color: #53625f; border: 1px solid var(--line); background: #fffaf2; font: 690 .48rem/1 var(--font-mono); cursor: pointer; }
|
||||
.task-switch button[aria-pressed="true"] { color: #edf3f1; border-color: #2f776c; background: #2f776c; }
|
||||
.task-switch p { margin: 0; padding: .35rem .5rem; text-align: right; }
|
||||
.task-switch p span { display: block; color: #6a7773; font: .42rem/1 var(--font-mono); }
|
||||
.task-switch p b { display: block; margin-top: .32rem; font: 750 .66rem/1 var(--font-mono); }
|
||||
.task-matrix { margin-top: 1rem; overflow: hidden; border: 1px solid var(--line); }
|
||||
.matrix-reading { display: grid; grid-template-columns: repeat(3,1fr); gap: 1px; margin-top: 1rem; border: 1px solid var(--line); background: var(--line); }
|
||||
.matrix-reading article { padding: .82rem; background: #eee9df; }
|
||||
.matrix-reading span { color: var(--orange); font: .45rem/1.2 var(--font-mono); }
|
||||
.matrix-reading b { display: block; margin-top: .4rem; font: 740 .68rem/1.15 var(--font-mono); }
|
||||
.matrix-reading p { margin: .4rem 0 0; color: #69736f; font-size: .51rem; line-height: 1.5; }
|
||||
.direction-controls { display: grid; grid-template-columns: 1.25fr repeat(3,1fr); gap: 1px; border: 1px solid var(--line); background: var(--line); }
|
||||
.direction-controls label, .direction-controls article { padding: .75rem; background: #eee9df; }
|
||||
.direction-controls select { width: 100%; margin-top: .4rem; padding: .42rem; border: 1px solid var(--line); background: #fffaf2; font: 650 .53rem/1.2 var(--font-mono); }
|
||||
.direction-controls b { display: block; margin-top: .4rem; font: 740 .65rem/1.1 var(--font-mono); }
|
||||
.direction-controls p { margin: .28rem 0 0; color: #6c7672; font-size: .44rem; }
|
||||
.zero-axis { margin-top: 1rem; overflow: hidden; border: 1px solid var(--line); }
|
||||
.zero-axis > header { display: grid; grid-template-columns: 1fr auto 1fr; padding: .6rem .8rem; color: #e9f0ee; background: #29434a; font: .43rem/1 var(--font-mono); }
|
||||
.zero-axis > header span:last-child { text-align: right; }
|
||||
.zero-axis > header b { color: #a8d2c9; }
|
||||
.round-correction { display: grid; grid-template-columns: 1fr auto 1fr; gap: .7rem; align-items: center; margin-top: 1rem; }
|
||||
.round-correction article { padding: .9rem; border: 1px solid var(--line); background: #eee9df; }
|
||||
.round-correction article.result { color: #eaf1ef; border: 0; background: #2e776c; }
|
||||
.round-correction span { font: .43rem/1.2 var(--font-mono); opacity: .72; }
|
||||
.round-correction b { display: block; margin-top: .42rem; font: 720 .65rem/1.2 var(--font-mono); }
|
||||
.round-correction p { margin: .38rem 0 0; opacity: .72; font-size: .51rem; line-height: 1.5; }
|
||||
.round-correction > i { color: var(--orange); font: 760 .72rem/1 var(--font-mono); }
|
||||
.direction-all { display: grid; grid-template-columns: repeat(4,1fr); gap: 1px; margin-top: 1rem; border: 1px solid var(--line); background: var(--line); }
|
||||
.direction-all article { padding: .75rem; background: #eee9df; }
|
||||
.direction-all span { color: #66736f; font: .43rem/1.2 var(--font-mono); }
|
||||
.direction-all b { display: block; margin-top: .38rem; font: 720 .58rem/1.2 var(--font-mono); }
|
||||
.direction-all p { margin: .32rem 0 0; color: #717b77; font: .44rem/1.4 var(--font-mono); }
|
||||
.repro-equation { display: grid; grid-template-columns: 1fr auto 1fr auto 1fr; gap: .65rem; align-items: center; }
|
||||
.repro-equation article { padding: .9rem; border: 1px solid var(--line); background: #eee9df; }
|
||||
.repro-equation article.result { color: #eaf2ef; border: 0; background: #2e776c; }
|
||||
.repro-equation span { font: .43rem/1.2 var(--font-mono); opacity: .7; }
|
||||
.repro-equation b { display: block; margin-top: .4rem; font: 730 .67rem/1.1 var(--font-mono); }
|
||||
.repro-equation p { margin: .35rem 0 0; opacity: .72; font-size: .49rem; }
|
||||
.repro-equation > i { color: var(--orange); font: 760 .78rem/1 var(--font-mono); }
|
||||
.repro-fields { display: grid; grid-template-columns: repeat(4,1fr); gap: 1px; margin-top: 1rem; border: 1px solid var(--line); background: var(--line); }
|
||||
.repro-fields article { padding: .72rem; background: #dfece7; }
|
||||
.repro-fields span { display: block; color: #58716b; font: .4rem/1.2 var(--font-mono); }
|
||||
.repro-fields b { display: block; margin-top: .34rem; color: #28675c; font: 740 .59rem/1.1 var(--font-mono); }
|
||||
.repro-fields i { display: block; margin-top: .25rem; color: #568178; font: .38rem/1 var(--font-mono); }
|
||||
.hash-chain { display: grid; grid-template-columns: 1fr auto 1fr auto 1fr auto 1fr auto 1fr; gap: .35rem; align-items: center; margin-top: 1rem; padding: .75rem; border: 1px solid var(--line); background: #eee9df; }
|
||||
.hash-chain article { min-width: 0; padding: .55rem; background: #fffaf2; }
|
||||
.hash-chain span { display: block; color: var(--orange); font: .39rem/1.2 var(--font-mono); }
|
||||
.hash-chain b { display: block; margin-top: .34rem; overflow: hidden; font: 650 .43rem/1.25 var(--font-mono); text-overflow: ellipsis; }
|
||||
.hash-chain p { margin: .28rem 0 0; color: #6f7975; font: .38rem/1.2 var(--font-mono); }
|
||||
.hash-chain > i { color: var(--orange); font: 700 .55rem/1 var(--font-mono); }
|
||||
.execution-contract { display: grid; grid-template-columns: repeat(4,1fr); gap: 1px; margin-top: 1rem; border: 1px solid var(--line); background: var(--line); }
|
||||
.execution-contract article { min-width: 0; padding: .76rem; background: #eee9df; }
|
||||
.execution-contract span { color: #64716d; font: .42rem/1.2 var(--font-mono); }
|
||||
.execution-contract b { display: block; margin-top: .38rem; overflow-wrap: anywhere; font: 680 .54rem/1.3 var(--font-mono); }
|
||||
.execution-contract p { margin: .34rem 0 0; color: #707975; font-size: .46rem; }
|
||||
.cross-source-lab > figcaption { display: grid; grid-template-columns: auto 1fr auto; gap: 1rem; align-items: center; padding: .9rem 1.1rem; color: #e2eae8; background: #203a42; }
|
||||
.cross-source-lab > figcaption b { color: #8dc8bd; font: 720 .5rem/1.2 var(--font-mono); }
|
||||
.cross-source-lab > figcaption span { font-size: .52rem; line-height: 1.5; }
|
||||
.cross-source-lab > figcaption code { color: #dda988; font: .4rem/1.3 var(--font-mono); }
|
||||
@media (max-width: 900px) {
|
||||
.cs-head, .cs-panel-lead { grid-template-columns: 1fr; }
|
||||
.cs-ledger { grid-template-columns: repeat(3,1fr); }
|
||||
.cs-tabs { grid-template-columns: 1fr 1fr; }
|
||||
.source-card-grid, .source-summary, .direction-all, .execution-contract { grid-template-columns: 1fr 1fr; }
|
||||
.hash-chain { grid-template-columns: 1fr; }
|
||||
.hash-chain > i { transform: rotate(90deg); text-align: center; }
|
||||
.cross-source-lab > figcaption { grid-template-columns: 1fr; }
|
||||
}
|
||||
@media (max-width: 640px) {
|
||||
.cs-panel { padding: .9rem; }
|
||||
.cs-head { padding: 1.2rem; }
|
||||
.cs-tabs, .cs-controls, .unit-equation, .task-switch, .matrix-reading,
|
||||
.direction-controls, .round-correction, .repro-equation { grid-template-columns: 1fr; }
|
||||
.task-switch { display: grid; }
|
||||
.task-switch > div { display: grid; grid-template-columns: 1fr 1fr; }
|
||||
.task-switch p { text-align: left; }
|
||||
.unit-equation > i, .round-correction > i, .repro-equation > i { transform: rotate(90deg); text-align: center; }
|
||||
.source-card-grid, .source-summary, .direction-all, .execution-contract, .repro-fields { grid-template-columns: 1fr 1fr; }
|
||||
}
|
||||
</style>
|
||||
|
||||
<style is:global>
|
||||
[data-cross-source-lab] [data-cs-source-cards] > article { min-width: 0; overflow: hidden; border: 1px solid var(--line); background: #eee9df; }
|
||||
[data-cross-source-lab] [data-cs-source-cards] header { display: flex; justify-content: space-between; gap: .5rem; padding: .7rem; border-bottom: 1px solid var(--line); }
|
||||
[data-cross-source-lab] [data-cs-source-cards] header span,
|
||||
[data-cross-source-lab] [data-cs-source-cards] header b { display: block; }
|
||||
[data-cross-source-lab] [data-cs-source-cards] header span { color: var(--orange); font: .4rem/1 var(--font-mono); }
|
||||
[data-cross-source-lab] [data-cs-source-cards] header b { margin-top: .3rem; font: 700 .54rem/1.15 var(--font-mono); }
|
||||
[data-cross-source-lab] [data-cs-source-cards] header em { color: #57736c; font: 650 .42rem/1.2 var(--font-mono); }
|
||||
[data-cross-source-lab] .seed-dots { display: grid; grid-template-columns: repeat(4,1fr); gap: 1px; height: 8.2rem; padding: .6rem; background: rgba(37,61,67,.12); }
|
||||
[data-cross-source-lab] .seed-dots > i { position: relative; display: flex; flex-direction: column; justify-content: space-between; align-self: end; height: var(--seed-height); min-height: 1.3rem; padding: .28rem; color: #f2f6f4; background: linear-gradient(to top,#2f776c,#72aea3); font-style: normal; }
|
||||
[data-cross-source-lab] .seed-dots > i.truncated { background: linear-gradient(to top,#a94f31,#d4936c); }
|
||||
[data-cross-source-lab] .seed-dots span { font: .37rem/1 var(--font-mono); }
|
||||
[data-cross-source-lab] .seed-dots b { font: 690 .43rem/1 var(--font-mono); }
|
||||
[data-cross-source-lab] [data-cs-source-cards] > article > p { margin: 0; padding: .6rem .7rem; color: #65716d; font: .4rem/1.45 var(--font-mono); }
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > header,
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > div { display: grid; grid-template-columns: 1.35fr repeat(4,1fr) .85fr; gap: 1px; align-items: stretch; background: var(--line); }
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > header > *,
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > div > * { min-width: 0; padding: .65rem .45rem; }
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > header > * { color: #e7efed; background: #29434a; font: 650 .4rem/1.2 var(--font-mono); text-align: center; }
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > header > b:first-child { text-align: left; }
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > div > span { background: #eee9df; font: 670 .48rem/1.2 var(--font-mono); }
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > div > b { color: #244a43; background: color-mix(in srgb,#338274 calc(var(--pass) * 20%),#f7f3ea); font: 740 .54rem/1 var(--font-mono); text-align: center; }
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > div > strong { color: #eaf1ef; background: #2f776c; font: 740 .54rem/1 var(--font-mono); text-align: center; }
|
||||
[data-cross-source-lab] [data-cs-direction-rows] > article { display: grid; grid-template-columns: 1fr 2.8fr .8fr; gap: .7rem; align-items: center; padding: .7rem .8rem; border-bottom: 1px solid var(--line); background: #eee9df; }
|
||||
[data-cross-source-lab] [data-cs-direction-rows] > article:last-child { border-bottom: 0; }
|
||||
[data-cross-source-lab] [data-cs-direction-rows] span { font: 650 .5rem/1.2 var(--font-mono); }
|
||||
[data-cross-source-lab] [data-cs-direction-rows] > article > i { position: relative; height: .44rem; background: linear-gradient(to right,#d99a75 0 49.7%,#345f59 49.7% 50.3%,#8fc8bd 50.3% 100%); }
|
||||
[data-cross-source-lab] [data-cs-direction-rows] u { position: absolute; top: 50%; left: var(--dot-position); width: .8rem; height: .8rem; border: .14rem solid #f7f3ea; border-radius: 50%; background: #2f776c; box-shadow: 0 0 0 1px #29434a; transform: translate(-50%,-50%); }
|
||||
[data-cross-source-lab] [data-cs-direction-rows] u.negative { background: #b65c38; }
|
||||
[data-cross-source-lab] [data-cs-direction-rows] b { font: 700 .49rem/1.2 var(--font-mono); text-align: right; }
|
||||
@media (max-width: 640px) {
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > header,
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > div { grid-template-columns: 1.2fr repeat(4,.7fr) .72fr; }
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > header > *,
|
||||
[data-cross-source-lab] [data-cs-task-matrix] > div > * { padding: .5rem .2rem; font-size: .36rem; }
|
||||
[data-cross-source-lab] [data-cs-direction-rows] > article { grid-template-columns: 1fr; }
|
||||
[data-cross-source-lab] [data-cs-direction-rows] b { text-align: left; }
|
||||
}
|
||||
</style>
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,941 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"protocol_id": "llm-atlas-deepseek-chat-cross-source-sampling-v1",
|
||||
"formal": {
|
||||
"path": "/tmp/deepseek-v2-lite-chat-cross-source-sampling-formal.json",
|
||||
"sha256": "f013132485f27adce008f03f781bed9982efc0d7939f13faede01c9f6f3d7f7c",
|
||||
"content_hash": "8fa14db0e9e6f1fd649a500796953ca948606a2c525d9c92c4b5f848beeaa3a7",
|
||||
"base_seeds": [
|
||||
2101325316,
|
||||
2511573438,
|
||||
1677220094,
|
||||
2412346607
|
||||
]
|
||||
},
|
||||
"rerun": {
|
||||
"path": "/tmp/deepseek-v2-lite-chat-cross-source-sampling-repro-r0.json",
|
||||
"sha256": "143dc9d0f7c914db4781e36b1401cdc9fbc2971a0dca71188bab0f8cedb002a6",
|
||||
"content_hash": "dc2706ed500e2ec3f7922b4aecbfd19247353d3b973162ea4bff44188b93e7f9",
|
||||
"base_seeds": [
|
||||
2101325316
|
||||
]
|
||||
},
|
||||
"rows": [
|
||||
{
|
||||
"source_id": "HumanEval/133",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/133",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/133",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/133",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/23",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/23",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/23",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/23",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/31",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/31",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/31",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/31",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/44",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/44",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/44",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "HumanEval/44",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/0144",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/0144",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/0144",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/0144",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1069",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1069",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1069",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1069",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1228",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1228",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1228",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1228",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1251",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1251",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1251",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "gsm8k/test/1251",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/1059",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/1059",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/1059",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/1059",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/3448",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/3448",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/3448",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/3448",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/4855",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/4855",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/4855",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/4855",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/8935",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/8935",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/8935",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "tnews/test/8935",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/0030",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/0030",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/0030",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/0030",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/0443",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/0443",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/0443",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/0443",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/2746",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/2746",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/2746",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/2746",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/2909",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/2909",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s0_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/2909",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_eos",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
},
|
||||
{
|
||||
"source_id": "wikitext2/raw-validation/2909",
|
||||
"base_seed": 2101325316,
|
||||
"condition": "s1_period",
|
||||
"run_seed_exact": true,
|
||||
"prompt_hash_exact": true,
|
||||
"generated_token_ids_exact": true,
|
||||
"decoded_text_exact": true,
|
||||
"eos_state_exact": true,
|
||||
"truncation_state_exact": true,
|
||||
"cpu_rng_pre_state_exact": true,
|
||||
"cuda_rng_pre_state_exact": true,
|
||||
"all_preregistered_fields_exact": true
|
||||
}
|
||||
],
|
||||
"summary": {
|
||||
"cells": 64,
|
||||
"all_preregistered_fields_exact": 64,
|
||||
"by_field": {
|
||||
"run_seed_exact": 64,
|
||||
"prompt_hash_exact": 64,
|
||||
"generated_token_ids_exact": 64,
|
||||
"decoded_text_exact": 64,
|
||||
"eos_state_exact": 64,
|
||||
"truncation_state_exact": 64,
|
||||
"cpu_rng_pre_state_exact": 64,
|
||||
"cuda_rng_pre_state_exact": 64
|
||||
}
|
||||
},
|
||||
"claim_boundary": [
|
||||
"Only the rerun seed subset is independently reproduced.",
|
||||
"Exact replay is scoped to the pinned software and hardware contract.",
|
||||
"Reproduction does not imply trajectories are seed-invariant."
|
||||
],
|
||||
"content_hash": "35562a6c5b5bdafd9472d8d638975b84e5ecf06d7d2061b64c341ddf28790da1"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -6,6 +6,7 @@ import DeepSeekArtifactLab from "@/components/DeepSeekArtifactLab.astro";
|
||||
import DeepSeekBehaviorLab from "@/components/DeepSeekBehaviorLab.astro";
|
||||
import DeepSeekCompletionDepthLab from "@/components/DeepSeekCompletionDepthLab.astro";
|
||||
import DeepSeekSamplingLab from "@/components/DeepSeekSamplingLab.astro";
|
||||
import DeepSeekCrossSourceSamplingLab from "@/components/DeepSeekCrossSourceSamplingLab.astro";
|
||||
import { deepseekBranches, deepseekLedgers, deepseekPaperChain, deepseekWaves } from "@/data/deepseek";
|
||||
|
||||
const toc = [
|
||||
@@ -35,21 +36,22 @@ const toc = [
|
||||
["23", "behavior", "Chat：最终生成行为"],
|
||||
["24", "completion-depth", "Chat：完成度与全深度"],
|
||||
["25", "sampling", "Chat：多种子采样稳健性"],
|
||||
["26", "branches", "别漏掉旁支"],
|
||||
["27", "audit", "事实、推导与教学模型"],
|
||||
["26", "cross-source-sampling", "Chat：跨题采样与统计单位"],
|
||||
["27", "branches", "别漏掉旁支"],
|
||||
["28", "audit", "事实、推导与教学模型"],
|
||||
["↳", "papers", "六十节点阅读链"],
|
||||
];
|
||||
---
|
||||
|
||||
<BaseLayout
|
||||
title="DeepSeek 技术谱系与真实权重深读：从 Dense、MoE、MLA 到 R1 与 V4"
|
||||
description="用二十四张问题账、十次技术转向、二十个交互实验、真实 V2-Lite Base / Chat 权重、512-token 完成度评测、29 阶段隐藏状态、26 层 MoE 路由追踪与 256 条多种子采样，完整理解 DeepSeek 的 MoE、MLA、FP8、DualPipe、GRPO、R1、V3.2 与 V4。"
|
||||
description="用二十四张问题账、十次技术转向、二十一个交互实验、真实 V2-Lite Base / Chat 权重、512-token 完成度评测、29 阶段隐藏状态、26 层 MoE 路由追踪，以及单题与跨题两轮各 256 条采样，完整理解 DeepSeek 的 MoE、MLA、FP8、DualPipe、GRPO、R1、V3.2 与 V4。"
|
||||
section="deepseek"
|
||||
>
|
||||
<header class="page-hero deepseek-hero">
|
||||
<div class="page-hero-inner">
|
||||
<div>
|
||||
<p class="eyebrow"><span>SPOTLIGHT / DEEPSEEK · ROUND 06</span> SAMPLING × COMPLETION × FULL DEPTH</p>
|
||||
<p class="eyebrow"><span>SPOTLIGHT / DEEPSEEK · ROUND 07</span> SOURCE COVERAGE × SAMPLING × FULL DEPTH</p>
|
||||
<h1>不要背模型名<br />要看懂每次为什么转向</h1>
|
||||
<p class="lead">
|
||||
这不是七篇报告的摘要，而是一套可追问、可计算、可反驳的技术谱系：
|
||||
@@ -61,9 +63,9 @@ const toc = [
|
||||
<div><dt>SPAN</dt><dd>2024.01 → 2026.06</dd></div>
|
||||
<div><dt>LEDGERS</dt><dd>24 张问题账</dd></div>
|
||||
<div><dt>LINEAGE</dt><dd>10 次技术转向</dd></div>
|
||||
<div><dt>LABS</dt><dd>20 个可操作实验</dd></div>
|
||||
<div><dt>LABS</dt><dd>21 个可操作实验</dd></div>
|
||||
<div><dt>EVIDENCE</dt><dd>60 个一手 / 官方节点</dd></div>
|
||||
<div><dt>STATUS</dt><dd>六轮 · 256 条采样</dd></div>
|
||||
<div><dt>STATUS</dt><dd>七轮 · 512 条采样</dd></div>
|
||||
</dl>
|
||||
</div>
|
||||
</header>
|
||||
@@ -825,8 +827,21 @@ const toc = [
|
||||
<DeepSeekSamplingLab />
|
||||
</section>
|
||||
|
||||
<section class="article-section" id="cross-source-sampling">
|
||||
<p class="eyebrow"><span>26</span> SEEDS ARE NOT TASKS</p>
|
||||
<h2>同一道题抽 64 次，仍然只有一道题：把预算移到 16 条预先冻结的 source</h2>
|
||||
<p class="lede">
|
||||
Round 06 证明了 sampled trajectory 会跨 seed 分叉，也能在固定执行合同下逐 token
|
||||
复现；但 Math 与 Code 各只有一道题。Round 07 保持 256 条正式输出的总预算不变，
|
||||
改为四域各四条 source、每格四个 seed，只保留
|
||||
<code>system off/on × EOS/句点</code> 四格。source 是主要覆盖单位，seed 是题内重复；
|
||||
方向先逐 source 计算，不把 64 个生成误写成 64 道独立任务。
|
||||
</p>
|
||||
<DeepSeekCrossSourceSamplingLab />
|
||||
</section>
|
||||
|
||||
<section class="article-section" id="branches">
|
||||
<p class="eyebrow"><span>26</span> THE MAIN LINE IS NOT THE WHOLE TREE</p>
|
||||
<p class="eyebrow"><span>27</span> THE MAIN LINE IS NOT THE WHOLE TREE</p>
|
||||
<h2>如果只读 V2 → V3 → R1 → V4，会漏掉五条反过来影响主线的旁支</h2>
|
||||
<div class="branch-grid">
|
||||
{deepseekBranches.map(([name, line, text, url]) => (
|
||||
@@ -846,7 +861,7 @@ const toc = [
|
||||
</section>
|
||||
|
||||
<section class="article-section" id="audit">
|
||||
<p class="eyebrow"><span>27</span> EVIDENCE AUDIT</p>
|
||||
<p class="eyebrow"><span>28</span> EVIDENCE AUDIT</p>
|
||||
<h2>同一张页面里有三种知识，它们的语气必须不同</h2>
|
||||
<div class="audit-grid">
|
||||
<article class="reported">
|
||||
|
||||
@@ -145,18 +145,18 @@ const paths = [
|
||||
</a>
|
||||
<a class="release-card deepseek-release" href="/deepseek/">
|
||||
<div>
|
||||
<p class="eyebrow"><span>NEW / DEEPSEEK ROUND 06</span> MULTI-SEED SAMPLING · EXACT REPLAY</p>
|
||||
<p class="eyebrow"><span>NEW / DEEPSEEK ROUND 07</span> CROSS-SOURCE SAMPLING · SOURCE-BLOCKED AUDIT</p>
|
||||
<h2>从 Dense 到百万上下文：每次创新都在偿还上一代最贵的一张账</h2>
|
||||
<p>
|
||||
在 512-token greedy 与 29-stage / 26-gate 全深度 trace 之后，再按预注册的
|
||||
8 个 seed 生成 256 条官方 nucleus samples：251 条自然 EOS、242 条不同完整
|
||||
token 轨迹；Math 62 / 64、Code 63 / 64，并在新进程复现 R0/R1 的 64 / 64 格。
|
||||
单题抽 64 次仍然只有一道题。新一轮保持 256 条预算不变，改用 16 条预先冻结的
|
||||
source：Math 四题从 8 / 16 到 16 / 16，Code 四题也从 8 / 16 到 16 / 16；
|
||||
English 甚至出现域均值与 3 / 4 source 方向相反。新进程 R0 仍 64 / 64 格 exact。
|
||||
</p>
|
||||
</div>
|
||||
<dl>
|
||||
<div><dt>LINEAGE</dt><dd>1991 → 2026 · 10 次转向</dd></div>
|
||||
<div><dt>NODES</dt><dd>60 个一手 / 官方节点</dd></div>
|
||||
<div><dt>LAB</dt><dd>20 · Base / Chat / sampling</dd></div>
|
||||
<div><dt>LAB</dt><dd>21 · Base / Chat / sampling</dd></div>
|
||||
</dl>
|
||||
<span class="release-arrow" aria-hidden="true">进入 DeepSeek 完整技术谱系 →</span>
|
||||
</a>
|
||||
|
||||
@@ -15,7 +15,7 @@ const workstreams = [
|
||||
{ label: "表示、位置与残差高速公路", value: 81, next: "加入真实 hidden-state / norm traces、长上下文位置外推复现与更多深层稳定性消融" },
|
||||
{ label: "Scaling Laws", value: 74, next: "加入真实拟合复现、置信区间与更多模型族对照" },
|
||||
{ label: "数据工程与预训练配方", value: 73, next: "逐图精读 FineWeb / DCLM，加入真实去重与 mixture traces" },
|
||||
{ label: "DeepSeek 专题", value: 99, next: "扩大 sampling 的 source/task 覆盖并推进干预式 mediation、SM90 FlashMLA、FP8/pipeline 与 R1-like RL" },
|
||||
{ label: "DeepSeek 专题", value: 99, next: "把跨题采样扩大到可做 task-level bootstrap，再推进 per-row RNG、干预式 mediation、SM90 FlashMLA 与 R1-like RL" },
|
||||
{ label: "指令微调与人类偏好", value: 75, next: "加入真实偏好分歧样本、RM 长度偏置与 PPO/DPO 小模型复现" },
|
||||
{ label: "推理与测试时扩展", value: 76, next: "真实模型采样曲线、PRM 案例与逐篇图表精读" },
|
||||
{ label: "工具使用与长程 Agent", value: 74, next: "补真实环境 traces、cross-harness 对照、Agent RL 训练曲线与安全案例" },
|
||||
@@ -50,7 +50,7 @@ const workstreams = [
|
||||
<div><dt>OVERALL</dt><dd>专题平均 {average}%</dd></div>
|
||||
<div><dt>READABLE</dt><dd>{published} 个首版可读专题</dd></div>
|
||||
<div><dt>ACTIVE</dt><dd>{researching} 个研究/写作中</dd></div>
|
||||
<div><dt>UPDATED</dt><dd>2026-07-30 01:50 CST</dd></div>
|
||||
<div><dt>UPDATED</dt><dd>2026-07-30 04:00 CST</dd></div>
|
||||
<div><dt>MODE</dt><dd>持续迭代，不锁死版本</dd></div>
|
||||
</dl>
|
||||
</div>
|
||||
@@ -97,12 +97,12 @@ const workstreams = [
|
||||
<article><span>✓</span><h3>K3 报告已结构化拆解</h3><p>47 页报告目录、151 条参考来源和架构/后训练/系统主线已经提取。</p></article>
|
||||
<article><span>✓</span><h3>17 专题知识图</h3><p>从语言模型基础到评测安全，包含先修依赖和三条贯穿案例。</p></article>
|
||||
<article><span>✓</span><h3>编辑式网站系统</h3><p>响应式导航、章节模板、侧栏、进度、论文链和证据提示组件。</p></article>
|
||||
<article><span>✓</span><h3>八十七个原创交互视图</h3><p>K3 三轴图、八联报告实验与四联开放工件实验，DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth 与 multi-seed sampling 三轮实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
|
||||
<article><span>✓</span><h3>八十八个原创交互视图</h3><p>K3 三轴图、八联报告实验与四联开放工件实验，DeepSeek 四联公式实验、十三联 Base 工件实验、Chat 行为、completion/full-depth、multi-seed 与 cross-source sampling 四轮实验，以及语言模型前史、Transformer、表示深度、长上下文、MoE、推理、Agent、多模态、训练系统、推理服务、Scaling、数据工程、数值、Alignment 与评测安全专题。</p></article>
|
||||
<article><span>✓</span><h3>十七篇首版长文</h3><p>K3、语言模型前史、Transformer、表示/位置/残差、DeepSeek、Scaling、数据工程、长上下文、MoE、后训练、推理、Agent、原生多模态、训练系统、推理服务、数值优化与评测安全专题。</p></article>
|
||||
<article><span>✓</span><h3>语言模型前史深度专题</h3><p>八张独立问题账、33 个正式节点、20 段长文与概率—向量—记忆—对齐四联实验。</p></article>
|
||||
<article><span>✓</span><h3>Transformer 深度专题</h3><p>十张独立问题账、40 个正式节点、21 段正文与 QKV—Mask—多头位置—Block 成本四联实验。</p></article>
|
||||
<article><span>✓</span><h3>表示、位置与残差高速公路深度专题</h3><p>二十张问题账、66 个一手节点、DeepSeek/Kimi 双谱系，以及 Token—位置—Norm—Residual/FFN 四联实验。</p></article>
|
||||
<article><span>✓</span><h3>DeepSeek 六轮真实权重里程碑</h3><p>在 512-token greedy 与 29-stage / 26-gate 全深度 trace 之后，预注册 8 个 SHA-256 seed，生成 256 条 official nucleus samples：251 条 natural EOS、242 个 unique trajectory hashes；GSM8K 62 / 64 strict exact、HumanEval 63 / 64 tests pass，新进程 R0/R1 的八项合同字段 64 / 64 exact。</p></article>
|
||||
<article><span>✓</span><h3>DeepSeek 七轮真实权重里程碑</h3><p>继单题多 seed 之后，保持 256 条预算并把覆盖扩大到 16 条预先冻结的 source：250 条 natural EOS、247 个 unique trajectories；Math 四题为 14/16、16/16、8/16、9/16，Code 四题为 16/16、8/16、12/16、16/16。source-blocked 方向揭示 English 均值与多数题相反，新进程 R0 八项合同字段 64 / 64 exact。</p></article>
|
||||
<article><span>✓</span><h3>Kimi K3 技术报告二轮深读</h3><p>三十二张问题账、Figure 1–16 / Table 1–5 审计、100 节点阅读链，以及 Delta—Decay—AttnRes—LatentMoE—SiTU—QB—MOPD—Cache 八联实验。</p></article>
|
||||
<article><span>✓</span><h3>Kimi K3 三轮开放工件里程碑</h3><p>固定官方 revisions，审计 96 个 shards、497,220 个 tensor entries 与真实 KDA / MLA / MoE / MoonViT shapes；四联实验分开显示层型、tensor anatomy、参数范围和复现边界。</p></article>
|
||||
<article><span>✓</span><h3>FlashKDA RTX 5090 执行闸门</h3><p>隔离 CUDA 13.0 / glibc 2.39 编译 sm_120a wheel；6/6 官方参考逐元素相等，并完成 fixed / varlen、三种 state mode 的 1,800 个 CUDA Event samples。</p></article>
|
||||
@@ -134,7 +134,7 @@ const workstreams = [
|
||||
<div class="queue-table">
|
||||
<div class="head"><b>优先级</b><b>专题</b><b>本轮交付</b><b>完成闸门</b></div>
|
||||
<div><span>P0</span><strong>K3 三轮</strong><p>开放权重 traces → FlashKDA / AttnRes / MoE 真实行为 → Figure 1–16 数值重绘与独立复现</p><em>运行证据 + 逐图复现</em></div>
|
||||
<div><span>P0</span><strong>DeepSeek 六轮后续</strong><p>扩大任务与语言 source 的 sampling 覆盖 → 干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
|
||||
<div><span>P0</span><strong>DeepSeek 七轮后续</strong><p>扩大到 task-level bootstrap → per-row RNG 对照 → 干预式 mediation → SM90 FlashMLA / FP8 / pipeline traces → R1-like RL 小模型复现</p><em>运行证据 + 独立复现</em></div>
|
||||
<div><span>P0</span><strong>Transformer 二轮</strong><p>多头电路逐图 → Pre/Post-LN 真实 traces → Flash/KV 配置与 kernel 对照</p><em>逐图笔记 + 实测边界</em></div>
|
||||
<div><span>P0</span><strong>表示、位置与残差二轮</strong><p>真实 hidden-state / norm traces → 长上下文位置外推 → mHC / AttnRes 深层稳定性消融</p><em>可复现实验 + 逐图笔记</em></div>
|
||||
<div><span>P0</span><strong>语言模型前史二轮</strong><p>Kneser–Ney / LSTM / Bahdanau 逐图 → 真实小语料复现 → tokenizer 公平性</p><em>可复现实验 + 逐图笔记</em></div>
|
||||
@@ -233,6 +233,10 @@ const workstreams = [
|
||||
<div><time>2026-07-29</time><b>停止、终点、可评测与正确分四张账</b><p>自然 EOS 不等于答对；fallback 不冒充 strict completion；HumanEval tests pass 不冒充代码安全。</p></div>
|
||||
<div><time>2026-07-29</time><b>全深度比较只保留 exact interior tokens</b><p>1,537 个 content tokens / condition 在八格中 ID exact；56 个跨字符边界 token 排除，不拿不同 token 比隐藏状态。</p></div>
|
||||
<div><time>2026-07-29</time><b>表示与路由分叉不是中介因果</b><p>29-stage hidden 与 26-gate route 曲线描述传播；没有干预式 mediation 前不解释输出或能力因果。</p></div>
|
||||
<div><time>2026-07-30</time><b>seed 是题内重复，不是任务覆盖</b><p>Round 07 用 16 sources × 4 seeds × 4 cells 保持 256 条预算；所有方向先在 source 内计算，再数四题方向，不把 seed 当独立 benchmark 题。</p></div>
|
||||
<div><time>2026-07-30</time><b>任务总数必须展开为逐题矩阵</b><p>Math 与 Code 的四题都从 8/16 跨到 16/16；跨不同 GSM8K gold 的 final-answer frequency 禁止聚合。</p></div>
|
||||
<div><time>2026-07-30</time><b>均值与 source 方向同时展示</b><p>English 句点 contrast 均值 −8.5，但 3/4 source 为正；Code 两道题 +1/−1 interaction 在域均值 0 中抵消。</p></div>
|
||||
<div><time>2026-07-30</time><b>跨题 sampling 仍要求精确复跑</b><p>R0/R1 63/64 同格分叉；新进程 R0 的 seed、prompt、token、text、stop 与 CPU/CUDA RNG pre-state 八字段 64/64 exact。</p></div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
|
||||
Reference in New Issue
Block a user