feat: add bootstrapped DeepSeek routing corpus

This commit is contained in:
wuyang
2026-07-29 15:32:03 +08:00
parent b7fef2350f
commit 5bcfd58452
12 changed files with 158485 additions and 32 deletions
+46
View File
@@ -54,3 +54,49 @@ python experiments/deepseek/compare_v2_lite_traces.py \
--second /path/to/trace-2.json \
--output src/data/deepseek-v2-lite-trace-repro.json
```
## Fixed public routing corpus
`v2_lite_routing_corpus.py` keeps the same official layer 0–6 execution boundary
but replaces the four authored prompts with 128 source-addressable public
prompts:
- 32 WikiText-2 raw validation passages;
- 32 CLUE TNEWS public-test sentences;
- 32 OpenAI HumanEval prompts, without solutions/tests or code execution;
- 32 OpenAI GSM8K test questions, without answers.
Selection is the ascending SHA-256 rank of a fixed salt, domain, and source ID.
Inputs are truncated to 96 DeepSeek tokens. The six MoE layers therefore produce
304,560 actual top-6 routed-expert selections over 8,460 valid tokens.
The output includes both token-weighted and prompt-balanced distributions. Its
95% intervals use 2,000 prompt-level bootstrap resamples within each domain,
rather than treating correlated tokens as independent observations.
```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps \
python -B experiments/deepseek/v2_lite_routing_corpus.py \
--artifact-dir /path/to/deepseek-v2-lite \
--human-eval /path/to/HumanEval.jsonl.gz \
--gsm8k /path/to/gsm8k/test.jsonl \
--tnews /path/to/tnews/test.json \
--tnews-archive /path/to/tnews_public.zip \
--wikitext /path/to/wikitext-validation.parquet \
--output src/data/deepseek-v2-lite-routing-corpus.json \
--per-domain 32 \
--max-tokens 96 \
--batch-size 16 \
--bootstrap 2000 \
--seed 20260729 \
--captured-at 2026-07-29T07:45:00+00:00
```
The committed independent rerun is byte-exact. Both JSON files have SHA-256:
```text
4678a1d15395de93ffba757598cc3642bf35e9e07f71d82ddd87c27fc38a09e4
```
See `research/DEEPSEEK_ROUTING_CORPUS_AUDIT.md` for corpus revisions and hashes,
metric definitions, interval semantics, results, and claim boundaries.