feat: add DeepSeek equal-length history control

This commit is contained in:
wuyang
2026-07-29 19:01:17 +08:00
parent bdd93a5222
commit 8130e3745f
11 changed files with 2854526 additions and 38 deletions
+55
View File
@@ -264,3 +264,58 @@ See `research/DEEPSEEK_ROUTING_HISTORY_FACTORIAL_AUDIT.md` for the factorial
definition, all CV intervals, the 24/24 system-edge TV attenuation pattern,
per-token stability, and the strict boundary between a message-history effect
and a role-semantic claim.
## Equal-length filler history control
`v2_lite_routing_history_distance_control.py` extends the preceding 2×2 probe
to six cells:
```text
system off/on × no history / repeated-token filler / fixed one-shot
```
The filler turn is:
```text
User: x x x x x x x x x
Assistant: x
```
Under the pinned official template, both filler and one-shot add exactly 17
tokens per source on both system levels. They therefore share the same
user/assistant roles, assistant EOS, target position, generation prompt, and
batch shape. The filler is deliberately called low-information rather than
semantics-free: repeated `x` tokens remain learned inputs.
```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
python -B experiments/deepseek/v2_lite_routing_history_distance_control.py \
--artifact-dir /path/to/deepseek-v2-lite \
--human-eval /path/to/HumanEval.jsonl.gz \
--gsm8k /path/to/gsm8k/test.jsonl \
--tnews /path/to/tnews/test.json \
--tnews-archive /path/to/tnews_public.zip \
--wikitext /path/to/wikitext-validation.parquet \
--output src/data/deepseek-v2-lite-routing-history-distance-control.json \
--per-domain 32 \
--content-tokens 23 \
--batch-prompts 5 \
--layers 7 \
--bootstrap 2000 \
--seed 20260729 \
--captured-at 2026-07-29T10:17:00+00:00
```
The six cells add 1,376,496 real top-6 route selections. The committed run and
independent rerun are byte-exact:
```text
423a095d92738d4bc6c2ace2afa9b35627e61811458c7883dca7b44aee4e648e
```
For exact target content under prompt-balanced aggregation, mean system-edge
TV is `0.0738 → 0.0378 → 0.0188` for none → filler → demo. Both steps decrease
in 24/24 layer×domain cells, with all 24 paired intervals below zero. See
`research/DEEPSEEK_ROUTING_HISTORY_DISTANCE_CONTROL_AUDIT.md` for the full
table, token-level route stability, BF16 batch-shape boundary, literature
context, and non-claims.