feat: add DeepSeek equal-length history control
This commit is contained in:
@@ -264,3 +264,58 @@ See `research/DEEPSEEK_ROUTING_HISTORY_FACTORIAL_AUDIT.md` for the factorial
|
||||
definition, all CV intervals, the 24/24 system-edge TV attenuation pattern,
|
||||
per-token stability, and the strict boundary between a message-history effect
|
||||
and a role-semantic claim.
|
||||
|
||||
## Equal-length filler history control
|
||||
|
||||
`v2_lite_routing_history_distance_control.py` extends the preceding 2×2 probe
|
||||
to six cells:
|
||||
|
||||
```text
|
||||
system off/on × no history / repeated-token filler / fixed one-shot
|
||||
```
|
||||
|
||||
The filler turn is:
|
||||
|
||||
```text
|
||||
User: x x x x x x x x x
|
||||
Assistant: x
|
||||
```
|
||||
|
||||
Under the pinned official template, both filler and one-shot add exactly 17
|
||||
tokens per source on both system levels. They therefore share the same
|
||||
user/assistant roles, assistant EOS, target position, generation prompt, and
|
||||
batch shape. The filler is deliberately called low-information rather than
|
||||
semantics-free: repeated `x` tokens remain learned inputs.
|
||||
|
||||
```bash
|
||||
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
|
||||
python -B experiments/deepseek/v2_lite_routing_history_distance_control.py \
|
||||
--artifact-dir /path/to/deepseek-v2-lite \
|
||||
--human-eval /path/to/HumanEval.jsonl.gz \
|
||||
--gsm8k /path/to/gsm8k/test.jsonl \
|
||||
--tnews /path/to/tnews/test.json \
|
||||
--tnews-archive /path/to/tnews_public.zip \
|
||||
--wikitext /path/to/wikitext-validation.parquet \
|
||||
--output src/data/deepseek-v2-lite-routing-history-distance-control.json \
|
||||
--per-domain 32 \
|
||||
--content-tokens 23 \
|
||||
--batch-prompts 5 \
|
||||
--layers 7 \
|
||||
--bootstrap 2000 \
|
||||
--seed 20260729 \
|
||||
--captured-at 2026-07-29T10:17:00+00:00
|
||||
```
|
||||
|
||||
The six cells add 1,376,496 real top-6 route selections. The committed run and
|
||||
independent rerun are byte-exact:
|
||||
|
||||
```text
|
||||
423a095d92738d4bc6c2ace2afa9b35627e61811458c7883dca7b44aee4e648e
|
||||
```
|
||||
|
||||
For exact target content under prompt-balanced aggregation, mean system-edge
|
||||
TV is `0.0738 → 0.0378 → 0.0188` for none → filler → demo. Both steps decrease
|
||||
in 24/24 layer×domain cells, with all 24 paired intervals below zero. See
|
||||
`research/DEEPSEEK_ROUTING_HISTORY_DISTANCE_CONTROL_AUDIT.md` for the full
|
||||
table, token-level route stability, BF16 batch-shape boundary, literature
|
||||
context, and non-claims.
|
||||
|
||||
Reference in New Issue
Block a user