feat: add DeepSeek message-history factorial probe

2026-07-29 18:05:59 +08:00
parent 28f8066626
commit efe3ffcb4e
11 changed files with 1890088 additions and 33 deletions
@@ -205,3 +205,62 @@ da1f10333b2fa269e64f9716a0ca6c1a656d23d3e70b8a25d6e59f8a5b3bc1b9

 See `research/DEEPSEEK_ROUTING_TEMPLATE_AUDIT.md` for the aligned-content
 contract, paired intervals, per-token route stability, and claim boundaries.
+
+## System × one-shot message-history factorial
+
+`v2_lite_routing_history_factorial_probe.py` reuses the exact 128-source,
+23-content-token cohort from the official-template probe and renders four
+official chat histories:
+
+```text
+S0F0  target user
+S1F0  fixed system + target user
+S0F1  fixed demo user/assistant + target user
+S1F1  fixed system + fixed demo user/assistant + target user
+```
+
+Every condition uses `add_generation_prompt=True`. The fixed system treatment
+adds 16 tokens per source in both `F0` and `F1`; the fixed one-shot treatment
+adds 17 tokens per source in both `S0` and `S1`. All four variants of one
+source execute in the same padded batch.
+
+The output contains:
+
+- full-input and exact target-content scopes;
+- token-weighted and prompt-balanced aggregation;
+- four paired factor edges;
+- system and one-shot main effects;
+- difference-in-differences interaction;
+- per-target-token ordered/set top-6 stability and Jaccard;
+- 2,000 source-prompt bootstrap replicates shared by all four cells.
+
+```bash
+PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
+python -B experiments/deepseek/v2_lite_routing_history_factorial_probe.py \
+  --artifact-dir /path/to/deepseek-v2-lite \
+  --human-eval /path/to/HumanEval.jsonl.gz \
+  --gsm8k /path/to/gsm8k/test.jsonl \
+  --tnews /path/to/tnews/test.json \
+  --tnews-archive /path/to/tnews_public.zip \
+  --wikitext /path/to/wikitext-validation.parquet \
+  --output src/data/deepseek-v2-lite-routing-history-factorial.json \
+  --per-domain 32 \
+  --content-tokens 23 \
+  --batch-prompts 8 \
+  --layers 7 \
+  --bootstrap 2000 \
+  --seed 20260729 \
+  --captured-at 2026-07-29T09:36:00+00:00
+```
+
+The four cells add 865,440 real top-6 route selections. The committed run and
+independent rerun are byte-exact:
+
+```text
+5765fbf85fa6fd948cca90f11e2237254fd58c32f373670f44530cff0a70c1fb
+```
+
+See `research/DEEPSEEK_ROUTING_HISTORY_FACTORIAL_AUDIT.md` for the factorial
+definition, all CV intervals, the 24/24 system-edge TV attenuation pattern,
+per-token stability, and the strict boundary between a message-history effect
+and a role-semantic claim.