feat: factor DeepSeek boundary and role blocks

2026-07-29 21:42:07 +08:00
parent 8cebb8cb30
commit 1d0a331e06
21 changed files with 8060293 additions and 33 deletions
@@ -425,3 +425,109 @@ target-token ordered top-6 routes, with zero target TV, JSD, and ΔCV. See
 depth maps, token-level alignment, cross-experiment BF16 batch-content audit,
 primary sources, and the boundary against full role semantics or Chat-model
 behavior.
+
+## Special-token family control
+
+`v2_lite_routing_special_token_family_control.py` keeps the same repeated-token
+history and changes the completed assistant boundary to four single IDs:
+
+```text
+system off/on × EOS / BOS / x / period
+```
+
+The pinned tokenizer has exactly two special-token IDs: BOS `100000` and EOS
+`100001`; PAD aliases EOS. EOS/BOS therefore exhaust the special inventory,
+while `x` and period are only two selected ordinary controls. The 2-vs-2
+family summary is descriptive for these four IDs and is not a
+population-level specialness claim.
+
+```bash
+PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
+python -B experiments/deepseek/v2_lite_routing_special_token_family_control.py \
+  --artifact-dir /path/to/deepseek-v2-lite \
+  --human-eval /path/to/HumanEval.jsonl.gz \
+  --gsm8k /path/to/gsm8k/test.jsonl \
+  --tnews /path/to/tnews/test.json \
+  --tnews-archive /path/to/tnews_public.zip \
+  --wikitext /path/to/wikitext-validation.parquet \
+  --output src/data/deepseek-v2-lite-routing-special-token-family-control.json \
+  --per-domain 32 \
+  --content-tokens 23 \
+  --batch-prompts 4 \
+  --layers 7 \
+  --bootstrap 2000 \
+  --seed 20260729 \
+  --captured-at 2026-07-29T12:57:00+00:00
+```
+
+The eight cells add 2,044,224 real top-6 route selections. All 256
+source×system groups preserve length and target position, and all 768
+counterfactuals change exactly one ID. The committed run and independent
+rerun are byte-exact:
+
+```text
+c372c1b03a8b15f615b54ded5d9257a8fc2cdb7728001735d3d4c1d8534af5bf
+```
+
+For exact target content under prompt-balanced aggregation, mean system-edge
+TV is `.037518 / .048018 / .053975 / .055289` for EOS / BOS / x / period.
+BOS−EOS is positive in 22/24 layer×domain cells; x−EOS and period−EOS are
+positive in 24/24. The selected ordinary-control mean exceeds the complete
+special-inventory mean by `.011864` in these four IDs only. See
+`research/DEEPSEEK_ROUTING_SPECIAL_TOKEN_FAMILY_AUDIT.md` for all paired
+intervals, direct edges, scope split, alignment, batch-content audit,
+literature context, and non-claims.
+
+## Full two-token role-marker factorial
+
+`v2_lite_routing_role_marker_block_factorial.py` treats the pre-target
+two-token marker as two independent factors:
+
+```text
+system off/on × head User/Assistant × delimiter colon/x
+
+User:       [5726, 25]      Assistant:  [77398, 25]
+User x      [5726, 87]      Assistant x [77398, 87]
+```
+
+The official assistant EOS and post-target generation suffix remain unchanged.
+The four blocks have zero, one, one, and two edited IDs relative to official
+`User:`; every edit is verified while length, target position, attention mask,
+and 32-row batch shape stay fixed.
+
+```bash
+PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
+python -B experiments/deepseek/v2_lite_routing_role_marker_block_factorial.py \
+  --artifact-dir /path/to/deepseek-v2-lite \
+  --human-eval /path/to/HumanEval.jsonl.gz \
+  --gsm8k /path/to/gsm8k/test.jsonl \
+  --tnews /path/to/tnews/test.json \
+  --tnews-archive /path/to/tnews_public.zip \
+  --wikitext /path/to/wikitext-validation.parquet \
+  --output src/data/deepseek-v2-lite-routing-role-marker-block-factorial.json \
+  --per-domain 32 \
+  --content-tokens 23 \
+  --batch-prompts 4 \
+  --layers 7 \
+  --bootstrap 2000 \
+  --seed 20260729 \
+  --captured-at 2026-07-29T13:06:00+00:00
+```
+
+The eight cells add 2,044,224 real top-6 route selections. The 256 official,
+512 one-ID, and 256 two-ID cells all pass their exact edit contracts. The
+committed run and independent rerun are byte-exact:
+
+```text
+a703dddb6d04182b1a608c213bfd341cfc32e1801be42c417dca30a505087e82
+```
+
+For exact target content under prompt-balanced aggregation, the four mean
+system-edge TVs are `.037304 / .037015 / .037239 / .036220`. The head,
+delimiter, and head×delimiter effects are small and mixed across the 24
+layer×domain cells. Direct head and delimiter edges remain nonzero; replacing
+colon with the single `x` control modestly reduces User-vs-Assistant direct TV
+in most cells, without a uniform system modulation. See
+`research/DEEPSEEK_ROUTING_ROLE_MARKER_BLOCK_AUDIT.md` for the exact factor
+coding, intervals, direct dependencies, full-input split, route alignment,
+cross-batch audit, sources, and non-claims.