llm-atlas/experiments/deepseek/README.md

# DeepSeek real-weight execution probes

The probes in this directory use official DeepSeek artifacts and keep their
scope deliberately narrower than a full-model benchmark.

## DeepSeek-V2-Lite truncated trace

`v2_lite_trace.py` executes layers 0–6 from the official BF16 checkpoint. Those
seven layers are fully contained in shard 1; layer 7 is split across shards 1
and 2 and is therefore outside the default evidence boundary.

Pinned model:

```text
deepseek-ai/DeepSeek-V2-Lite@604d5664dddd88a0433dbae533b7fe9472482de0
```

Required Python stack:

```text
torch==2.11.0+cu128
transformers==4.41.2
safetensors==0.8.0
```

The 2024 remote code does not import under Transformers 5.5 because
`is_torch_fx_available` was removed. The probe imports the official files as a
read-only local package; it does not patch the model source.

Download the metadata, tokenizer, remote code, index, and first shard with the
Hugging Face CLI, then run:

```bash
python experiments/deepseek/v2_lite_trace.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --output src/data/deepseek-v2-lite-trace.json
```

The result contains:

- real tokenizer pieces and model-derived hidden states;
- the actual `[B,T,576]` MLA compressed projection at each executed layer;
- the expanded key/value tensors stored by the Hugging Face eager cache;
- token-level top-6 routed expert IDs and weights for six MoE layers;
- per-layer and per-prompt expert-load summaries;
- explicit boundaries against global load, expert semantics, training traces,
  full-model generation, and production serving claims.

Run the probe twice and compare deterministic evidence while excluding timing:

```bash
python experiments/deepseek/compare_v2_lite_traces.py \
  --first /path/to/trace-1.json \
  --second /path/to/trace-2.json \
  --output src/data/deepseek-v2-lite-trace-repro.json
```

## Fixed public routing corpus

`v2_lite_routing_corpus.py` keeps the same official layer 0–6 execution boundary
but replaces the four authored prompts with 128 source-addressable public
prompts:

- 32 WikiText-2 raw validation passages;
- 32 CLUE TNEWS public-test sentences;
- 32 OpenAI HumanEval prompts, without solutions/tests or code execution;
- 32 OpenAI GSM8K test questions, without answers.

Selection is the ascending SHA-256 rank of a fixed salt, domain, and source ID.
Inputs are truncated to 96 DeepSeek tokens. The six MoE layers therefore produce
304,560 actual top-6 routed-expert selections over 8,460 valid tokens.

The output includes both token-weighted and prompt-balanced distributions. Its
95% intervals use 2,000 prompt-level bootstrap resamples within each domain,
rather than treating correlated tokens as independent observations.

```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps \
python -B experiments/deepseek/v2_lite_routing_corpus.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-corpus.json \
  --per-domain 32 \
  --max-tokens 96 \
  --batch-size 16 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T07:45:00+00:00
```

The committed independent rerun is byte-exact. Both JSON files have SHA-256:

```text
4678a1d15395de93ffba757598cc3642bf35e9e07f71d82ddd87c27fc38a09e4
```

See `research/DEEPSEEK_ROUTING_CORPUS_AUDIT.md` for corpus revisions and hashes,
metric definitions, interval semantics, results, and claim boundaries.

## Paired length-control cohort

The corpus runner can also select one fixed cohort by untruncated source length
and execute nested prefixes. The committed 16-token and 24-token traces use the
same 128 source prompts, all selected from records with at least 24 DeepSeek
tokens:

```bash
common_args=(
  --per-domain 32
  --batch-size 16
  --bootstrap 2000
  --sample-salt llm-atlas-deepseek-routing-length-control-v1
  --eligibility-min-tokens 24
)

python -B experiments/deepseek/v2_lite_routing_corpus.py \
  ...source arguments... \
  "${common_args[@]}" \
  --max-tokens 16 \
  --output src/data/deepseek-v2-lite-routing-matched16.json

python -B experiments/deepseek/v2_lite_routing_corpus.py \
  ...source arguments... \
  "${common_args[@]}" \
  --max-tokens 24 \
  --output src/data/deepseek-v2-lite-routing-matched24.json
```

The two real traces add 184,320 top-6 route selections. Their independent
reruns are byte-exact:

```text
matched-16  f8d437d5379ffb41ac8dca5a8e97c0f44ba10ce7b63f95d7be0b7c88ac0baebd
matched-24  bed54835ad243ca2ab46bf9574e53137e6c2c0e19267f581719b5f2f65546436
```

Use the paired comparison runner to resample identical prompt indices in the
short and long traces:

```bash
python -B experiments/deepseek/compare_routing_length_control.py \
  --short src/data/deepseek-v2-lite-routing-matched16.json \
  --long src/data/deepseek-v2-lite-routing-matched24.json \
  --output src/data/deepseek-v2-lite-routing-length-sensitivity.json \
  --bootstrap 2000 \
  --seed 20260729
```

See `research/DEEPSEEK_ROUTING_LENGTH_CONTROL_AUDIT.md` for the sampling bias
audit, paired CV/JSD deltas, total-variation accounting, and interpretation
boundaries.

## Official chat-template sensitivity

`v2_lite_routing_template_probe.py` renders three variants of one fixed
23-content-token prefix:

```text
raw         BOS + content
user        BOS + "User: " + content + "\n\n"
generation  user prefix + "Assistant:"
```

The latter two use the pinned official `chat_template` through
`apply_chat_template`. All three variants of one source prompt execute in the
same padded batch. Statistics are split between:

- the full operational input, including wrapper tokens;
- the exact intersection of `(relative character span, token ID)` inside the
  source content across all three variants.

The `user → generation` comparison is a causal negative control: the appended
suffix must not change routes on their shared prefix.

```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
python -B experiments/deepseek/v2_lite_routing_template_probe.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-template.json \
  --per-domain 32 \
  --content-tokens 23 \
  --batch-prompts 8 \
  --layers 7 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T08:30:00+00:00
```

The three variants add 382,032 real top-6 route selections. Across six MoE
layers, all 21,852 `user → generation` shared-prefix token routes are
ordered-top-6 exact. The committed run and independent rerun are byte-exact:

```text
da1f10333b2fa269e64f9716a0ca6c1a656d23d3e70b8a25d6e59f8a5b3bc1b9
```

See `research/DEEPSEEK_ROUTING_TEMPLATE_AUDIT.md` for the aligned-content
contract, paired intervals, per-token route stability, and claim boundaries.

## System × one-shot message-history factorial

`v2_lite_routing_history_factorial_probe.py` reuses the exact 128-source,
23-content-token cohort from the official-template probe and renders four
official chat histories:

```text
S0F0  target user
S1F0  fixed system + target user
S0F1  fixed demo user/assistant + target user
S1F1  fixed system + fixed demo user/assistant + target user
```

Every condition uses `add_generation_prompt=True`. The fixed system treatment
adds 16 tokens per source in both `F0` and `F1`; the fixed one-shot treatment
adds 17 tokens per source in both `S0` and `S1`. All four variants of one
source execute in the same padded batch.

The output contains:

- full-input and exact target-content scopes;
- token-weighted and prompt-balanced aggregation;
- four paired factor edges;
- system and one-shot main effects;
- difference-in-differences interaction;
- per-target-token ordered/set top-6 stability and Jaccard;
- 2,000 source-prompt bootstrap replicates shared by all four cells.

```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
python -B experiments/deepseek/v2_lite_routing_history_factorial_probe.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-history-factorial.json \
  --per-domain 32 \
  --content-tokens 23 \
  --batch-prompts 8 \
  --layers 7 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T09:36:00+00:00
```

The four cells add 865,440 real top-6 route selections. The committed run and
independent rerun are byte-exact:

```text
5765fbf85fa6fd948cca90f11e2237254fd58c32f373670f44530cff0a70c1fb
```

See `research/DEEPSEEK_ROUTING_HISTORY_FACTORIAL_AUDIT.md` for the factorial
definition, all CV intervals, the 24/24 system-edge TV attenuation pattern,
per-token stability, and the strict boundary between a message-history effect
and a role-semantic claim.

## Equal-length filler history control

`v2_lite_routing_history_distance_control.py` extends the preceding 2×2 probe
to six cells:

```text
system off/on × no history / repeated-token filler / fixed one-shot
```

The filler turn is:

```text
User: x x x x x x x x x
Assistant: x
```

Under the pinned official template, both filler and one-shot add exactly 17
tokens per source on both system levels. They therefore share the same
user/assistant roles, assistant EOS, target position, generation prompt, and
batch shape. The filler is deliberately called low-information rather than
semantics-free: repeated `x` tokens remain learned inputs.

```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
python -B experiments/deepseek/v2_lite_routing_history_distance_control.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-history-distance-control.json \
  --per-domain 32 \
  --content-tokens 23 \
  --batch-prompts 5 \
  --layers 7 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T10:17:00+00:00
```

The six cells add 1,376,496 real top-6 route selections. The committed run and
independent rerun are byte-exact:

```text
423a095d92738d4bc6c2ace2afa9b35627e61811458c7883dca7b44aee4e648e
```

For exact target content under prompt-balanced aggregation, mean system-edge
TV is `0.0738 → 0.0378 → 0.0188` for none → filler → demo. Both steps decrease
in 24/24 layer×domain cells, with all 24 paired intervals below zero. See
`research/DEEPSEEK_ROUTING_HISTORY_DISTANCE_CONTROL_AUDIT.md` for the full
table, token-level route stability, BF16 batch-shape boundary, literature
context, and non-claims.

## History-boundary single-token control

`v2_lite_routing_history_boundary_token_control.py` keeps the repeated-token
history from the preceding probe and changes exactly one token ID at the
completed assistant boundary:

```text
system off/on × official EOS / x / period / newline
```

The official template places EOS between the filler assistant content and the
next `User:` marker. The three controls replace only that EOS ID after official
tokenization. They are explicit counterfactual token sequences, not valid
official chat serializations. All eight conditions preserve sequence length,
target position, role markers, attention mask, and within-run batch shape.

```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
python -B experiments/deepseek/v2_lite_routing_history_boundary_token_control.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-history-boundary-token-control.json \
  --per-domain 32 \
  --content-tokens 23 \
  --batch-prompts 4 \
  --layers 7 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T11:12:00+00:00
```

The eight cells add 2,044,224 real top-6 route selections. All 256
source×system groups are equal-length and equal-position; all 768
counterfactual cells differ from their official sequence at exactly one input
ID. The committed run and independent rerun are byte-exact:

```text
9bb93834ffd8536aeebe4325e45d2179ba590554c6ff6b6fcceaba2f499b9c37
```

For exact target content under prompt-balanced aggregation, mean system-edge
TV is `.0374 / .0539 / .0553 / .0492` for EOS / x / period / newline. X and
period exceed EOS in 24/24 layer×domain cells, while newline does so in 23/24.
See `research/DEEPSEEK_ROUTING_HISTORY_BOUNDARY_TOKEN_AUDIT.md` for all paired
intervals, per-token route alignment, scope split, BF16 batch-shape audit,
primary literature, and the boundary between an input-ID intervention and a
chat-turn semantic claim.

## Role-marker-head single-token control

`v2_lite_routing_role_marker_head_control.py` keeps the official filler
history, EOS, colon, target span, generation prompt, mask, length, and 32-row
batch shape, while changing one ordinary token ID:

```text
system off/on × official / pre-target User→Assistant /
                    pre-target User→x / post-target Assistant→User
```

The pinned tokenizer maps `User`, `Assistant`, `:`, and `x` to IDs `5726`,
`77398`, `25`, and `87`. The pre-target controls identify the effect of the
first role-marker token only; the colon remains. The post-target replacement
is a causal suffix negative control. None of the three counterfactuals is a
valid official chat serialization.

```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
python -B experiments/deepseek/v2_lite_routing_role_marker_head_control.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-role-marker-head-control.json \
  --per-domain 32 \
  --content-tokens 23 \
  --batch-prompts 4 \
  --layers 7 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T12:30:00+00:00
```

The eight cells add 2,044,224 real top-6 route selections. All 256
source×system groups preserve length and target position; every one of the
768 counterfactuals differs from its official sequence at exactly one ID. The
committed run and independent rerun are byte-exact:

```text
9dc0e37fbce6581269428dcfcb84c7a17b466c5239c8171e6741f66d5eeb8caf
```

For exact target content, direct replacement TV is about `.02–.03`, but the
system-edge contrast has mixed direction: `12↑12↓` for User→Assistant and
`13↑11↓` for User→x. The post-target control is exact for all 34,488 aligned
target-token ordered top-6 routes, with zero target TV, JSD, and ΔCV. See
`research/DEEPSEEK_ROUTING_ROLE_MARKER_HEAD_AUDIT.md` for all paired intervals,
depth maps, token-level alignment, cross-experiment BF16 batch-content audit,
primary sources, and the boundary against full role semantics or Chat-model
behavior.

## Special-token family control

`v2_lite_routing_special_token_family_control.py` keeps the same repeated-token
history and changes the completed assistant boundary to four single IDs:

```text
system off/on × EOS / BOS / x / period
```

The pinned tokenizer has exactly two special-token IDs: BOS `100000` and EOS
`100001`; PAD aliases EOS. EOS/BOS therefore exhaust the special inventory,
while `x` and period are only two selected ordinary controls. The 2-vs-2
family summary is descriptive for these four IDs and is not a
population-level specialness claim.

```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
python -B experiments/deepseek/v2_lite_routing_special_token_family_control.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-special-token-family-control.json \
  --per-domain 32 \
  --content-tokens 23 \
  --batch-prompts 4 \
  --layers 7 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T12:57:00+00:00
```

The eight cells add 2,044,224 real top-6 route selections. All 256
source×system groups preserve length and target position, and all 768
counterfactuals change exactly one ID. The committed run and independent
rerun are byte-exact:

```text
c372c1b03a8b15f615b54ded5d9257a8fc2cdb7728001735d3d4c1d8534af5bf
```

For exact target content under prompt-balanced aggregation, mean system-edge
TV is `.037518 / .048018 / .053975 / .055289` for EOS / BOS / x / period.
BOS−EOS is positive in 22/24 layer×domain cells; x−EOS and period−EOS are
positive in 24/24. The selected ordinary-control mean exceeds the complete
special-inventory mean by `.011864` in these four IDs only. See
`research/DEEPSEEK_ROUTING_SPECIAL_TOKEN_FAMILY_AUDIT.md` for all paired
intervals, direct edges, scope split, alignment, batch-content audit,
literature context, and non-claims.

## Full two-token role-marker factorial

`v2_lite_routing_role_marker_block_factorial.py` treats the pre-target
two-token marker as two independent factors:

```text
system off/on × head User/Assistant × delimiter colon/x

User:       [5726, 25]      Assistant:  [77398, 25]
User x      [5726, 87]      Assistant x [77398, 87]
```

The official assistant EOS and post-target generation suffix remain unchanged.
The four blocks have zero, one, one, and two edited IDs relative to official
`User:`; every edit is verified while length, target position, attention mask,
and 32-row batch shape stay fixed.

```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps:/usr/lib/python3/dist-packages \
python -B experiments/deepseek/v2_lite_routing_role_marker_block_factorial.py \
  --artifact-dir /path/to/deepseek-v2-lite \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-routing-role-marker-block-factorial.json \
  --per-domain 32 \
  --content-tokens 23 \
  --batch-prompts 4 \
  --layers 7 \
  --bootstrap 2000 \
  --seed 20260729 \
  --captured-at 2026-07-29T13:06:00+00:00
```

The eight cells add 2,044,224 real top-6 route selections. The 256 official,
512 one-ID, and 256 two-ID cells all pass their exact edit contracts. The
committed run and independent rerun are byte-exact:

```text
a703dddb6d04182b1a608c213bfd341cfc32e1801be42c417dca30a505087e82
```

For exact target content under prompt-balanced aggregation, the four mean
system-edge TVs are `.037304 / .037015 / .037239 / .036220`. The head,
delimiter, and head×delimiter effects are small and mixed across the 24
layer×domain cells. Direct head and delimiter edges remain nonzero; replacing
colon with the single `x` control modestly reduces User-vs-Assistant direct TV
in most cells, without a uniform system modulation. See
`research/DEEPSEEK_ROUTING_ROLE_MARKER_BLOCK_AUDIT.md` for the exact factor
coding, intervals, direct dependencies, full-input split, route alignment,
cross-batch audit, sources, and non-claims.

## Full Chat-checkpoint generation behavior

`v2_lite_chat_special_token_behavior_probe.py` carries the special-token grid
into the full official SFT Chat checkpoint:

```text
deepseek-ai/DeepSeek-V2-Lite-Chat
@85864749cd611b4353ce1decdb286193298f64c7

system off/on × EOS / BOS / x / period
```

It reuses the pinned Base-routing source IDs and ranks but generates from the
full source text. All eight variants of one source run in one left-padded
batch. Decoding is deterministic greedy; BOS / x / period remain invalid-chat
single-ID counterfactuals.

The official BF16 checkpoint contains `31,412,968,448` tensor bytes. Because
the local RTX 5090 has less than the official 40GB single-GPU boundary, the
formal run uses Accelerate `device_map=auto`: embeddings and layers 0–24 are
placed on CUDA, while layers 25–26, final norm, and LM head are CPU-offloaded.

```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps \
python -B experiments/deepseek/v2_lite_chat_special_token_behavior_probe.py \
  --artifact-dir /path/to/deepseek-v2-lite-chat \
  --reference-routing-json \
    src/data/deepseek-v2-lite-routing-special-token-family-control.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-chat-behavior.json \
  --per-domain 4 \
  --max-new-tokens 128 \
  --gpu-memory 29GiB \
  --cpu-memory 80GiB
```

The formal artifact contains 16 sources and 128 outputs. Only 31 outputs hit
EOS; 97 reach the 128-token cap, so task-score fields are diagnostic and are
not used for capability ranking. A 4-source / 32-cell long-sequence rerun is
exact for prompt hashes, generated token IDs, decoded text, and EOS state.

```text
formal  54496955d0dd20a3116e6b758a55f56e2444c43dd99bfef94a2cb7b140e6196e
repro   b74c31606c0fc70a8e52cc6c3d6135c4e916d698b4d5dae51863124d025b0c72
```

See `research/DEEPSEEK_V2_LITE_CHAT_BEHAVIOR_PROTOCOL.md` and
`research/DEEPSEEK_V2_LITE_CHAT_BEHAVIOR_AUDIT.md` for the preregistered
contract, output-divergence table, offload device map, truncation boundary,
rerun coverage, execution fixes, primary sources, and forbidden conclusions.

## Completion-aware 512-token evaluation

The completion run uses the same 16 sources and eight-condition batches but
reruns all 128 cells with one uniform 512-token budget. The successful formal
placement leaves embeddings and layers 0–23 on CUDA and offloads layers 24–26,
final norm, and LM head. `expandable_segments:True` is part of the recorded
runtime contract.

```bash
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
PYTHONPATH=/path/to/transformers-4.41.2-deps \
python -B experiments/deepseek/v2_lite_chat_special_token_behavior_probe.py \
  --artifact-dir /path/to/deepseek-v2-lite-chat \
  --reference-routing-json \
    src/data/deepseek-v2-lite-routing-special-token-family-control.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-chat-completion-512.json \
  --per-domain 4 \
  --max-new-tokens 512 \
  --gpu-memory 28GiB \
  --cpu-memory 80GiB
```

The evaluator separates stopping, semantic terminal state, evaluator coverage,
and correctness. Each HumanEval candidate runs in a fresh pinned, networkless,
read-only Docker container with no host mounts:

```bash
python -B experiments/deepseek/v2_lite_chat_completion_evaluator.py \
  --behavior-json \
    src/data/deepseek-v2-lite-chat-completion-512.json \
  --baseline-json src/data/deepseek-v2-lite-chat-behavior.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --sandbox-image \
    python:3.11-alpine@sha256:25976e9d34a0fab1f278cae931f34c8303d97bf0c0d7f85b6b4dcf641d7702a4 \
  --output \
    src/data/deepseek-v2-lite-chat-completion-512-eval.json
```

The formal run reaches natural EOS in 121/128 cells, up from 31/128 at the
128-token budget. Strict-complete GSM8K numeric exact is 23/32; HumanEval
official-test pass is 24/32. These are four source tasks per domain, not
benchmark estimates. A fresh-process one-source-per-domain rerun reproduces
all 32 complete generated token sequences.

## Full 27-layer Chat hidden-state and router trace

`v2_lite_chat_full_depth_trace.py` performs a prompt-only `use_cache=False`
forward on the same checkpoint and inputs. It captures hashes and statistics
for embedding, every decoder-layer output, final norm, and every MoE gate.
Boundary-crossing tokenizer tokens are excluded from the exact target-content
scope.

```bash
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
PYTHONPATH=/path/to/transformers-4.41.2-deps \
python -B experiments/deepseek/v2_lite_chat_full_depth_trace.py \
  --artifact-dir /path/to/deepseek-v2-lite-chat \
  --reference-routing-json \
    src/data/deepseek-v2-lite-routing-special-token-family-control.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --output src/data/deepseek-v2-lite-chat-full-depth.json \
  --per-domain 4 \
  --gpu-memory 28GiB \
  --cpu-memory 80GiB
```

The formal trace covers 1,537 exact interior content tokens per condition,
29 hidden stages, 26 gates, and 1,918,176 top-6 route decisions. A fresh
four-source rerun reproduces 1,856 hidden tensor hashes, 1,664 ordered route
hashes, 1,664 route-weight hashes, and every derived comparison.

See `research/DEEPSEEK_V2_LITE_CHAT_COMPLETION_PROTOCOL.md` and
`research/DEEPSEEK_V2_LITE_CHAT_COMPLETION_DEPTH_AUDIT.md` for the OOM
amendment, task evaluators, completion table, depth curves, reproduction
audit, artifact hashes, and claim boundaries.

## Preregistered multi-seed Chat sampling

`v2_lite_chat_sampling_probe.py` keeps the same four first-ranked sources,
eight prompt conditions, 512-token cap, BF16 checkpoint, and offload contract.
It enables the pinned official sampling values and explicitly disables top-k:

```text
do_sample=true · temperature=.3 · top_p=.95 · top_k=0
4 sources × 8 SHA-256-derived seeds × 8 conditions = 256 outputs
```

The eight rows of one source remain in a fixed batch. They are batch-seed
aligned, not common-random-number pairs: Transformers samples the whole batch
with one `torch.multinomial` call.

```bash
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
PYTHONPATH=/path/to/transformers-4.41.2-deps \
python -B experiments/deepseek/v2_lite_chat_sampling_probe.py \
  --artifact-dir /path/to/deepseek-v2-lite-chat \
  --reference-routing-json \
    src/data/deepseek-v2-lite-routing-special-token-family-control.json \
  --greedy-json src/data/deepseek-v2-lite-chat-completion-512.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --base-seeds \
    19683830 1560062173 3978401375 1280933274 \
    1467459869 1297359489 2722953988 3330978061 \
  --max-new-tokens 512 \
  --gpu-memory 28GiB \
  --cpu-memory 80GiB \
  --output src/data/deepseek-v2-lite-chat-sampling.json
```

The independent evaluator preserves stopping, semantic terminal state,
coverage, and correctness as separate ledgers. Exact duplicate code candidates
reuse a cache keyed by candidate, task tests, and sandbox-harness hashes:

```bash
python -B experiments/deepseek/v2_lite_chat_sampling_evaluator.py \
  --sampling-json src/data/deepseek-v2-lite-chat-sampling.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --sandbox-image \
    python:3.11-alpine@sha256:25976e9d34a0fab1f278cae931f34c8303d97bf0c0d7f85b6b4dcf641d7702a4 \
  --output src/data/deepseek-v2-lite-chat-sampling-eval.json
```

A fresh Python process reruns R0/R1. The reproduction comparer checks run
seeds, prompt hashes, complete token IDs, text, stopping state, and CPU/CUDA
RNG pre-state hashes:

```bash
python -B experiments/deepseek/v2_lite_chat_sampling_reproduction.py \
  --formal-json src/data/deepseek-v2-lite-chat-sampling.json \
  --rerun-json \
    src/data/deepseek-v2-lite-chat-sampling-repro-r0r1.json \
  --output \
    src/data/deepseek-v2-lite-chat-sampling-reproduction.json
```

Formal results are 251/256 natural EOS and 242/256 globally unique complete
token trajectories. Strict GSM8K exact is 62/64; HumanEval official-test pass
is 63/64. These repeat one task per domain and are not benchmark estimates.
The R0/R1 fresh-process rerun reproduces all eight preregistered fields in
64/64 cells.

```text
formal  46c7edfce1409e798d6b0f06e905dd3a9d4ab6911acd8e42e2ec9d06d65345af
eval    078f486e021ffd0e4af8ad942d001a9f2429933b2fb0fb3523b7d1da5b24c4d8
rerun   72d050eace531bb38fb2592e39dc7f1b9c9003306c92c568dcd4acaaa691f6a0
compare 4d59a775943459604ea2d9976bb3be01ff7b3b08d71eb0ca22c376122ef82d15
```

See `research/DEEPSEEK_V2_LITE_CHAT_SAMPLING_PROTOCOL.md` and
`research/DEEPSEEK_V2_LITE_CHAT_SAMPLING_AUDIT.md` for seed derivation,
smoke gates, source-condition diversity tables, edge-set comparisons, task
failure cases, exact reproduction scope, primary sources, and non-claims.

## Source-blocked cross-source Chat sampling

`v2_lite_chat_cross_source_sampling_probe.py` reuses the audited sampling
runtime but changes the coverage contract before any outputs are inspected:

```text
16 preregistered sources
× 4 SHA-256-derived seeds
× 4 conditions (system off/on × EOS/period)
= 256 outputs
```

The four first-ranked sources in each of WikiText-2, TNEWS, HumanEval, and
GSM8K come from the frozen routing-corpus selection contract. Source is the
primary coverage unit; seed is a within-source repeat. The period condition is
a directed Round-06 follow-up, not a blind independent confirmation.

```bash
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
PYTHONPATH=/path/to/transformers-4.41.2-deps \
python -B \
  experiments/deepseek/v2_lite_chat_cross_source_sampling_probe.py \
  --artifact-dir /path/to/deepseek-v2-lite-chat \
  --reference-routing-json \
    src/data/deepseek-v2-lite-routing-special-token-family-control.json \
  --greedy-json src/data/deepseek-v2-lite-chat-completion-512.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --base-seeds 2101325316 2511573438 1677220094 2412346607 \
  --max-new-tokens 512 \
  --gpu-memory 28GiB \
  --cpu-memory 80GiB \
  --output \
    src/data/deepseek-v2-lite-chat-cross-source-sampling.json
```

The independent evaluator uses the same pinned, networkless, read-only
HumanEval sandbox and avoids pooling final-answer frequencies across different
GSM8K tasks:

```bash
python -B \
  experiments/deepseek/v2_lite_chat_cross_source_sampling_evaluator.py \
  --sampling-json \
    src/data/deepseek-v2-lite-chat-cross-source-sampling.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --sandbox-image \
    python:3.11-alpine@sha256:25976e9d34a0fab1f278cae931f34c8303d97bf0c0d7f85b6b4dcf641d7702a4 \
  --output \
    src/data/deepseek-v2-lite-chat-cross-source-sampling-eval.json
```

The source-blocked analysis computes each contrast within source, then reports
the four source directions in each domain. It deliberately emits no p-values
or population confidence intervals:

```bash
python -B \
  experiments/deepseek/v2_lite_chat_cross_source_sampling_analysis.py \
  --sampling-json \
    src/data/deepseek-v2-lite-chat-cross-source-sampling.json \
  --evaluation-json \
    src/data/deepseek-v2-lite-chat-cross-source-sampling-eval.json \
  --reproduction-json \
    src/data/deepseek-v2-lite-chat-cross-source-sampling-reproduction.json \
  --output \
    src/data/deepseek-v2-lite-chat-cross-source-sampling-analysis.json
```

A fresh Python process reruns R0 for all 16 sources. The existing reproduction
comparer verifies the eight preregistered fields in all 64
source-condition cells.

Formal results are 250/256 natural EOS and 247/256 unique complete token
trajectories. Strict task totals are Math 47/64 and Code 52/64, but the
per-task totals range from 8/16 to 16/16 in both domains. Period shortens mean
generation length for 11/16 sources. English is the important counterexample:
the first source is −121 tokens while the other three are +44.5, +41, and
+1.375, so the domain mean is negative even though three of four sources are
positive.

```text
formal   f013132485f27adce008f03f781bed9982efc0d7939f13faede01c9f6f3d7f7c
eval     e88b274599fc5951561f9e5e7438fb4d6d25f341ae3bc4a2c4121877a0db8975
rerun    143dc9d0f7c914db4781e36b1401cdc9fbc2971a0dca71188bab0f8cedb002a6
compare  ec4a47894953f5d73bb62211b588632ef7032da0e915218992b7fc3ef9c2a556
analysis d0dece388998fee419d34ff33f140695a9fedef6e79799cdf42eb283b047bc84
compact  d0ab65646c6119bdeafeb451103dc6afebff3624a1e05f13a45a52ad965be1af
```

See `research/DEEPSEEK_V2_LITE_CHAT_CROSS_SOURCE_SAMPLING_PROTOCOL.md` and
`research/DEEPSEEK_V2_LITE_CHAT_CROSS_SOURCE_SAMPLING_AUDIT.md` for the
preregistered source frame, task matrices, source-direction contrasts,
failure identities, hash chain, exact replay scope, and non-claims.

## Selected-task bootstrap with explicit common random numbers

`v2_lite_chat_task_bootstrap_crn_probe.py` expands the task frame to all 32
frozen HumanEval and 32 frozen GSM8K sources.  The primary analysis runs one
preregistered tape T0 on every task.  Four tasks per domain additionally run
T1-T3 as a crossed tape-sensitivity diagnostic:

```text
64 tasks × T0 × 4 conditions                         = 256 outputs
8 diagnostic tasks × 3 additional tapes × 4 conditions = 96 outputs
formal total                                            = 352 outputs
```

Unlike the preceding batch-seed-aligned studies, Round 08 constructs a true
step-index common-random-number tape.  The same SHA-256-derived `u_t` is
mapped through each condition's official `.3/.95` temperature/top-p
distribution using a float32 token-ID-ordered inverse CDF.  It does not call
`transformers.generate` or `torch.multinomial`.

The frozen manifest and all 256 Chat prompt hashes must be built before any
model outputs:

```bash
node scripts/build-deepseek-chat-task-bootstrap-manifest.mjs \
  --reference \
    src/data/deepseek-v2-lite-routing-special-token-family-control.json \
  --output \
    research/DEEPSEEK_V2_LITE_CHAT_TASK_BOOTSTRAP_MANIFEST.json

PYTHONPATH=/path/to/transformers-4.41.2-deps \
python -B scripts/freeze-deepseek-chat-task-bootstrap-prompts.py \
  --artifact-dir /path/to/deepseek-v2-lite-chat \
  --manifest \
    research/DEEPSEEK_V2_LITE_CHAT_TASK_BOOTSTRAP_MANIFEST.json \
  --reference-routing-json \
    src/data/deepseek-v2-lite-routing-special-token-family-control.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet
```

The formal runner writes one hash-checked journal per source and supports
`--resume`:

```bash
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
PYTHONPATH=/path/to/transformers-4.41.2-deps:experiments/deepseek \
python -B \
  experiments/deepseek/v2_lite_chat_task_bootstrap_crn_probe.py \
  --execution-mode formal \
  --artifact-dir /path/to/deepseek-v2-lite-chat \
  --manifest \
    research/DEEPSEEK_V2_LITE_CHAT_TASK_BOOTSTRAP_MANIFEST.json \
  --reference-routing-json \
    src/data/deepseek-v2-lite-routing-special-token-family-control.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --tnews /path/to/tnews/test.json \
  --tnews-archive /path/to/tnews_public.zip \
  --wikitext /path/to/wikitext-validation.parquet \
  --journal-dir /path/to/formal-journal \
  --resume \
  --output \
    src/data/deepseek-v2-lite-chat-task-bootstrap-crn.json
```

Evaluation keeps T0 primary rows separate from T1-T3 diagnostics:

```bash
PYTHONPATH=experiments/deepseek \
python -B \
  experiments/deepseek/v2_lite_chat_task_bootstrap_crn_evaluator.py \
  --sampling-json \
    src/data/deepseek-v2-lite-chat-task-bootstrap-crn.json \
  --human-eval /path/to/HumanEval.jsonl.gz \
  --gsm8k /path/to/gsm8k/test.jsonl \
  --sandbox-image \
    python:3.11-alpine@sha256:25976e9d34a0fab1f278cae931f34c8303d97bf0c0d7f85b6b4dcf641d7702a4 \
  --output \
    src/data/deepseek-v2-lite-chat-task-bootstrap-crn-eval.json
```

The analysis resamples the 32 selected tasks 10,000 times within each domain
while preserving condition pairing.  These are selected-task resampling
bands under T0, not benchmark-population or generation-seed confidence
intervals:

```bash
python -B \
  experiments/deepseek/v2_lite_chat_task_bootstrap_crn_analysis.py \
  --sampling-json \
    src/data/deepseek-v2-lite-chat-task-bootstrap-crn.json \
  --evaluation-json \
    src/data/deepseek-v2-lite-chat-task-bootstrap-crn-eval.json \
  --reproduction-json \
    src/data/deepseek-v2-lite-chat-task-bootstrap-crn-reproduction.json \
  --output \
    src/data/deepseek-v2-lite-chat-task-bootstrap-crn-analysis.json
```

The formal grid contains 343/352 natural EOS outputs and 320 unique complete
trajectories.  T0 correctness totals are Code 59/128 and Math 71/128 across
32 tasks × four prompt conditions; they are not standard benchmark scores.
All correctness contrast bands cross zero.  The clearest length interaction
is system-on minus system-off under the period boundary: Code is -130.875
tokens with band [-178.688, -83.218], while Math is +25.125 with band
[7.874, 44.531].  The preregistered 64-cell cross-process replay is exact in
all twelve audited fields.

```text
formal   ea0607f2b197fac3f794655c1538ce1f9a1cb072d637eb31d35573682e311809
eval     82b2fc5d1f854a7e0cd7aba7c70ff74d733d24221522a4f92b962478c76177ab
replay   6519947e2fa4327c1ba2cc506b0861f172a6bdbd4f2d6787447edaf8fbcac508
compare  63ed39e5dcdbc2a30e516172f3657dfa453ff3b23d5bd5c49c734939242a70f5
analysis 9ab17561ced930a668c082141f7c6e013cbda70e42b09de63d41f1b82c01a6ae
compact  082e3c56373e5f91d51d88b1430d14ad3b68b0f200e80901171c2fb62a1e57bf
```

Run the complete offline hash and invariant audit with:

```bash
node scripts/check-deepseek-chat-task-bootstrap-crn-data.mjs
```

See `research/DEEPSEEK_V2_LITE_CHAT_TASK_BOOTSTRAP_PROTOCOL.md` and
`research/DEEPSEEK_V2_LITE_CHAT_TASK_BOOTSTRAP_CRN_AUDIT.md` for the
pre-output corrections, explicit sampler, task bootstrap, transition tables,
multi-tape matrices, exact replay, reported gold-loading process deviation,
and claim boundaries.