feat: add DeepSeek Chat behavior evidence

This commit is contained in:
wuyang
2026-07-29 22:47:03 +08:00
parent b615224798
commit 96443d4dfc
18 changed files with 35709 additions and 34 deletions
+55
View File
@@ -531,3 +531,58 @@ in most cells, without a uniform system modulation. See
`research/DEEPSEEK_ROUTING_ROLE_MARKER_BLOCK_AUDIT.md` for the exact factor
coding, intervals, direct dependencies, full-input split, route alignment,
cross-batch audit, sources, and non-claims.
## Full Chat-checkpoint generation behavior
`v2_lite_chat_special_token_behavior_probe.py` carries the special-token grid
into the full official SFT Chat checkpoint:
```text
deepseek-ai/DeepSeek-V2-Lite-Chat
@85864749cd611b4353ce1decdb286193298f64c7
system off/on × EOS / BOS / x / period
```
It reuses the pinned Base-routing source IDs and ranks but generates from the
full source text. All eight variants of one source run in one left-padded
batch. Decoding is deterministic greedy; BOS / x / period remain invalid-chat
single-ID counterfactuals.
The official BF16 checkpoint contains `31,412,968,448` tensor bytes. Because
the local RTX 5090 has less than the official 40GB single-GPU boundary, the
formal run uses Accelerate `device_map=auto`: embeddings and layers 0–24 are
placed on CUDA, while layers 25–26, final norm, and LM head are CPU-offloaded.
```bash
PYTHONPATH=/path/to/transformers-4.41.2-deps \
python -B experiments/deepseek/v2_lite_chat_special_token_behavior_probe.py \
--artifact-dir /path/to/deepseek-v2-lite-chat \
--reference-routing-json \
src/data/deepseek-v2-lite-routing-special-token-family-control.json \
--human-eval /path/to/HumanEval.jsonl.gz \
--gsm8k /path/to/gsm8k/test.jsonl \
--tnews /path/to/tnews/test.json \
--tnews-archive /path/to/tnews_public.zip \
--wikitext /path/to/wikitext-validation.parquet \
--output src/data/deepseek-v2-lite-chat-behavior.json \
--per-domain 4 \
--max-new-tokens 128 \
--gpu-memory 29GiB \
--cpu-memory 80GiB
```
The formal artifact contains 16 sources and 128 outputs. Only 31 outputs hit
EOS; 97 reach the 128-token cap, so task-score fields are diagnostic and are
not used for capability ranking. A 4-source / 32-cell long-sequence rerun is
exact for prompt hashes, generated token IDs, decoded text, and EOS state.
```text
formal 54496955d0dd20a3116e6b758a55f56e2444c43dd99bfef94a2cb7b140e6196e
repro b74c31606c0fc70a8e52cc6c3d6135c4e916d698b4d5dae51863124d025b0c72
```
See `research/DEEPSEEK_V2_LITE_CHAT_BEHAVIOR_PROTOCOL.md` and
`research/DEEPSEEK_V2_LITE_CHAT_BEHAVIOR_AUDIT.md` for the preregistered
contract, output-divergence table, offload device map, truncation boundary,
rerun coverage, execution fixes, primary sources, and forbidden conclusions.
File diff suppressed because it is too large Load Diff