feat: add DeepSeek Chat behavior evidence
This commit is contained in:
@@ -531,3 +531,58 @@ in most cells, without a uniform system modulation. See
|
||||
`research/DEEPSEEK_ROUTING_ROLE_MARKER_BLOCK_AUDIT.md` for the exact factor
|
||||
coding, intervals, direct dependencies, full-input split, route alignment,
|
||||
cross-batch audit, sources, and non-claims.
|
||||
|
||||
## Full Chat-checkpoint generation behavior
|
||||
|
||||
`v2_lite_chat_special_token_behavior_probe.py` carries the special-token grid
|
||||
into the full official SFT Chat checkpoint:
|
||||
|
||||
```text
|
||||
deepseek-ai/DeepSeek-V2-Lite-Chat
|
||||
@85864749cd611b4353ce1decdb286193298f64c7
|
||||
|
||||
system off/on × EOS / BOS / x / period
|
||||
```
|
||||
|
||||
It reuses the pinned Base-routing source IDs and ranks but generates from the
|
||||
full source text. All eight variants of one source run in one left-padded
|
||||
batch. Decoding is deterministic greedy; BOS / x / period remain invalid-chat
|
||||
single-ID counterfactuals.
|
||||
|
||||
The official BF16 checkpoint contains `31,412,968,448` tensor bytes. Because
|
||||
the local RTX 5090 has less than the official 40GB single-GPU boundary, the
|
||||
formal run uses Accelerate `device_map=auto`: embeddings and layers 0–24 are
|
||||
placed on CUDA, while layers 25–26, final norm, and LM head are CPU-offloaded.
|
||||
|
||||
```bash
|
||||
PYTHONPATH=/path/to/transformers-4.41.2-deps \
|
||||
python -B experiments/deepseek/v2_lite_chat_special_token_behavior_probe.py \
|
||||
--artifact-dir /path/to/deepseek-v2-lite-chat \
|
||||
--reference-routing-json \
|
||||
src/data/deepseek-v2-lite-routing-special-token-family-control.json \
|
||||
--human-eval /path/to/HumanEval.jsonl.gz \
|
||||
--gsm8k /path/to/gsm8k/test.jsonl \
|
||||
--tnews /path/to/tnews/test.json \
|
||||
--tnews-archive /path/to/tnews_public.zip \
|
||||
--wikitext /path/to/wikitext-validation.parquet \
|
||||
--output src/data/deepseek-v2-lite-chat-behavior.json \
|
||||
--per-domain 4 \
|
||||
--max-new-tokens 128 \
|
||||
--gpu-memory 29GiB \
|
||||
--cpu-memory 80GiB
|
||||
```
|
||||
|
||||
The formal artifact contains 16 sources and 128 outputs. Only 31 outputs hit
|
||||
EOS; 97 reach the 128-token cap, so task-score fields are diagnostic and are
|
||||
not used for capability ranking. A 4-source / 32-cell long-sequence rerun is
|
||||
exact for prompt hashes, generated token IDs, decoded text, and EOS state.
|
||||
|
||||
```text
|
||||
formal 54496955d0dd20a3116e6b758a55f56e2444c43dd99bfef94a2cb7b140e6196e
|
||||
repro b74c31606c0fc70a8e52cc6c3d6135c4e916d698b4d5dae51863124d025b0c72
|
||||
```
|
||||
|
||||
See `research/DEEPSEEK_V2_LITE_CHAT_BEHAVIOR_PROTOCOL.md` and
|
||||
`research/DEEPSEEK_V2_LITE_CHAT_BEHAVIOR_AUDIT.md` for the preregistered
|
||||
contract, output-divergence table, offload device map, truncation boundary,
|
||||
rerun coverage, execution fixes, primary sources, and forbidden conclusions.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user