wuyang
51 lines
1.9 KiB
Markdown
51 lines
1.9 KiB
Markdown
# Attention Residuals local mixer-path diagnostics
|
|
|
|
This directory implements preregistered protocol
|
|
`llm-atlas-k3-attnres-local-path-v1`.
|
|
|
|
It is a targeted follow-up to Round 06. It exact-replays the same depth-32
|
|
Block training and keeps the learned forward unchanged while switching source
|
|
value-gradient coefficients only at frozen mixer scopes. It is not a Kimi K3
|
|
checkpoint run, a trainable variant, an additive attribution, or a reproduction
|
|
of unpublished Figure 5 telemetry.
|
|
|
|
## Frozen environment
|
|
|
|
```text
|
|
Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
|
|
PyTorch 2.11.0+cu128
|
|
GPU NVIDIA GeForce RTX 5090
|
|
CUBLAS_WORKSPACE_CONFIG=:4096:8
|
|
```
|
|
|
|
The preregistration was committed as `6911efc` before the runner or any result
|
|
file existed.
|
|
|
|
## Step-0 smoke
|
|
|
|
```bash
|
|
CUBLAS_WORKSPACE_CONFIG=:4096:8 \
|
|
/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
|
|
experiments/k3/attnres_local_path/train.py \
|
|
--run-kind smoke \
|
|
--seed 2026073001 \
|
|
--cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
|
|
--data-manifest experiments/k3/attnres_gradient/manifest.json \
|
|
--parent-manifest experiments/k3/attnres_spike/manifest.json \
|
|
--manifest experiments/k3/attnres_local_path/manifest.json \
|
|
--output /home/wuyang/.cache/llm-atlas/k3-attnres-local-path-v1/smoke/seed-2026073001.json
|
|
```
|
|
|
|
The first smoke passed all 14-mode forward-identity, selector, Round 06 endpoint,
|
|
initialization-negative-control, parent-learned, and loss-scale gates. Its
|
|
canonical content hash is
|
|
`f708200f4fb122f61f30b97393839382a2094a71a8ea5d48cf47fb7fa094e69b`.
|
|
|
|
Formal cells use the same command with `--run-kind formal`, one of the three
|
|
manifest seeds, and a new output path. The independent seed-2026073001 run uses
|
|
`--run-kind replay`.
|
|
|
|
Only `analyze.py` may calculate the global log gap, sufficiency/restoration
|
|
scores, and preregistered gates. The site consumes its frozen aggregate rather
|
|
than reimplementing thresholds in TypeScript.
|