experiment: implement AttnRes forward training runner

2026-07-30 15:14:15 +08:00
parent d81aacfc68
commit 7ea91caabb
11 changed files with 17842 additions and 1 deletions
@@ -0,0 +1,58 @@
+# Attention Residuals train-time forward intervention
+
+This directory implements preregistered protocol
+`llm-atlas-k3-attnres-forward-training-v1`:
+
+- `research/K3_ATTNRES_FORWARD_TRAINING_SCOPING.md`
+- `research/K3_ATTNRES_FORWARD_TRAINING_PROTOCOL.md`
+- `research/K3_ATTNRES_FORWARD_TRAINING_GROK_REVIEW.md`
+
+It is a depth-32 reduced Block AttnRes architecture ablation. It is not a
+Kimi-K3 checkpoint forward pass and does not claim to recover unpublished
+Figure 5 telemetry.
+
+## Frozen environment
+
+```text
+Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
+PyTorch 2.11.0+cu128
+GPU NVIDIA GeForce RTX 5090
+CUBLAS_WORKSPACE_CONFIG=:4096:8
+maximum concurrency 2
+```
+
+## Pre-result gates
+
+The checked-in gate artifacts must pass before formal output:
+
+```bash
+CUBLAS_WORKSPACE_CONFIG=:4096:8 \
+/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
+  experiments/k3/attnres_forward/verify.py step-zero \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
+  --parent-manifest experiments/k3/attnres_gradient/manifest.json \
+  --study-manifest experiments/k3/attnres_forward/manifest.json \
+  --output experiments/k3/attnres_forward/results/gates/step-zero.json
+```
+
+`learned_reference` is smoke-only. Its 20-step result is compared with a
+fresh parent Round 05 smoke using `verify.py smoke-compare`.
+
+## Formal matrix
+
+```bash
+/home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
+  experiments/k3/attnres_forward/run_matrix.py \
+  --python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python \
+  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
+  --parent-manifest experiments/k3/attnres_gradient/manifest.json \
+  --study-manifest experiments/k3/attnres_forward/manifest.json \
+  --output-dir experiments/k3/attnres_forward/results/raw \
+  --phase all \
+  --concurrency 2
+```
+
+This runs 12 formal cells and one full replay. The analyzer reads all cells,
+the frozen historical paired references, and generates the only authoritative
+status, interaction map, and website compact artifact.
+