research: lock AttnRes gradient runner
This commit is contained in:
@@ -0,0 +1,57 @@
|
||||
# Attention Residuals activation-gradient/depth study
|
||||
|
||||
This directory implements preregistered protocol
|
||||
`llm-atlas-k3-attnres-gradient-scale-v1`:
|
||||
|
||||
- `research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md`
|
||||
- `research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md`
|
||||
|
||||
It is an independent reduced mechanism experiment. It is not a Kimi K3
|
||||
checkpoint forward pass and does not claim to recover the paper's unpublished
|
||||
Figure 5 telemetry definition.
|
||||
|
||||
## Frozen environment
|
||||
|
||||
```text
|
||||
Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
|
||||
PyTorch 2.11.0+cu128
|
||||
GPU NVIDIA GeForce RTX 5090
|
||||
CUBLAS_WORKSPACE_CONFIG=:4096:8
|
||||
```
|
||||
|
||||
## Build the manifest
|
||||
|
||||
```bash
|
||||
python experiments/k3/attnres_gradient/build_dataset.py \
|
||||
--cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
|
||||
--manifest experiments/k3/attnres_gradient/manifest.json
|
||||
```
|
||||
|
||||
## Run a smoke cell
|
||||
|
||||
```bash
|
||||
CUBLAS_WORKSPACE_CONFIG=:4096:8 \
|
||||
python experiments/k3/attnres_gradient/train.py \
|
||||
--run-kind smoke \
|
||||
--architecture block \
|
||||
--depth 32 \
|
||||
--seed 2026073001 \
|
||||
--cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
|
||||
--manifest experiments/k3/attnres_gradient/manifest.json \
|
||||
--output /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1/smoke-a/depth-32-block.json
|
||||
```
|
||||
|
||||
Smoke is fixed to 20 steps. Formal and replay runs are fixed to 8,000 steps;
|
||||
the runner rejects alternative budgets. The same command uses
|
||||
`--run-kind formal` or `--run-kind replay` and omits an explicit `--steps`.
|
||||
|
||||
Formal output keys use:
|
||||
|
||||
```text
|
||||
formal/depth-{16|32}-{baseline|block}-seed-{seed}.json
|
||||
replay/depth-32-block-seed-2026073001.json
|
||||
```
|
||||
|
||||
Raw parquet/binary files and working runs remain in the local cache. The
|
||||
manifest, runner, complete result JSON, compact website payload, reproduction
|
||||
hashes, protocol, and audit enter the public repository.
|
||||
Reference in New Issue
Block a user