# Attention Residuals activation-gradient/depth study

This directory implements preregistered protocol
`llm-atlas-k3-attnres-gradient-scale-v1`:

- `research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md`
- `research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md`

It is an independent reduced mechanism experiment. It is not a Kimi K3
checkpoint forward pass and does not claim to recover the paper's unpublished
Figure 5 telemetry definition.

## Frozen environment

```text
Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
PyTorch 2.11.0+cu128
GPU NVIDIA GeForce RTX 5090
CUBLAS_WORKSPACE_CONFIG=:4096:8
```

## Build the manifest

```bash
python experiments/k3/attnres_gradient/build_dataset.py \
  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
  --manifest experiments/k3/attnres_gradient/manifest.json
```

## Run a smoke cell

```bash
CUBLAS_WORKSPACE_CONFIG=:4096:8 \
python experiments/k3/attnres_gradient/train.py \
  --run-kind smoke \
  --architecture block \
  --depth 32 \
  --seed 2026073001 \
  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
  --manifest experiments/k3/attnres_gradient/manifest.json \
  --output /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1/smoke-a/depth-32-block.json
```

Smoke is fixed to 20 steps. Formal and replay runs are fixed to 8,000 steps;
the runner rejects alternative budgets. The same command uses
`--run-kind formal` or `--run-kind replay` and omits an explicit `--steps`.

Formal output keys use:

```text
formal/depth-{16|32}-{baseline|block}-seed-{seed}.json
replay/depth-32-block-seed-2026073001.json
```

Raw parquet/binary files and working runs remain in the local cache. The
manifest, runner, complete result JSON, compact website payload, reproduction
hashes, protocol, and audit enter the public repository.