llm-atlas/experiments/k3/attnres_gradient/README.md

Attention Residuals activation-gradient/depth study

This directory implements preregistered protocol llm-atlas-k3-attnres-gradient-scale-v1:

• research/K3_ATTNRES_GRADIENT_DEFINITION_AUDIT.md
• research/K3_ATTNRES_GRADIENT_SCALE_PROTOCOL.md

It is an independent reduced mechanism experiment. It is not a Kimi K3 checkpoint forward pass and does not claim to recover the paper's unpublished Figure 5 telemetry definition.

Frozen environment

Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
PyTorch 2.11.0+cu128
GPU NVIDIA GeForce RTX 5090
CUBLAS_WORKSPACE_CONFIG=:4096:8

Build the manifest

python experiments/k3/attnres_gradient/build_dataset.py \
  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
  --manifest experiments/k3/attnres_gradient/manifest.json

Run a smoke cell

CUBLAS_WORKSPACE_CONFIG=:4096:8 \
python experiments/k3/attnres_gradient/train.py \
  --run-kind smoke \
  --architecture block \
  --depth 32 \
  --seed 2026073001 \
  --cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1 \
  --manifest experiments/k3/attnres_gradient/manifest.json \
  --output /home/wuyang/.cache/llm-atlas/k3-attnres-gradient-scale-v1/smoke-a/depth-32-block.json

Smoke is fixed to 20 steps. Formal and replay runs are fixed to 8,000 steps; the runner rejects alternative budgets. The same command uses --run-kind formal or --run-kind replay and omits an explicit --steps.

Formal output keys use:

formal/depth-{16|32}-{baseline|block}-seed-{seed}.json
replay/depth-32-block-seed-2026073001.json

Raw parquet/binary files and working runs remain in the local cache. The manifest, runner, complete result JSON, compact website payload, reproduction hashes, protocol, and audit enter the public repository.