wuyang
65 lines
2.2 KiB
Markdown
65 lines
2.2 KiB
Markdown
# Reduced Attention Residuals reproduction
|
|
|
|
This directory implements protocol
|
|
`llm-atlas-k3-attnres-reduced-v1`, frozen in
|
|
`research/K3_ATTNRES_REDUCED_PROTOCOL.md`.
|
|
|
|
The experiment is a reduced independent mechanism probe. It is not a Kimi K3
|
|
checkpoint forward pass and not a reproduction of the paper-scale training run.
|
|
|
|
## Environment
|
|
|
|
The pinned execution environment used by this project is:
|
|
|
|
```text
|
|
Python /home/wuyang/.pyenv/versions/3.10.14/envs/navi-router-cu128/bin/python
|
|
PyTorch 2.11.0+cu128
|
|
GPU NVIDIA GeForce RTX 5090
|
|
CUBLAS_WORKSPACE_CONFIG=:4096:8
|
|
```
|
|
|
|
## Build the frozen dataset
|
|
|
|
```bash
|
|
python experiments/k3/attnres/build_dataset.py \
|
|
--cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1 \
|
|
--manifest experiments/k3/attnres/manifest.json
|
|
```
|
|
|
|
## Run one cell
|
|
|
|
```bash
|
|
CUBLAS_WORKSPACE_CONFIG=:4096:8 \
|
|
python experiments/k3/attnres/train.py \
|
|
--architecture baseline \
|
|
--seed 2026073001 \
|
|
--cache-dir /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1 \
|
|
--manifest experiments/k3/attnres/manifest.json \
|
|
--output /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/runs/baseline-2026073001.json
|
|
```
|
|
|
|
Raw parquet and checkpoints stay in the local cache. Frozen manifests, metric
|
|
JSON, analyses, code, checksums, and a compact website payload enter the public
|
|
repository.
|
|
|
|
## Validate and aggregate the complete study
|
|
|
|
After the nine formal cells, the preregistered replay, and the paired smoke runs
|
|
exist in the cache:
|
|
|
|
```bash
|
|
python experiments/k3/attnres/analyze.py \
|
|
--formal-dir /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/formal \
|
|
--smoke-dir /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/smoke \
|
|
--replay /home/wuyang/.cache/llm-atlas/k3-attnres-reduced-v1/replay/block-2026073001.json \
|
|
--manifest experiments/k3/attnres/manifest.json \
|
|
--output src/data/k3-attnres-reduced.json \
|
|
--compact-output src/data/k3-attnres-reduced-compact.json \
|
|
--reproduction-output experiments/k3/attnres/reproduction.json
|
|
```
|
|
|
|
The aggregator fails closed on protocol identity, grid completeness, byte
|
|
budget, schedule hashes, shared initialization, non-finite metrics, diagnostic
|
|
shape, smoke mismatch, or formal replay mismatch. Timing is recorded but is not
|
|
required to replay bit-for-bit.
|