feat: add remote workspaces and verified deliverables

2026-07-26 12:58:04 +08:00
parent 37f83eaac7
commit 0b4d799216
30 changed files with 1595 additions and 55 deletions
@@ -0,0 +1,89 @@
+# Operations runbook
+
+## Production layout
+
+- Public URL: `https://agent.k1412.top`
+- NAS project: `/boot/config/plugins/compose.manager/projects/k1412-agent`
+- Public host port: `12004`
+- Execution provider: `ssh-docker`
+- Execution host Docker root: `/srv/k1412-data/docker`
+- Execution host workspace volumes: Docker volumes named
+  `k1412-ws-data-<hashed-user-id>`
+
+No credential belongs in this document or repository.
+
+## Deployment
+
+Build and push immutable `linux/amd64` images:
+
+```bash
+docker buildx build --platform linux/amd64 -f docker/web.Dockerfile \
+  -t docker.k1412.top/wuyang/k1412-agent-web:<release> --push .
+docker buildx build --platform linux/amd64 -f docker/runtime.Dockerfile \
+  -t docker.k1412.top/wuyang/k1412-agent-runtime:<release> --push .
+docker buildx build --platform linux/amd64 -f docker/gateway.Dockerfile \
+  -t docker.k1412.top/wuyang/k1412-agent-gateway:<release> --push .
+docker buildx build --platform linux/amd64 -f docker/workspace.Dockerfile \
+  -t docker.k1412.top/wuyang/k1412-agent-workspace:<release> --push .
+```
+
+Copy `deploy/docker-compose.yml`, `deploy/docker-compose.override.yml`, and
+`deploy/docker-compose.ssh.yml` to the NAS project. Update only immutable image
+tags and non-secret settings in the protected `.env`, validate with
+`docker compose config`, pull, and recreate.
+
+The SSH override removes `/var/run/docker.sock` from Gateway and mounts the
+dedicated SSH directory read-only at `/root/.ssh`.
+
+## Workspace migration
+
+Before changing execution hosts:
+
+1. stop Gateway so no new workspace mutation can begin;
+2. enumerate containers and volumes with the
+   `app.k1412.component=user-workspace` label/name prefix;
+3. stop each workspace container;
+4. stream a tar archive of each named volume to an identically named volume on
+   the new Docker host;
+5. pull the exact workspace image on the new host;
+6. start Gateway with the new SSH target;
+7. verify file listing and one read for every migrated volume;
+8. retain the old volumes until the verification window ends.
+
+Volume names contain only a user-ID hash, not email or display name.
+
+## Verification
+
+For every release:
+
+```bash
+.venv/bin/pytest
+RUN_DOCKER_INTEGRATION=1 .venv/bin/pytest tests/test_docker_workspace.py
+./scripts/verify-e2e.sh
+./scripts/audit-images.sh
+```
+
+Production smoke checks must cover:
+
+- anonymous requests are redirected/rejected;
+- an approved user can use Chat;
+- Chat can upgrade to Work but cannot downgrade;
+- Work creates and verifies a real file;
+- the file browser lists and downloads that file;
+- a second user cannot see it;
+- Gateway health reports `ssh-docker`;
+- the workspace container exists on the physical execution host, not the NAS.
+
+## Backup priorities
+
+Back up:
+
+1. PostgreSQL;
+2. the Open WebUI data volume;
+3. Runtime state in PostgreSQL;
+4. per-user workspace volumes on the execution host;
+5. the protected deployment `.env` and SSH directory through the private
+   infrastructure backup process.
+
+Redis is reconstructible coordination state and is lower priority than the
+database and workspace volumes.