# Parity and ship

> [!NOTE]
> Every WebGPU parity check in CI (`pnpm test:gpu` and `gpu-engine parity` on the repository examples) ran on a software adapter (Mesa llvmpipe through Dawn, the `webgpu` package). Real GPUs and real browsers have not been run. See [Status](https://gpu-engine.dev/docs/start/status).

## Backend parity rules

Trainer, CPU runtime and WGSL runtime must agree within each model's declared `parity.tolerance`.

| Pair | Requirement | Where it runs |
| --- | --- | --- |
| Trainer and CPU | Bitwise identical (both call `forward`) | CI |
| CPU and WGSL (software adapter) | Identical decoded output on every fixture; logits within `parity.tolerance` | CI, on Dawn (`webgpu` package) over Mesa lavapipe |
| CPU and WGSL (real GPU) | Same as above | Locally, recorded in `parity.json` and `MODEL_CARD.md` |
| CPU and WGSL (real browser) | Same as above | Locally, recorded in `parity.json` and `MODEL_CARD.md` |

Whether real browser parity can also run in CI is an [open question](https://gpu-engine.dev/docs/start/status#open-questions).

### Running `gpu-engine parity`

```sh
npm install --save-dev webgpu@^0.6.2
gpu-engine parity
```

On Linux without a hardware GPU, also install `mesa-vulkan-drivers` so Dawn can run on Mesa lavapipe. `webgpu` is resolved from the model package folder and imported only when `parity` runs; if it is missing, fails to load or finds no adapter, `parity` exits 1 with `PARITY_ADAPTER_MISSING`.

What `gpu-engine parity` checks:

- Loads `report.json` (`VERIFY_REPORT_INVALID` if missing or invalid) and the promoted weights `weights/<name>.bin`, verifying its SHA-256 matches `report.weights.sha256` (`VERIFY_WEIGHTS_CHANGED`).
- Builds fixtures from every dev input (in file and line order) plus four built-in edge inputs: empty text, one token, exactly `input.max` tokens and `input.max + 3` tokens. It never reads the test set and takes no digest.
- For a `retrieve` model, compares scores against `eval.table` (trainer side runs `forward` followed by the score step).
- **Trainer vs CPU:** compares the trainer's forward pass on decoded weights against `loadModel(weights, { backend: 'cpu' })`. Logits must be bitwise identical; any difference or non-finite logit fails with `PARITY_TOLERANCE_EXCEEDED`.
- **CPU vs WGSL:** compares CPU against `loadModel(weights, { backend: 'webgpu', gpu })`. Every fixture must decode to the same result (`eval.decode(output, input)` or raw head output, with numbers compared within `eval.parity.tolerance`, else `PARITY_DECODE_DIFFERS`), and the largest absolute logit difference over all fixtures must be at most `eval.parity.tolerance` (`PARITY_TOLERANCE_EXCEEDED`).

On a pass, `parity` writes `parity.json` (`date`, `weights`, `adapter`, `tolerance`, `fixtures`, `trainerCpu: "identical"`, `maxDifference`, `decodeDiffers`, `realGpu: "not run"`, `realBrowser: "not run"`) and regenerates `MODEL_CARD.md`. When only `date` would change, `parity.json` keeps its existing bytes so repeated runs leave the working tree clean. On failure, it prints every diagnostic, writes nothing and exits 1.

The repository's CI `gpu` job builds gpu-engine and runs `gpu-engine parity` in every `examples/*` folder that has a `report.json` (`examples/tag`, `examples/pooled`, `examples/retrieve`).

## Library GPU tests (`pnpm test:gpu`)

`pnpm test:gpu` runs in CI on Ubuntu with Dawn over Mesa lavapipe and fails (never skips) if no adapter is available:

- **Per-op parity (`test/gpu/*.test.ts`).** Every layer (`embed`, `conv`, `biScan`, `dense`) and all three heads (`tag`, class and values `pooled` including a 1-token model, and `retrieve` head op plus dot-product score step) match their CPU forward pass within `1e-5` on random inputs for 1, 2 and maximum tokens.
- **Whole-model parity (`test/gpu/model.test.ts`, `test/gpu/retrieve.test.ts`).** Checks a full `tag` model across 50 random inputs (empty, single-token, truncated, punctuation, Unicode and flags) within `1e-3` with identical decoded tags, a `pooled` model across 20 random inputs within `1e-3` with identical decoded labels, and a `retrieve` model across 30 inputs against 100 table rows within `1e-3` with matching top score. Also checks `'auto'` threshold switching, lost devices, missing tables (`RUN_TABLE_MISSING`) and WGSL validation failures (`RUN_GPU_FAILED`).

All of these run on a software adapter (llvmpipe) only. Real GPU and real browser runs have not been executed yet; each op notes this in the [layer catalog](https://gpu-engine.dev/docs/reference/layers).

## Generated release files

`gpu-engine promote` generates three files using pure renderers. `MODEL_CARD.md` and `RESULTS.md` are rendered from `report.json` (`renderModelCard` and `renderResults`, exported from `gpu-engine/harness`), while `src/weights.ts` is rendered from the promoted weights bytes and SHA-256:

- **`MODEL_CARD.md`.** States the promotion date, commit, definition hash, weights path and Brotli size, parameter and multiply-add counts, test seal level and test read count from `reads.jsonl`, dev and test score tables, per-seed dev scores, gate lines, locked data hashes and parity status. When `parity.json` matches the promoted weights SHA-256, it names the adapter, labels known software rasterizers explicitly (such as `software adapter (llvmpipe)`), and reports the `realGpu` and `realBrowser` fields (`not run` until recorded). Without matching `parity.json`, it states that parity has not run.
- **`RESULTS.md`.** Contains the dev score table, test score table and gate lines.
- **`src/weights.ts`.** Exports `weightsBase64` holding the promoted `weights/<name>.bin` bytes.

Never edit these generated files by hand. Both `gpu-engine promote --verify` and `gpu-engine doctor` regenerate them in memory and fail with `VERIFY_FILE_STALE` on any byte difference.

## CI verification

Run the promotion verifier in CI:

```sh
gpu-engine promote --verify
```

`promote --verify` reads no test data and needs no level 3 key. It runs the level 1 scan, validates `report.json`, checks `data.lock.json` and section 7 data rules, verifies `weights/<name>.bin` hashes and sizes, re-scores dev to match `report.dev`, checks `gate.json`, budgets and dev floors, confirms `reads.jsonl` holds at least `report.testReads` reads, and verifies that `MODEL_CARD.md`, `RESULTS.md` and `src/weights.ts` match byte for byte.

## Building and ship checks

A scaffolded model package ships as one zero-dependency ES module. Running `npm run build` executes `scripts/build.ts`, which strips types and bundles `src/index.ts`, `src/decode.ts`, `src/weights.ts`, the `gpu-engine` runtime (and `src/table.ts`, regenerated from `data/table.jsonl` for `retrieve` models) into `dist/index.mjs` with zero imports. Any `node:` or Node built-in import stops the build.

```sh
npm run build
gpu-engine doctor
```

Whenever `dist/` exists, `gpu-engine doctor` runs the ship checks (`src/harness/ship.ts`) against `package.json`, the file list reported by `npm pack --dry-run --json --ignore-scripts`, and `weights/<name>.bin`, without writing any file:

| Check | Code |
| --- | --- |
| `package.json` exists and parses, and `npm pack --dry-run --json` lists the packed files | `SHIP_PACKAGE_INVALID` |
| `package.json` has no `dependencies`, `peerDependencies`, `optionalDependencies`, `bundleDependencies` or `bundledDependencies` field, and no packed `.js`, `.mjs` or `.cjs` file contains an `import`, `export ... from` or dynamic `import(...)` | `SHIP_DEPENDENCY_FOUND` |
| Once `weights/<name>.bin` exists, a packed file contains it as base64 (rebuild after every `promote`) | `SHIP_BUILD_STALE` |
| Every packed file is `package.json` or an exact path listed in `eval.ship.files` (such as `dist/index.mjs`) | `SHIP_FILE_NOT_ALLOWED` |
| Summed Brotli bytes (quality 11) of all packed files except `package.json` are at most `eval.ship.brotliBytes` | `SHIP_SIZE_EXCEEDED` |

Without `ship` in `<name>.eval.ts`, the last two checks do not run and `doctor` prints a note. Without a `dist/` folder, `doctor` prints a note that nothing is built to check.

Both the repository's CI `gate` job (across every `examples/*` folder with a `report.json`) and every scaffolded package's `.github/workflows/ci.yml` run `npm run build`, `gpu-engine doctor` and `gpu-engine promote --verify` under `set -euo pipefail`.

See [Diagnostics](https://gpu-engine.dev/docs/reference/diagnostics) for every `PARITY_*`, `VERIFY_*` and `SHIP_*` code.

Source: SPEC sections 6, 9, 10, 11 and 12.

<!-- vitehub-docs:section-links -->
## In this section

- [Define a model](https://gpu-engine.dev/raw/docs/build/define-a-model.md)
- [Data and splits](https://gpu-engine.dev/raw/docs/build/data-and-splits.md)
- [Train and tune](https://gpu-engine.dev/raw/docs/build/train-and-tune.md)
- [Evaluate and promote](https://gpu-engine.dev/raw/docs/build/evaluate-and-promote.md)
- [Run a model](https://gpu-engine.dev/raw/docs/build/run-a-model.md)
- [Parity and ship](https://gpu-engine.dev/raw/docs/build/parity-and-ship.md) (this page)

All pages: [llms.txt](https://gpu-engine.dev/llms.txt)
