# Define a model

Each model package has one `<name>.model.ts` file that default-exports `defineModel(...)`. `defineModel` is a plain function with no dependency on the host app. It validates the definition, throws a `GpuEngineError` with a fix if the layout cannot work, and returns its argument unchanged.

```ts [my-model.model.ts]
import { defineModel, features as f, int6, layers as l } from 'gpu-engine'

export default defineModel({
  name: 'my-model',
  input: f.tokens({ max: 64, rows: [f.shape(), f.hash({ buckets: 128, k: 2 })] }),
  net: [l.embed(20), l.conv({ taps: 3 }), l.biScan(20), l.dense(20, 'relu')],
  head: { kind: 'tag', labels: ['start', 'inside', 'none'] },
  quant: int6({ qatFrom: 0.7 }),
  budgets: { brotliBytes: 5_000, macsPerToken: 3_200 },
})
```

## Configuration fields

### `input`

`f.tokens({ max, rows })` configures the shared featurizer. `tokenize(text)` splits the input string into tokens, keeps at most the first `max` tokens, and assigns each kept token the slots declared in `rows`:

| Builder | What each token receives | Slots | Embedding rows |
| --- | --- | --- | --- |
| `f.shape()` | Character class (8 classes) and code-point length bucket (5 buckets) | 2 | 13 |
| `f.hash({ buckets, k })` | `k` distinct 32-bit FNV-1a hashed rows of the lowercased, NFC-normalised UTF-8 token | `k` | `buckets` |
| `f.flag(name)` | One caller-supplied boolean per token (`-1` when false or omitted) | 1 | 1 |

Hashing replaces a learned vocabulary so any token maps into a fixed row range. Trainer, CPU runtime and WebGPU runtime all call the same CPU featurizer and MUST produce identical rows for the same input.

`tokenize` is exported from `gpu-engine`. It walks Unicode code points without splitting surrogate pairs and returns `{ text, start, end }[]` using UTF-16 half-open offsets:

```ts
import { tokenize } from 'gpu-engine'

tokenize('tomorrow at 5pm').map((token) => token.text) // ['tomorrow', 'at', '5', 'pm']
```

Letters (`\p{L}`) join letters, decimal digits (`\p{Nd}`) join digits, marks (`\p{M}`) attach to the preceding token, whitespace (`\p{White_Space}`) ends a token without entering the list, and any other code point forms its own token (`don't` becomes `don`, `'`, `t`). See SPEC section 5 for every rule.

### `net`

`net` is always an explicit list of ops from the closed v1 catalog. It must start with one `embed` layer and cannot contain a second `embed`. There are no architecture presets.

| Builder | What it computes |
| --- | --- |
| `l.embed(size)` | Sums the embedding table rows for a token's active slots plus a bias |
| `l.conv({ taps })` | Depthwise 1D convolution over an odd number of neighbouring tokens with `tanh` |
| `l.biScan(size)` | Bidirectional gated affine scan; outputs `2 * size` channels (forward then backward) |
| `l.dense(size, activation)` | Per-token linear transform with `'relu'`, `'tanh'` or `'none'` |

Full formulas, initialisations, parameter counts and tradeoffs are in the [layer catalog](https://gpu-engine.dev/docs/reference/layers).

### `head`

The head maps the final layer's output to labels, numbers or a ranked candidate table.

| Kind | Shape | Output |
| --- | --- | --- |
| `tag` | `{ kind: 'tag', labels }` | One label per token |
| `pooled` | `{ kind: 'pooled', labels }` or `{ kind: 'pooled', outputs }` | One class label or `outputs` numbers for the whole input |
| `retrieve` | `{ kind: 'retrieve', size }` | One `{ id, score }` per row of a candidate table, highest score first |

Heads with `labels` require at least two distinct non-empty strings. A `pooled` head accepts either `labels` (class form) or a positive integer `outputs` (values form), never both.

#### How the `retrieve` head works

A `retrieve` head scores an input against the `{ id, text }` rows of a candidate table through one shared tower:

- The tower is `net` followed by the `pooled` pool-and-linear op with `out = size` (a positive integer), turning any text into a vector of `size` floats. Both the input and every table row pass through the same tower weights.
- The score step is a separate op with no tensors: `score[r] = sum over i of query[i] * table[r, i]`, accumulated in doubles and stored as float32. It costs `rows * size` multiply-adds per run for a table of `rows` rows, which `macsPerToken` does not count.
- Decode ranks every table row as `{ id, score }`, highest score first, preserving table order on a tie.

```ts [command-lookup.model.ts]
import { defineModel, features as f, int6, layers as l } from 'gpu-engine'

export default defineModel({
  name: 'command-lookup',
  input: f.tokens({ max: 12, rows: [f.hash({ buckets: 8192, k: 2 })] }),
  net: [l.embed(16), l.dense(16, 'tanh')],
  head: { kind: 'retrieve', size: 16 },
  quant: int6({ qatFrom: 0.7 }),
  budgets: { brotliBytes: 30_000, macsPerToken: 1_000 },
})
```

### `quant`

`int6({ qatFrom })` enables straight-through int6 quantization-aware training starting at share `qatFrom` (`0` to `1`) of the planned training steps. Omitting `quant` keeps weights in float32. See [Train and tune](https://gpu-engine.dev/docs/build/train-and-tune#quantization-aware-training).

### `budgets`

`budgets` caps model size and compute:

- `brotliBytes`: maximum size of the Brotli-compressed weights file (quality 11).
- `macsPerToken`: maximum multiply-adds per token from the plan.

`defineModel` enforces both before training starts. The multiply-add cap is exact. The byte check is a safe floor: it rejects a model only when the packed int6 bytes of every tensor outside `embed.table` exceed twice `budgets.brotliBytes`, because trained tensors do not compress below half their packed int6 size while sparse embedding tables can compress further. The actual Brotli size is enforced after training (`BUDGET_BYTES_EXCEEDED`).

## Define-time errors

`defineModel` calls `plan` and `checkBudgets`, throwing a `GpuEngineError` on any violation. `error.diagnostic` contains `{ code, message, why, fix }`.

| Code | Trigger |
| --- | --- |
| `DEFINE_NET_ORDER` | `net` is empty, does not begin with `embed`, or includes `embed` more than once |
| `DEFINE_LAYER_SHAPE` | Non-positive or non-integer layer size (or `retrieve` head size), even or non-positive `taps`, unknown activation, or unknown layer kind |
| `DEFINE_HEAD_LABELS` | Fewer than two labels, empty or duplicate label, both or neither of `labels` and `outputs` on `pooled`, non-positive `outputs`, `outputs` on `tag`, or unknown head kind |
| `DEFINE_FEATURE_ROW` | Invalid `max`, `buckets` or `k`, empty `rows`, unknown row kind, more than one `shape` row, or empty or duplicate flag name |
| `DEFINE_BUDGET_EXCEEDED` | `macsPerToken` exceeds `budgets.macsPerToken`, or the non-embedding int6 floor exceeds twice `budgets.brotliBytes` |

For example, putting `biScan(64)` under a 3,200 multiply-add cap fails at define time:

```ts
import { GpuEngineError, defineModel, features as f, layers as l } from 'gpu-engine'

try {
  defineModel({
    name: 'too-big',
    input: f.tokens({ max: 64, rows: [f.shape(), f.hash({ buckets: 128, k: 2 })] }),
    net: [l.embed(64), l.biScan(64)],
    head: { kind: 'tag', labels: ['start', 'inside', 'none'] },
    budgets: { brotliBytes: 5_000, macsPerToken: 3_200 },
  })
} catch (error) {
  if (error instanceof GpuEngineError) console.log(error.diagnostic.code, error.diagnostic.fix)
}
```

Formatted with `formatDiagnostic(error.diagnostic)`, that error reads:

```txt
DEFINE_BUDGET_EXCEEDED: the layers need 17152 multiply-adds per token, budgets.macsPerToken is 3200
  why: The layers already exceed a budget before any training: multiply-adds per token, or a packed int6 size that cannot fit the Brotli budget.
  fix: Shrink a layer (embed size, scan size, dense size, hash buckets) or raise the budget in `budgets`.
```

Runtime inference behaves differently: `model.run(input)` returns `[error, data]` and never throws.

## Inspecting op costs with `plan`

`plan(definition)` in `gpu-engine/harness` returns the ordered ops (`net` layers followed by the head) with each op's `inWidth`, `outWidth`, `tensors` and `macs`, plus model totals: `size` (parameter count), `macsPerToken`, `slots`, `rows`, `maxTokens`, `outWidth` and the 16-hex definition `hash`.

```ts
import { plan } from 'gpu-engine/harness'
import model from './my-model.model.ts'

const planned = plan(model)
console.log(planned.size, planned.macsPerToken) // 5483 2640
for (const op of planned.ops) console.log(op.spec.kind, op.inWidth, op.outWidth, op.macs)
```

For `my-model` above:

| Op | Input width | Output width | Multiply-adds per token |
| --- | --- | --- | --- |
| `embed(20)` | 4 | 20 | 80 |
| `conv({ taps: 3 })` | 20 | 20 | 60 |
| `biScan(20)` | 20 | 40 | 1,640 |
| `dense(20, 'relu')` | 40 | 20 | 800 |
| `tag` (3 labels) | 20 | 3 | 60 |
| Total | | | 2,640 (within 3,200 budget) |

The [playground](https://gpu-engine.dev/playground) typechecks example model files in the browser against the library's type declarations; it does not run `defineModel` or `plan` and does not render this cost table.

## Type inference from head labels and tables

`defineModel` preserves literal label types (`const` type parameter). `ModelOutput<typeof model, Id>` derives the raw head output type:

| Head | `ModelOutput<typeof model, Id>` |
| --- | --- |
| `tag` with `labels` | `Labels[number][]` |
| `pooled` with `labels` | `Labels[number]` |
| `pooled` with `outputs` | `number[]` |
| `retrieve` | `RetrieveMatch<Id>[]` (each `{ readonly id: Id; readonly score: number }`, where `Id` defaults to `string` and narrows to the table's literal ids when `table` is passed inline or `as const` to `loadModel`) |

```ts
import type { ModelOutput } from 'gpu-engine'
import type model from './my-model.model.ts'

// ('start' | 'inside' | 'none')[]
type Tags = ModelOutput<typeof model>
```

If you rename or remove a label in `<name>.model.ts`, any `decode` or eval code referencing that label fails typechecking immediately.

## Keep `decode` in plain code

Converting tags, numbers or table rankings into your application type always belongs in plain TypeScript or JavaScript outside gpu-engine. Pass that function to `defineEval` so scoring compares decoded results, and call the same function after `model.run(input)` in your app.

Source: SPEC sections 4, 5, 9 and 11, and `src/types.ts`.

<!-- vitehub-docs:section-links -->
## In this section

- [Define a model](https://gpu-engine.dev/raw/docs/build/define-a-model.md) (this page)
- [Data and splits](https://gpu-engine.dev/raw/docs/build/data-and-splits.md)
- [Train and tune](https://gpu-engine.dev/raw/docs/build/train-and-tune.md)
- [Evaluate and promote](https://gpu-engine.dev/raw/docs/build/evaluate-and-promote.md)
- [Run a model](https://gpu-engine.dev/raw/docs/build/run-a-model.md)
- [Parity and ship](https://gpu-engine.dev/raw/docs/build/parity-and-ship.md)

Next: [Data and splits](https://gpu-engine.dev/raw/docs/build/data-and-splits.md)

All pages: [llms.txt](https://gpu-engine.dev/llms.txt)
