# Slates and preferences

> Runs started together on one task to be compared, and the judgements between them.

> **Info:** Added in 0.3. The matching [producer capability](https://orchestrationcontextprotocol.io/specification/0.3/document#producer) is `slate`.

A **slate** is a set of runs started together on one task, from one commit, to be compared. A pair is a slate of two. Runs on the same task and the same starting point compare configurations directly, without differences in task difficulty in the way. A **preference** is a judgement between slate members: which one is better, or a tie.

Each run is its own document, so each member's file carries the same `slate` object, and every preference is copied into every member's file.

```json
"slate": {
  "id": "slt_17",
  "members": ["run_17a", "run_17b"],
  "base_commit": "a1b2c3d",
  "isolated": true,
  "blinded": true
},
"preferences": [
  {
    "id": "prf_17",
    "slate": "slt_17",
    "winner": "run_17a",
    "members": ["run_17a", "run_17b"],
    "judge": { "kind": "referee", "model": "gpt-6-astra", "blinded": true },
    "observed_at": "2026-09-23T10:56:00-07:00",
    "tier": "reported"
  }
]
```

## Slate

Runs started together on one task to be compared (a pair is a slate of two). Every member's file carries the same object (E194).

- `id` ([Id](https://orchestrationcontextprotocol.io/specification/0.3/schema#id), required)

- `members` (string[], required): Run ids.

- `base_commit` (string): At most 200 characters.

- `isolated` (boolean): Members ran in separate worktrees or sandboxes.

- `blinded` (boolean): Judges did not know which member ran which configuration.

- `ext` ([Ext](https://orchestrationcontextprotocol.io/specification/0.3/schema#ext))

- `isolated` says the members ran in separate worktrees or sandboxes, so one could not see or disturb another's changes.
- `blinded` says the judges did not know which member ran which configuration.

## Preferences

A pairwise judgement between slate members. Copied into every member's file; readers deduplicate by id.

- `id` ([Id](https://orchestrationcontextprotocol.io/specification/0.3/schema#id), required)

- `slate` (string, required): At most 200 characters.

- `winner` (string, required): A member run id, or `tie` (E194).

  At most 200 characters.

- `members` (string[])

- `judge` (object): *properties*

  - `kind` (string): Recommended values (open set): `referee`, `user`, `model`.

    At most 100 characters.

  - `model` (string): At most 200 characters.

  - `blinded` (boolean)

  - `ext` ([Ext](https://orchestrationcontextprotocol.io/specification/0.3/schema#ext))

- `observed_at` ([Timestamp](https://orchestrationcontextprotocol.io/specification/0.3/schema#timestamp))

- `tier` ([OutcomeTier](https://orchestrationcontextprotocol.io/specification/0.3/schema#outcometier))

- `ext` ([Ext](https://orchestrationcontextprotocol.io/specification/0.3/schema#ext))

- A reader MUST deduplicate preferences by `id`, since the same preference appears in every member's file.
- `winner` is a member's run id, or `tie`.
- `judge.kind` says who judged: a `referee` agent, the `user`, or a `model`. `tier` says how the judgement is known.

## Requirements

These are checked as [E194](https://orchestrationcontextprotocol.io/specification/0.3/conformance#e194):

- The run's id MUST be one of `slate.members`.
- When both are present, `task.base_commit` MUST equal `slate.base_commit`.
- A preference that names a slate MUST name this run's slate. Its `winner` MUST be `tie` or one of the preference's `members`, or of the slate's members when the preference lists none.
- Documents with the same `slate.id` MUST share `task.id` and the base commit (`task.base_commit`, else `slate.base_commit`). A checker can only see this when it reads several files at once, as `loopmath ocp validate` does when given more than one file.

A preference is a judgement between slate members, and each member's file carries the same `slate` object. The reference checker does not test either: it does not check a preference's own `members` against the slate, and across files it compares only the task and the base commit. See [what the checker does not test](https://orchestrationcontextprotocol.io/specification/0.3/conformance#checker-limits).