Slates and preferences

"slate": {
  "id": "slt_17",
  "members": ["run_17a", "run_17b"],
  "base_commit": "a1b2c3d",
  "isolated": true,
  "blinded": true
},
"preferences": [
  {
    "id": "prf_17",
    "slate": "slt_17",
    "winner": "run_17a",
    "members": ["run_17a", "run_17b"],
    "judge": { "kind": "referee", "model": "gpt-6-astra", "blinded": true },
    "observed_at": "2026-09-23T10:56:00-07:00",
    "tier": "reported"
  }
]

​

Slate

• isolated says the members ran in separate worktrees or sandboxes, so one could not see or disturb another’s changes.
• blinded says the judges did not know which member ran which configuration.

​

Preferences

• A reader MUST deduplicate preferences by id, since the same preference appears in every member’s file.
• winner is a member’s run id, or tie.
• judge.kind says who judged: a referee agent, the user, or a model. tier says how the judgement is known.

​

Requirements

• The run’s id MUST be one of slate.members.
• When both are present, task.base_commit MUST equal slate.base_commit.
• A preference that names a slate MUST name this run’s slate. Its winner MUST be tie or one of the preference’s members, or of the slate’s members when the preference lists none.
• Documents with the same slate.id MUST share task.id and the base commit (task.base_commit, else slate.base_commit). A checker can only see this when it reads several files at once, as loopmath ocp validate does when given more than one file.