> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aicmelabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Rewards

> What a reward holds: four scores, flags, and its status

A reward is created when you send an answer to `POST /v1/rewards`, and read with `GET /v1/rewards/{id}`:

```json theme={null}
{
  "object": "reward",
  "id": "rw_00d1h3x3y8evb35vv5bvvbxe2q4g3rrxg6jr",
  "status": "scored",
  "episode_id": null,
  "task": {"id": "aicme/example-001", "version": "tv_3f9a1c07b2e45d18"},
  "judge_version": "j3",
  "scores": {"reward": 0.75, "partial": 0.75, "must_pass": 1.0, "all_pass": 0.0},
  "flags": [],
  "failure": null,
  "client_ref": null,
  "created_at": "2026-10-28T21:31:32.440Z",
  "scored_at": "2026-10-28T21:31:52.940Z"
}
```

## Status

| Status | Meaning |
| - | - |
| `pending` | Still being scored. Returned with `202` and a `Retry-After` header. Poll again |
| `scored` | Final. `scores` holds the reward |
| `failed` | Final. We couldn't score the answer. See [A failed reward](#a-failed-reward) |

## The scores

| Field | Range | What it is |
| - | - | - |
| `reward` | 0 to 1 | The reward for the answer |
| `partial` | 0 to 1 | Partial credit: a denser signal than `reward` |
| `must_pass` | 0 or 1 | Whether the answer met everything it must |
| `all_pass` | 0 or 1 | Whether the answer met everything |

Use `reward` to compare with AICME's published results. `partial` is the denser signal, and `must_pass` and
`all_pass` are binary rewards, if your trainer wants one.

## The flags

Each flag explains a reward of 0. All four scores are 0 when one is set.

| Flag | Meaning |
| - | - |
| `empty_answer` | The answer was empty, or only whitespace |
| `over_length` | The answer was longer than the task's soft cap ([Limits](https://docs.aicmelabs.com/limits.md)) |
| `instructs_judge` | The answer addressed the scorer instead of answering the task |
| `unscorable` | The answer couldn't be scored, after repeated attempts |

New flags may be added. Treat a flag you don't know as a reward of 0 with a reason.

## A failed reward

`status: failed` means we couldn't score the answer. It is never caused by the answer, and it is never a 0:

```json theme={null}
{"status": "failed", "scores": null, "failure": {"reason": "infra", "retryable": true}}
```

We retry a reward ourselves before we report it as `failed`. To try again, send the same answer: a failed reward
isn't reused, so that creates a new one. `reason` is `judge_error` or `infra`, and new reasons may be added.

## Versions

* **Send the `version` you downloaded and ran,** so the answer is scored against that version. The package's
  verifier always does.
* **Leave out `version` and `judge_version`** to use the ones your lab is pinned to. Your first reward on a task pins
  the versions current then.
* **Name a version** to score against it. It doesn't move your pin.

## Sending the same answer again

The same answer, sent again for the same task version and `judge_version`, returns the first reward. It isn't scored
twice. `client_ref` is optional: your own label of up to 256 characters, returned on the reward.

## Waiting instead of polling

Send `Prefer: wait=<seconds>`, up to 120, to have us hold the request until the reward is final. See
[Limits](https://docs.aicmelabs.com/limits.md#waiting-for-a-reward).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.