Skip to main content
A task comes as a package: one Harbor task, as a plain tar. Download it with GET /v1/tasks/{task_id}/versions/{version}/package. The task ID keeps its slash in the path.

​
What’s inside

There is no rubric, reference solution or judge in it: those stay with us.

​
Check the bytes

The same version always returns the same bytes. Their SHA-256 is in the ETag and Repr-Digest response headers, and AICME sends it with the version’s ID. Check it before you use the package:

​
Run it with Harbor

Set two things before you run it:
  • Your model’s host. The agent’s network is an allowlist, and it ships empty. Add the host your agent calls.
  • Your key, in your environment, as AICME_API_KEY. task.toml passes it to the verifier. The key is never written into the package.
Optionally, set AICME_CLIENT_REF to your own label for the run, such as your rollout ID, up to 256 characters. The verifier sends it as the reward’s client_ref, and we return it with the reward. An answer you’ve already had scored returns that first reward, with the client_ref it was sent with. When the agent finishes, the verifier posts its answer, /app/answer.md, to POST /v1/rewards and waits for the reward. It writes the four scores to /logs/verifier/reward.json, which Harbor reads, and the reward’s ID and flags to /logs/verifier/aicme.json.