> ## Documentation Index
> Fetch the complete documentation index at: https://docs.verial.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Benchmarks

> Collections of tasks that test specific healthcare AI agent capabilities.

A benchmark is a named collection of tasks that test your agent's ability to perform specific healthcare workflows. Each task binds input data, optional scenario steps, and [criteria](/guides/concepts/criteria) that the [verification engine](/guides/concepts/verification) scores after a rollout. Benchmarks are linked to an [environment](/guides/concepts/environments) that defines the simulated systems the agent will drive.

## Structure

A benchmark contains:

* **Name and slug/version.** Published benchmarks are identified by `slug@version`.
* **Environment.** The simulated health system the agent will run against.
* **Tasks.** Individual test cases, each with input data bindings, optional scenario steps, and criteria.
* **Criteria.** Typed assertions per task, scored by the verification engine.
* **Configuration.** Timeout and concurrency settings.

```mermaid theme={null}
graph TD
  B["Benchmark"]:::bench --> T1["Task 1: Submit Prior Auth"]:::task
  B --> T2["Task 2: Check Auth Status"]:::task

  T1 --> C1["Criterion (portal-state-match)"]:::crit
  T1 --> C2["Criterion (voice-transcript)"]:::crit
  T2 --> C3["Criterion (portal-state-match)"]:::crit

  classDef bench fill:#2563eb,color:#fff,stroke:#1d4ed8
  classDef task fill:#3b82f6,color:#fff,stroke:#2563eb
  classDef crit fill:#e0f2fe,color:#1e3a5f,stroke:#3b82f6
```

## Creating a Benchmark

```typescript theme={null}
const benchmark = await verial.benchmarks.create({
  name: "Prior Authorization Workflow",
  environmentId: environment.id,
  timeout: 300,
  concurrency: 1,
});
```

## Tasks

Tasks are separate resources linked to a benchmark. Each task carries its `task_item` (instruction, trigger), an optional pre-rollout `scenario`, `entities` that scope it to specific dataset rows, and the typed [criteria](/guides/concepts/criteria) the verification engine scores. See the [Tasks concept](/guides/concepts/tasks) page for the full field reference and examples.

## Multi-Interface Tasks

Tasks can span multiple simulators. A single task can require the agent to read a chart in FHIR, call an IVR line, submit a request to a payer portal, and fax documentation. The verification engine runs one criterion per observable outcome against the relevant sandbox state.

## Configuration

### Timeout

Per-task timeout (seconds). A task that exceeds its timeout is recorded with `verdict: "fail"` and score `0`.

### Concurrency

How many task rollouts may run in parallel within a benchmark run. Default `1` (sequential).

## Next Steps

<CardGroup cols={2}>
  <Card title="Runs" icon="play" href="/guides/concepts/runs">
    Execute benchmarks and review results.
  </Card>

  <Card title="Criteria" icon="check" href="/guides/concepts/criteria">
    Write typed assertions for each check type.
  </Card>
</CardGroup>