> ## Documentation Index
> Fetch the complete documentation index at: https://docs.verial.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Criteria
> Typed assertions attached to a task. The verification engine runs them after a rollout to score the task.
Verial is a healthcare simulated environment platform. You define **environments** (simulated EHRs, phone lines, fax, clearinghouses, portals), group **tasks** into **benchmarks**, and each task has **criteria**, typed assertions that the **verification engine** runs after a rollout to score the task.
A **criterion** is a single typed assertion. It lives on a [Task](/api-reference/resources/tasks) and, after the task run completes, produces one [Criterion Run](/api-reference/resources/criterion-runs) with `passed`, `score`, `details`, and `evidence`.
## How Criteria Work
```mermaid theme={null}
graph LR
A["Agent Rollout"]:::input --> S["Sandbox State
(FHIR, HL7, SFTP, portal, voice, X12)"]:::state
S --> V["Verification Engine"]:::engine
C["Criteria (assertion specs)"]:::input --> V
V --> R["Criterion Runs
(passed, score, evidence)"]:::output
classDef input fill:#e0f2fe,color:#1e3a5f,stroke:#3b82f6
classDef state fill:#fef3c7,color:#78350f,stroke:#f59e0b
classDef engine fill:#2563eb,color:#fff,stroke:#1d4ed8
classDef output fill:#f0fdf4,color:#166534,stroke:#22c55e
```
Unlike the legacy `eval` approach (a single natural language `assert` string judged by an LLM), a criterion has a structured `assertion` object. The verification engine dispatches to a dedicated **check** implementation keyed by `assertion.assert`.
## Anatomy of a Criterion
| Field | Description |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `label` | Short human-readable description |
| `weight` | Relative contribution to the task score. The task score is a weighted mean of per-criterion scores |
| `axis` | Optional scoring axis. Criteria sharing an axis contribute to a per-axis score (for example `correctness`, `safety`, `efficiency`) |
| `input_entity_id` | Optional [DatasetEntity](/api-reference/resources/datasets) the criterion is scoped to (e.g. "the referral the agent should have processed") |
| `assertion` | Typed assertion spec. Discriminated on `assert` |
## Supported Checks
Each check is documented in full on the [Criteria API reference](/api-reference/resources/criteria). A quick tour:
Assert that a FHIR search returns a resource with the expected field values after the rollout.
Assert field values on HL7v2 outbound messages (ADT, ORU, ORM, SIU).
Assert that a row in simulated portal state has the expected values after submission.
Assert that a file was uploaded to the SFTP endpoint, optionally checking parsed JSON contents.
Assert that required phrases appear (and forbidden phrases do not) in the call transcript. Phrase matching is LLM-assisted.
Assert field values on an X12 EDI response (270/271/276/277/278).
## Annotated Examples
### FHIR: Appointment booked
```json theme={null}
{
"label": "Follow-up appointment booked with Dr. Rivera",
"weight": 1.0,
"axis": "correctness",
"assertion": {
"assert": "fhir-resource-state",
"resource_type": "Appointment",
"search": { "patient": "Patient/john-smith", "status": "booked" },
"fields": [
{ "path": "participant.0.actor.display", "expected": "Dr. Rivera" }
]
}
}
```
The engine runs a FHIR search for `Appointment?patient=Patient/john-smith&status=booked`, then asserts the first result has the expected participant display name.
### Voice: required disclosures
```json theme={null}
{
"label": "Agent collected member ID and DOB",
"weight": 0.5,
"axis": "correctness",
"assertion": {
"assert": "voice-transcript",
"speaker": "agent",
"contains": ["member ID", "date of birth"],
"not_contains": ["social security number"]
}
}
```
The engine fetches the voice transcript for the task run's playground and checks the agent's turns for required and forbidden phrases.
### SFTP: claim file uploaded
```json theme={null}
{
"label": "Claim file written to outbound/",
"weight": 1.0,
"assertion": {
"assert": "sftp-file-present",
"path_pattern": "outbound/claims/*.json",
"parse_json": true,
"fields": [
{ "path": "claim.patient.member_id", "expected": "BCB123456789" },
{ "path": "claim.total_charges", "expected": 420.00 }
]
}
}
```
### Portal: prior auth submitted
```json theme={null}
{
"label": "Prior auth submitted with correct CPT",
"weight": 1.0,
"axis": "correctness",
"assertion": {
"assert": "portal-state-match",
"correlate_by": { "resource": "prior_auth_requests", "key": "request_id" },
"assertions": [
{ "path": "status", "expected": "submitted" },
{ "path": "cpt_code", "expected": "72148" }
]
}
}
```
### HL7: ADT sent
```json theme={null}
{
"label": "ADT^A01 admission message sent for correct patient",
"weight": 1.0,
"assertion": {
"assert": "hl7-structural",
"correlate_by": { "MSH.9.1": "ADT", "MSH.9.2": "A01" },
"fields": [
{ "path": "PID.5.1", "expected": "Smith" },
{ "path": "PV1.2", "expected": "I" }
]
}
}
```
### X12: 271 eligibility response
```json theme={null}
{
"label": "Eligibility 271 returned active coverage",
"weight": 1.0,
"assertion": {
"assert": "x12-response",
"transaction": "271",
"fields": [
{ "path": "EB.1", "expected": "1" },
{ "path": "EB.3", "expected": "30" }
]
}
}
```
## Writing Good Criteria
* Prefer precise field assertions over free-form natural language.
* Group related criteria under an `axis` so you can see a per-axis score breakdown in the task score.
* Weight critical outcomes higher. Verial uses weighted means, so `weight: 2` doubles a criterion's contribution relative to `weight: 1`.
* Test negative behaviors too. For example a `voice-transcript` criterion with `not_contains: ["social security number"]`.
* Start narrow. One criterion per observable outcome is better than one compound criterion.
## Next Steps
How the verification engine scores criteria into a task score.
REST endpoints and full assertion spec reference.