Criteria

A Criterion is a typed assertion attached to a Task. After a task rollout completes, the verification engine evaluates each criterion against the recorded environment state (FHIR resources, HL7 messages, SFTP uploads, portal state, voice transcripts, X12 responses) and produces a Criterion Run with a pass or fail result, a numeric score, and evidence.

Endpoints

Method	Endpoint	Description
`GET`	`/criteria?task_id={task_id}`	List criteria for a task
`POST`	`/criteria`	Create a criterion
`GET`	`/criteria/{id}`	Get criterion details
`PATCH`	`/criteria/{id}`	Update a criterion
`DELETE`	`/criteria/{id}`	Delete a criterion

Criterion Object

Field	Type	Description
`id`	string	Unique identifier
`task_id`	string	Parent Task
`input_entity_id`	string \| null	Optional DatasetEntity the criterion is scoped to
`label`	string	Short human-readable label
`assertion`	object	Typed assertion spec. Shape depends on `assertion.assert` (see below)
`weight`	number	Relative weight for task scoring. Higher means more important
`axis`	string \| null	Optional scoring axis for per-axis breakdown
`created_at`	datetime	Creation timestamp

Assertion Specs

The assertion field is a discriminated union on the assert key. The verification engine dispatches to a check implementation based on this value.

fhir-resource-state

Asserts that, after the rollout, a FHIR resource search returns a resource whose fields match expected values.

Field	Type	Description
`assert`	`"fhir-resource-state"`	Discriminator
`resource_type`	string	FHIR resource type, for example `"Appointment"`
`search`	object	FHIR search params as key or value strings. Default `{}`
`fields`	array	Field assertions. Each has `path` (dotted path into the resource) and `expected` (string, number, boolean, or null)

{
  "assert": "fhir-resource-state",
  "resource_type": "Appointment",
  "search": { "patient": "Patient/john-smith", "status": "booked" },
  "fields": [
    { "path": "participant.0.actor.display", "expected": "Dr. Rivera" },
    { "path": "serviceType.0.coding.0.code", "expected": "follow-up" }
  ]
}

hl7-structural

Asserts field values on HL7v2 outbound messages captured by the sandbox.

Field	Type	Description
`assert`	`"hl7-structural"`	Discriminator
`correlate_by`	object	Key or value map used to select the matching HL7 message. Default `{}`
`fields`	array	Each entry has `path` (HL7 segment-field path, e.g. `PID.5.1`) and `expected` (string)

{
  "assert": "hl7-structural",
  "correlate_by": { "MSH.9.1": "ADT" },
  "fields": [
    { "path": "PID.5.1", "expected": "Smith" },
    { "path": "PV1.2", "expected": "I" }
  ]
}

portal-state-match

Asserts that after a web portal simulation ends, a given resource row has the expected fields.

Field	Type	Description
`assert`	`"portal-state-match"`	Discriminator
`portal`	string \| null	Optional portal slug
`correlate_by`	object	`{ resource, key }`: which sandbox-state resource to load, and which key identifies the row
`assertions`	array	Each entry has `path` (dotted JSON path) and `expected` (any JSON value)

{
  "assert": "portal-state-match",
  "correlate_by": { "resource": "prior_auth_requests", "key": "request_id" },
  "assertions": [
    { "path": "status", "expected": "submitted" },
    { "path": "cpt_code", "expected": "72148" }
  ]
}

sftp-file-present

Asserts that a file was uploaded to the sandbox SFTP endpoint, optionally parsing JSON contents and asserting fields.

Field	Type	Description
`assert`	`"sftp-file-present"`	Discriminator
`path`	string \| null	Exact path
`path_pattern`	string \| null	Glob pattern with `*` wildcard. Use instead of `path` for multiple candidate files
`parse_json`	boolean	Parse the file as JSON before checking fields. Default `false`
`fields`	array	Each entry has `key` or `path` and `expected`. Use `key` for sidecar-style keys, `path` for dotted JSON paths

{
  "assert": "sftp-file-present",
  "path_pattern": "outbound/claims/*.json",
  "parse_json": true,
  "fields": [
    { "path": "claim.total_charges", "expected": 420.00 },
    { "path": "claim.status", "expected": "submitted" }
  ]
}

voice-transcript

Asserts the content of a voice call transcript. This is the check that uses LLM-assisted matching for loose phrase presence.

Field	Type	Description
`assert`	`"voice-transcript"`	Discriminator
`speaker`	`"agent" \| "caller" \| "ivr" \| "any"`	Which speaker’s turns to match against. Default `"any"`
`contains`	string[]	Phrases that must appear in the transcript
`not_contains`	string[]	Phrases that must not appear

{
  "assert": "voice-transcript",
  "speaker": "agent",
  "contains": ["member ID", "date of birth"],
  "not_contains": ["social security number"]
}

x12-response

Asserts field values on an X12 EDI response (270/271/276/277/278) produced by the sandbox clearinghouse.

Field	Type	Description
`assert`	`"x12-response"`	Discriminator
`transaction`	`"270" \| "271" \| "276" \| "277" \| "278" \| null`	Optional transaction set filter
`correlate_by`	object	Key or value map to pick the matching response
`fields`	array	Each has `path` and `expected` (string, number, boolean, or null)

{
  "assert": "x12-response",
  "transaction": "271",
  "fields": [
    { "path": "EB.1", "expected": "1" },
    { "path": "EB.3", "expected": "30" }
  ]
}

HTTP Example

# Create a criterion
curl -X POST https://api.verial.ai/criteria \
  -H "Authorization: Bearer $VERIAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "task_id": "task_abc123",
    "label": "Appointment booked for correct patient",
    "weight": 1.0,
    "axis": "correctness",
    "assertion": {
      "assert": "fhir-resource-state",
      "resource_type": "Appointment",
      "search": { "patient": "Patient/john-smith", "status": "booked" },
      "fields": [
        { "path": "participant.0.actor.display", "expected": "Dr. Rivera" }
      ]
    }
  }'

Response:

{
  "id": "crit_01H...",
  "task_id": "task_abc123",
  "input_entity_id": null,
  "label": "Appointment booked for correct patient",
  "assertion": {
    "assert": "fhir-resource-state",
    "resource_type": "Appointment",
    "search": { "patient": "Patient/john-smith", "status": "booked" },
    "fields": [
      { "path": "participant.0.actor.display", "expected": "Dr. Rivera" }
    ]
  },
  "weight": 1.0,
  "axis": "correctness",
  "created_at": "2026-04-20T17:04:12.000Z"
}

SDK Example

// Create a criterion (SDK uses camelCase)
const criterion = await verial.criteria.create({
  taskId: "task_abc123",
  label: "Appointment booked for correct patient",
  weight: 1.0,
  axis: "correctness",
  assertion: {
    assert: "fhir-resource-state",
    resource_type: "Appointment",
    search: { patient: "Patient/john-smith", status: "booked" },
    fields: [
      { path: "participant.0.actor.display", expected: "Dr. Rivera" },
    ],
  },
});

// List criteria for a task
const criteria = await verial.criteria.list({ taskId: "task_abc123" });

// Update weight
await verial.criteria.update({ id: criterion.id, weight: 2.0 });

// Delete
await verial.criteria.delete({ id: criterion.id });

The assertion object is passed through unchanged by the API. Keys inside assertion are not snake-to-camel transformed. Use the exact field names documented above (for example resource_type, correlate_by, parse_json, not_contains).

Getting Started

Resources

Sandbox Protocols

Endpoints

Criterion Object

Assertion Specs

HTTP Example

SDK Example

​Endpoints

​Criterion Object

​Assertion Specs

​HTTP Example

​SDK Example

Endpoints

Criterion Object

Assertion Specs

HTTP Example

SDK Example