Skip to main content
A Criterion is a typed assertion attached to a Task. After a task rollout completes, the verification engine evaluates each criterion against the recorded environment state (FHIR resources, HL7 messages, SFTP uploads, portal state, voice transcripts, X12 responses) and produces a Criterion Run with a pass or fail result, a numeric score, and evidence. Criteria replace the legacy Evals resource. Both entities are stored in the same underlying table, but criteria use a structured assertion object keyed by check type rather than a single natural language assert string.

​
Endpoints

MethodEndpointDescription
GET/criteria?task_id={task_id}List criteria for a task
POST/criteriaCreate a criterion
GET/criteria/{id}Get criterion details
PATCH/criteria/{id}Update a criterion
DELETE/criteria/{id}Delete a criterion

​
Criterion Object

FieldTypeDescription
idstringUnique identifier
task_idstringParent Task
input_entity_idstring | nullOptional DatasetEntity the criterion is scoped to
labelstringShort human-readable label
assertionobjectTyped assertion spec. Shape depends on assertion.assert (see below)
weightnumberRelative weight for task scoring. Higher means more important
axisstring | nullOptional scoring axis for per-axis breakdown
created_atdatetimeCreation timestamp

​
Assertion Specs

The assertion field is a discriminated union on the assert key. The verification engine dispatches to a check implementation based on this value.
Asserts that, after the rollout, a FHIR resource search returns a resource whose fields match expected values.
FieldTypeDescription
assert"fhir-resource-state"Discriminator
resource_typestringFHIR resource type, for example "Appointment"
searchobjectFHIR search params as key or value strings. Default {}
fieldsarrayField assertions. Each has path (dotted path into the resource) and expected (string, number, boolean, or null)
{
  "assert": "fhir-resource-state",
  "resource_type": "Appointment",
  "search": { "patient": "Patient/john-smith", "status": "booked" },
  "fields": [
    { "path": "participant.0.actor.display", "expected": "Dr. Rivera" },
    { "path": "serviceType.0.coding.0.code", "expected": "follow-up" }
  ]
}
Asserts field values on HL7v2 outbound messages captured by the sandbox.
FieldTypeDescription
assert"hl7-structural"Discriminator
correlate_byobjectKey or value map used to select the matching HL7 message. Default {}
fieldsarrayEach entry has path (HL7 segment-field path, e.g. PID.5.1) and expected (string)
{
  "assert": "hl7-structural",
  "correlate_by": { "MSH.9.1": "ADT" },
  "fields": [
    { "path": "PID.5.1", "expected": "Smith" },
    { "path": "PV1.2", "expected": "I" }
  ]
}
Asserts that after a web portal simulation ends, a given resource row has the expected fields.
FieldTypeDescription
assert"portal-state-match"Discriminator
portalstring | nullOptional portal slug
correlate_byobject{ resource, key }: which sandbox-state resource to load, and which key identifies the row
assertionsarrayEach entry has path (dotted JSON path) and expected (any JSON value)
{
  "assert": "portal-state-match",
  "correlate_by": { "resource": "prior_auth_requests", "key": "request_id" },
  "assertions": [
    { "path": "status", "expected": "submitted" },
    { "path": "cpt_code", "expected": "72148" }
  ]
}
Asserts that a file was uploaded to the sandbox SFTP endpoint, optionally parsing JSON contents and asserting fields.
FieldTypeDescription
assert"sftp-file-present"Discriminator
pathstring | nullExact path
path_patternstring | nullGlob pattern with * wildcard. Use instead of path for multiple candidate files
parse_jsonbooleanParse the file as JSON before checking fields. Default false
fieldsarrayEach entry has key or path and expected. Use key for sidecar-style keys, path for dotted JSON paths
{
  "assert": "sftp-file-present",
  "path_pattern": "outbound/claims/*.json",
  "parse_json": true,
  "fields": [
    { "path": "claim.total_charges", "expected": 420.00 },
    { "path": "claim.status", "expected": "submitted" }
  ]
}
Asserts the content of a voice call transcript. This is the check that uses LLM-assisted matching for loose phrase presence.
FieldTypeDescription
assert"voice-transcript"Discriminator
speaker"agent" | "caller" | "ivr" | "any"Which speaker’s turns to match against. Default "any"
containsstring[]Phrases that must appear in the transcript
not_containsstring[]Phrases that must not appear
{
  "assert": "voice-transcript",
  "speaker": "agent",
  "contains": ["member ID", "date of birth"],
  "not_contains": ["social security number"]
}
Asserts field values on an X12 EDI response (270/271/276/277/278) produced by the sandbox clearinghouse.
FieldTypeDescription
assert"x12-response"Discriminator
transaction"270" | "271" | "276" | "277" | "278" | nullOptional transaction set filter
correlate_byobjectKey or value map to pick the matching response
fieldsarrayEach has path and expected (string, number, boolean, or null)
{
  "assert": "x12-response",
  "transaction": "271",
  "fields": [
    { "path": "EB.1", "expected": "1" },
    { "path": "EB.3", "expected": "30" }
  ]
}

​
HTTP Example

# Create a criterion
curl -X POST https://api.verial.ai/criteria \
  -H "Authorization: Bearer $VERIAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "task_id": "task_abc123",
    "label": "Appointment booked for correct patient",
    "weight": 1.0,
    "axis": "correctness",
    "assertion": {
      "assert": "fhir-resource-state",
      "resource_type": "Appointment",
      "search": { "patient": "Patient/john-smith", "status": "booked" },
      "fields": [
        { "path": "participant.0.actor.display", "expected": "Dr. Rivera" }
      ]
    }
  }'
Response: 
{
  "id": "crit_01H...",
  "task_id": "task_abc123",
  "input_entity_id": null,
  "label": "Appointment booked for correct patient",
  "assertion": {
    "assert": "fhir-resource-state",
    "resource_type": "Appointment",
    "search": { "patient": "Patient/john-smith", "status": "booked" },
    "fields": [
      { "path": "participant.0.actor.display", "expected": "Dr. Rivera" }
    ]
  },
  "weight": 1.0,
  "axis": "correctness",
  "created_at": "2026-04-20T17:04:12.000Z"
}

​
SDK Example

// Create a criterion (SDK uses camelCase)
const criterion = await verial.criteria.create({
  taskId: "task_abc123",
  label: "Appointment booked for correct patient",
  weight: 1.0,
  axis: "correctness",
  assertion: {
    assert: "fhir-resource-state",
    resource_type: "Appointment",
    search: { patient: "Patient/john-smith", status: "booked" },
    fields: [
      { path: "participant.0.actor.display", expected: "Dr. Rivera" },
    ],
  },
});

// List criteria for a task
const criteria = await verial.criteria.list({ taskId: "task_abc123" });

// Update weight
await verial.criteria.update({ id: criterion.id, weight: 2.0 });

// Delete
await verial.criteria.delete({ id: criterion.id });
The assertion object is passed through unchanged by the API. Keys inside assertion are not snake-to-camel transformed. Use the exact field names documented above (for example resource_type, correlate_by, parse_json, not_contains).