Eligibility Check

User Prompt

“Set up an environment with a clearinghouse and test whether my agent can verify patient eligibility.”

Tool Call Sequence

Create clearinghouse simulator

Call: simulators

{
  "action": "create",
  "type": "Clearinghouse",
  "name": "Availity Clearinghouse"
}

Response:

{
  "id": "sim_ch_01",
  "type": "Clearinghouse",
  "name": "Availity Clearinghouse",
  "config": null,
  "createdAt": "2026-04-06T12:00:00Z"
}

What the agent learns: The clearinghouse simulator is created. It will accept 270/271 eligibility transactions once provisioned in a playground.

Create environment and link simulator

Call 1: environments

{
  "action": "create",
  "name": "Eligibility Testing",
  "description": "Clearinghouse environment for verifying patient insurance eligibility"
}

Response:

{
  "id": "env_elig_01",
  "name": "Eligibility Testing",
  "description": "Clearinghouse environment for verifying patient insurance eligibility",
  "simulators": [],
  "createdAt": "2026-04-06T12:00:01Z"
}

Call 2: environments

{
  "action": "addSimulator",
  "id": "env_elig_01",
  "simulatorId": "sim_ch_01"
}

Response:

{
  "id": "env_elig_01",
  "name": "Eligibility Testing",
  "simulators": [
    { "id": "sim_ch_01", "type": "Clearinghouse", "name": "Availity Clearinghouse" }
  ]
}

What the agent learns: The environment is ready with the clearinghouse attached.

Create dataset with patient data

The agent creates a dataset containing a FHIR patient bundle with insurance coverage information.Call: datasets

{
  "action": "create",
  "name": "Eligibility Test Patient",
  "format": "fhir",
  "data": {
    "resourceType": "Bundle",
    "type": "collection",
    "entry": [
      {
        "resource": {
          "resourceType": "Patient",
          "id": "pat-001",
          "name": [{ "family": "Chen", "given": ["Maria"] }],
          "birthDate": "1985-03-14",
          "gender": "female",
          "address": [{ "state": "CA", "postalCode": "90210" }]
        }
      },
      {
        "resource": {
          "resourceType": "Coverage",
          "id": "cov-001",
          "status": "active",
          "beneficiary": { "reference": "Patient/pat-001" },
          "payor": [{ "display": "BlueCross BlueShield" }],
          "subscriberId": "BCB-998877",
          "class": [
            {
              "type": { "coding": [{ "code": "plan" }] },
              "value": "PPO Gold"
            }
          ],
          "period": {
            "start": "2026-01-01",
            "end": "2026-12-31"
          }
        }
      }
    ]
  }
}

Response:

{
  "id": "ds_elig_01",
  "name": "Eligibility Test Patient",
  "format": "fhir",
  "resourceCount": 2,
  "createdAt": "2026-04-06T12:00:02Z"
}

What the agent learns: The dataset includes a patient (Maria Chen) with active BlueCross PPO Gold coverage. This data will be loaded into the sandbox so the clearinghouse returns realistic eligibility responses.

Create playground and load data

The agent provisions the environment, finds the sandbox, and loads the patient dataset.Call 1: playgrounds

{
  "action": "create",
  "environmentId": "env_elig_01"
}

Response:

{
  "id": "pg_elig_01",
  "environmentId": "env_elig_01",
  "status": "Active",
  "sandboxes": [
    {
      "id": "sbx_ch_01",
      "simulatorId": "sim_ch_01",
      "type": "Clearinghouse",
      "status": "Active",
      "endpoint": "https://ch.verial.dev/sbx_ch_01"
    }
  ],
  "createdAt": "2026-04-06T12:00:03Z"
}

Call 2: sandboxes

{
  "action": "list",
  "playgroundId": "pg_elig_01"
}

Response:

{
  "items": [
    {
      "id": "sbx_ch_01",
      "simulatorId": "sim_ch_01",
      "type": "Clearinghouse",
      "status": "Active",
      "endpoint": "https://ch.verial.dev/sbx_ch_01",
      "datasets": []
    }
  ]
}

Call 3: sandboxes

{
  "action": "addDataset",
  "id": "sbx_ch_01",
  "datasetId": "ds_elig_01"
}

Response:

{
  "id": "sbx_ch_01",
  "datasets": [
    { "id": "ds_elig_01", "name": "Eligibility Test Patient" }
  ]
}

What the agent learns: The playground is live and the clearinghouse sandbox now has the patient data loaded. When the agent-under-test queries eligibility for Maria Chen (subscriber BCB-998877), the clearinghouse will return active PPO Gold coverage.

Create benchmark with eligibility eval

The agent defines the benchmark, task, and eval in sequence.Call 1: benchmarks

{
  "action": "create",
  "name": "Eligibility Verification",
  "environmentId": "env_elig_01",
  "timeout": 120
}

Response:

{
  "id": "bm_elig_01",
  "name": "Eligibility Verification",
  "environmentId": "env_elig_01",
  "timeout": 120,
  "tasks": [],
  "createdAt": "2026-04-06T12:00:04Z"
}

Call 2: tasks

{
  "action": "create",
  "benchmarkId": "bm_elig_01",
  "name": "Verify patient eligibility",
  "instruction": "Check insurance eligibility for patient Maria Chen (DOB 1985-03-14, subscriber ID BCB-998877) through the clearinghouse. Confirm whether coverage is active and retrieve the plan details."
}

Response:

{
  "id": "task_elig_01",
  "benchmarkId": "bm_elig_01",
  "name": "Verify patient eligibility",
  "instruction": "Check insurance eligibility for patient Maria Chen (DOB 1985-03-14, subscriber ID BCB-998877) through the clearinghouse. Confirm whether coverage is active and retrieve the plan details.",
  "criteria": [],
  "createdAt": "2026-04-06T12:00:05Z"
}

Call 3: criteria

{
  "action": "create",
  "taskId": "task_elig_01",
  "name": "eligibility-verified",
  "assertion": { "type": "x12-response", "spec": { "transaction": "271", "coverage_status": "active" } },
  "weight": 1.0
}

Response:

{
  "id": "crit_elig_01",
  "taskId": "task_elig_01",
  "name": "eligibility-verified",
  "weight": 1.0,
  "createdAt": "2026-04-06T12:00:06Z"
}

What the agent learns: The benchmark is fully defined with one task and one criterion. The criterion checks that the agent-under-test successfully verified eligibility and received an active coverage response.

Agent Response

Your eligibility check environment and benchmark are ready:
Resource ID Detail
Environment env_elig_01 Clearinghouse simulator attached
Dataset ds_elig_01 Maria Chen, BCB PPO Gold, subscriber BCB-998877
Playground pg_elig_01 Active at https://ch.verial.dev/sbx_ch_01
Benchmark bm_elig_01 1 task, 1 criterion, 120s timeout
Point your agent at the clearinghouse endpoint and run the benchmark with benchmark_runs create to start testing.

Resource	ID	Detail
Environment	`env_elig_01`	Clearinghouse simulator attached
Dataset	`ds_elig_01`	Maria Chen, BCB PPO Gold, subscriber BCB-998877
Playground	`pg_elig_01`	Active at `https://ch.verial.dev/sbx_ch_01`
Benchmark	`bm_elig_01`	1 task, 1 criterion, 120s timeout

Summary

Metric	Value
Tool calls	9
Tools used	`simulators`, `environments`, `datasets`, `playgrounds`, `sandboxes`, `benchmarks`, `tasks`, `criteria`
Key technique	Load patient data into the sandbox before running benchmarks so the clearinghouse returns realistic eligibility responses. The create-then-link pattern (simulator, environment, addSimulator, playground, addDataset) builds up the simulation layer by layer.

​User Prompt

​Tool Call Sequence

​Agent Response

​Summary

User Prompt

Tool Call Sequence

Agent Response

Summary