Skip to main content

​
User Prompt

“Set up an environment with a clearinghouse and test whether my agent can verify patient eligibility.”

​
Tool Call Sequence

1

Create clearinghouse simulator

Call: simulators
{
  "action": "create",
  "type": "Clearinghouse",
  "name": "Availity Clearinghouse"
}
Response:
{
  "id": "sim_ch_01",
  "type": "Clearinghouse",
  "name": "Availity Clearinghouse",
  "config": null,
  "createdAt": "2026-04-06T12:00:00Z"
}
What the agent learns: The clearinghouse simulator is created. It will accept 270/271 eligibility transactions once provisioned in a playground.
2

Create environment and link simulator

Call 1: environments
{
  "action": "create",
  "name": "Eligibility Testing",
  "description": "Clearinghouse environment for verifying patient insurance eligibility"
}
Response:
{
  "id": "env_elig_01",
  "name": "Eligibility Testing",
  "description": "Clearinghouse environment for verifying patient insurance eligibility",
  "simulators": [],
  "createdAt": "2026-04-06T12:00:01Z"
}
Call 2: environments
{
  "action": "addSimulator",
  "id": "env_elig_01",
  "simulatorId": "sim_ch_01"
}
Response:
{
  "id": "env_elig_01",
  "name": "Eligibility Testing",
  "simulators": [
    { "id": "sim_ch_01", "type": "Clearinghouse", "name": "Availity Clearinghouse" }
  ]
}
What the agent learns: The environment is ready with the clearinghouse attached.
3

Create dataset with patient data

The agent creates a dataset containing a FHIR patient bundle with insurance coverage information.Call: datasets
{
  "action": "create",
  "name": "Eligibility Test Patient",
  "format": "fhir",
  "data": {
    "resourceType": "Bundle",
    "type": "collection",
    "entry": [
      {
        "resource": {
          "resourceType": "Patient",
          "id": "pat-001",
          "name": [{ "family": "Chen", "given": ["Maria"] }],
          "birthDate": "1985-03-14",
          "gender": "female",
          "address": [{ "state": "CA", "postalCode": "90210" }]
        }
      },
      {
        "resource": {
          "resourceType": "Coverage",
          "id": "cov-001",
          "status": "active",
          "beneficiary": { "reference": "Patient/pat-001" },
          "payor": [{ "display": "BlueCross BlueShield" }],
          "subscriberId": "BCB-998877",
          "class": [
            {
              "type": { "coding": [{ "code": "plan" }] },
              "value": "PPO Gold"
            }
          ],
          "period": {
            "start": "2026-01-01",
            "end": "2026-12-31"
          }
        }
      }
    ]
  }
}
Response:
{
  "id": "ds_elig_01",
  "name": "Eligibility Test Patient",
  "format": "fhir",
  "resourceCount": 2,
  "createdAt": "2026-04-06T12:00:02Z"
}
What the agent learns: The dataset includes a patient (Maria Chen) with active BlueCross PPO Gold coverage. This data will be loaded into the sandbox so the clearinghouse returns realistic eligibility responses.
4

Create playground and load data

The agent provisions the environment, finds the sandbox, and loads the patient dataset.Call 1: playgrounds
{
  "action": "create",
  "environmentId": "env_elig_01"
}
Response:
{
  "id": "pg_elig_01",
  "environmentId": "env_elig_01",
  "status": "Active",
  "sandboxes": [
    {
      "id": "sbx_ch_01",
      "simulatorId": "sim_ch_01",
      "type": "Clearinghouse",
      "status": "Active",
      "endpoint": "https://ch.verial.dev/sbx_ch_01"
    }
  ],
  "createdAt": "2026-04-06T12:00:03Z"
}
Call 2: sandboxes
{
  "action": "list",
  "playgroundId": "pg_elig_01"
}
Response:
{
  "items": [
    {
      "id": "sbx_ch_01",
      "simulatorId": "sim_ch_01",
      "type": "Clearinghouse",
      "status": "Active",
      "endpoint": "https://ch.verial.dev/sbx_ch_01",
      "datasets": []
    }
  ]
}
Call 3: sandboxes
{
  "action": "addDataset",
  "id": "sbx_ch_01",
  "datasetId": "ds_elig_01"
}
Response:
{
  "id": "sbx_ch_01",
  "datasets": [
    { "id": "ds_elig_01", "name": "Eligibility Test Patient" }
  ]
}
What the agent learns: The playground is live and the clearinghouse sandbox now has the patient data loaded. When the agent-under-test queries eligibility for Maria Chen (subscriber BCB-998877), the clearinghouse will return active PPO Gold coverage.
5

Create benchmark with eligibility eval

The agent defines the benchmark, task, and eval in sequence.Call 1: benchmarks
{
  "action": "create",
  "name": "Eligibility Verification",
  "environmentId": "env_elig_01",
  "timeout": 120
}
Response:
{
  "id": "bm_elig_01",
  "name": "Eligibility Verification",
  "environmentId": "env_elig_01",
  "timeout": 120,
  "tasks": [],
  "createdAt": "2026-04-06T12:00:04Z"
}
Call 2: tasks
{
  "action": "create",
  "benchmarkId": "bm_elig_01",
  "name": "Verify patient eligibility",
  "instruction": "Check insurance eligibility for patient Maria Chen (DOB 1985-03-14, subscriber ID BCB-998877) through the clearinghouse. Confirm whether coverage is active and retrieve the plan details."
}
Response:
{
  "id": "task_elig_01",
  "benchmarkId": "bm_elig_01",
  "name": "Verify patient eligibility",
  "instruction": "Check insurance eligibility for patient Maria Chen (DOB 1985-03-14, subscriber ID BCB-998877) through the clearinghouse. Confirm whether coverage is active and retrieve the plan details.",
  "evals": [],
  "createdAt": "2026-04-06T12:00:05Z"
}
Call 3: evals
{
  "action": "create",
  "taskId": "task_elig_01",
  "name": "eligibility-verified",
  "assert": "Eligibility was verified and returned active coverage for the patient",
  "weight": 1.0
}
Response:
{
  "id": "eval_elig_01",
  "taskId": "task_elig_01",
  "name": "eligibility-verified",
  "assert": "Eligibility was verified and returned active coverage for the patient",
  "weight": 1.0,
  "createdAt": "2026-04-06T12:00:06Z"
}
What the agent learns: The benchmark is fully defined with one task and one eval. The eval checks that the agent-under-test successfully verified eligibility and received an active coverage response.

​
Agent Response

Your eligibility check environment and benchmark are ready:
ResourceIDDetail
Environmentenv_elig_01Clearinghouse simulator attached
Datasetds_elig_01Maria Chen, BCB PPO Gold, subscriber BCB-998877
Playgroundpg_elig_01Active at https://ch.verial.dev/sbx_ch_01
Benchmarkbm_elig_011 task, 1 eval, 120s timeout
Point your agent at the clearinghouse endpoint and run the benchmark with benchmark_runs create to start testing.

​
Summary

MetricValue
Tool calls9
Tools usedsimulators, environments, datasets, playgrounds, sandboxes, benchmarks, tasks, evals
Key techniqueLoad patient data into the sandbox before running benchmarks so the clearinghouse returns realistic eligibility responses. The create-then-link pattern (simulator, environment, addSimulator, playground, addDataset) builds up the simulation layer by layer.