Protocol matrix
Each row’s verification check is documented alongside its peers on the Criteria page.
Lifecycle
- Create the simulator once, configure it (e.g. portal flavor, IVR flow, determination rules).
- Link it to one or more environments. Environments compose simulators plus datasets.
- Provision a sandbox when a playground is created or a benchmark run starts. The sandbox gets its own credentials, isolated state, and evidence stream.
- Drive the sandbox from your agent using the
/v1/benchmark-runs/{id}/endpoints (or the direct sandbox endpoints during authoring). - Tear down at the end of the run. The sandbox stops accepting traffic, but all recorded interactions remain for review and verification.
Creating and linking
Simulators are standalone resources that you link to environments. Configuration is set at create time and can be updated until the simulator is provisioned:Provisioning
When a playground or benchmark run starts, Verial walks the environment’s simulator list and provisions each one:- Allocate resources. FHIR stores are created in GCP Healthcare, phone numbers are leased from the voice pool, SFTP drops are initialized, portal logins are minted, CDS Hooks endpoints are registered.
- Load datasets. Any datasets linked to the simulator are seeded into the fresh sandbox (patients into FHIR, files into SFTP, determination rules into the portal).
- Return credentials. The benchmark-run response includes a
endpointsmap (for path-routed simulators) and, where applicable, each sandbox’scredentialspayload (phone numbers, portal URLs, usernames, fax numbers). - Record interactions. Every request to the sandbox is captured as evidence. Voice calls produce transcripts, fax submissions produce OCR’d documents, portal submissions produce UI events.
- Tear down. After the last task run completes, sandboxes release their resources. Interaction evidence persists so criteria can score the rollout.
Configuration vs runtime credentials
Simulator configuration and runtime credentials are distinct.- Configuration is set at
POST /simulatorstime (and editable viaPATCH /simulators/{id}until provisioning). It describes the shape of the simulated system: which payer portal to render, which IVR menus to expose, which determination rules to apply, which CDS cards to return. - Runtime credentials are generated when the sandbox is provisioned. They are unique per sandbox, scoped to that sandbox’s lifetime, and include things like a FHIR base URL, a leased phone number, a portal username and password, a fax number, or an SFTP root path. The agent reads them out of the benchmark-run response or the sandbox’s
credentialspayload.
Next Steps
FHIR
Provision a FHIR R4 EHR and drive it with SMART on FHIR.
Web Portal
Simulated RadMD and CoverMyMeds payer portals.
Voice
Phone / IVR lines with recorded transcripts.
Criteria
The typed assertions the verification engine runs against sandbox state.