patient-view, order-select, medication-prescribe, etc.), and record user feedback on the cards that come back. Use it to benchmark agents that either consume CDS cards surfaced in a provider workflow or produce cards as a CDS service reacting to EHR activity.
How your agent connects
The CDS sandbox exposes a CDS Hooks-style surface on the direct-service path. Register CDS services via discovery, invoke hooks, and submit feedback; all traffic is recorded as interaction evidence and correlates with the sandbox’s FHIR resources.CDS Hooks endpoints are not currently mounted on the run-scoped
/v1/benchmark-runs/{id}/cds/* surface. During a benchmark run, resolve the
CDS sandbox from the sandboxes list on the run response and call the direct
/sandboxes/{id}/cds/* endpoints with an organization API key.Endpoints
| Method | Endpoint | Description |
|---|---|---|
POST | /sandboxes/{sandbox_id}/cds/services | Register CDS services by pointing to a discovery URL |
GET | /sandboxes/{sandbox_id}/cds/services | List CDS services currently registered with the sandbox |
POST | /sandboxes/{sandbox_id}/cds/hooks/{hook} | Fire a hook (e.g. patient-view, order-select) with a simulated context; returns the cards produced |
POST | /sandboxes/{sandbox_id}/cds/feedback | Record user feedback on a CDS card (accepted, overridden, ignored) |
Driving the rollout
Register CDS services by pointing at a discovery URL (CDS Hooks publishes a/cds-services endpoint per vendor):
Configuration
At create time you configure the hooks the sandbox will advertise and, optionally, canned cards the sandbox will return when a hook is invoked.Verification
There is no dedicated CDS Hooks check type in the current criterion catalog. CDS evidence is scored in two ways:- Interaction evidence. Every service registration, hook invocation, and feedback submission is logged as an interaction. The request context, returned cards, and feedback outcomes are attached to criterion-run evidence.
- Cross-simulator criteria. When a CDS card leads to a change in FHIR (for example the agent accepts a
create-medicationsuggestion), usefhir-resource-stateon the resulting resource to score the outcome. When it leads to a portal action, useportal-state-match.
Next Steps
FHIR
CDS context objects reference FHIR resources; pair this simulator with a FHIR simulator.
CDS API reference
Direct-access CDS Hooks endpoints for authoring.
Criteria
The assertions the verification engine currently supports.
Simulators overview
Lifecycle, provisioning, and configuration patterns.