Tested runs
Every run on this page used synthetic records on open-source software or our own test apps, on our machines. Each row says what was run, when, the result in plain words, and how the result was checked.
Updated 2026-10-18
The runs
| System | Task | Result | How it was checked | Date |
|---|---|---|---|---|
| MockMed, our synthetic clinic app | The quickstart with --break-it: one entry, then one with the save rejected | Done and checked; with the save rejected, it stopped and reported Check the record | Read of the saved note | 18 Oct 2026 |
| OpenEMR 8.0.0.3 | Referral intake, timed against an AI agent (Claude Sonnet 5.5, claude-sonnet-5-5), 63 synthetic referrals each | Both saved 62 of 63; median 29.7 s (OpenAdapt), 36.5 s (agent, about $0.08 a referral) | Database read of each chart | 14 to 15 Oct 2026 |
| OpenEMR 8.0.0.3 | The same referrals, replayed from a recording without page-structure identity | 62 of 63 saved; median 38.6 s, slower than the agent | Database read of each chart | 14 to 15 Oct 2026 |
| openIMIS 25.10 | Claims intake, 6 synthetic claims | 5 saved; 1 stopped before saving | Test setup read-back, 15 of 15 fields | 12 Oct 2026 |
| OpenEMR 8.0.0.3 | Referral intake, 8 synthetic referrals | 7 saved; 1 stopped (Check the record) | Test setup read-back, 12 of 12 fields | 11 Oct 2026 |
| OpenEMR 8.0.0.3 | New patient | 3 of 3 Done and checked | API and database read | Jul 2026 |
| OpenEMR, local | New patient, engine and direct-API comparison | 12 of 12 correct with 0 model calls | API read, database read, table changes | 16 Jul 2026 |
| OpenEMR public demo | Add a note | 19 of 20 passed | A screen check of the saved note | 8 Jul 2026 |
| openIMIS 25.10 | Eligibility look-ups | 3 of 3 active confirmed; 3 of 3 expired stopped | Read-only database query | Jul 2026 |
| openIMIS 25.10 | One claim per run | 3 of 3 checked | Database read | Jul 2026 |
| Frappe Lending 16.2.0 | Loan application | 6 of 6 checked; 5 of 5 faults stopped | API, database and table-change audit | 16 Jul 2026 |
| Our scheduling app over real remote desktop | Book an appointment | 3 of 3 confirmed; wrong-patient and changed-screen runs stopped | Read-only database query | 30 Jul 2026 |
| Our Windows test app | A form task | 3 of 3 confirmed | Database read | 17 Jul 2026 |
| MockMed, our synthetic clinic app | Triage note tutorial | 3 of 3 checked; 15 of 15 fault runs stopped | Read of the saved note | 26 Jul 2026 |
| A stand-in imitating Citrix | Normal and changed-screen runs | 3 normal; 3 changed-screen stops | Separate read | Lab only |
Not tested: a commercial EMR, claims or policy system; a real Citrix session; a US payer or carrier portal; real patient or member data. Process pages: Processes.
Sources
- OpenAdapt quickstart with --break-it (openadapt launcher 1.17.0), MockMed synthetic clinic app, 18 Oct 2026 (the run's reports): the normal entry ended Done and checked by a read of the saved note; with the save rejected on purpose, the screen looked the same as the good save, and the run stopped and reported Check the record without retrying.
- Referral timing run, local OpenEMR 8.0.0.3 test copy, 14 to 15 Oct 2026 (run data): 8 synthetic referrals, OpenAdapt (a program compiled to find each field by the page's structure, 0 model calls) against Claude Sonnet 5.5 (claude-sonnet-5-5) as a computer-use agent in a capped harness, warm starts, interleaved blocks, each chart read back from the database. Blocks 2 to 10, normal referrals saved correctly: 62 runs each; median 29.7 s (OpenAdapt) and 36.5 s (agent); agent median $0.083 a run at list price. Both 62 of 63 normal referrals correct; on the possible duplicate, no new chart in 9 of 9 for both. Block 1 left out (CPU shared with another process); with it (blocks 1 to 10), 69 of 70 correct for both, 29.7 s and 37.6 s (n=69 each). The same task from the 10 Oct recording without the page's structure: 62 of 63, 38.6 s median, slower than the agent.
- OpenAdapt on openIMIS 25.10 (open-source insurance system), local test copy, 12 Oct 2026. A program learned from one demonstration entered 6 synthetic health-facility claims in one batch, with 0 AI model calls. 5 were saved; our test setup read each back through SQL and the openIMIS API, and 15 of 15 fields matched the claim document. On the 6th, the insured number had two digits swapped: openIMIS found no member, the screen no longer matched the demonstration, and the program stopped before saving (no claim was written). The engine labels the 5 saved runs "Finished, not checked", because this program has no record check of its own yet.
- OpenAdapt on OpenEMR 8.0.0.3 (open-source EMR), local test copy, 11 Oct 2026. One program ran over 8 synthetic eReferrals in one batch, with 0 AI model calls. 7 new patients were saved; our test setup read each chart back through the OpenEMR API and database, and 12 of 12 fields matched the referral. The 8th (Rosa Delgado, same name as an existing chart, different birth date) stopped at the EMR's duplicate check; afterwards no new chart existed. The engine labels the saved runs "Finished, not checked" (practice profile) and the stop "Check the record".
- OpenAdapt on OpenEMR 8.0.0.3, July 2026: new-patient registration, 3 of 3 runs "Done and checked" through the OpenEMR API and a database read, 56.9 to 63.9 s each, synthetic patients.
- OpenAdapt engine benchmarks, OpenEMR public demo, 8 Jul 2026 (docs/BENCHMARKS.md): an 18-step add-a-note task on the public demo (synthetic patients, resets daily). The compiled program passed 19 of 20 runs at 39.2 s median with 0 model calls; in the 20th the note stayed in the unsaved form. A computer-use agent, Claude Sonnet 5 (claude-sonnet-5), passed 10 of 10 at 70.4 s median, about $0.55 a run at list price. Both judged by the same screen check of the saved note, not a database read. Engine version 0.1.0.
- OpenAdapt on openIMIS 25.10 (open-source insurance system, not a US payer portal), July 2026, synthetic members: 3 of 3 active policies confirmed, 3 of 3 expired policies stopped, 0 active policies stopped by mistake. Checked by a read-only database query of the policy, product and dates.
- OpenAdapt on openIMIS 25.10, July 2026: one health-facility claim per run, 3 of 3 checked by a direct database read (exactly one claim, status Entered, right member), 0 duplicates, 25.6 to 30.3 s a run, 0 AI model calls.
- OpenAdapt on Frappe Lending 16.2.0 (open source), 16 Jul 2026, synthetic applicants: 6 of 6 loan applications checked through the API, a database read and a table-change audit (3 normal runs, 3 after a cosmetic screen change); 5 of 5 injected faults stopped before saving.
- OpenAdapt remote-desktop test, 30 Jul 2026: a fictional clinic scheduling app over a real remote-desktop connection (FreeRDP, not Citrix), synthetic patients. 3 of 3 bookings confirmed by a read-only database query; 3 of 3 wrong-patient runs stopped with 0 rows written; 3 of 3 runs on a changed screen stopped. 0 AI model calls.
- OpenAdapt on our own Windows test app (UI Automation), 17 Jul 2026: 3 of 3 runs with saves confirmed by a database read; missing and ambiguous targets refused 3 of 3.
See it on your process
Bring one task your team types every day. We'll tell you whether it fits, and what a test on your own system would cover.