AI for Biometrics
IRT / RTSM User Acceptance Testing
A visual, runnable guide, from the biostatistics lens
Watch / explore / read / build, pick your mode. This page shows the UAT (diagrams + a live dashboard computed from the data), the runnable R Shiny app lets you run one, the build-it walkthrough shows how the app is made, and the Excel workbook is what you file. Operations-only, synthetic, no PHI. We say participant, not subject.
What a UAT is, in one picture
You are confirming, in a test environment, that the vendor’s randomization and drug-supply build does exactly what your specs say, before a single real participant is randomized.
Where it actually runs, the systems
The tracker you see below is a picture. The real UAT happens across three live places, and the workbook is the paper trail that ties them together.
The workbook does not connect to the IRT. You transcribe results into it, on purpose, so the record is attributable and stands on its own for an auditor. Every number comes from the validated systems (the IRT, the test list, your reconciliation), never from an assistant.
The live tracker, computed from the data
This is the same logic the Shiny app and the Excel gate use, rendered here from the three data files: test_scripts.csv, defect_log.csv, and traceability_matrix.csv (in the templates/ folder). Edit those CSVs (or run the app) and these move.
29
Test scripts
22
Passed
2
Failed
3
Open defects
97%
Requirements traced
NOT ready - blocked
✘2 critical test(s) not yet passing
✘2 open critical/major defect(s)
✘1 requirement(s) without a test
The gate packages the evidence; a named human signs. It does not grant approval.
The gate is deliberately conjunctive: all three conditions must hold. In the seed data it reads blocked because a critical test is still failing, two critical/major defects are open, and one requirement has no test yet. Fix those in the app and watch it flip.
| Defect | Severity | In test | Description |
|---|---|---|---|
| DEF-01 | Critical | RAND-04 | Duplicate randomization number under concurrent site requests (race condition on the number sequence). |
| DEF-03 | Major | SUP-03 | Temperature-excursion kit remained dispensable; quarantine hold not enforced. |
| DEF-04 | Minor | NOT-01 | Enrollment-notification email omits site number in the subject line (cosmetic). |
Run it, and change it
The interactive tool is a real R Shiny app you own. From the package root:
install.packages(c("shiny", "bslib", "DT", "ggplot2")) # once
shiny::runApp("uat_irt/shiny")Select a test, mark it Pass/Fail, close a defect, and the KPIs and the go-live gate recompute live. Download your results to CSV to drop into the Excel workbook and the eTMF.
You want to build these yourself. shiny/R/logic.R holds the gate and KPI math (the functions this page also called), and shiny/app.R is the UI and reactivity. The build-it walkthrough takes you through the complete source, line by line. Change the CSV to your study’s tests and it is your tracker.
How to write a test that is actually executable
A good test has a specific, verifiable expected result, not “works correctly”. Two worked examples:
- Precondition: test list loaded in staging; the <70 kg stratum, block 1, has an unused slot; you know the next expected list entry.
- Steps: log in as Investigator, register participant 101-001 at weight 68.0 kg, Randomize, record the kit and confirmation, open the allocation report as the unblinded statistician.
- Expected: 101-001 is allocated to the next unused <70 kg / block-1 entry; arm, sequence, and kit match the list; the investigator sees a kit ID only, not the arm.
- Verify: put the test list next to the IRT output and confirm cell-for-cell across every stratum and both arms of the ratio, not one happy-path participant.
- Precondition: a randomized test participant exists; you hold the unblinding-authorized role.
- Steps: open the participant, invoke emergency unblind, enter a reason, confirm, check the audit trail, then log in as a blinded role and confirm the arm is still hidden.
- Expected: arm revealed to the authorized role only; unblinding written to the audit trail (who / when / why); participant flagged; blinded role still cannot see the arm; any required safety notification fires.
Defect severity drives the gate
| Severity | Meaning | Blocks go-live? |
|---|---|---|
| Critical | Could compromise safety, break the blind, mis-randomize, mis-dose, or corrupt integrity | Yes |
| Major | Significant functional error in a key workflow (resupply, wrong kit, missing safety alert) | Yes, until fixed |
| Minor | Cosmetic / non-functional (a label typo, a missing site number in a subject line) | No; fix or waive with rationale |
The other formats
| Mode | Artifact |
|---|---|
| Explore / run | Run & build the Shiny app, then run it locally |
| Share (no R) | Dashboard, the tracker as a self-contained page |
| Present | Slides |
| Skim | One-page cheat-sheet (PDF) |
| File | Excel workbook, the signed record for the eTMF |
| Build | Build-it walkthrough, the complete R source |
Synthetic data throughout. UAT sign-off is a named-human decision.