IRT / RTSM User Acceptance Testing

Confirming the build before the first participant is randomized

What a UAT is

You confirm, in a test environment, that the vendor’s randomization and drug-supply build does exactly what your specs say, before a single real participant is randomized.

URS + specs

Test cases

Execute in
staging IRT

Defects

Traceability

Go-live gate

Production

Operations-only, synthetic, no PHI. We say participant, not subject.

Where it runs: the systems

  • Staging IRT / RTSM (vendor): the web app you operate in a browser, plus the approved test list.
  • You, the biostatistician: execute each case, capture dated screenshot evidence, and transcribe results into your workbook.
  • Vendor defect tracker: log failures; the vendor promotes a fixed build back to staging for retest.
  • eTMF: the signed sign-off package is filed as the record.

The workbook does not connect to the IRT. You transcribe on purpose, so the record is attributable and stands alone for an auditor.

The live gate, computed now

NOT ready - blocked
✘ 2 critical test(s) not yet passing
✘ 2 open critical/major defect(s)
✘ 1 requirement(s) without a test

Conjunctive: all critical pass AND 0 open critical/major defects AND 100% traceability.

Snapshot

29
Test scripts
22
Passed
2
Failed
3
Open defects
97%
Traced

Every number here comes from the validated data, never from an assistant.

Execution by area

How to write an executable test

RAND-02, stratified allocation. A good test has a specific, verifiable expected result, not “works correctly”.

  • Precondition: test list loaded; the <70 kg stratum, block 1, has an unused slot.
  • Steps: register participant 101-001 at 68.0 kg, randomize, record kit + confirmation, open the allocation report as the unblinded statistician.
  • Expected: allocated to the next unused <70 kg / block-1 entry; arm, sequence, and kit match the list; the investigator sees a kit ID only, not the arm.
  • Verify: compare cell-for-cell across every stratum and both arms, not one happy-path participant.

Severity drives the gate

Severity Meaning Blocks go-live?
Critical Could break the blind, mis-randomize, mis-dose, or corrupt integrity Yes
Major Significant functional error in a key workflow (resupply, wrong kit, missing alert) Yes, until fixed
Minor Cosmetic / non-functional (label typo, missing site number) No; fix or waive with rationale

One open Critical or Major defect holds the gate. That is the whole point: severity, not count, decides.

Run it, and build it

The interactive tool is a real R Shiny app you own:

shiny::runApp("uat_irt/shiny")
# needs: install.packages(c("shiny","bslib","DT","ggplot2"))

Mark a test Pass/Fail, close a defect, and the KPIs and go-live gate recompute live. The gate math lives in shiny/R/logic.R, shared by the app, these slides, and the unit tests. Change the CSV to your study’s tests and it is your tracker.

Pick your mode

Synthetic data throughout. UAT sign-off is a named-human decision.