Stage 5: QC and validation
Executed QC ledger: re-derivation by a second method
Synthetic data only: simulated for this worked example; not real trial data and not a regulatory deliverable.
Same-author re-derivation, not independent double programming. One person wrote the primary programs and these checks from the same SAP. The checks re-derive each result by a different method and compare with diffdf at the tolerances shown. They can catch coding slips and engine defaults; they cannot catch a misreading of the SAP that both routes share.
- Checks40
- Pass40
- Fail0
- Values compared5100
| ID | Item | Variable | Primary route | QC route | Compared | Tolerance | Mismatches | Max abs diff | Status |
|---|---|---|---|---|---|---|---|---|---|
| Q1 | NCA parameters per profile | CMAX | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-09 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | TMAX | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-09 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | TLST | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-09 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | CLST | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-09 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | AUCLST | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-06 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | AUCIFO | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-06 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | AUCPEO | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-06 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | LAMZHL | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-06 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | LAMZ | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-09 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | LAMZNPT | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | exact | 0 | 0 | pass |
| Q1 | NCA parameters per profile | R2ADJ | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-09 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | LAMZLL | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-09 | 0 | ≤ 1e-10 | pass |
| Q1 | NCA parameters per profile | LAMZUL | PKNCA 0.12.1 pk.nca() | NonCompart 0.8.4 tblNCA(), plus the zero-segment trapezoid in base R | 79 | 1e-09 | 0 | ≤ 1e-10 | pass |
| Q1b | NCA parameter rules: values flagged per profile | AUCIFO_EXCL | PKNCA exclude() (01_nca.R) | base-R rule on the NonCompart values | 79 | exact | 0 | pass | |
| Q1b | NCA parameter rules: values flagged per profile | LAMZ_EXCL | PKNCA exclude() (01_nca.R) | base-R rule on the NonCompart values | 79 | exact | 0 | pass | |
| Q2 | BE ratio, 90% CI and within-participant CV | ratio | lm() GLM (02_be.R) | two-sample t-test on half period differences (closed form) | 3 | 1e-10 | 0 | ≤ 1e-10 | pass |
| Q2 | BE ratio, 90% CI and within-participant CV | lower | lm() GLM (02_be.R) | two-sample t-test on half period differences (closed form) | 3 | 1e-10 | 0 | ≤ 1e-10 | pass |
| Q2 | BE ratio, 90% CI and within-participant CV | upper | lm() GLM (02_be.R) | two-sample t-test on half period differences (closed form) | 3 | 1e-10 | 0 | ≤ 1e-10 | pass |
| Q2 | BE ratio, 90% CI and within-participant CV | cv_within | lm() GLM (02_be.R) | two-sample t-test on half period differences (closed form) | 3 | 1e-10 | 0 | ≤ 1e-10 | pass |
| Q2 | BE ratio, 90% CI and within-participant CV | df | lm() GLM (02_be.R) | two-sample t-test on half period differences (closed form) | 3 | exact | 0 | 0 | pass |
| Q3 | BE ratio, 90% CI and within-participant CV | ratio | lm() GLM (02_be.R) | nlme::lme() REML, random participant | 3 | 1e-06 | 0 | ≤ 1e-10 | pass |
| Q3 | BE ratio, 90% CI and within-participant CV | lower | lm() GLM (02_be.R) | nlme::lme() REML, random participant | 3 | 1e-06 | 0 | 6.1e-10 | pass |
| Q3 | BE ratio, 90% CI and within-participant CV | upper | lm() GLM (02_be.R) | nlme::lme() REML, random participant | 3 | 1e-06 | 0 | 6.9e-10 | pass |
| Q3 | BE ratio, 90% CI and within-participant CV | cv_within | lm() GLM (02_be.R) | nlme::lme() REML, random participant | 3 | 1e-06 | 0 | 1.6e-07 | pass |
| Q3 | BE ratio, 90% CI and within-participant CV | df | lm() GLM (02_be.R) | nlme::lme() REML, random participant | 3 | exact | 0 | 0 | pass |
| Q4 | ADPC relative times | NFRLT | admiral 1.5.0 ad_adpc.R pattern | base-R difftime() on PCDTC and EXSTDTC | 1185 | exact | 0 | 0 | pass |
| Q4 | ADPC relative times | ARRLT | admiral 1.5.0 ad_adpc.R pattern | base-R difftime() on PCDTC and EXSTDTC | 1185 | 1e-06 | 0 | 3.3e-09 | pass |
| Q4 | ADPC relative times | AFRLT | admiral 1.5.0 ad_adpc.R pattern | base-R difftime() on PCDTC and EXSTDTC | 1185 | 1e-06 | 0 | 3.3e-08 | pass |
| Q5 | ADSL population flags | SAFFL | admiral ADSL plus 05_adpp.R rules | base R on DM, EX, PC, AE and PP | 40 | exact | 0 | pass | |
| Q5 | ADSL population flags | PKCFL | admiral ADSL plus 05_adpp.R rules | base R on DM, EX, PC, AE and PP | 40 | exact | 0 | pass | |
| Q5 | ADSL population flags | PKPFL | admiral ADSL plus 05_adpp.R rules | base R on DM, EX, PC, AE and PP | 40 | exact | 0 | pass | |
| Q5 | ADSL population flags | BEFL | admiral ADSL plus 05_adpp.R rules | base R on DM, EX, PC, AE and PP | 40 | exact | 0 | pass | |
| Q6 | Table 14.3.1.1 displayed counts | N | rtables/tern table cells | base-R count on AE and EX | 2 | exact | 0 | 0 | pass |
| Q6 | Table 14.3.1.1 displayed counts | any | rtables/tern table cells | base-R count on AE and EX | 2 | exact | 0 | 0 | pass |
| Q6 | Table 14.3.1.1 displayed counts | events | rtables/tern table cells | base-R count on AE and EX | 2 | exact | 0 | 0 | pass |
| Q6 | Table 14.3.1.1 displayed counts | related | rtables/tern table cells | base-R count on AE and EX | 2 | exact | 0 | 0 | pass |
| Q6 | Table 14.3.1.1 displayed counts | severe | rtables/tern table cells | base-R count on AE and EX | 2 | exact | 0 | 0 | pass |
| Q6 | Table 14.3.1.1 displayed counts | serious | rtables/tern table cells | base-R count on AE and EX | 2 | exact | 0 | 0 | pass |
| Q7 | Control: planted 0.1% change in one AUC(0-t); pass = exactly 1 mismatch found | AUCLST | PKNCA results | the same results with one value changed | 79 | 1e-06 | 1 | 1.4e+00 | pass |
| Q8 | Control: NonCompart AUC(0-t) without the zero-segment trapezoid; pass = mismatches in exactly the 1 profile(s) with a BLQ value between quantifiable samples | AUCLST | PKNCA results | NonCompart tblNCA() as it comes | 79 | 1e-06 | 1 | 6.9e+01 | pass |
How to read it
- Q1 runs a second NCA engine (NonCompart) on the same ADPC input with the same settings: linear-up/log-down AUC, BLQ as zero, at least 3 terminal points after tmax chosen by adjusted R-squared. The input includes 1 profile(s) with a BLQ value between quantifiable samples (scenario 6), so the BLQ rule is compared, not only assumed. NonCompart's log-down formula gives a segment that falls to zero no area; the SAP uses the linear trapezoid there, as PKNCA does, so the QC route adds that trapezoid in base R (1 segment(s)).
- Q1b applies the SAP parameter rules (adjusted R-squared at least 0.90; AUC(0-inf) at most 20% extrapolated) to the NonCompart values in base R and compares the flags with PKNCA
exclude(). - Q2 is the closed-form crossover analysis: half the within-participant period difference, compared between sequences with a two-sample t-test. For complete 2x2 data it gives the same estimate, interval and residual variance as the GLM.
- Q3 refits the model with a random participant effect (
nlme::lme, REML, tight convergence settings). With every participant contributing both periods it reproduces the GLM's estimate, interval and residual variance; the 1e-6 tolerance allows for the iterative fit. With the default settings the CV agreed only to about 5e-6 percentage points, so the settings were tightened rather than the tolerance widened. - Q4 checks the admiral-derived times, including
derive_var_nfrlt(), which admiral 1.5.0 labels experimental. - Q5 and Q6 rebuild the population flags and the adverse-event counts from the SDTM-like domains without admiral or tern.
- Q7 is a control: a 0.1% change planted in one value must be found exactly once, which shows the comparison can fail.
- Q8 is a control for the BLQ scenario: NonCompart's AUC(0-t) as it comes must differ from PKNCA in exactly the profiles with a BLQ value between quantifiable samples, which shows that the scenario reaches the comparison and that the adjustment in Q1 is confined to it.
Files: qc_ledger.csv, qc_ledger.json, qc_nca_side_by_side.csv (PKNCA and NonCompart values for every profile). Program: study/05_qc/qc.R.
What the QC found while this example was built
The first full run had 6 failures out of the 37 checks it had then, all resolved; the fixes are in the code, not in the tolerances:
- TLST, LAMZLL and LAMZUL differed by about 3e-9 h because ADPC was rounded to 10 significant digits after PKNCA had already run on the unrounded times. ADPC is now rounded when it is built, so both engines and the XPT file see the same times.
- The degrees of freedom were stored as integer in one route and as double in the other; the comparison now uses values, not storage types.
- With nlme's default convergence settings the REML residual variance matched the GLM only to about 5e-6 percentage points of CV; the refit now uses tight convergence settings and meets the pre-set 1e-6 tolerance.
A later review added a BLQ value between quantifiable samples (scenario 6) so that the ICH M13A BLQ setting is exercised. Q1 then failed for that profile's AUC(0-t), AUC(0-inf) and extrapolated percentage: NonCompart gives a segment that falls to zero no area, while the SAP and PKNCA use the linear trapezoid. The QC route now adds that trapezoid, and Q8 checks that the difference is confined to that profile. The same review found that the SAP's adjusted R-squared and extrapolation thresholds were passed to PKNCA as options, which do nothing on their own; they are now applied with exclude() and checked in Q1b.
Not covered
- No second programmer, no blinded re-implementation from the specification alone.
- The SDTM-like inputs are simulated by the same code base; there is no check against source documents or an EDC export.
- No Pinnacle 21 or CDISC CORE conformance run; the metadata checks in stage 3 are metatools and xportr checks against this study's own specification.