Hybrid AI — Trial-Operations Operating Wiki

An agentic project-management, trial-management, and monitoring layer built on the hybrid Claude + local stack: 75 automations across the study lifecycle — recurrent data-validity QC, SOP/WI/study-document ingestion, eTMF filing, Smartsheet timeline automation, and risk-based monitoring — each routed by data sensitivity, tiered by risk, and gated by a human.

The 60-second version

1. One gateway fronts a local model (sensitive data), the Claude API (de-identified reasoning), and a RAG store of your SOPs/WIs/study docs.
2. An orchestrator fires scheduled and event-driven agents that read your systems (eTMF, Smartsheet, EDC, Pinnacle 21) via connectors.
3. Each agent proposes a change into a human-approval queue, with a full 21 CFR Part 11 audit trail.
4. On approval, the change is written to the system of record. Nothing regulated moves without a human.

It is the same hybrid architecture from the AI strategy — here, pointed at operations.

The platform — one gateway, many agents

Every agent calls one OpenAI-compatible endpoint. The gateway routes by data classification, enforces guardrails, and logs everything; connectors expose your systems as tools.

Platform components

Project & Timeline Management

Milestone & critical-path tracking, Smartsheet timeline automation, status reporting, action/decision capture, vendor oversight. Smartsheet's engine computes the plan; the AI explains, prioritizes, and proposes — the PM approves before any committed-date write.

eTMF & Document Control

Auto-classify & file to the DIA TMF Reference Model, completeness/expiry checks, and SOP/WI/study-doc ingestion into the RAG store for grounded compliance Q&A. Filing to the record-of-truth is human-approved; RAG answers are draft/advisory and always cited.

Data Validity & Recurrent QC

Trial Monitoring & RBQM

ICH E6(R3) risk-based quality management: KRI/QTL breach signals, central/statistical monitoring, deviation surveillance, safety triage, DSMB packet assembly. Unblinded/participant-level signals run on the local model; reported safety numbers come from validated tools with medical review.

Additional / easily-missed

The easily-missed operations the completeness audit surfaced — consent version tracking, IP/drug accountability, randomization/IXRS oversight, regulatory-submission & safety-clock tracking, training compliance, data-transfer scheduling, and more.

Governance & routing

The tier model

The three hard rules

1. PHI / unblinded / randomization data never leaves — always the local frozen model (zero egress). De-identified text may use cloud Claude.
2. Every reported number and record-of-truth comes from a validated tool (Pinnacle 21, Phoenix, the EDC/CTMS/eTMF), never an LLM.
3. A human approves any change to a regulated record or timeline — via the approval queue, with a Part 11 audit trail.

How it's enforced

• Data-classification router + Presidio firewall at the gateway — PHI is blocked from the cloud egress by policy, not by trust.
• Engine-of-record registry + model-freeze — the authoritative list of which validated engine / frozen model is in production; LLMs explicitly excluded from producing reported numbers.
• Unified 21 CFR Part 11 audit trail — every gateway call and connector action: who/which agent, model + backend, prompt + classification, sources, proposal, approver, timestamps (ALCOA++).
• Least-privilege connectors — read-mostly; writes to a record-of-truth are gated through the human queue.

Build roadmap — start small, earn trust

Stand it up in stages; each new agent goes live behind the human-approval queue and graduates as validation and trust accrue.

Phase 0 — the stack (from the hybrid AI strategy)

• Gateway (LiteLLM) + local vLLM model + Claude API under BAA + Presidio guardrail.
• The Part 11 audit trail and the human-approval queue.

Phase 1 — the RAG knowledge base (highest value, lowest risk)

• Ingest approved SOPs / WIs / protocols / plans into the local vector store.
• Ship the SOP/compliance Q&A agent first — pure GREEN, cited answers, immediate daily value.

Phase 2 — read-only connectors + scheduled agents

• Connect Smartsheet, the eTMF, and Pinnacle 21 outputs as read tools.
• Turn on the nightly QC-triage and weekly status-report agents (AMBER — everything to the approval queue).

Phase 3 — event-driven automation & monitoring

• Timeline-cascade on data cut / lock, eTMF filing proposals, action-item capture.
• KRI/QTL breach signals and central-monitoring agents.
• Gated writes graduate to lighter review as each agent earns a track record.

Agent & prompt patterns

The standard agent loop

trigger (schedule / event)
  -> gather context: connectors (read) + RAG (cited)
  -> classify data sensitivity  ->  route to LOCAL or CLOUD via gateway
  -> reason: draft / classify / triage / summarize
  -> PROPOSE change to the human-approval queue (with rationale + sources)
  -> on human approval: write via connector to the system of record
  -> log everything to the Part 11 audit trail

QC finding-triage prompt (local model)

You are triaging CDISC conformance findings. Pinnacle 21 has ALREADY produced the authoritative Error/Warning/Notice list (you NEVER change a verdict). Cluster findings by rule and domain; diff against the last clean run and surface only NEW findings; for each, draft a plain-language explanation and a proposed disposition (real defect vs expected/explained-in-cSDRG-ADRG, grounded in the reviewer guides), and name the programmer to route it to. Output a table; flag low confidence.

SOP / compliance Q&A prompt (RAG)

Answer ONLY from the approved SOP/WI knowledge base. Quote the exact clause and cite the document ID + version. If the SOPs do not cover it, say so. Never infer a regulatory requirement that isn't written.

Weekly status-report prompt

Draft the weekly study status report in the house template from: the Smartsheet plan-of-record (milestones, % complete, slips), open risks/issues, action-item aging, and vendor deliverables. Pull numbers from the systems of record; invent nothing. Mark any reported metric for PM verification. Keep it to one page.

Worked example — a day in the ops layer

02:00 — nightly QC triage

A new ADaM build lands. Pinnacle 21 runs; the local agent diffs against the last clean run, finds 3 genuinely new findings among 240 (the rest pre-explained from the ADRG), drafts dispositions, and routes them to two programmers' worklists. The CDISC lead opens a clean, pre-triaged queue at 08:00 instead of a 4-hour manual slog.

10:30 — data cut declared

The data-cut event fires. The agent recomputes the Smartsheet critical path, sees the dry-run TLF date now drives the finish, drafts targeted notifications (“you are now blocked”), and posts the recomputed plan to the approval queue. The PM approves; Smartsheet and the stakeholders update in minutes.

13:00 — a KRI breaches

Central monitoring flags a site whose query rate crossed its QTL. The local agent assembles the signal context (de-identified), drafts a follow-up for the CRA, and logs it to the RBQM register — medical review owns the call.

15:00 — eTMF filing + an SOP question

Three monitoring reports arrive; the agent proposes TMF Reference Model zones/artifacts and flags one as a probable duplicate — the doc controller approves the filing. Meanwhile a programmer asks the wiki agent “what does our SOP require for a derived-variable spec?” and gets the exact clause, cited.

16:00 — the weekly status report

The Friday agent drafts the status report from the plan-of-record and open risks; the PM edits two lines and sends it. Every regulated number came from a validated system; no patient data left the building; and a human approved every committed change.

FAQ

Does the AI ever change a record on its own? No. Every change to a regulated record, timeline, or reported output is proposed to the human-approval queue. The LLM is out of the decision path for anything regulated (RED).

Where do the numbers come from? Always a validated tool — Pinnacle 21 (conformance), Phoenix WinNonlin (PK), the EDC/CTMS/eTMF of record. The AI triages and explains; it never produces the authoritative number.

What about patient data? PHI, unblinded, and participant-level data are processed only on the local, air-gapped model. The Presidio guardrail blocks PHI from the cloud egress by policy. De-identified text may use Claude.

How is this different from the Copilot wiki? The Copilot wiki is the M365-native, ops-memory layer for email. This is the full agentic operations platform on the hybrid stack — broader scope, deeper integration (eTMF, Smartsheet, Pinnacle 21, EDC), and validated-tool + human-gated governance throughout.

Is it inspection-defensible? That is the design intent: validated engines of record, model-freeze, a unified Part 11 audit trail, least-privilege connectors, and a human approving every regulated change.

Walkthrough transcript

The complete narration of the walkthrough video, as readable text.

1. Part 1

This walks through a hybrid AI trial-operations layer — an agentic system, built on the Claude-plus-local stack, that handles the project management, trial management, and monitoring work across a study. Seventy-five automations: recurrent QC checks, document and SOP ingestion, eTMF filing, Smartsheet timeline updates, and risk-based monitoring. And one principle holds throughout: the AI drafts, triages, and routes; validated tools and humans own every regulated number and record.

2. Part 2

Trial operations are fragmented. The knowledge and the work are spread across the EDC, the eTMF, Smartsheet, Pinnacle 21, email, and a dozen SOPs — and the team spends its time on manual QC triage, document filing, status decks, and chasing action items. It is exactly the kind of repetitive, cross-system busywork that an AI layer can absorb — if, and only if, it is governed properly.

3. Part 3

So we point the hybrid stack at operations. Seventy-five governed agents, each one routed by data sensitivity, tiered by risk, and gated by a human. The AI reads your systems, reasons over de-identified text on Claude or sensitive data on the local model, and proposes a change — but nothing regulated moves until a person approves it, with a full Part 11 audit trail.

4. Part 4

Here is how it is wired. An orchestrator fires agents on a schedule or on study events. Every agent calls one gateway, which routes by data classification, runs the Presidio guardrail that blocks patient data from the cloud, and logs everything. Behind it sit three things: the local model for sensitive data, the Claude API for de-identified reasoning, and a RAG knowledge base of your SOPs and study docs. Connectors expose your systems as tools. And every regulated change lands in a human-approval queue before it is written.

5. Part 5

The governance is the whole point. Three tiers: green for drafts and cited answers; amber for triage and proposed changes with a human verify; red for anything touching a reported number, a record of truth, or unblinded data — where a validated tool and a human decide, and the model is out of the path. Three hard rules: patient data never leaves the local model; every reported number comes from a validated tool; and a human approves every change to a regulated record or timeline.

6. Part 6

Take project management. When a data cut is declared, an agent recomputes the critical path in Smartsheet, sees what is now driving the finish date, drafts the you-are-now-blocked notifications, and posts the recomputed plan to the approval queue. Smartsheet's engine does the math; the AI explains and proposes; the project manager approves before any committed date is written. And the weekly status report drafts itself from the plan of record.

7. Part 7

On documents: agents auto-classify incoming files to the TMF Reference Model, flag missing or expiring documents, and propose filings, which the document controller approves. And every approved SOP and work instruction is ingested into the knowledge base — so anyone can ask, what does our SOP require for a derived-variable spec, and get the exact clause, cited.

8. Part 8

This is the marquee one — recurrent data-validity QC at every stage. Each night, after a new dataset build, Pinnacle 21 produces the authoritative conformance findings. The local agent then clusters them, diffs against the last clean run to surface only what is genuinely new, drafts a disposition grounded in the reviewer guides, and routes each one to the right programmer. It never changes a verdict. A four-hour manual triage becomes a clean, pre-explained queue at eight a.m.

9. Part 9

For monitoring, under ICH E6(R3) risk-based quality management: agents watch the key risk indicators and quality tolerance limits, detect breaches and central-monitoring signals like site outliers or enrollment anomalies, surveil protocol deviations, and assemble the DSMB packet. Unblinded signals run on the local model; the reported safety numbers come from validated tools with medical review.

10. Part 10

Every agent follows the same loop, and it is worth seeing once. A trigger fires; the agent gathers context from your systems and the cited knowledge base; it classifies the data and routes to local or cloud; it drafts, triages, or summarizes; it proposes the change to the human queue with its rationale and sources; and only on approval is anything written to the system of record — all of it logged. That loop is what makes seventy-five different automations safe and consistent.

11. Part 11

You build it in stages, and you start small. Phase zero is the stack — gateway, local model, Claude under a BAA, the guardrail and the approval queue. Phase one is the knowledge base — ingest the SOPs and ship the compliance Q&A agent first; it is pure green, cited, and instantly useful. Phase two adds read-only connectors and the nightly QC-triage and weekly status agents. Phase three turns on the event-driven automations and the monitoring signals. Each agent earns lighter review as it builds a track record.

12. Part 12

A day in the layer: at two a.m. the QC agent surfaces three genuinely new findings among two hundred forty and pre-triages the rest. At ten-thirty a data cut fires and the timeline cascade updates in minutes. At one p.m. a key risk indicator breaches and the signal is drafted for the monitor. At three, a programmer gets an SOP answer, cited, in seconds, and three documents are filed to the eTMF on approval. At four, the weekly status report drafts itself. Every regulated number came from a validated system; no patient data left the building; a human approved every committed change.

13. Part 13

So that is the operations layer. Seventy-five governed automations on the hybrid stack, absorbing the cross-system busywork while validated tools and humans keep ownership of everything that is regulated. Start with the knowledge base and the SOP agent, keep everything behind the approval queue, and expand as trust grows. The full catalog — every automation, its engine, its integration, and its human checkpoint — is in the operating wiki alongside this. Thank you.