TLF Interpretation Drafter QUARTO + ON-PREM SLM

Before you start

Quick-start & how to run

Tick each step as you finish it (full detail for the model pieces is in Runtime & hardware and the README):

• Install R packages: install.packages(c("reticulate","digest"))
• Create a user Python venv and pip install llama-cpp-python (no admin, no daemon)
• Put a GGUF model on disk and point tlf.slm.model_path / TLF_SLM_GGUF at it
• Run the guardrail self-test — Rscript selftest.R (no model needed) → expect SELFTEST PASS
• (optional) Render a TLF: quarto render interpret_tlf.qmd — drafts come out marked DRAFT — review required
• Biostatistician reviews / edits / signs off — the mandatory human gate before anything is APPROVED

Run it

The self-test proves the guardrails with no model installed (pure-R validators on a known-good draft and a hallucinated one). Run it from the tlf_interpret/ folder so the relative source("R/…") paths resolve:

cd tlf_interpret
Rscript selftest.R

Success ends with SELFTEST PASS (exit 0): the faithful draft → DRAFT_OK, the hallucinated one → REJECTED with its invented numbers (27, 90.0, 3) listed as unmatched. To draft with a real model, set the GGUF path and (optionally) full GPU offload, then render:

export TLF_SLM_GGUF="$HOME/models/medgemma-1.5-4b-it-Q4_K_M.gguf"
export TLF_SLM_GPU_LAYERS=999   # whole ~2.6 GB 4B on the 4 GB VRAM; drop toward ~28 on VRAM OOM. 0 = pure CPU
quarto render interpret_tlf.qmd

What it does

A pipeline that drafts the narrative interpretation that accompanies each Table/Listing/Figure (e.g. the AE-overview paragraph, the PK-parameter summary text), using an on-prem small language model (SLM), rendered with Quarto, and wired to the deterministic TLF outputs from ../r_tlf_pseudocode/ (or ../sas_tlf_pseudocode/).

This is a DESIGN + runnable skeleton, not a validated system. It is a drafting aid behind hard guardrails — it does not replace the biostatistician and it does not produce any reported number.

The one hard invariant

The SLM never sources a number. Every reported number comes from the validated TLF result dataset (which itself traces to Phoenix WinNonlin / Pinnacle 21 / EDC per SOP). The model only writes the prose around numbers it is handed, and that prose is then checked so that every number in the draft must already exist in the deterministic facts — otherwise it is rejected as a hallucination. The rendered table/figure in the report is the validated output, not anything the model produced.

Pipeline

Why each piece

• Numbers stay deterministic. The fact extractor is plain R operating on the aggregate TLF result (n's, %'s, geometric means already computed by the validated program). No participant-level data and no PHI ever reach the model. The model cannot compute, only restate.
• Local SLM, on-prem, NO daemon, NO admin (reticulate → llama-cpp-python, GGUF). Runs the model in-process from R via a user-space Python wheel — no Ollama, no system service, no network, no admin rights (the things that get blocked on locked-down work laptops). Sovereignty + no data egress + a frozen GGUF file (pinned by name + SHA) so the same facts reproduce the same draft.
• Deterministic decoding. temperature = 0, fixed seed → reproducible drafts; logged in the audit record alongside the model digest and a hash of the prompt and facts.
• Validators are not the model. The numeric-consistency check (regex-extract every number from the draft; each must match a facts value within tolerance) is the core safety control — it makes a hallucinated statistic structurally impossible to pass silently. The claim guard flags efficacy/causal/“well tolerated”-style language for mandatory human judgement.
• Human gate is mandatory. Nothing is APPROVED without biostatistician sign-off. The Quarto doc renders drafts as DRAFT — biostatistician review required and shows exactly which facts the draft was grounded on, so review is fast and auditable.
• Quarto is the orchestration + rendering + freeze layer: one parameterized .qmd per TLF (or a master doc looping the TLF list), freeze: true + renv lockfile for reproducible reports.

Architecture — the runtime is NOT Ollama

Runtime & hardware (32 GB RAM / 4 GB VRAM target)

The default is a dense 4B medical model that fits in the 4 GB VRAM at Q4 — a change from the earlier 26B-A4B MoE, which had to run CPU-first from the 32 GB RAM. Either works; pick by how much VRAM you have and how much capacity you want.

Getting a MedGemma GGUF (Google ships safetensors only — there is no official GGUF)

• convert it yourself (recent llama.cpp):

  python convert_hf_to_gguf.py google/medgemma-1.5-4b-it --outfile medgemma-1.5-4b-it-f16.gguf
  ./llama-quantize medgemma-1.5-4b-it-f16.gguf ~/models/medgemma-1.5-4b-it-Q4_K_M.gguf Q4_K_M

  or
• use a trusted community quant (e.g. an unsloth/bartowski GGUF, when available) — verify the publisher and record the SHA.
• We load the text model only (no mmproj/vision projector); the facts→prose task needs no image input.
• A medical model volunteers clinical conclusions more readily — the claim-guard (below) is essential, not optional.

Running it on 4 GB VRAM (the 4B now fits, unlike the MoE)

• simplest: n_gpu_layers = 0 (pure CPU/RAM), n_ctx = 4096 — always works.
• fast (now viable for a 4B): TLF_SLM_GPU_LAYERS=999 puts the whole ~2.6 GB model on the GPU; drop toward ~28 if you hit a VRAM OOM (the KV cache + compute buffer also need room).
• for the 26B-A4B alternative instead: keep experts on CPU, offload the backbone — options(tlf.slm.tensor_overrides = ".ffn_.*_exps.=CPU", tlf.slm.gpu_layers = 999) (recent llama-cpp-python; = llama.cpp --override-tensor ".ffn_.*_exps.=CPU" -ngl 999).

Expect tens of tok/s with the 4B on the GPU (or ~5–15 tok/s CPU-only) → ~seconds per draft; fine for review-gated batch drafting. Keep n_ctx small (4096) — the facts+prompt+draft is <1K tokens, so a large context just wastes RAM, and on a busy work laptop (Outlook/Teams/browser open) close heavy apps to avoid swap.

Install once in a user venv: pip install llama-cpp-python; drop the .gguf on disk; point tlf.slm.model_path at it. No admin, no daemon, no network. If even reticulate is blocked, the same wheel exposes a local OpenAI-compatible server (python -m llama_cpp.server) on localhost — but in-process reticulate is preferred (no port, no process to manage).

Honest limits

• Small local models are weak writers; this is draft-assist + validate, never autonomous authoring. The guardrails (number-validation + human gate), not the model's competence, are what make it safe.
• The numeric validator catches fabricated/altered numbers; it does not certify the clinical correctness of the interpretation — that is the reviewer's job (the claim guard only surfaces risky phrasings). Two specific boundaries the reviewer must cover: (a) it checks each number exists in the facts, not that it is attached to the right quantity — a model could pair a valid number with the wrong noun (“1 death” when the 1 was actually the SAE count); (b) spelled-out numbers (“twenty-four”) bypass the digit regex — the prompt requires digits, but the human gate is the backstop.
• It does not, and must not, generate any regulated/reported number. Tables/figures shown are the validated outputs.
• Not validated software; treat outputs as draft text for human authoring, under ops governance.

Wire-in

Point extract_facts() at the result tibble a TLF program returns (e.g. the AE-overview counts, the t_pk_param_summary summary). The same pattern serves SAS: emit the result dataset, read it in R for fact extraction + rendering, or call the SLM client from SAS via a shell/PROC HTTP step.

Browse the code

R/ — pipeline scripts

Quarto render layer

Tests & docs

Troubleshooting

Real failure modes from R/slm_client.R and README.md — symptom → why → fix.