On-device Small Language Models + SAS/R — the 75-component stack

We took the 75-component hybrid trial-operations platform and re-assessed every component for a SMALL open-weight model running on-device (a 1–9B quantized model on a workstation, fully offline) paired with SAS/R. The principle is strict: SAS/R owns every number and check; the small model only classifies, extracts, routes, and drafts — always behind a human gate.

Validated tools (Pinnacle 21, Phoenix WinNonlin, the EDC/CTMS) still own every reported number and record — the “RED” governance is unchanged. What changes here is that the language helper is a small, local, offline model, so the data-sovereignty story is the simplest of any option — at the cost of model capability.

Why on-device SLM + SAS/R

This is the third variant of the same 75-component, early-phase clin-pharm trial-operations stack. The first ran on cloud Claude; the second on a big local GPU behind a model-agnostic gateway. This one asks a narrower question: how far do we get when the only model is a small open-weight model running on hardware we already own, fully offline, paired with the SAS/R we already validate against? The honest answer is: further than a skeptic expects on the language plumbing, and not as far as the marketing on anything that needs real reasoning. The division of labor is what makes it work.

What "on-device SLM" actually means here

A 1–9B-parameter open-weight model — Qwen3 (0.6B–8B), Gemma 4 (E2B/E4B), Phi-4-mini (3.8B), IBM Granite 4.1 (3B/8B), Llama 3.3/3.2 (8B/3B/1B) — quantized to 4–8 bit (GGUF) and served locally by Ollama, llama.cpp, or LM Studio on an ordinary workstation or laptop, CPU or a single consumer GPU. (Ollama and LM Studio both wrap the llama.cpp inference engine; hard grammar-constrained decoding — GBNF in llama.cpp — is most directly controlled through llama.cpp / llama-server, which is what we standardize on for any call that must return schema-valid JSON.) No GPU farm, no cloud account, no telemetry, no network call leaving the box. SAS/R reaches it over 127.0.0.1 (PROC HTTP / httr2) for the language step only.

The case for the small-model pairing

• Maximum sovereignty — the simplest possible HIPAA story. The data physically never leaves the workstation. There is no cloud egress to negotiate, no BAA to scope around an AI vendor, no EU-data-boundary question. The control is one sentence: the box stays offline.
• No new hardware. A 3B–9B quantized model runs on a workstation or laptop the team already has — a 3.8B model fits in a few GB of VRAM, and 4-bit 7B–9B models run on CPU. No GPU procurement, no server build-out as a precondition to start.
• Negligible marginal cost. After setup there is no per-token metering and no usage budget to police; the only cost is compute time and power. One workstation does serialize calls, so throughput — not price — is the practical limit.
• Freezable for reproducibility — with one honest caveat. We pin the model name + quantization + digest, the decoding settings (temperature 0, fixed seed), and the exact prompt; that bundle is an archivable, re-runnable artifact under change control. Re-run on the same pinned build and hardware and you get the same behavior. The honest caveat: temperature 0 + a fixed seed do not guarantee bitwise-identical tokens across different GPUs/drivers or under batching/caching — CPU, single-threaded, fixed-build execution is the reproducible configuration we validate against. This is far closer to a GAMP-5 / 21 CFR Part 11 story than the cloud variant can offer (we control the binary and it never changes underneath us), but the regulatory guarantee comes from the deterministic SAS/R validator and the human gate, not from the model being perfectly deterministic.
• No vendor or network dependency. Nothing breaks because a model was deprecated, an API changed, or the link went down. The capability is on the box, and it stays the version we validated.

The capability ceiling — stated plainly

A small quantized model hallucinates more than a frontier model and is more sensitive to prompt and format. It degrades well before its advertised context window. The bigger-is-better instinct is correct: when the language work needs genuine reasoning, a 7B quantized model is not enough. So we never let it near these:

• Anything numeric or quantitative — math, counts, reconciliation, statistics. SAS/R owns all numbers, always. Reported and regulated values come only from validated engines (Pinnacle 21, Phoenix WinNonlin, the EDC/CTMS) — never the model.
• Multi-document synthesis or long-context reasoning — small models fall apart here first.
• Nuanced clinical, safety, or regulatory narrative, judgement, or adjudication.
• Faithful long summaries; cross-correlating many signals.

The division of labor — non-negotiable

SAS/R is the spine. It owns data access, every computation, check, reconciliation, and statistic, the scheduling and orchestration, the validator that checks the model's structured output against the allowed values, every write to a record of truth, and the audit log. The model is a sidecar SAS/R calls for one bounded language task, with JSON/grammar-constrained output and a human approving before anything touches a record.

Where it is reliably good — and only with RAG grounding, constrained output, a SAS/R validator, and a narrow prompt — is short-text classification into fixed buckets, field extraction from a short passage, routing and severity tagging, near-duplicate grouping, short templated drafting grounded in retrieved text (a query line, a reminder email, a candidate code for human confirmation), and yes/no/which-bucket calls over a small context. That is exactly the language seam across this trial-ops stack, and SAS/R does everything around it.

How the 75 components landed

We assigned every component exactly one verdict for the on-device-SLM + SAS/R variant.

The shape of that distribution is the honest verdict on the variant. Only 12 components are clean on-device wins; 36 are feasible but only behind real guardrails; and 15 are simply Beyond a small model — kept deterministic in SAS/R or escalated. The remaining 12 are infrastructure, and this is where the offline posture pays off: a single offline local model means there is no cloud egress to firewall (the control collapses to "the box stays offline"), and the model-agnostic gateway collapses to "SAS/R calls one local endpoint." The plumbing gets simpler precisely because the sovereignty is absolute.

What a small model can and can't do

An on-device small model is a narrow language helper, not a reasoning engine. Knowing exactly where its useful work ends is the whole game: scope it to the tasks it does reliably, scaffold those tasks tightly, and keep everything else in SAS/R. Across the 75-component stack only 12 components land in the clean on-device zone; 36 are feasible only behind guardrails, 15 are genuinely beyond a small model, and 12 are pure platform plumbing. That distribution is the honest shape of the envelope: a small model earns its place on short, bounded, structured language sub-tasks and nowhere else.

What an on-device SLM actually is

An on-device small language model (SLM) is a 1–9B-parameter open-weight model, quantized to roughly 4–8 bits and shipped as a GGUF file, served locally by a runtime such as Ollama, llama.cpp, or LM Studio on an ordinary workstation or laptop (CPU, or a single consumer GPU), fully offline with no telemetry. Representative families in mid-2026 include Qwen3 (0.6B–8B dense), Gemma 4 (E2B/E4B), Phi-4-mini (3.8B), IBM Granite 4.1 (3B/8B), and Llama 3.3/3.2 (8B/3B/1B). "Open-weight" means the weights are downloadable — it does not mean OSI open-source, and licences differ: Qwen3, Granite 4.1 and Gemma 4 ship under Apache 2.0, whereas Llama carries a custom community licence with acceptable-use restrictions. Confirm the licence for the specific model and version before any regulated use.

Reliable — but only with guardrails

These are the language sub-tasks a 1–9B quantized model does dependably, and only when wrapped in all four guardrails at once: RAG grounding (retrieve a short, relevant passage first), constrained output (JSON-schema or grammar-restricted), a deterministic SAS/R validator that checks every field against an allowlist, and a narrow prompt. Remove any one of the four and the reliability drops.

• Short text classification into a fixed set of buckets — e.g. labelling a finding by type or a document by TMF artifact.
• Field / named-entity extraction from a short passage — a date, an expiry, a site number, an owner pulled from one paragraph, not a whole document.
• Routing & triage tagging — which owner, queue, or severity bucket an item belongs in.
• Near-duplicate / similarity grouping — is this finding new, or the same as one we already raised?
• Short, templated drafting grounded in retrieved text — a query text, a reminder email, a one-line status note, or a candidate code for a human to confirm. The human, not the model, decides.
• Yes / no / which-bucket decisions over a small context.

The common thread: the input is short, the output space is small and pre-defined, and a deterministic check can confirm the answer is one of the allowed values. That is exactly where a small model is safe.

Do NOT trust a small model with

These are not deployment details to tune around — they are the capability ceiling. A small quantized model will fail on them in ways that look fluent and confident, which is worse than failing loudly.

• Anything numeric or quantitative — math, counts, reconciliation, statistics. SAS/R owns every number, always. The model is never in the number path.
• Multi-document synthesis or long-context reasoning — small models degrade well before their advertised context window, and cross-correlating many signals is exactly what they cannot hold.
• Nuanced clinical, safety, or regulatory narrative, judgement, or adjudication — the stakes and the nuance are both too high.
• Faithful long summaries — the longer the source and the longer the output, the more a small model invents or drops.

Division of labor

SAS/R owns data access, all computation, checks, reconciliation, and statistics, the orchestration and scheduling, the validator that checks the model's structured output against allowed values, every write, and the audit log. The SLM is a constrained language sidecar that SAS/R calls over a local loopback endpoint (PROC HTTP or httr2 to a 127.0.0.1 Ollama or llama.cpp server) for the language sub-task only, with schema-validated output and a human gate before anything reaches a record of truth. Reported and regulated numbers come only from validated engines (Pinnacle 21, Phoenix WinNonlin, EDC/CTMS) — never the SLM.

Quantization: the quality / footprint trade

An on-device model is shipped as a GGUF file at a chosen quantization — the weights are compressed from 16-bit to roughly 4–8 bits so the model fits in ordinary RAM and runs at usable speed. The trade is direct: lower bit-width means a smaller, faster model but degraded instruction-following and schema adherence.

• Q4_K_M — the common footprint choice; a 7–8B model is roughly 4–5 GB on disk and about 5–6 GB resident at runtime (weights plus KV cache and overhead), so plan for materially more system RAM than the file size — on the order of 16 GB+ for comfortable CPU inference. It runs on CPU, just slowly; a consumer GPU is much faster. Good enough for many classification and tagging tasks if you validate it on yours.
• Q8_0 — higher fidelity, larger footprint (roughly double Q4_K_M); closer to the full model's behaviour, worth it where schema adherence at Q4 is shaky.

The rule: pick the smallest model and lowest bit-width that still pass your validation set for the specific task — never assume a quantization is fine because it works in a demo.

Why constrained output + a SAS/R validator is mandatory

A small model, left to free-form, will wrap its answer in prose, invent a category that was never offered, or drift the format between runs. Two mechanisms make it usable — and a human still closes the loop, because neither one certifies that the answer is correct:

Constrained output controls the shape; the validator confirms the value is permitted. Crucially, a grammar guarantees syntax, not meaning — the model can still emit a perfectly valid, perfectly wrong enum value (and the renormalization that enforces the grammar can even nudge quality down). So the model is treated as an untrusted source: its structure is enforced, its choice is bounds-checked, and a human ratifies the substance.

Why RAG grounding is what makes a small model usable

A small model's own "knowledge" is unreliable and unverifiable. RAG — retrieve-then-read — flips that: SAS/R first retrieves the short, relevant passage (the study spec, the SOP excerpt, the matching prior finding), hands only that to the model, and asks it to answer from the supplied text. The model is reduced to reading and labelling a short context it was given, not recalling from training. This both raises accuracy and keeps the context short — the regime small models handle well. Grounded on a short retrieved passage, a small model is useful; asked to answer from memory or over a long document, it is not. RAG reduces, but does not eliminate, hallucination — which is why the validator and the human gate still apply.

Effective vs advertised context window

A model card may advertise a large context window, but a small quantized model's effective window — where it still reasons reliably — is much shorter. Accuracy falls off, and information in the middle of a long input is quietly lost, well before the advertised limit. Treat the advertised number as a hard ceiling you stay far below: keep inputs short, retrieve narrowly with RAG, and never rely on the model holding a long document in mind. This is precisely why long summaries and multi-document synthesis sit in the "do not trust" list.

Determinism: pin the decoding AND the environment

For regulated work a run must be reproducible. SAS/R pins the decoding to greedy sampling (temperature 0) with a fixed seed, and pins the model identity — name + quantization + digest — plus the exact prompt. But temperature 0 and a fixed seed are not sufficient on their own: floating-point and batching behaviour can vary across hardware, GPU vs CPU, runtime version, and concurrency, so the same prompt can drift if the environment changes. Reproducibility is therefore a property of the frozen environment, not the seed alone: pin the runtime build, the backend (e.g. CPU-only), the context length, and the batch conditions, then verify by re-execution and record the result. Treated that way — pinned, constrained, validated, human-gated, and re-run to confirm — the language step is a controlled, qualifiable artifact under a GAMP-5 / 21 CFR Part 11 framework. The point is not that the model is "smart"; it is that the step is documented, bounded, and re-runnable, with the regulated numbers still coming only from the validated engines.

The SLM + SAS/R architecture

This is the same governance as the other companion stacks — a deterministic engine does all the real work, a human approves before anything touches a record of truth, and a Part 11 audit trail records all of it. The only change here is the language helper: instead of cloud Claude behind a gateway, a small open-weight model runs locally on the box, fully offline. SAS/R still owns every number; the model only does a narrow language sub-task, and only when called.

Across the 75-component trial-ops stack the assessment assigns each component one verdict for the language sub-task: Ready 12 (the language sub-task sits squarely in the reliable zone), Guardrails 36 (feasible on-device only with tight scaffolding and a stated residual risk), Beyond 15 (needs frontier-level reasoning or long-context synthesis a small quantized model cannot do reliably — keep deterministic or escalate to a larger model off-box), and Platform 12 (pure infrastructure, not a language task). These four sum to 75; they are this assessment's judgement calls, not a measured benchmark. Most useful work falls in the Guardrails tier, which is the honest answer: a small local model earns its place only when it is boxed in.

What an “on-device SLM” actually is

A roughly 1–9B-parameter open-weight model (Qwen3 0.6B–8B, Gemma 4 (E2B/E4B), Phi-4-mini, IBM Granite 4.1 (3B/8B), Llama 3.3/3.2 (8B/3B/1B)), typically 4–8 bit quantized to a GGUF file, served by Ollama / llama.cpp / LM Studio on an ordinary workstation or laptop (CPU, or a single consumer GPU). No internet, no telemetry. It is a constrained language sidecar, not the system — and it hallucinates more than a frontier model and is more sensitive to prompt and format, so it never gets handed a number, a long document, or a clinical judgement.

The loop

Division of labor (always)

Reported and regulated values come only from validated engines — Pinnacle 21, Phoenix WinNonlin, the EDC/CTMS — never the SLM. The model classifies, extracts, routes, and drafts a candidate; it does not compute, reconcile, or report.

Reproducibility — a frozen, re-runnable artifact

Pin the model name + quantization + digest, fix the decoding (temperature 0, top_k 1, fixed seed), set an explicit context length, and version the prompt and schema. Record that whole tuple in the audit log. One honest caveat: temperature 0 plus a fixed seed does not by itself guarantee bit-identical tokens. On a GPU, output can vary run-to-run because reduction order and batch scheduling are non-deterministic; reproducibility therefore also requires pinning the runtime — ideally CPU inference (or deterministic kernels), a fixed build, a fixed context length, and single-stream serving with no concurrent batching. With the stack pinned that way the step becomes practically reproducible, which is what GAMP-5 / Part 11 asks of a qualified component. Two further points keep it defensible regardless: the SLM's output is never authoritative on its own — the deterministic SAS/R validator constrains it to allowed values, and a human approves — so the record of truth never depends on the model reproducing a token exactly.

How the “Platform” controls simplify here

An offline local model makes several infrastructure controls cheaper than in the cloud companions:

• Egress firewall (Presidio / DLP) — collapses to a physical fact: the box stays offline, so model egress is impossible. There is no cloud call to inspect or scrub. (This control becomes “keep the box offline” — itself a verified, change-controlled state, not a free pass.)
• Model-agnostic gateway — collapses to SAS/R calls one local 127.0.0.1 endpoint. No router, no per-model cost ledger.
• Model-freeze registry — is just the pinned name+quant+digest (plus runtime build and decoding params) recorded on disk, under change control.

The approval queue, RBAC/secrets, and the audit trail remain as they are in the other companions — SAS/R-owned, human-gated, fully logged.

Choosing an on-device model

For this stack the model is the least important decision — the scaffolding around it (constrained output, RAG grounding, the SAS/R validator, the human gate) is what makes a small model usable, and that scaffolding is the same regardless of which open-weight model you pick. So choose conservatively. A handful of well-supported open-weight families all run offline under Ollama or llama.cpp on hardware you already own, and any of them is a reasonable starting point for the narrow language jobs in this stack (classify, extract, route, draft short grounded text). The job here is to pick one that is licence-clean, well-supported, and small enough to validate — then prove it on a fixed test set for your specific sub-task. Do not assume a model is adequate because it appears in this table; capability is established by your validation run, not by the model card.

Representative on-device open-weight models (June 2026)

Sizes, licences, and recommended variants are as of June 2026 — verify the current version, the exact licence on the specific size you download, and the model card before deploying. Licences and available sizes change between releases (Gemma moving from a custom licence in Gemma 2/3 to Apache-2.0 in Gemma 4, and Qwen consolidating to uniform Apache-2.0 in Qwen3, are recent live examples), and terms can still differ across sizes within a family.

How to choose — practical guidance

• Prefer Apache-2.0 or MIT. Phi-4-mini (MIT), Granite 4.1 (Apache-2.0), Qwen3 (Apache-2.0 across its dense lineup) and Gemma 4 (Apache-2.0 as of the April 2026 release) carry the cleanest, OSI-approved licences and minimise legal review. Llama’s Community License is usable for a CRO but is a custom, non-OSI licence with conditions that legal must read in full — do not treat it as “effectively Apache.”
• Pick the smallest model that passes your validation on the task. Run the candidate against a fixed, representative test set for the specific language sub-task (e.g. “label this finding new-vs-known”) and choose the smallest model that clears your acceptance bar. A 3B model that passes is better than an 8B you never validated — smaller is cheaper, faster, and easier to freeze. Do not assume bigger is needed until a smaller one demonstrably fails — but equally, do not assume any on-device model passes until it does. If none clears the bar, that is a signal the task is “Beyond” on-device SLM, not a prompt to ship a failing model.
• Favour instruct / structured-output-capable variants. Use the instruct-tuned build, not the base model, and confirm it follows JSON/grammar-constrained output reliably. In llama.cpp this is enforced with a GBNF grammar (Ollama exposes a JSON/format mode over the same engine); that constrained, schema-shaped output is exactly what the SAS/R validator checks against allowed values before anything reaches the human gate.
• Quantize deliberately. Pull a GGUF and choose the quantization for the job: Q4_K_M for the smallest footprint and fastest CPU inference (the usual default), or Q8_0 when you want maximum fidelity and prompt-adherence and can spend the RAM. Heavier quantization shrinks the model but degrades instruction-following and increases hallucination, so re-run your validation set at the quantization you will actually deploy — not at full precision.
• Pin what you ship, and pin where it runs. Record model name + size + quantization + file digest, fix decoding (temperature 0, fixed seed, greedy/top_k=1), and freeze the prompt. Be honest about the limit of this: temperature 0 plus a fixed seed makes sampling deterministic, but it does not by itself guarantee bit-identical output — GPU floating-point reduction order, batching, and KV-cache layout can still cause small run-to-run variation. For a reproducible, Part 11 / GAMP-5-defensible artifact, also pin the execution environment (e.g. CPU inference, or a deterministic-inference build) and qualify it by re-running the validation set, validating against behaviour on the test set rather than asserting bit-for-bit identity. That qualified tuple — not “Llama 8B” in the abstract — is what you register.

Project & Timeline Management

Status drafting, action capture, vendor oversight, timeline notes — mostly short classification/extraction/drafting a small model can do behind a human gate. SAS/R owns all the date math and the writes.

eTMF & Document Control

Document classification to the TMF model, metadata/expiry extraction, SOP Q&A over RAG — a strong fit for a small model on short passages with constrained output. Filing of record stays human + validated eTMF.

Data-Validity & Recurrent QC

Trial Monitoring & RBQM

KRI/QTL signal labelling, deviation classification, MVR action extraction. Nuanced safety narrative and multi-signal synthesis are Beyond a small model — SAS/R computes, the model tags, humans decide.

The platform layer

These are infrastructure, not model tasks — and the on-device variant simplifies most of them: one offline local model means there is no cloud egress to firewall and no model-agnostic gateway to run; SAS/R calls a single local endpoint, pins the model, and writes the audit trail.

Additional / easily-missed

The easily-missed ops: consent-version tracking, IP accountability, safety-clock and submission tracking, training compliance, data-transfer receipt. Extraction and reminders suit a small model; the regulated records stay validated + human.

Standing it up with IT — instructions & hurdles

This is the section to hand IT, Security, and QA. It is two parts: a concrete, numbered enablement playbook for deploying an on-device small model alongside SAS/R, and a candid hurdles table naming what each function will raise and exactly how to clear it. Throughout, hold the line: SAS/R and the validated tools (Pinnacle 21, Phoenix WinNonlin, the EDC/CTMS) own every number and every record. The small model is a constrained, offline language helper behind a human gate. Nothing here asks IT to trust an AI with a regulated value.

Part 1 — the enablement playbook

1. Pick the use cases and pin the language task to a schema. Choose only from the Ready (green) and Guardrails (amber) components — short classification, field extraction, routing/triage tags, near-duplicate grouping, or a short RAG-grounded draft. For each, write down the exact language sub-task and the output schema (the allowed labels/enum, the fields, the JSON shape). If you cannot reduce it to a fixed-vocabulary, short-context decision, it is Beyond a small model — leave it deterministic SAS/R or escalate to a larger model; do not force it on-device.
2. Size the hardware — CPU-first. A modern workstation runs a 7–8B model quantized to Q4 (e.g. Q4_K_M GGUF) at a usable, non-interactive speed on CPU alone; 3–4B models are comfortably faster. Memory rule of thumb: at Q4 the weights are roughly 0.5 GB per billion parameters (~4–5 GB for an 8B), but you must add the KV cache, the context, and runtime overhead — so budget closer to ~6 GB resident for an 8B Q4 model and provision 16 GB+ of system RAM to run it comfortably alongside SAS/R. Add one consumer GPU only where throughput demands it — for triage-style batch jobs that run nightly or in small bursts, CPU is usually enough. A single 12–16 GB GPU fits an 8B model in VRAM and multiplies tokens/sec if a queue ever justifies it. (Note: moving from CPU to GPU can change exact byte-for-byte outputs — see step 5 on reproducibility.)
3. Choose a runtime and install it in the locked-down environment. Ollama for the simplest operations (one local service, easy model management); llama.cpp when you want maximum control or to embed the engine (Ollama wraps llama.cpp, so they share the GGUF runtime); LM Studio for desktop trials and quick evaluation; vLLM only later if you genuinely need server-grade throughput on a GPU host. Install from a signed release pinned to a version, mirrored on the internal artifact repository (or delivered as an approved container image). Pull the chosen model on a connected staging box, record and checksum the model file (SHA-256 digest), then transfer that verified file to the offline workstation — the production box never reaches the internet.
4. Wire SAS/R to the local endpoint and bolt on the validator + human gate. SAS calls the loopback endpoint with PROC HTTP; R uses httr2 — both target 127.0.0.1 only. SAS/R does all data access and computation, then sends just the short language sub-task. The model returns schema-constrained JSON (enforced with the runtime's grammar / JSON-schema / structured-output mode, not just asked for in the prompt); a SAS/R validator then parses it and checks every field against the allowed values, rejecting and logging anything malformed or off-list. Nothing reaches a record of truth until a human reviews and approves it.
5. Pin the configuration for best-effort reproducibility; register it; do not claim bit-exactness. Freeze model name + quantization + SHA-256 digest, the decoding settings (temperature 0 / greedy, top_k and top_p pinned, repeat/frequency penalties pinned, a fixed seed for any stochastic path), and the full runtime context: runtime build/version, hardware, thread count, and single-stream batch = 1 execution. Be honest with QA about what this buys you: temperature 0 makes the token-selection rule deterministic, but it does not guarantee byte-identical output across different hardware or under varying batch/load, because floating-point addition is non-associative and kernels accumulate in different orders. Pinning the runtime, the hardware, and batch = 1 (CPU single-stream is the most reproducible path) gets you run-to-run stable output on that box; treat cross-hardware bit-exactness as not assured. Enter the frozen tuple in the engine-of-record / model-freeze registry exactly as a version of a qualified tool — no silent model, quant, runtime, or hardware swaps.
6. Validate (GAMP 5 Second Edition, risk-based) and wire the Part 11 audit trail — which is the real backbone. Scope the model as low-to-medium-risk decision-support, not a system of record, and follow GAMP 5 (2nd ed.) with the ISPE GAMP RDI Good Practice Guide, Appendix D11 (AI/ML): treat it as a probabilistic component, not deterministic software. IQ the install (runtime version, model digest, offline state, batch = 1 config). OQ the pinned model against a fixed, labeled test set — measure label accuracy and JSON-schema adherence, set an acceptance threshold, and include a re-run stability check (same inputs run repeatedly on the production box) rather than assuming one golden output. PQ on the real workflow with humans in the loop. Because exact reproduction is configuration-dependent, the 21 CFR Part 11 audit trail is what makes this defensible: SAS/R writes, for every item, prompt + model digest + runtime/config + raw model output as actually returned + the validator result + the human decision + timestamp + user. You audit the output that actually occurred and who approved it — you do not rely on being able to regenerate it bit-for-bit.
7. Define the support model and phase the rollout. Name an owner; treat the model like a qualified instrument under change control (any change to model, quantization, runtime build, or the hardware/decoding config is a controlled change with re-validation). Set a patch cadence for the runtime. Roll out in phases: one narrow Ready use case on one box first, prove the OQ/PQ and the audit trail, then widen to the next.

Part 2 — anticipated hurdles

What IT, Security, and QA will raise — and the honest answer for each. None of these requires overselling the model.

What a small on-device model can’t do — and why

The 15 components marked Beyond are the honest line: the language work needs frontier-level reasoning, long-context synthesis, or nuanced clinical/regulatory narrative that a small quantized model will not do reliably. Keep these deterministic SAS/R or escalate to a larger local/cloud model.

The SAS/R × SLM glue — %slm_* macros & R functions

A small helper library so calling the local model is a few lines: SAS/R sends the prompt to the offline endpoint, forces schema-constrained JSON, gets the structured answer, and a SAS/R validator checks every field against an allowlist before anything is shown to a human. The model is pinned (name + quantization + digest + temperature 0 + seed) so a run is reproducible.

Below is the complete, actual program, every line. SAS/R owns every number and every check; the on-device small model only drafts language, never a reported number. These are the real shipped files (also linked above).

The one file you edit per study: slm_config.sas

SAS · 20 lines · the real shipped file, in full — nothing elided.

slm_config.sas is the single per-deployment configuration file for LOCALMIND, the on-device small-language-model triage library in the SLM wiki macro library. It holds nothing but constants: the pinned model tag, the loopback endpoint, the reproducibility seed, a fail-loud contact, and the controlled vocabularies the model's drafted output is validated against. You %include it before the driver, so the driver macros read fixed values instead of hard-coded literals. Nothing here calls the model or produces a number: every reported number and every check still comes from validated SAS/R, the model only drafts language, and a human owns every decision.

/*====================================================================================
  slm_config.sas  -  per-deployment constants for LOCALMIND. %include before the driver.
  Pin the model here; keep the allowlists (the controlled vocabularies the model's output
  is validated against) in one place so adding a label is a config edit, not a code edit.
====================================================================================*/
%global CFG_MODEL CFG_BASE CFG_SEED CFG_BACKUP
        CFG_OWNERS CFG_SEVERITY CFG_NOVELTY;

/* The PINNED model: exact tag + quantization. Must be `ollama pull`-ed already.
   Prefer an Apache-2.0/MIT model (Qwen2.5, Granite, Phi) to avoid licence friction.    */
%let CFG_MODEL = qwen2.5:7b-instruct-q8_0;
%let CFG_BASE  = http://127.0.0.1:11434;     /* loopback only - %slm_init refuses anything else */
%let CFG_SEED  = 42;                          /* fixed seed + temperature 0 = reproducible       */
%let CFG_BACKUP= biostat.backup@example.com;  /* fail-loud target                                */

/* The controlled vocabularies (pipe-delimited). The model's output is rejected if it is
   not on these lists - the in-code trust boundary, analogous to SHEETLINK's allowlist.   */
%let CFG_OWNERS   = DM|PROGRAMMING|PK|MEDICAL CODING|SITE|BIOSTAT;
%let CFG_SEVERITY = CRITICAL|MAJOR|MINOR;
%let CFG_NOVELTY  = NEW|KNOWN;

Walkthrough, part by part

File header comment. The opening block states the file's contract: these are the per-deployment constants for LOCALMIND, and you %include this file before the driver. It records the design intent that the model is pinned here and that the allowlists (the controlled vocabularies the model's output is validated against) live in one place, so adding a label is a config edit rather than a code edit. This keeps the trust boundary auditable: a reviewer can see exactly which model and which vocabularies a run used by reading one short file.

%global declaration of the seven configuration macro variables. The %global statement declares CFG_MODEL, CFG_BASE, CFG_SEED, CFG_BACKUP, CFG_OWNERS, CFG_SEVERITY, and CFG_NOVELTY so they exist in the global symbol table and remain visible to the driver macros after this file finishes including. Declaring them global before assignment guarantees the driver reads the study's intended values and not a stale or locally scoped copy. If you add a configuration key later, it must be declared here as well as assigned below, or the downstream macros will not see it.

CFG_MODEL: the pinned model tag and quantization. %let CFG_MODEL = qwen2.5:7b-instruct-q8_0; pins the exact model, including its quantization (q8_0), so every run uses an identical frozen local model. The comment directs you to have it ollama pull-ed already and to prefer an Apache-2.0/MIT model such as Qwen2.5, Granite, or Phi to avoid licence friction. Pinning the tag is what makes the drafting step reproducible and defensible: the same weights produce the same behavior across runs, and this is language drafting only, never a source of a reported number.

CFG_BASE: the loopback-only endpoint. %let CFG_BASE = http://127.0.0.1:11434; sets the model server address to the local loopback interface on Ollama's default port. The inline comment notes that %slm_init refuses anything else, so this value cannot silently point the library at a remote host. Keeping inference on 127.0.0.1 is the data-sovereignty guarantee: participant-adjacent text drafted by the model never leaves the machine.

CFG_SEED: the fixed reproducibility seed. %let CFG_SEED = 42; supplies the fixed random seed for the model. The comment states that the fixed seed plus temperature 0 make generation reproducible, so the same prompt yields the same draft text. This supports validation and audit of the language the model produces, while the numbers in any deliverable still come from validated SAS/R rather than from this seeded generation.

CFG_BACKUP: the fail-loud contact. %let CFG_BACKUP = biostat.backup@example.com; defines the constant the comment labels as the fail-loud target, a named human contact for the deployment. It exists so that a failure surfaces to a person rather than being swallowed silently, which is consistent with the governance stance that a person, not the model, owns every decision. Replace the placeholder value with the real biostatistics backup contact for the study before running.

CFG_OWNERS: the controlled vocabulary of functional owners. %let CFG_OWNERS = DM|PROGRAMMING|PK|MEDICAL CODING|SITE|BIOSTAT; is the pipe-delimited allowlist of the only functional owners the model may assign. The comment frames these lists as the in-code trust boundary: the model's output is rejected if it is not on the list, analogous to SHEETLINK's allowlist. This is how a free-text drafting step is constrained to a validated controlled vocabulary, so an owner the model invents is caught and refused rather than reported.

CFG_SEVERITY: the controlled vocabulary of severity levels. %let CFG_SEVERITY = CRITICAL|MAJOR|MINOR; is the pipe-delimited allowlist of permitted severity labels. Any severity the model emits that is not one of CRITICAL, MAJOR, or MINOR is rejected at the same trust boundary. Constraining severity to three fixed values keeps the drafted triage language on a known scale that a human reviewer can act on, and the severity assignment remains a suggestion for a person to confirm, not a decision the model makes.

CFG_NOVELTY: the controlled vocabulary of novelty flags. %let CFG_NOVELTY = NEW|KNOWN; is the pipe-delimited allowlist for the novelty flag, restricting the model's output to NEW or KNOWN. As with the other vocabularies, a value outside this list is rejected rather than passed through. To extend any of these three allowlists you edit the pipe-delimited string here, which is the config-edit-not-code-edit design the header describes, and the validated SAS/R downstream still enforces the boundary and owns every check.

The complete SAS side of the glue: slm_macros.sas

SAS · 146 lines · the real shipped file, in full — nothing elided.

slm_macros.sas is the complete SAS side of the LOCALMIND triage library: the set of %slm_* macros that let a SAS program call a local, offline small language model (Ollama or llama.cpp) for the language layer only. The library is invoked at the top of any triage or routing job, before any per-row classification loop, and it enforces the governing pattern in code: the on-device model classifies, extracts, routes, and drafts under a schema constraint, while SAS owns every number and every check and a human owns every decision. The header comment states the model is an untrusted text source whose output is grammar-constrained at the server and allowlist-validated here (belt and suspenders), and that the pipeline is frozen for GAMP-5 and 21 CFR Part 11 via temperature 0, a fixed seed, and a pinned model. The model never produces, reconciles, or decides a reported number; those come from the validated SAS/R tools.

/*====================================================================================
  slm_macros.sas  -  LOCALMIND: a SAS macro library to call a LOCAL, OFFLINE small
                     language model (Ollama / llama.cpp) for the LANGUAGE layer only.
  ------------------------------------------------------------------------------------
  Pattern : SAS/R owns every number and check; the on-device SLM only classifies /
            extracts / routes / drafts, with SCHEMA-CONSTRAINED output, a SAS validator,
            and a human gate. The model never produces a reported number.
  Prefix  : every macro is %slm_<name>.  Base SAS 9.4M5+ (PROC HTTP + JSON libname).
  OFFLINE : the endpoint MUST be loopback (127.0.0.1 / localhost). %slm_init refuses any
            routable host -> "nothing leaves the box" is enforced in code, not by policy.
  REPRO   : temperature 0 + fixed seed + a PINNED model (tag + quantization + sha256
            digest) = a frozen, re-runnable artifact for GAMP-5 / 21 CFR Part 11.
  TRUST   : the model is an UNTRUSTED text source. Output is grammar-constrained at the
            server AND allowlist-validated here (belt + suspenders). Reject, never coerce.
====================================================================================*/

%global SLM_BASE SLM_MODEL SLM_SEED SLM_DIGEST SLM_RC SLM_BACKUP;

/*==== %slm_assert : FAIL LOUD. ========================================================*/
%macro slm_assert(cond=, msg=, abort=N);
  %if not (&cond) %then %do;
    %let SLM_RC=1;
    %put ERROR: [LOCALMIND] ASSERT FAILED: &msg;
    %if %length(&SLM_BACKUP) %then %do;
      filename _slmm email to=("&SLM_BACKUP") subject="LOCALMIND FAILURE - &msg" importance="High";
      data _null_; file _slmm; put "LOCALMIND job FAILED: &msg"; run; filename _slmm clear;
    %end;
    %if &abort=Y %then %abort cancel;
  %end;
%mend slm_assert;

/*==== %slm_init : pin the model, ASSERT loopback + offline, assert the model is pulled,
       capture the model digest for provenance. base defaults to the Ollama loopback.    */
%macro slm_init(model=, base=http://127.0.0.1:11434, seed=42, backup=);
  %let SLM_BASE=&base; %let SLM_MODEL=&model; %let SLM_SEED=&seed; %let SLM_RC=0; %let SLM_BACKUP=&backup;
  /* GUARD: only a loopback host may ever be called (no egress) */
  %slm_assert(cond=%sysfunc(prxmatch(%str(/^http:\/\/(127\.0\.0\.1|localhost|\[::1\])(:\d+)?(\/|$)/i),&SLM_BASE)),
              msg=non-loopback endpoint blocked (&SLM_BASE) - egress not allowed, abort=Y);
  /* liveness probe (local only) */
  filename _slmv temp;
  proc http url="&SLM_BASE/api/version" method="GET" out=_slmv; run;
  %slm_assert(cond=(&SYS_PROCHTTP_STATUS_CODE=200), msg=local model server not reachable at &SLM_BASE - is it running?, abort=Y);
  filename _slmv clear;
  /* assert the exact model tag is already pulled (no silent download) */
  filename _slmt temp;
  proc http url="&SLM_BASE/api/tags" method="GET" out=_slmt; run;
  libname _slmj JSON fileref=_slmt;
  %local _has; %let _has=0;
  proc sql noprint; select count(*) into :_has trimmed from _slmj.models where upcase(name)=upcase("&SLM_MODEL"); quit;
  libname _slmj clear; filename _slmt clear;
  %slm_assert(cond=(&_has>0), msg=model "&SLM_MODEL" is not pulled (run: ollama pull &SLM_MODEL), abort=Y);
  /* capture the model digest (provenance / model-freeze) */
  filename _slms temp; filename _slmsi temp;
  data _null_; file _slmsi; put '{"model":"' "&SLM_MODEL" '"}'; run;
  proc http url="&SLM_BASE/api/show" method="POST" in=_slmsi out=_slms ct="application/json"; run;
  libname _slmj JSON fileref=_slms;
  proc sql noprint; select digest into :SLM_DIGEST trimmed from _slmj.details; quit;   /* may be .model_info; see README */
  libname _slmj clear; filename _slms clear; filename _slmsi clear;
  %put NOTE: [LOCALMIND] init OK - model=&SLM_MODEL seed=&SLM_SEED endpoint=&SLM_BASE (loopback, offline) digest=&SLM_DIGEST;
%mend slm_init;

/*==== %slm_esc : JSON-escape helper text for a request body (call inside a data step).  */
%macro slm_esc(src, dst);
  &dst = tranwrd(strip(&src),'\','\\'); &dst = tranwrd(&dst,'"','\"');
  &dst = tranwrd(&dst,'0A'x,' ');       &dst = tranwrd(&dst,'0D'x,' ');   /* no raw newlines */
%mend slm_esc;

/*==== %slm_chat : the core call. POST /api/chat with stream=false, temp 0 + seed, and a
       caller-supplied JSON-SCHEMA string (&schema) as the grammar constraint. Double-
       parses (envelope -> message.content is itself a JSON string -> parse again) into
       &out. &sys / &usr are escaped. num_predict generous so the JSON object isn't cut.  */
%macro slm_chat(out=slm_out, num_predict=512, num_ctx=8192);   /* reads globals _SLM_SYS _SLM_USR _SLM_SCHEMA */
  filename _req temp; filename _resp temp; filename _hdr temp;
  data _null_; file _req lrecl=60000; length line $58000 s $20000 u $20000;
    %slm_esc(symget('_SLM_SYS'), s); %slm_esc(symget('_SLM_USR'), u);
    line = cats('{"model":"', "&SLM_MODEL", '","stream":false,'
      , '"keep_alive":"-1","options":{"temperature":0,"seed":', "&SLM_SEED"
      , ',"top_k":1,"top_p":1,"repeat_penalty":1,"num_predict":', "&num_predict"
      , ',"num_ctx":', "&num_ctx", '},'
      , '"format":', symget('_SLM_SCHEMA'), ','
      , '"messages":[{"role":"system","content":"', strip(s), '"},'
      , '{"role":"user","content":"', strip(u), '"}]}');
    l=length(line); put line $varying58000. l;
  run;
  proc http url="&SLM_BASE/api/chat" method="POST" in=_req out=_resp headerout=_hdr ct="application/json"; run;
  %slm_assert(cond=(&SYS_PROCHTTP_STATUS_CODE=200), msg=local model HTTP &SYS_PROCHTTP_STATUS_CODE, abort=Y);
  /* envelope: message.content holds the schema JSON as a STRING */
  libname _o JSON fileref=_resp;
  %local _slmcontent _msgtab;
  %if %sysfunc(exist(_o.message)) %then %let _msgtab=_o.message; %else %let _msgtab=_o.root;
  data _null_; set &_msgtab; call symputx('_slmcontent', content, 'L'); run;
  libname _o clear;
  filename _c temp; data _null_; file _c lrecl=60000; length z $58000; z=symget('_slmcontent'); l=length(z); put z $varying58000. l; run;
  libname _p JSON fileref=_c;
  data &out; set _p.root; run;     /* flat object -> ROOT (or ALLDATA; see README) */
  libname _p clear;
  filename _req clear; filename _resp clear; filename _hdr clear; filename _c clear;
%mend slm_chat;

/*==== %slm_validate : the trust boundary. Check a label var against a pipe-delimited
       allowlist (UPCASE) and confidence in [0,1]; reject+fail-loud on any miss.          */
%macro slm_validate(ds=, field=, allow=, conf=confidence);
  %local bad; %let bad=0;
  data &ds; set &ds; length _v $200;
    _v=upcase(&field);
    /* EXACT whole-element compare vs the pipe list (indexw would split a value with a
       space, e.g. "MEDICAL CODING", and false-reject it) */
    _ok=0; do _k=1 to countw("&allow",'|');
      if strip(_v)=strip(upcase(scan("&allow",_k,'|'))) then _ok=1; end;
    if not _ok then do;
      put "ERROR: [LOCALMIND] off-allowlist &field=" _v " (allowed: &allow)"; call symputx('bad','1'); end;
    %if %length(&conf) %then %do;
      if not missing(&conf) and (&conf<0 or &conf>1) then do;
        put "ERROR: [LOCALMIND] &conf out of range=" &conf; call symputx('bad','1'); end;
    %end;
    /* provenance stamp on every row */
    slm_model="&SLM_MODEL"; slm_digest="&SLM_DIGEST"; slm_seed=&SLM_SEED; slm_temp=0; slm_endpoint="&SLM_BASE";
    drop _v _ok _k;
  run;
  %slm_assert(cond=(&bad=0), msg=model output failed validation - NOT writing anything, abort=Y);
%mend slm_validate;

/*==== %slm_classify : convenience = build a one-enum-field schema from &allow, call the
       model, and validate. Returns &out with the label (&field), confidence, rationale,
       and provenance columns. The flagship for triage/routing/labelling tasks.           */
%macro slm_classify(sys=, usr=, field=label, allow=, out=slm_out);
  /* _SLM_SYS / _SLM_USR carry the prompt text (set here if passed, else the caller set
     them via CALL SYMPUTX for per-row free text that may contain commas/quotes).         */
  %global _SLM_SCHEMA _SLM_SYS _SLM_USR;
  %if %length(%superq(sys)) %then %let _SLM_SYS=%superq(sys);
  %if %length(%superq(usr)) %then %let _SLM_USR=%superq(usr);
  /* build the enum-constrained JSON schema from the pipe-delimited allowlist */
  data _null_; length enumj $4000 sch $6000;
    do i=1 to countw("&allow",'|');
      enumj=cats(enumj, ifc(i>1,',',''), '"', strip(scan("&allow",i,'|')), '"'); end;
    sch=cats('{"type":"object","properties":{"', "&field", '":{"type":"string","enum":[', enumj, ']},'
      , '"confidence":{"type":"number"},"rationale":{"type":"string"}},'
      , '"required":["', "&field", '","confidence","rationale"],"additionalProperties":false}');
    call symputx('_SLM_SCHEMA', sch, 'L');
  run;
  %slm_chat(out=&out);
  %slm_validate(ds=&out, field=&field, allow=&allow);
  %put NOTE: [LOCALMIND] classify ok (&field in {&allow});
%mend slm_classify;

/*==== End slm_macros.sas. See slm_config.sas, triage_driver.sas, slm_companion.R, README. */

Walkthrough, part by part

Header block and design contract. The opening comment declares LOCALMIND's operating rule: SAS/R owns every number and check while the on-device model only classifies, extracts, routes, or drafts under schema-constrained output, a SAS validator, and a human gate. It fixes the naming convention (every macro is %slm_ prefixed) and the platform floor (Base SAS 9.4M5+ for PROC HTTP plus the JSON libname engine). It also states the three non-negotiables the code later enforces: the endpoint must be loopback so nothing leaves the box, reproducibility comes from temperature 0 plus a fixed seed plus a pinned model (tag, quantization, sha256 digest), and the model is treated as an untrusted source whose output is rejected rather than coerced. This matters because it is the governance charter the macros implement, not a description of aspiration.

Global macro variable declarations. The %global statement declares SLM_BASE, SLM_MODEL, SLM_SEED, SLM_DIGEST, SLM_RC, and SLM_BACKUP so the pinned configuration and run state survive across every macro call in the session. SLM_RC is the library return code that %slm_assert flips to 1 on failure, and SLM_DIGEST is the captured model digest stamped onto output for provenance. Making these global is what lets one %slm_init call configure the whole job and later macros read the same frozen settings.

%slm_assert. This is the fail-loud primitive: it takes cond=, msg=, and an abort= flag (default N), and when the condition is false it sets SLM_RC=1 and writes an ERROR: line to the log. If SLM_BACKUP holds an address, it also sends a high-importance failure email via a filename email fileref so a human is notified out of band. When called with abort=Y it issues %abort cancel to stop the job rather than let a bad run continue. Every downstream guard routes through this macro, so a violated invariant halts work instead of silently producing something.

%slm_init. This macro pins the run and enforces the offline guarantees before any classification happens. It records base (defaulting to the Ollama loopback http://127.0.0.1:11434), model, and seed (default 42) into the globals, then asserts with a prxmatch regex that the endpoint is a loopback host (127.0.0.1, localhost, or the IPv6 [::1]) and aborts otherwise, so egress is blocked in code and not merely by policy. It then probes /api/version for a 200 liveness check, calls /api/tags and counts models whose name matches SLM_MODEL in the parsed JSON to assert the exact tag is already pulled (no silent download), and posts to /api/show to capture the digest from the parsed details table into SLM_DIGEST for the model-freeze provenance. Each of these steps aborts on failure, so a job cannot proceed against a routable host, a dead server, an unpinned model, or an unknown digest.

%slm_esc. This helper JSON-escapes free text before it is placed in a request body, and it is meant to be called inside a data step against a source variable src writing to dst. It backslash-escapes existing backslashes first, then double quotes, then replaces raw line-feed (0Ax) and carriage-return (0Dx) bytes with spaces so no raw newline breaks the JSON. It matters because prompt text is participant or study free text that routinely contains quotes and line breaks, and an unescaped body would produce malformed JSON or, worse, let content alter the request structure.

%slm_chat. This is the core call that POSTs to /api/chat with stream=false. It builds the request body in a data step, escaping the system and user prompts (read from globals _SLM_SYS and _SLM_USR) via %slm_esc, and it hard-codes the reproducibility and load options: temperature 0, the fixed seed, top_k 1, top_p 1, repeat_penalty 1, keep_alive -1, and the num_predict and num_ctx parameters (defaults 512 and 8192) so the JSON object is not truncated. It passes the caller's JSON schema (_SLM_SCHEMA) as the server-side grammar constraint via the format field, asserts a 200 status, then double-parses the response: the outer envelope's message.content (falling back to the JSON root table if no message table exists) is itself a JSON string, so it is written back to a fileref and parsed again into the output dataset named by out= (default slm_out). This double-parse is why the model's constrained answer lands as ordinary SAS columns rather than as an opaque string.

%slm_validate. This macro is the trust boundary that runs after the model answers. It re-checks the field value against the same pipe-delimited allowlist the schema used, comparing whole elements after upcase and strip (deliberately not indexw, which would split a multi-word value like MEDICAL CODING and false-reject it), and it range-checks the confidence variable to lie in [0,1] when the conf= variable is present and non-missing. Any off-allowlist label or out-of-range confidence writes an ERROR: line, sets a local bad flag via call symputx, and triggers %slm_assert with abort=Y so nothing is written on a miss. It also stamps every row with provenance columns (slm_model, slm_digest, slm_seed, slm_temp fixed at 0, and slm_endpoint), which is what makes each labeled row auditable and re-runnable. Because the server grammar and this validator are independent, an untrusted model cannot slip a value past both.

%slm_classify. This is the flagship convenience macro for triage, routing, and labeling: it assembles a one-enum-field JSON schema from the pipe-delimited allow list, calls the model, and validates the result in one step. It sets the prompt globals _SLM_SYS and _SLM_USR via %superq when sys and usr are passed (otherwise the caller sets them by CALL SYMPUTX for per-row free text with commas or quotes), then builds a schema object requiring the field label, confidence, and rationale with additionalProperties:false. It invokes %slm_chat to get the output dataset and immediately runs %slm_validate against the same allowlist, so the enum constraint is enforced at both the server and the validator. The returned dataset carries the label, confidence, rationale, and the provenance columns, and the model here still only proposes a category: the allowlist, the validation, and any human review own what is accepted.

Closing pointer comment. The final comment marks the end of the file and points to its companions: slm_config.sas, triage_driver.sas, slm_companion.R, and the README. This tells the biostatistician that this file defines the macros but does not run a job on its own: configuration lives in slm_config.sas, an actual per-row triage loop lives in triage_driver.sas, and the R side handles its share of the deterministic work. Reading those alongside this file is how you go from the macro definitions to a runnable, validated triage pipeline.

The driver: deterministic checks first, then language only: triage_driver.sas

SAS · 67 lines · the real shipped file, in full — nothing elided.

triage_driver.sas is the runnable worked example for the LOCALMIND small-language-model (SLM) macro library: a scheduled QC-finding triage where validated SAS owns every number and every check, the local model only labels and drafts language, and a human approves before anything is written. It runs against a set of QC findings that already came out of validated work (Pinnacle 21 plus the study's own edit-checks and reconciliation), classifies each finding for routing and novelty, assembles a ranked worklist, and on human approval writes a Part 11 audit line per actioned row. The header comment states the contract plainly: nothing the model emits is written until a human approves it, and the model emits no number. The program is 68 lines and executes as an ordinary SAS program once the config and macro includes resolve.

/*====================================================================================
  triage_driver.sas  -  the worked example as runnable SAS: a scheduled QC-finding triage
  where SAS owns every number/check and a LOCAL small model only labels, routes, and drafts.
  ------------------------------------------------------------------------------------
  Flow : config -> macros -> init (loopback+pinned model) -> SAS owns the findings
         (deterministic) -> per finding, the SLM returns NEW/KNOWN + owner + draft (JSON,
         validated) -> assemble ranked worklist -> HUMAN approves -> write + Part 11 audit.
  Nothing the model emits is written until a human approves it. The model emits no number.
====================================================================================*/
%let ROOT = /sasdata/cp101/localmind;
%include "&ROOT/slm_config.sas";
%include "&ROOT/slm_macros.sas";

/* 1) PIN + go offline-safe: loopback guard, liveness, model-pulled, capture digest */
%slm_init(model=&CFG_MODEL, base=&CFG_BASE, seed=&CFG_SEED, backup=&CFG_BACKUP);

/* 2) SAS owns the findings. These come from validated work (Pinnacle 21 + the study's
      own edit-checks / reconciliation). The MODEL NEVER PRODUCES A COUNT. Here: a stub. */
data findings;
  length finding_id $12 rule $20 findtext $600;
  finding_id="SD0064"; rule="AE date order"; findtext="AESTDTC is after AEENDTC for 3 records in domain AE."; output;
  finding_id="SD0011"; rule="Required var";  findtext="LBSTRESN missing for 12 records where LBSTRESC is numeric."; output;
  finding_id="PK0003"; rule="Timing dev";    findtext="ADPC actual sampling time deviates >10% from nominal at the 4h timepoint for 2 participants."; output;
run;

/* 3) Per finding, ask the local model THREE narrow things as schema-constrained JSON,
      each validated against a controlled vocabulary. Set the per-row text via SYMPUTX so
      free text (commas/quotes) is safe. */
%macro triage(ds=findings, out=worklist);
  %local n i; proc sql noprint; select count(*) into :n trimmed from &ds; quit;
  proc datasets nolist lib=work; delete &out; quit;
  %do i=1 %to &n;
    data _null_; set &ds(firstobs=&i obs=&i);
      call symputx('fid', finding_id); call symputx('_SLM_USR', findtext, 'G');  /* global so %slm_chat reads it */
    run;
    /* (a) new-vs-known, (b) owner routing - two constrained classifications */
    %slm_classify(sys=%nrstr(You triage ONE clinical-data QC finding for routing. Use only the schema. Never invent or change any number.),
                  field=owner, allow=&CFG_OWNERS, out=_own);
    %slm_classify(field=novelty, allow=&CFG_NOVELTY, out=_nov);   /* reuses the same _SLM_USR/_SLM_SYS */
    data _row; merge _own(keep=owner confidence rationale slm_model slm_digest slm_seed)
                     _nov(keep=novelty); length finding_id $12; finding_id="&fid"; run;
    data &out; set %if &i>1 %then &out; _row; run;
  %end;
%mend triage;
%triage(ds=findings, out=worklist);

/* 4) SAS assembles + ranks the worklist deterministically (severity/aging - SAS's math,
      not the model's). Join back the SAS-owned facts; the model's label rides ALONGSIDE. */
proc sql; create table worklist2 as
  select f.finding_id, f.rule, f.findtext, w.novelty, w.owner, w.confidence, w.rationale,
         w.slm_model, w.slm_digest
  from findings f left join worklist w on f.finding_id=w.finding_id
  order by w.owner, f.finding_id; quit;

/* 5) HUMAN GATE: the biostatistician reviews worklist2 (SAS facts beside the model label
      + draft) and sets approved=1 on the rows to action. (Here: assume reviewed.)        */
data approved; set worklist2; approved=1; /* <- set by the reviewer, not the model */ run;

/* 6) Only approved rows are written, each with a Part 11 audit line incl. the model digest.
      The query text of record is issued by SAS to the EDC; the model's draft was advisory. */
data _null_; set approved; where approved=1;
  file "&ROOT/audit/triage_audit.txt" mod;
  put finding_id +1 owner +1 novelty +1 "model=" slm_model +1 "digest=" slm_digest +1
      "seed=&CFG_SEED temp=0 reviewer=&SYSUSERID datetime=" "%sysfunc(datetime(),e8601dt.)";
run;
%slm_assert(cond=(&SLM_RC=0), msg=LOCALMIND triage finished with errors - see log);
%put NOTE: [LOCALMIND] triage complete - SAS owned the numbers, the model only labelled/drafted, a human approved.;

Walkthrough, part by part

Header comment and the ownership contract. The opening block comment states what the program is: the worked example as runnable SAS, a scheduled QC-finding triage in which SAS owns every number and check and a local small model only labels, routes, and drafts. It lays out the flow in order: config, macros, init (loopback plus pinned model), SAS owns the deterministic findings, per finding the SLM returns a NEW/KNOWN novelty plus owner plus a draft as validated JSON, assemble a ranked worklist, a human approves, then write plus a Part 11 audit. Two sentences carry the governance that the rest of the file enforces: nothing the model emits is written until a human approves it, and the model emits no number. This matters because it fixes the boundary up front, the model produces language only and the biostatistician stays the decision-maker.

ROOT macro variable and the config and macro includes. %let ROOT pins the on-disk base path /sasdata/cp101/localmind for this study's LOCALMIND install, and the two %include statements pull in slm_config.sas and slm_macros.sas from under that root. The config include defines the CFG_* settings referenced later (model name, base URL, seed, backup, and the controlled vocabularies CFG_OWNERS and CFG_NOVELTY), while the macro include defines the %slm_* macros this driver calls. Resolving ROOT once and building every later path from &ROOT keeps the program relocatable to another study directory by editing a single line. Nothing here talks to the model yet, it only stages configuration and the macro library.

Step 1, slm_init: pin the model and go offline-safe. %slm_init is called with the config values (model=&CFG_MODEL, base=&CFG_BASE, seed=&CFG_SEED, backup=&CFG_BACKUP). Per the comment on the numbered step, it pins the model and goes offline-safe: it applies a loopback guard (the endpoint must be local, not an external service), checks liveness, confirms the model is pulled, and captures the model digest. Pinning a specific model plus a fixed seed and capturing the digest is what makes a run reproducible and auditable, the same frozen local model answers the same way and the digest later lands in the audit line. This is the step that keeps the small model on the participant-data-safe local host rather than reaching out over the network.

Step 2, the findings data step (SAS owns the counts). The data findings step declares typed columns finding_id, rule, and findtext, then hard-codes three example rows and outputs each. The step comment is load-bearing: these findings come from validated work (Pinnacle 21 plus the study's own edit-checks and reconciliation), and the model never produces a count, this is only a stub standing in for that validated source. Every number that appears (AESTDTC after AEENDTC for 3 records, LBSTRESN missing for 12 records, a greater-than-10% timing deviation for 2 participants at the 4h timepoint) originates in SAS and its validated inputs, not the model. In a real run you would replace this stub with a read of the actual validated findings dataset, and the counts would still be SAS-owned.

Step 3, the triage macro definition. %macro triage(ds=findings, out=worklist) defines the per-finding loop that asks the local model narrow things as schema-constrained, controlled-vocabulary-validated JSON. It declares locals n and i, uses proc sql to count the rows of &ds into &n, and runs proc datasets to delete any prior &out so the run starts clean. The comment states the design intent: the model is asked only to classify and draft, each answer validated against a controlled vocabulary, and per-row free text is passed by SYMPUTX so commas and quotes in findtext are safe. The macro contains no arithmetic on the findings, it routes and labels, it does not recompute or reconcile any count.

Step 3, per-row setup inside the loop (fid and _SLM_USR). The %do i=1 %to &n loop reads exactly one finding per iteration via set &ds(firstobs=&i obs=&i). A data _null_ step uses call symputx to place that row's finding_id into macro variable fid and its findtext into _SLM_USR with the 'G' (global) scope so %slm_chat can read it as the user prompt. Using symputx rather than string-building the prompt inline is deliberate: it moves the finding's free text (which contains commas, quotes, and comparison operators) into a macro variable safely, avoiding quoting breakage. The finding text becomes the model's input, but the model is being asked to classify it, not to trust or restate its numbers as fact.

Step 3, the two slm_classify calls (owner and novelty). Two %slm_classify calls perform the constrained classifications. The first sets field=owner with allow=&CFG_OWNERS and output out=_own, and passes a system prompt (via %nrstr) instructing the model to triage one clinical-data QC finding for routing, use only the schema, and never invent or change any number. The second sets field=novelty with allow=&CFG_NOVELTY and out=_nov, and its comment states it reuses the same _SLM_USR and _SLM_SYS from the first call. Constraining each answer to the allow vocabulary is what keeps the model's output a validated label from a fixed set rather than free-form text, and the explicit no-number instruction reinforces that the model routes and names, it does not decide a value.

Step 3, merging the per-row result and appending to the worklist. A data _row step merges _own (keeping owner, confidence, rationale, slm_model, slm_digest, slm_seed) with _nov (keeping novelty), declares finding_id with length 12, and stamps it from &fid. The next step appends _row to &out, reading the existing &out first only when &i>1 so the first iteration creates the table and later ones grow it. Carrying slm_model and slm_digest on every row is what lets the later worklist and audit record which pinned model produced each label. The result of the macro is one worklist row per finding, each row holding the model's labels beside the provenance fields, and %triage(ds=findings, out=worklist) invokes it.

Step 4, SAS assembles and ranks the worklist deterministically. A proc sql creates worklist2 by left-joining the SAS-owned findings table to the model-labelled worklist on finding_id, keeping the SAS facts (finding_id, rule, findtext) alongside the model outputs (novelty, owner, confidence, rationale, slm_model, slm_digest), ordered by owner then finding_id. The comment is explicit that ranking (severity and aging) is SAS's math, not the model's, and that the model's label rides alongside the SAS-owned facts rather than replacing them. Driving the join from findings means every validated finding survives even if a model label is missing, the deterministic side stays the spine of the table. Ordering and any severity or aging logic live in SAS, keeping the worklist's structure reproducible and independent of the model.

Step 5, the human gate. A data approved step copies worklist2 and sets approved=1, with an inline comment stating this flag is set by the reviewer, not the model, and that the current line only assumes the rows were reviewed. This is the point where the biostatistician reads each row (the SAS facts beside the model's label and draft) and decides which findings to action. In real use this hard-coded assignment is replaced by the reviewer's actual per-row decision, the program deliberately stops short of automating it. This gate is the mechanism that keeps a human owning every decision, no model output becomes actionable without an explicit human approval.

Step 6, write approved rows with a Part 11 audit line. A data _null_ step reads approved, subsets to approved=1, and appends to &ROOT/audit/triage_audit.txt using file with the mod option. Each put writes the finding_id, owner, and novelty, then model= with slm_model and digest= with slm_digest, then a literal seed=&CFG_SEED temp=0, the reviewer via &SYSUSERID, and an ISO 8601 datetime from %sysfunc(datetime(),e8601dt.). The comment states the query text of record is issued by SAS to the EDC and the model's draft was only advisory, so what gets written is the SAS-owned action, annotated with which pinned model advised it and who approved. Only approved rows are written, and each carries a reproducibility-and-attribution trail (model, digest, seed, temperature, reviewer, timestamp) suitable for a Part 11 audit.

Closing assertion and completion note. %slm_assert(cond=(&SLM_RC=0), msg=LOCALMIND triage finished with errors - see log) checks the library's return code &SLM_RC and fails loudly if the triage finished with errors, pointing to the log. The final %put NOTE: restates the invariant in one line: SAS owned the numbers, the model only labelled and drafted, and a human approved. Ending on an asserted return code plus a plain-language restatement of the contract makes the run self-documenting, a reader of the log sees both whether it succeeded and what division of responsibility it enforced. Together these two lines close the program on the same governance stance it opened with: validated tools own the numbers and checks, the model owns language only, and a human owns the decision.

The same glue in R: slm_companion.R

R · 71 lines · the real shipped file, in full — nothing elided.

slm_companion.R is the R twin of the LOCALMIND SAS macro library (slm_macros.sas): it calls a local, offline small language model (via Ollama on the loopback interface) for the language layer only, using httr2 for HTTP and jsonlite for JSON. It runs when a validated SAS/R pipeline has already produced numbers and checks and you want the model to draft or label text about a finding, never to compute, reconcile, or decide a number. The file enforces the same contract as the SAS side: loopback-only (no egress), schema-constrained output, an allowlist validator, deterministic decoding (temperature 0 plus a fixed seed), a pinned model, and a human gate. Every reported number comes from the validated SAS/R tools and every decision belongs to a person; the model only supplies language.

# =====================================================================================
# slm_companion.R  -  LOCALMIND R companion: call a LOCAL, OFFLINE small language model
# (Ollama) for the language layer only. httr2 + jsonlite. Same contract as slm_macros.sas:
# loopback-only (no egress), schema-constrained output, allowlist validator, temp0+seed,
# pinned model, human gate. The model NEVER produces a number.
# =====================================================================================
suppressPackageStartupMessages({ library(httr2); library(jsonlite) })

# ---- pin the model + assert loopback/offline + assert the model is pulled -------------
slm_init <- function(model, base = "http://127.0.0.1:11434", seed = 42L) {
  if (!grepl("^http://(127\\.0\\.0\\.1|localhost|\\[::1\\])(:[0-9]+)?(/|$)", base))
    stop("[LOCALMIND] non-loopback endpoint blocked: ", base, " - egress not allowed", call. = FALSE)
  live <- tryCatch(request(base) |> req_url_path_append("api", "version") |> req_timeout(5) |>
                     req_perform() |> resp_status(), error = function(e) 0)
  if (!identical(live, 200L)) stop("[LOCALMIND] local model server not reachable at ", base, call. = FALSE)
  tags <- request(base) |> req_url_path_append("api", "tags") |> req_perform() |> resp_body_json()
  names <- vapply(tags$models, function(m) tolower(m$name), character(1))
  if (!tolower(model) %in% names)
    stop("[LOCALMIND] model '", model, "' is not pulled (run: ollama pull ", model, ")", call. = FALSE)
  show <- request(base) |> req_url_path_append("api", "show") |> req_method("POST") |>
    req_body_json(list(model = model)) |> req_perform() |> resp_body_json()
  digest <- show$details$digest %||% show$model_info$general.digest %||% NA_character_
  structure(list(model = model, base = base, seed = as.integer(seed), digest = digest), class = "slm")
}

# ---- the trust boundary: validate a parsed result against an allowlist + range --------
slm_validate <- function(x, field, allow, conf = "confidence") {
  v <- toupper(as.character(x[[field]]))
  if (!v %in% toupper(allow))
    stop(sprintf("[LOCALMIND] off-allowlist %s: %s (allowed: %s)", field, v, paste(allow, collapse = ", ")), call. = FALSE)
  if (!is.null(x[[conf]]) && (!is.numeric(x[[conf]]) || x[[conf]] < 0 || x[[conf]] > 1))
    stop(sprintf("[LOCALMIND] %s out of range: %s", conf, x[[conf]]), call. = FALSE)
  invisible(TRUE)
}

# ---- classify/route/label: build a one-enum schema, call locally, validate, stamp -----
# sys = system instruction; usr = the SHORT finding text. allow = character vector of labels.
slm_classify <- function(con, sys, usr, field = "label", allow, num_predict = 512L, num_ctx = 8192L) {
  stopifnot(inherits(con, "slm"))
  # build the enum-constrained JSON schema (set the label field by name)
  props <- list(); props[[field]] <- list(type = "string", enum = as.list(allow))
  props$confidence <- list(type = "number"); props$rationale <- list(type = "string")
  schema <- list(type = "object", properties = props,
                 required = as.list(c(field, "confidence", "rationale")), additionalProperties = FALSE)

  resp <- request(con$base) |> req_url_path_append("api", "chat") |>
    req_body_json(list(model = con$model, stream = FALSE, keep_alive = "-1",
      options = list(temperature = 0, seed = con$seed, top_k = 1L, top_p = 1, repeat_penalty = 1,
                     num_predict = num_predict, num_ctx = num_ctx),
      format = schema,                                   # server-side grammar constraint
      messages = list(list(role = "system", content = sys),
                      list(role = "user",   content = usr))), auto_unbox = TRUE) |>
    req_timeout(180) |> req_perform() |> resp_body_json()

  out <- fromJSON(resp$message$content)                  # message.content is a JSON string -> parse again
  slm_validate(out, field = field, allow = allow)        # reject -> stop() (fail loud, no write)
  c(out, list(slm_model = con$model, slm_digest = con$digest, slm_seed = con$seed,
              slm_temp = 0, slm_endpoint = con$base))    # provenance
}

`%||%` <- function(a, b) if (is.null(a) || length(a) == 0 || is.na(a[1])) b else a

# ---- example -------------------------------------------------------------------------
if (sys.nframe() == 0) {
  con <- slm_init("llama3.1:8b-instruct-q8_0")
  r <- slm_classify(con,
    sys = "You triage a CDISC conformance finding. Reply only as the schema. Do not invent numbers.",
    usr = "SD0064: AESTDTC is after AEENDTC for 3 records in domain AE.",
    field = "owner", allow = c("DM", "PROGRAMMING", "PK", "MEDICAL CODING", "SITE"))
  str(r)
}

Walkthrough, part by part

Header contract and dependencies. The banner comment fixes the operating contract in one place: the R companion talks to a LOCAL, OFFLINE model through Ollama, handles the language layer only, and mirrors slm_macros.sas point for point (loopback-only, schema-constrained output, allowlist validator, temp0 plus seed, pinned model, human gate). The last line of the banner is the load-bearing rule: the model NEVER produces a number. The single executable line loads httr2 and jsonlite inside suppressPackageStartupMessages so package chatter does not contaminate logs or captured output.

slm_init: pin the model and prove the endpoint is local, live, and pulled. slm_init takes a model name, a base URL defaulting to http://127.0.0.1:11434, and a seed defaulting to 42L. It has three hard gates that call stop(): first a regex check that base is a loopback address (127.0.0.1, localhost, or the bracketed IPv6 form), which is the egress block that keeps participant-adjacent text on the machine; second a check that the server answers status 200 on api/version; third a check that lists pulled models via api/tags and errors with a copy-pasteable ollama pull hint if the requested model is absent. After those gates pass it reads api/show to capture a model digest, falling back through show$details$digest, then show$model_info$general.digest, then NA_character_ if neither is present (so a missing digest does not halt). It returns an object of class slm holding model, base, the integer seed, and digest, so downstream calls carry a pinned, identifiable model rather than a moving target.

slm_validate: the trust boundary against an allowlist and a confidence range. slm_validate is where model output stops being trusted text and must earn its way through a gate; it takes the parsed result x, a field, an allow set, and a conf key name defaulting to "confidence". It uppercases the value at field and calls stop() if it is not in the uppercased allow set, so the model can only ever return one of a closed list of labels, never free-form or invented categories. If the conf key is present it must be numeric and within the 0 to 1 range, otherwise the function errors; the function returns invisible(TRUE) on success. This is a deliberate fail-loud design: an off-allowlist or out-of-range result raises rather than silently passing, and because the model emits only labels and a rationale, this validator never touches or blesses a reported number.

slm_classify: build an enum schema, call locally, validate, and stamp provenance. slm_classify is the one working entry point: given an slm connection con, a system instruction sys, the short finding text usr, a target field (default label), an allow vector, and generation caps num_predict (default 512L) and num_ctx (default 8192L), it constructs a JSON schema whose field is a string constrained to enum = allow, plus a numeric confidence and a string rationale, all three listed in required and with additionalProperties = FALSE. It posts to api/chat with deterministic decoding options (temperature = 0, the pinned seed, top_k = 1L, top_p = 1, repeat_penalty = 1) alongside num_predict and num_ctx, passes the schema as format so the server enforces the grammar, sets keep_alive = "-1" to hold the model resident, and applies a 180-second request timeout. It parses resp$message$content a second time with fromJSON (the content arrives as a JSON string), runs slm_validate so a bad answer halts before anything is written, and returns the validated result augmented with provenance fields (slm_model, slm_digest, slm_seed, slm_temp set to 0, and slm_endpoint). The design keeps the model in its lane: it routes or labels a finding and explains why, while every number and every final decision stays with the validated SAS/R tools and the human reviewer.

The null-coalescing helper %||%. %||% is a small infix operator that returns its left operand unless that operand is NULL, has length 0, or has an NA first element, in which case it returns the right operand. It is defined here specifically to back the digest fallback chain in slm_init, where the digest may live under show$details$digest or show$model_info$general.digest or be absent (NA_character_). Defining it near the bottom is fine because R resolves the function at call time, not at parse time, so slm_init can reference it even though the operator appears later in source order.

The guarded example block. The trailing block runs only when the file is sourced as the top-level script (sys.nframe() == 0), so importing these functions into a larger program does not trigger a live model call. It demonstrates the intended flow: pin llama3.1:8b-instruct-q8_0, then triage a CDISC conformance finding into an owner drawn from the allowlist c("DM", "PROGRAMMING", "PK", "MEDICAL CODING", "SITE"), with a system instruction that explicitly tells the model to reply only as the schema and not to invent numbers. The example targets field = "owner" rather than the default label, showing the same function routes a finding to a responsible team, and str(r) prints the validated, provenance-stamped result for a human to review before any action is taken.

Scheduling it: the whole setup, once

The token and any credentials are read at run time from an environment variable or a permissions-600 file, never hard-coded and never written to a log. Point the driver at your install (the %let paths and the config file), dry-run once, then schedule it under a dedicated service account, after your validated nightly extract refreshes.

• SAS on Linux (cron): 0 2 * * * sas -nodms -sysin slm/triage_driver.sas -log slm/log/monitor_$(date +\%F).log
• SAS on Windows (Task Scheduler): a daily task running sas.exe -sysin C:\slm\triage_driver.sas -log C:\slm\log\monitor.log
• R: cronR::cron_add(command = cronR::cron_rscript("slm_companion.R"), frequency = "daily", at = "02:00"), or taskscheduleR::taskscheduler_create() on Windows. Pin packages with renv.
• Dry-run first, then trust the heartbeat: the job fails loud on any error and emits a completion NOTE, so a stale run announces itself instead of masquerading as fresh.

A morning of on-device triage — one offline workstation

A scheduled SAS/R QC run plus a small local model doing only the language layer, fully offline: SAS/R computes the findings, the model labels and drafts, SAS/R validates, the biostatistician approves, SAS/R writes and audits.

What one scheduled run prints to the log

There is nothing to watch in a UI, and nothing leaves the workstation. Every step is an auditable line from the shipped %slm_* library. SAS/R owns the numbers; the local model only drafts language, and the validator refuses anything off the allowlist. Read the run top to bottom.

# triage_driver.sas, run on one offline workstation; nothing leaves 127.0.0.1
NOTE: [LOCALMIND] init OK - model=qwen2.5:7b-instruct-q8_0 seed=42 endpoint=http://127.0.0.1:11434 (loopback, offline) digest=sha256:9f2c8ab1
# 1) SAS/R computes the numbers first; the model is handed a finding to label, never a record to score
NOTE: [LOCALMIND] classify ok (owner in {DM|PROGRAMMING|PK|MEDICAL CODING|SITE|BIOSTAT})
NOTE: [LOCALMIND] classify ok (severity in {CRITICAL|MAJOR|MINOR})
# the SAS/R validator refuses any off-allowlist label the model emits:
ERROR: [LOCALMIND] off-allowlist owner=SPONSOR (allowed: DM|PROGRAMMING|PK|MEDICAL CODING|SITE|BIOSTAT)   # refused in code, queued for a human
NOTE: [LOCALMIND] triage complete - SAS owned the numbers, the model only labelled/drafted, a human approved.

Silence is never assumed: a failed run mails the fail-loud address, and the external watchdog alerts if the heartbeat stops.

The run, beat by beat

SAS/R computes the numbers first. The deterministic checks run exactly as they do without any model: SAS/R owns every count, threshold, and flag. The on-device model is never handed a raw participant record to score, and it never produces a reported number.

The local model drafts language, offline. For each flag the pinned model (a quantized open-weight model on 127.0.0.1, seed fixed, temperature 0) drafts a short label and narrative under a constrained JSON schema. It runs on loopback only, so participant-adjacent text never leaves the workstation, and the fixed seed makes the draft reproducible.

The validator checks every field against an allowlist. Before anything reaches a person, a SAS/R validator checks each drafted field against a controlled vocabulary (owner, severity, novelty). An off-list value, such as an invented SPONSOR owner, is refused in code, not passed through. This is the same in-code trust boundary as the monitoring allowlist.

A human approves before any write. The accepted drafts are queued for a person to confirm or edit; nothing is auto-sent and nothing is written of record. The model labelled and drafted, the validated tools owned the numbers, and a human made every decision.

Governance & honest limits

The governance is the same as every companion in this program; only the model changes. The line that does not move: SAS/R and the validated engines own every number and every record of truth. The small on-device model never produces a value, never reconciles, never decides. It classifies, extracts, routes, and drafts words — and a human approves before anything is written. Swapping cloud Claude for a small (roughly 1–9B-parameter), quantized model on a workstation tightens the data-egress story; it does not loosen the controls.

What governs the small model

• SAS/R owns the truth. Every number, check, reconciliation, and stat is deterministic SAS/R; every reported/regulated value comes from a validated engine. The model is never in the number path.
• The model only handles words. Its job is bounded: label into a fixed bucket set, extract a field from a short passage, tag an owner/queue/severity, group near-duplicates, or draft a short templated text grounded in retrieved spec. Nothing more.
• Constrained output, then validated. The model is forced toward well-formed JSON by grammar-constrained decoding (llama.cpp GBNF / JSON-schema, or Ollama structured output), which masks tokens that violate the schema during sampling. This is not a guarantee — constrained decoding can still emit a truncated or incomplete object if generation runs out of tokens — so a deterministic SAS/R validator independently parses the output, checks every field against the allowed values, and drops or flags anything malformed, incomplete, or out-of-set. Free text is never trusted as-is.
• Human gate before any write. Every model-touched item (a label, a route, a draft query) lands in an approval queue. A biostatistician approves; only then does SAS/R write and audit.
• Pinned, version-controlled model. Pin the model name + quantization + weight digest, fix the decoding policy (greedy / temperature 0), and version the prompt and the RAG sources. That tuple is a frozen, re-runnable configuration — the qualified instrument is defined exactly and cannot change underneath you without the digest changing. Note the honest caveat below: pinning makes the configuration reproducible, not necessarily the token-for-token output.
• Part 11 audit trail. SAS/R logs the prompt, the pinned model digest, the model output, the validator result, and the human decision (who, when, approve/reject) for every item. The decision record is inspectable end-to-end.
• Local-only, network-isolated. The model and runtime live on the workstation; the endpoint is loopback (127.0.0.1) and the box is held on an isolated/air-gapped network by configuration and policy. This is an enforced, verifiable control, not a law of physics — the machine still has hardware interfaces, so the control is “no outbound path is configured or permitted, and that is auditable,” not “egress is impossible.” It is still the simplest data-governance story of any AI option in the program: there is no cloud endpoint to trust, no BAA to negotiate, and no egress firewall to maintain.

Roles, stated plainly

A note on the model licences

“Open-weight” is not the same as OSI “open-source,” and the difference matters to legal and QA. Confirm the licence per model before qualification: Qwen3, Gemma 4, Phi-4-mini (MIT), and IBM Granite 4.1 ship under genuinely permissive (Apache 2.0 / MIT) terms, whereas Llama 3.3/3.2 carries the Meta Llama Community License (custom, with an acceptable-use policy and a >700M-MAU clause) — redistributable for our use but not OSI-approved, with flow-down/use restrictions. None of this changes the data-governance story (the weights run locally either way); it is a procurement and IP checklist item, not a control on the data.

Validate before production

Treat the pinned model as a qualified analytical instrument, not software you trust on faith. Risk-based GAMP 5 qualification: installation qualification on the runtime, operational qualification of the pinned model against a fixed test set (does the constrained output hold; does the validator catch the bad cases), and performance qualification on the real workflow with the human gate live. Validate against a sandbox first; promote to production only after the pinned-model configuration passes on representative study data. Re-qualify on any change to the model, quantization, runtime, decoding policy, or prompt — the digest or configuration changes, so the qualified state changes.

FAQ

Is a small AI model safe to use in regulated work? Only in the narrow shape we deploy it: the on-device model never touches a record of truth, never produces a number, and never decides anything. SAS/R does the deterministic work and writes the audit log; the model emits schema-constrained output that a SAS/R validator checks against an allowlist; a human approves before anything is committed. Inside that envelope — short classification, extraction from a short passage, routing, near-duplicate grouping, templated drafting grounded in retrieved text — residual risk is low but not zero, so the human gate stays. Outside it, the model is the wrong tool, which is why 15 of the 75 components are marked Beyond.

Does the small model touch any of the numbers? No. SAS/R owns every count, check, reconciliation, and statistic, and reported or regulated values come only from validated engines — Pinnacle 21, Phoenix WinNonlin, the EDC/CTMS. A small quantized model is the least trustworthy thing in the building for anything quantitative; it hallucinates more than a frontier model and is more sensitive to prompt and format. So it is fenced out of arithmetic entirely. It labels, extracts short fields, groups duplicates, and drafts a query or note for a human to confirm — nothing else.

How is this different from the cloud-Claude hybrid? Same division of labor, very different capability and sovereignty trade. The hybrid sends de-identified language work to a frontier model that can reason across documents, hold long context, and write nuanced narrative — so its Beyond list is short. This option uses a 1–9B model that runs offline on hardware you already own, so the data-residency story is the simplest of any option, but the capability ceiling is real: the work the hybrid sends to Claude is exactly the work a small model can't do reliably. A small quantized model also degrades well before its advertised context window, so “it has a long context” is not the same as “it reasons well over one.” Pick on-device where sovereignty and reproducibility dominate and the language task is genuinely small; escalate to the hybrid when the language work needs real reasoning.

And how is it different from the M365 Copilot option? Copilot is a cloud service with no on-prem model and no model freeze — good for de-identified ops, but it can't keep unblinded or participant-level text on the box and can't be pinned for re-runnable reproducibility. The on-device SLM inverts that: it can run on a network-isolated box (loopback-only client config, plus no egress route at the infrastructure layer), and it is straightforward to pin and freeze, at the cost of the model's reasoning power. Different ends of the same spectrum — Copilot trades sovereignty for capability; the local SLM trades capability for sovereignty.

What hardware do we actually need? Less than people expect. A 3–8B model at 4–8 bit quantization (GGUF) runs on an ordinary workstation or laptop — CPU works, a single consumer GPU is faster — served locally by Ollama, llama.cpp, or LM Studio. No server farm, no cloud account. The constraint is throughput, not feasibility: keep the prompts short and the batches sized to the box. If a task needs a model big enough to need real GPU infrastructure, that is a signal the task is Beyond a small model in the first place.

Is it validated and reproducible? The pipeline is built to be, with an honest definition of “reproducible.” We pin the model name, quantization, and digest; set decoding to temperature 0 with a fixed seed; fix the context length and other decode options; and freeze the prompt — together a re-runnable artifact. Output is deterministic only within a pinned environment: the same runtime build, the same hardware and backend, the same context settings. Across different builds or GPUs, kernel and batching differences can still shift output, so we don't claim bitwise reproducibility on arbitrary hardware. That is fine, because the regulatory burden does not rest on the model: under GAMP-5 and 21 CFR Part 11, validation lives in the SAS/R validator, the allowlist, the audit trail, and the human gate. The model is a frozen, constrained language sidecar, not a validated instrument and never treated as one.

What happens to the data? It stays on the workstation. SAS/R calls the model at a local 127.0.0.1 endpoint (PROC HTTP / httr2), and the box runs with no telemetry and no egress — nothing leaves. The loopback-only client config is necessary but not sufficient on its own; the real control is that the box has no route off the network, which is an infrastructure control we assert and inspect, not just a line of code. That is the whole point of the on-device choice: the data-sovereignty control collapses from “firewall the cloud egress” to “the box stays offline.” PHI, unblinded, and participant-level text are safe here precisely because there is nowhere for them to go.

Which model should we start with? A current 7–9B instruct model at 4–8 bit, with a 3B fallback for the lightest classification tasks. Check the licence before you commit, because “open-weight” is not “open-source”: Qwen3, Gemma 4 and IBM Granite 4.1 8B ship under Apache 2.0, while Llama 3.3/3.2 (Meta Community License + Acceptable Use Policy, MAU cap, flow-down terms) carries a custom restrictive licence that legal should clear for our use. Bigger-is-better is the correct instinct for the language work: don't reach for a 1.5B model to save memory if an 8B one clears the task. Pick one, pin model+quantization+digest, validate it against the allowlist on your own examples, and freeze it. The exact name matters less than locking it down once it works.

Walkthrough transcript

The complete narration of the “morning of on-device triage” video, as readable text.

1. Part 1

This is one overnight cycle on Study CP-101, a Phase 1 clin-pharm study. A scheduled SAS or R job runs the deterministic checks and Pinnacle 21 produces the conformance findings, exactly as they do today. The only thing that is new is a small open-weight model running locally on the workstation, called by the job to handle the language layer — labeling each finding, sorting it into a queue, and drafting a candidate query. SAS/R owns every number; the model only helps with words; and a human approves before anything touches a record of truth.

2. Part 2

The setup is deliberately boring. An ordinary workstation you already own runs Ollama, serving one quantized model — say Qwen2.5 7B at Q4_K_M, Apache-2.0 licensed — on the loopback address 127.0.0.1. The model name, quantization, and digest are pinned and registered, decoding is fixed at temperature zero with a set seed, so the same input gives the same output every run. The box stays offline: there is no cloud account, no telemetry, and the network cable is unplugged — the data-egress story is simply that nothing can leave.

3. Part 3

Overnight, the scheduled job runs the validated work. Pinnacle 21 executes the CDISC conformance run over SDTM and Define-XML, and the study's own SAS or R programs run the cross-form edit checks, the PK timing and BLQ rules, and the lab and IXRS reconciliation. This morning it produces two hundred fourteen findings. Every one of those counts, comparisons, and rule evaluations came from validated, double-programmed code — the small model has not been called yet and will never compute a number.

4. Part 4

Now SAS/R calls the local model, one finding at a time, using PROC HTTP or httr2 to the loopback endpoint. For each short finding it asks three narrow, in-envelope questions: is this new or a known recurring issue, what type and severity bucket does it fall in, and which owner or queue should it route to. The model must answer as JSON constrained to a fixed schema with allowed values only — no free prose, no numbers. This is short-text classification and routing, squarely in what a small quantized model can do reliably.

5. Part 5

Before trusting a single word, the SAS/R validator parses the JSON and checks every field against the allowed enum — any malformed or out-of-vocabulary answer is dropped and logged, never guessed. For findings that pass, the model drafts a one-line candidate query, but only grounded by retrieval over the study's own specifications and edit-check definitions, so the wording stays anchored to real source text. Every draft is labeled DRAFT and is unsent; SAS/R then assembles the survivors into a single ranked worklist, ordered by severity and aging.

6. Part 6

At the workstation the biostatistician opens the ranked worklist. Each row shows the deterministic facts from SAS/R — the rule, the records, the counts — next to the model's suggested label and its draft query text. The reviewer approves, edits, or rejects each one; a wrong label is a two-second fix, not a silent error. Nothing the model produced becomes real until a qualified human has signed off on it.

7. Part 7

Only the approved items get written. SAS/R issues the queries to the EDC and writes a 21 CFR Part 11 audit line for each one, recording the prompt, the pinned model name, quantization and digest, the draft text, and who approved it and when. Because the model is frozen and decoding is fixed, that audit line is a re-runnable artifact an inspector can reproduce. And the offline guarantee is physical, not a policy promise — the cable is unplugged, so the only thing that ever left the box was a query you approved into your own EDC.

8. Part 8

Step back and the division of labor is clean. SAS/R and Pinnacle 21 owned the data, every check and count, the validator, every write, and the audit trail. The small local model did one narrow language job — sorting findings and drafting candidate words — and it never touched a number or a record on its own. The biostatistician made every decision that mattered. That is the honest shape of an on-device win: the model saved triage time on a narrow, validated task, and the truth stayed exactly where it always was.