AI Radiograph → Insurance-Narrative Pipeline — Build Recipe

Take a bare machine to a running copy of an AI dental radiograph → insurance narrative pipeline. Upload a radiograph and a deterministic computer-vision pass detects + FDI-numbers the teeth (YOLOv8m), segments them (SAM 2.1), and runs a pathology stack (modality classifier, apex regressor, lesion / restoration / caries detectors). Those structured findings drive a tooth chart, image overlays, a rules-engine procedure suggester, a payer-specific documentation checklist, an approval-likelihood meter, and an audit-trail PDF — and, optionally, an LLM writes the final insurance narrative from the findings (it never sees the radiograph on the grounded path).

The system runs as two co-located containers on one compose network — a CV service and a narrative service. CPU-only — no GPU required.

Status: ✅ Verified 2026-08-01 — built from this clean clone with docker compose up --build. Both images build (cv ≈ 6.2 GB incl. ~1.1 GB of bundled weights + torch/SAM2; narrative ≈ 250 MB), both come up healthy, and the full pipeline ran end-to-end, key-free on a bundled CC BY 4.0 bitewing: the real detector found 10 FDI-numbered teeth with SAM masks and caries pathology (e.g. tooth #30 @ 0.33), the rules engine suggested ★ Core Buildup, the approval meter scored the claim at 58 % with a breakdown, the overlays rendered on the radiograph (see demo.png), and the WeasyPrint PDF export produced a valid PDF. The LLM-off gate returns a friendly notice, and with a local Ollama endpoint the full CV-grounded narrative generated live (_meta.path: "findings") — real radiograph → real CV → real prose, no API key. All three tiers are covered — the prebuilt tarball (docker save radiograph-cv radiograph-narrative, ~6069 MB, verified 2026-08-01), a from-source build, and bare-metal. The clean clone lives in clone/.

PHI-safety — read this. This is a dental-imaging system, so be explicit: no patient data ships, by construction. The narrative service is stateless — radiographs flow through RAM and are discarded; there is no database, no uploads dir, no bundled patient images. The author's CV research repo holds ~16 GB / ~100k radiograph-type images; that entire image surface is excluded — the clone ships only source code, the bundled model weights (learned tensors, no recoverable patient images), and exactly two public, CC BY 4.0 research radiographs (samples/, see ATTRIBUTIONS.md) — never a real patient image, never a bulk dataset. All secrets are placeholders.

What it is

Two services on one compose network:

• cv (cv/, FastAPI, port 8237, internal) — wraps a curated subset of the CV code + the production weights. POST /detect runs FDI detection → SAM masks → pathology and returns structured findings JSON. CPU-only (torch==2.11+cpu); a GPU is auto-detected if present.
• narrative (narrative/, FastAPI, port 8248, published) — the product: proxies images to cv, turns findings into a text prompt for an LLM, and adds the rules engine, 12 payer rulesets, a patient-context effect matrix, HITL review, overlays, the approval meter, and two WeasyPrint PDFs. No ML in-process.

Prerequisites

New machine? See ../SETUP.md for the base tools.

Tier 1 — Run the prebuilt container(s) (recommended)

Availability: the prebuilt image bundle is available on request — it is not published or linked anywhere. Ask Kevin for it, or build everything from source via Tier 2/3 below.

Both images ship in one tarball. Produce it once on a machine that has built them, then load-and-run anywhere with no build step:

# --- produce the tarball once (on a machine that has already built the images) ---
docker save radiograph-cv radiograph-narrative -o radiograph-images.tar   # ~6069 MB

# --- consume it anywhere (no --build; the images are already loaded) ---
docker load -i radiograph-images.tar
cd clone
docker compose up -d                  # → http://localhost:8248
curl -s localhost:8248/health         # → {"status":"ok"}

The tarball is ~6069 MB (5.9 GB) — most of it is the cv image, which bundles ~1.1 GB of production weights plus CPU torch/SAM 2. That is the honest price of a self-contained, key-free CV pipeline: no download step, no API key. The opt-in LiteLLM gateway (--profile llm) uses a separate public image (ghcr.io/berriai/litellm) that is pulled on demand and is not in this tarball.

Tier 2 — Build the image from source

cd clone
docker compose up -d --build          # → http://localhost:8248

Open http://localhost:8248, drag in samples/bitewing_demo.jpg, and click 🔍 Detect & Review. The real detector labels the teeth, SAM masks + FDI numbers + pathology badges render on the radiograph, and the rules engine suggests procedures — all key-free.

First detection is slow. The cv container cold-loads the model stack on the first /detect (~1–2 min on CPU incl. the 856 MB SAM base); subsequent calls are ~8–13 s. A GPU is a pure speed-up, not a requirement. The build also takes a few minutes (torch + SAM 2 from source + a one-time timm cache warm).

Tier 3 — Bare-metal (two processes, Linux/macOS)

cd clone
# CV service (needs torch/ultralytics/onnxruntime/opencv + sam2 from git; weights in cv/app/roboflow/)
cd cv && python -m venv .venv && . .venv/bin/activate
pip install torch==2.11.0 torchvision==0.26.0 --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt && pip install "git+https://github.com/facebookresearch/sam2.git"
python -c "import timm; timm.create_model('efficientnet_b0', pretrained=True, in_chans=4)"   # warm cache
ROBOFLOW_DIR=$PWD/app/roboflow python -m uvicorn inference_service:app --app-dir app --port 8237 &

# Narrative service (separate shell)
cd narrative && python -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt          # + WeasyPrint OS libs (see narrative/Dockerfile)
INFERENCE_URL=http://localhost:8237 python -m uvicorn main:app --app-dir src --port 8248

Optional: the AI narrative (LLM)

The prose step talks to any OpenAI-compatible endpoint and is off by default — the /analyze route returns a friendly notice until you enable it. Everything else (CV findings, overlays, rules, PDFs) works without it. See .env.example; the turnkey path is the bundled LiteLLM proxy:

# put a provider key in .env, then:
docker compose --profile llm up -d --build

Or point at any OpenAI-compatible base URL — e.g. a local Ollama (ollama serve + ollama pull llama3.2): set LITELLM_URL=http://host.docker.internal:11434, LITELLM_KEY=ollama, LLM_CHAT_MODELS=llama3.2 and it runs key-free (this is how the live narrative above was verified).

Verify

curl -s localhost:8248/health                 # → {"status":"ok"}
curl -s localhost:8248/findings/health        # → inference ok
# Deterministic PDF export works without any LLM:
curl -s -X POST localhost:8248/export/pdf -H 'Content-Type: application/json' \
     -d '[{"procedure_type":"core_buildup","tooth":"30","narrative":"…","cdt_codes":["D2950"]}]' \
     -o out.pdf   # → a valid PDF

Then open the UI, upload samples/bitewing_demo.jpg, and click Detect & Review.

Provenance — models, training data & hardware

Model weights (~1.1 GB, bundled). The author's custom-trained detectors — a YOLOv8m FDI detector (F1 0.9608 on a held-out 48-image / 295-tooth benchmark), a SAM 2.1-L LoRA fine-tune (val Dice 0.9657, over Meta's SAM 2.1 Hiera-L base), and the pathology stack (modality ONNX + apex regressor on EfficientNet-B0, plus YOLOv11 lesion / restoration / caries detectors). These are learned tensors with no recoverable patient images — PHI-safe to distribute.

Training data & licenses. The models were trained on an assembled corpus of 15+ public dental-radiograph datasets plus the author's own office radiographs (~150 hand-annotated for FDI). The principal public sources:

The corpus carries mixed licenses, several explicitly non-commercial (DENTEX CC-BY-NC-SA, PRAD research-scope, some Mendeley sets CC BY-NC). The project is personal-use, non-commercial — which is why the entire image corpus is excluded from this clone; only the learned weights and two CC BY 4.0 sample radiographs ship (see samples/ATTRIBUTIONS.md). Evaluation used strictly held-out benchmarks (the 48-image FDI test set; a public Mendeley 100-bitewing caries set, CC BY-NC 3.0), kept disjoint from the training data by perceptual-hash checks.

Training hardware & automation. The detectors' weights were produced on rented cloud GPUs — Vast.ai and RunPod — under a rent → train → export → download → tear-down discipline (pods destroyed after each run's weights are pulled; costs accounted per run). The FDI detector was pushed through a closed-loop retrain campaign (five cycles): each cycle added a data source, retrained on an H100/H200, and either adopted or rejected the result against a held-out floor. The adopted champion (Cycle 2, an H200 run, ~$1.94) reached F1 0.9608; the whole campaign cost ≈ $9.13 of ~$17 in credit. The honest outcome: a regression of F1 against training-instance count came out essentially flat, so later single-source cycles stopped helping — a real ~0.96 data ceiling on the author's own sensor distribution, documented rather than papered over.

No training in this clone. The clone ships the finished weights and runs CPU-only — there is no dataset, no GPU, and no retrain step here. Reproducing the training would require the public datasets above (per their individual licenses) and a GPU; this recipe reproduces the running pipeline.

What was stripped from the personal version (de-identification + minimalism)

• The hardcoded LiteLLM master key (present in 5 places incl. source + Dockerfile) → removed everywhere; AI made opt-in (--profile llm, placeholder keys), the app now speaks to any OpenAI-compatible endpoint via env, and the model-group names are configurable.
• The entire image/dataset surface (~16 GB / ~100k radiographs: datasets, annotation queues, outputs, test sets, even the training-run mosaics next to the weights) — never copied. The clone is code + weights + two public samples.
• .env secrets (Roboflow / HuggingFace / Label-Studio tokens) → never copied; a placeholder .env.example ships instead.
• Personal paths in the CV code comments (C:\Users\…) → genericized. The CV code is a curated 28-file subset of the research monorepo (detector + SAM + pathology package only), not the full repo.
• A CDN dependency — the UI's html2canvas was loaded from jsDelivr → vendored locally so the app is self-contained.
• Kept as-is (the author's call): the 12 payer rulesets with their real carrier names — they encode public provider-reference documentation patterns and carry no PHI.

Known limitations (stated honestly)

Personal-LAN tool: no authentication, permissive CORS (*), and both services are stateless (nothing persisted). CPU inference is slow — the first /detect cold-loads the models (~1–2 min) and warm calls are ~8–13 s for the full FDI+SAM+pathology stack (a GPU cuts this sharply). The cv image is large (~6 GB) because it bundles torch + SAM 2 + the weights. Put a reverse proxy with auth in front before exposing it. The modality classifier can mis-call BW vs PA on some inputs — that is the real model's output, surfaced honestly.