⚙ How it works 📊 Results 🧰 Skills & tech
01 Overview
This isn't one narrow model — it's one dental-radiograph computer-vision system with three jobs on the same codebase: identify and number every tooth, outline each one, and flag the pathology and restorations the training data can actually teach a model to recognize. It's the vision component behind the insurance-narrative pipeline — a companion project that turns these findings into a drafted claim narrative — and it attempts to detect, segment, and identify pathology in a single pass. You can run the live demo to try it on a radiograph yourself.
Worth knowing before you judge the output: it was trained almost entirely on public dental-radiograph datasets — 15-plus of them, spanning bitewings, periapicals, and panoramics, at resolutions from 224×300 up to 2652×1536, some of them photographs of X-ray film rather than direct sensor captures, and a few mislabeled by their own publishers. That variance is the hard part of the problem, not a footnote to it. The one non-public ingredient was a small set of my own hand-annotated radiographs, which turned out to be the single biggest accuracy lever: 150 in-domain annotations moved detector F1 from 0.863 to 0.951.
FDI tooth detector
A custom-trained YOLOv8m that finds and numbers all 32 teeth in FDI notation, 32 classes (11–48). F1 0.9608 on a held-out 48-image / 295-tooth test set — count-based micro-F1, on images it never trained on. Production weight is the Cycle-2 retrain, trained on rented cloud GPUs for about $9 across the whole campaign (see mlops-gpu).
SAM tooth masks
SAM 2.1-L segmentation, fine-tuned locally with hand-rolled LoRA adapters — a small LoRALinear module injected onto SAM 2.1-L's Hiera attention, co-trained with the mask decoder. Val Dice 0.9657, up from a 0.9505 baseline, trained against 686 human-verified tooth masks from the public DenPAR periapical dataset.
Findings pipeline
A lesion detector (caries, PARL) plus a restoration detector (v1.5b5c: filling, crown, root canal, implant) — the full per-tooth finding set the pipeline surfaces. Bridges are folded into the crown class; bone loss is not detected. Every finding carries a confidence score.
02 Why I built it
Nothing downstream works without knowing which tooth is which — a pathology finding doesn't mean anything, and a claim narrative can't reference "tooth #14," without correct numbering first. So FDI numbering came first, and it stayed the foundation everything else in this system builds on top of.
Pathology and restorations came second, deliberately scoped to what public dental-radiograph datasets can actually teach: caries, periapical lesions, fillings, crowns, root canals, and implants are all reasonably well represented across the public literature. A clinician's full diagnostic judgment is not something a detector trained on box and mask labels can replace, so the system's job is to surface per-tooth findings with a confidence score — not a diagnosis.
03 What I built & how it works
A single-machine pipeline: a modality classifier routes the image, an FDI detector numbers every tooth, SAM outlines each one, and a modality-routed pathology stack flags findings — one radiograph in, a per-tooth findings table out.
Fig. 1 — one pipeline, one process: modality routing → FDI numbering → SAM outlines → pathology stack. Every stage runs on the same machine; there's no service boundary or network hop between them.
- Classify modality — an EfficientNet-B0 classifier routes the image as bitewing or periapical, so pathology thresholds and crop strategy match the view instead of assuming one. (It's a two-class model; panoramics are handled by a separate panoramic-specific detector rather than by this router.)
- Number the teeth — the YOLOv8m detector finds and numbers all 32 teeth in FDI notation in one forward pass; F1 0.9608 on held-out data it never trained on.
- Outline each tooth — SAM 2.1-L, prompted by the detector's boxes and fine-tuned locally with LoRA adapters, draws a per-tooth mask (val Dice 0.9657).
- Detect pathology & restorations — a lesion detector flags caries and periapical lesions; a restoration detector (v1.5b5c) flags fillings, crowns, root canals, and implants — each modality-routed and confidence-scored.
- Merge per tooth — the FDI number, mask, and findings combine into one per-tooth record. That record is what both the tester's findings table and the narrative pipeline consume.
04 🧰 Skills & tech used
05 Notable challenges & decisions
Most of the interesting engineering here was in the honesty: what a fine-tune actually improved, what a metric actually proved, and what more data could not fix.
Hand-rolled LoRA adapters on SAM 2.1-L
Stock SAM masks were good but not clinical-grade at the tooth margins. I wrote my own low-rank adapters — a small LoRALinear module plus an injection pass — onto SAM 2.1-L's Hiera attention, and co-trained them with the mask decoder. Trained against 686 human-verified tooth masks from the public DenPAR periapical dataset and scored on a held-out validation split, where Dice moved from a 0.9505 baseline to 0.9657.
Using panoramic data to train a bitewing model
The pathology models needed more data than the bitewing and periapical sets could supply, and panoramic images — the big wraparound X-rays — look nothing like them, so on the face of it that data was unusable. The observation that unlocked it: at the level of a single tooth, a 224×224 crop from a panoramic is nearly indistinguishable from the same crop taken from a periapical. Cropping normalizes the source modality away. So the pipeline trains a separate panoramic-specific tooth detector (per-tooth top-1 0.976 vs 0.701 for the bitewing/periapical detector on those images), uses it to cut per-tooth crops out of four panoramic datasets, and trains the pathology models on crops instead of whole images. That change fixed a case where the whole-image model scored an obvious clinical root canal at zero confidence.
Train-time metrics can hide production bugs
A more aggressive LoRA variant (attention + MLP) tied the winner on training-time Dice — so on paper it was a wash. In the real inference harness it silently failed on 5 of 144 masks, rejected outright by the SAM predictor. I discarded it. The rule I kept: validate on the production path, not the training metric, because the two can disagree quietly.
Finding the data ceiling instead of chasing a number
I ran a closed-loop retrain campaign to push the detector toward a higher target. A regression of F1 against training-instance count came out essentially flat, and later single-source data cycles stopped improving — the two after the champion regressed rather than gaining. The finding was that more data of the same distribution would not move the number much — a real limit, documented rather than papered over.
The earlier segmentation approach didn't ship — and that's the honest part. "DenPAR" is a public periapical-radiograph dataset, not a codename. Before this detector-based pipeline existed, an earlier effort tried a segmentation-first approach — nnU-Net tooth segmentation plus ViT landmark regression, annotated with MONAI Label / 3D Slicer, built around that same public dataset. It failed on real office radiographs — FDI numbering broke under domain shift, and CPU inference was too slow to use — so it didn't ship as a whole system. It still earned its keep: it produced the bitewing-vs-periapical view classifier this pipeline still uses, and the 686 human-verified tooth masks that became the SAM fine-tune's ground truth above.
06 The inference tester
Everything above is exercised through one Gradio tool — the same one this page's Live Demo tab runs, minus anything that could touch real patient data.
Upload a bitewing or periapical radiograph — or pick one of the bundled public samples — and it runs FDI numbering, SAM masks, and pathology detection in one pass, then shows a per-tooth findings table. It exposes real controls, not a cut-down showcase: a YOLO model-variant picker (default the YOLOv8m champion), a SAM mask-variant picker (default the 2.1-L LoRA fine-tune), conf/IoU/TTA/NMS dials, overlay and opacity toggles, a clickable tooth-picker that force-anchors FDI numbers under a dental-arch constraint, and a Debug panel with a per-detection table, inference timing, and raw-prediction JSON.
FDI numbers + SAM outlines + pathology badges on a sample radiograph
07 Results
Sources: routed evaluation on the held-out test set (count-based micro-F1), the SAM Dice harness against 686 verified DenPAR masks, and the training-campaign ledger — see mlops-gpu for the per-run cost breakdown.
08 Honest status
The FDI detector, SAM masks, and pathology stack are real, trained, and evaluated — every number above is measured, not estimated. The system runs as a single-machine pipeline: one codebase, one Gradio process, one set of weights, with no service boundary between the detection, segmentation, and pathology stages.
The Live Demo tab is the de-identified version of the exact tester used during development, not a simplified mockup — stateless, key-free, and limited to public CC-licensed sample radiographs in place of any real patient image.
Dental-CV Demo — Build Recipe
Take a bare machine to a running copy of a dental-radiograph computer-vision demo: upload a bitewing or periapical radiograph and a Gradio UI runs a custom-trained YOLOv8m FDI tooth detector, SAM 2.1 per-tooth masks, and a pathology pipeline (modality classifier, apex regressor, and caries / restoration / lesion detectors), then renders the findings as overlays, a per-tooth table, and raw JSON. One container, one machine, CPU-only — no GPU required.
Status: the clone (
clone/) is a complete, de-identified, code-only copy — app code, samples, and the container/build layer are all written and reviewed. It has not been built or run yet: the model weights (~1 GB, not part of this source clone — see below) still need to be staged intoclone/models/beforedocker compose up --buildcan actually produce a working image. That build-and-verify pass is a separate, later step, not covered by this document's own testing.PHI-safety — read this. This is a dental-imaging system, so be explicit: no patient data ships, by construction. The app is stateless — an uploaded radiograph is processed in memory and discarded; there is no database, no upload directory, no bundled patient images. The author's CV research project holds a large private corpus of radiograph-type training images; none of that ships here. The clone contains only source code, the bundled model weights (learned tensors, no recoverable patient images), and exactly two public, CC BY 4.0 research radiographs (
clone/samples/, seesamples/ATTRIBUTIONS.md) — never a real patient image, never a bulk dataset. All secrets are placeholders (there are none required to run the demo at all).
What it is
One service, one container:
dental-cv(clone/, Gradio, port 7860) — a flat Python app (app.py) wrapping the detector + SAM mask provider + pathology package. Upload an image, click Detect, get FDI-numbered teeth, tooth masks, and pathology/restoration badges rendered on the radiograph, plus a per-detection table and a raw-JSON export. A debug toggle exposes every inference dial (model variant, confidence/IoU, mask backend, overlay opacities) for anyone who wants to go past the default view.
Prerequisites
| Tier | You need |
|---|---|
| 1 — prebuilt container (recommended) | Docker Desktop / Engine. Load the on-request image and run — nothing to build, no other deps. |
| 2 — build from source | Docker Desktop / Engine + the model weight set (below) staged into clone/models/. docker compose up --build compiles the image locally — CPU torch, SAM 2, all deps, and a sample-result cache bake happen during the build (a few minutes). |
| 3 — bare-metal | A host with Python 3.11, the weight set staged into models/, and the same pip steps the Dockerfile runs (CPU torch, requirements.txt, sam2 from git). |
Tier 1 — Run the prebuilt container (recommended)
Availability: the prebuilt image is available on request — it is not published or linked anywhere. Ask Kevin for it, or build from source via Tier 2/3 below.
docker load -i dental-cv-clone.tar # produced once via: docker save dental-cv-clone -o dental-cv-clone.tar
cd clone
docker compose up -d # → http://localhost:7860
The image bundles the full stack — CPU torch, SAM 2, and ~1 GB of production weights — so there is no download step and no API key at runtime.
Tier 2 — Build the image from source
cd clone
# Stage weights first (see "Model weights" below) — same relative paths as
# the code expects, e.g. models/yolo_m_best.pt, models/pathology/*.pt, etc.
docker compose up -d --build # → http://localhost:7860
Open http://localhost:7860, click one of the two bundled sample radiographs, and hit Detect. The real detector labels the teeth, SAM masks + FDI numbers + pathology badges render on the radiograph, and the per-detection table + raw JSON populate below — all key-free.
First detection is slow. The container cold-loads the model stack on the first click (roughly a minute or two on CPU); warm calls are faster. The two bundled samples are pre-baked into a cache at build time (
precompute_samples.pyruns as part ofdocker build), so they load instantly regardless of cold-start. A GPU is a pure speed-up, not a requirement. The build itself also takes a few minutes (CPU torch + SAM 2 from source + a one-timetimmcache warm).
Tier 3 — Bare-metal (single process)
cd clone
python -m venv .venv && . .venv/bin/activate # Windows: .venv\Scripts\activate
pip install torch==2.11.0 torchvision==0.26.0 --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
pip install "git+https://github.com/facebookresearch/sam2.git"
python -c "import timm; timm.create_model('efficientnet_b0', pretrained=True, in_chans=4)" # warm cache
# Stage weights into models/ (see below), then optionally pre-bake the sample cache:
python precompute_samples.py
python app.py # → http://localhost:7860
Model weights
The code expects a models/ directory (~1 GB, ~10 files: the FDI detector,
a SAM 2.1 LoRA fine-tune, and the pathology sub-detectors/classifier) plus
one larger SAM base checkpoint at the project root — none of it is part of
this source clone. It is available on request for anyone who wants to
build Tier 2/3 themselves; the prebuilt image (Tier 1) already has it baked
in. These are learned tensors with no recoverable patient images —
PHI-safe to distribute.
Optional: advanced mask variant
The debug-mode "SAM 3" mask option fetches a gated model from Hugging Face
Hub on first use. See clone/.env.example — an optional HF_TOKEN raises
the rate limit / enables that one variant; everything else (the default
detector + mask + pathology stack) needs no token.
Verify
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:7860/ # → 200
Then open the UI, click a bundled sample, and hit Detect — expect FDI tooth labels, mask overlays, and (if pathology weights are present) badges above teeth with findings.
Provenance — models, training data & hardware
Model weights (~1 GB, bundled). A custom-trained YOLOv8m FDI detector (F1 0.9608 on a held-out 48-image / 295-tooth benchmark), a SAM 2.1-L LoRA fine-tune (val Dice 0.9657, over Meta's SAM 2.1 Hiera-L base), and a pathology stack (modality-classifier ONNX, an EfficientNet-B0 apex regressor, and YOLO lesion / restoration / caries detectors — the restoration detector's held-out combined validation mAP@50 is 0.739). All are learned tensors with no recoverable patient images.
Training data & licenses. The models were trained on an assembled corpus of 15+ public dental-radiograph datasets plus a small set of the author's own hand-annotated radiographs. The principal public sources:
| Dataset | Role in training | License |
|---|---|---|
| DenPAR — Rasnayaka S., Leuke Bandara D., Jayasundara A., et al., "DenPAR: Annotated Intra-Oral Periapical Radiographs Dataset for Machine Learning," Sci Data 12, 1615 (2025); Zenodo DOI 10.5281/zenodo.16645076; paper DOI 10.1038/s41597-025-05906-9 | per-tooth PA masks + FDI pairing; SAM fine-tune ground truth | CC BY 4.0 |
| DENTEX (panoramic challenge set) | OPG pathology (PARL) + the separate OPG-FDI detector | CC BY-NC-SA (non-commercial) |
| PRAD-5K — Zhou Z. et al., "PRAD," MICCAI 2025, pp. 475-484 (Springer); Nankai University | restoration + PARL supplement | research/personal scope; © Nankai University |
| sam-ybj0w/dental-7vwaw (Roboflow Universe) | bitewing caries recall booster | CC BY 4.0 |
| Additional: Mendeley bitewing/caries sets, NLM Open-i, Roboflow Universe ICCMS caries sets, Bitewing_allside | caries / FDI / restoration supplements | mixed — CC BY 4.0 / CC BY-NC / Public Domain (per source) |
The corpus carries mixed licenses, several explicitly non-commercial
(DENTEX CC-BY-NC-SA, PRAD research-scope, some Mendeley sets CC BY-NC).
This project is personal-use, non-commercial — which is why the entire
training-image corpus is excluded from this clone; only the learned
weights and two CC BY 4.0 sample radiographs ship (see
clone/samples/ATTRIBUTIONS.md). Evaluation used strictly held-out
benchmarks, kept disjoint from the training data by perceptual-hash checks.
Training hardware. The detectors' weights were produced on rented cloud GPU instances, rented, trained, and torn down per run (no hardware kept running between training jobs). The FDI detector went through a closed-loop retrain campaign (five cycles), each either adopted or rejected against a held-out floor; the adopted champion reached F1 0.9608 for roughly $2 of cloud compute.
No training in this clone. The clone ships the finished weights and runs CPU-only — there is no dataset, no GPU, and no retrain step here. Reproducing the training would require the public datasets above (per their individual licenses) and a GPU; this recipe reproduces the running demo.
What was stripped from the personal version (de-identification + minimalism)
- The author's full CV research repo (a large private image/dataset surface, plus training/eval/label-review tooling not relevant to running the demo) — never copied. The clone is a curated source subset (the Gradio app + the detector/mask/pathology package) plus the two public samples.
.envsecrets (any API tokens the author's own copy uses for optional gated-model downloads) → never copied; a placeholder.env.exampleships instead, and every documented feature works with no token at all.- Personal machine paths and hostnames in code comments → genericized.
- Kept as-is: the model weights themselves and the pathology class taxonomy — they're learned tensors and public terminology, no PHI.
Known limitations (stated honestly)
Personal-project demo: no authentication, and the container is stateless (nothing persisted). CPU inference is slow — the first request cold-loads the models (roughly a minute or two) and warm calls are tens of seconds for the full FDI + SAM + pathology stack; a GPU would cut this sharply but isn't required. The modality classifier can mis-call bitewing vs. periapical on some inputs — that's the real model's output, surfaced honestly rather than hidden. A few pathology sub-detectors are best-effort and clearly labeled as such in the UI. Put a reverse proxy with auth in front before exposing this beyond a trusted network.