SmileVision AI — Build Recipe

Take a bare machine to a running copy of a dental "ideal smile" / whitening visualization web app. Upload a smile photo and a CPU-only computer-vision pipeline segments the visible teeth (Segformer face-parsing → SAM 2 promptable masking), runs a clinical-esthetics analysis (lip line, dental midline, buccal corridors, smile arc), builds an inpainting mask, generates an "ideal smile" into that region (mock by default), composites the result back pixel-exactly, validates that pixels outside the mask are untouched, and presents a zoomed before/after slider for chairside patient discussion.

Key-free and GPU-free by default. Out of the box the app runs the whole vision pipeline on CPU and uses a built-in mock inpainting backend (SV_METHOD=dummy) — a whitened composite blended into the detected mouth mask. No GPU, no model download for generation, no API keys. Real Stable-Diffusion generation is opt-in against your own ComfyUI, and the LLM age-estimator is opt-in against your own endpoint (both off by default).

This is a partial clone of a larger personal project — the single canonical web app + CPU pipeline, collapsed from a three-version / multi-machine deployment down to one self-contained build (see What was stripped).

Status: ✅ Verified 2026-08-02 — built from this clone on a Linux Docker host (docker compose up -d --build). The service came up healthy on :8231 key-free and GPU-free (SV_METHOD=dummy default); a real end-to-end run on the bundled synthetic face (upload → Segformer/SAM segmentation → clinical analysis → mock composite → validation) completed on CPU in ~75 s and produced a before/after result. The UI rendered with only a benign favicon 404, and the "Recent Patients" thumbnail is the unmistakably-synthetic sample face. All three tiers covered; the prebuilt tarball (docker save smile-vision-clone) size is recorded in ../PREBUILT_IMAGES.md.

Sensitive data: none — PHI-safe by construction. All ~638 patient photos on the live server live only in Docker data volumes and were never in the git source tree; this clone ships zero patient imagery — the only faces are synthetic, PIL-drawn samples (see clone/samples/ATTRIBUTIONS.md). No data//jobs/ dir ships; real generation is opt-in against your own ComfyUI.

PHI-safety — read this. SmileVision's real inputs are patient smile photographs. None of them ship here — none, by construction. The original app's ~638 patient and test photos live only in Docker data volumes on the author's server; the git-tracked source tree this clone was built from contains zero images (verified with find over the copied tree — the only images anywhere in the clone are three synthetic, PIL-drawn faces under clone/samples/, see below). The clone ships only source code, a self-contained Dockerfile, and that synthetic sample face. No data/, no jobs/, no meta.json, no patient filenames, no bundled dataset. All secrets are placeholders; the author's LiteLLM key and .env were never copied.

What it is

One container: a FastAPI service (src/api.py, port 8231) that serves a build-less React 18 single-page UI (src/static/index.html, no build step) and runs the pipeline (src/pipeline.py) in a background thread. The vision stack is ~14 Python modules — face parser, SAM 2 masker, clinical analysis, mask post-processing, golden-ratio guide, compositor (alpha / Poisson / lab-matched blend), validation, and a multi-backend teeth generator whose default backend is the GPU-free mock.

Prerequisites

New machine? See ../SETUP.md for the base tools. Then, per tier:

Tier 1 — Run the prebuilt container (recommended)

Availability: the prebuilt image is available on request — it is not published or linked anywhere. Ask Kevin for it, or build from source (Tier 2).

# --- produce the tarball once (on a machine that has already built the image) ---
docker save smile-vision-clone -o smile-vision-clone-image.tar

# --- consume it anywhere (no --build; the image is already loaded) ---
docker load -i smile-vision-clone-image.tar
cd clone
docker compose up -d                  # → http://localhost:8231
curl -s localhost:8231/health         # → {"status":"ok","service":"smile-vision"}

The image is honestly large — it bundles CPU torch + transformers plus ~500 MB of pre-downloaded vision models (Segformer ~323 MB, SAM 2-small ~185 MB). That is the price of a self-contained, key-free CV pipeline with no runtime model download for the vision stages.

Tier 2 — Build from source

cd clone
docker compose up -d --build          # → http://localhost:8231

Open http://localhost:8231, upload a smile photo (or drag in samples/synthetic_smile.png), and click Generate. With the default SV_METHOD=dummy the pipeline runs end-to-end on CPU — face parse → SAM 2 mask → clinical analysis → mock whitened composite → before/after slider — with no GPU and no keys.

First request is slow. The container cold-loads (and, if it could not pre-download at build time, fetches) the vision models on the first generation. After warm-up, the CPU vision stages are quick (the original measured a face parse and a SAM 2 mask at ~2 s each on a downscaled working image). A GPU is a pure speed-up for the optional Stable-Diffusion path, never a requirement for the default flow.

Tier 3 — Bare-metal (one process)

cd clone
python -m venv .venv && . .venv/bin/activate     # Windows: .venv\Scripts\activate
# CPU torch keeps the install lean:
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
# (needs OpenCV's system libs — libgl1, libglib2.0-0 — on Linux; see Dockerfile)
SV_METHOD=dummy SV_DEVICE=cpu PYTHONPATH=src \
  python -m uvicorn api:app --app-dir src --host 0.0.0.0 --port 8231
# → http://localhost:8231 ; jobs are written to clone/data/jobs/ (gitignored)

There is also a GPU-free, key-free, photo-free smoke test that draws a synthetic face and runs the whole pipeline mechanically (with a synthetic-mask fallback so it needs no models at all):

PYTHONPATH=src python src/quick_test.py

Optional: real Stable-Diffusion generation (your own ComfyUI + GPU)

The dummy backend is a placeholder composite, not a real smile simulation. To generate an actual inpainted smile, point the app at your own ComfyUI server — nothing is bundled, and no model is downloaded by this clone:

# in clone/.env  (copy from .env.example)
SV_METHOD=comfyui
COMFYUI_URL=http://your-comfyui-host:8188
COMFYUI_CHECKPOINT=sd-v1-5-inpainting.ckpt

The teeth generator sends a workflow (CheckpointLoader → CLIPTextEncode(±prompt) → VAEEncodeForInpaint → KSampler → VAEDecode → SaveImage), polls, and cancels on timeout. This is a GPU workload you host yourself; the original ran it on a GTX 1080 Ti at roughly ~57 s per 512×512 / 30-step generation.

Optional: LLM age estimation (your own endpoint)

Age estimation auto-suggests an age bracket (which tunes the generation prompt) from the portrait. It is off by default and speaks to any OpenAI-compatible endpoint:

# in clone/.env
LITELLM_URL=http://your-litellm:4000       # or a local Ollama, key-free
LITELLM_API_KEY=your-key-or-ollama
LITELLM_VISION_MODEL=gpt-4o-mini           # a vision-capable model your endpoint serves

With LITELLM_URL unset, age_estimator.py short-circuits and the clinician just picks the bracket manually — the pipeline is unaffected.

The sample face — provenance

The before/after showcase uses only a 100% synthetic, PIL-drawn face — a cartoon of ovals, arcs, and lines, not a photograph of anyone. It is generated by the exact code that ships in src/quick_test.py (create_test_image) and src/teeth_generator.py (_generate_dummy), so its provenance is verifiable by construction. See clone/samples/ATTRIBUTIONS.md. The three files (synthetic_smile.png, synthetic_smile_after_dummy.png, and a before_after_dummy.png strip) are public-domain / CC0. No patient photo, and no third-party photograph, appears anywhere in this clone.

Verify

curl -s localhost:8231/health                 # → {"status":"ok","service":"smile-vision"}
curl -s localhost:8231/api/jobs               # → [] on a fresh boot (empty — no bundled jobs)

Then open the UI, upload samples/synthetic_smile.png, and click Generate — the mask, clinical overlay, and a mock whitened before/after render, key-free.

Provenance — models, data & measured facts

Models (downloaded from Hugging Face at build/first-run, not authored here). Segformer jonathandinu/face-parsing (~323 MB) for 19-class face parsing, and Meta's SAM 2.1 Hiera-small (~185 MB) for promptable tooth-region masking. These are public pretrained weights; no model was trained for this project, and no training data is involved in running it.

Measured facts (from the author's development notes, stated honestly).

Applies to what this clone ships (the mar26 lineage): - SAM 2 mask quality: ~0.877 IoU typical on the author's inputs. - Optional SD generation timing (opt-in ComfyUI path): see below.

From a later "Current" build's whitening/quality overhaul — described here as development history; this mar26-based clone does not include these stages (its compositor is blend-only and there is no per-variation quality gate): - Whitening stability: switching from fixed offsets to adaptive per-pixel LAB normalization to a target L* cut run-to-run variance from ±8 L* to <1 L* (fixed offsets gave Vita A2 one run, A3 the next under SD variance). - Whitening targets: upper arch L*=88 (≈ Vita A1/B1), lower-arch pass L*=87, ceiling 93, with a 12 L* minimum-perceptible-change floor. - Quality gate per generated variation: brightness>145, P25>100, std>18, SSIM outside the mask >0.99 — with up to 3 silent regenerations per slot. - CPU working-resolution speedups: face parse and SAM 2 each ~15 s → 2 s, SSIM ~65 s → 5 s, after downscaling large photos to a ≤1500 px working copy. - Optional SD generation: ~57 s per 512×512 / 30-step run on a GTX 1080 Ti (the author's GPU host — not part of this clone).

The original was ~6 k lines across the "Current" build and ~5.6 k across the "mar26" snapshot this clone is based on; it grew from a Phase-1 drop of 9 files / ~2,369 lines over 25 commits (2026-03-27 → 03-31).

What was stripped from the personal version (de-identification + minimalism)

• The three-version / multi-machine deployment. The original ran three side-by-side builds (a "Current" overhaul, and Mar-26 / Mar-27 snapshots) behind a Node/dockerode wake-proxy launcher, with Stable-Diffusion delegated over the LAN to a ComfyUI on a separate GPU PC. Collapsed to one self-contained container that runs the CPU pipeline + mock backend locally; real SD is opt-in against your ComfyUI.
• The broken base image. mar26's Dockerfile was FROM smile-vision-new:latest (a personal layered image that has since been deleted, so it could not rebuild). Replaced with a full self-contained build (python:3.11-slim + requirements.txt
• a cached model-download layer).
• A hardcoded LAN IP — the compose COMFYUI_URL pointed at a private LAN address (the author's GPU host) → an unset placeholder (.env.example); the default flow never dials it.
• A hardcoded internal LiteLLM alias — age_estimator.py had http://litellm:4000/... baked in → made env-driven and opt-in (LITELLM_URL, unset = disabled), with the personal model alias (free-vision) replaced by a configurable LITELLM_VISION_MODEL.
• The LiteLLM key. The mar26 compose referenced ${SMILE_VISION_LITELLM_KEY} from a server-side .env holding a real key — that .env was never copied; a placeholder .env.example ships instead.
• A personal hardware reference — a specific workstation model named in requirements.txt and a quick_test.py hint → genericized to "your own GPU / your own ComfyUI".
• The legacy JSX UI (SmileVisionApp.jsx, unused, hardcoded a dev port) → dropped; only the production index.html ships.
• A server's local hostname in a design doc → localhost.
• The entire patient/test image surface (~638 photos across four Docker data volumes, with meta.json clinical geometry and phone-camera filenames) — never in the source tree, never copied. Fresh empty volumes; the app boots with an empty job list.
• The SAM 2 pre-download mismatch — the original image pre-fetched sam2.1-hiera-base-plus while the code loads sam2.1-hiera-small; the clone pre-fetches hiera-small so the baked layer is actually used.

Known limitations (stated honestly)

• The default backend is a mock, not a smile simulator. SV_METHOD=dummy composites a flat lightened fill into the mask — it demonstrates the pipeline (segmentation, clinical analysis, mask building, compositing, validation), not a realistic generated smile. A real result needs the opt-in ComfyUI + GPU path.
• This clone is the mar26 lineage — blend-only compositor, no whitening or quality-gate stages. The adaptive-LAB whitening, 3-pass artifact scrubber, and per-variation quality-gate-with-regeneration described in Provenance were a later "Current" build overhaul and are not in this shipped code (the compositor here does alpha / Poisson / LAB-matched blending only). Kept honest rather than back-porting an unshipped feature.
• The proportion-system picker has no effect on generated output. The golden-ratio guide is drawn and saved as an intermediate, but it is not fed to the generator (there is no ControlNet node in the workflow) — it is a visual guide, not a generation constraint. Kept as-is, flagged here for honesty.
• Personal-LAN tool: no authentication, permissive CORS (*), one shared workspace. Put a reverse proxy with auth in front before exposing it.
• Camera capture needs a secure context. getUserMedia requires HTTPS or localhost; over plain http://<host>:8231 browsers hide it — use file upload.
• The UI loads React and fonts from CDNs (esm.sh, Google Fonts), so the demo machine needs internet for the page to render.
• The image is large (CPU torch + transformers + ~500 MB of vision models) — the honest cost of a self-contained, key-free CV pipeline.
• No training here. The clone runs finished public pretrained models on CPU; there is no dataset, no GPU step, and no retrain in this recipe.