How we render scenes in 2.3 seconds

Twelve months ago a QuickStyle render took 11.4 seconds at the median. Today it takes 2.3. The GPU we use is the same generation. The model is roughly the same parameter count. What changed is six small, ugly, eminently shippable engineering decisions.

Here they are, in the order we shipped them and roughly the order of their impact.

1. Move the cutout step off the critical path (−4.1s)

The single largest win. Originally our pipeline did: receive image → cutout → render → return. Cutout took 4–5 seconds and blocked everything downstream. We moved cutout into a preflight step that runs the moment a user opens the upload modal — before they have even clicked “render.” By the time they pick a preset, the cutout is already done.

This is a UX trick masquerading as a perf trick. The cutout still takes 4–5 seconds. We just make sure it overlaps with the human looking at the preset library.

2. Cache compiled diffusion graphs across requests (−2.4s)

Our first-pass implementation re-traced the diffusion model graph on every request. Compilation overhead was real and grew worse as we added schedulers. We moved to a single warm container with the graph traced once at boot and re-used across requests, with a per-request session that resets only the latents and conditioning.

The trade is concurrency: one container can only serve one render at a time, so we run more of them. This matters less than it sounds because containers are cheap on Modal and warm-pool sizing handles burst.

3. Switch to FP8 weights with a calibrated re-quant pass (−1.2s)

Off-the-shelf FP8 quantization gave us speed but tanked render quality on dark backgrounds. The fix wasn’t avoiding FP8 — it was running a 200-image calibration pass against our actual scene library and re-quantizing per-tensor instead of per-layer. Result: same quality, 22% faster inference.

4. Stream the response while the last 10% renders (−0.9s perceived)

The diffusion process produces a usable image well before the final denoise step finishes. We stream a low-resolution preview at step 18 of 25, then ship the full output at step 25. The user sees something in 1.4s, the final result lands at 2.3s. This is purely a perceived-latency win, not a real one — but perceived latency is the one that actually controls whether a user rates the tool fast.

5. Pre-warm three GPU containers per region (−0.6s p99)

Cold starts were brutal. A first-request user could wait 18 seconds while a container booted. We now keep three warm containers per region (US-East, EU-Central, AP-Southeast). The cost is real — about $340/month — but it removed 12-second cold starts from the experience entirely.

This is the single decision I argued against and was wrong about. Cold-start UX is the difference between “this tool is fast” and “this tool is sometimes fast.”

6. Parallelize background scene composition (−0.3s)

The scene background — the marble counter, the editorial backdrop, whatever — is composited from a deterministic pre-rendered atlas, not regenerated per request. While the diffusion model paints the product, a separate worker pulls the matching scene from R2, applies lighting/color-grade, and prepares the final composite layer. The two finish at roughly the same time and merge in 80ms.

What we did not do

We did not switch to a smaller model. The temptation is real and the headline numbers look great, but the failure modes — fabric texture, edge sharpness, tricky materials like metal or velvet — get bad fast at smaller parameter counts. The 2.3s number is on a model that’s only 12% smaller than our launch model. The other 88% of the win came from the boring engineering above.

We did not switch GPUs. We considered upgrading from H100s to a hypothetical newer chip and the math did not pencil. Container density matters more than per-chip throughput at our scale.

The point

If your team is staring at a 10-second render and the discussion is about a faster model — the discussion is on the wrong axis. Look at what runs sequentially that could run in parallel. Look at what’s recomputed that could be cached. Look at what blocks the user that could be hidden behind a UX seam. The chip is rarely the bottleneck. The pipeline is.