Why try-on still misses fabric drape (and what to do about it)
The hardest unsolved problem in virtual try-on isn't pose, fit, or lighting. It's drape — the way fabric falls under gravity. Here's why current models can't render it, and the four workarounds that actually ship.
Look at any current virtual try-on output side-by-side with a real catalog photo of the same garment. Pose looks right. Lighting looks right. Skin looks right. The garment fits the body believably. And then your eye catches on something: the fabric isn’t falling the way fabric falls. The drape is wrong.
This is the unsolved problem. Pose, fit, and lighting are mostly solved. Drape is not. Here’s why.
What “drape” actually means
Fabric drape is the geometry of how a textile folds under gravity, given its weight, weave density, stiffness, and the shape it’s hanging on. A heavy wool coat drapes one way. A silk slip drapes another. A bias-cut cotton dress drapes a third way.
To render drape correctly, a model has to know:
- The physical properties of the specific fabric in the photo.
- The 3D geometry of the body underneath.
- How gravity, friction, and tension interact between them.
Current diffusion-based try-on models don’t have any of these. They have a 2D garment image and a 2D person image, and they paint the former onto the latter using a learned distribution of how garments-on-people tend to look. That distribution gets average drape mostly right and edge-case drape badly wrong.
Why physics-based simulation isn’t the answer (yet)
The textbook fix is to add a physics simulator. Cloth simulators have existed in animation for 25 years. The problem is parameter inference: given a single photo of a garment, you cannot reliably infer the physical properties needed to drive a simulation. Weave density, drape coefficient, bending stiffness — none of these are recoverable from pixels.
There are research papers attempting this with multi-view capture, but they require specialized turntables and 4–8 camera setups. That doesn’t run in 2.3 seconds on a render farm. It barely runs in 8 hours on a workstation.
The four workarounds that actually ship
1. Restrict the garment library
The simplest solution. If your try-on tool only handles structured garments — t-shirts, button-downs, jackets — drape barely matters because these garments don’t drape much in the first place. Brands selling structured apparel can ship try-on today.
This breaks down for fashion brands selling dresses, skirts, scarves, or anything cut on the bias.
2. Pre-photograph drape, then composite
What we use for our most demanding partners. The brand sends us 3–5 reference photos of the garment on a real body, captured from canonical angles. We use these as conditioning images during the try-on render. The model copies the drape from the reference rather than inventing it.
3. Restrict the pose library
Drape errors are most visible when the garment is in motion or off-center on the body. They are least visible in static, symmetrical poses. If you constrain try-on output to front-facing, neutral-stance, hands-at-sides shots, you can get away with current models even on flowing garments.
This is the catalog-photography compromise: use try-on for the spec shot, use a real photographer for the campaign shot. Honest, and it ships.
4. Use try-on for fit signal, not for hero imagery
The cynical version, but the most pragmatic one. Try-on does not need to render hero-level imagery to be useful. It needs to give a shopper enough confidence that the garment will look right on them to click “add to cart.” That bar is much lower than catalog quality.
We’ve seen brands deploy try-on as a sub-page widget — small, optional, framed as “preview, not photography” — and get conversion lift even from outputs that wouldn’t pass a catalog review.
Where we think this goes
Three things are improving simultaneously:
- Per-fabric models. We’re seeing strong results from models trained on a single garment-type-and-fabric combination. A model trained only on cotton t-shirts gets drape and texture right almost always. Specialization beats generalization for this problem.
- Differentiable physics. Recent work on simulators with learnable parameters lets us infer drape coefficients from a single photo at training time. It’s not real-time yet, but it’s close.
- Synthetic training data. As physically-simulated try-on rendering improves, it produces enough labeled training pairs to fine-tune diffusion models on rare drape scenarios — bias cuts, asymmetric hems, layered garments.
We expect the 20% bad-drape rate to be 5% by end of year, 1% within two years. It will not be 0%. There will always be a long tail of strange garments where physics matters and pixels don’t capture it.
What to do with this today
If you sell apparel and you’re considering virtual try-on, the question isn’t “is the technology ready?” The question is “is the technology ready for my garments?”
Walk through your top 20 SKUs. For each, ask: is this garment structured (drape doesn’t matter) or flowy (drape matters)? If structured-dominant, ship try-on. If flowy-dominant, either restrict to spec poses, use reference conditioning for the bestsellers, or hold off until per-fabric models mature.
Drape is the last hard problem. The rest is shipping.