The scene is decomposed into its objects (each object's 3D Gaussians cropped from the block by its PandaSet bounding box). car-04 is a well-covered example: the capture vehicle drove past it, so about three-quarters of it was seen. From that we reconstruct it many ways and drop each into one comparison viewer.
The methods split into two families — and the split is the whole story.
Reconstruction · real pixels
Built only from what the cameras actually saw. Faithful to the real car, but limited to the covered angles — the unseen side is missing or noisy.
Generative · image-to-3D
A neural model hallucinates a complete object from one image using learned priors. Always whole, but the unseen parts are plausible invention, not measurement.
Source 3DGS
- Real captured light — photoreal, view-dependent
- The benchmark everything else is judged against
- ~¾ coverage, floaters, not a solid/editable asset
TRELLIS.2 · best
- Complete, sharp PBR texture — the standout reconstruction
- Proportions snapped to the vehicle's real width (spec-corrected)
- Needs a 24GB GPU; the fully-unseen side is inferred
TRELLIS (GS)
- TRELLIS's native splat — dense, smooth, complete all sides
- Photoreal-ish, view-dependent
- Gaussians, not a solid mesh; larger file; far side generated
SF3D
- Clean lean mesh, crisp 1024² texture, game-ready — on 8GB
- Single view → the back is guessed
- Very sensitive to input mask quality (was flat until fixed)
InstantMesh
- Complete; multiview-diffusion (invents 6 views, then reconstructs)
- Chunkier / wider — less faithful than TRELLIS
TripoSR
- Instant, complete, real 3D volume — free & local
- Blobby geometry, muddy colors, no UV texture
GaussianObject (GS)
- Only generative-free complete-ish result — real multi-view pixels + poses
- Genuine reconstruction, not invention
- Coarse (diffusion repair skipped) → fuzzy, weak unseen side
- Optimized per object (~25 min each), not one-shot
Poisson mesh
- An actual mesh straight from real geometry
- Noisy, blobby, wide skirt from ground/neighbour bleed
Can we push the best reconstruction (TRELLIS) further using only license-clean signals? Once the vehicle is identified — a 2010 Ford Escape in Luxor SF taxi livery (read by open OCR + a self-hosted vision-LLM, so the ID is just a label, never a baked asset) — you get two clean levers: OEM dimensions (facts, not copyrightable) to snap the geometry to spec, and the make/model as a text prompt for a self-hosted diffusion model. Nothing third-party ever enters the 3D output. Three variants came out of it:
TRELLIS +refine
- Sharper livery + panel detail — the cleanest of the three
- Robust, cheap, runs on the local card
- Cosmetic — polishes the invented texture, doesn't add real fidelity
TRELLIS +real-tex
- ~½ the surface is the real captured livery — highest potential fidelity
- Patchy/washed — TRELLIS's generated geometry is ~10 cm off the capture, so pixels smear
TRELLIS +real+refine
- More coherent than raw real-tex — smooths the real↔fill seam
- Still bottlenecked by the same geometry-registration ceiling
The finding: generative refinement (+refine) is the robust win; real-pixel re-texturing is gated by how well the reconstruction's geometry matches the capture. A clean mesh (TRELLIS) doesn't line up; the geometry that does (the source splat) is too noisy. True real-pixel fidelity needs geometry that's clean and capture-matched — non-rigid registration or a better multi-view reconstruction — not more diffusion.
Which to use when
- • Most faithful to the real car → Source 3DGS (or GaussianObject, once its repair stage is run).
- • Best clean, complete, sim/game-ready asset → TRELLIS.2 mesh — the standout.
- • Photoreal complete splat → TRELLIS (GS).
- • No cloud GPU → SF3D (textured) or TripoSR (fast) — with clean SAM masks.
The core trade-off: reconstructions are true but incomplete; generative methods are complete but partly invented. TRELLIS wins because its priors are strong enough that the invented parts look right — but for anything a driving sim must trust as real geometry, the real-pixel reconstructions are the honest source.