Driving-world database

A driving simulator needs two things a map product can't give you at once: a road you can own — drive, collide against, and edit — and a world that actually looks real. Google's photorealistic 3D tiles look great from above but melt at street level, so here the drivable geometry is generated from open data + sensor capture, and the photorealism comes from a 3D Gaussian splat of the very same block.

The source is a real capture — PandaSet: six synchronized camera feeds + LiDAR driving one SoMa block. Those frames are turned into a 3D Gaussian Splat (3DGS) — the scene reconstructed as roughly a million tiny colored 3D Gaussians, optimized by differentiable rendering until they reproduce the captured photos from every angle. The result is photoreal and renders in real time right in the browser: the 3DGS build.

A raw splat bakes the captured traffic and the capture vehicle itself into the scene as translucent ghosts. The 3DGS (Masked) build strips them: vehicles and the ego are masked out (from PandaSet's 3D bounding boxes), erased from the images with LaMa inpainting, and the road they hid is filled back in with a frame-consistent top-down IPM road mosaic — leaving the clean static world a sim would populate with its own actors.

Cameras only see road where the car actually drove — about a third of this street was unobserved or caught at angles too oblique to use. The street surface is completed in three layers, each labeled by how much you can trust it. Measured: where cameras saw the road well, the real pixels, untouched. Drawn: lane markings recovered as vectors from LiDAR retroreflectivity and camera paint, then redrawn crisply — their geometry is measured even where no photo exists. Synthesized: the bare asphalt is filled by a diffusion model kept on a short leash — handed a rough color fill and allowed only to match grain and exposure at low strength, never to invent layout, with the markings stamped back on from the vector layer. Every pixel records which layer it came from, so nothing synthesized is ever shown as measurement — and one void the LiDAR says isn't road is left empty on purpose.

Everything owned is license-clean — OpenStreetMap, USGS/open data, and CC-BY PandaSet; Google and Street View are shown only for side-by-side reference, never used as a generation input.