Improving Photogrammetric Digital Surface Models via Pretrained Diffusion Models and Multimodal Conditioning

Meta description: Stable Diffusion 3 refines satellite DSMs with Pléiades imagery, cutting dense urban RMSE from 6.00 to 3.45 m in France and to 2.77 m in Bordeaux.

Tags: Stable Diffusion 3, Digital Surface Models, satellite photogrammetry, LiDAR, diffusion models

Featured image: Abstract technology style: a 3D city elevation grid with colored height contours, satellite imagery patches, and diffusion network nodes; no people, logos, or text.

A modified Stable Diffusion 3 model is being tested for DSM refinement, using satellite-derived elevation grids and Pléiades imagery to correct noisy surface maps. The result matters because it addresses a long-standing trade-off: photogrammetry makes wide-area 3D elevation products cheaper to create, while LiDAR supplies the high-accuracy vertical values that many applications need. In dense urban areas, the work reports root mean square error falling from 6.00 m to 3.45 m in in-context cities and from 4.16 m to 2.77 m in the held-out city of Bordeaux.

From cheap maps to noisy gaps

Satellite stereo-photogrammetry, a method that derives height from overlapping views, offers a low-cost route to large-area Digital Surface Models, or DSMs. A DSM represents the elevation of the visible surface, including buildings, trees and terrain. It is useful for city-scale 3D modeling, terrain analysis and planning. Yet the finished 3D products frequently carry three defects: scattered noise, isolated outliers and empty gaps. Noise roughens smooth surfaces, outliers put blocks or streets at the wrong height, and gaps leave areas without usable values.

Aerial LiDAR, which measures elevation with laser returns, sets a stricter vertical standard, but the price is substantially higher. It is often used where accuracy is worth the added cost. The study is placed between these two workflows: it wants to keep the breadth of photogrammetric DSMs and move their vertical quality closer to LiDAR-derived references.

Development: Stable Diffusion 3 tuned for elevation

Diffusion models are generative models that learn to remove noise from a target signal. Here, the model is conditioned on two sources: a photogrammetric DSM and Pléiades imagery. Pléiades is a high-resolution satellite imagery source. The task is to refine vertically co-registered DSMs, meaning the elevation surfaces have been aligned in height so remaining differences can be treated as correctable errors rather than misregistration.

The key architecture change is a modified Stable Diffusion 3 model. Stable Diffusion 3 is best known for generating images from text prompts, but the text stream is pruned. Patch-wise normalization is also added, normalizing data at the local patch level. Those changes are intended to make training on LiDAR data stable and to allow transfer from natural-image patterns to elevation-map patterns. DSM grids differ from photographs in value scale, geometry and what each pixel means, so a standard image generator may not adapt without adjustments.

Latest results: French cities and Bordeaux generalization

The experiments take place in French cities. Multimodal conditioning improves elevation accuracy in dense urban settings. Dense Urban RMSE drops from 6.00 m to 3.45 m in the in-context cities. These are the locations included in the conditioning setup, so the model can draw directly on their patterns.

The held-out city of Bordeaux gives the more important signal for transfer. There, Dense Urban RMSE drops from 4.16 m to 2.77 m. A reduction to 2.77 m in a city outside the in-context setting suggests the method is not only cleaning sites used for the demonstration. It still leaves open how far the approach can go beyond the tested urban contexts.

Current stage: technical proof with numbers attached

The work currently sits at a proof-of-concept stage. It does not announce a production map service or a fully automated replacement for LiDAR. It reports a concrete technical milestone: Stable Diffusion 3 can be adapted with a pruned text stream and patch-wise normalization to refine DSMs, and the French-city tests produce Dense Urban RMSE values of 3.45 m in in-context cities and 2.77 m in Bordeaux.

What to watch next is whether the same error pattern holds across more varied terrain types, building densities, satellite image conditions and registration quality. The cost question also remains open: if the method needs LiDAR training data and careful co-registration, the savings over direct LiDAR acquisition may be narrower than the photogrammetric starting point suggests.