GeoVerse: Novel View Synthesis in Geometric Latent Space with World Consistency

Meta description: GeoVerse uses a geometric latent diffusion model, Wan2.2 VACE appearance priors and global spatial memory to improve sparse-image novel view synthesis.

Tags: GeoVerse, Novel View Synthesis, Wan2.2 VACE, 3D Foundation Model, Diffusion Models

Featured image: Abstract technology-style visualization of sparse camera views, 3D point clouds, latent diffusion grids and shared spatial memory, with no real people, no company logos and no text.

GeoVerse is a framework for novel view synthesis from sparse images that tries to keep observed scene structure intact while inventing plausible detail for areas no camera captured. In the reported evaluations, it delivers 2.23 dB higher PSNR on DL3DV and 32.4% lower ATE on Mip-NeRF360 than GLD. Those numbers matter because the field has been split between methods that are geometrically careful and methods that are visually convincing, and sparse-image 3D work needs both.

Background: The Tradeoff Between Seen and Unseen

The task starts with only a few images of a scene. The system must render new viewpoints that remain consistent with those inputs. This is useful in workflows where capture is limited, such as 3D content creation and immersive media. The challenge is that a small set of views contains real evidence only in some regions. The rest must be inferred.

Geometry-focused approaches are good at preserving surfaces that were observed. They anchor reconstruction to the camera inputs, which helps maintain scale and structure. But when the camera moves to an unseen area, those same systems can struggle to fill the missing visual content convincingly. Video generative models solve part of that problem. They can produce realistic textures and plausible scene details. Yet when they generate a sequence of views, errors can compound. Each new frame may look attractive on its own, but the set of frames may no longer describe one stable 3D scene.

Development Process: Geometry First, Appearance Injected

GeoVerse addresses that split by combining a geometric backbone with an appearance prior. The generation process operates inside the geometric latent space of a pretrained 3D foundation model. That gives the system a spatial reference frame instead of starting from isolated image synthesis.

The appearance side comes from Wan2.2 VACE. The framework extracts multilevel features from this video generative model and injects them into a geometric latent diffusion model through a ControlNet-style adapter. This incorporates video-learned appearance priors to enhance structural completion without overriding the underlying geometry.

The second major component is a global spatial memory. It aggregates both observed and synthesized content. The memory reprojects target-aligned guidance so that later generated views are anchored to a shared scene representation. This is designed to prevent each new viewpoint from drifting away from the scene that previous views established.

Latest Progress: What the DL3DV and Mip-NeRF360 Results Show

In extensive experiments across diverse datasets, the highlighted results are on DL3DV and Mip-NeRF360. On DL3DV, GeoVerse records 2.23 dB higher PSNR than GLD. PSNR, peak signal-to-noise ratio, is an image fidelity score; higher values generally mean the synthesized view is closer to the reference view. On Mip-NeRF360, GeoVerse records 32.4% lower ATE than GLD. ATE, absolute trajectory error, is a lower-is-better measure of trajectory alignment, so the reported gain points to stronger geometric consistency across views.

The use of both metrics is important. PSNR captures how well individual images match reference quality, while ATE captures how well the synthesized sequence stays aligned with the expected viewpoint trajectory. A method could improve one and regress the other, which is why the paired result is more informative than a single visual quality score.

Current Stage: A Joint Geometry and Appearance Evaluation

GeoVerse now sits in a stage where sparse-image novel view synthesis is being tested as a joint geometry-appearance problem. The method's reported gains on DL3DV and Mip-NeRF360 suggest that a geometric latent space, video-derived appearance priors, and shared spatial memory can improve both reconstruction fidelity and cross-view consistency. The open questions are how stable those gains remain when inputs are even sparser, when scenes contain more ambiguous texture, and when the global spatial memory must scale to larger or more dynamic environments.