학술논문

XScale- NVS: Cross-Scale Novel View Synthesis with Hash Featurized Manifold
Document Type
Conference
Source
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) CVPR Computer Vision and Pattern Recognition (CVPR), 2024 IEEE/CVF Conference on. :21029-21039 Jun, 2024
Subject
Computing and Processing
Manifolds
Computer vision
Scalability
Color
Benchmark testing
Rendering (computer graphics)
Distortion
Novel View Synthesis
Neural Rendering
Large-scale Scene Reconstruction
Language
ISSN
2575-7075
Abstract
We propose XScale-NVS for high-fidelity cross-scale novel view synthesis of real-world large-scale scenes. Ex-isting representations based on explicit surface suffer from discretization resolution or $UV$ distortion, while implicit volumetric representations lack scalability for large scenes due to the dispersed weight distribution and surface ambi-guity. In light of the above challenges, we introduce hash featurized manifold, a novel hash-based featurization cou-pled with a deferred neural rendering framework. This approach fully unlocks the expressivity of the representation by explicitly concentrating the hash entries on the $2D$ manifold, thus effectively representing highly detailed contents independent of the discretization resolution. We also in-troduce a novel dataset, namely $Gig{a}NVS$, to benchmark cross-scale, high-resolution novel view synthesis of real-world large-scale scenes. Our method significantly outper-forms competing baselines on various real-world scenes, yielding an average LPIPS that is $\sim$ 40% lower than prior state-of-the-art on the challenging $Gig{a}NVS$ benchmark. Please see our project page at: xscalenvs.github.io.