Recent advances in radiance fields and novel view synthesis enable creation of realistic digital twins from photographs. However, current methods struggle with flat, texture-less surfaces, creating uneven and semi-transparent reconstructions, due to an ill-conditioned photometric reconstruction objective. Surface reconstruction methods solve this issue but sacrifice visual quality. We propose a novel hybrid 2D/3D representation that jointly optimizes constrained planar (2D) Gaussians for modeling flat surfaces and freeform (3D) Gaussians for the rest of the scene. Our end-to-end approach dynamically detects and refines planar regions, improving both visual fidelity and geometric accuracy. It achieves state-of-the-art depth estimation on ScanNet++ and ScanNetv2, and excels at mesh extraction without overfitting to a specific camera model, showing its effectiveness in producing high-quality reconstruction of indoor scenes.

Our hybrid representation enables precise reconstruction of volumetric details such as fur, while accurately capturing planar geometry for solid flat surfaces like walls, without compromising either geometry or appearance, as shown in the figure below. Render of a fox scene (left), checkerboard pattern that shows our detected 3D plane in the scene (middle), depth render (right).

Our method achieves a photorealistic quality on par with fully 3D approaches, while improving geometry over surface reconstruction methods, e.g. no visible hole in the middle of the `garden' scene from Mip-NeRF360.

Proposed method reduced popping artifacts on planar surfaces, e.g. on the floor and the door in the scene below, compared to baselines.

The method faithfully reconstructs planar surfaces and reduces floaters, e.g. table and fridge in the scene below, while preserving high-quality details on non-planar parts of the scene.