Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al. 2023] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [Rückert et al. 2022] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud.
In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions.
Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage.
基于点的辐射场渲染已经在新视角合成方面展示出令人印象深刻的结果，提供了渲染质量和计算效率的引人注目的结合。然而，即使是该领域的最新方法也不是没有缺点。3D高斯喷溅[Kerbl和Kopanas等，2023]在渲染高细节场景时会遇到困难，因为它会导致模糊和云状伪影。另一方面，ADOP [Rückert等，2022]能够呈现更清晰的图像，但是神经重建网络降低了性能，它还难以处理时间上的不稳定性，并且无法有效应对点云中的大型间隙。
在本文中，我们介绍了TRIPS（三线性点喷溅），这是一种结合了高斯喷溅和ADOP思想的方法。我们新技术背后的基本概念涉及将点光栅化成屏幕空间图像金字塔，金字塔层的选择由投影点的大小决定。这种方法允许使用单次三线性写入来渲染任意大的点。然后使用一个轻量级神经网络来重建包括超出喷溅分辨率细节的无孔图像。重要的是，我们的渲染管线是完全可微分的，允许对点的大小和位置进行自动优化。
我们的评估表明，TRIPS在渲染质量方面超越了现有的最先进方法，同时在现成硬件上保持每秒60帧的实时帧率。这种性能扩展到具有挑战性的场景，如具有复杂几何形状、广阔景观和自动曝光镜头的场景。