HFGS: 4D Gaussian Splatting with Emphasis on Spatial and Temporal High-Frequency Components for Endoscopic Scene Reconstruction
Robot-assisted minimally invasive surgery benefits from enhancing dynamic scene reconstruction, as it improves surgical outcomes. While Neural Radiance Fields (NeRF) have been effective in scene reconstruction, their slow inference speeds and lengthy training durations limit their applicability. To overcome these limitations, 3D Gaussian Splatting (3D-GS) based methods have emerged as a recent trend, offering rapid inference capabilities and superior 3D quality. However, these methods still struggle with under-reconstruction in both static and dynamic scenes. In this paper, we propose HFGS, a novel approach for deformable endoscopic reconstruction that addresses these challenges from spatial and temporal frequency perspectives. Our approach incorporates deformation fields to better handle dynamic scenes and introduces Spatial High-Frequency Emphasis Reconstruction (SHF) to minimize discrepancies in spatial frequency spectra between the rendered image and its ground truth. Additionally, we introduce Temporal High-Frequency Emphasis Reconstruction (THF) to enhance dynamic awareness in neural rendering by leveraging flow priors, focusing optimization on motion-intensive parts. Extensive experiments on two widely used benchmarks demonstrate that HFGS achieves superior rendering quality.
机器人辅助的微创手术受益于增强动态场景重建，因为这可以改善手术结果。尽管神经辐射场（NeRF）在场景重建方面有效，但其缓慢的推理速度和漫长的训练时间限制了其应用性。为了克服这些限制，基于三维高斯喷溅（3D-GS）的方法已经成为最近的趋势，提供快速的推理能力和优越的3D质量。然而，这些方法在静态和动态场景中仍然面临重建不足的问题。在这篇文章中，我们提出了一种新的方法HFGS，用于可变形的内窥镜重建，从空间和时间频率的角度来解决这些挑战。我们的方法包括变形场以更好地处理动态场景，并引入了空间高频强调重建（SHF），以最小化渲染图像与其真实图像之间的空间频率谱的差异。此外，我们引入了时间高频强调重建（THF），通过利用流先验来增强神经渲染中的动态意识，专注于运动密集部分的优化。在两个广泛使用的基准测试上的广泛实验表明，HFGS实现了优越的渲染质量。