Interactive 3D segmentation in radiance fields is an appealing task since its importance in 3D scene understanding and manipulation. However, existing methods face challenges in either achieving fine-grained, multi-granularity segmentation or contending with substantial computational overhead, inhibiting real-time interaction. In this paper, we introduce Segment Any 3D GAussians (SAGA), a novel 3D interactive segmentation approach that seamlessly blends a 2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D segmentation results generated by the segmentation foundation model into 3D Gaussian point features through well-designed contrastive training. Evaluation on existing benchmarks demonstrates that SAGA can achieve competitive performance with state-of-the-art methods. Moreover, SAGA achieves multi-granularity segmentation and accommodates various prompts, including points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation within milliseconds, achieving nearly 1000x acceleration compared to previous SOTA.
交互式3D辐射场分割是一个吸引人的任务，因为它在3D场景理解和操纵中非常重要。然而，现有方法在实现细粒度、多粒度分割或应对大量计算开销方面面临挑战，这限制了实时互动。在这篇论文中，我们介绍了“分割任意3D高斯”（SAGA），这是一种新颖的3D交互式分割方法，它将2D分割基础模型与3D高斯喷溅（3DGS）无缝融合，后者是辐射场的最新突破。SAGA通过精心设计的对比训练，高效地将分割基础模型生成的多粒度2D分割结果嵌入到3D高斯点特征中。在现有基准测试上的评估表明，SAGA能够与最先进的方法竞争。此外，SAGA实现了多粒度分割，并适应各种提示，包括点、涂鸦和2D蒙版。值得注意的是，SAGA可以在几毫秒内完成3D分割，与以前的最先进技术相比，几乎实现了1000倍的加速。