With the emergence of large-scale Text-to-Image(T2I) models and implicit 3D representations like Neural Radiance Fields (NeRF), many text-driven generative editing methods based on NeRF have appeared. However, the implicit encoding of geometric and textural information poses challenges in accurately locating and controlling objects during editing. Recently, significant advancements have been made in the editing methods of 3D Gaussian Splatting, a real-time rendering technology that relies on explicit representation. However, these methods still suffer from issues including inaccurate localization and limited manipulation over editing. To tackle these challenges, we propose GSEditPro, a novel 3D scene editing framework which allows users to perform various creative and precise editing using text prompts only. Leveraging the explicit nature of the 3D Gaussian distribution, we introduce an attention-based progressive localization module to add semantic labels to each Gaussian during rendering. This enables precise localization on editing areas by classifying Gaussians based on their relevance to the editing prompts derived from cross-attention layers of the T2I model. Furthermore, we present an innovative editing optimization method based on 3D Gaussian Splatting, obtaining stable and refined editing results through the guidance of Score Distillation Sampling and pseudo ground truth. We prove the efficacy of our method through extensive experiments.
随着大规模文本生成图像（Text-to-Image, T2I）模型和神经辐射场（Neural Radiance Fields, NeRF）等隐式3D表示的兴起，许多基于NeRF的文本驱动生成编辑方法相继出现。然而，隐式编码几何和纹理信息的方式在编辑中准确定位和控制对象方面仍面临挑战。最近，基于3D Gaussian Splatting 的实时渲染技术在编辑方法上取得了显著进展，该技术依赖显式表示。然而，这些方法仍然存在定位不准确和编辑操控性有限的问题。 为了解决这些挑战，我们提出了 GSEditPro，一种全新的3D场景编辑框架，允许用户仅使用文本提示进行多种创造性且精确的编辑。通过利用3D高斯分布的显式特性，我们引入了一种基于注意力的渐进定位模块，在渲染过程中为每个高斯添加语义标签。该模块通过T2I模型交叉注意力层生成的编辑提示，基于高斯的相关性对其进行分类，从而实现编辑区域的精准定位。 此外，我们提出了一种基于3D Gaussian Splatting 的创新编辑优化方法，结合得分蒸馏采样（Score Distillation Sampling）和伪真实值（pseudo ground truth）的引导，获得稳定且精细的编辑结果。通过广泛的实验验证，我们证明了该方法的有效性。