PSAvatar: A Point-based Morphable Shape Model for Real-Time Head Avatar Creation with 3D Gaussian Splatting
Despite much progress, creating real-time high-fidelity head avatar is still difficult and existing methods have to trade-off between speed and quality. 3DMM based methods often fail to model non-facial structures such as eyeglasses and hairstyles, while neural implicit models suffer from deformation inflexibility and rendering inefficiency. Although 3D Gaussian has been demonstrated to possess promising capability for geometry representation and radiance field reconstruction, applying 3D Gaussian in head avatar creation remains a major challenge since it is difficult for 3D Gaussian to model the head shape variations caused by changing poses and expressions. In this paper, we introduce PSAvatar, a novel framework for animatable head avatar creation that utilizes discrete geometric primitive to create a parametric morphable shape model and employs 3D Gaussian for fine detail representation and high fidelity rendering. The parametric morphable shape model is a Point-based Morphable Shape Model (PMSM) which uses points instead of meshes for 3D representation to achieve enhanced representation flexibility. The PMSM first converts the FLAME mesh to points by sampling on the surfaces as well as off the meshes to enable the reconstruction of not only surface-like structures but also complex geometries such as eyeglasses and hairstyles. By aligning these points with the head shape in an analysis-by-synthesis manner, the PMSM makes it possible to utilize 3D Gaussian for fine detail representation and appearance modeling, thus enabling the creation of high-fidelity avatars. We show that PSAvatar can reconstruct high-fidelity head avatars of a variety of subjects and the avatars can be animated in real-time (≥ 25 fps at a resolution of 512 x 512 )
尽管取得了很大进展，但创建实时高保真头部化身仍然是一项挑战，现有方法必须在速度和质量之间进行权衡。基于3DMM的方法通常无法模拟非面部结构，如眼镜和发型，而神经隐式模型则受困于变形的不灵活性和渲染效率低下。 尽管3D高斯在几何表示和辐射场重建方面已显示出有希望的能力，但在头部化身创建中应用3D高斯仍然是一个主要挑战，因为3D高斯难以模拟由变化的姿势和表情引起的头部形状变化。在这篇文章中，我们介绍了PSAvatar，这是一个用于可动画头部化身创建的新颖框架，它利用离散几何原语来创建参数化可变形状模型，并采用3D高斯进行细节表示和高保真渲染。这个参数化可变形状模型是一个基于点的可变形状模型（PMSM），它使用点而不是网格进行3D表示，以实现更强的表示灵活性。PMSM首先通过在表面上以及网格外采样将FLAME网格转换为点，以实现不仅能重建表面类结构，而且能重建如眼镜和发型等复杂几何结构。通过以分析综合的方式将这些点与头部形状对齐，PMSM使得可以利用3D高斯进行细节表示和外观建模，从而实现高保真化身的创建。我们展示了PSAvatar可以重建各种对象的高保真头部化身，这些化身可以实时动画化（≥25 fps，分辨率为512 x 512）。