An excellent representation is crucial for reinforcement learning (RL) performance, especially in vision-based reinforcement learning tasks. The quality of the environment representation directly influences the achievement of the learning task. Previous vision-based RL typically uses explicit or implicit ways to represent environments, such as images, points, voxels, and neural radiance fields. However, these representations contain several drawbacks. They cannot either describe complex local geometries or generalize well to unseen scenes, or require precise foreground masks. Moreover, these implicit neural representations are akin to a ``black box", significantly hindering interpretability. 3D Gaussian Splatting (3DGS), with its explicit scene representation and differentiable rendering nature, is considered a revolutionary change for reconstruction and representation methods. In this paper, we propose a novel Generalizable Gaussian Splatting framework to be the representation of RL tasks, called GSRL. Through validation in the RoboMimic environment, our method achieves better results than other baselines in multiple tasks, improving the performance by 10%, 44%, and 15% compared with baselines on the hardest task. This work is the first attempt to leverage generalizable 3DGS as a representation for RL.
在强化学习（RL）中，尤其是视觉基础的强化学习任务中，出色的表现对于性能至关重要。环境表征的质量直接影响学习任务的成果。以往基于视觉的RL通常使用显式或隐式的方式来表征环境，如图像、点、体素和神经辐射场。然而，这些表征存在一些缺陷。它们要么无法描述复杂的局部几何形状，要么在未见场景中泛化能力差，或者需要精确的前景遮罩。此外，这些隐式的神经表征类似于“黑盒”，极大地阻碍了可解释性。3D高斯喷涂（3DGS），凭借其显式的场景表现和可微渲染的特性，被认为是重建和表征方法的革命性变革。在本文中，我们提出了一种名为GSRL的新型可泛化高斯喷涂框架，作为RL任务的表征。通过在RoboMimic环境中的验证，我们的方法在多个任务中比其他基线模型取得了更好的结果，在最难的任务上相比基线模型提高了10%、44%和15%的性能。这项工作是首次尝试利用可泛化的3DGS作为RL的表征。