Sim-to-Real refers to the process of transferring policies learned in simulation to the real world, which is crucial for achieving practical robotics applications. However, recent Sim2real methods either rely on a large amount of augmented data or large learning models, which is inefficient for specific tasks. In recent years, radiance field-based reconstruction methods, especially the emergence of 3D Gaussian Splatting, making it possible to reproduce realistic real-world scenarios. To this end, we propose a novel real-to-sim-to-real reinforcement learning framework, RL-GSBridge, which introduces a mesh-based 3D Gaussian Splatting method to realize zero-shot sim-to-real transfer for vision-based deep reinforcement learning. We improve the mesh-based 3D GS modeling method by using soft binding constraints, enhancing the rendering quality of mesh models. We then employ a GS editing approach to synchronize rendering with the physics simulator, reflecting the interactions of the physical robot more accurately. Through a series of sim-to-real robotic arm experiments, including grasping and pick-and-place tasks, we demonstrate that RL-GSBridge maintains a satisfactory success rate in real-world task completion during sim-to-real transfer. Furthermore, a series of rendering metrics and visualization results indicate that our proposed mesh-based 3D Gaussian reduces artifacts in unstructured objects, demonstrating more realistic rendering performance.
Sim-to-Real 是指将模拟中学习的策略转移到现实世界，这对于实现实际的机器人应用至关重要。然而，现有的 Sim2Real 方法往往依赖于大量增强数据或庞大的学习模型，这对于某些特定任务而言效率较低。近年来，基于辐射场的重建方法，尤其是3D高斯分布（3D Gaussian Splatting）的出现，使得再现逼真的现实场景成为可能。为此，我们提出了一种新颖的“真实-模拟-真实”强化学习框架，称为RL-GSBridge，利用基于网格的3D高斯分布方法实现视觉深度强化学习的零样本Sim-to-Real迁移。我们通过软绑定约束改进了基于网格的3D GS建模方法，提升了网格模型的渲染质量。随后，我们采用高斯分布编辑方法，将渲染与物理模拟器同步，更准确地反映了物理机器人交互。通过一系列Sim-to-Real的机械臂实验，包括抓取和挑拣任务，我们展示了RL-GSBridge在Sim-to-Real迁移过程中保持了令人满意的任务完成成功率。此外，一系列渲染指标和可视化结果表明，我们提出的基于网格的3D高斯方法减少了非结构化物体中的伪影，展现出更逼真的渲染效果。