Sim2Real transfer, particularly for manipulation policies relying on RGB images, remains a critical challenge in robotics due to the significant domain shift between synthetic and real-world visual data. In this paper, we propose SplatSim, a novel framework that leverages Gaussian Splatting as the primary rendering primitive to reduce the Sim2Real gap for RGB-based manipulation policies. By replacing traditional mesh representations with Gaussian Splats in simulators, SplatSim produces highly photorealistic synthetic data while maintaining the scalability and cost-efficiency of simulation. We demonstrate the effectiveness of our framework by training manipulation policies within SplatSim}and deploying them in the real world in a zero-shot manner, achieving an average success rate of 86.25%, compared to 97.5% for policies trained on real-world data.
Sim2Real 转移，尤其是对于依赖 RGB 图像的操作策略，在机器人学中仍然是一个关键挑战，因为合成与真实世界视觉数据之间存在显著的领域差异。在本文中，我们提出了 SplatSim，一种利用高斯投影作为主要渲染基元的全新框架，以减少 RGB 基操作策略的 Sim2Real 差距。通过在模拟器中用高斯投影替代传统的网格表示，SplatSim 能生成高度逼真的合成数据，同时保持模拟的可扩展性和成本效益。我们展示了该框架的有效性，操控策略在 SplatSim 中训练并直接部署到现实世界中进行零样本测试，取得了平均 86.25% 的成功率，相比于在真实数据上训练的策略成功率为 97.5%。