Efficient acquisition of real-world embodied data has been increasingly critical. However, large-scale demonstrations captured by remote operation tend to take extremely high costs and fail to scale up the data size in an efficient manner. Sampling the episodes under a simulated environment is a promising way for large-scale collection while existing simulators fail to high-fidelity modeling on texture and physics. To address these limitations, we introduce the RoboGSim, a real2sim2real robotic simulator, powered by 3D Gaussian Splatting and the physics engine. RoboGSim mainly includes four parts: Gaussian Reconstructor, Digital Twins Builder, Scene Composer, and Interactive Engine. It can synthesize the simulated data with novel views, objects, trajectories, and scenes. RoboGSim also provides an online, reproducible, and safe evaluation for different manipulation policies. The real2sim and sim2real transfer experiments show a high consistency in the texture and physics. Moreover, the effectiveness of synthetic data is validated under the real-world manipulated tasks. We hope RoboGSim serves as a closed-loop simulator for fair comparison on policy learning.
高效获取真实世界的具身数据变得越来越重要。然而，通过远程操作捕获的大规模示范数据成本极高，难以有效扩大数据规模。在模拟环境下采样任务片段是一种有前景的大规模数据收集方式，但现有模拟器在纹理和物理建模的高保真性方面存在不足。为了解决这些限制，我们提出了 RoboGSim，一个基于真实到模拟再回归真实（real2sim2real）流程的机器人模拟器，由 3D 高斯点绘制和物理引擎驱动。 RoboGSim 主要包括四个模块：高斯重构器（Gaussian Reconstructor）、数字孪生构建器（Digital Twins Builder）、场景编辑器（Scene Composer）以及交互引擎（Interactive Engine）。它可以合成具有新视角、对象、轨迹和场景的模拟数据，并提供在线、可复现且安全的环境，用于评估不同的操作策略。通过真实到模拟和模拟到真实的实验验证，RoboGSim 在纹理和物理表现上展现了高度一致性。此外，实验还表明，合成数据在真实世界中的操作任务中具有显著的有效性。 我们希望 RoboGSim 能作为一个闭环模拟器，为策略学习的公平比较提供支持。