Bug when update to 1.5.0

[Wu-Fisher]

System Info

robosuite                  1.5.0

Information

1. After updating to version 1.5.0, the screen freezes after rendering for a while, but the environment (env) should still be able to step and reset normally.
2. IK controller for part-robot isnt work well and no template config(xx.json) for arm ik control.
3. Some demos do not work. I thk controller module still has some problems.

Reproduction

1. for any demo

for i_episode in range(20):
      ### **next time here off screen will freeze**
        observation = env.reset()
        for t in range(500):
            env.render()
            action = env.action_space.sample()
            observation, reward, terminated, truncated, info = env.step(action)
            if terminated or truncated:
                print("Episode finished after {} timesteps".format(t + 1))
                observation, info = env.reset()
                env.close()
                break

2. Repo didnt contain such file "controllers/config/parts/{}.json"

def load_part_controller_config(custom_fpath=None, default_controller=None):
###...
        custom_fpath = os.path.join(
            os.path.dirname(__file__), "..", "controllers/config/parts/{}.json".format(default_controller.lower())
        )

###...
    return controller_config

3. python robosuite/demos/demo_control.py >[1] Lift >[3] Panda > [0-4] >

  File "/home/wufisher/workspace/dev/robosuite/robosuite/demos/demo_control.py", line 95, in <module>
    options["controller_configs"] = suite.load_part_controller_config(default_controller=controller_name)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wufisher/workspace/dev/robosuite/robosuite/controllers/parts/controller_factory.py", line 48, in load_part_controller_config
    default_controller in ALL_PART_CONTROLLERS
AssertionError: Error: Unknown default controller specified. Requested BASIC, available controllers: ['JOINT_VELOCITY', 'JOINT_TORQUE', 'JOINT_POSITION', 'OSC_POSITION', 'OSC_POSE', 'IK_POSE']

Expected behavior

No response

[kevin-thankyou-lin]

@youliangtan, seems like these tests/demos didn't get fixed in that PR - do you think you could take a stab?

@Wu-Fisher For now, please use the [collect_human_demos](https://github.com/ARISE-Initiative/robosuite/blob/master/robosuite/scripts/collect_human_demonstrations.py) script for running different controllers

[Wu-Fisher]

@kevin-thankyou-lin Thank you for providing the script for reference. However, recently I’ve been focusing on a robotic arm project (the script seems more suited for humanoid robots). Robosuite 1.5 indeed introduced many updates❤️, such as inverse kinematics no longer relying on Bullet (a week ago, when I was working with version 1.4, I found the initialization for fixed position IK solutions to be quite cumbersome, like using set_goal(delta, set_ik=True)). However, after the update yesterday, I noticed that there are still many deficiencies in the configuration, so I’m looking forward to seeing more comprehensive support for robotic arms soon. If I can help, I’ll also try to submit a PR.

[kevin-thankyou-lin]

Certainly - what were you referring to by I noticed that there are still many deficiencies in the configuration?

For fixed arm testing, this script should work: python robosuite/scripts/collect_human_demonstrations.py --robots Panda

[Wu-Fisher]

My imprecise description led to a misunderstanding😢. robosuite/scripts/collect_human_demonstrations.py --robots Panda" works really well and serves as a great reference for collecting expert data. My previous comment was referring to the fact that after migrating from version 1.4 to 1.5 yesterday, controller configurations such as "load_part_controller_config" don’t seem to have corresponding support.

[kevin-thankyou-lin]

Gotcha, thanks for raising this issue - we'll look into a fix

[ModiShi]

Gotcha, thanks for raising this issue - we'll look into a fix

Hello, I have met the same question, seems that there is no controllers/config/parts/{}.json for default controllers, what should I do if I wanna use OSC_POSE controller, could I use this config file?

[Wu-Fisher]

Gotcha, thanks for raising this issue - we'll look into a fix

Hello, I have met the same question, seems that there is no controllers/config/parts/{}.json for default controllers, what should I do if I wanna use OSC_POSE controller, could I use this config file?

Do you mean using impl part OSC_POSE controller(not highly depend on cls Robot())?
If just ok for a single-arm robot, the default controller is OSC_POSE, and It works well. config(robosuite/scripts/collect_human_demonstrations.py)

[ModiShi]

Hello, I have met the same question, seems that there is no controllers/config/parts/{}.json for default controllers, what should I do if I wanna use OSC_POSE controller, could I use this config file?

Do you mean using impl part OSC_POSE controller(not highly depend on cls Robot())? If just ok for a single-arm robot, the default controller is OSC_POSE, and It works well. config(robosuite/scripts/collect_human_demonstrations.py)

Thanks for your reply. Actually, I am just trying to load controller config like this

from robosuite import load_part_controller_config
config = load_part_controller_config(default_controller='OSC_POSE')

then I get

Traceback (most recent call last):
  File ".../robosuite/robosuite/controllers/parts/controller_factory.py", line 63, in load_part_controller_config
    with open(custom_fpath) as f:
         ^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '.../robosuite/robosuite/controllers/parts/../controllers/config/parts/osc_pose.json'

[Wu-Fisher]

@ModiShi Another suggestion, now i think that "load_part_controller_config" might have no enough support and exmple too.
"Part(Arm)" controller load refer:robot.py
And merge your arm config with a robot and it will work.

    def _load_arm_controllers(self):
        urdf_loaded = False
        # Load composite controller configs for both left and right arm
        for arm in self.arms:
            # Assert that the controller config is a dict file:
            #             NOTE: "type" must be one of: {JOINT_POSITION, JOINT_TORQUE, JOINT_VELOCITY,
            #                                           OSC_POSITION, OSC_POSE, IK_POSE}
            assert (
                type(self.part_controller_config[arm]) == dict
            ), "Inputted controller config must be a dict! Instead, got type: {}".format(
                type(self.part_controller_config[arm])
            )

            # Add to the controller dict additional relevant params:
            #   the robot name, mujoco sim, eef_name, actuator_range, joint_indexes, timestep (model) freq,
            #   policy (control) freq, and ndim (# joints)
            self.part_controller_config[arm]["robot_name"] = self.name
            self.part_controller_config[arm]["sim"] = self.sim
            self.part_controller_config[arm]["ref_name"] = self.gripper[arm].important_sites["grip_site"]
            self.part_controller_config[arm]["part_name"] = arm
            self.part_controller_config[arm]["naming_prefix"] = self.robot_model.naming_prefix

            self.part_controller_config[arm]["eef_rot_offset"] = self.eef_rot_offset[arm]
            self.part_controller_config[arm]["ndim"] = self._joint_split_idx
            self.part_controller_config[arm]["policy_freq"] = self.control_freq
            self.part_controller_config[arm]["lite_physics"] = self.lite_physics
            (start, end) = (None, self._joint_split_idx) if arm == "right" else (self._joint_split_idx, None)
            self.part_controller_config[arm]["joint_indexes"] = {
                "joints": self.arm_joint_indexes[start:end],
                "qpos": self._ref_arm_joint_pos_indexes[start:end],
                "qvel": self._ref_arm_joint_vel_indexes[start:end],
            }
            self.part_controller_config[arm]["actuator_range"] = (
                self.torque_limits[0][start:end],
                self.torque_limits[1][start:end],
            )

            # Only load urdf the first time this controller gets called
            self.part_controller_config[arm]["load_urdf"] = True if not urdf_loaded else False
            urdf_loaded = True

            # # Instantiate the relevant controller

            if self.has_gripper[arm]:
                # Load gripper controllers
                assert "gripper" in self.part_controller_config[arm], "Gripper controller config not found!"
                gripper_name = self.get_gripper_name(arm)
                self.part_controller_config[gripper_name] = {}
                self.part_controller_config[gripper_name]["type"] = self.part_controller_config[arm]["gripper"]["type"]
                self.part_controller_config[gripper_name]["use_action_scaling"] = self.part_controller_config[arm][
                    "gripper"
                ].get("use_action_scaling", True)
                self.part_controller_config[gripper_name]["robot_name"] = self.name
                self.part_controller_config[gripper_name]["sim"] = self.sim
                self.part_controller_config[gripper_name]["eef_name"] = self.gripper[arm].important_sites["grip_site"]
                self.part_controller_config[gripper_name]["part_name"] = gripper_name
                self.part_controller_config[gripper_name]["naming_prefix"] = self.robot_model.naming_prefix
                self.part_controller_config[gripper_name]["ndim"] = self.gripper[arm].dof
                self.part_controller_config[gripper_name]["policy_freq"] = self.control_freq
                self.part_controller_config[gripper_name]["joint_indexes"] = {
                    "joints": self.gripper_joints[arm],
                    "actuators": self._ref_joint_gripper_actuator_indexes[arm],
                    "qpos": self._ref_gripper_joint_pos_indexes[arm],
                    "qvel": self._ref_gripper_joint_vel_indexes[arm],
                }
                low = self.sim.model.actuator_ctrlrange[self._ref_joint_gripper_actuator_indexes[arm], 0]
                high = self.sim.model.actuator_ctrlrange[self._ref_joint_gripper_actuator_indexes[arm], 1]

                self.part_controller_config[gripper_name]["actuator_range"] = (low, high)

[ModiShi]

@Wu-Fisher got it, thank you🥳🥳

[kevin-thankyou-lin]

Hi folks, great to see you've come up with a solution. We've updated the latest master branch based on this PR to support the part controllers, with an example in demo_control.py

[kevin-thankyou-lin]

@Wu-Fisher this PR recently fixed the IK controller - you can pull from master branch and try it out - thanks!

[kevin-thankyou-lin]

Closing this issue for now - please re-open if there are other issues. Thanks!