Mujoco rendering produces black image for every MjModel instance called after the first

[sriddle97]

I am running a python program that iterates through multiple mujoco simulations using a different xml file each time. This requires me to execute the following lines for each iteration of the loop:

model_mj = mujoco.MjModel.from_xml_path(file_path)
data_mj = mujoco.MjData(model_mj)
renderer = mujoco.Renderer(model_mj, 400, 900)
mujoco.mj_forward(model_mj,data_mj)
renderer.update_scene(data_mj, camera='fixed')
media.show_image(renderer.render())

The rendering of the first instance of model_mj looks as it should when show_image is executed. However, when show_image is executed in every subsequent iteration, the rendering shows a black screen rather than the proper model. I know the model is still being run in the physics engine as I get expected numerical data outputs from data_mj regardless of the image being correct or blank.

In the past I was able to fix this issue by running each iteration individually and restarting the python kernel manually each time. I assume this was just a roundabout way of restarting the mujoco simulation each time as well. However, I cannot do this when I need the python code to continue running for the following iterations. Is there a way to restart the mujoco environment within the code or at very least some way to get the images to render properly without having to restart the python code? While I am able to get the same numerical data from my simulation, the rendered video is a vital part of my analysis for this project.

Any help would be appreciated.

[sriddle97]

I've tried running mujoco.mj_resetData(model_mj, data_mj) at the end of each loop but it did not fix the issue.

[saran-t]

This looks like a question for https://github.com/google-deepmind/mujoco rather than mujoco-py.

[hqm666]

Have you solved this problem? I have met the same situation recently.

[ruiqiwang96]

This happened to me too. Running the example notebook on google colab works fine, but running it on my PC shows black image from the second time. Renderer gives expected image the first time only, after that it's all black. Since the model works fine and media works fine, I reckon the problem lies in renderer. The use of renderer replies on GPU and it needs to turn on headless rendering model. Based on the example code to set up the EGL on google colab, I confirmed I do have "libEGL_nvidia.so.0" on my PC and created the missing json file / ICD config "10_nvidia.json". Problem solved!

[saran-t]

For future reference:

• The code snippets suggests that this issue relates to the "new" MuJoCo bindings (released in 2022). You should post questions on google-deepmind/mujoco. This repo is for OpenAI's old "mujoco-py" that's no longer maintained or developed. MuJoCo developers don't monitor issues here.

• You mentioning media.show_image suggests that you're trying to visualise interactively. You should check out official Python viewer which is going to give you a lot more than just renders.

[sriddle97]

@hqm666 I've figured out a fix. I have each of my simulations saved as a separate xml file and the paths for these files stored in an array. Each time I iterate to the next sim I call model = mujoco.MjModel.from_xml_path(array[i]) and then the following:
renderer = mujoco.Renderer(model)

Then after running the sim and generating the frames each timestep I run the following:
media.write_video(vid_names, frames, fps=framerate)
renderer.close()

The renderer.close() after media.write_video() should "clear the cache" so to speak and make way for the next mujoco.Renderer() command when the next sim is iterated.