[Proposal] Support custom width, height, and scene geometries in the MuJoCo renderer

[guyazran]

Proposal

I want to add the option to choose the width and height of the offscreen renderer outputs individually and not rely on the visual settings in the XML. I also want to be able to control the maxgeom limit set in the base renderer class.

Motivation

The window viewer implementation for mujoco rendering in gymnasium is the best one out there. But using this viewer and another viewer for offscreen rendering yields glfw errors. The offscreen viewer class implementation has two flaws that prevent me from using it in my project:

1. offscreen renderers have dimensions that are specific to the visual settings in the XML.
2. the maxgeom setting is at a low 1000 and my scenes are very cluttered.

Pitch

1. support for offscreen renderers to use specific height and width, similar to the mujoco.Renderer class.
2. control the maxgeom setting when creating the scene object in the base viewer class.

Alternatives

No response

Additional context

I have a pull request lined up with the required implementation that I will upload shortly

Checklist

• I have checked that there is no similar issue in the repo

[Kallinteris-Andreas]

1. what MuJoCo Model are you running
2. MuJuCo environments already have height and width arguments

[guyazran]

1. You mean XML model? we use our own custom model
2. yes, but if I want to render multiple cameras then I cannot control their resolution independently

[Kallinteris-Andreas]

1. are you using your own custom MuJocoEnv class?
2. the Gymnasium API does not support multiple render view ports (i. e. multiple cameras), render() returns a single image not a list of images

[guyazran]

1. yes
2. I have a fetch robot with a camera on its head and another bird's-eye view camera. both cameras are a part of the observation space and have different resolutions. is this not possible?

[Kallinteris-Andreas]

2. Unfortunately the current API does not support, multiple cameras (and by extension not multiple cameras with different resolutions)

The API we should probably extend it to support that (we have gotten few requests for that)

Never mind my previous comments regarding the gymnasium API not supporting multiple camera, that was wrong sorry (

Gymnasium/gymnasium/envs/mujoco/mujoco_env.py

Line 39 in 54d2ba7

width: int = DEFAULT_SIZE,

)

That being said the MuJoCo renderer does not currently support that feature, that should be patched in

3. do you want to use both your cameras with rgb_array

[guyazran]

3. currently yes, but in the future I would like to also use depth, and even segmentation images.

[guyazran]

here is an example use-case: example.zip

I ran this on MacOS 14.0 via anaconda python version 3.11. This uses the code from my pull request #731, installed using the provided environment.yml file. with the PR code, feature works as expected