Multi agent gym environment support

[jaskarannagi19]

How do you guys plan to support multi agent environment. I came though a blog post for multiple model Mario Kart and I want to run it very bad. Can someone help me in make this work for multi agent environment support?

Thanks in advance.
Currently it doesnt support multi agent environment

[jaskarannagi19]

@bzier Any support would be helpful. I updated input bot plugin to start 2 servers for input controller but where the controllers are being initialized so that I can create a second player controller to make selection and move forward.

[bzier]

Hi @jaskarannagi19. Sorry I never responded to your message. I'm in the middle of transitioning between jobs and moving at the moment and haven't had the time to spend on this recently. I read that same blog post (I assume it was the same one) and reached out to the person who wrote it to see if they would share their code, but never heard back.

I haven't fully thought out how best to implement support for multiple agents. Getting the controller inputs working is an important first step, but how to coordinate multiple agents within the gym framework isn't clear to me. If you have any ideas/suggestions, I'm open to them.

Within the environment, the ControllerHTTPServer is defined here. That class is initialized with a port number here. If you've updated the input bot plugin to connect to multiple servers on different ports, this is where you'd start them.

Hope that helps a little. When things settle down for me, I'll try to get back to this again.

[jaskarannagi19]

Hello @bzier I already achieved what you said. I edited the controller.c file to include 2 ports and in config.yml where port numbers are defined. And changed the controllerHTTPServer to start two different servers at 2 different port and return 4 results instead of two. Now I believe, I am at the state where I have successfully added 2 controllers to the game but when I change navigation menu function and select two players. Here the issue is the second player needs to be selected from second controller. There is some class called controllerState which is controlling the inputs over there. I some how needs to initialize second controller here and use it to move ahead. I also tried to change MarioKart basic file where controllers are initialized but I am struck at point where I have to consume the second controller. Some where we need to add something like ControllerState1 and use ControllerState1 to select the navigation panel. Hope this explains my problem

[jaskarannagi19]

As you can see I can select 1st player but I need to consume second controller to select that player to move ahead.

[bzier]

I came across this multi-agent RL library, PettingZoo, which has a comparable API to gym. Noting it here for future reference. May be a decent approach to adding multi-agent support here. Maybe worth maintaining a separate branch, or separate fork. Need to consider how to refactor the codebase in such a way that both could easily be supported with minimal redundancy.