Nearly All Pack Grocery Exp failed for Collision

[BobHuangC]

Hello, I'm well interested in LLM Agent's application in robot-control case. Recently I found you repo and it helps me a lot.
I tried to replicate the experiment, all the results strongly follow data in paper RoCo: Dialectic Multi-Robot Collaboration with Large Language Models except the Pack Grocery Task.

I tried the following commands to run the pack experiment.

python run_dialog.py --task pack -llm gpt-4
python run_dialog.py --task pack --skip_display -nruns 20 -t 20
python run_dialog.py --task pack --skip_display -nruns 5 -t 20 --comm_mode plan

I suppose the above commands test both the communication mode of dialog and plan.
But in my limited experiments, they all failed to complete the task.

The fail reason is almost the same: Collision

In detailed, the first two steps plan successfully and can generate specific action to take, but in the following steps, it almost failed to find a path to do the action, the reason is almost ReasonCollisionAtStart_time0_iter0.

The output looks like

Stesp: 12 Plan success: False, reason: ReasonCollisionAtStart_time0_iter0
Plan 0 failed to execute.
...
...
Stesp: 13 Plan success: False, reason: ReasonCollisionAtStart_time0_iter0
Plan 0 failed to execute.

I suppose perhaps there's some problem with the pack env? It would be very nice of you if you can help me!
Many thanks again for your wonderful idea.