The view adjust agents broken.

[LiuXuan66]

command:python3 main.py -y config/roan-10-not-wide.yaml -p 'move the viewpoint to 10m ahead' -s demo
output:
[User prompt] move the viewpoint to 10m ahead

[Project Manager] decomposing tasks
[Raw Response>>>] The broken down actions for this requirement are:

{1: 'Move the viewpoint to 10m ahead'}

This only requires a single action without any additional information about the scene or any vehicles.
[Extracted Response>>>] {1: 'Move the viewpoint to 10m ahead'}

[Performing Single Prompt] Move the viewpoint to 10m ahead

[Project Manager] dispatching each task
[Raw Response>>>] {'operation': 3}
[Extracted Response>>>] 3. (adjusting the viewpoint)

[View Adjust Agent LLM] reasoning the view motion
[Raw Response>>>] This description is related to view motion. Therefore, the dictionary would be:

{'if_view_motion': 1}
[Extracted Response>>>] {'if_view_motion': 1}

[View Adjust Agent LLM] generating the ego motion
[Raw Response>>>] I'm sorry, as an AI language model, I don't have the capability to actually move your viewpoint physically. However, I can provide you with assistance and information related to computer vision and image processing. Is there anything specific that you would like help with?
substring not found
Traceback (most recent call last):
File "/home/ChatSim/chatsim/agents/view_adjust_agent.py", line 115, in llm_view_motion_gen
start = answer.index("{")
ValueError: substring not found
Traceback (most recent call last):
File "/home/ChatSim/main.py", line 135, in
chatsim.execute_llms(args.prompt)
File "/home/ChatSim/main.py", line 99, in execute_llms
self.project_manager.dispatch_task(self.scene, task, self.tech_agents)
File "/home/ChatSim/chatsim/agents/project_manager.py", line 173, in dispatch_task
self.view_adjust_operation(scene, task, tech_agents)
File "/home/ChatSim/chatsim/agents/project_manager.py", line 278, in view_adjust_operation
start_frame_in_nerf, end_frame_in_nerf = view_adjust_agent.llm_view_motion_gen(scene, task)
ValueError: too many values to unpack (expected 2)

[LiuXuan66]

The chatgpt cannot reply right json, neither 3.5 nor 4.

[yifanlu0227]

It's possible that OpenAI has changed their model so that some of the responses from GPT have changed from previous behavior. You can try to modify the prompt, for example by underlining that a json dict should be returned in your prompt.

[yifanlu0227]

You can also set a low temperature in the API call, which will make the response robust and lessen unexpected behaviors once you have a nice prompt.

Also, check hyperparameters like temperature, top_k and frequency_penalty in
Instruction for OpenAI API.