AttributeError: 'NoneType' object has no attribute 'shape' when running simple_rl_train.py - Carla

[tornado20092008]

Description:

When running simple_rl_train.py, I encounter an error where the environment (Carla) seems to attempt a reset multiple times, causing an AttributeError due to obs_space being None. This happens after the environment appears to initialize successfully.

Error Traceback:

python simple_rl_train.py 
pygame 2.6.1 (SDL 2.28.4, Python 3.7.16)
Hello from the pygame community. https://www.pygame.org/contribute.html
[ENV] Register environments: ['SimpleCarla-v1', 'ScenarioCarla-v1'].
------ Run Carla on Port: 9000, GPU: 0 ------
[SIMULATOR] Not providing TM port, try finding free
[SIMULATOR] Using TM port: 55293
------ Run Carla on Port: 9004, GPU: 0 ------
[SIMULATOR] Not providing TM port, try finding free
[SIMULATOR] Using TM port: 37631
[11-14 09:23:12] INFO     [RANK0]: DI-engine DRL Policy                                                                                                                                                   base_learner.py:338
                          DQNRLModel(                                                                                                                                                                                        
                            (_encoder): BEVSpeedConvEncoder(                                                                                                                                                                 
                              (_relu): ReLU()                                                                                                                                                                                
                              (_model): Sequential(                                                                                                                                                                          
                                (0): Conv2d(5, 64, kernel_size=(3, 3), stride=(2, 2))                                                                                                                                        
                                (1): ReLU()                                                                                                                                                                                  
                                (2): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2))                                                                                                                                      
                                (3): ReLU()                                                                                                                                                                                  
                                (4): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2))                                                                                                                                     
                                (5): ReLU()                                                                                                                                                                                  
                                (6): Flatten(start_dim=1, end_dim=-1)                                                                                                                                                        
                              )                                                                                                                                                                                              
                              (_mid): Linear(in_features=2304, out_features=256, bias=True)                                                                                                                                  
                            )                                                                                                                                                                                                
                            (_head): DuelingHead(                                                                                                                                                                            
                              (A): Sequential(                                                                                                                                                                               
                                (0): Sequential(                                                                                                                                                                             
                                  (0): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                  
                                  (1): ReLU()                                                                                                                                                                                
                                )                                                                                                                                                                                            
                                (1): Sequential(                                                                                                                                                                             
                                  (0): Linear(in_features=512, out_features=21, bias=True)                                                                                                                                   
                                )                                                                                                                                                                                            
                              )                                                                                                                                                                                              
                              (V): Sequential(                                                                                                                                                                               
                                (0): Sequential(                                                                                                                                                                             
                                  (0): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                  
                                  (1): ReLU()                                                                                                                                                                                
                                )                                                                                                                                                                                            
                                (1): Sequential(                                                                                                                                                                             
                                  (0): Linear(in_features=512, out_features=1, bias=True)                                                                                                                                    
                                )                                                                                                                                                                                            
                              )                                                                                                                                                                                              
                            )                                                                                                                                                                                                
                          )                                                                                                                                                                                                  
pygame 2.6.1 (SDL 2.28.4, Python 3.7.16)
Traceback (most recent call last):
  File "simple_rl_train.py", line 143, in <module>
    main(args)
  File "simple_rl_train.py", line 104, in main
    cfg.policy.eval.evaluator, evaluate_env, policy.eval_mode, exp_name=cfg.exp_name
  File "/home/a_mohame/DI-drive/core/eval/serial_evaluator.py", line 59, in __init__
    super().__init__(cfg, env, policy, tb_logger=tb_logger, exp_name=exp_name, instance_name=instance_name)
  File "/home/a_mohame/DI-drive/core/eval/base_evaluator.py", line 49, in __init__
    self.env = env
  File "/home/a_mohame/DI-drive/core/eval/serial_evaluator.py", line 76, in env
    self._env_manager.launch()
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/site-packages/ding/envs/env_manager/subprocess_env_manager.py", line 350, in launch
    self._create_state()
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/site-packages/ding/envs/env_manager/subprocess_env_manager.py", line 251, in _create_state
    shape = obs_space.shape
AttributeError: 'NoneType' object has no attribute 'shape'

Setup Details:

DI-engine: 0.4
DI-drive: 0.3.4
Gym: 0.20.0
Carla: 0.9.10
Python: 3.7.16

Steps to Reproduce:

Run the command python simple_rl_train.py.
The environment initializes and registers, and the simulation begins is reset.
After resetting, the environment unexpectedly attempts another reset.
The program throws an AttributeError in subprocess_env_manager.py due to obs_space being None.

[tornado20092008]

This was fixed by switching of shared_memory in subprocess_env_manager.py. However now I am getting this:

pygame 2.6.1 (SDL 2.28.4, Python 3.7.16)
Hello from the pygame community. https://www.pygame.org/contribute.html
[ENV] Register environments: ['SimpleCarla-v1', 'ScenarioCarla-v1'].
------ Run Carla on Port: 9000, GPU: 0 ------
[SIMULATOR] Not providing TM port, try finding free
[SIMULATOR] Using TM port: 56485
------ Run Carla on Port: 9004, GPU: 0 ------
[SIMULATOR] Not providing TM port, try finding free
[SIMULATOR] Using TM port: 42571
[11-14 11:13:40] INFO     [RANK0]: DI-engine DRL Policy                                                                                                                                                   base_learner.py:338
                          DQNRLModel(                                                                                                                                                                                        
                            (_encoder): BEVSpeedConvEncoder(                                                                                                                                                                 
                              (_relu): ReLU()                                                                                                                                                                                
                              (_model): Sequential(                                                                                                                                                                          
                                (0): Conv2d(5, 64, kernel_size=(3, 3), stride=(2, 2))                                                                                                                                        
                                (1): ReLU()                                                                                                                                                                                  
                                (2): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2))                                                                                                                                      
                                (3): ReLU()                                                                                                                                                                                  
                                (4): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2))                                                                                                                                     
                                (5): ReLU()                                                                                                                                                                                  
                                (6): Flatten(start_dim=1, end_dim=-1)                                                                                                                                                        
                              )                                                                                                                                                                                              
                              (_mid): Linear(in_features=2304, out_features=256, bias=True)                                                                                                                                  
                            )                                                                                                                                                                                                
                            (_head): DuelingHead(                                                                                                                                                                            
                              (A): Sequential(                                                                                                                                                                               
                                (0): Sequential(                                                                                                                                                                             
                                  (0): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                  
                                  (1): ReLU()                                                                                                                                                                                
                                )                                                                                                                                                                                            
                                (1): Sequential(                                                                                                                                                                             
                                  (0): Linear(in_features=512, out_features=21, bias=True)                                                                                                                                   
                                )                                                                                                                                                                                            
                              )                                                                                                                                                                                              
                              (V): Sequential(                                                                                                                                                                               
                                (0): Sequential(                                                                                                                                                                             
                                  (0): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                  
                                  (1): ReLU()                                                                                                                                                                                
                                )                                                                                                                                                                                            
                                (1): Sequential(                                                                                                                                                                             
                                  (0): Linear(in_features=512, out_features=1, bias=True)                                                                                                                                    
                                )                                                                                                                                                                                            
                              )                                                                                                                                                                                              
                            )                                                                                                                                                                                                
                          )                                                                                                                                                                                                  
pygame 2.6.1 (SDL 2.28.4, Python 3.7.16)
Hello from the pygame community. https://www.pygame.org/contribute.html
[ENV] Register environments: ['SimpleCarla-v1', 'ScenarioCarla-v1'].
------ Run Carla on Port: 9000, GPU: 0 ------
[SIMULATOR] Not providing TM port, try finding free
[SIMULATOR] Using TM port: 40265
------ Run Carla on Port: 9004, GPU: 0 ------
[SIMULATOR] Not providing TM port, try finding free
[SIMULATOR] Using TM port: 35481
**[11-14 11:13:59] ERROR    Env 0 reset has exceeded max retries(5)**                                                                                                                                                  func.py:62
Traceback (most recent call last):
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/site-packages/ding/utils/system_helper.py", line 57, in run
    self.ret = self._target(*self._args, **self._kwargs)
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/site-packages/ding/envs/env_manager/subprocess_env_manager.py", line 459, in _reset
    self.close()
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/site-packages/ding/envs/env_manager/subprocess_env_manager.py", line 764, in close
    p.send(['close', None, None])
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
    self._send(header + buf)
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "simple_rl_train_My.py", line 143, in <module>
    main(args)
  File "simple_rl_train_My.py", line 104, in main
    cfg.policy.eval.evaluator, evaluate_env, policy.eval_mode, exp_name=cfg.exp_name
  File "/home/a_mohame/DI-drive/core/eval/serial_evaluator.py", line 59, in __init__
    super().__init__(cfg, env, policy, tb_logger=tb_logger, exp_name=exp_name, instance_name=instance_name)
  File "/home/a_mohame/DI-drive/core/eval/base_evaluator.py", line 49, in __init__
    self.env = env
  File "/home/a_mohame/DI-drive/core/eval/serial_evaluator.py", line 76, in env
    self._env_manager.launch()
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/site-packages/ding/envs/env_manager/subprocess_env_manager.py", line 351, in launch
    self.reset(reset_param)
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/site-packages/ding/envs/env_manager/subprocess_env_manager.py", line 408, in reset
    t.join()
  File "/home/a_mohame/anaconda3/envs/my_env/lib/python3.7/site-packages/ding/utils/system_helper.py", line 64, in join
    raise RuntimeError('Exception in thread({})'.format(id(self))) from self.exc
RuntimeError: Exception in thread(140351789910736)

[PaParaZz1]

There are 2 possible methods to figure out this problem:

1. You should add the definition of obs_space in the environment class you used in DI-drive
2. add shared_memory=False in the config field of env, since the subprocess_env_manager uses the obs_space to create shared memory buffer before running, set it to False can skip this process.

[tornado20092008]

For method 2, I already tried making shared_memory=False in the subprocess_env_manager and the error I am sharing is after doing that.

For method 1, I see its already defined in the 'simple_carla_env.py' file:

`
def get_observations(self) -> Dict:
"""
Get observations from simulator. The sensor data, navigation, state and information in simulator
are used, while not all these are added into observation dict.

    :Returns:
        Dict: Observation dict.
    """
    obs = dict()
    state = self._simulator.get_state()
    navigation = self._simulator.get_navigation()
    sensor_data = self._simulator.get_sensor_data()
    information = self._simulator.get_information()

    self._simulator_databuffer['state'] = state
    self._simulator_databuffer['navigation'] = navigation
    self._simulator_databuffer['information'] = information
    if 'action' not in self._simulator_databuffer:
        self._simulator_databuffer['action'] = dict()
    if not navigation['agent_state'] == 4 or self._ignore_light:
        self._stuck_detector.tick(state['speed'])

    obs.update(sensor_data)
    obs.update(
        {
            'tick': information['tick'],
            'timestamp': np.float32(information['timestamp']),
            'agent_state': navigation['agent_state'],
            'node': navigation['node'],
            'node_forward': navigation['node_forward'],
            'target': np.float32(navigation['target']),
            'target_forward': np.float32(navigation['target_forward']),
            'command': navigation['command'],
            'speed': np.float32(state['speed']),
            'speed_limit': np.float32(navigation['speed_limit']),
            'location': np.float32(state['location']),
            'forward_vector': np.float32(state['forward_vector']),
            'acceleration': np.float32(state['acceleration']),
            'velocity': np.float32(state['velocity']),
            'angular_velocity': np.float32(state['angular_velocity']),
            'rotation': np.float32(state['rotation']),
            'is_junction': np.float32(state['is_junction']),
            'tl_state': state['tl_state'],
            'tl_dis': np.float32(state['tl_dis']),
            'waypoint_list': navigation['waypoint_list'],
            'direction_list': navigation['direction_list'],
        }
    )

    if self._visualizer is not None:
        if self._visualize_cfg.type not in sensor_data:
            raise ValueError("visualize type {} not in sensor data!".format(self._visualize_cfg.type))
        self._render_buffer = sensor_data[self._visualize_cfg.type].copy()
        if self._visualize_cfg.type == 'birdview':
            self._render_buffer = visualize_birdview(self._render_buffer)
    return obs

`

[PaParaZz1]

for method 1, maybe you should check why obs_space is NoneType in the error message, maybe you can initialize the obs_space variable as early as possible.