ValueError: need at least one array to concatenate

[True-deng]

Nice works!!
When I was running 'start simulation', executing this line of code: python main.py -y config/waymo-1137.yaml -p 'add a straight driving car in the scene' -s demo
I encountered this error."

I found that the issue was because the values for 'centerline' and 'boundary' read from the map.pkl file are empty.

Additionally, when I am doing Data Preparation,the instruction is: "python process_waymo_script.py --waymo_data_dir=../data/waymo_tfrecords/1.4.2 --nerf_data_dir=../data/waymo_multi_view" .
I encounter this display,

and I'm unsure if it's related to the issue mentioned above.

Could you please advise me on how to resolve this problem?
Thanks a lot!

[yifanlu0227]

That's abnormal. What's your tensorflow and waymo-open-dataset version? Some previous version can not read the map data

tensorflow==2.11.0
waymo-open-dataset-tf-2-11-0==1.6.1

[True-deng]

The version in the current environment is as follows:

[yifanlu0227]

This is where map.pkl generate. Maybe you can put a breakpoint here and debug.

And when running python process_waymo_script.py --waymo_data_dir=../data/waymo_tfrecords/1.4.2 --nerf_data_dir=../data/waymo_multi_view, my output is

2024-05-15 16:23:16.783479: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-15 16:23:21.441163: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib64:/usr/local/lib:/usr/local/cuda/lib64:/usr/local/cuda/cudnn/lib:/usr/local/cuda-11.8/lib:/home/ubuntu/.mujoco/mujoco210/bin:/usr/lib/nvidia
2024-05-15 16:23:21.441365: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib64:/usr/local/lib:/usr/local/cuda/lib64:/usr/local/cuda/cudnn/lib:/usr/local/cuda-11.8/lib:/home/ubuntu/.mujoco/mujoco210/bin:/usr/lib/nvidia
2024-05-15 16:23:21.441402: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Processing file  ['../data/waymo_tfrecords/1.4.2/segment-14333744981238305769_5658_260_5678_260_with_camera_labels.tfrecord']
2024-05-15 16:23:37.807362: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-15 16:23:38.742267: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 22276 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-05-15 16:23:38.743587: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 22276 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:25:00.0, compute capability: 8.9
2024-05-15 16:23:38.744646: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:2 with 11962 MB memory:  -> device: 2, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:41:00.0, compute capability: 8.9
2024-05-15 16:23:38.745900: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:3 with 12179 MB memory:  -> device: 3, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:61:00.0, compute capability: 8.9
2024-05-15 16:23:38.746963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:4 with 18661 MB memory:  -> device: 4, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:81:00.0, compute capability: 8.9
2024-05-15 16:23:38.747956: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:5 with 22276 MB memory:  -> device: 5, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:a1:00.0, compute capability: 8.9
2024-05-15 16:23:38.748982: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:6 with 22276 MB memory:  -> device: 6, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:c1:00.0, compute capability: 8.9
2024-05-15 16:23:38.750247: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:7 with 22276 MB memory:  -> device: 7, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:e1:00.0, compute capability: 8.9
0it [00:00, ?it/s]2024-05-15 16:23:41.246392: I tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:630] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
2024-05-15 16:23:41.490738: I tensorflow/core/util/cuda_solvers.cc:179] Creating GpuSolver handles for stream 0xe107f90
45it [01:48,  2.44s/it]

I think it's more of an environmental issue.

Do you have any idea? @vfishc

[True-deng]

When running python process_waymo_script.py --waymo_data_dir=../data/waymo_tfrecords/1.4.2 --nerf_data_dir=../data/waymo_multi_view , my output is:

2024-05-15 08:46:57.258018: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-15 08:46:57.850541: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-11.7/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:
2024-05-15 08:46:57.850609: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-11.7/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:
2024-05-15 08:46:57.850620: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Processing file  ['../data/waymo_tfrecords/1.4.2/segment-14869732972903148657_2420_000_2440_000_with_camera_labels.tfrecord']
2024-05-15 08:47:00.417327: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/miniconda3/envs/chatsim/lib/python3.8/site-packages/cv2/../../lib64:/usr/local/cuda-11.7/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:
2024-05-15 08:47:00.417377: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2024-05-15 08:47:00.417761: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
184it [06:55,  2.26s/it]

It does seem to be an environment issue. I attempted to resolve it earlier, but without success.
Do you have any methods to address it?

[vfishc]

When running python process_waymo_script.py --waymo_data_dir=../data/waymo_tfrecords/1.4.2 --nerf_data_dir=../data/waymo_multi_view , my output is:

2024-05-15 08:46:57.258018: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-15 08:46:57.850541: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-11.7/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:
2024-05-15 08:46:57.850609: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-11.7/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:
2024-05-15 08:46:57.850620: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Processing file  ['../data/waymo_tfrecords/1.4.2/segment-14869732972903148657_2420_000_2440_000_with_camera_labels.tfrecord']
2024-05-15 08:47:00.417327: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/miniconda3/envs/chatsim/lib/python3.8/site-packages/cv2/../../lib64:/usr/local/cuda-11.7/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:
2024-05-15 08:47:00.417377: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2024-05-15 08:47:00.417761: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
184it [06:55,  2.26s/it]

It does seem to be an environment issue. I attempted to resolve it earlier, but without success. Do you have any methods to address it?

This may caused by your environmental versions? Make sure each process can be run without errors to check if the incorrect lib versions do not support the functions in data preparation. Moreover, the waymo data .tfrecord files must keep 1.4.2 version, any earlier or later version would not contain the map data. So in my view, first check your lib versions to keep the libs the same version as README, and also check the version of .tfrecord files because incorrect versions do not contain map.

[True-deng]

When running python process_waymo_script.py --waymo_data_dir=../data/waymo_tfrecords/1.4.2 --nerf_data_dir=../data/waymo_multi_view , my output is:

2024-05-15 08:46:57.258018: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-15 08:46:57.850541: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-11.7/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:
2024-05-15 08:46:57.850609: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-11.7/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:
2024-05-15 08:46:57.850620: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Processing file  ['../data/waymo_tfrecords/1.4.2/segment-14869732972903148657_2420_000_2440_000_with_camera_labels.tfrecord']
2024-05-15 08:47:00.417327: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /root/miniconda3/envs/chatsim/lib/python3.8/site-packages/cv2/../../lib64:/usr/local/cuda-11.7/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:
2024-05-15 08:47:00.417377: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2024-05-15 08:47:00.417761: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
184it [06:55,  2.26s/it]

It does seem to be an environment issue. I attempted to resolve it earlier, but without success. Do you have any methods to address it?

This may caused by your environmental versions? Make sure each process can be run without errors to check if the incorrect lib versions do not support the functions in data preparation. Moreover, the waymo data .tfrecord files must keep 1.4.2 version, any earlier or later version would not contain the map data. So in my view, first check your lib versions to keep the libs the same version as README, and also check the version of .tfrecord files because incorrect versions do not contain map.

I found where the issue lies. It's a problem with the version of the waymo dataset I downloaded.
Thank you very much.