render or lift question when running samesh

[longshot-pting]

when i run samesh on obj object, i mee
**
Rendering Multiviews...: 100%|██████████| 12/12 [00:02<00:00, 5.64it/s]
Computing SAM Masks for norms: 0it [00:00, ?it/s]/home/easter/pting/project/omini6dpose/code/samesh-main/third_party/segmentanymesh/sam2/modeling/sam/transformer.py:270: UserWarning: Memory efficient kernel not used because: (Triggered internally at ../aten/src/ATen/native/transformers/cuda/sdp_utils.cpp:607.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/home/easter/pting/project/omini6dpose/code/samesh-main/third_party/segmentanymesh/sam2/modeling/sam/transformer.py:270: UserWarning: Memory Efficient attention has been runtime disabled. (Triggered internally at ../aten/src/ATen/native/transformers/sdp_utils_cpp.h:495.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/home/easter/pting/project/omini6dpose/code/samesh-main/third_party/segmentanymesh/sam2/modeling/sam/transformer.py:270: UserWarning: Flash attention kernel not used because: (Triggered internally at ../aten/src/ATen/native/transformers/cuda/sdp_utils.cpp:609.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/home/easter/pting/project/omini6dpose/code/samesh-main/third_party/segmentanymesh/sam2/modeling/sam/transformer.py:270: UserWarning: Expected query, key and value to all be of dtype: {Half, BFloat16}. Got Query dtype: float, Key dtype: float, and Value dtype: float instead. (Triggered internally at ../aten/src/ATen/native/transformers/sdp_utils_cpp.h:98.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/home/easter/pting/project/omini6dpose/code/samesh-main/third_party/segmentanymesh/sam2/modeling/sam/transformer.py:270: UserWarning: CuDNN attention kernel not used because: (Triggered internally at ../aten/src/ATen/native/transformers/cuda/sdp_utils.cpp:611.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/home/easter/pting/project/omini6dpose/code/samesh-main/third_party/segmentanymesh/sam2/modeling/sam/transformer.py:270: UserWarning: The CuDNN backend needs to be enabled by setting the enviornment variableTORCH_CUDNN_SDPA_ENABLED=1 (Triggered internally at ../aten/src/ATen/native/transformers/cuda/sdp_utils.cpp:409.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/home/easter/miniconda3/envs/SAmesh/lib/python3.11/site-packages/torch/nn/modules/module.py:1541: UserWarning: Flash Attention kernel failed due to: No available kernel. Aborting execution.
Falling back to all available kernels for scaled_dot_product_attention (which may have a slower speed).
return forward_call(*args, **kwargs)
Computing SAM Masks for norms: 12it [00:15, 1.29s/it]
Rendering Multiviews...: 100%|██████████| 12/12 [00:01<00:00, 11.76it/s]
Computing SAM Masks for sdf: 12it [00:14, 1.25s/it]
Saving items to cache...
Visualizing images: 12it [00:00, 66.78it/s]
Visualizing SDF: 12it [00:00, 62.46it/s]
Visualizing FaceIDs: 12it [00:00, 41.62it/s]
Visualizing SAM Masks: 12it [00:00, 38.90it/s]
Visualizing Normals: 12it [00:00, 43.54it/s]
Visualizing Normals Mask: 12it [00:00, 45.11it/s]
lift begin
norms: <class 'list'>, length: 12
depth: <class 'list'>, length: 12
matte: <class 'list'>, length: 12
faces: <class 'list'>, length: 12
poses: <class 'list'>, length: 12
norms_masked: <class 'list'>, length: 12
sdf: <class 'list'>, length: 12
bmasks: <class 'list'>, length: 12
cmasks: <class 'list'>, length: 12

IndexError Traceback (most recent call last)
Cell In[2], line 3
1 config = OmegaConf.load('../configs/mesh_segmentation.yaml')
2 # mesh = segment_mesh('/home/easter/pting/project/omini6dpose/code/Omni6DPoseAPI/Omni6DPoseAPI-main/data/Omni6DPose/PAM/object_meshes/google_scan-box_0002/Aligned.obj', config, visualize=True)
----> 3 mesh = segment_mesh('/home/easter/pting/project/omini6dpose/code/Omni6DPoseAPI/Omni6DPoseAPI-main/assets/assets_omni6DPose/objects/omni6DPose/fork/omni6DPose_fork_000/Aligned.obj', config, visualize=True)

File ~/pting/project/omini6dpose/code/samesh-main/src/samesh/models/sam_mesh.py:642, in segment_mesh(filename, config, visualize, extension, target_labels, texture, output_folder)
640 # run sam grounded mesh and optionally visualize renders
641 visualize_path = f'{config.output}/{filename.stem}_visualized' if visualize else None
--> 642 faces2label, _ = model(tmesh, visualize_path=visualize_path, target_labels=target_labels)
644 # colormap and save mesh
645 os.makedirs(config.output, exist_ok=True)

File ~/miniconda3/envs/SAmesh/lib/python3.11/site-packages/torch/nn/modules/module.py:1532, in Module._wrapped_call_impl(self, *args, **kwargs)
1530 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1531 else:
-> 1532 return self._call_impl(*args, **kwargs)

File ~/miniconda3/envs/SAmesh/lib/python3.11/site-packages/torch/nn/modules/module.py:1541, in Module._call_impl(self, *args, **kwargs)
1536 # If we don't have any hooks, we want to skip the rest of the logic in
1537 # this function, and just call forward.
1538 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1539 or _global_backward_pre_hooks or _global_backward_hooks
1540 or _global_forward_hooks or _global_forward_pre_hooks):
...
-> 4831 slices_having_nans = np.isnan(arr[-1, ...])
4832 else:
4833 slices_having_nans = None

IndexError: index -1 is out of bounds for axis 0 with size 0
**
,lifting is not complete normally for some values have nothing. can you help me solve this question?

[gtangg12]

Sorry, where in the code are you getting the issues? Can you point me to the line number and provide more context?

[longshot-pting]

Thank you for replying ! . I met this question in 'lift' module, the face2label_views has empty pairs instead of the valid pairs, so the error "IndexError: index -1 is out of bounds for axis 0 with size 0" means there is no labels for every face. I check the obj mesh, I can not find any question, i check the code as well, i can not find any more. Can you give me some suggestions for solving this question?

[longshot-pting]

hi! I think I found the reason, in the function “compute_face2label”, there is a value called "threshold_counts" ,for some special obj like many faces are folded, the default value 16 is too big to filter all faces, I tried set this value smaller, I can get final segmentation

[kaitolucifer]

@longshot-pting
Hi, what value did you use? And seems like using small value makes the split function super slow.