running errors using the pretrained networks

[cb7877]

Hi there,

I encountered errors when running the following lines:

---

inside the single_dataset_pathlib.py

dataset_directory = Path("/data001/data")

active_zone_mod_path = Path("/data001/data")
vesicle_mod_path = Path("/data001/data")

pl2 = cryovesnet.Pipeline(dataset_directory,pattern='*.mrc')
pl2.setup_cryovesnet_dir(initialize=False, make_masks=True)

pl2.run_deep(force_run=False,augmentation_level=4,rescale=None, weight_path=None)
pl2.rescale(force_run=False,slice_range=[10,130])

---

WARNING:tensorflow:From /home/software/anaconda3/envs/CryoVesNet/lib/python3.9/site-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
CryoVesNet Pipeline: the pipeline is created for /data001/data
*.mrc
Pixel size: 1.2119999885559083
CryoVesNet Pipeline: setting up cryovesnet directory
/data001/data/cryovesnet
Painting section 140
Painting section 140
Rescale Factor: 0.550909085707231
CryoVesNet pipeline: Running unet segmentation if there are less than 4 file in ./deep directory
(?, 32, 32, 32, 1)
5
ROI: 24
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [03:14<00:00, 48.54s/it]
(78, 564, 564)
(78, 564, 564)
ROI: 24
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [03:12<00:00, 48.06s/it]
(78, 564, 564)
(78, 564, 564)
ROI: 24
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [03:05<00:00, 46.45s/it]
(78, 564, 564)
(78, 564, 564)
ROI: 24
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [03:05<00:00, 46.38s/it]
(78, 564, 564)
(78, 564, 564)
CryoVesNet Pipeline: zooming the unet mask
(141, 1024, 1024)
last output array name: deep_mask
last mrc file saved : cryovesnet/PFC_935_8-148_zoomed_mask.mrc
CryoVesNet Pipeline: running label_vesicles
CryoVesNet Pipeline: restricting labels to segmentation region
last output array name: deep_mask
last mrc file saved : cryovesnet/PFC_935_8-148_zoomed_mask.mrc
Threshold: 0.002
last output array name: deep_labels
last mrc file saved : cryovesnet/PFC_935_8-148_deep_labels.mrc
CryoVesNet Pipeline: running label_vesicles_adaptive
Tabel computed!
Collision vesicles:
Empty DataFrame
Columns: [label, area, centroid-0, centroid-1, centroid-2, bbox-0, bbox-1, bbox-2, bbox-3, bbox-4, bbox-5, extent, area_zscore, extent_zscore]
Index: []
Tabel computed!
Vesicles to remove:
Empty DataFrame
Columns: [area, centroid-0, centroid-1, centroid-2, bbox-0, bbox-1, bbox-2, bbox-3, bbox-4, bbox-5, extent, area_zscore, extent_zscore]
Index: []
Tabel computed!
last output array name: clean_deep_labels
last mrc file saved : cryovesnet/PFC_935_8-148_clean_deep_labels.mrc
CryoVesNet Pipeline: Making vesicles spherical.
fitting sphere to vesicles: 0%| | 0/1 [00:00<?, ?it/s]get_shift_between_images failed, shift set to 0,0,0
fitting sphere to vesicles: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 302.16it/s]
0
Traceback (most recent call last):
File "/data001/scripts/single_dataset_pathlib.py", line 45, in
df = pl2.make_spheres(tight=True, keep_ellipsoid=False)
File "/home/software/CryoVesNet/cryovesnet/pipeline.py", line 621, in make_spheres
_ , ellipsoid_tags = self.compute_sphere_dataframe(input_array_name, tight= tight,keep_ellipsoid=keep_ellipsoid)
File "/home/software/CryoVesNet/cryovesnet/pipeline.py", line 597, in compute_sphere_dataframe
sphere_df['p'] = 1 - chi2.cdf(sphere_df['mahalanobis'], 2)
File "/home/software/anaconda3/envs/CryoVesNet/lib/python3.9/site-packages/scipy/stats/_distn_infrastructure.py", line 2077, in cdf
place(output, (1-cond0)+np.isnan(x), self.badvalue)
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

---

end of error message.

I tried to change multiple variables, such as with or without mod files, or specifying the weight path, or using different augmentation 1 or 4, etc. None of them generates reasonable segmentations of the vesicles.

Did I miss something when following your protocols?

Thanks in advance
Bob

[amineuron]

Hi Bob,

I noticed the reported threshold of 0.002 is unusually low since the algorithm typically searches for threshold between 0.8 and 1. Could you please confirm if you've manually set the threshold_coef argument? (If you haven't manually set the threshold_coef, this might indicate a bug)

Also recommend inspecting visually look into the output of the network. You can do this by opening the probability maps using napari. The file path you'd use is something like deep/PFC_935_8-148__wreal_mask.tiff. This can help to assess whether the network is producing sensible outputs or subsequent steps. Either drag and drop your file into napari, or load it via the command line: (while environment is activated)
napari deep/PFC_935_8-148__wreal_mask.tiff

Let me know how it goes or if there’s anything else I can help with.

Best,
Amin

[cb7877]

Hi,

Thanks for your quick response.

I think the results of the wreal_mask.tiff does not look right either.

I tried different levels of thresholds. such as default 0.8 to 1, and 0.002. The "cytomask.mrc" have a binary mask containing the segmented synaptic regions of the interest. However, when choosing a augmentation level 4, the output zoomed_mask.mrc has about the identical region with the cytomask but with a value of about 0.004 (the tomo size does not change either). And the deep_labels.mrc has nothing inside regardless of the threshold from 0.002 to 1.

I also tried to figure out the threshold by myself by checking out the outputs of the find_threshold function. Regardless of the thresholding values from 0 to 1, the output image_shells have about the identical shape. I have attached it here as well.

Thanks

Bob