Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filename metadata to multidimdataloader #441

Merged
merged 3 commits into from
Oct 21, 2024
Merged

Add filename metadata to multidimdataloader #441

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4e6a952
add filename_or_obj metatata to multidim dataloader to create io maps
Oct 18, 2024
7ff2644
run pre-commit
Oct 18, 2024
57f9b32
add timepoint to filename
Oct 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions configs/data/local/segmentation_all_cells_mask_from_zarr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ batch_size: 1
pin_memory: True
#persistent_workers: False


csv_path:
img_path_column: movie_path
channel_column: bf_channel
Expand All @@ -14,5 +13,3 @@ out_key: ${source_col}
transforms:
- _target_: monai.transforms.ToTensor
- _target_: monai.transforms.NormalizeIntensity


8 changes: 4 additions & 4 deletions configs/experiment/local/eval_scale1_new.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,17 +28,17 @@ trainer:

model:
compile: True
#save_dir: /allen/aics/assay-dev/users/Suraj/EMT_Work/image_analysis_test/EMT_image_analysis/Colony_mask_training_inference/eval_whole_movie_multiscale_patch1_zarr_aws
save_dir: /allen/aics/assay-dev/users/Suraj/EMT_Work/image_analysis_test/EMT_image_analysis/Colony_mask_training_inference/eval_whole_movie_multiscale_patch1_zarr_aws
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If any configs use an /allen path, then the instructions need to tell people to edit it. Instead, I recommend using relative paths (starting with ./) for portability. I've filed an issue to track this going forward.


data:
csv_path: /allen/aics/assay-dev/users/Suraj/EMT_Work/image_analysis_test/EMT_image_analysis/Colony_mask_training_inference/sample_csv/predict_all_cells_mask_zarr_aws_v0.csv #path to csv containing test movies
#batch_size: 1
_aux:
_aux:
patch_shape: [16, 128, 128]

callbacks:
predict_saving:
_target_: cyto_dl.callbacks.ImageSaver
save_dir: /allen/aics/assay-dev/users/Suraj/EMT_Work/image_analysis_test/EMT_image_analysis/Colony_mask_training_inference/eval_whole_movie_multiscale_patch1_zarr_aws
save_dir: ${model.save_dir}
stages: ["predict"]
save_input: False
save_input: False
2 changes: 1 addition & 1 deletion configs/model/local/segmentation_all_cells_mask.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ _aux:
- - ${target_col}
- _target_: cyto_dl.nn.BaseHead
loss:
_target_: monai.losses.GeneralizedDiceFocalLoss ##Main loss
_target_: monai.losses.GeneralizedDiceFocalLoss ##Main loss
sigmoid: True
postprocess:
input:
Expand Down
4 changes: 3 additions & 1 deletion cyto_dl/callbacks/image_saver.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ def __init__(
Parameters
----------
save_dir: Union[str, Path]
Directory to save images
Directory to save images. Only testing saves in this directory, prediction
saves in model save directory
save_every_n_epochs:int=1
Frequency to save images
stages:List[str]=["train", "val"]
Expand Down Expand Up @@ -49,6 +50,7 @@ def on_predict_batch_end(
if outputs is None:
# image has already been saved
return

for i, head_io_map in enumerate(io_map.values()):
for k, save_path in head_io_map.items():
self._save(save_path, outputs[k]["pred"][i])
Expand Down
6 changes: 6 additions & 0 deletions cyto_dl/datamodules/multidim_image.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,13 +98,18 @@ def _get_timepoints(self, row, img):
timepoints = range(start, stop + 1, step) if stop > 0 else range(img.dims.T)
return list(timepoints)

def _get_filename(self, image_input_path):
return image_input_path.split("/")[-1].split(".")[0]

def get_per_file_args(self, df):
img_data = []
for row in df.itertuples():
row = row._asdict()
img = BioImage(row[self.img_path_column])
scenes = self._get_scenes(row, img)
timepoints = self._get_timepoints(row, img)
filename = self._get_filename(row[self.img_path_column])

for scene in scenes:
for timepoint in timepoints:
img_data.append(
Expand All @@ -115,6 +120,7 @@ def get_per_file_args(self, df):
"scene": scene,
"T": timepoint,
"original_path": row[self.img_path_column],
"filename_or_obj": filename + f"_{timepoint}", # needs to be part of metadata to generate IO maps
}
)
img_data.reverse()
Expand Down