KeyError: 'extra_metadata' when running example "setup_images_to_stack.ipynb"

[byee4]

🐛 Bug

KeyError: 'extra_metadata' when running example "setup_images_to_stack.ipynb"

To Reproduce

Steps to reproduce the behavior:

1. Install punctatools
2. Run example notebook "setup_images_to_stack.ipynb"

The following code returns the number of stacks that were found
If executing this cell results in an error, make sure the channel and z-position codes are provided correctly. Note that the codes are case-sensitive.
#################################
#  Don't modify the code below  #
#################################
​
n = get_number_of_stacks(input_dir, channel_code=channel_code, z_position_code=z_position_code)
n
/opt/conda/lib/python3.9/site-packages/pandas/core/dtypes/common.py:1645: DeprecationWarning: Converting `np.integer` or `np.signedinteger` to a dtype is deprecated. The current result is `np.dtype(np.int_)` which is not strictly correct. Note that the result depends on the system. To ensure stable results use may want to use `np.int64` or `np.int32`.
  npdtype = np.dtype(dtype)
/opt/conda/lib/python3.9/site-packages/pandas/core/dtypes/common.py:1645: DeprecationWarning: Converting `np.integer` or `np.signedinteger` to a dtype is deprecated. The current result is `np.dtype(np.int_)` which is not strictly correct. Note that the result depends on the system. To ensure stable results use may want to use `np.int64` or `np.int32`.
  npdtype = np.dtype(dtype)
/opt/conda/lib/python3.9/site-packages/pandas/core/dtypes/common.py:1645: DeprecationWarning: Converting `np.integer` or `np.signedinteger` to a dtype is deprecated. The current result is `np.dtype(np.int_)` which is not strictly correct. Note that the result depends on the system. To ensure stable results use may want to use `np.int64` or `np.int32`.
  npdtype = np.dtype(dtype)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Input In [7], in <cell line: 5>()
      1 #################################
      2 #  Don't modify the code below  #
      3 #################################
----> 5 n = get_number_of_stacks(input_dir, channel_code=channel_code, z_position_code=z_position_code)
      6 n

File /opt/conda/lib/python3.9/site-packages/punctatools/lib/convert.py:130, in get_number_of_stacks(input_dir, channel_code, z_position_code)
    109 """
    110 Return the number of stacks in the dataset.
    111 
   (...)
    127 
    128 """
    129 src = __get_source(input_dir, channel_code, z_position_code)
--> 130 n_stacks = intake_io.imload(src, metadata_only=True)['npartitions']
    131 return n_stacks

File /opt/conda/lib/python3.9/site-packages/intake_io/io.py:50, in imload(uri, partition, metadata_only, **kwargs)
     48 if isinstance(uri, intake.DataSource):
     49     if metadata_only:
---> 50         return uri.discover()
     51     out = _to_xarray(uri, partition=partition)
     52     if not isinstance(out, xr.Dataset):

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:155, in DataSourceBase.discover(self)
    153 def discover(self):
    154     """Open resource and populate the source attributes."""
--> 155     self._load_metadata()
    157     return dict(
    158         dtype=self.dtype,
    159         shape=self.shape,
    160         npartitions=self.npartitions,
    161         metadata=self.metadata,
    162     )

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:84, in DataSourceBase._load_metadata(self)
     82 """load metadata only if needed"""
     83 if self._schema is None:
---> 84     self._schema = self._get_schema()
     85     self.dtype = self._schema.dtype
     86     self.shape = self._schema.shape

File /opt/conda/lib/python3.9/site-packages/intake_io/source/filepattern.py:255, in FilePatternSource._get_schema(self)
    254 def _get_schema(self) -> Schema:
--> 255     header = self._files.get_file_metadata()
    256     metadata = header["metadata"]
    257     shape = self._set_shape_metadata(self._files.axes, self._files.shape, metadata["spacing"], metadata["spacing_units"], self._files.coords)

File /opt/conda/lib/python3.9/site-packages/intake_io/source/filepattern.py:188, in FilePattern.get_file_metadata(self)
    186 file = self.files.iloc[0]["file"]
    187 if self.source is None:
--> 188     with autodetect(file) as src:
    189         return src.discover()
    190 else:

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:245, in DataSourceBase.__enter__(self)
    244 def __enter__(self):
--> 245     self._load_metadata()
    246     return self

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:88, in DataSourceBase._load_metadata(self)
     86 self.shape = self._schema.shape
     87 self.npartitions = self._schema.npartitions
---> 88 self.metadata.update(self._schema.extra_metadata)

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:18, in Schema.__getattr__(self, item)
     17 def __getattr__(self, item):
---> 18     return self[item]

KeyError: 'extra_metadata'

Expected behavior

Environment

• Operation System: Linux
• conda version (conda --version): conda 23.1.0
• pip version (pip --version): pip 22.0.4 from /opt/conda/lib/python3.9/site-packages/pip (python 3.9)
• punctatools version: 0.2.0
• Any other relevant information:
  environment.txt

Additional context

[lsxucsf]

I'm also receiving this error, albeit in the setup_roi_segmentation.ipynb. This error is also occurring when I use the example data stacks, so I imagine it's not intrinsic to my data. Any suggestions?

[Louison5]

Hey, I think downgrading the library "intake" to 0.7.0 works for me