Crash with granularity='frame'

[dgoodwin208]

Hi,

First - this is a great package, thank you for the contribution and the maintenance :)

Second - I'm getting some pretty significant motion correction artifacts when I use the default granularity of row in the hidden markov 2d.

#Specify the granularity: row, column or frame
correction_approach = sima.motion.HiddenMarkov2D(num_states_retained=50,
                                                      max_displacement=[20, 30],
                                                         verbose=True,
                                                         granularity=('frame',1),
                                                         n_processes=2)

Result:
Traceback (most recent call last): File "runsima.py", line 52, in <module> sequences, dataset_path, channel_names=['Arcon'], File "/Users/Goody/Neuro/Arcon/venv/lib/python2.7/site-packages/sima/motion/motion.py", line 130, in correct frame_shape) File "/Users/Goody/Neuro/Arcon/venv/lib/python2.7/site-packages/sima/motion/motion.py", line 201, in _trim_coords for d in it.chain.from_iterable(displacements)) File "/Users/Goody/Neuro/Arcon/venv/lib/python2.7/site-packages/sima/motion/motion.py", line 201, in <genexpr> for d in it.chain.from_iterable(displacements)) File "/Users/Goody/Neuro/Arcon/venv/lib/python2.7/site-packages/sima/motion/motion.py", line 232, in _observation_counts z, y, x = displacements ValueError: need more than 2 values to unpack

if I change to granularity=('row',1) it works fine. Any ideas? (using 1.3.0 from pip)

Thanks!

[pkaifosh]

Thanks for the bug report @dgoodwin208 .

The problem is that the relevant line in the _observation_counts function is assuming that the displacements are 3D (z, y, x), whereas you only have a 2D displacement (y, x). You will see just below in the code for that function that for another case, it handles this possibility with the lines:

if len(d) == 2:
    d = [0] + d`

Perhaps something like that would work here. You may have to check the dimensionality of the untrimmed_shape argument passed to this function.

Based on these pointers, are you able to try to fix this and submit a pull request?

[dgoodwin208]

Hey @pkaifosh,

I've been trying to contribute but I'm having trouble getting running with the master. I'm getting stuck as the setup.py can't see a /sima/motion/_motion.c file. What do you think I should do?

Thanks! I attached output below

(venv)~/sima-master 🍔   python setup.py install
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'install_requires'
  warnings.warn(msg)
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'setup_requires'
  warnings.warn(msg)
running install
running build
running build_py
creating build
creating build/lib.macosx-10.10-intel-2.7
creating build/lib.macosx-10.10-intel-2.7/sima
copying sima/__init__.py -> build/lib.macosx-10.10-intel-2.7/sima
copying sima/extract.py -> build/lib.macosx-10.10-intel-2.7/sima
copying sima/imaging.py -> build/lib.macosx-10.10-intel-2.7/sima
copying sima/imaging_parameters.py -> build/lib.macosx-10.10-intel-2.7/sima
copying sima/ROI.py -> build/lib.macosx-10.10-intel-2.7/sima
copying sima/sequence.py -> build/lib.macosx-10.10-intel-2.7/sima
copying sima/spikes.py -> build/lib.macosx-10.10-intel-2.7/sima
creating build/lib.macosx-10.10-intel-2.7/sima/misc
copying sima/misc/__init__.py -> build/lib.macosx-10.10-intel-2.7/sima/misc
copying sima/misc/align.py -> build/lib.macosx-10.10-intel-2.7/sima/misc
copying sima/misc/convert.py -> build/lib.macosx-10.10-intel-2.7/sima/misc
copying sima/misc/imagej.py -> build/lib.macosx-10.10-intel-2.7/sima/misc
copying sima/misc/tifffile.py -> build/lib.macosx-10.10-intel-2.7/sima/misc
creating build/lib.macosx-10.10-intel-2.7/sima/motion
copying sima/motion/__init__.py -> build/lib.macosx-10.10-intel-2.7/sima/motion
copying sima/motion/dftreg.py -> build/lib.macosx-10.10-intel-2.7/sima/motion
copying sima/motion/frame_align.py -> build/lib.macosx-10.10-intel-2.7/sima/motion
copying sima/motion/hmm.py -> build/lib.macosx-10.10-intel-2.7/sima/motion
copying sima/motion/motion.py -> build/lib.macosx-10.10-intel-2.7/sima/motion
copying sima/motion/transform.py -> build/lib.macosx-10.10-intel-2.7/sima/motion
package init file 'sima/motion/tests/__init__.py' not found (or not a regular file)
creating build/lib.macosx-10.10-intel-2.7/sima/motion/tests
copying sima/motion/tests/test_dftreg.py -> build/lib.macosx-10.10-intel-2.7/sima/motion/tests
copying sima/motion/tests/test_frame_align.py -> build/lib.macosx-10.10-intel-2.7/sima/motion/tests
copying sima/motion/tests/test_hmm.py -> build/lib.macosx-10.10-intel-2.7/sima/motion/tests
copying sima/motion/tests/test_motion.py -> build/lib.macosx-10.10-intel-2.7/sima/motion/tests
creating build/lib.macosx-10.10-intel-2.7/sima/segment
copying sima/segment/__init__.py -> build/lib.macosx-10.10-intel-2.7/sima/segment
copying sima/segment/ca1pc.py -> build/lib.macosx-10.10-intel-2.7/sima/segment
copying sima/segment/normcut.py -> build/lib.macosx-10.10-intel-2.7/sima/segment
copying sima/segment/oPCA.py -> build/lib.macosx-10.10-intel-2.7/sima/segment
copying sima/segment/segment.py -> build/lib.macosx-10.10-intel-2.7/sima/segment
copying sima/segment/stica.py -> build/lib.macosx-10.10-intel-2.7/sima/segment
package init file 'sima/segment/tests/__init__.py' not found (or not a regular file)
creating build/lib.macosx-10.10-intel-2.7/sima/segment/tests
copying sima/segment/tests/test_normcut.py -> build/lib.macosx-10.10-intel-2.7/sima/segment/tests
copying sima/segment/tests/test_oPCA.py -> build/lib.macosx-10.10-intel-2.7/sima/segment/tests
copying sima/segment/tests/test_segment.py -> build/lib.macosx-10.10-intel-2.7/sima/segment/tests
creating build/lib.macosx-10.10-intel-2.7/sima/tests
copying sima/tests/test_extract.py -> build/lib.macosx-10.10-intel-2.7/sima/tests
copying sima/tests/test_imaging.py -> build/lib.macosx-10.10-intel-2.7/sima/tests
copying sima/tests/test_ROI.py -> build/lib.macosx-10.10-intel-2.7/sima/tests
copying sima/tests/test_sequence.py -> build/lib.macosx-10.10-intel-2.7/sima/tests
copying sima/tests/test_spikes.py -> build/lib.macosx-10.10-intel-2.7/sima/tests
creating build/lib.macosx-10.10-intel-2.7/sima/tests/data
creating build/lib.macosx-10.10-intel-2.7/sima/tests/data/example.sima
copying sima/tests/data/example.sima/dataset.pkl -> build/lib.macosx-10.10-intel-2.7/sima/tests/data/example.sima
copying sima/tests/data/example.sima/displacements.pkl -> build/lib.macosx-10.10-intel-2.7/sima/tests/data/example.sima
copying sima/tests/data/example.tif -> build/lib.macosx-10.10-intel-2.7/sima/tests/data
copying sima/tests/data/example.h5 -> build/lib.macosx-10.10-intel-2.7/sima/tests/data
copying sima/tests/data/example-volume.h5 -> build/lib.macosx-10.10-intel-2.7/sima/tests/data
copying sima/tests/data/imageJ_ROIs.zip -> build/lib.macosx-10.10-intel-2.7/sima/tests/data
creating build/lib.macosx-10.10-intel-2.7/sima/tests/data/example-tiffs
copying sima/tests/data/example-tiffs/0.tif -> build/lib.macosx-10.10-intel-2.7/sima/tests/data/example-tiffs
copying sima/tests/data/example-tiffs/1.tif -> build/lib.macosx-10.10-intel-2.7/sima/tests/data/example-tiffs
copying sima/tests/data/example-tiffs/2.tif -> build/lib.macosx-10.10-intel-2.7/sima/tests/data/example-tiffs
package init file 'sima/motion/tests/__init__.py' not found (or not a regular file)
package init file 'sima/segment/tests/__init__.py' not found (or not a regular file)
running build_ext
building 'sima.motion._motion' extension
creating build/temp.macosx-10.10-intel-2.7
creating build/temp.macosx-10.10-intel-2.7/sima
creating build/temp.macosx-10.10-intel-2.7/sima/motion
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/Users/Goody/Neuro/Arcon/venv/lib/python2.7/site-packages/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c sima/motion/_motion.c -o build/temp.macosx-10.10-intel-2.7/sima/motion/_motion.o
clang: error: no such file or directory: 'sima/motion/_motion.c'
clang: error: no input files
error: command 'cc' failed with exit status 1

[pkaifosh]

If you are compiling from source, then the .c file needs to be made with cython from the .pyx file. If you have Cython installed, then I believe this should happen automatically.