Installation issue

[Shreyas-Bhat]

Hello,

Thanks a lot for open-sourcing this code. It is very helpful.
I am facing the following error. I tried to downgrade my numpy to several previous versions. However, this issues hasn't been resolved. Can the maintainers of synthseg kindly look into this issue and let me know what I can do to solve this?

Best,

2024-09-09 22:45:54.444118: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:92
61] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN wh
en one has already been registered                                                     
2024-09-09 22:45:54.444265: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:60
7] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT whe
n one has already been registered                                                      
2024-09-09 22:45:54.792866: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1
515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS
 when one has already been registered                                                  
2024-09-09 22:45:55.454141: I tensorflow/core/platform/cpu_feature_guard.cc:182] This Te
nsorFlow binary is optimized to use available CPU instructions in performance-critical o
perations.                                                                             
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow
with the appropriate compiler flags.                                                   
2024-09-09 22:46:03.084162: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-T
RT Warning: Could not find TensorRT                                                    
                                                                                       
SynthSeg 2.0                                                                           
                                                                                       
using 1 thread                                                                         
Traceback (most recent call last):                                                     
  File "/home/sbb36/SynthSeg/scripts/commands/SynthSeg_predict.py", line 113, in <module
>                                                                                      
    predict(path_images=args['i'],                                                     
  File "/home/sbb36/SynthSeg/SynthSeg/predict_synthseg.py", line 87, in predict        
    labels_segmentation, _ = utils.get_list_labels(label_list=labels_segmentation)     
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^     
  File "/home/sbb36/SynthSeg/ext/lab2im/utils.py", line 227, in get_list_labels        
    label_list = np.array(reformat_to_list(label_list, load_as_numpy=True, dtype='int'))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sbb36/SynthSeg/ext/lab2im/utils.py", line 335, in reformat_to_list       
    if isinstance(var, (int, float, np.int, np.int32, np.int64, np.float, np.float32, np
.float64)):                                                                            
                                    ^^^^^^                                             
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site
-packages/numpy/__init__.py", line 319, in __getattr__                                 
    raise AttributeError(__former_attrs__[attr])                                       
AttributeError: module 'numpy' has no attribute 'int'.  

[hvgazula]

File "/home/sbb36/SynthSeg/ext/lab2im/utils.py", line 335, in reformat_to_list       
    if isinstance(var, (int, float, np.int, np.int32, np.int64, np.float, np.float32, np
.float64)):

As suggested by the error, simply remove np.int and np.float from the offending line in utils.py and you should be alright.

[Shreyas-Bhat]

Thank you so much for your response @hvgazula !
There seems to be a followup error and I wonder if it has anything to do with the numpy version. Thanks

Traceback (most recent call last):                                                                                                                                                                                              
  File "/home/sbb36/SynthSeg/scripts/commands/SynthSeg_predict.py", line 28, in <module>                                                                                                                                        
    from SynthSeg.predict_synthseg import predict                                                                                                                                                                               
  File "/home/sbb36/SynthSeg/SynthSeg/__init__.py", line 1, in <module>                                                                                                                                                         
    from . import brain_generator                                                                                                                                                                                               
  File "/home/sbb36/SynthSeg/SynthSeg/brain_generator.py", line 21, in <module>                                                                                                                                                 
    from SynthSeg.model_inputs import build_model_inputs                                                                                                                                                                        
  File "/home/sbb36/SynthSeg/SynthSeg/model_inputs.py", line 22, in <module>                                                                                                                                                    
    from ext.lab2im import utils                                                                                                                                                                                                
  File "/home/sbb36/SynthSeg/ext/lab2im/__init__.py", line 1, in <module>                                                                                                                                                       
    from . import edit_tensors                                                                                                                                                                                                  
  File "/home/sbb36/SynthSeg/ext/lab2im/edit_tensors.py", line 28, in <module>                                                                                                                                                  
    import tensorflow as tf                                                                                                                                                                                                     
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/__init__.py", line 48, in <module>                                                                                 
    from tensorflow._api.v2 import __internal__                                                                                                                                                                                 
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/_api/v2/__internal__/__init__.py", line 8, in <module>                                                             
    from tensorflow._api.v2.__internal__ import autograph                                                                                                                                                                       
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/_api/v2/__internal__/autograph/__init__.py", line 8, in <module>                                                   
    from tensorflow.python.autograph.core.ag_ctx import control_status_ctx # line: 34 
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/python/autograph/core/ag_ctx.py", line 21, in <module>
    from tensorflow.python.autograph.utils import ag_logging
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/python/autograph/utils/__init__.py", line 17, in <module>
    from tensorflow.python.autograph.utils.context_managers import control_dependency_on_returns
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/python/autograph/utils/context_managers.py", line 19, in <module>
    from tensorflow.python.framework import ops
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/python/framework/ops.py", line 45, in <module>
    from tensorflow.python.eager import context
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/python/eager/context.py", line 37, in <module>
    from tensorflow.python.eager import execute
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/python/eager/execute.py", line 21, in <module>
    from tensorflow.python.framework import dtypes
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/tensorflow/python/framework/dtypes.py", line 21, in <module>
    import ml_dtypes
  File "/autofs/vast/qtim/tools/pyenv/versions/3.11.3/envs/brainmets/lib/python3.11/site-packages/ml_dtypes/__init__.py", line 32, in <module>
    from ml_dtypes._custom_floats import bfloat16
ImportError: numpy.core.umath failed to import

[neuronflow]

Not sure if it is related after running:

conda create -n synthseg_38 python=3.8 tensorflow-gpu=2.2.0 keras=2.3.1 nibabel matplotlib -c anaconda -c conda-forge

I saw:

  File "/home/florian/miniconda3/envs/synthseg_38/lib/python3.8/site-packages/tensorflow/python/framework/dtypes.py", line 513, in <module>
    np.object,
  File "/home/florian/miniconda3/envs/synthseg_38/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'object'.
`np.object` was a deprecated alias for the builtin `object`. To avoid this error in existing code, use `object` by itself. Doing this will not modify any behavior and is safe. 
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

after downgrading to numpy 1.19 it seems to work.

[BBillot]

Hi, if you look at https://github.com/BBillot/SynthSeg/blob/master/requirements_python3.8.txt it says that with python 3.8 numpy should be version 1.23.5.
@neuronflow Glad to see that you were able to make it work with numpy 1.19
@Shreyas-Bhat it looks like this is a numpy version issue, as you too are using numpy 1.20. I guess try downgrading to 1.19 or upgrading to 1.23.5.
Let me know how it goes :)

[Shreyas-Bhat]

Hi @BBillot ,

I sincerely appreciate your help out here! While I am trying your suggestions, I need to mention the tensorflow version, which isn't mentioned here: https://github.com/BBillot/SynthSeg/blob/master/requirements_python3.6.txt. Could you please let me know this information?

Thanks a lot,
Shreyas

[hvgazula]

SynthSeg/requirements_python3.6.txt

Line 42 in 2a2aa3b

tensorflow-gpu==2.0.0

[Shreyas-Bhat]

Thanks @hvgazula, but as I understand tensorflow-gpu has been outdated (https://pypi.org/project/tensorflow-gpu/). So which tensorflow version would you suggest?

[BBillot]

well if you're using python 3.6 tensorflow-gpu should still be valid. If not just use straight tensorflow

[Shreyas-Bhat]

Hi @BBillot, thanks for the reply. It might be that the problem is with me not using a pyenv, instead of a conda env, but installing tensorflow-gpu and tensorflow are both causing errors. The following error is thrown when installing TensorFlow. Are you familiar with this error? I have followed all the requirements given in the repo.

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
    
    
    cdef class CallbackWrapper:
    
        def __cinit__(self, object future, object loop, CallbackFailureHandler failure_handler):
            self.context.functor.functor_run = self.functor_run
                                                   ^
    ------------------------------------------------------------
    
    src/python/grpcio/grpc/_cython/_cygrpc/aio/callback_common.pyx.pxi:36:47: Cannot assign type 'void (grpc_completion_queue_functor *, int) except *' to 'void (*)(grpc_completion_queue_functor *, int) noexcept'. Exception values are incompatible. Suggest adding 'noexcept' to the type of 'functor_run'.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/local_mount/space/glacier/2/scratch/pip-install-qup507uh/grpcio/setup.py", line 559, in <module>
        cmdclass=COMMAND_CLASS,
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/envs/synthseg_new/lib/python3.6/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/envs/synthseg_new/lib/python3.6/site-packages/setuptools/command/install.py", line 68, in run
        return orig.install.run(self)
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/command/install.py", line 545, in run
        self.run_command('build')
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/envs/synthseg_new/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 79, in run
        _build_ext.run(self)
      File "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/lib/python3.6/distutils/command/build_ext.py", line 339, in run
        self.build_extensions()
      File "/local_mount/space/glacier/2/scratch/pip-install-qup507uh/grpcio/src/python/grpcio/commands.py", line 278, in build_extensions
        self.extensions = try_cythonize(self.extensions)
      File "/local_mount/space/glacier/2/scratch/pip-install-qup507uh/grpcio/src/python/grpcio/commands.py", line 197, in try_cythonize
        compiler_directives=cython_compiler_directives)
      File "/local_mount/space/glacier/2/scratch/pip-install-qup507uh/grpcio/.eggs/Cython-3.0.11-py3.6.egg/Cython/Build/Dependencies.py", line 1154, in cythonize
        cythonize_one(*args)
      File "/local_mount/space/glacier/2/scratch/pip-install-qup507uh/grpcio/.eggs/Cython-3.0.11-py3.6.egg/Cython/Build/Dependencies.py", line 1321, in cythonize_one
        raise CompileError(None, pyx_file)
    Cython.Compiler.Errors.CompileError: src/python/grpcio/grpc/_cython/cygrpc.pyx
    
    ----------------------------------------
Command "/autofs/vast/qtim/tools/pyenv/versions/3.6.15/envs/synthseg_new/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/local_mount/space/glacier/2/scratch/pip-install-qup507uh/grpcio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /local_mount/space/glacier/2/scratch/pip-record-cbjxvlei/install-record.txt --single-version-externally-managed --compile --install-headers /autofs/vast/qtim/tools/pyenv/versions/3.6.15/envs/synthseg_new/include/site/python3.6/grpcio" failed with error code 1 in /local_mount/space/glacier/2/scratch/pip-install-qup507uh/grpcio/

[hvgazula]

@Shreyas-Bhat you mentioned "It might be that the problem is with me not using a pyenv, instead of a conda env,..." If I understood correctly, you are a conda user. In that case, the following steps worked fine for me (on Rocky Linux).

conda create -n ss_36 python=3.6 -y
conda activate ss_36
pip install -r requirements_python3.6.txt

[Shreyas-Bhat]

Hi @BBillot @neuronflow , I really appreciate your help here and I've been able to resolve my issues with regards to installation. Thanks a lot!