mriSuperresolution in antspynet

[limacora45]

Hi encountered the errors, while running image superresolution module of antspynet

        import keras as ke
        import tensorflow as tf
        import numpy as np
        image = ants.image_read(ants.get_ants_data('r16'))
        import antspynet.architectures as apa
        import antspynet.utilities as apu
        image_sr = apu.apply_super_resolution_model_to_image(image, apu.get_pretrained_network("dbpn4x"))
        Downloading data from https://ndownloader.figshare.com/files/13347617
        145252352/145251944 [==============================] - 9s 0us/step
        145260544/145251944 [==============================] - 9s 0us/step
        Traceback (most recent call last):
        File "", line 1, in
        File "/home/uname/anaconda3/lib/python3.8/site-packages/antspynet/utilities/super_resolution_utilities.py", line 279, in apply_super_resolution_model_to_image
        model = load_model(model)
        File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
        raise e.with_traceback(filtered_tb) from None
        File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py", line 349, in init
        raise TypeError(
        TypeError: Expected trainable argument to be a boolean, but got: None

And while running the doSuperresolution.py from https://github.com/ANTsXNet/MRISuperResolution:
python Scripts/doSuperResolution.py Data/Example/1097782_defaced_MPRAGE_downsampled.nii.gz 1097782_defaced_MPRAGE_superResolution.nii.gz
I encountered the following errors:
ValueError: Layer count mismatch when loading weights from file. Model expected 125 layers, found 43 saved layers.

Really need help to get it working.
my python version
python 3.8.8
tensorflow 2.8.0

[ntustison]

The first example uses a 2-D image (super-resolution is 3-D) and the second example, as stated in the README, is deprecated. Try using the second image in the first example.

[limacora45]

import ants
import antspynet
import antspynet.architectures as apa
import antspynet.utilities as apu
import tensorflow as tf
import numpy as np
t1_file="/mnt/4e9ca5bb-8e5f-43b4-850a-00a96aa2d8b4/mri_upsample/MRISuperResolution/Data/Example/1097782_defaced_MPRAGE_downsampled.nii.gz"
image = ants.image_read(t1_file)
image_sr=apu.apply_super_resolution_model_to_image(image, apu.get_pretrained_network("mriSuperResolution"))

Still facing error:

TypeError Traceback (most recent call last)
/tmp/ipykernel_1632866/2303928518.py in
7 t1_file="/mnt/4e9ca5bb-8e5f-43b4-850a-00a96aa2d8b4/mri_upsample/MRISuperResolution/Data/Example/1097782_defaced_MPRAGE_downsampled.nii.gz"
8 image = ants.image_read(t1_file)
----> 9 image_sr=apu.apply_super_resolution_model_to_image(image, apu.get_pretrained_network("mriSuperResolution"))

~/anaconda3/lib/python3.8/site-packages/antspynet/utilities/super_resolution_utilities.py in apply_super_resolution_model_to_image(image, model, target_range, batch_size, regression_order, verbose)
277 tflite_flag = True
278 else:
--> 279 model = load_model(model)
280 shape_length = len(model.input_shape)
281

~/anaconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.traceback)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb

~/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py in init(self, trainable, name, dtype, dynamic, **kwargs)
347 (isinstance(trainable, (tf.Tensor, tf.Variable)) and
348 trainable.dtype is tf.bool)):
--> 349 raise TypeError(
350 'Expected trainable argument to be a boolean, '
351 f'but got: {trainable}')

TypeError: Expected trainable argument to be a boolean, but got: None

​

[limacora45]

Please let me know how I can troubleshoot this to get this module working. I am using pop os 21.10 (Linux) as the operating system, in case that is necessary. I tried in ubuntu 20.04 LTS, with no luck as well. And in addition, I also received the same error when trying: image_srr=antspynet.utilities.mri_super_resolution(image)

[ntustison]

I just ran it and didn't have any problem (see below). You might want to clean out your ANTsXNet cache (~/.keras/ANTsXNet) to make sure you have the correct model.

(base) ntustison@Histograms Desktop % python3
Python 3.8.5 (default, Sep  4 2020, 02:22:02) 
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ants
>>> import antspynet
INFO:tensorflow:Enabling eager execution
INFO:tensorflow:Enabling v2 tensorshape
INFO:tensorflow:Enabling resource variables
INFO:tensorflow:Enabling tensor equality
INFO:tensorflow:Enabling control flow v2
>>> t1 = ants.image_read("testData2mm.nii")
>>> image_sr = antspynet.mri_super_resolution(t1, verbose=True)
2022-03-08 08:10:30.404671: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-03-08 08:10:30.924440: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
>>> ants.image_write(image_sr, "image_sr.nii.gz")

[limacora45]

Hi, I tried to see whether ANTsXNet cache had files that were not correct. So I cleaned the directory (~/.keras/ANTsXNet) and re run it in two different linux distros.
pop os 21.10:
Python 3.9.7 (default, Sep 16 2021, 13:09:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import ants
import antspynet
t1=ants.image_read("t1.nii")
image_sr = antspynet.mri_super_resolution(t1, verbose=True)
Downloading data from https://ndownloader.figshare.com/files/24128618
34881536/34878800 [==============================] - 15s 0us/step
34889728/34878800 [==============================] - 15s 0us/step
Traceback (most recent call last):
File "", line 1, in
File "/home/uname/anaconda3/lib/python3.9/site-packages/antspynet/utilities/mri_super_resolution.py", line 45, in mri_super_resolution
model_sr = tf.keras.models.load_model(model_and_weights_file_name, compile=False)
File "/home/uname/anaconda3/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/uname/anaconda3/lib/python3.9/site-packages/keras/engine/base_layer.py", line 349, in init
raise TypeError(
TypeError: Expected trainable argument to be a boolean, but got: None

and the download in the process resulted in writing mriSuperResolution.h5 in the keras cache.

ubuntu 20.04 LTS:
Python 3.8.8 (default, Apr 13 2021, 19:58:26)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import ants
import antspynet
2022-03-08 18:20:14.240196: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-03-08 18:20:14.240214: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
t1=ants.image_read("bb.nii")
image_sr = antspynet.mri_super_resolution(t1, verbose=True)
Downloading data from https://ndownloader.figshare.com/files/24128618
34881536/34878800 [==============================] - 3s 0us/step
34889728/34878800 [==============================] - 3s 0us/step
Traceback (most recent call last):
File "", line 1, in
File "/home/uname/anaconda3/lib/python3.8/site-packages/antspynet/utilities/mri_super_resolution.py", line 45, in mri_super_resolution
model_sr = tf.keras.models.load_model(model_and_weights_file_name, compile=False)
File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py", line 349, in init
raise TypeError(
TypeError: Expected trainable argument to be a boolean, but got: None

so the model seems to be correct if this mriSuperResolution.h5 weight file is supposed to be written in the keras cache.

Would be nice to get it running, so please let me know what I should do. I am also new to antspynet and antspy, so your help here would be much appreciated.

[ntustison]

I can't reproduce this error so there's not much I can suggest. However, I would recommend looking at the error that's being thrown in mri_super_resolution.py. It's simply loading the model weights. For some reason, I'm able to load it and you are not---I don't know why.

>>> model_and_weights_file_name = antspynet.get_pretrained_network("mriSuperResolution")
>>> model_sr = tf.keras.models.load_model(model_and_weights_file_name, compile=False)

Perhaps you have an outdated version of keras/tensorflow.

[limacora45]

Hi, Ntustison,
First of all thank you for such quick responses, happy that after your responses, I got the urge to dig deeper.
Firstly, the keras/tensorflow version that i have is the most recent ones, 2.8.0. And I created a virtual environment to install different tensorflow versions, and tried to install antspynet, and in one of the venv, i let antspynet install whatever tensorflow it wants to run with.
With version 2.6.0,
tf, there was a bug noted in tf also something known when I searched online, that didn't allow me to load antspynet , then,
in tf version 2.7.0,
the command model_sr = tf.keras.models.load_model(model_and_weights_file_name, compile=False) threw this error:
This version of TensorFlow Probability requires TensorFlow version >= 2.8; Detected an installation of version 2.7.0. Please upgrade TensorFlow to proceed.

So in order to run the newest antspynet edition properly tensorflow 2.8 seems important.

now the error that I mentioned earlier, was an error that comes from the model file mriSuperResolution.h5, which has to do with setting some kind of trainable argument, I am not an expert here, just starting on antspy and antspynet. which threw the error in
File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py", line 349, in init
raise TypeError(
TypeError: Expected trainable argument to be a boolean, but got: None
so I opened the base_layer.py command, and commented out the boolean expectation, and re ran the command..
It worked.

Python 3.9.7 (default, Sep 16 2021, 13:09:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import ants
import antspynet
t1=ants.image_read("/media/uname/diskone/mri_ants_superresolution/MRISuperResolution/Data/Example/t1.nii")
image_sr = antspynet.mri_super_resolution(t1, verbose=True)
2022-03-09 06:15:11.387914: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-03-09 06:15:11.836725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 12910 MB memory: -> device: 0, name: NVIDIA RTX A4000, pci bus id: 0000:21:00.0, compute capability: 8.6
2022-03-09 06:15:14.208739: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8302
image_sr
ANTsImage (RPI)
Pixel Type : float (float32)
Components : 1
Dimensions : (96, 128, 128)
Spacing : (2.0, 2.0, 2.0)
Origin : (-91.5265, 110.0629, -142.0)
Direction : [ 1. 0. 0. 0. -1. 0. 0. 0. 1.]

ants.image_write(image_sr,"/media/uname/diskone/mri_ants_superresolution/MRISuperResolution/Data/Example/t1_super.nii.gz")

It turns out "/home/uname/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py", line 349, in init
raise TypeError(
TypeError: Expected trainable argument to be a boolean, but got: None"
is the key.

Now that this module is working for me, I do have some follow up questions, although these are not really issues in running the model, but more of the details in understanding the architectures available, and their usage, etc, but I am not sure whether this is the right platform. Its more like a discussion rather than an issue. So it would be great if you could let me know the correct platform for such queries, and discussion.

Thank you again for such quick response, the entire ANTS community do care a lot about their users.

[ntustison]

Now that this module is working for me, I do have some follow up questions, although these are not really issues in running the model, but more of the details in understanding the architectures available, and their usage, etc, but I am not sure whether this is the right platform. Its more like a discussion rather than an issue. So it would be great if you could let me know the correct platform for such queries, and discussion.

Just start a separate issue.