Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CREST fmriprep: ICA-AROMA failed (FAST1050, job 1320777) #17

Open
ariveradompenciel opened this issue Jun 4, 2020 · 1 comment
Open

CREST fmriprep: ICA-AROMA failed (FAST1050, job 1320777) #17

ariveradompenciel opened this issue Jun 4, 2020 · 1 comment
Labels
argon fmriprep issue when running fmriprep singularity

Comments

@ariveradompenciel
Copy link

User = sodoma

Script/Job File

#!/bin/bash

#$ -pe smp 16
#$ -q UI
#$ -m bea
#$ -M [email protected]
#$ -o /Shared/vosslabhpc/Projects/CREST/code/fmriprep/out/
#$ -e /Shared/vosslabhpc/Projects/CREST/code/fmriprep/err/
OMP_NUM_THREADS=10
singularity run -H ${HOME}/singularity_home -B /Shared/vosslabhpc:/mnt \
/Shared/vosslabhpc/UniversalSoftware/SingularityContainers/fmriprep-1.2.1.simg \
/mnt/Projects/CREST/ /mnt/Projects/CREST/derivatives \
participant --participant_label FAST1050 \
-w /nfsscratch/Users/sodoma/work/CRESTfmriprep \
--write-graph --mem_mb 35000 --omp-nthreads 10 --nthreads 16 --use-aroma \
--output-space template \
--template MNI152NLin2009cAsym \
--fs-license-file /mnt/UniversalSoftware/freesurfer_license.txt

Email Message

Job 1320777 (sub-FAST1050.job) Complete
User             = sodoma
Queue            = [email protected]
Host             = argon-itf-cf45-34.hpc
Start Time       = 06/03/2020 01:12:22
End Time         = 06/03/2020 09:52:22
User Time        = 00:24:29
System Time      = 00:00:08
Wallclock Time   = 08:40:00
CPU              = 21:52:20
Max vmem         = 8.999G
Exit Status      = 1

Output File

200603-04:55:18,536 nipype.interface WARNING:
	 q-forms of reference and mask are substantially different

200603-09:51:55,484 nipype.interface WARNING:
	 No signal components were classified
200603-09:51:55,489 nipype.workflow WARNING:
	 [Node] Error on "fmriprep_wf.single_subject_FAST1050_wf.func_preproc_task_rest_wf.ica_aroma_wf.ica_aroma_confound_extraction" (/nfsscratch/Users/sodoma/work/CRESTfmriprep/fmriprep_wf/single_subject_FAST1050_wf/func_preproc_task_rest_wf/ica_aroma_wf/ica_aroma_confound_extraction)
200603-09:51:57,53 nipype.workflow ERROR:
	 Node ica_aroma_confound_extraction failed to run on host argon-itf-cf45-34.hpc.
200603-09:51:57,58 nipype.workflow ERROR:
	 Saving crash info to /mnt/Projects/CREST/derivatives/fmriprep/sub-FAST1050/log/20200603-011231_51b53cc0-ffe7-4477-83f8-dd3784fbdb2c/crash-20200603-095157-sodoma-ica_aroma_confound_extraction-71b82bc1-7eb8-46c8-953d-7ab2739ee2f3.txt
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py", line 69, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 471, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 555, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.6/site-packages/nipype/interfaces/base/core.py", line 522, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/interfaces/confounds.py", line 123, in _run_interface
    raise RuntimeError('ICA-AROMA failed')
RuntimeError: ICA-AROMA failed

200603-09:52:14,300 nipype.workflow INFO:
	 [Node] Finished "fmriprep_wf.single_subject_FAST1050_wf.func_preproc_task_rest_wf.ica_aroma_wf.add_nonsteady".
200603-09:52:20,981 nipype.workflow ERROR:
	 could not run node: fmriprep_wf.single_subject_FAST1050_wf.func_preproc_task_rest_wf.ica_aroma_wf.ica_aroma_confound_extraction

Error (stderr file)

/usr/local/miniconda/lib/python3.6/site-packages/bids/grabbids/__init__.py:6: FutureWarning: grabbids has been renamed to layout in version 0.6.5, and will be removed in version 0.8
  warnings.warn("grabbids has been renamed to layout in version 0.6.5, and will be removed in version 0.8", FutureWarning)
[WARNING] This document format requires a nonempty <title> element.
  Please specify either 'title' or 'pagetitle' in the metadata.
  Falling back to 'CITATION'
/usr/local/miniconda/lib/python3.6/site-packages/bids/grabbids/__init__.py:6: FutureWarning: grabbids has been renamed to layout in version 0.6.5, and will be removed in version 0.8
  warnings.warn("grabbids has been renamed to layout in version 0.6.5, and will be removed in version 0.8", FutureWarning)
/usr/local/miniconda/lib/python3.6/site-packages/bids/layout/bids_layout.py:116: UserWarning: 'dataset_description.json' file is missing from project root. You may want to set the root path to a valid BIDS project.
  warnings.warn("'dataset_description.json' file is missing from "
/usr/local/miniconda/lib/python3.6/site-packages/nitime/utils.py:980: FutureWarning: Conversion of the second argument of issubdtype from `complex` to `np.complexfloating` is deprecated. In future, it will be treated as `np.complex128 == np.dtype(complex).type`.
  complex_result = (np.issubdtype(in1.dtype, np.complex) or
/usr/local/miniconda/lib/python3.6/site-packages/nitime/utils.py:981: FutureWarning: Conversion of the second argument of issubdtype from `complex` to `np.complexfloating` is deprecated. In future, it will be treated as `np.complex128 == np.dtype(complex).type`.
  np.issubdtype(in2.dtype, np.complex))
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.6/site-packages/nipype/algorithms/confounds.py:1022: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.6/site-packages/nilearn/image/resampling.py:518: UserWarning: Casting data from int32 to float32
  warnings.warn("Casting data from %s to %s" % (data.dtype.name, aux))
/usr/local/miniconda/lib/python3.6/site-packages/nilearn/image/resampling.py:518: UserWarning: Casting data from int32 to float32
  warnings.warn("Casting data from %s to %s" % (data.dtype.name, aux))
/usr/local/miniconda/lib/python3.6/site-packages/nilearn/image/resampling.py:518: UserWarning: Casting data from int32 to float32
  warnings.warn("Casting data from %s to %s" % (data.dtype.name, aux))
/usr/local/miniconda/lib/python3.6/site-packages/nilearn/image/resampling.py:518: UserWarning: Casting data from int32 to float32
  warnings.warn("Casting data from %s to %s" % (data.dtype.name, aux))
/usr/local/miniconda/lib/python3.6/site-packages/nilearn/image/resampling.py:518: UserWarning: Casting data from int32 to float32
  warnings.warn("Casting data from %s to %s" % (data.dtype.name, aux))
/usr/local/miniconda/lib/python3.6/site-packages/nilearn/image/resampling.py:518: UserWarning: Casting data from int32 to float32
  warnings.warn("Casting data from %s to %s" % (data.dtype.name, aux))
/usr/local/miniconda/lib/python3.6/site-packages/nilearn/image/resampling.py:518: UserWarning: Casting data from int32 to float32
  warnings.warn("Casting data from %s to %s" % (data.dtype.name, aux))
/usr/local/miniconda/lib/python3.6/site-packages/nilearn/image/resampling.py:518: UserWarning: Casting data from int32 to float32
  warnings.warn("Casting data from %s to %s" % (data.dtype.name, aux))
/usr/local/miniconda/lib/python3.6/site-packages/nilearn/image/resampling.py:518: UserWarning: Casting data from int32 to float32
  warnings.warn("Casting data from %s to %s" % (data.dtype.name, aux))
/usr/local/miniconda/lib/python3.6/site-packages/nipype/algorithms/confounds.py:1022: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
Errors occurred while generating reports for participants: FAST1050 (1).
@ariveradompenciel ariveradompenciel added argon fmriprep issue when running fmriprep singularity labels Jun 4, 2020
@ariveradompenciel ariveradompenciel changed the title CREST fmriprep: ICA AROMA failed (FAST1050, job 1320777) CREST fmriprep: ICA-AROMA failed (FAST1050, job 1320777) Jun 4, 2020
@jdkent
Copy link
Member

jdkent commented Jun 4, 2020

This is always a fun error. The most relevant bit is: No signal components were classified from ICA-AROMA, meaning according to ICA-AROMA, all the components derived were classified as noise.

In the past, this has meant the brainmask was unexpectedly misshapen or the transformation from bold space (native) to template space (MNI) was severely sub-optimal.

I haven't seen this before:
q-forms of reference and mask are substantially different

which could be a hint on what to look for, and that message is in #9 as well.

It will be helpful to have access to this participant's working directory and look at some of the intermediate outputs (I think this directory):
/nfsscratch/Users/sodoma/work/CRESTfmriprep/fmriprep_wf/single_subject_FAST1050_wf/

Could this directory be copied somewhere on the server that we all can access?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
argon fmriprep issue when running fmriprep singularity
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jdkent @ariveradompenciel