Registration sometimes iterates 100x slower #998

dzenanz · 2023-11-30T17:43:58Z

dzenanz
Nov 30, 2023

Sometimes, registration is a lot slower. Debugging, I tried different versions, see InsightSoftwareConsortium/ITKElastix#265, and the behavior seems the same in all of them. It can be triggered by various differences. Applying N4 bias field correction to images before registration seems to be a big trigger. I even managed to swap fast/slow behavior with parameter_map['RandomSeed'] = ['30101983']. Below is the log of a good run, and a bad one, for the same set of parameters (all except RandomSeed is the same), and for the same images, for similarity transform. It also happens for BSpline transform, but speed difference is smaller (20x-30x). I don't remember whether results are identical, or merely close. Do you have any suggestions of how to avoid slowness? What else consumes iteration time besides evaluating metrics?

Good run log:

ELASTIX version: 5.1.0
Command line options from ElastixBase:
-out      MRI_to_MRI\F05\F95_Neck/
-priority unspecified, so NORMAL process priority
-threads  unspecified, so all available threads are used
WARNING: The parameter "UseDirectionCosines", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.

WARNING: The option "UseDirectionCosines" was not found in your parameter file.
  From elastix 4.8 it defaults to true!
This may change the behavior of your registrations considerably.

Command line options from TransformBase:
-t0       MRI_to_MRI\F05\F95_Neck\fiducial_transform.txt
Command line options from CorrespondingPointsEuclideanDistanceMetric:
-fp       MRI_to_MRI\F05\F95_Neck\fixed_point_set.txt
-mp       MRI_to_MRI\F05\F95_Neck\moving_point_set.txt

Reading images...
Reading images took 0 ms.

Reading the elastix parameters from file ...

WARNING: The parameter "HowToCombineTransforms", requested at entry number 0, does not exist at all.
  The default value "Compose" is used instead.
WARNING: the fixed pyramid schedule is not fully specified!
  A default pyramid schedule is used.
WARNING: the moving pyramid schedule is not fully specified!
  A default pyramid schedule is used.
WARNING: The parameter "AutomaticTransformInitialization", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "AutomaticTransformInitializationMethod", requested at entry number 0, does not exist at all.
  The default value "GeometricalCenter" is used instead.
Transform parameters are initialized as: [0, 0, 0, 0, 0, 0, 1]
Scales are estimated automatically.
Scales for transform parameters are: [35676.94029036686, 35680.293689377584, 37639.56580517747, 1, 1, 1, 13624.599973115224]
Loading landmarks for Metric1:CorrespondingPointsEuclideanDistanceMetric.
  Reading landmark file: MRI_to_MRI\F05\F95_Neck\fixed_point_set.txt
  Landmarks are specified in world coordinates.
  Number of specified points: 3
Loading landmarks for Metric1:CorrespondingPointsEuclideanDistanceMetric.
  Reading landmark file: MRI_to_MRI\F05\F95_Neck\moving_point_set.txt
  Landmarks are specified in world coordinates.
  Number of specified points: 3
Initialization of all components (before registration) took: 5 ms.
Preparation of the image pyramids took: 1342 ms.

Resolution: 0
WARNING: The parameter "ShowExactMetricValue", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "UseMultiThreadingForMetrics", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.
WARNING: The parameter "ShowExactMetricValue", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
Setting the fixed masks took: 0 ms.
Setting the moving masks took: 0 ms.
WARNING: The parameter "UseRelativeWeights", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "FixedImageBSplineInterpolationOrder", requested at entry number 0, does not exist at all.
  The default value "1" is used instead.
WARNING: The parameter "UseRandomSampleRegion", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "NumberOfHistogramBins", requested at entry number 0, does not exist at all.
  The default value "32" is used instead.
WARNING: The parameter "NumberOfFixedHistogramBins", requested at entry number 0, does not exist at all.
  The default value "32" is used instead.
WARNING: The parameter "NumberOfMovingHistogramBins", requested at entry number 0, does not exist at all.
  The default value "32" is used instead.
WARNING: The parameter "FixedLimitRangeRatio", requested at entry number 0, does not exist at all.
  The default value "0.01" is used instead.
WARNING: The parameter "MovingLimitRangeRatio", requested at entry number 0, does not exist at all.
  The default value "0.01" is used instead.
WARNING: The parameter "FixedKernelBSplineOrder", requested at entry number 0, does not exist at all.
  The default value "0" is used instead.
WARNING: The parameter "MovingKernelBSplineOrder", requested at entry number 0, does not exist at all.
  The default value "3" is used instead.
WARNING: The parameter "UseFastAndLowMemoryVersion", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.
WARNING: The parameter "UseJacobianPreconditioning", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "FiniteDifferenceDerivative", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "SP_A", requested at entry number 0, does not exist at all.
  The default value "20" is used instead.

WARNING: You have set MaximumNumberOfSamplingAttempts to 8.
  This functionality is known to cause problems (stack overflow) for large values.
  If elastix stops or segfaults for no obvious reason, reduce this value.
  You may select the RandomSparseMask image sampler to fix mask-related problems.

WARNING: The parameter "SigmoidInitialTime", requested at entry number 0, does not exist at all.
  The default value "0" is used instead.
WARNING: The parameter "MaxBandCovSize", requested at entry number 0, does not exist at all.
  The default value "192" is used instead.
WARNING: The parameter "NumberOfBandStructureSamples", requested at entry number 0, does not exist at all.
  The default value "10" is used instead.
WARNING: The parameter "UseAdaptiveStepSizes", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.
WARNING: The parameter "UseConstantStep", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "MaximumStepLengthRatio", requested at entry number 0, does not exist at all.
  The default value "1" is used instead.
WARNING: The parameter "MaximumStepLength", requested at entry number 0, does not exist at all.
  The default value "0.70315" is used instead.
WARNING: The parameter "NumberOfGradientMeasurements", requested at entry number 0, does not exist at all.
  The default value "0" is used instead.
WARNING: The parameter "NumberOfJacobianMeasurements", requested at entry number 0, does not exist at all.
  The default value "1000" is used instead.
WARNING: The parameter "SigmoidScaleFactor", requested at entry number 0, does not exist at all.
  The default value "0.1" is used instead.
Elastix initialization of all components (for this resolution) took: 0 ms.
Initialization of AdvancedMattesMutualInformation metric took: 38 ms.
Initialization of CorrespondingPointsEuclideanDistance metric took: 0 ms.
Starting automatic parameter estimation for AdaptiveStochasticGradientDescent ...
WARNING: The parameter "ASGDParameterEstimationMethod", requested at entry number 0, does not exist at all.
  The default value "Original" is used instead.
  Computing JacobianTerms ...
  Computing the Jacobian terms took 0.001309s
  NumberOfGradientMeasurements to estimate sigma_i: 5
  Sampling gradients ...
  Sampling the gradients took 0.012342s
Automatic parameter estimation took 0.01s
1:ItNr	2:Metric	2:Metric0	2:Metric1	3a:Time	3b:StepSize	4:||Gradient0||	4:||Gradient1||	4:||Gradient||	Time0[ms]	Time1[ms]	Time[ms]
0	-0.371916	-0.380681	1.372297	0.000000	16.579673	0.981848	18.751915	0.010654	0.8	0.0	53.0
1	-0.379819	-0.388531	1.353903	0.000000	16.579673	1.037432	19.996484	0.010114	0.9	0.0	1.2
2	-0.377315	-0.385965	1.343976	0.000000	16.579673	0.613462	22.388481	0.009921	0.9	0.0	1.3
3	-0.377481	-0.386215	1.360523	0.000000	16.579673	2.022232	24.381236	0.014349	0.8	0.0	1.2
4	-0.362779	-0.371609	1.394404	0.000000	16.579673	0.656607	22.670786	0.006344	0.9	0.0	1.2
5	-0.365175	-0.374094	1.409775	0.000000	16.579673	1.374662	20.621834	0.008070	0.9	0.0	1.2
6	-0.356354	-0.365423	1.448515	0.000000	16.579673	1.201290	19.051091	0.008624	0.8	0.0	1.2
7	-0.373869	-0.383152	1.473524	0.000000	16.579673	1.457586	17.916623	0.011508	0.8	0.0	1.2
8	-0.337802	-0.347161	1.524760	0.000000	16.579673	0.705133	14.297747	0.005929	0.9	0.0	1.2
9	-0.366803	-0.376485	1.559901	0.000000	16.579673	0.315199	14.083972	0.005559	0.9	0.0	1.2
10	-0.360036	-0.369899	1.602683	0.000000	16.579673	0.234393	12.947334	0.005916	0.9	0.0	1.3
11	-0.354823	-0.364925	1.655469	0.000000	16.579673	0.758665	10.871057	0.007700	0.8	0.0	1.2
12	-0.381569	-0.392101	1.714302	0.000000	16.579673	0.525661	10.798590	0.005778	0.9	0.0	1.2
13	-0.379178	-0.389938	1.761992	0.000000	16.579673	1.064546	10.570370	0.008059	1.2	0.0	1.6
14	-0.372087	-0.383115	1.822549	0.000000	16.579673	1.102724	10.816817	0.007490	0.9	0.0	1.3
15	-0.378544	-0.389855	1.872415	0.000000	16.579673	0.584234	10.822676	0.007014	1.2	0.0	1.6
16	-0.363395	-0.374944	1.934920	0.000000	16.579673	1.012252	10.540787	0.007333	1.0	0.0	1.4
17	-0.372565	-0.384329	1.968489	0.000000	16.579673	1.283464	10.452669	0.009100	1.0	0.0	1.3

Bad run log:

ELASTIX version: 5.1.0
Command line options from ElastixBase:
-out      MRI_to_MRI\F05\F95_Neck/
-priority unspecified, so NORMAL process priority
-threads  unspecified, so all available threads are used
WARNING: The parameter "UseDirectionCosines", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.

WARNING: The option "UseDirectionCosines" was not found in your parameter file.
  From elastix 4.8 it defaults to true!
This may change the behavior of your registrations considerably.

Command line options from TransformBase:
-t0       MRI_to_MRI\F05\F95_Neck\fiducial_transform.txt
Command line options from CorrespondingPointsEuclideanDistanceMetric:
-fp       MRI_to_MRI\F05\F95_Neck\fixed_point_set.txt
-mp       MRI_to_MRI\F05\F95_Neck\moving_point_set.txt

Reading images...
Reading images took 0 ms.

Reading the elastix parameters from file ...

WARNING: The parameter "HowToCombineTransforms", requested at entry number 0, does not exist at all.
  The default value "Compose" is used instead.
WARNING: the fixed pyramid schedule is not fully specified!
  A default pyramid schedule is used.
WARNING: the moving pyramid schedule is not fully specified!
  A default pyramid schedule is used.
WARNING: The parameter "AutomaticTransformInitialization", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "AutomaticTransformInitializationMethod", requested at entry number 0, does not exist at all.
  The default value "GeometricalCenter" is used instead.
Transform parameters are initialized as: [0, 0, 0, 0, 0, 0, 1]
Scales are estimated automatically.
Scales for transform parameters are: [35676.94029036686, 35680.293689377584, 37639.56580517747, 1, 1, 1, 13624.599973115224]
Loading landmarks for Metric1:CorrespondingPointsEuclideanDistanceMetric.
  Reading landmark file: MRI_to_MRI\F05\F95_Neck\fixed_point_set.txt
  Landmarks are specified in world coordinates.
  Number of specified points: 3
Loading landmarks for Metric1:CorrespondingPointsEuclideanDistanceMetric.
  Reading landmark file: MRI_to_MRI\F05\F95_Neck\moving_point_set.txt
  Landmarks are specified in world coordinates.
  Number of specified points: 3
Initialization of all components (before registration) took: 4 ms.
Preparation of the image pyramids took: 1332 ms.

Resolution: 0
WARNING: The parameter "ShowExactMetricValue", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "UseMultiThreadingForMetrics", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.
WARNING: The parameter "ShowExactMetricValue", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
Setting the fixed masks took: 0 ms.
Setting the moving masks took: 0 ms.
WARNING: The parameter "UseRelativeWeights", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "FixedImageBSplineInterpolationOrder", requested at entry number 0, does not exist at all.
  The default value "1" is used instead.
WARNING: The parameter "UseRandomSampleRegion", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "NumberOfHistogramBins", requested at entry number 0, does not exist at all.
  The default value "32" is used instead.
WARNING: The parameter "NumberOfFixedHistogramBins", requested at entry number 0, does not exist at all.
  The default value "32" is used instead.
WARNING: The parameter "NumberOfMovingHistogramBins", requested at entry number 0, does not exist at all.
  The default value "32" is used instead.
WARNING: The parameter "FixedLimitRangeRatio", requested at entry number 0, does not exist at all.
  The default value "0.01" is used instead.
WARNING: The parameter "MovingLimitRangeRatio", requested at entry number 0, does not exist at all.
  The default value "0.01" is used instead.
WARNING: The parameter "FixedKernelBSplineOrder", requested at entry number 0, does not exist at all.
  The default value "0" is used instead.
WARNING: The parameter "MovingKernelBSplineOrder", requested at entry number 0, does not exist at all.
  The default value "3" is used instead.
WARNING: The parameter "UseFastAndLowMemoryVersion", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.
WARNING: The parameter "UseJacobianPreconditioning", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "FiniteDifferenceDerivative", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "SP_A", requested at entry number 0, does not exist at all.
  The default value "20" is used instead.

WARNING: You have set MaximumNumberOfSamplingAttempts to 8.
  This functionality is known to cause problems (stack overflow) for large values.
  If elastix stops or segfaults for no obvious reason, reduce this value.
  You may select the RandomSparseMask image sampler to fix mask-related problems.

WARNING: The parameter "SigmoidInitialTime", requested at entry number 0, does not exist at all.
  The default value "0" is used instead.
WARNING: The parameter "MaxBandCovSize", requested at entry number 0, does not exist at all.
  The default value "192" is used instead.
WARNING: The parameter "NumberOfBandStructureSamples", requested at entry number 0, does not exist at all.
  The default value "10" is used instead.
WARNING: The parameter "UseAdaptiveStepSizes", requested at entry number 0, does not exist at all.
  The default value "true" is used instead.
WARNING: The parameter "UseConstantStep", requested at entry number 0, does not exist at all.
  The default value "false" is used instead.
WARNING: The parameter "MaximumStepLengthRatio", requested at entry number 0, does not exist at all.
  The default value "1" is used instead.
WARNING: The parameter "MaximumStepLength", requested at entry number 0, does not exist at all.
  The default value "0.70315" is used instead.
WARNING: The parameter "NumberOfGradientMeasurements", requested at entry number 0, does not exist at all.
  The default value "0" is used instead.
WARNING: The parameter "NumberOfJacobianMeasurements", requested at entry number 0, does not exist at all.
  The default value "1000" is used instead.
WARNING: The parameter "SigmoidScaleFactor", requested at entry number 0, does not exist at all.
  The default value "0.1" is used instead.
Elastix initialization of all components (for this resolution) took: 0 ms.
Initialization of AdvancedMattesMutualInformation metric took: 2727 ms.
Initialization of CorrespondingPointsEuclideanDistance metric took: 0 ms.
Starting automatic parameter estimation for AdaptiveStochasticGradientDescent ...
WARNING: The parameter "ASGDParameterEstimationMethod", requested at entry number 0, does not exist at all.
  The default value "Original" is used instead.
  Computing JacobianTerms ...
  Computing the Jacobian terms took 0.917768s
  NumberOfGradientMeasurements to estimate sigma_i: 5
  Sampling gradients ...
  Sampling the gradients took 10.316487s
Automatic parameter estimation took 11.23s
1:ItNr	2:Metric	2:Metric0	2:Metric1	3a:Time	3b:StepSize	4:||Gradient0||	4:||Gradient1||	4:||Gradient||	Time0[ms]	Time1[ms]	Time[ms]
0	-0.379434	-0.388236	1.372297	0.000000	16.579673	1.227569	18.751915	0.011040	1.5	0.0	16236.5
1	-0.376453	-0.385149	1.354039	0.000000	16.579673	0.547429	21.036774	0.010379	1.4	0.0	943.5
2	-0.376267	-0.384987	1.359110	0.000000	16.579673	1.994400	23.783733	0.014176	1.3	0.0	928.4
3	-0.361611	-0.370393	1.386075	0.000000	16.579673	0.749641	23.689085	0.006894	1.5	0.0	957.1
4	-0.364250	-0.373091	1.395240	0.000000	16.579673	1.306802	22.740870	0.007736	1.7	0.0	938.8
5	-0.355370	-0.364337	1.429043	0.000000	16.579673	1.220397	21.564943	0.009166	2.0	0.0	1296.5
6	-0.372409	-0.381585	1.453571	0.000000	16.579673	1.546021	20.249420	0.012338	1.6	0.0	1099.1
7	-0.337544	-0.346767	1.497708	0.000000	16.579673	0.722362	16.979470	0.005991	1.5	0.0	973.9
8	-0.366691	-0.376246	1.534828	0.000000	16.579673	0.275546	15.933911	0.005788	1.6	0.0	960.2
9	-0.359921	-0.369667	1.579469	0.000000	16.579673	0.258675	14.425550	0.006200	1.4	0.0	948.0
10	-0.354369	-0.364361	1.634142	0.000000	16.579673	0.826325	12.131923	0.008100	1.6	0.0	923.6
11	-0.381587	-0.392016	1.693856	0.000000	16.579673	0.601636	11.786207	0.006136	1.6	0.0	938.5
12	-0.378841	-0.389505	1.743218	0.000000	16.579673	1.072948	11.173149	0.008189	1.5	0.0	966.4
13	-0.371775	-0.382705	1.803136	0.000000	16.579673	1.109345	11.272871	0.007534	1.5	0.0	1238.0
14	-0.378345	-0.389555	1.852405	0.000000	16.579673	0.617715	11.167814	0.007088	1.5	0.0	964.7
15	-0.363283	-0.374729	1.914440	0.000000	16.579673	1.009196	10.876155	0.007275	1.7	0.0	923.6
16	-0.372339	-0.383996	1.947409	0.000000	16.579673	1.276445	10.915026	0.009156	1.6	0.0	949.8
17	-0.378207	-0.390129	1.994363	0.000000	16.579673	1.377497	9.209004	0.011836	1.4	0.0	1342.5

dzenanz · 2023-11-30T18:08:30Z

dzenanz
Nov 30, 2023
Author

dzenanz/ITKElastix@325971e is an unsuccessful attempt at creating a minimum working example to demonstrate the difference.

1 reply

dzenanz Dec 1, 2023
Author

I just confirmed that the problem happens on a Linux computer too. Ubuntu 22.04, Python 3.10.12, itk-elastix 0.19.0, itk-5.4rc1.

N-Dekker · 2023-12-04T17:10:04Z

N-Dekker
Dec 4, 2023
Maintainer

Thanks for discussing this problem. Honestly, I don't know what might make the registration sometimes slower.

In your "Bad run log", I see:

Sampling the gradients took 10.316487s

Which must be from:

elastix/Components/Optimizers/AdaptiveStochasticGradientDescent/elxAdaptiveStochasticGradientDescent.hxx

Line 602 in c3fb0eb

    
           log::info(std::ostringstream{} << "  Sampling the gradients took " << Conversion::SecondsToDHMS(timer3.GetMean(), 6));

That seems to suggest that the following call is a bottleneck:

this->SampleGradients(this->GetScaledCurrentPosition(), sigma4, gg, ee);

SampleGradients uses ImageGridSampler, which is slightly faster now, when the current main revision from elastix is taken, as it includes:

pull request PERF: ImageGridSampler twice as fast, doing sampleVector.reserve(numberOfSamplesOnGrid) #994 commit 21cb9e9

We expect further ImageGridSampler performance improvements in the future by adding multi-threading, but that's still work in progress.

Anyway, I don't know if that would entirely solve the problem you're discussing here. To be continued...!

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Registration sometimes iterates 100x slower #998

{{title}}

Replies: 2 comments 1 reply

{{title}}

{{title}}

{{title}}

Select a reply

Registration sometimes iterates 100x slower #998

dzenanz Nov 30, 2023

Replies: 2 comments · 1 reply

dzenanz Nov 30, 2023 Author

dzenanz Dec 1, 2023 Author

N-Dekker Dec 4, 2023 Maintainer

dzenanz
Nov 30, 2023

Replies: 2 comments 1 reply

dzenanz
Nov 30, 2023
Author

dzenanz Dec 1, 2023
Author

N-Dekker
Dec 4, 2023
Maintainer