Problem with pip and anaconda scipy versions due to MKL

[verginer]

The results of functions which rely on scipy linear algebra (e.g. scipy.sparse.linalg.eigen) can give significant different results depending on whether scipy was compiled with MKL (i.e. conda install) or without (i.e. pip install).

The differences in results are significant. For example from the tests:

test_eigen_centrality
...
1.7205880187995493 != 1.7461339874793727
2.007652443416882 != 2.030946758

Similarly convergence is inconsistent depending on MKL.

To make this more transparent it might be good to explicitely tell the user when using affected functions through a log message (i.e. INFO) that the results might be different if an other scipy version is used.

The following invocation will list any MKL compile time flags:

In [1]: import scipy
In [2]: scipy.show_config()

openblas_lapack_info:
  NOT AVAILABLE
mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['anaconda3/envs/pathpy/lib']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['anaconda3/envs/pathpy/include']
lapack_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/anaconda3/envs/pathpy/lib']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/anaconda3/envs/pathpy/include']
lapack_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/anaconda3/envs/pathpy/lib']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/anaconda3/envs/pathpy/include']
blas_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/anaconda3/envs/pathpy/lib']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/anaconda3/envs/pathpy/include']
blas_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/anaconda3/envs/pathpy/lib']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/anaconda3/envs/pathpy/include']

[verginer]

Currently solved by setting the test to xfail(i.e. expected fail) and reduce the absolute accepted deviation from 1e-16 to 1e-3

[IngoScholtes]

now mentioned in list of known issues in Readme

[IngoScholtes]

Kamino cloned this issue to uzhdag/pathpy