macでimport numpyをするとinit_dgelsd failed initして使えない問題

pyenvが使えるようになったので、python 3.8.6をインストールした。 blog.hayashikun.com

いつもどおりnumpyやらscipyやらmatplotlibなどをインストールしていたら、matplotlibのインストール中にエラーが起きた。

原因はnumpyが使える状態になっていなくて、この状態でimport numpyをすると、

Python 3.8.6 (default, Nov 14 2020, 10:49:48) 
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Python(46878,0x10ae1be00) malloc: can't allocate region
:*** mach_vm_map(size=18446744071929176064, flags: 100) failed (error code=3)
Python(46878,0x10ae1be00) malloc: *** set a breakpoint in malloc_error_break to debug
init_dgelsd failed init
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/numpy/__init__.py", line 286, in <module>
    raise RuntimeError(msg)
RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, see site.cfg.example for information. Otherwise report this to the vendor that provided NumPy.
RankWarning: Polyfit may be poorly conditioned

とエラーが出た。

これは、numpyをimportしたときに走るmacosでちゃんと動くかなチェックで、BLASとか関連の問題ぽい。

https://github.com/numpy/numpy/blob/master/numpy/__init__.py#L338

実際に、numpyのconfigを見てみると、

>>> from numpy.__config__ import show
>>> show()
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
atlas_3_10_blas_threads_info:
  NOT AVAILABLE
atlas_3_10_blas_info:
  NOT AVAILABLE
atlas_blas_threads_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
accelerate_info:
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
blas_opt_info:
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
  NOT AVAILABLE
openblas_clapack_info:
  NOT AVAILABLE
flame_info:
  NOT AVAILABLE
atlas_3_10_threads_info:
  NOT AVAILABLE
atlas_3_10_info:
  NOT AVAILABLE
atlas_threads_info:
  NOT AVAILABLE
atlas_info:
  NOT AVAILABLE
lapack_opt_info:
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]

とNOT AVAILABLEばかりで、brewで入れているはずのopenblasとかが認識されていない。

brewで入れたpython 3.9では何もしていないけどちゃんとopenblasが認識されていて、

>>> import numpy as np
>>> np.show_config()
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
lapack_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

となる。

対処法は、HOME下に.numpy-site.cfgを置き、--no-binaryにする。（--no-binaryにしなくてもいけるかな？試してない）

ksaw.hatenablog.com

$ cat ~/.numpy-site.cfg
[openblas]
libraries = openblas
library_dirs = /usr/local/opt/openblas/lib
include_dirs = /usr/local/opt/openblas/include
runtime_library_dirs = /usr/local/opt/openblas/lib

ちなみに、$(brew --prefix openblas)/libとかにすると認識されなかった。 --verboseを付けて見ていると、読めへんわとエラーがでていた。

はやし雑記

はやしです

macでimport numpyをするとinit_dgelsd failed initして使えない問題