问题背景
Cython是用来加速Python程序性能的一个工具,其基本使用逻辑就是将类Python代码(*.pyx扩展格式)编译成\(*.c,*.so\)动态链接库文件,然后就可以在正常的Python脚本文件中调用动态链接库的内部函数。编译过程中因为会去索引一些头文件,如果找不到路径就有可能报错。
测试案例
我们可以用Cython做一个简单的基于numpy array输入的求和函数:
# test_sum.pyx
import numpy as np
cimport numpy as np
cpdef double my_sum(double[:] arr):
cdef double s = 0.0
cdef int i
for i in range(arr.shape[0]):
s += arr[i]
return s
def main():
a = np.array([1.0, 2.0, 3.0])
print(my_sum(a))
然后编译:
$ cythonize -i test_sum.pyx
running build_ext
building 'test_sum' extension
creating /home/mindsponge/tests/tmpwo3gq_ad/home
creating /home/mindsponge/tests/tmpwo3gq_ad/home/mindsponge
creating /home/mindsponge/tests/tmpwo3gq_ad/home/mindsponge/tests
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/python-3.7.5/include/python3.7m -c /home/mindsponge/tests/test_sum.c -o /home/mindsponge/tests/tmpwo3gq_ad/home/mindsponge/tests/test_sum.o
/home/mindsponge/tests/test_sum.c:1240:10: fatal error: numpy/arrayobject.h: No such file or directory
#include "numpy/arrayobject.h"
^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1
这个报错是找不到numpy下的一个头文件,那么解决问题的思路就很直接,找到相关头文件的路径,添加到gcc编译的环境变量中即可。先看一下numpy的安装路径:
$ python3 -m pip show numpy
Name: numpy
Version: 1.21.6
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email:
License: BSD
Location: /usr/local/python-3.7.5/lib/python3.7/site-packages
Requires:
Required-by: bitshuffle, fabio, h5py, matplotlib, mindinsight, mindspore, mindspore-serving, pandas, pyopencl, scikit-learn, scipy, silx, Xponge
然后在这个Location中找到对应的头文件:
$ find /usr/local/python-3.7.5/lib/python3.7/site-packages -name 'arrayobject.h'
/usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h
找到以后先查看一下环境变量中是否已有值,然后再将上面这个路径添加到环境变量中:
$ export | grep C_INCLUDE_PATH
$ export C_INCLUDE_PATH=/usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/
再次执行编译:
# cythonize -i test_sum.pyx
running build_ext
building 'test_sum' extension
creating /home/mindsponge/tests/tmpw_icoc3b/home
creating /home/mindsponge/tests/tmpw_icoc3b/home/mindsponge
creating /home/mindsponge/tests/tmpw_icoc3b/home/mindsponge/tests
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/python-3.7.5/include/python3.7m -c /home/mindsponge/tests/test_sum.c -o /home/mindsponge/tests/tmpw_icoc3b/home/mindsponge/tests/test_sum.o
In file included from /usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1969:0,
from /usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from /home/mindsponge/tests/test_sum.c:1240:
/usr/local/python-3.7.5/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it with " \
^~~~~~~
gcc -pthread -shared /home/mindsponge/tests/tmpw_icoc3b/home/mindsponge/tests/test_sum.o -L/usr/local/python-3.7.5/lib -lpython3.7m -o /home/mindsponge/tests/test_sum.cpython-37m-x86_64-linux-gnu.so
编译顺利通过,并且会在当前路径下生成一个*.c
文件和一个*.so
动态链接库文件。然后就可以在python中直接引用动态链接库的内部函数:
In [1]: import numpy as np
In [2]: a = np.array([1.0, 2.0, 3.0])
In [3]: from test_erf import my_sum
In [4]: my_sum(a)
Out[4]: 6.0
总结概要
本文介绍了一个在使用Cython进行Python高性能编程时有可能遇到的一个问题,就是找不到的对应的C语言的头文件,例如numpy中的一些头文件。解决思路就是先在本地找到相应的头文件路径,然后将其添加到编译器的环境变量中即可。
版权声明
本文首发链接为:https://www.cnblogs.com/dechinphy/p/win11-vmd.html
作者ID:DechinPhy
更多原著文章:https://www.cnblogs.com/dechinphy/
请博主喝咖啡:https://www.cnblogs.com/dechinphy/gallery/image/379634.html