在Alpine Linux安装机器学习环境
警告
我尚未完成这个安装部署,原因是 Alpine Linux 默认使用 musl 库会中断 为glibc发行版编译的 TensorFlow wheels。解决方式是需要自己从源代码编译,或者使用预先编译好的 musl-compatible wheel
我准备后续在部署了 GitLab CI/CD 环境后,再尝试自己完成编译发布
待续...
在 Alpine Docker镜像 构建 alpine-dev 镜像,其中部署 Machine Learning 工作环境涉及安装 scikit-learn :
# install scikit-learn need python3-dev, some python modules need build from source
sudo apk add python3 python3-dev
sh -c "cd /home/${USER} && python3 -m venv venv/ml"
sh -c "source /home/${USER}/venv/ml/bin/activate && pip install --upgrade pip && pip install numpy scipy matplotlib scikit-learn tensorflow-cpu==2.20.0 pillow"
安装报错处理
Python环境安装scikit-learn报错处理
涉及安装 scikit-learn 报错显示缺少 pkgconfig :
scikit-learn 报错信息 Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [31 lines of output]
+ meson setup /tmp/pip-install-ee5bw9jl/scikit-learn_dbc81d3a0c6d451894cd93ee9fcb0e83 /tmp/pip-install-ee5bw9jl/scikit-learn_dbc81d3a0c6d451894cd93ee9fcb0e83/.mesonpy-w_rmp7ki -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/tmp/pip-install-ee5bw9jl/scikit-learn_dbc81d3a0c6d451894cd93ee9fcb0e83/.mesonpy-w_rmp7ki/meson-python-native-file.ini
The Meson build system
Version: 1.9.1
Source dir: /tmp/pip-install-ee5bw9jl/scikit-learn_dbc81d3a0c6d451894cd93ee9fcb0e83
Build dir: /tmp/pip-install-ee5bw9jl/scikit-learn_dbc81d3a0c6d451894cd93ee9fcb0e83/.mesonpy-w_rmp7ki
Build type: native build
Project name: scikit-learn
Project version: 1.7.2
C compiler for the host machine: cc (gcc 14.2.0 "cc (Alpine 14.2.0) 14.2.0")
C linker for the host machine: cc ld.bfd 2.44
C++ compiler for the host machine: c++ (gcc 14.2.0 "c++ (Alpine 14.2.0) 14.2.0")
C++ linker for the host machine: c++ ld.bfd 2.44
Cython compiler for the host machine: cython (cython 3.1.6)
Host machine cpu family: x86_64
Host machine cpu: x86_64
Compiler for C supports arguments -Wno-unused-but-set-variable: YES
Compiler for C supports arguments -Wno-unused-function: YES
Compiler for C supports arguments -Wno-conversion: YES
Compiler for C supports arguments -Wno-misleading-indentation: YES
Library m found: YES
Program sklearn/_build_utils/tempita.py found: YES (/home/admin/venv/ml/bin/python3 /tmp/pip-install-ee5bw9jl/scikit-learn_dbc81d3a0c6d451894cd93ee9fcb0e83/sklearn/_build_utils/tempita.py)
Program python found: YES (/home/admin/venv/ml/bin/python3)
Run-time dependency OpenMP for c found: YES 4.5
Program cython found: YES (/tmp/pip-build-env-e8ycri1e/overlay/bin/cython)
Did not find pkg-config by name 'pkg-config'
Found pkg-config: NO
Run-time dependency python found: NO (tried pkgconfig, pkgconfig and sysconfig)
../sklearn/meson.build:205:5: ERROR: Python dependency not found
A full log can be found at /tmp/pip-install-ee5bw9jl/scikit-learn_dbc81d3a0c6d451894cd93ee9fcb0e83/.mesonpy-w_rmp7ki/meson-logs/meson-log.txt
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> scikit-learn
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
上述报错是因为操作系统没有安装 python3-dev 软件包导致的,该软件包在系统级别提供了Python开发头文件,用于编译源代码。所以解决方法是先安装 python3-dev 软件包(如果系统已经安装了 python3 ),然后再重新在 Python virtualenv 环境安装 scikit-learn 。
备注
已修订 Alpine Docker镜像 构建 alpine-dev 镜像
Python环境安装tensorflow-cpu提示python版本过高
我最初执行 pip install tensorflow-cpu 没有指定版本,我以为会自动安装最新版本,其实不是,没有指定版本则会安装版本 1.9.1 ,这时会提示报错:
tensorflow-cpu 提示python版本过高ERROR: Ignored the following versions that require a different python version: 1.10.0 Requires-Python >=3.8,<3.12; 1.10.0rc1 Requires-Python >=3.8,<3.12; 1.10.0rc2 Requires-Python >=3.8,<3.12; 1.10.1 Requires-Python >=3.8,<3.12; 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11; 1.6.2 Requires-Python >=3.7,<3.10; 1.6.3 Requires-Python >=3.7,<3.10; 1.7.0 Requires-Python >=3.7,<3.10; 1.7.1 Requires-Python >=3.7,<3.10; 1.7.2 Requires-Python >=3.7,<3.11; 1.7.3 Requires-Python >=3.7,<3.11; 1.8.0 Requires-Python >=3.8,<3.11; 1.8.0rc1 Requires-Python >=3.8,<3.11; 1.8.0rc2 Requires-Python >=3.8,<3.11; 1.8.0rc3 Requires-Python >=3.8,<3.11; 1.8.0rc4 Requires-Python >=3.8,<3.11; 1.8.1 Requires-Python >=3.8,<3.11; 1.9.0 Requires-Python >=3.8,<3.12; 1.9.0rc1 Requires-Python >=3.8,<3.12; 1.9.0rc2 Requires-Python >=3.8,<3.12; 1.9.0rc3 Requires-Python >=3.8,<3.12; 1.9.1 Requires-Python >=3.8,<3.12
ERROR: Could not find a version that satisfies the requirement tensorflow-cpu (from versions: none)
ERROR: No matching distribution found for tensorflow-cpu
实际上 tensorflow-cpu 最新版本 2.20.0 是支持Python3系列到 python 3.13 版本的,但是非常奇怪,即使我指定了版本( pip install tensorflow-cpu==2.20.0 ),依然报相同的错误
tensorflow-cpu 需要指定版本pip install tensorflow-cpu==2.20.0
按照TensorFlow官方文档 Install TensorFlow 2 ,看起来直接使用 pip install tensorflow 就是安装CPU版本,如果要安装GPU版本则使用 pip install 'tensorflow[and-cuda]'
如果使用官方文档安装方法:
tensorflowsh -c "cd /home/${USER} && python3 -m venv venv/ml"
ource /home/${USER}/venv/ml/bin/activate && \
pip install --upgrade pip && \
pip install tensorflow
但是现在报错显示:
tensorflow 报错找不到匹配的发行版...
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)
ERROR: No matching distribution found for tensorflow
原因是 TensorFlow 二进制包 依赖 glibc ,在 Alpine Linux 上通常是采用 Docker 容器方式运行官方提供的镜像
如果要在Alpine Linux上原生运行TensorFlow,需要从源代码编译。 The simplest way to make Alpine TensorFlow work like it should 提到:
Alpine’s default musl libc often breaks TensorFlow wheels built for glibc-based distros. The fastest fix is to rebuild TensorFlow from source or use a pre-compiled musl-compatible wheel. Either choice yields predictable, repeatable behavior across clusters.