Background to the issue
Although the GPU hardware backend is not supported in official releases after MindSpore-2.3, there is actually support for the GPU backend in the development branch versions:
However, you may encounter some problems or report errors during the installation process, here is a reproduction of the installation process in my Ubuntu-20.04 environment.
Pip Installation
The basic installation process goes like this, first create a python-3.9 virtual environment using anaconda, since python-3.7 is no longer supported after MindSpore-2.4 version:
$ conda create -name mindspore-master python=3.9
Then, depending on your local environment, execute the appropriate pip install command, for example:
$ python3 -m pip install mindspore-dev -i /simple
If there is a timeout problem during pip installation, just re-execute the above process again. After installation, execute the following command to verify the installed MindSpore:
$ python -c "import mindspore;mindspore.set_context(device_target='GPU');mindspore.run_check()"
What follows is a moment to deal with various issues.
version XXX not found
The first type of problem that can arise is a mismatch between the versions of various compilation tools, for example:
$ python -c "import mindspore;mindspore.set_context(device_target='GPU');mindspore.run_check()"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/__init__.py", line 18, in <module>
from mindspore.run_check import run_check
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/run_check/__init__.py", line 17, in <module>
from ._check_version import check_version_and_env_config
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/run_check/_check_version.py", line 28, in <module>
from mindspore._c_expression import MSContext, ms_ctx_param
ImportError: /home/dechin/anaconda3/envs/mindspore-master/bin/../lib/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/_c_expression.cpython-39-x86_64)
In this case it just can't be found.CXXABI_1.3.8This software version. But if you check the software version inside the system:
$ strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep CXXABI
CXXABI_1.3
CXXABI_1.3.1
CXXABI_1.3.2
CXXABI_1.3.3
CXXABI_1.3.4
CXXABI_1.3.5
CXXABI_1.3.6
CXXABI_1.3.7
CXXABI_1.3.8
CXXABI_1.3.9
CXXABI_1.3.10
CXXABI_1.3.11
CXXABI_1.3.12
CXXABI_TM_1
CXXABI_FLOAT128
We foundCXXABI_1.3.8is present, and the reason for this error is that the version does not exist in this mindspore virtual environment created by anaconda:
$ strings /home/dechin/anaconda3/envs/mindspore-master/lib/libstdc++.so.6 | grep CXXABICXXABI_1.3
CXXABI_1.3.1
CXXABI_1.3.2
CXXABI_1.3.3
CXXABI_1.3.4
CXXABI_1.3.5
CXXABI_1.3.6
CXXABI_1.3.7
CXXABI_TM_1
Then the solution is this, we can directly make a soft connection to this dynamic link library under the virtual environment of mindspore, and link it to the corresponding dynamic link library inside the system library:
$ ln -sf /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /home/dechin/anaconda3/envs/mindspore-master/lib/libstdc++.so.6
Rerunning it again solves the current problem, and similar errors are reported:
$ python3 -c "import mindspore;mindspore.set_context(device_target='GPU');mindspore.run_check()"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/__init__.py", line 18, in <module>
from mindspore.run_check import run_check
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/run_check/__init__.py", line 17, in <module>
from ._check_version import check_version_and_env_config
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/run_check/_check_version.py", line 28, in <module>
from mindspore._c_expression import MSContext, ms_ctx_param
ImportError: /home/dechin/anaconda3/envs/mindspore-master/bin/../lib/.1: version `GOMP_4.0' not found (required by /home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/lib/libmindspore_backend.so)
It can be handled in the same way.
cannot open shared object file
After configuring the above environment, there is still a possibility of this error message:
$ python3 -c "import mindspore;mindspore.set_context(device_target='GPU');mindspore.run_check()"
[WARNING] ME(232647,7ff51906b4c0,python3):2024-11-18-09:54:31.123.673 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:65] GetNvccRealPath] Invalid environment variable CUDA_HOME [/home], can not find nvcc file [/home/bin/nvcc], please check the CUDA_HOME.
/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/train/metrics/hausdorff_distance.py:20: UserWarning: A NumPy version >=1.22.4 and <2.3.0 is required for this version of SciPy (detected version 1.22.3)
from import morphology
[ERROR] ME(232647:140690663584960,MainProcess):2024-11-18-09:54:32.148.524 [mindspore/run_check/_check_version.py:218] (need by mindspore-gpu) is not found. Please confirm that libmindspore_gpu.so is in directory:/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/run_check/../lib/plugin and the correct cuda version has been installed, you can refer to the installation guidelines: /install
[ERROR] ME(232647:140690663584960,MainProcess):2024-11-18-09:54:32.148.726 [mindspore/run_check/_check_version.py:218] (need by mindspore-gpu) is not found. Please confirm that libmindspore_gpu.so is in directory:/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/run_check/../lib/plugin and the correct cuda version has been installed, you can refer to the installation guidelines: /install
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/_checkparam.py", line 1367, in wrapper
return func(*args, **kwargs)
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/", line 1861, in set_context
ctx.set_device_target(kwargs['device_target'])
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/", line 495, in set_device_target
self.set_param(ms_ctx_param.device_target, target)
File "/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/", line 187, in set_param
self._context_handle.set_param(param, value)
RuntimeError: Unsupported device target GPU. This process only supports one of the ['CPU']. Please check whether the GPU environment is installed and configured correctly, and check whether current mindspore wheel package was built with "-e GPU". For details, please refer to "Device load error message".
----------------------------------------------------
- Device load error message:
----------------------------------------------------
Load dynamic library: libmindspore_ascend.so.2 failed. libge_runner.so: cannot open shared object file: No such file or directory
Load dynamic library: libmindspore_gpu.so.11.6 failed. .11: cannot open shared object file: No such file or directory
Load dynamic library: libmindspore_gpu.so.11.1 failed. .11: cannot open shared object file: No such file or directory
Load dynamic library: libmindspore_gpu.so.10.1 failed. .7: cannot open shared object file: No such file or directory
----------------------------------------------------
- C++ Call Stack: (For framework developers)
----------------------------------------------------
mindspore/core/utils/ms_context.cc:287 SetDeviceTargetFromInner
The hint here is that it can't be foundlibmindspore_gpu.so.11.6And so on the address of the dynamic link libraries. Then the solution is this, we first go to the system to search for these libraries, if there is a corresponding version number, we put the location of thelibThe path is configured to theLD_LIBRARY_PATHThe center will be fine:
$ sudo find / -name *
/home/dechin/anaconda3/envs/mindspore-latest/lib/
/home/dechin/anaconda3/envs/mindspore-latest/lib/.11.3.0.106
/home/dechin/anaconda3/envs/mindspore-latest/lib/.11
/home/dechin/anaconda3/envs/mindsponge/lib/
/home/dechin/anaconda3/envs/mindsponge/lib/.11.3.0.106
/home/dechin/anaconda3/envs/mindsponge/lib/.11
/home/dechin/anaconda3/envs/mindspore-master/lib/
/home/dechin/anaconda3/envs/mindspore-master/lib/.10
/home/dechin/anaconda3/envs/mindspore-master/lib/.10.2.2.89
/usr/lib/x86_64-linux-gnu/.10.2.1.243
/usr/lib/x86_64-linux-gnu/.10.1.0.105
/usr/lib/x86_64-linux-gnu/stubs/
/usr/lib/x86_64-linux-gnu/
/usr/lib/x86_64-linux-gnu/.10
Here we find that in our newly createdmindspore-masterenvironment did not have the corresponding versions of the DLLs, but instead the old version of the mindspore environment had the corresponding versions of these DLLs, so my solution was to put the old version of the mindspore environment in thelibConfiguring it to an environment variable solves the problem:
$ export LD_LIBRARY_PATH=/home/dechin/anaconda3/envs/mindspore-master/lib:/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/lib:/home/dechin/anaconda3/envs/mindsponge/lib
Run the test again:
$ python3 -c "import mindspore;mindspore.set_context(device_target='GPU');mindspore.run_check()"
[WARNING] ME(232736,7f562eca06c0,python3):2024-11-18-09:55:58.717.253 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:65] GetNvccRealPath] Invalid environment variable CUDA_HOME [/home], can not find nvcc file [/home/bin/nvcc], please check the CUDA_HOME.
/home/dechin/anaconda3/envs/mindspore-master/lib/python3.9/site-packages/mindspore/train/metrics/hausdorff_distance.py:20: UserWarning: A NumPy version >=1.22.4 and <2.3.0 is required for this version of SciPy (detected version 1.22.3)
from import morphology
MindSpore version: 2.4.0.dev20241103
The result of multiplication calculation is correct, MindSpore has been installed on platform [GPU] successfully!
As you can see, although there are some warning messages, the final run result is correct and can be run if you need to ignore the warning messages:
$ export GLOG_v=4
to configure the mindspore logging level.
One problem here is that if the user's environment does not have an older version of MindSpore installed. then I personally think that a more convenient option is if the system environment has otherlibcublasIf you have a framework environment, such as Jax or Torch, there are versions of these related software, so you can configure the path to them directly into the environment variable. If you don't have any, my suggestion is to create a virtual environment and install an old version of MindSpore, for examplemindspore-gpu-2.2After you have ensured a successful installation, configure the path to this old version of lib into the environment variable under the new version.
Unsupported device target GPU
If there is aUnsupported device target GPUand automatically indexes the Ascend backend libraries, which happens because there is no configuration of theCUDA_HOMEThis environment variable. It should be, the new version of mindspore underlying logic to determine the hardware platform is by getting the environment variable, so you need to manually configure aCUDA_HOMEparameter will do, for example:
$ export CUDA_HOME=/home
Although this arbitrary configuration may lead to some warning messages, it does not affect the results of the correct operation of the program.
Summary outline
This article describes how to install the latest MindSpore-2.4-for-GPU version on an Ubuntu-20.04 system, and some of the issues that may arise during the installation process. Although the GPU hardware backend is no longer supported in the official version of MindSpore, the development version continues to be supported and includes new features from versions 2.3 and 2.4, but no updates or optimizations at the arithmetic level. This is good news for MindSpore users with GPU backends.
copyright statement
This article was first linked to:/dechinphy/p/
Author ID: DechinPhy
More original articles:/dechinphy/
Buy the blogger coffee:/dechinphy/gallery/image/
Reference Links
- /install/