I. Environment Setup
1. Install cuda, I use the 12.1 version, download address:/cuda-12-1-1-download-archive
2. Install conda3./m0_73634846/article/details/136378350
3. Prepare the code environment
Original text:/s/PQVrlr5FoVb89Mivzi7pLA
Sequential execution:
git clone /Ucas-HaoranWei/GOT-OCR2.
conda create -n got python=3.10 -y #create virtual environment
conda activate got # activate virtual environment
pip install -e . # Install the dependencies using the project
Install Flash-Attention:
pip install ninja
pip install flash-attn --no-build-isolation
ps: the above is the official installation tutorial, but in fact, will encounter many problems, the following one by one instructions, not guaranteed to solve all the problems, but most of the problems can be solved!
II. Sequence of issues:
mountingcudaAn error message appears：AssertionError: Torch not compiled with CUDA enabled
/qq_44750293/article/details/129685556
The essence of the problem is pytorch can not call cuda, the reason is that the domestic acceleration mirror will be installed by default cpu version, install the gpu version can be, and .is_available output is false that is on behalf of the installation of the version for the cpu version, the installation of the gpu version can be output as true!
###1, .is_available() display false solution.
Use pytorch official recommended version to install:
/
For example, the install command for cuda12.1:
conda: conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
pip: pip install torch torchvision torchaudio
pip install torch torchvision torchaudio --index-url /whl/cu121 (works, this takes 2 hours + to download)
AliCloud Acceleration (using this is the fastest, recommended, only need to download 2min+):
pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 -f /pytorch-wheels/cu121
Encountering a deepspeed installation error
Python｜Windows Installation DeepSpeed installation method and error Unable to pre-compile async_io Handling.
/Changxing_J/article/details/139789110
Configuring visual studio
Run Anaconda Powershell Prompt, execute Set-Item Env:\DS_BUILD_OPS 0, switch to the depspeed directory, execute again : . \build_win.bat
pip error
ERROR: Could not find a version that satisfies the requirement setuptools_scm (from versions: none)
Try switchingpipTry the source.，pip install -i source address
conda error (in computing)
I activate anaconda in VS interface, but it reports error "CondaError: Run 'conda init' before 'conda activate'", how to solve it?
Open Anaconda Prompt
conda init
conda activate environment
Encountering a library error that depends on the C++ environment
[Error report] fatal error C1189
visual studio version is too high or too low cause, the current version of 2019 compatibility is relatively good
/thisjuly/article/details/136260069
flash_attn fails to execute after installation, essentially compiling the package incorrectly
text-generation-webui loading codellama reported error DLL load failed while importing flash_attn_2_cuda: The specified module could not be found. :
/dandandancpop/article/details/134729988
Solution: /Dao-AILab/flash-attention/releases , in this link to download and local cuda and torch and python corresponding to the version can be, cuda12.1 download cu123 pro test available, cp310 for python3.10, cp311 for python3.11 other The same reason
Large models cannot be started locally
Because the default source is hugging-face, but the domestic can not be accessed, the domestic can use modelscope (magic tower community) of the model
The download of hugging-face models did not work?
/p/722248326?utm_campaign=shareopn&utm_medium=social&utm_psn=1824225242303819776&utm_source=wechat_session
Use the model provided by modelscope
cudnn deployment
/retainenergy/article/details/126183711
cuDNN download link:/rdp/cudnn-archive I use 8.9.7, just configure it in the installation link!
Download links to the resources you need to use (only the ones that download very slowly have been uploaded):
Files shared via Netflix: GOT-OCR Large Model Deployment Related
Link./s/1s18O0kS3apBvs-L35KB-qw?pwd=y4ht Extract code: y4ht