Technical background
In some previous articles, we have introducedLoading DeepSeek Distillation Model on Linux Platform using Ollama，Deploy DeepSeek Local Model on Windows Platform using Ollama. In addition to using Ollama to interact with model files, you can also useandKTransformerThis is the case. There are also some client tools such as ChatBox, AnythingLLM and PageAssist.
Here we consider such a usage scenario, in our useollama pullOr after downloading a local model from ModelScope, Hugging Face and other platforms, for example, it has been loaded into Ollama, how can I export this model to other hardware?
Ollama Model Path
If you do not manually configure the Model path of Ollama, the default Ollama model path on Linux is:
/usr/share/ollama/.ollama/models/
It is a folder that requires root permissions. On Windows platforms, this path is generally:
C:\Users\user_name\.ollama\models
Ollama path meaning
There are two subdirectories under the models path:
drwxr-xr-x 2 ollama ollama 4096 February 20 14:47 blobs/
drwxr-xr-x 3 ollama ollama 4096 February 20 14:53 manifests/inblobsIt contains a series of model files,sha256Start with a long list of hashes:
$ ll blobs/
Total dosage 106751780
drwxr-xr-x 2 ollama ollama 4096 February 20 14:47 ./
drwxr-xr-x 4 ollama ollama 4096 Feb 5 12:24 ../
-rw-r--r-- 1 ollama ollama 412 February 14 15:43 sha256-066df13e4debff21aeb0bb1ce8db63b9f19498d06b477ba3a4fa066adafd2949
-rwxr-xr-x 1 root root 7712 February 20 14:46 sha256-0b4284c1f87029e67654c7953afa16279961632cf73dcfe33374c4c2f298fa35
-rwxr-xr-x 1 root root 5963057248 February 20 14:46 sha256-11f274007f093fefeec994a5dbbb33d0733a4feb87f7ab66dcd7c1069fef0068These files are actually the GGUF files of the model, and can be used directly with the suffix of gguf. But the key is that sometimes a model may correspond to multiple GGUF files, so an index file is needed.manifestsThe directory stores the index of each model:
$ tree manifests/
manifests/
└──
└── library
├── deepseek-r1
│ ├── 14b
│ ├── 32b
│ ├── 32b-q2k
│ ├── 32b-q40
│ └── 70b-q2k
├── llama3-vision
│ └── 11b
└── nomic-embed-text-v1.5
└── latest
5 directories, 7 files
This is a basic structure of Ollama model storage.
Find the corresponding model file
In fact, the index file in the previous chapter clearly contains the comparison path of the model file, such as opening an llama3 index file:
{"schemaVersion":2,"mediaType":"application/.v2+json","config":{"mediaType":"application/.v1+json","digest":"sha256:fbd313562bb706ac00f1a18c0aad8398b3c22d5cd78c47ff6f7246c4c3438576","size":572},"layers":[{"mediaType":"application/","digest":"sha256:11f274007f093fefeec994a5dbbb33d0733a4feb87f7ab66dcd7c1069fef0068","size":5963057248},{"mediaType":"application/","digest":"sha256:ece5e659647a20a5c28ab9eea1c12a1ad430bc0f2a27021d00ad103b3bf5206f","size":1938763584,"from":"/Users/ollama/.ollama/models/blobs/sha256-ece5e659647a20a5c28ab9eea1c12a1ad430bc0f2a27021d00ad103b3bf5206f"},{"mediaType":"application/","digest":"sha256:715415638c9c4c0cb2b78783da041b97bd1205f8b9f9494bd7e5a850cb443602","size":269},{"mediaType":"application/","digest":"sha256:0b4284c1f87029e67654c7953afa16279961632cf73dcfe33374c4c2f298fa35","size":7712},{"mediaType":"application/","digest":"sha256:fefc914e46e6024467471837a48a24251db2c6f3f58395943da7bf9dc6f70fb6","size":32}]}
The ones with sha256 here are the gguf files that are required for the model. Compared with this method of directly opening the index file to retrieve, Ollama actually has a more elegant way to view the model storage address:
$ ollama show --modelfile llama3-vision:11b
# Modelfile generated by "ollama show"
# To build a new Modelfile based on this, replace FROM with:
# FROM llama3-vision:11b
FROM /usr/share/ollama/.ollama/models/blobs/sha256-11f274007f093fefeec994a5dbbb33d0733a4feb87f7ab66dcd7c1069fef0068
FROM /usr/share/ollama/.ollama/models/blobs/sha256-ece5e659647a20a5c28ab9eea1c12a1ad430bc0f2a27021d00ad103b3bf5206f
TEMPLATE """{{- range $index, $_ := .Messages }}<|start_header_id|>{{ .Role }}<|end_header_id|>
{{ .Content }}
{{- if gt (len (slice $.Messages $index)) 1 }}<|eot_id|>
{{- else if ne .Role "assistant" }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{{ end }}
{{- end }}"""
PARAMETER temperature 0.6
PARAMETER top_p 0.9
LICENSE "LLAMA 3.2 COMMUNITY LICENSE AGREEMENT
Llama 3.2 Version Release Date: September 25, 2024
“Agreement” means the terms and conditions for use, reproduction, distribution
and modification of the Llama Materials set forth herein.
“Documentation” means the specifications, manuals and documentation accompanying Llama 3.2
distributed by Meta at /doc/overview.
“Licensee” or “you” means you, or your employer or any other person or entity (if you are
entering into this Agreement on such person or entity’s behalf), of the age required under
applicable laws, rules or regulations to provide legal consent and that has legal authority
to bind your employer or such other person or entity if you are entering in this Agreement
on their behalf.
“Llama 3.2” means the foundational large language models and software and algorithms, including
machine-learning model code, trained model weights, inference-enabling code, training-enabling code,
fine-tuning enabling code and other elements of the foregoing distributed by Meta at
/llama-downloads.
“Llama Materials” means, collectively, Meta’s proprietary Llama 3.2 and Documentation (and
any portion thereof) made available under this Agreement.
“Meta” or “we” means Meta Platforms Ireland Limited (if you are located in or,
if you are an entity, your principal place of business is in the EEA or Switzerland)
and Meta Platforms, Inc. (if you are located outside of the EEA or Switzerland).
By clicking “I Accept” below or by using or distributing any portion or element of the Llama Materials,
you agree to be bound by this Agreement.
1. License Rights and Redistribution.
a. Grant of Rights. You are granted a non-exclusive, worldwide,
non-transferable and royalty-free limited license under Meta’s intellectual property or other rights
owned by Meta embodied in the Llama Materials to use, reproduce, distribute, copy, create derivative works
of, and make modifications to the Llama Materials.
b. Redistribution and Use.
i. If you distribute or make available the Llama Materials (or any derivative works thereof),
or a product or service (including another AI model) that contains any of them, you shall (A) provide
a copy of this Agreement with any such Llama Materials; and (B) prominently display “Built with Llama”
on a related website, user interface, blogpost, about page, or product documentation. If you use the
Llama Materials or any outputs or results of the Llama Materials to create, train, fine tune, or
otherwise improve an AI model, which is distributed or made available, you shall also include “Llama”
at the beginning of any such AI model name.
ii. If you receive Llama Materials, or any derivative works thereof, from a Licensee as part
of an integrated end user product, then Section 2 of this Agreement will not apply to you.
iii. You must retain in all copies of the Llama Materials that you distribute the
following attribution notice within a “Notice” text file distributed as a part of such copies:
“Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms,
Inc. All Rights Reserved.”
iv. Your use of the Llama Materials must comply with applicable laws and regulations
(including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for
the Llama Materials (available at /llama3_2/use-policy), which is hereby
incorporated by reference into this Agreement.
2. Additional Commercial Terms. If, on the Llama 3.2 version release date, the monthly active users
of the products or services made available by or for Licensee, or Licensee’s affiliates,
is greater than 700 million monthly active users in the preceding calendar month, you must request
a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to
exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights.
3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE LLAMA MATERIALS AND ANY OUTPUT AND
RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND META DISCLAIMS
ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES
OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE
FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE LLAMA MATERIALS AND ASSUME ANY RISKS ASSOCIATED
WITH YOUR USE OF THE LLAMA MATERIALS AND ANY OUTPUT AND RESULTS.
4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT,
FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN
IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
5. Intellectual Property.
a. No trademark licenses are granted under this Agreement, and in connection with the Llama Materials,
neither Meta nor Licensee may use any name or mark owned by or associated with the other or any of its affiliates,
except as required for reasonable and customary use in describing and redistributing the Llama Materials or as
set forth in this Section 5(a). Meta hereby grants you a license to use “Llama” (the “Mark”) solely as required
to comply with the last sentence of Section . You will comply with Meta’s brand guidelines (currently accessible
at /brand/resources/meta/company-brand/). All goodwill arising out of your use of the Mark
will inure to the benefit of Meta.
b. Subject to Meta’s ownership of Llama Materials and derivatives made by or for Meta, with respect to any
derivative works and modifications of the Llama Materials that are made by you, as between you and Meta,
you are and will be the owner of such derivative works and modifications.
c. If you institute litigation or other proceedings against Meta or any entity (including a cross-claim or
counterclaim in a lawsuit) alleging that the Llama Materials or Llama 3.2 outputs or results, or any portion
of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable
by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or
claim is filed or instituted. You will indemnify and hold harmless Meta from and against any claim by any third
party arising out of or related to your use or distribution of the Llama Materials.
6. Term and Termination. The term of this Agreement will commence upon your acceptance of this Agreement or access
to the Llama Materials and will continue in full force and effect until terminated in accordance with the terms
and conditions herein. Meta may terminate this Agreement if you are in breach of any term or condition of this
Agreement. Upon termination of this Agreement, you shall delete and cease use of the Llama Materials. Sections 3,
4 and 7 shall survive the termination of this Agreement.
7. Governing Law and Jurisdiction. This Agreement will be governed and construed under the laws of the State of
California without regard to choice of law principles, and the UN Convention on Contracts for the International
Sale of Goods does not apply to this Agreement. The courts of California shall have exclusive jurisdiction of
any dispute arising out of this Agreement. "
Use this show instruction to view the model storage address. In addition to the model address, there is also the entire LICENSE. But here we also found a very important issue:The GGUF model files listed through the ollama show are incomplete! ! !. Therefore, it is recommended to open the configuration file in manifests directly to view it, and then migrate the relevant model files.
Model migration and loading
After understanding the storage structure of Ollama's model file before, we need to migrate the model and the idea is clear: just copy the model file and index file to the corresponding path. For example, copy the gguf file of sha256 to the target device under the blobs path, and then copy the index file to the path of manifests according to the directory structure of manifests. If it is a Linux operating system, the following structure is fixed:
manifests/
└──
└── library
Then create a directory named model name under library, e.g.model-name/, and then put the configuration fileconfigCopy to this path, the instructions you need to use when loading the model are:
$ ollama run model-name:config
It should be noted that when you first migrate the model, you may not be able to find the local model you just migrated by using PageAssist and other tools. You need to firstollama runOnly once can you find it in the model list:
$ ollama list
NAME ID SIZE MODIFIED
llama3-vision:11b 085a1fdae525 7.9 GB About an hour ago
At this point, Ollama's model migration is completed.
Summary
To facilitate local big model deployment and migration, this article provides a method for model local migration of Ollama. Since the models downloaded directly from Ollama Hub, or the larger GGUF model file, are often divided into multiples, and the file name is sha256 hash transformation in the Ollama path. Therefore, we need to obtain the corresponding file name from the index file and then perform local model migration.
Copyright Statement
The first link to this article is:/dechinphy/p/
Author ID: DechinPhy
More original articles:/dechinphy/
Please ask the blogger to have coffee:/dechinphy/gallery/image/