Joseph717171
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1 |
---
|
2 |
language:
|
3 |
- en
|
4 |
-
author: froggeric (https://huggingface.co/datasets/froggeric/imatrix/edit/main/README.md)
|
5 |
---
|
6 |
# All credit for this wonderful Repo Card detailing and explaining the similarities and differences of computed imatrices and detailing and explaining the differences, similarities, and, highlighted significances of training datasets and their purported purposes for particular large language models, goes to froggeric.
|
7 |
|
@@ -9,7 +9,7 @@ author: froggeric (https://huggingface.co/datasets/froggeric/imatrix/edit/main/R
|
|
9 |
|
10 |
# Note: Imatrices uploaded to this repo follow the following naming convention: model-name_training-dataset.imatrix (hyphens are purely used in this example to enhance readability...)
|
11 |
|
12 |
-
# Just download the imatrix for your chosen LLM (Large Language Model), and quantize to your preferred QuantType. (Note the following example already assumes you converted your model to GGUF)
|
13 |
```
|
14 |
llama.cpp % ./quantize --imatrix path_to_imatrix path_to_model/ggml-model-f16.gguf model_name-QuantType.gguf QuantType
|
15 |
```
|
@@ -111,6 +111,11 @@ Small Wikipedia dump. Unclean, contains many unwanted tags.
|
|
111 |
exllamav2 calibration data taken from:\
|
112 |
https://github.com/turboderp/exllamav2/tree/master/conversion/standard_cal_data
|
113 |
|
|
|
|
|
|
|
|
|
|
|
114 |
## How to quantize using an imatrix, with llama.cpp
|
115 |
|
116 |
1. Get one of the input files collected here, or elsewhere.
|
|
|
1 |
---
|
2 |
language:
|
3 |
- en
|
4 |
+
author: Joseph717171 & froggeric (https://huggingface.co/datasets/froggeric/imatrix/edit/main/README.md)
|
5 |
---
|
6 |
# All credit for this wonderful Repo Card detailing and explaining the similarities and differences of computed imatrices and detailing and explaining the differences, similarities, and, highlighted significances of training datasets and their purported purposes for particular large language models, goes to froggeric.
|
7 |
|
|
|
9 |
|
10 |
# Note: Imatrices uploaded to this repo follow the following naming convention: model-name_training-dataset.imatrix (hyphens are purely used in this example to enhance readability...)
|
11 |
|
12 |
+
# Just download the imatrix for your chosen LLM (Large Language Model), and quantize to your preferred QuantType. (Note the following example already assumes you converted your model to GGUF - if you need detailed steps to convert your LLM to GGUF, [please scroll to the bottom of the page]() )
|
13 |
```
|
14 |
llama.cpp % ./quantize --imatrix path_to_imatrix path_to_model/ggml-model-f16.gguf model_name-QuantType.gguf QuantType
|
15 |
```
|
|
|
111 |
exllamav2 calibration data taken from:\
|
112 |
https://github.com/turboderp/exllamav2/tree/master/conversion/standard_cal_data
|
113 |
|
114 |
+
## How to convert (Supported) LLMs (Large Language Model) to GGUF format:
|
115 |
+
```
|
116 |
+
llama.cpp % python convert.py path_to_model --outtype f16
|
117 |
+
```
|
118 |
+
|
119 |
## How to quantize using an imatrix, with llama.cpp
|
120 |
|
121 |
1. Get one of the input files collected here, or elsewhere.
|