Joseph717171 commited on
Commit
38a9b34
·
verified ·
1 Parent(s): 1cc6b00

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -2
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  language:
3
  - en
4
- author: froggeric (https://huggingface.co/datasets/froggeric/imatrix/edit/main/README.md)
5
  ---
6
  # All credit for this wonderful Repo Card detailing and explaining the similarities and differences of computed imatrices and detailing and explaining the differences, similarities, and, highlighted significances of training datasets and their purported purposes for particular large language models, goes to froggeric.
7
 
@@ -9,7 +9,7 @@ author: froggeric (https://huggingface.co/datasets/froggeric/imatrix/edit/main/R
9
 
10
  # Note: Imatrices uploaded to this repo follow the following naming convention: model-name_training-dataset.imatrix (hyphens are purely used in this example to enhance readability...)
11
 
12
- # Just download the imatrix for your chosen LLM (Large Language Model), and quantize to your preferred QuantType. (Note the following example already assumes you converted your model to GGUF)
13
  ```
14
  llama.cpp % ./quantize --imatrix path_to_imatrix path_to_model/ggml-model-f16.gguf model_name-QuantType.gguf QuantType
15
  ```
@@ -111,6 +111,11 @@ Small Wikipedia dump. Unclean, contains many unwanted tags.
111
  exllamav2 calibration data taken from:\
112
  https://github.com/turboderp/exllamav2/tree/master/conversion/standard_cal_data
113
 
 
 
 
 
 
114
  ## How to quantize using an imatrix, with llama.cpp
115
 
116
  1. Get one of the input files collected here, or elsewhere.
 
1
  ---
2
  language:
3
  - en
4
+ author: Joseph717171 & froggeric (https://huggingface.co/datasets/froggeric/imatrix/edit/main/README.md)
5
  ---
6
  # All credit for this wonderful Repo Card detailing and explaining the similarities and differences of computed imatrices and detailing and explaining the differences, similarities, and, highlighted significances of training datasets and their purported purposes for particular large language models, goes to froggeric.
7
 
 
9
 
10
  # Note: Imatrices uploaded to this repo follow the following naming convention: model-name_training-dataset.imatrix (hyphens are purely used in this example to enhance readability...)
11
 
12
+ # Just download the imatrix for your chosen LLM (Large Language Model), and quantize to your preferred QuantType. (Note the following example already assumes you converted your model to GGUF - if you need detailed steps to convert your LLM to GGUF, [please scroll to the bottom of the page]() )
13
  ```
14
  llama.cpp % ./quantize --imatrix path_to_imatrix path_to_model/ggml-model-f16.gguf model_name-QuantType.gguf QuantType
15
  ```
 
111
  exllamav2 calibration data taken from:\
112
  https://github.com/turboderp/exllamav2/tree/master/conversion/standard_cal_data
113
 
114
+ ## How to convert (Supported) LLMs (Large Language Model) to GGUF format:
115
+ ```
116
+ llama.cpp % python convert.py path_to_model --outtype f16
117
+ ```
118
+
119
  ## How to quantize using an imatrix, with llama.cpp
120
 
121
  1. Get one of the input files collected here, or elsewhere.