sophosympatheia
commited on
Commit
•
8bc751c
1
Parent(s):
047ab85
Update README.md
Browse filesAdded links to more quants
README.md
CHANGED
@@ -165,11 +165,18 @@ You could also try ChatML.
|
|
165 |
```
|
166 |
|
167 |
### Quantizations
|
168 |
-
*
|
169 |
-
* [
|
170 |
-
* [
|
171 |
-
*
|
172 |
-
* [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
173 |
* If you don't see something you're looking for, [try searching Hugging Face](https://huggingface.co/models?search=midnight-miqu-70b). There may be newer quants available than what I've documented here.
|
174 |
|
175 |
### Licence and usage restrictions
|
|
|
165 |
```
|
166 |
|
167 |
### Quantizations
|
168 |
+
* GGUF
|
169 |
+
* [ooooz/midnight-miqu-70b-v1.0-GGUF](https://huggingface.co/ooooz/midnight-miqu-70b-v1.0-GGUF/tree/main) -- Various GGUF quants
|
170 |
+
* [mradermacher/Midnight-Miqu-70B-v1.0-GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.0-GGUF) -- Q4_K_M quant so far, maybe more to come
|
171 |
+
* GPTQ
|
172 |
+
* [Kotokin/sophosympatheia_Midnight-Miqu-70B-v1.0_GPTQ32G](https://huggingface.co/Kotokin/sophosympatheia_Midnight-Miqu-70B-v1.0_GPTQ32G) -- 4-bit 32g GPTQ quant
|
173 |
+
* Exllama2
|
174 |
+
* 2.24bpw: [Dracones/Midnight-Miqu-70B-v1.0_exl2_2.24bpw](https://huggingface.co/Dracones/Midnight-Miqu-70B-v1.0_exl2_2.24bpw)
|
175 |
+
* 3.0bpw: [Dracones/Midnight-Miqu-70B-v1.0_exl2_3.0bpw](https://huggingface.co/Dracones/Midnight-Miqu-70B-v1.0_exl2_3.0bpw)
|
176 |
+
* 3.75bpw: [altomek/Midnight-Miqu-70B-v1.0-3.75bpw-EXL2](https://huggingface.co/altomek/Midnight-Miqu-70B-v1.0-3.75bpw-EXL2)
|
177 |
+
* 4.0bpw: [Dracones/Midnight-Miqu-70B-v1.0_exl2_4.0bpw](https://huggingface.co/Dracones/Midnight-Miqu-70B-v1.0_exl2_4.0bpw)
|
178 |
+
* 4.65bpw: [Dracones/Midnight-Miqu-70B-v1.0_exl2_4.65bpw](https://huggingface.co/Dracones/Midnight-Miqu-70B-v1.0_exl2_4.65bpw)
|
179 |
+
* 5.0bpw: [Dracones/Midnight-Miqu-70B-v1.0_exl2_5.0bpw](https://huggingface.co/Dracones/Midnight-Miqu-70B-v1.0_exl2_5.0bpw)
|
180 |
* If you don't see something you're looking for, [try searching Hugging Face](https://huggingface.co/models?search=midnight-miqu-70b). There may be newer quants available than what I've documented here.
|
181 |
|
182 |
### Licence and usage restrictions
|