Update README.md
Browse filesAdded IQ1_S, IQ2_XXS, IQ2_XS, IQ3_XXS, and IQ4_XS imatrix GGUF quants (20k_random_data/100 chunks)
README.md
CHANGED
@@ -18,9 +18,9 @@ tags:
|
|
18 |
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6303ca537373aacccd85d8a7/vmCAhJCpF0dITtCVxlYET.jpeg)
|
19 |
|
20 |
- HF: [wolfram/miquliz-120b-v2.0](https://huggingface.co/wolfram/miquliz-120b-v2.0)
|
21 |
-
- GGUF: Q2_K | IQ3_XXS | Q4_K_M | Q5_K_M
|
22 |
-
- [dranger003's IQ2_XS | IQ2_XXS | IQ3_XXS | Q8_0](https://huggingface.co/dranger003/miquliz-120b-v2.0-iMat.GGUF)
|
23 |
-
- [KnutJaegersberg's IQ2_XS](https://huggingface.co/KnutJaegersberg/2-bit-LLMs)
|
24 |
- [mradermacher's i1-IQ1_S – i1-Q5_K_M](https://huggingface.co/mradermacher/miquliz-120b-v2.0-i1-GGUF)
|
25 |
- [mradermacher's Q2_K – Q8_0](https://huggingface.co/mradermacher/miquliz-120b-v2.0-GGUF)
|
26 |
- EXL2: [2.4bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-2.4bpw-h6-exl2) | [2.65bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-2.65bpw-h6-exl2) | [3.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-3.0bpw-h6-exl2) | [3.5bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-3.5bpw-h6-exl2) | [4.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-4.0bpw-h6-exl2) | [5.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-5.0bpw-h6-exl2)
|
@@ -35,11 +35,18 @@ Thanks for the additional quants, [DAN™](https://huggingface.co/dranger003), [
|
|
35 |
|
36 |
Also available: [miqu-1-120b](https://huggingface.co/wolfram/miqu-1-120b) – Miquliz's older, purer sister; only Miqu, inflated to 120B.
|
37 |
|
38 |
-
### ❗ Q4_K_M and Q5_K_M files are split and require joining
|
39 |
|
40 |
-
**Note:** HF does not support uploading files larger than 50GB. Therefore I have uploaded the Q4_K_M and Q5_K_M files as split files.
|
41 |
|
42 |
-
<details><summary>Click for instructions regarding Q4_K_M and Q5_K_M files</summary>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
43 |
|
44 |
#### Q4_K_M
|
45 |
|
@@ -60,6 +67,8 @@ To join the files, do the following:
|
|
60 |
Linux and macOS:
|
61 |
|
62 |
```sh
|
|
|
|
|
63 |
cat miquliz-120b-v2.0.Q4_K_M.gguf-split-* > miquliz-120b-v2.0.Q4_K_M.gguf && rm miquliz-120b-v2.0.Q4_K_M.gguf-split-*
|
64 |
|
65 |
cat miquliz-120b-v2.0.Q5_K_M.gguf-split-* > miquliz-120b-v2.0.Q5_K_M.gguf && rm miquliz-120b-v2.0.Q5_K_M.gguf-split-*
|
@@ -68,6 +77,9 @@ cat miquliz-120b-v2.0.Q5_K_M.gguf-split-* > miquliz-120b-v2.0.Q5_K_M.gguf && rm
|
|
68 |
Windows command line:
|
69 |
|
70 |
```cmd
|
|
|
|
|
|
|
71 |
COPY /B miquliz-120b-v2.0.Q4_K_M.gguf-split-a + miquliz-120b-v2.0.Q4_K_M.gguf-split-b miquliz-120b-v2.0.Q4_K_M.gguf
|
72 |
DEL miquliz-120b-v2.0.Q4_K_M.gguf-split-a miquliz-120b-v2.0.Q4_K_M.gguf-split-b
|
73 |
|
|
|
18 |
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6303ca537373aacccd85d8a7/vmCAhJCpF0dITtCVxlYET.jpeg)
|
19 |
|
20 |
- HF: [wolfram/miquliz-120b-v2.0](https://huggingface.co/wolfram/miquliz-120b-v2.0)
|
21 |
+
- GGUF: IQ1_S* | IQ2_XXS* | IQ2_XS* | Q2_K | IQ3_XXS* | IQ3_XXS.old | IQ4_XS* | Q4_K_M | Q5_K_M (*: imatrix=20k_random_data/100 chunks)
|
22 |
+
- [dranger003's IQ2_XS | IQ2_XXS | IQ3_XXS | Q8_0](https://huggingface.co/dranger003/miquliz-120b-v2.0-iMat.GGUF) (imatrix=100K wiki.train.raw/200 chunks)
|
23 |
+
- [KnutJaegersberg's IQ2_XS](https://huggingface.co/KnutJaegersberg/2-bit-LLMs) (imatrix=20k_random_data/100 chunks)
|
24 |
- [mradermacher's i1-IQ1_S – i1-Q5_K_M](https://huggingface.co/mradermacher/miquliz-120b-v2.0-i1-GGUF)
|
25 |
- [mradermacher's Q2_K – Q8_0](https://huggingface.co/mradermacher/miquliz-120b-v2.0-GGUF)
|
26 |
- EXL2: [2.4bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-2.4bpw-h6-exl2) | [2.65bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-2.65bpw-h6-exl2) | [3.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-3.0bpw-h6-exl2) | [3.5bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-3.5bpw-h6-exl2) | [4.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-4.0bpw-h6-exl2) | [5.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-5.0bpw-h6-exl2)
|
|
|
35 |
|
36 |
Also available: [miqu-1-120b](https://huggingface.co/wolfram/miqu-1-120b) – Miquliz's older, purer sister; only Miqu, inflated to 120B.
|
37 |
|
38 |
+
### ❗ IQ4_XS, Q4_K_M, and Q5_K_M files are split and require joining
|
39 |
|
40 |
+
**Note:** HF does not support uploading files larger than 50GB. Therefore I have uploaded the IQ4_XS, Q4_K_M, and Q5_K_M files as split files.
|
41 |
|
42 |
+
<details><summary>Click for instructions regarding IQ4_XS, Q4_K_M, and Q5_K_M files</summary>
|
43 |
+
|
44 |
+
#### IQ4_XS
|
45 |
+
|
46 |
+
Please download:
|
47 |
+
|
48 |
+
- `miquliz-120b-v2.0.IQ4_XS.gguf-split-a`
|
49 |
+
- `miquliz-120b-v2.0.IQ4_XS.gguf-split-b`
|
50 |
|
51 |
#### Q4_K_M
|
52 |
|
|
|
67 |
Linux and macOS:
|
68 |
|
69 |
```sh
|
70 |
+
cat miquliz-120b-v2.0.IQ4_XS.gguf-split-* > miquliz-120b-v2.0.IQ4_XS.gguf && rm miquliz-120b-v2.0.IQ4_XS.gguf-split-*
|
71 |
+
|
72 |
cat miquliz-120b-v2.0.Q4_K_M.gguf-split-* > miquliz-120b-v2.0.Q4_K_M.gguf && rm miquliz-120b-v2.0.Q4_K_M.gguf-split-*
|
73 |
|
74 |
cat miquliz-120b-v2.0.Q5_K_M.gguf-split-* > miquliz-120b-v2.0.Q5_K_M.gguf && rm miquliz-120b-v2.0.Q5_K_M.gguf-split-*
|
|
|
77 |
Windows command line:
|
78 |
|
79 |
```cmd
|
80 |
+
COPY /B miquliz-120b-v2.0.IQ4_XS.gguf-split-a + miquliz-120b-v2.0.IQ4_XS.gguf-split-b miquliz-120b-v2.0.IQ4_XS.gguf
|
81 |
+
DEL miquliz-120b-v2.0.IQ4_XS.gguf-split-a miquliz-120b-v2.0.IQ4_XS.gguf-split-b
|
82 |
+
|
83 |
COPY /B miquliz-120b-v2.0.Q4_K_M.gguf-split-a + miquliz-120b-v2.0.Q4_K_M.gguf-split-b miquliz-120b-v2.0.Q4_K_M.gguf
|
84 |
DEL miquliz-120b-v2.0.Q4_K_M.gguf-split-a miquliz-120b-v2.0.Q4_K_M.gguf-split-b
|
85 |
|