wolfram commited on
Commit
d060218
1 Parent(s): 98b4b75

Update README.md

Browse files

Added IQ1_S, IQ2_XXS, IQ2_XS, IQ3_XXS, and IQ4_XS imatrix GGUF quants (20k_random_data/100 chunks)

Files changed (1) hide show
  1. README.md +18 -6
README.md CHANGED
@@ -18,9 +18,9 @@ tags:
18
  ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6303ca537373aacccd85d8a7/vmCAhJCpF0dITtCVxlYET.jpeg)
19
 
20
  - HF: [wolfram/miquliz-120b-v2.0](https://huggingface.co/wolfram/miquliz-120b-v2.0)
21
- - GGUF: Q2_K | IQ3_XXS | Q4_K_M | Q5_K_M
22
- - [dranger003's IQ2_XS | IQ2_XXS | IQ3_XXS | Q8_0](https://huggingface.co/dranger003/miquliz-120b-v2.0-iMat.GGUF)
23
- - [KnutJaegersberg's IQ2_XS](https://huggingface.co/KnutJaegersberg/2-bit-LLMs)
24
  - [mradermacher's i1-IQ1_S – i1-Q5_K_M](https://huggingface.co/mradermacher/miquliz-120b-v2.0-i1-GGUF)
25
  - [mradermacher's Q2_K – Q8_0](https://huggingface.co/mradermacher/miquliz-120b-v2.0-GGUF)
26
  - EXL2: [2.4bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-2.4bpw-h6-exl2) | [2.65bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-2.65bpw-h6-exl2) | [3.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-3.0bpw-h6-exl2) | [3.5bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-3.5bpw-h6-exl2) | [4.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-4.0bpw-h6-exl2) | [5.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-5.0bpw-h6-exl2)
@@ -35,11 +35,18 @@ Thanks for the additional quants, [DAN™](https://huggingface.co/dranger003), [
35
 
36
  Also available: [miqu-1-120b](https://huggingface.co/wolfram/miqu-1-120b) – Miquliz's older, purer sister; only Miqu, inflated to 120B.
37
 
38
- ### ❗ Q4_K_M and Q5_K_M files are split and require joining
39
 
40
- **Note:** HF does not support uploading files larger than 50GB. Therefore I have uploaded the Q4_K_M and Q5_K_M files as split files.
41
 
42
- <details><summary>Click for instructions regarding Q4_K_M and Q5_K_M files</summary>
 
 
 
 
 
 
 
43
 
44
  #### Q4_K_M
45
 
@@ -60,6 +67,8 @@ To join the files, do the following:
60
  Linux and macOS:
61
 
62
  ```sh
 
 
63
  cat miquliz-120b-v2.0.Q4_K_M.gguf-split-* > miquliz-120b-v2.0.Q4_K_M.gguf && rm miquliz-120b-v2.0.Q4_K_M.gguf-split-*
64
 
65
  cat miquliz-120b-v2.0.Q5_K_M.gguf-split-* > miquliz-120b-v2.0.Q5_K_M.gguf && rm miquliz-120b-v2.0.Q5_K_M.gguf-split-*
@@ -68,6 +77,9 @@ cat miquliz-120b-v2.0.Q5_K_M.gguf-split-* > miquliz-120b-v2.0.Q5_K_M.gguf && rm
68
  Windows command line:
69
 
70
  ```cmd
 
 
 
71
  COPY /B miquliz-120b-v2.0.Q4_K_M.gguf-split-a + miquliz-120b-v2.0.Q4_K_M.gguf-split-b miquliz-120b-v2.0.Q4_K_M.gguf
72
  DEL miquliz-120b-v2.0.Q4_K_M.gguf-split-a miquliz-120b-v2.0.Q4_K_M.gguf-split-b
73
 
 
18
  ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6303ca537373aacccd85d8a7/vmCAhJCpF0dITtCVxlYET.jpeg)
19
 
20
  - HF: [wolfram/miquliz-120b-v2.0](https://huggingface.co/wolfram/miquliz-120b-v2.0)
21
+ - GGUF: IQ1_S* | IQ2_XXS* | IQ2_XS* | Q2_K | IQ3_XXS* | IQ3_XXS.old | IQ4_XS* | Q4_K_M | Q5_K_M (*: imatrix=20k_random_data/100 chunks)
22
+ - [dranger003's IQ2_XS | IQ2_XXS | IQ3_XXS | Q8_0](https://huggingface.co/dranger003/miquliz-120b-v2.0-iMat.GGUF) (imatrix=100K wiki.train.raw/200 chunks)
23
+ - [KnutJaegersberg's IQ2_XS](https://huggingface.co/KnutJaegersberg/2-bit-LLMs) (imatrix=20k_random_data/100 chunks)
24
  - [mradermacher's i1-IQ1_S – i1-Q5_K_M](https://huggingface.co/mradermacher/miquliz-120b-v2.0-i1-GGUF)
25
  - [mradermacher's Q2_K – Q8_0](https://huggingface.co/mradermacher/miquliz-120b-v2.0-GGUF)
26
  - EXL2: [2.4bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-2.4bpw-h6-exl2) | [2.65bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-2.65bpw-h6-exl2) | [3.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-3.0bpw-h6-exl2) | [3.5bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-3.5bpw-h6-exl2) | [4.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-4.0bpw-h6-exl2) | [5.0bpw](https://huggingface.co/wolfram/miquliz-120b-v2.0-5.0bpw-h6-exl2)
 
35
 
36
  Also available: [miqu-1-120b](https://huggingface.co/wolfram/miqu-1-120b) – Miquliz's older, purer sister; only Miqu, inflated to 120B.
37
 
38
+ ### ❗ IQ4_XS, Q4_K_M, and Q5_K_M files are split and require joining
39
 
40
+ **Note:** HF does not support uploading files larger than 50GB. Therefore I have uploaded the IQ4_XS, Q4_K_M, and Q5_K_M files as split files.
41
 
42
+ <details><summary>Click for instructions regarding IQ4_XS, Q4_K_M, and Q5_K_M files</summary>
43
+
44
+ #### IQ4_XS
45
+
46
+ Please download:
47
+
48
+ - `miquliz-120b-v2.0.IQ4_XS.gguf-split-a`
49
+ - `miquliz-120b-v2.0.IQ4_XS.gguf-split-b`
50
 
51
  #### Q4_K_M
52
 
 
67
  Linux and macOS:
68
 
69
  ```sh
70
+ cat miquliz-120b-v2.0.IQ4_XS.gguf-split-* > miquliz-120b-v2.0.IQ4_XS.gguf && rm miquliz-120b-v2.0.IQ4_XS.gguf-split-*
71
+
72
  cat miquliz-120b-v2.0.Q4_K_M.gguf-split-* > miquliz-120b-v2.0.Q4_K_M.gguf && rm miquliz-120b-v2.0.Q4_K_M.gguf-split-*
73
 
74
  cat miquliz-120b-v2.0.Q5_K_M.gguf-split-* > miquliz-120b-v2.0.Q5_K_M.gguf && rm miquliz-120b-v2.0.Q5_K_M.gguf-split-*
 
77
  Windows command line:
78
 
79
  ```cmd
80
+ COPY /B miquliz-120b-v2.0.IQ4_XS.gguf-split-a + miquliz-120b-v2.0.IQ4_XS.gguf-split-b miquliz-120b-v2.0.IQ4_XS.gguf
81
+ DEL miquliz-120b-v2.0.IQ4_XS.gguf-split-a miquliz-120b-v2.0.IQ4_XS.gguf-split-b
82
+
83
  COPY /B miquliz-120b-v2.0.Q4_K_M.gguf-split-a + miquliz-120b-v2.0.Q4_K_M.gguf-split-b miquliz-120b-v2.0.Q4_K_M.gguf
84
  DEL miquliz-120b-v2.0.Q4_K_M.gguf-split-a miquliz-120b-v2.0.Q4_K_M.gguf-split-b
85