martinkozle
commited on
Commit
•
79fdfaa
1
Parent(s):
a3d3ae3
Update README.md
Browse files
README.md
CHANGED
@@ -5,6 +5,7 @@ language:
|
|
5 |
- en
|
6 |
tags:
|
7 |
- axolotl
|
|
|
8 |
---
|
9 |
|
10 |
# MKLLM-7B-Instruct-exl2
|
@@ -13,7 +14,8 @@ EXL2 quants of [trajkovnikola/MKLLM-7B-Instruct](https://huggingface.co/trajkovn
|
|
13 |
|
14 |
<b>The "main" branch only contains the measurement.json (which can be used for further conversions), download one of the other branches for the model:</b>
|
15 |
|
16 |
-
[5
|
|
|
17 |
[4.25 bits per weight with 6 lm_head bits](https://huggingface.co/martinkozle/MKLLM-7B-Instruct-exl2/tree/4.25bpw-h6)
|
18 |
[3.5 bits per weight with 6 lm_head bits](https://huggingface.co/martinkozle/MKLLM-7B-Instruct-exl2/tree/3.5bpw-h6)
|
19 |
|
@@ -34,4 +36,4 @@ To download a specific branch, use the `--revision` parameter.
|
|
34 |
|
35 |
```shell
|
36 |
huggingface-cli download martinkozle/MKLLM-7B-Instruct-exl2 --revision 5.0bpw-h6 --local-dir MKLLM-7B-Instruct-exl2-5.0bpw-h6 --local-dir-use-symlinks False
|
37 |
-
```
|
|
|
5 |
- en
|
6 |
tags:
|
7 |
- axolotl
|
8 |
+
- exl2
|
9 |
---
|
10 |
|
11 |
# MKLLM-7B-Instruct-exl2
|
|
|
14 |
|
15 |
<b>The "main" branch only contains the measurement.json (which can be used for further conversions), download one of the other branches for the model:</b>
|
16 |
|
17 |
+
[6.5 bits per weight with 6 lm_head bits](https://huggingface.co/martinkozle/MKLLM-7B-Instruct-exl2/tree/6.5bpw-h6)
|
18 |
+
[5.0 bits per weight with 6 lm_head bits](https://huggingface.co/martinkozle/MKLLM-7B-Instruct-exl2/tree/5.0bpw-h6)
|
19 |
[4.25 bits per weight with 6 lm_head bits](https://huggingface.co/martinkozle/MKLLM-7B-Instruct-exl2/tree/4.25bpw-h6)
|
20 |
[3.5 bits per weight with 6 lm_head bits](https://huggingface.co/martinkozle/MKLLM-7B-Instruct-exl2/tree/3.5bpw-h6)
|
21 |
|
|
|
36 |
|
37 |
```shell
|
38 |
huggingface-cli download martinkozle/MKLLM-7B-Instruct-exl2 --revision 5.0bpw-h6 --local-dir MKLLM-7B-Instruct-exl2-5.0bpw-h6 --local-dir-use-symlinks False
|
39 |
+
```
|