Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,8 @@ EXL2 quants of alpindale/goliath-120b (https://huggingface.co/alpindale/goliath-
|
|
| 5 |
|
| 6 |
Calibration dataset is wikitext. I've added a measurement.json file on the main branch if you want to do your own quants.
|
| 7 |
|
|
|
|
|
|
|
| 8 |
[4.85bpw](https://huggingface.co/Panchovix/goliath-120b-exl2/tree/4.85bpw)
|
| 9 |
|
| 10 |
[4.5bpw](https://huggingface.co/Panchovix/goliath-120b-exl2/tree/4.5bpw)
|
|
|
|
| 5 |
|
| 6 |
Calibration dataset is wikitext. I've added a measurement.json file on the main branch if you want to do your own quants.
|
| 7 |
|
| 8 |
+
IMPORTANT: For the 3BPW quant, and if using ooba text gen, disable BOS Token, else you will get gibberish, see https://huggingface.co/Panchovix/goliath-120b-exl2/discussions/1
|
| 9 |
+
|
| 10 |
[4.85bpw](https://huggingface.co/Panchovix/goliath-120b-exl2/tree/4.85bpw)
|
| 11 |
|
| 12 |
[4.5bpw](https://huggingface.co/Panchovix/goliath-120b-exl2/tree/4.5bpw)
|