File size: 384 Bytes
ade1317 e92d4f4 0d5014b e92d4f4 0d5014b e92d4f4 11e951c e92d4f4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 |
---
license: apache-2.0
---
I know, I know, 6.5bpw is enough for perfection.
For some people.
But, for those who want the best they can load, here's an 8bpw quant.
Makes a difference for me, I think.
tweaked exl2 quant parameters a bit because I run 6-8k contexts:
python3 convert.py -i ../models/Smaug-Mixtral_v0.1 -o smaug_mixtral -cf smaug_mixtral -l 4096 -b 8 -hb 8 -ss 4096
|