TheYuriLover commited on
Commit
54579f2
1 Parent(s): 8688ddd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -0
README.md CHANGED
@@ -1,3 +1,10 @@
1
  ---
2
  license: other
3
  ---
 
 
 
 
 
 
 
 
1
  ---
2
  license: other
3
  ---
4
+ This is the gptq 4bit quantization of this model: https://huggingface.co/elinas/chronos-13b
5
+
6
+ This quantization was made by using this repository: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton
7
+
8
+ And I used the triton branch with all the gptq implementations available (true_sequential + act_order + groupsize 128)
9
+
10
+ CUDA_VISIBLE_DEVICES=0 python llama.py ./chronos-13b-GPTQ-Triton c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors manticore-13b-4bit-128g.safetensors