rayliuca commited on
Commit
948ea6b
1 Parent(s): 2795885

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -14,6 +14,14 @@ pipeline_tag: text2text-generation
14
 
15
  Merged and GPTQ quantized version of [rayliuca/TRagx-internlm2-7b](https://huggingface.co/rayliuca/TRagx-internlm2-7b)
16
 
 
 
 
 
 
 
 
 
17
 
18
  ## GPTQ Dataset
19
 
 
14
 
15
  Merged and GPTQ quantized version of [rayliuca/TRagx-internlm2-7b](https://huggingface.co/rayliuca/TRagx-internlm2-7b)
16
 
17
+ Note: I'm having some difficulties quantizing the models using GPTQ. Mistral and NeuralOmniBeagle's GPTQ models have significantly degraded output, while quantized TowerInstruct v0.2 was not working out right
18
+
19
+ While this quantized model for InternLM2 seems to work all right, the translation accuracy is not validated.
20
+
21
+ These AWQ quantized models are recommended:
22
+ - [rayliuca/TRagx-AWQ-NeuralOmniBeagle-7B](https://huggingface.co/rayliuca/TRagx-AWQ-NeuralOmniBeagle-7B)
23
+ - [rayliuca/TRagx-AWQ-Mistral-7B-Instruct-v0.2](https://huggingface.co/rayliuca/TRagx-AWQ-Mistral-7B-Instruct-v0.2)
24
+
25
 
26
  ## GPTQ Dataset
27