Update README.md
Browse files
README.md
CHANGED
@@ -14,6 +14,14 @@ pipeline_tag: text2text-generation
|
|
14 |
|
15 |
Merged and GPTQ quantized version of [rayliuca/TRagx-internlm2-7b](https://huggingface.co/rayliuca/TRagx-internlm2-7b)
|
16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
|
18 |
## GPTQ Dataset
|
19 |
|
|
|
14 |
|
15 |
Merged and GPTQ quantized version of [rayliuca/TRagx-internlm2-7b](https://huggingface.co/rayliuca/TRagx-internlm2-7b)
|
16 |
|
17 |
+
Note: I'm having some difficulties quantizing the models using GPTQ. Mistral and NeuralOmniBeagle's GPTQ models have significantly degraded output, while quantized TowerInstruct v0.2 was not working out right
|
18 |
+
|
19 |
+
While this quantized model for InternLM2 seems to work all right, the translation accuracy is not validated.
|
20 |
+
|
21 |
+
These AWQ quantized models are recommended:
|
22 |
+
- [rayliuca/TRagx-AWQ-NeuralOmniBeagle-7B](https://huggingface.co/rayliuca/TRagx-AWQ-NeuralOmniBeagle-7B)
|
23 |
+
- [rayliuca/TRagx-AWQ-Mistral-7B-Instruct-v0.2](https://huggingface.co/rayliuca/TRagx-AWQ-Mistral-7B-Instruct-v0.2)
|
24 |
+
|
25 |
|
26 |
## GPTQ Dataset
|
27 |
|