metadata
license: other
language:
- vi
model_name: PhoGPT 7B5 Instruct
inference: false
model_creator: VinAI Research
model_link: https://huggingface.co/vinai/PhoGPT-7B5-Instruct
model_type: mpt
pipeline_tag: text-generation
quantized_by: nguyenviet
base_model: vinai/PhoGPT-7B5-Instruct
PhoGPT-7B5-Instruct.GGUF
GGUF format files of the model vinai/PhoGPT-7B5-Instruct.
Model Details
For detailed information about the original model, please refer to phoGPT's repository.
Uses
Select and download the quantization version that fits the needs.
License
PhoGPT is licensed under the PhoGPT Community License, Copyright (c) VinAI. All Rights Reserved.
Provided files
| Name | Quant method | Size | Use case |
|---|---|---|---|
| PhoGPT-7B5-Instruct-q2_k.gguf | Q2_K | 3.8 GB | smallest, significant quality loss - not recommended for most purposes |
| PhoGPT-7B5-Instruct-q3_k_s.gguf | Q3_K_S | 4.07 GB | very small, high quality loss |
| PhoGPT-7B5-Instruct-q3_k_m.gguf | Q3_K_M | 4.66 GB | very small, high quality loss |
| PhoGPT-7B5-Instruct-q3_k_l.gguf | Q3_K_L | 4.98 GB | small, substantial quality loss |
| PhoGPT-7B5-Instruct-q4_0.gguf | Q4_0 | 5.06 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
| PhoGPT-7B5-Instruct-q4_k_s.gguf | Q4_K_S | 5.1 GB | small, greater quality loss |
| PhoGPT-7B5-Instruct-q4_k_m.gguf | Q4_K_M | 5.54 GB | medium, balanced quality - recommended |
| PhoGPT-7B5-Instruct-q4_1.gguf | Q4_1 | 5.53 GB | legacy; higher accuracy than Q4_0 but not as high as Q5_0, however has quicker inference than Q5 models. |
| PhoGPT-7B5-Instruct-q5_0.gguf | Q5_0 | 6 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
| PhoGPT-7B5-Instruct-q5_k_s.gguf | Q5_K_S | 6 GB | large, low quality loss - recommended |
| PhoGPT-7B5-Instruct-q5_k_m.gguf | Q5_K_M | 6.35 GB | large, very low quality loss - recommended |
| PhoGPT-7B5-Instruct-q5_1.gguf | Q5_1 | 6.46 GB | legacy; even higher accuracy, resource usage and slower inference. |
| PhoGPT-7B5-Instruct-q6_k.gguf | Q6_K | 6.99 GB | very large, extremely low quality loss |
| PhoGPT-7B5-Instruct-q8_0.gguf | Q8_0 | 9.05 GB | almost indistinguishable from float16. High resource use and slow, not recommended for most users |
| PhoGPT-7B5-Instruct-f16.gguf | float16 | 17 GB | very large, extremely low quality loss - not recommended |