nguyenviet/PhoGPT-7B5-Instruct-GGUF

PhoGPT-7B5-Instruct.GGUF

GGUF format files of the model vinai/PhoGPT-7B5-Instruct.

Model Details

For detailed information about the original model, please refer to phoGPT's repository.

Uses

Select and download the quantization version that fits the needs.

License

PhoGPT is licensed under the PhoGPT Community License, Copyright (c) VinAI. All Rights Reserved.

Provided files

Name	Quant method	Size	Use case
PhoGPT-7B5-Instruct-q2_k.gguf	Q2_K	3.8 GB	smallest, significant quality loss - not recommended for most purposes
PhoGPT-7B5-Instruct-q3_k_s.gguf	Q3_K_S	4.07 GB	very small, high quality loss
PhoGPT-7B5-Instruct-q3_k_m.gguf	Q3_K_M	4.66 GB	very small, high quality loss
PhoGPT-7B5-Instruct-q3_k_l.gguf	Q3_K_L	4.98 GB	small, substantial quality loss
PhoGPT-7B5-Instruct-q4_0.gguf	Q4_0	5.06 GB	legacy; small, very high quality loss - prefer using Q3_K_M
PhoGPT-7B5-Instruct-q4_k_s.gguf	Q4_K_S	5.1 GB	small, greater quality loss
PhoGPT-7B5-Instruct-q4_k_m.gguf	Q4_K_M	5.54 GB	medium, balanced quality - recommended
PhoGPT-7B5-Instruct-q4_1.gguf	Q4_1	5.53 GB	legacy; higher accuracy than Q4_0 but not as high as Q5_0, however has quicker inference than Q5 models.
PhoGPT-7B5-Instruct-q5_0.gguf	Q5_0	6 GB	legacy; medium, balanced quality - prefer using Q4_K_M
PhoGPT-7B5-Instruct-q5_k_s.gguf	Q5_K_S	6 GB	large, low quality loss - recommended
PhoGPT-7B5-Instruct-q5_k_m.gguf	Q5_K_M	6.35 GB	large, very low quality loss - recommended
PhoGPT-7B5-Instruct-q5_1.gguf	Q5_1	6.46 GB	legacy; even higher accuracy, resource usage and slower inference.
PhoGPT-7B5-Instruct-q6_k.gguf	Q6_K	6.99 GB	very large, extremely low quality loss
PhoGPT-7B5-Instruct-q8_0.gguf	Q8_0	9.05 GB	almost indistinguishable from float16. High resource use and slow, not recommended for most users
PhoGPT-7B5-Instruct-f16.gguf	float16	17 GB	very large, extremely low quality loss - not recommended