Quantization
Collection
A collection of quantized models. All the models can be fine-tuned by adding a LoRA Adapter.
•
83 items
•
Updated
•
3
The Quantized Google Gemma 2 27B Instruction Tuned Model
Original Base Model: google/gemma-2-27b-it.
Link: https://huggingface.co/google/gemma-2-27b-it
Quantization Configurations
"quantization_config": {
"bits": 4,
"checkpoint_format": "gptq",
"damp_percent": 0.1,
"desc_act": true,
"group_size": 128,
"model_file_base_name": null,
"model_name_or_path": null,
"quant_method": "gptq",
"static_groups": false,
"sym": true,
"true_sequential": true
},
Source Codes
Source Codes: https://github.com/vkola-lab/medpodgpt/tree/main/quantization.
- Downloads last month
- 741
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.