---
license: apache-2.0
base_model:
- open-r1/OpenR1-Qwen-7B
---

AWQ 4bits version of open-r1/OpenR1-Qwen-7B

```python
from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer
model_name = "open-r1/OpenR1-Qwen-7B"
model = AutoAWQForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
quant_config = { "zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "GEMM" }
model.quantize(tokenizer, quant_config=quant_config)
```