---
license: apache-2.0
library_name: transformers
language:
- en
tags:
- chat
- conversational
base_model:
- Qwen/Qwen2.5-32B
- AiCloser/Qwen2.5-32B-AGI
- EVA-UNIT-01/EVA-Qwen2.5-32B-v0.2
- fblgit/TheBeagle-v2beta-32B-MGS
- huihui-ai/Qwen2.5-32B-Instruct-abliterated
- huihui-ai/QwQ-32B-Preview-abliterated
- Qwen/QwQ-32B-Preview
- rombodawg/Rombos-LLM-V2.5-Qwen-32b
- nbeerbower/Qwen2.5-Gutenberg-Doppel-32B
---

# Qwentile 2.5 32B Instruct

Qwentile 2.5 32B Instruct is a *normalized denoised fourier interpolation* of the following models:

```yaml
output_base_model: "Qwen/Qwen2.5-32B"
finetune_merge:
  - { "model": "AiCloser/Qwen2.5-32B-AGI", "base": "Qwen/Qwen2.5-32B", "alpha": 0.3 }
  - { "model": "EVA-UNIT-01/EVA-Qwen2.5-32B-v0.2", "base": "Qwen/Qwen2.5-32B", "alpha": 0.7 }
  - { "model": "fblgit/TheBeagle-v2beta-32B-MGS", "base": "Qwen/Qwen2.5-32B", "alpha": 0.6 }
  - { "model": "huihui-ai/Qwen2.5-32B-Instruct-abliterated", "base": "Qwen/Qwen2.5-32B-Instruct", "alpha": 1.0 }
  - { "model": "huihui-ai/QwQ-32B-Preview-abliterated", "base": "Qwen/Qwen2.5-32B", "alpha": 1.0 }
  - { "model": "Qwen/QwQ-32B-Preview", "base": "Qwen/Qwen2.5-32B", "alpha": 0.8, "is_input": true }
  - { "model": "rombodawg/Rombos-LLM-V2.5-Qwen-32b", "base": "Qwen/Qwen2.5-32B", "alpha": 1.0, "is_output": true }
  - { "model": "nbeerbower/Qwen2.5-Gutenberg-Doppel-32B", "base": "Qwen/Qwen2.5-32B-Instruct", "alpha": 0.4 }
```

In other words, all of these models get warped and interpolated in signal space, and then jammed back on top of the instruct model.

### What is this?

I started my experiment because of QwQ is a really nifty model, but it was giving me problems with xml output - which is what I use for my thought tokens. So, I thought... lets just merge it in!

The first model worked pretty well, but I got a sense that the balances could be tweaked. Why not throw in some other models as well for fun and see if I can't run out of disk space in the process?

### Initial Results

It's a little crispier than Awqward, but does generate stable output. Since it is based on Qwen2.5 base instead of instruct, maybe it can score over zero on the math leaderboard?

## Citation

If you find our work helpful, feel free to give us a cite.

```
@misc{qwentile2.5-32b-instruct,
    title = {Qwentile 2.5 32B Instruct},
    url = {https://huggingface.co/maldv/Qwentile2.5-32B-Instruct},
    author = {Praxis Maldevide},
    month = {December},
    year = {2024}
}
```