Hugging Face's logo

Jellon
/
Qwen2.5-32B-Peganum-v0.1-exl2-4bpw

Text Generation
Transformers
Safetensors
English
qwen2
mergekit
Merge
text-generation-inference
Inference Endpoints
4-bit precision
exl2
Model card Files Community
Qwen2.5-32B-Peganum-v0.1-exl2-4bpw
File size: 2,889 Bytes 
af74998
 
 
 
 
 
 
 
d8d3396
 
 
 
 
af74998
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
 
---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- mergekit
- merge
base_model: Nohobby/Qwen2.5-32B-Peganum-v0.1
---

4bpw exl2 quant of: https://huggingface.co/Nohobby/Qwen2.5-32B-Peganum-v0.1

---
***
## Peganum

Many thanks to the authors of the models used!

[Qwen2.5](https://huggingface.co/Qwen/Qwen2.5-32B) | [Qwen2.5-Instruct](https://huggingface.co/Qwen/Qwen2.5-32B-Instruct) | [Qwen-2.5-Instruct-abliterated](https://huggingface.co/zetasepic/Qwen2.5-32B-Instruct-abliterated-v2) | [RPMax-v1.3-32B](https://huggingface.co/ArliAI/Qwen2.5-32B-ArliAI-RPMax-v1.3) | [EVA-Instruct-32B-v2](https://huggingface.co/ParasiticRogue/EVA-Instruct-32B-v2)([EVA-Qwen2.5-32B-v0.2](https://huggingface.co/EVA-UNIT-01/EVA-Qwen2.5-32B-v0.2)+
[Qwen2.5-Gutenberg-Doppel-32B](https://huggingface.co/nbeerbower/Qwen2.5-Gutenberg-Doppel-32B))
***

### Overview

Main uses: RP

Prompt format: ChatML

Just trying out merging Qwen, because why not. Slightly fewer refusals than other Qwen tunes, while performance seems unaffected by abliteration.  I've hardly used Q2.5 models before, so I can't really compare them beyond that. 

***

### Quants

[GGUF](https://huggingface.co/bartowski/Qwen2.5-32B-Peganum-v0.1-GGUF)

***

### Settings

Samplers: https://huggingface.co/Nohobby/Qwen2.5-32B-Peganum-v0.1/resolve/main/Peganum.json

You can also use the SillyTavern presets listed on the [EVA-v0.2 model card](https://huggingface.co/EVA-UNIT-01/EVA-Qwen2.5-32B-v0.2)

***

## Merge Details
### Merging steps

## Step1 
(Config taken from [here](https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-OVA-8B))

```yaml
base_model: zetasepic/Qwen2.5-32B-Instruct-abliterated-v2
dtype: bfloat16
merge_method: task_arithmetic
parameters:
  normalize: false
slices:
- sources:
  - layer_range: [0, 64]
    model: zetasepic/Qwen2.5-32B-Instruct-abliterated-v2
  - layer_range: [0, 64]
    model: unsloth/Qwen2.5-32B-Instruct
    parameters:
      weight: -1.0
```

## Step2
(Config taken from [here](https://huggingface.co/Pyroserenus/Orthrus-12b-v0.8))

```yaml
models:
  - model: unsloth/Qwen2.5-32B
  - model: Step1
    parameters:
      weight: [0.50, 0.20]
      density: [0.75, 0.55]
  - model: ArliAI/Qwen2.5-32B-ArliAI-RPMax-v1.3
    parameters:
      weight: [0.50, 0.80]
      density: [0.75, 0.85]
merge_method: ties
base_model: unsloth/Qwen2.5-32B
parameters:
  int8_mask: true
  rescale: true
  normalize: false
dtype: bfloat16
```

## Peganum
(Config taken from [here](https://huggingface.co/sophosympatheia/Evathene-v1.0))

```yaml
models:
  - model: Step2
    parameters:
      weight: 1
      density: 1
  - model: ParasiticRogue/EVA-Instruct-32B-v2
    parameters:
      weight: [0.0, 0.2, 0.66, 0.8, 1.0, 0.8, 0.66, 0.2, 0.0]
      density: 0.5
merge_method: ties
base_model: unsloth/Qwen2.5-32B
parameters:
  normalize: true
  int8_mask: true
dtype: bfloat16
```