File size: 3,555 Bytes
a0c0a1e 6f0f252 a0c0a1e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 |
---
base_model:
- Sao10K/L3-8B-Stheno-v3.2
- chujiezheng/Llama-3-Instruct-8B-SimPO-ExPO
- TheDrummer/Llama-3SOME-8B-v2
- NeverSleep/Llama-3-Lumimaid-8B-v0.1
- Hastagaras/Jamet-8B-L3-MK.V-Blackroot
library_name: transformers
tags:
- mergekit
- merge
- roleplay
- sillytavern
- llama3
- not-for-all-audiences
license: cc-by-nc-4.0
language:
- en
---
- Upgraded SimPO.
- A touch of 3SOME, Lumimaid and Jamet Blackroot resulting a slightly different prose and wider RP vocab.
- Leans slightly more on nsfw than the original.
## SillyTavern
## Text Completion presets
```
temp 0.9
top_k 30
top_p 0.75
min_p 0.2
rep_pen 1.1
smooth_factor 0.25
smooth_curve 1
```
## Advanced Formatting
[Context & Instruct preset by Virt-io](https://huggingface.co/Virt-io/SillyTavern-Presets/tree/main/Prompts/LLAMA-3/v1.9)
Instruct Mode: Enabled
# merge
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
This model was merged using the slerp merge method.
### Models Merged
The following models were included in the merge:
* [Sao10K/L3-8B-Stheno-v3.2](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2)
* [chujiezheng/Llama-3-Instruct-8B-SimPO-ExPO](https://huggingface.co/chujiezheng/Llama-3-Instruct-8B-SimPO-ExPO)
* [TheDrummer/Llama-3SOME-8B-v2](https://huggingface.co/TheDrummer/Llama-3SOME-8B-v2)
* [NeverSleep/Llama-3-Lumimaid-8B-v0.1](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-8B-v0.1)
* [Hastagaras/Jamet-8B-L3-MK.V-Blackroot](https://huggingface.co/Hastagaras/Jamet-8B-L3-MK.V-Blackroot)
### Configuration
The following YAML configuration was used to produce this model:
```yaml
slices:
- sources:
- model: NeverSleep/Llama-3-Lumimaid-8B-v0.1
layer_range: [0, 32]
- model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot
layer_range: [0, 32]
merge_method: slerp
base_model: NeverSleep/Llama-3-Lumimaid-8B-v0.1
parameters:
t:
- filter: self_attn
value: [0.7, 0.3, 0.3, 0.3]
- filter: mlp
value: [0.3, 0.7, 0.7, 0.7]
- value: 0.4
dtype: bfloat16
L3-Lumimaid-Jamet-Blackroot-8B
slices:
- sources:
- model: tannedbum/L3-Lumimaid-Jamet-Blackroot-8B
layer_range: [0, 32]
- model: chujiezheng/Llama-3-Instruct-8B-SimPO-ExPO
layer_range: [0, 32]
merge_method: slerp
base_model: tannedbum/L3-Lumimaid-Jamet-Blackroot-8B
parameters:
t:
- filter: self_attn
value: [0.3, 0.7, 0.7, 0.7]
- filter: mlp
value: [0.7, 0.3, 0.3, 0.3]
- value: 0.6
dtype: bfloat16
L3-SimPO-Lumimaid-Jamet-Blackroot-8B
slices:
- sources:
- model: Sao10K/L3-8B-Stheno-v3.2
layer_range: [0, 32]
- model: TheDrummer/Llama-3SOME-8B-v2
layer_range: [0, 32]
merge_method: slerp
base_model: Sao10K/L3-8B-Stheno-v3.2
parameters:
t:
- filter: self_attn
value: [0.3, 0.3, 0.7, 0.3]
- filter: mlp
value: [0.7, 0.7, 0.3, 0.7]
- value: 0.4
dtype: bfloat16
L3-Stheno-3SOME-8B
slices:
- sources:
- model: tannedbum/L3-Stheno-3SOME-8B
layer_range: [0, 32]
- model: tannedbum/L3-SimPO-Lumimaid-Jamet-Blackroot-8B
layer_range: [0, 32]
merge_method: slerp
base_model: tannedbum/L3-Stheno-3SOME-8B
parameters:
t:
- filter: self_attn
value: [0.4, 0.3, 0.3, 0.6]
- filter: mlp
value: [0.6, 0.7, 0.7, 0.4]
- value: 0.4
dtype: bfloat16
L3-Nymeria-v2-8B
```
Want to support my work ? My Ko-fi page: https://ko-fi.com/tannedbum |