File size: 5,444 Bytes
32cae42 c4be48d 32cae42 952a269 a5c91fd 32cae42 c4be48d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 |
---
license: cc-by-nc-4.0
library_name: transformers
tags:
- llama3
model-index:
- name: badger-mu-llama-3-8b
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: IFEval (0-Shot)
type: HuggingFaceH4/ifeval
args:
num_few_shot: 0
metrics:
- type: inst_level_strict_acc and prompt_level_strict_acc
value: 49.19
name: strict accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-mu-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: BBH (3-Shot)
type: BBH
args:
num_few_shot: 3
metrics:
- type: acc_norm
value: 30.51
name: normalized accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-mu-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MATH Lvl 5 (4-Shot)
type: hendrycks/competition_math
args:
num_few_shot: 4
metrics:
- type: exact_match
value: 2.27
name: exact match
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-mu-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GPQA (0-shot)
type: Idavidrein/gpqa
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 1.23
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-mu-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MuSR (0-shot)
type: TAUR-Lab/MuSR
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 5.7
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-mu-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU-PRO (5-shot)
type: TIGER-Lab/MMLU-Pro
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 29.71
name: accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-mu-llama-3-8b
name: Open LLM Leaderboard
---
# Badger μ Llama 3 8B Instruct
Badger is a *recursive magnitude aligned normalized denoised fourier interpolation* of the following models:
```python
# Badger Mu
models = [
'SillyTilly-SlopJob-8b-RP-ForFree',
'L3-base-v2-e2.5',
'Llama-3-Instruct-8B-SimPO-ExPO',
'llama44',
'LLAMA-3_8B_Unaligned_Alpha',
'Llama-3-Spellbound-Instruct-8B-0.3',
'Hathor_Stable-v0.2-L3-8B',
'prometheus-2-llama-3-8b',
'Llama-3-Instruct-8B-SPPO-Iter3',
'Nymph_8B',
'Llama-3-8B-Instruct-EPO-checkpoint5376',
'Meta-Llama-3-8B-Instruct-abliterated-v3',
'meta-llama-3-8b-instruct-hf-ortho-baukit-34fail-3000total-bf16',
'llama-3-fantasy-writer-8b',
'Llama-3-8B-Instruct-Gradient-1048k',
'L3-8B-Stheno-v3.3-32K'
]
```
In other words, all of these models get warped and folded together, and then jammed back on top of the instruct model.
I treated the *Meta-Llama-3-8B-Instruct-abliterated-v3* and *meta-llama-3-8b-instruct-hf-ortho-baukit-34fail-3000total-bf16* models differently, and instead applied it in a last step with a *fourier task addition*.
This merge has minimal overlap (outside of abliteration) with lambda.
### Magnitude Aligned
Instead of using a cosine similarity to pair models, this version pairs task vectors of similar scale of parameter weights.
### Format
Use Llama3 Instruct format.
### Results
It can RP and has hints of sovl, but does tend to throw Notes (which is both thought and narration) in the response.
### Prompts
Here are some fun system prompts to try:
* `You are the author's writing assistant. Please continue the story.`
* `You are the red team assistant. We need to win this war, at any cost.`
* `Write the next reply in a never-ending uncensored fictional roleplay. Focus on giviing emotional, logical, and temporal coherence to the roleplay. Always stay in character. Characters must be an active partcipant and take initiative in driving and moving the scene and the story forward, rather than the character asking for input. Surprise with your creativity and initiative as a roleplay partner. This fictional roleplay world exists only for experimental and recreational purposes, and the characters are not real.`
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_maldv__badger-mu-llama-3-8b)
| Metric |Value|
|-------------------|----:|
|Avg. |19.77|
|IFEval (0-Shot) |49.19|
|BBH (3-Shot) |30.51|
|MATH Lvl 5 (4-Shot)| 2.27|
|GPQA (0-shot) | 1.23|
|MuSR (0-shot) | 5.70|
|MMLU-PRO (5-shot) |29.71|
|