File size: 7,380 Bytes
4ef9200 ffc720b d08dda6 ffc720b 4ef9200 cac5f01 d08dda6 f91e61f 36a158e f91e61f 3845cd5 1d8134d 4ef9200 e91b0c7 4ef9200 77e7ed0 4ef9200 77e7ed0 4ef9200 77e7ed0 4ef9200 36a158e 4ef9200 36a158e e91b0c7 ffc720b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 |
---
license: cc-by-nc-4.0
library_name: transformers
tags:
- fourier
- task addition
- merge
base_model:
- maldv/badger-lambda-llama-3-8b
- maldv/llama-3-fantasy-writer-8b
- dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5
model-index:
- name: badger-writer-llama-3-8b
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: IFEval (0-Shot)
type: HuggingFaceH4/ifeval
args:
num_few_shot: 0
metrics:
- type: inst_level_strict_acc and prompt_level_strict_acc
value: 53.03
name: strict accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: BBH (3-Shot)
type: BBH
args:
num_few_shot: 3
metrics:
- type: acc_norm
value: 26.88
name: normalized accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MATH Lvl 5 (4-Shot)
type: hendrycks/competition_math
args:
num_few_shot: 4
metrics:
- type: exact_match
value: 6.57
name: exact match
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GPQA (0-shot)
type: Idavidrein/gpqa
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 5.26
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MuSR (0-shot)
type: TAUR-Lab/MuSR
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 3.2
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU-PRO (5-shot)
type: TIGER-Lab/MMLU-Pro
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 30.67
name: accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
name: Open LLM Leaderboard
---
Badger Writer is a *normalized fourier task addition* of [maldv/badger-lambda-llama-3-8b](/maldv/badger-lambda-llama-3-8b) and [maldv/llama-3-fantasy-writer-8b](/maldv/llama-3-fantasy-writer-8b).
I also used the first and last layer directly from [dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5](/dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5) due to the obvious advantages. I didn't train either the lm_head or embed_tokens layers on the fantasy-writer, but opus is part of lambda ; so they all fit nicely together.
Rep-pen 1.05-1.1 ; Min-p 0.01 ; Temp 0.7 ; Dynatemp 0.4 ; <s>32k</s>24k context ; llama 3 instruct template
```
<|start_header_id|>system<|end_header_id|>
It is so good for you to sit and let me watch you write, master of sci-fi.
Author Isaac Asimov ; Bruce Sterling
Title Terminal Connection
Tags adventure, computer science, body swapping
<|eot_id|>
```
````
{{[INPUT]}}
```story
Setting: 20th November 2050
Location: Las Vegas
Time: 11:17 pm
Character: Sally
Purpose: Exposition ; Descriptive ; Visual Detail, character appearance
```
// Chapter 1
{{[OUTPUT]}}
````
> The neon sky behind Sally was a strobe-lit smear. The street before her had the sheen of wet pavement under the headlights of a hundred different cars, each one bearing people away from this party, towards some more obscure destination.
>
> She took another drag off her cigarette, letting the smoke dance between her lips as she thought about this meeting, and how it came together.
>
> Sally stood tall at five foot two inches, but her athletic build made her seem taller. Her hair was jet black and trimmed short. She wore a low-cut yellow sweater that showed off an ample bosom, paired with black leather pants that hugged her hips perfectly. A silver necklace rested over her bare shoulders, studded with a small golden snake on its chain. Her skin was darkened, not by a sunburn, but by a tan lotion. Deep-brown eyes stared out into the night, squinting slightly against the glare. She looked like a girl who didn’t give a damn, except she did.
>
> Over her left shoulder hung a door to the penthouse suite in the Silver Star Hotel. Behind it, people danced, laughed and drank the night away. But not Sally; she had no interest in any of them, right now. They would have to wait until later.
>
> There were fifty other people on the street, milling around, some watching the view or waiting for friends, others lost in their own thoughts, maybe smoking too. People from all walks of life - tourists, locals, gamblers, street performers and dealers - they passed her by, always in a hurry to get somewhere else, and never lingered long enough to notice what might be the most interesting thing standing there.
>
> The music was loud inside, beating through the walls to create a bass-thumping rhythm that vibrated against Sally’s feet. It came from the DJ in the corner spinning tracks that kept the party moving along, his work supplemented by the pulsating strobe lights above. Sally could see it all from where she stood, even with the doors closed: the bartender working furiously as he handed out drinks; the waitress pouring wine, never stopping as the bottles glowed neon green under the black light that shone down on the tablecloths; men and women dancing together, losing themselves in the music or each other; and at the far end of the room, five or six poker games going full tilt with chips flying between them. The air smelled of smoke, sweat and sex as it flowed out onto the sidewalk.
>
> This was one of the many parties Vegas had to offer, but this particular party attracted a unique clientele. It was known as "The Gathering," and if you knew what to look for, you could find your way here easily enough. That’s why Sally found herself standing outside these doors, staring into the night.
````
{{[INPUT]}}
// Continue
{{[OUTPUT]}}
````
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_maldv__badger-writer-llama-3-8b)
| Metric |Value|
|-------------------|----:|
|Avg. |20.93|
|IFEval (0-Shot) |53.03|
|BBH (3-Shot) |26.88|
|MATH Lvl 5 (4-Shot)| 6.57|
|GPQA (0-shot) | 5.26|
|MuSR (0-shot) | 3.20|
|MMLU-PRO (5-shot) |30.67|
|