Update README.md
Browse files
README.md
CHANGED
|
@@ -2,6 +2,7 @@
|
|
| 2 |
base_model:
|
| 3 |
- maldv/badger-lambda-llama-3-8b
|
| 4 |
- maldv/llama-3-fantasy-writer-8b
|
|
|
|
| 5 |
library_name: transformers
|
| 6 |
tags:
|
| 7 |
- fourier
|
|
@@ -14,6 +15,8 @@ license: cc-by-nc-4.0
|
|
| 14 |
|
| 15 |
Badger Writer is a *normalized fourier task addition* of maldv/badger-lambda-llama-3-8b and maldv/llama-3-fantasy-writer-8b.
|
| 16 |
|
|
|
|
|
|
|
| 17 |
Rep-pen 1.1 ; Min-p 0.01 ; Temp 0.7 ; Dynatemp 0.4 ; 32k context ; llama 3 instruct template
|
| 18 |
|
| 19 |
```
|
|
|
|
| 2 |
base_model:
|
| 3 |
- maldv/badger-lambda-llama-3-8b
|
| 4 |
- maldv/llama-3-fantasy-writer-8b
|
| 5 |
+
- dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5
|
| 6 |
library_name: transformers
|
| 7 |
tags:
|
| 8 |
- fourier
|
|
|
|
| 15 |
|
| 16 |
Badger Writer is a *normalized fourier task addition* of maldv/badger-lambda-llama-3-8b and maldv/llama-3-fantasy-writer-8b.
|
| 17 |
|
| 18 |
+
I also used the first and last layer directly from dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5 due to the obvious advantages. I didn't train either the lm_head or embed_token layers on the fantasy-writer, but opus is part of lambda ; so they all fit nicely together.
|
| 19 |
+
|
| 20 |
Rep-pen 1.1 ; Min-p 0.01 ; Temp 0.7 ; Dynatemp 0.4 ; 32k context ; llama 3 instruct template
|
| 21 |
|
| 22 |
```
|