Commit
•
8535fab
1
Parent(s):
6bbc271
Upload model
Browse files- README.md +1 -83
- adapter_model.bin +1 -1
README.md
CHANGED
@@ -1,91 +1,9 @@
|
|
1 |
---
|
2 |
library_name: peft
|
3 |
-
license: apache-2.0
|
4 |
-
datasets:
|
5 |
-
- ThatOneShortGuy/SongLyrics
|
6 |
-
language:
|
7 |
-
- en
|
8 |
-
tags:
|
9 |
-
- music
|
10 |
---
|
11 |
-
# Musical Falcon
|
12 |
-
|
13 |
-
[OpenAssistant/falcon-7b-sft-mix-2000](https://huggingface.co/OpenAssistant/falcon-7b-sft-mix-2000) model fine tuned using PEFT on
|
14 |
-
[Song Lyrics](https://huggingface.co/datasets/ThatOneShortGuy/SongLyrics) to write lyrics to songs.
|
15 |
-
|
16 |
-
## Model Details
|
17 |
-
- **Finetuned from**: [OpenAssistant/falcon-7b-sft-mix-2000]
|
18 |
-
- **Model Type**: Causal decoder-only transformer language model
|
19 |
-
- **Language**: English (and limited capabilities in German, Spanish, French, Italian, Portuguese, Polish, Dutch, Romanian, Czech, Swedish);
|
20 |
-
- **License**: Apache 2.0
|
21 |
-
- **Contact**: Lol don't. This is just for fun.
|
22 |
-
|
23 |
-
## Usage
|
24 |
-
The basic basic format getting it in is:
|
25 |
-
```python
|
26 |
-
from peft import PeftModel, PeftConfig
|
27 |
-
from transformers import AutoModelForCausalLM
|
28 |
-
|
29 |
-
config = PeftConfig.from_pretrained("ThatOneShortGuy/MusicalFalcon")
|
30 |
-
model = AutoModelForCausalLM.from_pretrained("OpenAssistant/falcon-7b-sft-mix-2000")
|
31 |
-
model = PeftModel.from_pretrained(model, "ThatOneShortGuy/MusicalFalcon")
|
32 |
-
```
|
33 |
-
|
34 |
-
## Prompting
|
35 |
-
Considering this comes from [OpenAssistant/falcon-7b-sft-mix-2000](https://huggingface.co/OpenAssistant/falcon-7b-sft-mix-2000), it uses the same structure.
|
36 |
-
Two special tokens are used to mark the beginning of user and assistant turns: `<|prompter|>` and `<|assistant|>`. Each turn ends with a `<|endoftext|>` token.
|
37 |
-
The training prompt used the structure:
|
38 |
-
```
|
39 |
-
<|prompter|>Come up with the lyrics for a song from "{artist}" {"from " + year if year else ""} titled "{title}".<|endoftext|>
|
40 |
-
<|assistant|>Sure! Here are the lyrics:
|
41 |
-
{lyrics}
|
42 |
-
<|endoftext|>
|
43 |
-
```
|
44 |
-
|
45 |
-
However, it still seems to work just fine using:
|
46 |
-
```
|
47 |
-
<|prompter|>Write me a song titled "{title}"<|endoftext|><|assistant|>
|
48 |
-
```
|
49 |
-
or any anything of similar nature. Feel free to add a description of the song in there too.
|
50 |
-
|
51 |
-
## Examples
|
52 |
-
Prompt
|
53 |
-
```
|
54 |
-
<|prompter|>Write a song with the lyrics for a song from artist "Eminem" from 2018 titled "Suzy's Lemonade Stand"<|endoftext|>
|
55 |
-
<|assistant|>Sure! Here are the lyrics:
|
56 |
-
```
|
57 |
-
Result
|
58 |
-
```
|
59 |
-
[Intro]
|
60 |
-
I'm gonna make you famous, baby
|
61 |
-
|
62 |
-
[Verse 1]
|
63 |
-
It's been a long time since I had a lemonade stand
|
64 |
-
But I'm back in town and I'ma do it again
|
65 |
-
You know what? I've been on my own since I was born
|
66 |
-
So I'll be here all day long, so come on, let's go
|
67 |
-
I got some new tricks up my sleeve that I'd like to show
|
68 |
-
And I'mma get your name out of your mouth, so please don't blow
|
69 |
-
Let me see if I can get you to say my name
|
70 |
-
If not, then I'l just have to take it back to the drawing board
|
71 |
-
I gotta get this money, I's a little bit broke
|
72 |
-
I need some cash, I need some dough, I gotta get these clothes off
|
73 |
-
I wanna be a star, I wanna be the man
|
74 |
-
I want to be a superstar, I want to get my hands on them bands
|
75 |
-
```
|
76 |
-
|
77 |
-
|
78 |
## Training procedure
|
79 |
|
80 |
-
|
81 |
-
- load_in_4bit: False
|
82 |
-
- llm_int8_threshold: 6.0
|
83 |
-
- llm_int8_skip_modules: None
|
84 |
-
- llm_int8_enable_fp32_cpu_offload: True
|
85 |
-
- llm_int8_has_fp16_weight: False
|
86 |
-
- bnb_4bit_quant_type: fp4
|
87 |
-
- bnb_4bit_use_double_quant: False
|
88 |
### Framework versions
|
89 |
|
90 |
|
91 |
-
- PEFT 0.5.0.dev0
|
|
|
1 |
---
|
2 |
library_name: peft
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4 |
## Training procedure
|
5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
### Framework versions
|
7 |
|
8 |
|
9 |
+
- PEFT 0.5.0.dev0
|
adapter_model.bin
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 130641741
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:1992417a1f2689c9b04845f565bc68d1189f0687cbf4d8b305cb3b2cfab058d8
|
3 |
size 130641741
|