Update README.md
Browse files
README.md
CHANGED
@@ -9,11 +9,75 @@ tags:
|
|
9 |
|
10 |
An experimental model, fine-tuned using the ["multiplicative-LoRA" method](#the-multiplicative-lora-method) on [c4ai-command-r-v01](https://huggingface.co/CohereForAI/c4ai-command-r-v01).
|
11 |
|
12 |
-
Other experimental models
|
13 |
|
14 |
- [creative-writer-v0.1-bravo-35b](https://huggingface.co/jukofyork/creative-writer-v0.1-bravo-35b) - Scaled the pre-softmax logits by `1.1` during training (and then reset after training).
|
15 |
- [creative-writer-v0.1-charlie-35b](https://huggingface.co/jukofyork/creative-writer-v0.1-charlie-35b) - Scaled the pre-softmax logits by `0.9` during training (and didn't reset after training).
|
16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
---
|
18 |
|
19 |
# Usage
|
|
|
9 |
|
10 |
An experimental model, fine-tuned using the ["multiplicative-LoRA" method](#the-multiplicative-lora-method) on [c4ai-command-r-v01](https://huggingface.co/CohereForAI/c4ai-command-r-v01).
|
11 |
|
12 |
+
Other experimental models which attempt to encourage more diverse/creative text generation:
|
13 |
|
14 |
- [creative-writer-v0.1-bravo-35b](https://huggingface.co/jukofyork/creative-writer-v0.1-bravo-35b) - Scaled the pre-softmax logits by `1.1` during training (and then reset after training).
|
15 |
- [creative-writer-v0.1-charlie-35b](https://huggingface.co/jukofyork/creative-writer-v0.1-charlie-35b) - Scaled the pre-softmax logits by `0.9` during training (and didn't reset after training).
|
16 |
|
17 |
+
<details> <summary>Click to see some (brief) tests on effect of these changes</summary>
|
18 |
+
|
19 |
+
#### Using `command-r-3-2024` with `temperature = 1` and `min-p = 0.01`:
|
20 |
+
|
21 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/GqrDZnKk-fqRvihfxL014.png)
|
22 |
+
|
23 |
+
#### Using `creative-writer-v0.1-alfa:35b` with `temperature = 1` and `min-p = 0.01`:
|
24 |
+
|
25 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/imba_ELU1lCXR309u4CGY.png)
|
26 |
+
|
27 |
+
#### Using `creative-writer-v0.1-bravo:35b` with `temperature = 1` and `min-p = 0.01`:
|
28 |
+
|
29 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/7IzQQ_f5GGGI6-AFqG4mT.png)
|
30 |
+
|
31 |
+
#### Using `creative-writer-v0.1-charlie:35b` with `temperature = 1` and `min-p = 0.01`:
|
32 |
+
|
33 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/b4kJeQbbA9Dor3pgNZlVA.png)
|
34 |
+
|
35 |
+
---
|
36 |
+
|
37 |
+
#### Using `command-r-3-2024` with `temperature = 1` and `min-p = 0.01`:
|
38 |
+
|
39 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/aHEODwL9oEvWOQe52SUMy.png)
|
40 |
+
|
41 |
+
#### Using `creative-writer-v0.1-alfa:35b` with `temperature = 1` and `min-p = 0.01`:
|
42 |
+
|
43 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/2BDugSmPlJQCWp8QuHXDh.png)
|
44 |
+
|
45 |
+
#### Using `creative-writer-v0.1-bravo:35b` with `temperature = 1` and `min-p = 0.01`:
|
46 |
+
|
47 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/a7VEVHFgC1BBzkZIAECYW.png)
|
48 |
+
|
49 |
+
#### Using `creative-writer-v0.1-charlie:35b` with `temperature = 1` and `min-p = 0.01`:
|
50 |
+
|
51 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/ejs2KklluH1dhVCv9kQBl.png)
|
52 |
+
|
53 |
+
---
|
54 |
+
|
55 |
+
#### Using `command-r-3-2024` with `temperature = 1` and `min-p = 0.01`:
|
56 |
+
|
57 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/4cHQfo1PvJI8bB2tyxXSS.png)
|
58 |
+
|
59 |
+
#### Using `creative-writer-v0.1-alfa:35b` with `temperature = 1` and `min-p = 0.01`:
|
60 |
+
|
61 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/Kf9x9GgGBh1ed5xINtill.png)
|
62 |
+
|
63 |
+
#### Using `creative-writer-v0.1-alfa:35b` with `temperature = 1.1` and `min-p = 0.01`:
|
64 |
+
|
65 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/CpL_86aPtdxl2rUAJgGdX.png)
|
66 |
+
|
67 |
+
#### Using `creative-writer-v0.1-bravo:35b` with `temperature = 1` and `min-p = 0.01`:
|
68 |
+
|
69 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/KJZj-8bPYI78m9VZ45gNr.png)
|
70 |
+
|
71 |
+
#### Using `creative-writer-v0.1-bravo:35b` with `temperature = 0.9` and `min-p = 0.01`:
|
72 |
+
|
73 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/qSeuI1MCIc__4VG9YC_pn.png)
|
74 |
+
|
75 |
+
#### Using `creative-writer-v0.1-charlie:35b` with `temperature = 1` and `min-p = 0.01`:
|
76 |
+
|
77 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/PwnDkctZ273zMHC-Ta_YU.png)
|
78 |
+
|
79 |
+
</details>
|
80 |
+
|
81 |
---
|
82 |
|
83 |
# Usage
|