File size: 3,685 Bytes
d063bd6 a408785 d063bd6 f491588 d063bd6 f7ff432 d063bd6 29cb535 f7ff432 f491588 f7ff432 609a2e9 29cb535 d063bd6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
---
library_name: transformers
tags:
- not-for-all-audiences
datasets:
- crestf411/LimaRP-DS
- Gryphe/Sonnet3.5-Charcard-Roleplay
- anthracite-org/c2_logs_32k_mistral-v3_v1.2_no_system
- anthracite-org/kalo-opus-instruct-22k-no-refusal-no-system
- anthracite-org/kalo-opus-instruct-3k-filtered-no-system
- anthracite-org/nopm_claude_writing_fixed
base_model:
- mistralai/Mistral-Small-Instruct-2409
---
Sunfall (2024-10-28) v0.7.0 on top of [Mistral Small Instruct 2409](mistralai/Mistral-Small-Instruct-2409).
It also contains samples from [Antracite.Org](https://huggingface.co/anthracite-org) datasets. See bottom for details.
Significant revamping of the dataset metadata generation process, resulting in higher quality dataset overall. *The "Diamond Law" experiment has been removed as it didn't seem to affect the model output enough to warrant set up complexity.*
Recommended starting point:
* Temperature: **1**
* MinP: **0.05~0.1**
* DRY: **0.8 1.75 2 0**
At early context, I recommend keeping XTC disabled. Once you hit higher context sizes (10k+), enabling XTC at 0.1 / 0.5 seems to significantly improve the output, but YMMV. If the output drones on and is uninspiring, XTC can be extremely effective.
*[User](https://huggingface.co/crestf411/MS-sunfall-v0.7.0/discussions/1#672104795c98f6ace037ba61) suggests adding this as an author note/world info In-chat @ Depth '0' is effective:*
```
[OOC: Guidelines: take into account the character's personality, background, relationships and previous history before intimate scenes. Intimate scenes should have a logical explanation and result from a current or past situation. {{char}} will not submit to {{user}} without good reason.]
```
General heuristic:
* Lots of slop? Temperature is too low. Raise it, or enable XTC. For early context, temp bump is probably preferred.
* Is the model making mistakes about subtle or obvious details in the scene? Temperature is too high, OR XTC is enabled and/or XTC settings are too high. Lower temp and/or disable XTC.
*Mergers/fine-tuners: [there is a LoRA of this model](https://huggingface.co/crestf411/sunfall-peft/tree/main/mistral-small-instruct-2409). Consider merging that instead of merging this model.*
This model has been trained on context that mimics that of Silly Tavern's "Mistral V2 & V3" preset, with character names added.
Silly Tavern output example (Henry is the human, Beth the bot):
```
[INST] Henry: I poke Beth.[/INST] Beth: Beth yelps.</s>[INST] Henry: ...
```
The model has also been trained to do interactive storywriting. You may steer the model towards specific content by "responding" to the model like so:
```
Continue writing adhering to the following scenario: (things you want to happen next)
```
Additional inclusions (random sampled sub-set, cursorily quality-checked) from:
- [Gryphe/Sonnet3.5-Charcard-Roleplay](https://huggingface.co/datasets/Gryphe/Sonnet3.5-Charcard-Roleplay)
- [anthracite-org/c2_logs_32k_mistral-v3_v1.2_no_system](https://huggingface.co/datasets/anthracite-org/c2_logs_32k_mistral-v3_v1.2_no_system)
- [anthracite-org/kalo-opus-instruct-22k-no-refusal-no-system](https://huggingface.co/datasets/anthracite-org/kalo-opus-instruct-22k-no-refusal-no-system)
- [anthracite-org/kalo-opus-instruct-3k-filtered-no-system](https://huggingface.co/datasets/anthracite-org/kalo-opus-instruct-3k-filtered-no-system)
- [anthracite-org/nopm_claude_writing_fixed](https://huggingface.co/datasets/anthracite-org/nopm_claude_writing_fixed)
As such, the dataset is not 100% slop free, but this addition likely helps the model be a better roleplayer. At some point, I intend to clean up and release the samples, deslopped.
|