|
This repo contains my preferred SillyTavern settings for Mistral Small models. |
|
|
|
Mistral Small is a fantastic model, but its user/assistant turn structure means that the context/prompt composed by SillyTavern can mess up the instruct formatting for things like Lorebook/World Info injection or Author's Note injection, or result in repeated Assistant or User turns that can confuse the model. |
|
|
|
I've had the best luck composing the context as a **single user turn**, asking the model to respond to what's essentially a very long single-turn prompt. |
|
|
|
The composed context tends to look something like this: |
|
``` |
|
<s>[INST] System prompt: Use the following information to write {{char}}'s next response in the Exchange below. |
|
|
|
Character description |
|
Persona description |
|
Scenario description |
|
World Info |
|
|
|
Exchange: |
|
{{char}}: First message</s> |
|
{{user}}: First response |
|
{{char}}: Second message</s> |
|
{{user}}: Second response |
|
|
|
Author's Note: Write 2 paragraphs using vivid and natural prose.[/INST] {{char}}: |
|
``` |
|
|
|
This prevents instruct formatting issues from AN/WI insertions, and avoids problems with repeated turns depending on whether the first message in the chat is from the Character or the User. |
|
|
|
When using AN with this template, it should work correctly with Before/After Story String insertion. For more immediate insertion, Depth 1 (as shown in the example above) is least likely to confuse the model by interrupting the chat history. |
|
|
|
Note: `</s>` is added after each Character message in the chat history to prevent problems with the AI not ending its turn properly when writing the newest response. |