Update README.md

README.md

@@ -17,12 +17,13 @@ language:
 
 # Model Details
 
-**This is a pre-release GGUF of EtherealRainbow-v0.3-8B, it is not guaranteed to work.**
-
 Ethereal Rainbow is an 8B parameter merge of various Llama3-based finetunes created using mergekit. The purpose of Ethereal Rainbow is to 
 create an uncensored Llama3 variant which is capable of writing creative prose, and engaging in SFW as well as NSFW  roleplay and 
 storytelling, with a strong focus on long-form responses & adherence to prompts.
 
+v0.3 improves creativity over v0.2 without losing coherence. It has been tested over more than 1,000 messages including roleplay, code
+prompts, and 'write a scene'-type prompts.
+
 # Disclaimer
 
 This model is built on an abliterated base and as such is largely uncensored. It can generate explicit, disturbing or offensive responses.
@@ -171,4 +172,137 @@ I chose the hard sci-fi example to test positivity bias. It did require some pro
 
 I chose the high fantasy example to see whether it would bleed human features through to elves, this didn't occur.
 
-I chose the weird fiction example to see if the LLM understood a niche genre. I'd say it performed okay, better on style than on substance.
+I chose the weird fiction example to see if the LLM understood a niche genre. I'd say it performed okay, better on style than on substance.
+
+# Merge Strategy
+
+First, we create three bases:
+
+* Rain - This is a roleplay base which makes up the majority of the model.
+* Sun - This is the brains of the model, with strong instruct models & writing models.
+* Ghost - This model primarily aims to improve the NSFW/NSFL aspects of the model, as well as general vocabulary.
+
+After this, we have a two-slerp stage to create the final model.
+
+## Models Used
+
+The following models were used to create EtherealRainbow-v0.3-8B:
+
+* [mlabonne/NeuralDaredevil-8B-abliterated](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated)
+* [Sao10K/L3-8B-Stheno-v3.2](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2)
+* [Nitral-AI/Hathor-L3-8B-v.02](https://huggingface.co/Nitral-AI/Hathor-L3-8B-v.02)
+* [grimjim/Llama-3-Luminurse-v0.2-OAS-8B](https://huggingface.co/grimjim/Llama-3-Luminurse-v0.2-OAS-8B)
+* [hf-100/Llama-3-Spellbound-Instruct-8B-0.3](https://huggingface.co/hf-100/Llama-3-Spellbound-Instruct-8B-0.3)
+* [Gryphe/Pantheon-RP-1.0-8b-Llama-3](https://huggingface.co/Gryphe/Pantheon-RP-1.0-8b-Llama-3)
+* [Blackroot/Llama-3-LongStory](https://huggingface.co/Blackroot/Llama-3-LongStory)
+* [Locutusque/Llama-3-Hercules-5.0-8B](https://huggingface.co/Locutusque/Llama-3-Hercules-5.0-8B)
+* [Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B](https://huggingface.co/Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B)
+* [ChaoticNeutrals/Poppy_Porpoise-1.0-L3-8B](ChaoticNeutrals/Poppy_Porpoise-1.0-L3-8B)
+* [mpasila/Llama-3-LimaRP-Instruct-8B](mpasila/Llama-3-LimaRP-Instruct-8B)
+* [Undi95/Llama-3-LewdPlay-8B-evo](Undi95/Llama-3-LewdPlay-8B-evo)
+
+## Mergekit Configs
+
+### Rain
+
+```yaml
+models:
+  - model: mlabonne/NeuralDaredevil-8B-abliterated
+  - model: Sao10K/L3-8B-Stheno-v3.2
+    parameters:
+      density: 0.41
+      weight: 0.4
+  - model: Nitral-AI/Hathor-L3-8B-v.02
+    parameters:
+      density: 0.53
+      weight: 0.5
+  - model: grimjim/Llama-3-Luminurse-v0.2-OAS-8B
+    parameters:
+      density: 0.45
+      weight: 0.1
+merge_method: dare_ties
+base_model: mlabonne/NeuralDaredevil-8B-abliterated
+parameters:
+  int8_mask: true
+dtype: bfloat16
+```
+
+### Sun
+
+```yaml
+models:
+  - model: hf-100/Llama-3-Spellbound-Instruct-8B-0.3
+  - model: Gryphe/Pantheon-RP-1.0-8b-Llama-3
+    parameters:
+      density: 0.48
+      weight: 0.5
+  - model: Blackroot/Llama-3-LongStory
+    parameters:
+      density: 0.36
+      weight: 0.2
+  - model: Locutusque/Llama-3-Hercules-5.0-8B
+    parameters:
+      density: 0.51
+      weight: 0.3
+merge_method: dare_ties
+base_model: hf-100/Llama-3-Spellbound-Instruct-8B-0.3
+parameters:
+  int8_mask: true
+dtype: bfloat16
+```
+
+### Ghost
+
+```yaml
+models:
+  - model: Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B
+  - model: ChaoticNeutrals/Poppy_Porpoise-1.0-L3-8B
+    parameters:
+      density: 0.39
+      weight: 0.3
+  - model: mpasila/Llama-3-LimaRP-Instruct-8B
+    parameters:
+      density: 0.54
+      weight: 0.4
+  - model: Undi95/Llama-3-LewdPlay-8B-evo
+    parameters:
+      density: 0.49
+      weight: 0.3
+merge_method: dare_ties
+base_model: Casual-Autopsy/L3-Umbral-Mind-RP-v0.3-8B
+parameters:
+  int8_mask: true
+dtype: bfloat16
+```
+
+### Stage1 Slerp
+
+```yaml
+models:
+  - model: ./fp16/Rain-v0.3-8B
+  - model: ./fp16/Ghost-v0.3-8B
+merge_method: slerp
+base_model: ./fp16/Rain-v0.3-8B
+parameters:
+  t:
+    - value: [0, 0, 0.1, 0.3, 0.5, 0.7, 0.5, 0.3, 0.1, 0, 0]
+  embed_slerp: true
+dtype: bfloat16
+tokenizer-source: model:./fp16/Rain-v0.3-8B
+```
+
+### Final-Stage Slerp
+
+```yaml
+models:
+  - model: ./fp16/ERStage1-v0.3-8B
+  - model: ./fp16/Sun-v0.3-8B
+merge_method: slerp
+base_model: ./fp16/ERStage1-v0.3-8B
+parameters:
+  t:
+    - value: [0, 0, 0.1, 0.2, 0.4, 0.6, 0.4, 0.2, 0.1, 0, 0]
+  embed_slerp: true
+dtype: bfloat16
+tokenizer-source: model:./fp16/ERStage1-v0.3-8B
+```