Update README.md

README.md

@@ -1,3 +1,138 @@
----
-license: llama3
----
+---
+base_model: []
+library_name: transformers
+tags:
+- mergekit
+- merge
+- not-for-all-audiences
+license: llama3
+language:
+- en
+---
+
+<div align="center">
+  <b style="font-size: 36px;">EtherealRainbow-v0.2-8B (GGUF)</b>
+
+  <img src="https://huggingface.co/invisietch/EtherealRainbow-v0.2-8B/resolve/main/ethrheader.png" style="width:60%">
+</div>
+
+# Model Details
+
+Ethereal Rainbow is an 8B parameter merge of various Llama3-based finetunes created using mergekit. The purpose of Ethereal Rainbow is to 
+create an uncensored Llama3 variant which is capable of writing creative prose, and engaging in SFW as well as NSFW  roleplay and 
+storytelling, with a strong focus on long-form responses &amp; adherence to prompts.
+
+I don't consider v0.2 to be a 'finished product' by any means, but I've been getting some good results out of it so I wanted to share.
+
+# Formats
+
+[Safetensors](https://huggingface.co/invisietch/EtherealRainbow-v0.2-8B) | [GGUF](https://huggingface.co/invisietch/EtherealRainbow-v0.2-8B-GGUF)
+
+# Disclaimer
+
+This model is built on an abliterated base and as such is largely uncensored. It can generate explicit, disturbing or offensive responses.
+Use responsibly. I am not responsible for your use of this model.
+
+# Known Issues
+
+There are a couple of known issues with the model which I hope to fix in a future release:
+
+* It prefers asterisks for actions and bare speech roleplay style. You can work around this by swiping or editing the first 2-3 responses
+  to match the style you want, and then it tends to follow your style from there.
+* It runs 'hot' by default, so the temperature you use for other models is likely to be too high. I recommend starting with 0.7 &amp;
+  slowly increasing from there.
+
+# Merge Details
+
+## Merge Methodology
+
+EtherealRainbow is a two-stage merge. I create two base models -- **Rain** & **Sun** -- via `dare_ties`, and then using **Rain** as base, I use 
+`slerp` to bring in part of **Sun**.
+
+## Rain
+
+The purpose of **Rain** is to create an uncensored model focused purely on roleplay. For this, I chose four component models:
+
+* **mlabonne/NeuralDaredevil-8B-abliterated (base)** because it's a mostly-uncensored, coherent, high MMLU Llama-3 base.
+* **Sao10K/L3-8B-Stheno-v3.2** because it has my favorite creative writing of all Llama-3 RP variants.
+* **Nitral-AI/Hathor-L3-8B-v.02** to broaden the model's knowledge of creative writing styles.
+* **grimjim/Llama-3-Luminurse-v0.2-OAS-8B** to improve ERP with more anatomical knowledge.
+
+This is the mergekit config that I used to build **Rain**:
+
+```yaml
+models:
+  - model: mlabonne/NeuralDaredevil-8B-abliterated
+  - model: Sao10K/L3-8B-Stheno-v3.2
+    parameters:
+      density: 0.53
+      weight: 0.4
+  - model: Nitral-AI/Hathor-L3-8B-v.02
+    parameters:
+      density: 0.37
+      weight: 0.4
+  - model: grimjim/Llama-3-Luminurse-v0.2-OAS-8B
+    parameters:
+      density: 0.26
+      weight: 0.2
+merge_method: dare_ties
+base_model: mlabonne/NeuralDaredevil-8B-abliterated
+parameters:
+  int8_mask: true
+dtype: bfloat16
+```
+
+## Sun
+
+The purpose of **Sun** is to improve intelligence, particularly on two fronts: creative writing ability and anatomical knowledge.
+For this, I chose four more component models:
+
+* **Gryphe/Pantheon-RP-1.0-8b-Llama-3** because it's trained on large RP datasets that aren't in other models I used.
+* **aaditya/Llama3-OpenBioLLM-8B** for its anatomical and medical knowledge.
+* **Blackroot/Llama-3-LongStory** because it's uniquely trained on much longer story texts, to improve output length.
+* **Locutusque/Llama-3-Hercules-5.0-8B** for similar reasons to Blackroot/Llama-3-LongStory.
+
+This is the config that I used to create **Sun**:
+
+```yaml
+models:
+  - model: Gryphe/Pantheon-RP-1.0-8b-Llama-3
+  - model: aaditya/Llama3-OpenBioLLM-8B
+    parameters:
+      density: 0.36
+      weight: 0.2
+  - model: Blackroot/Llama-3-LongStory
+    parameters:
+      density: 0.40
+      weight: 0.3
+  - model: Locutusque/Llama-3-Hercules-5.0-8B
+    parameters:
+      density: 0.49
+      weight: 0.5
+merge_method: dare_ties
+base_model: Gryphe/Pantheon-RP-1.0-8b-Llama-3
+parameters:
+  int8_mask: true
+dtype: bfloat16
+```
+
+*Note: **Sun** is unusable by itself, it's very prone to endless generation issues, but after 400+ messages I have not seen*
+*this issue with **EtherealRainbow**.*
+
+## EtherealRainbow
+
+This is the config that I used to create **EtherealRainbow v0.2**:
+
+```yaml
+models:
+  - model: ./fp16/rain-8B
+  - model: ./fp16/sun-8B
+merge_method: slerp
+base_model: ./fp16/rain-8B
+parameters:
+  t:
+    - value: [0, 0, 0.1, 0.3, 0.5, 0.7, 0.5, 0.3, 0.1, 0, 0]
+  embed_slerp: true
+dtype: bfloat16
+tokenizer_source: model:./fp16/rain-8B
+```