Nitral-AI commited on
Commit
fe6e82f
·
verified ·
1 Parent(s): e82b855

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +41 -0
README.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - Epiculous/Violet_Twilight-v0.2
4
+ - Nitral-AI/Captain_BMO-12B
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+
12
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/642265bc01c62c1e4102dc36/V4OFnx7IBvXhYphdkUr5F.png)
13
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/642265bc01c62c1e4102dc36/Shj4ZhtUMgIpyhnfUsk98.png)
14
+ # Instruct/Context import + Textgen preset both available: [Here](https://huggingface.co/Nitral-AI/Captain_Violet-V0.420-12B/tree/main/ST)
15
+
16
+
17
+ ## Original Models used in the merge:
18
+ [Epiculous/Violet_Twilight-v0.2](https://huggingface.co/Epiculous/Violet_Twilight-v0.2)
19
+ [Nitral-AI/Captain_BMO-12B](https://huggingface.co/Nitral-AI/Captain_BMO-12B)
20
+
21
+
22
+ ### The following YAML configuration was used to produce this model:
23
+
24
+ ```yaml
25
+ slices:
26
+ - sources:
27
+ - model: Nitral-AI/Captain_BMO-12B
28
+ layer_range: [0, 40]
29
+ - model: Epiculous/Violet_Twilight-v0.2
30
+ layer_range: [0, 40]
31
+ merge_method: slerp
32
+ base_model: Nitral-AI/Captain_BMO-12B
33
+ parameters:
34
+ t:
35
+ - filter: self_attn
36
+ value: [0, 0.5, 0.3, 0.7, 1]
37
+ - filter: mlp
38
+ value: [1, 0.5, 0.7, 0.3, 0]
39
+ - value: 0.420
40
+ dtype: bfloat16
41
+ ```