Aryanne commited on
Commit
71e6cc1
·
verified ·
1 Parent(s): a4e889b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +50 -0
README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - NeuralNovel/Senzu-7B-v0.1-DPO
4
+ - senseable/WestLake-7B-v2
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+ # merged
12
+
13
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
+
15
+ ## Merge Details
16
+ ### Merge Method
17
+
18
+ This model was merged using the task_swapping merge method using [NeuralNovel/Senzu-7B-v0.1-DPO](https://huggingface.co/NeuralNovel/Senzu-7B-v0.1-DPO) as a base.
19
+
20
+ ### Models Merged
21
+
22
+ The following models were included in the merge:
23
+ * [senseable/WestLake-7B-v2](https://huggingface.co/senseable/WestLake-7B-v2)
24
+
25
+ ### Configuration
26
+
27
+ The following YAML configuration was used to produce this model:
28
+
29
+ ```yaml
30
+ base_model:
31
+ model:
32
+ path: NeuralNovel/Senzu-7B-v0.1-DPO
33
+ dtype: bfloat16
34
+ merge_method: task_swapping
35
+ slices:
36
+ - sources:
37
+ - layer_range: [0, 32]
38
+ model:
39
+ model:
40
+ path: senseable/WestLake-7B-v2
41
+ parameters:
42
+ diagonal_offset: 2.0
43
+ random_mask: 0.3333
44
+ random_mask_seed: 98557.0
45
+ weight: 0.75
46
+ - layer_range: [0, 32]
47
+ model:
48
+ model:
49
+ path: NeuralNovel/Senzu-7B-v0.1-DPO
50
+ ```