ryzen88 commited on
Commit
810a8fd
1 Parent(s): a51ebbf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -48
README.md CHANGED
@@ -1,48 +1,61 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # model
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the breadcrumbs_ties merge method using I:\Llama-3-70B-Instruct-Gradient-262k as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * E:\Llama-3-Lumimaid-70B-v0.1-OAS
22
- * I:\Tess-2.0-Llama-3-70B-v0.2
23
-
24
- ### Configuration
25
-
26
- The following YAML configuration was used to produce this model:
27
-
28
- ```yaml
29
- models:
30
- - model: I:\Llama-3-70B-Instruct-Gradient-262k
31
- parameters:
32
- weight: 0.20
33
- density: 0.90
34
- gamma: 0.01
35
- - model: I:\Tess-2.0-Llama-3-70B-v0.2
36
- parameters:
37
- weight: 0.20
38
- density: 0.90
39
- gamma: 0.01
40
- - model: E:\Llama-3-Lumimaid-70B-v0.1-OAS
41
- parameters:
42
- weight: 0.60
43
- density: 0.90
44
- gamma: 0.01
45
- merge_method: breadcrumbs_ties
46
- base_model: I:\Llama-3-70B-Instruct-Gradient-262k
47
- dtype: bfloat16
48
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Lumitess
2
+
3
+ This model was a attempt to create a good llama 3 uncencored model with long context.
4
+ At it worked like a charm
5
+
6
+ Did a merge with breadcrumbs_ties method.
7
+ Instruct gradient, Lumimaid and Tess.
8
+
9
+ Uses llama 3 context
10
+
11
+ Sampler wise it has a very wide optimal so works with lots of different settings.
12
+
13
+
14
+
15
+ base_model: []
16
+ library_name: transformers
17
+ tags:
18
+ - mergekit
19
+ - merge
20
+
21
+ ---
22
+ # model
23
+
24
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
25
+
26
+ ## Merge Details
27
+ ### Merge Method
28
+
29
+ This model was merged using the breadcrumbs_ties merge method using I:\Llama-3-70B-Instruct-Gradient-262k as a base.
30
+
31
+ ### Models Merged
32
+
33
+ The following models were included in the merge:
34
+ * E:\Llama-3-Lumimaid-70B-v0.1-OAS
35
+ * I:\Tess-2.0-Llama-3-70B-v0.2
36
+
37
+ ### Configuration
38
+
39
+ The following YAML configuration was used to produce this model:
40
+
41
+ ```yaml
42
+ models:
43
+ - model: I:\Llama-3-70B-Instruct-Gradient-262k
44
+ parameters:
45
+ weight: 0.20
46
+ density: 0.90
47
+ gamma: 0.01
48
+ - model: I:\Tess-2.0-Llama-3-70B-v0.2
49
+ parameters:
50
+ weight: 0.20
51
+ density: 0.90
52
+ gamma: 0.01
53
+ - model: E:\Llama-3-Lumimaid-70B-v0.1-OAS
54
+ parameters:
55
+ weight: 0.60
56
+ density: 0.90
57
+ gamma: 0.01
58
+ merge_method: breadcrumbs_ties
59
+ base_model: I:\Llama-3-70B-Instruct-Gradient-262k
60
+ dtype: bfloat16
61
+ ```