win10 commited on
Commit
db1621d
1 Parent(s): c223e91

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +55 -49
README.md CHANGED
@@ -1,49 +1,55 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # llama3
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the passthrough merge method.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
-
27
- ```yaml
28
- slices:
29
- - sources:
30
- - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct # embed_tokens comes along with the ride with whatever is the first layer
31
- layer_range: [0, 1]
32
- - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct # add dummy second model with 0 weight so tokenizer-based merge routine is invoked for embed_tokens
33
- layer_range: [0, 1]
34
- - sources:
35
- - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
36
- layer_range: [1, 24]
37
- - sources:
38
- - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
39
- layer_range: [8, 20]
40
- - sources:
41
- - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
42
- layer_range: [18, 32]
43
- - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
44
- layer_range: [18, 32]
45
- merge_method: passthrough
46
- dtype: bfloat16
47
-
48
-
49
- ```
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ language:
4
+ - en
5
+ pipeline_tag: text-generation
6
+ tags:
7
+ - pytorch
8
+ - llama
9
+ - llama-3
10
+ - mergekit
11
+ - merge
12
+ license: llama3
13
+
14
+ ---
15
+ # llama3
16
+
17
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
18
+
19
+ ## Merge Details
20
+ ### Merge Method
21
+
22
+ This model was merged using the passthrough merge method.
23
+
24
+ ### Models Merged
25
+
26
+ The following models were included in the merge:
27
+ * D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
28
+
29
+ ### Configuration
30
+
31
+ The following YAML configuration was used to produce this model:
32
+
33
+ ```yaml
34
+ slices:
35
+ - sources:
36
+ - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct # embed_tokens comes along with the ride with whatever is the first layer
37
+ layer_range: [0, 1]
38
+ - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct # add dummy second model with 0 weight so tokenizer-based merge routine is invoked for embed_tokens
39
+ layer_range: [0, 1]
40
+ - sources:
41
+ - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
42
+ layer_range: [1, 24]
43
+ - sources:
44
+ - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
45
+ layer_range: [8, 20]
46
+ - sources:
47
+ - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
48
+ layer_range: [18, 32]
49
+ - model: D:/text-generation-webui/models/meta-llama_Meta-Llama-3-8B-Instruct
50
+ layer_range: [18, 32]
51
+ merge_method: passthrough
52
+ dtype: bfloat16
53
+
54
+
55
+ ```