T145 commited on
Commit
8a004e1
·
verified ·
1 Parent(s): e448344

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +67 -67
README.md CHANGED
@@ -1,67 +1,67 @@
1
- ---
2
- base_model:
3
- - unsloth/Meta-Llama-3.1-8B
4
- - T145/KRONOS-8B-V1-P2
5
- - unsloth/Meta-Llama-3.1-8B-Instruct
6
- - T145/KRONOS-8B-V1-P1
7
- - T145/KRONOS-8B-V1-P3
8
- library_name: transformers
9
- tags:
10
- - mergekit
11
- - merge
12
-
13
- ---
14
- # Untitled Model (1)
15
-
16
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
17
-
18
- ## Merge Details
19
- ### Merge Method
20
-
21
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [unsloth/Meta-Llama-3.1-8B](https://huggingface.co/unsloth/Meta-Llama-3.1-8B) as a base.
22
-
23
- ### Models Merged
24
-
25
- The following models were included in the merge:
26
- * [T145/KRONOS-8B-V1-P2](https://huggingface.co/T145/KRONOS-8B-V1-P2)
27
- * [unsloth/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/unsloth/Meta-Llama-3.1-8B-Instruct)
28
- * [T145/KRONOS-8B-V1-P1](https://huggingface.co/T145/KRONOS-8B-V1-P1)
29
- * [T145/KRONOS-8B-V1-P3](https://huggingface.co/T145/KRONOS-8B-V1-P3)
30
-
31
- ### Configuration
32
-
33
- The following YAML configuration was used to produce this model:
34
-
35
- ```yaml
36
- base_model: unsloth/Meta-Llama-3.1-8B
37
- dtype: bfloat16
38
- merge_method: ties
39
- parameters:
40
- density: 1.0
41
- weight: 1.0
42
- slices:
43
- - sources:
44
- - layer_range: [0, 32]
45
- model: T145/KRONOS-8B-V1-P1
46
- parameters:
47
- density: 1.0
48
- weight: 1.0
49
- - layer_range: [0, 32]
50
- model: T145/KRONOS-8B-V1-P2
51
- parameters:
52
- density: 1.0
53
- weight: 1.0
54
- - layer_range: [0, 32]
55
- model: T145/KRONOS-8B-V1-P3
56
- parameters:
57
- density: 1.0
58
- weight: 1.0
59
- - layer_range: [0, 32]
60
- model: unsloth/Meta-Llama-3.1-8B-Instruct
61
- parameters:
62
- density: 1.0
63
- weight: 1.0
64
- - layer_range: [0, 32]
65
- model: unsloth/Meta-Llama-3.1-8B
66
- tokenizer_source: unsloth/Meta-Llama-3.1-8B-Instruct
67
- ```
 
1
+ ---
2
+ base_model:
3
+ - unsloth/Meta-Llama-3.1-8B
4
+ - T145/KRONOS-8B-V1-P2
5
+ - unsloth/Meta-Llama-3.1-8B-Instruct
6
+ - T145/KRONOS-8B-V1-P1
7
+ - T145/KRONOS-8B-V1-P3
8
+ library_name: transformers
9
+ tags:
10
+ - mergekit
11
+ - merge
12
+ license: llama3.1
13
+ ---
14
+ # KRONOS-8B-V2
15
+
16
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
17
+
18
+ ## Merge Details
19
+ ### Merge Method
20
+
21
+ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [unsloth/Meta-Llama-3.1-8B](https://huggingface.co/unsloth/Meta-Llama-3.1-8B) as a base.
22
+
23
+ ### Models Merged
24
+
25
+ The following models were included in the merge:
26
+ * [T145/KRONOS-8B-V1-P2](https://huggingface.co/T145/KRONOS-8B-V1-P2)
27
+ * [unsloth/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/unsloth/Meta-Llama-3.1-8B-Instruct)
28
+ * [T145/KRONOS-8B-V1-P1](https://huggingface.co/T145/KRONOS-8B-V1-P1)
29
+ * [T145/KRONOS-8B-V1-P3](https://huggingface.co/T145/KRONOS-8B-V1-P3)
30
+
31
+ ### Configuration
32
+
33
+ The following YAML configuration was used to produce this model:
34
+
35
+ ```yaml
36
+ base_model: unsloth/Meta-Llama-3.1-8B
37
+ dtype: bfloat16
38
+ merge_method: ties
39
+ parameters:
40
+ density: 1.0
41
+ weight: 1.0
42
+ slices:
43
+ - sources:
44
+ - layer_range: [0, 32]
45
+ model: T145/KRONOS-8B-V1-P1
46
+ parameters:
47
+ density: 1.0
48
+ weight: 1.0
49
+ - layer_range: [0, 32]
50
+ model: T145/KRONOS-8B-V1-P2
51
+ parameters:
52
+ density: 1.0
53
+ weight: 1.0
54
+ - layer_range: [0, 32]
55
+ model: T145/KRONOS-8B-V1-P3
56
+ parameters:
57
+ density: 1.0
58
+ weight: 1.0
59
+ - layer_range: [0, 32]
60
+ model: unsloth/Meta-Llama-3.1-8B-Instruct
61
+ parameters:
62
+ density: 1.0
63
+ weight: 1.0
64
+ - layer_range: [0, 32]
65
+ model: unsloth/Meta-Llama-3.1-8B
66
+ tokenizer_source: unsloth/Meta-Llama-3.1-8B-Instruct
67
+ ```