v000000 commited on
Commit
5418cd2
1 Parent(s): 1c36fe6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +70 -0
README.md CHANGED
@@ -12,3 +12,73 @@ tags:
12
  This model was converted to GGUF format from [`v000000/L3-11.5B-DuS-MoonRoot`](https://huggingface.co/v000000/L3-11.5B-DuS-MoonRoot) using llama.cpp
13
  Refer to the [original model card](https://huggingface.co/v000000/L3-11.5B-DuS-MoonRoot) for more details on the model.'
14
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  This model was converted to GGUF format from [`v000000/L3-11.5B-DuS-MoonRoot`](https://huggingface.co/v000000/L3-11.5B-DuS-MoonRoot) using llama.cpp
13
  Refer to the [original model card](https://huggingface.co/v000000/L3-11.5B-DuS-MoonRoot) for more details on the model.'
14
 
15
+ ### Llama-3-11.5B-Depth-Upscaled-MoonRoot
16
+ experiement, no continued finetuning
17
+
18
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64f74b6e6389380c77562762/lNgAEcW3pWd6x0x-4C3q1.png)
19
+
20
+ # Pretty good understanding gets the percentage wrong but understands.
21
+ ```bash
22
+ user: A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector.
23
+ If the radiation detector detects radiation, it will release the poison. The box is opened one day later.
24
+ What is the probability of the cat being alive?
25
+
26
+ assistant: The answer is 100%. Since the cat is already dead when it was placed in the box,
27
+ there is no possibility for it to be alive when the box is opened...
28
+ ```
29
+
30
+ Shows similar emergent language nuance abilities compared to 8B.
31
+
32
+ Unaligned and somewhat lazy.
33
+
34
+ # Quants
35
+ * [Q8_0 static, imatrix](https://huggingface.co/v000000/L3-11.5B-DuS-MoonRoot-Q8_0-GGUF)
36
+ * [Q6_K static, imatrix](https://huggingface.co/v000000/L3-11.5B-DuS-MoonRoot-Q6_K-GGUF)
37
+
38
+ # merge
39
+
40
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
41
+
42
+ ## Merge Details
43
+ ### Merge Method
44
+
45
+ This model was merged using the passthrough merge method.
46
+
47
+ ### Models Merged
48
+
49
+ The following models were included in the merge:
50
+ * [Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B](https://huggingface.co/Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B)
51
+ * [v000000/L3-8B-Poppy-Moonfall-C](https://huggingface.co/v000000/L3-8B-Poppy-Moonfall-C)
52
+
53
+ ### Configuration
54
+
55
+ The following YAML configuration was used to produce this model:
56
+
57
+ ```yaml
58
+ slices:
59
+ - sources:
60
+ - model: v000000/L3-8B-Poppy-Moonfall-C
61
+ layer_range: [0, 24]
62
+ - sources:
63
+ - model: Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B
64
+ layer_range: [8, 32]
65
+ merge_method: passthrough
66
+ dtype: bfloat16
67
+ ```
68
+
69
+ ---
70
+ base_model:
71
+ - Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B
72
+ - v000000/L3-8B-Poppy-Moonfall-C
73
+
74
+ # Prompt Template:
75
+ ```bash
76
+ <|begin_of_text|><|start_header_id|>system<|end_header_id|>
77
+
78
+ {system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
79
+
80
+ {input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
81
+
82
+ {output}<|eot_id|>
83
+
84
+ ```