File size: 3,815 Bytes
e4f09d5 6eda04b e4f09d5 2ebeb4e 6eda04b e4f09d5 429dd44 6eda04b 2ebeb4e e4f09d5 5418cd2 23f27a4 5418cd2 20b71b5 5418cd2 2559b8d 5418cd2 2559b8d 5418cd2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 |
---
base_model: v000000/L3-11.5B-DuS-MoonRoot
library_name: transformers
tags:
- mergekit
- merge
- llama
- llama-cpp
---
# Quants in repo: Q8_0 imatrix, static
# v000000/L3-11.5B-DuS-MoonRoot-Q8_0-GGUF
This model was converted to GGUF format from [`v000000/L3-11.5B-DuS-MoonRoot`](https://huggingface.co/v000000/L3-11.5B-DuS-MoonRoot) using llama.cpp
Refer to the [original model card](https://huggingface.co/v000000/L3-11.5B-DuS-MoonRoot) for more details on the model.'
### Llama-3-11.5B-Depth-Upscaled-MoonRoot
experiemental solar-like llama3 frankenmerge, no continued finetuning
# Pretty good understanding gets the percentage wrong but understands.
```bash
user: A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector.
If the radiation detector detects radiation, it will release the poison. The box is opened one day later.
What is the probability of the cat being alive?
assistant: The answer is 100%. Since the cat is already dead when it was placed in the box,
there is no possibility for it to be alive when the box is opened...
```
Shows similar emergent language nuance abilities compared to 8B.
Unaligned and somewhat lazy. Use rep_pen 1.1
# merge
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the passthrough merge method.
### Models Merged
The following models were included in the merge:
* [Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B](https://huggingface.co/Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B)
* [v000000/L3-8B-Poppy-Moonfall-C](https://huggingface.co/v000000/L3-8B-Poppy-Moonfall-C)
### Configuration
The following YAML configuration was used to produce this model:
---Step 3
```yaml
slices:
- sources:
- model: v000000/L3-8B-Poppy-Moonfall-C
layer_range: [0, 24]
- sources:
- model: Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B
layer_range: [8, 32]
merge_method: passthrough
dtype: bfloat16
```
---Step 2
```yaml
slices:
- sources:
- model: v000000/L3-8B-Poppy-Sunspice-experiment-c+Blackroot/Llama-3-8B-Abomination-LORA
layer_range: [0, 32]
- model: v000000/L3-8B-Poppy-Sunspice-experiment-c+ResplendentAI/BlueMoon_Llama3
layer_range: [0, 32]
merge_method: slerp
base_model: v000000/L3-8B-Poppy-Sunspice-experiment-c+Blackroot/Llama-3-8B-Abomination-LORA
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: bfloat16
random_seed: 0
```
---Step 1
```yaml
models:
- model: crestf411/L3-8B-sunfall-abliterated-v0.2
parameters:
weight: 0.1
density: 0.18
- model: Hastagaras/HALU-8B-LLAMA3-BRSLURP
parameters:
weight: 0.1
density: 0.3
- model: Nitral-Archive/Poppy_Porpoise-Biomix
parameters:
weight: 0.1
density: 0.42
- model: cgato/L3-TheSpice-8b-v0.8.3
parameters:
weight: 0.2
density: 0.54
- model: Sao10K/L3-8B-Stheno-v3.2
parameters:
weight: 0.2
density: 0.66
- model: Nitral-AI/Poppy_Porpoise-0.72-L3-8B
parameters:
weight: 0.3
density: 0.78
merge_method: dare_ties
base_model: NousResearch/Meta-Llama-3-8B-Instruct
parameters:
int8_mask: true
dtype: bfloat16
```
---
base_model:
- Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B
- v000000/L3-8B-Poppy-Moonfall-C
# Prompt Template:
```bash
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{output}<|eot_id|>
``` |