v000000/L3-11.5B-DuS-MoonRoot

Llama-3-11.5B-Depth-Upscaled-MoonRoot

experiemental solar-like llama3 frankenmerge, no continued finetuning

Pretty good understanding gets the percentage wrong but understands.

user: A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector.
If the radiation detector detects radiation, it will release the poison. The box is opened one day later.
What is the probability of the cat being alive?

assistant: The answer is 100%. Since the cat is already dead when it was placed in the box,
there is no possibility for it to be alive when the box is opened...

Shows similar emergent language nuance abilities compared to 8B.

Unaligned and somewhat lazy. Use rep_pen 1.1

Thanks mradermacher for the quants!

• GGUF
• GGUF imatrix

Quants

• Q8_0 static, imatrix
• Q6_K static, imatrix

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the passthrough merge method.

Models Merged

The following models were included in the merge:

• Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B
• v000000/L3-8B-Poppy-Moonfall-C

Configuration

The following YAML configuration was used to produce this model:

---Step 3

slices:
  - sources:
    - model: v000000/L3-8B-Poppy-Moonfall-C
      layer_range: [0, 24]
  - sources:
    - model: Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B
      layer_range: [8, 32]
merge_method: passthrough
dtype: bfloat16

---Step 2

slices:
  - sources:
      - model: v000000/L3-8B-Poppy-Sunspice-experiment-c+Blackroot/Llama-3-8B-Abomination-LORA
        layer_range: [0, 32]
      - model: v000000/L3-8B-Poppy-Sunspice-experiment-c+ResplendentAI/BlueMoon_Llama3
        layer_range: [0, 32]
merge_method: slerp
base_model: v000000/L3-8B-Poppy-Sunspice-experiment-c+Blackroot/Llama-3-8B-Abomination-LORA
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16
random_seed: 0

---Step 1

models:
  - model: crestf411/L3-8B-sunfall-abliterated-v0.2
    parameters:
      weight: 0.1
      density: 0.18
  - model: Hastagaras/HALU-8B-LLAMA3-BRSLURP
    parameters:
      weight: 0.1
      density: 0.3
  - model: Nitral-Archive/Poppy_Porpoise-Biomix
    parameters:
      weight: 0.1
      density: 0.42
  - model: cgato/L3-TheSpice-8b-v0.8.3
    parameters:
      weight: 0.2
      density: 0.54
  - model: Sao10K/L3-8B-Stheno-v3.2
    parameters:
      weight: 0.2
      density: 0.66
  - model: Nitral-AI/Poppy_Porpoise-0.72-L3-8B
    parameters:
      weight: 0.3
      density: 0.78
merge_method: dare_ties
base_model: NousResearch/Meta-Llama-3-8B-Instruct
parameters:
  int8_mask: true
dtype: bfloat16

---

base_model:

• Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B
• v000000/L3-8B-Poppy-Moonfall-C

Prompt Template:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>

{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{output}<|eot_id|>