---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- mergekit
- merge
base_model:
- arcee-ai/Virtuoso-Small
- Qwen2.5-14B-Qwenvergence-model_stock
- sometimesanotion/Qwen2.5-14B-Qwenvergence-model_stock
metrics:
- accuracy
pipeline_tag: text-generation
new_version: sometimesanotion/Lamarck-14B-v0.5
---
![Lamarck.webp](https://huggingface.co/sometimesanotion/Lamarck-14B-v0.4-Qwenvergence/resolve/main/Lamarck.webp)
---

Lamarck 14B v0.4 Qwenvergence:  it's a big step up for Lamarck in terms of quality.  All the same ingredients are involved as in previous releases of Lamarck; they are more effectively combined.  This model features slightly improved reasoning from 0.3, but the multi-language and prose are greatly improved.

## Merge Details

This model was initialized from model_stock, and refined from there.  No fine-tuning, or use of models apart from those listed as the contents of Qwen2.5-14B-Qwenvergence-model_stock except for a very mild application of [huihui-ai/Qwen2.5-14B-Instruct-abliterated-v2](https://huggingface.co/huihui-ai/Qwen2.5-14B-Instruct-abliterated-v2) which does not seem to have removed most of the refusals.

### Models Merged

**Top influences:** These ancestors are in the Qwenvergence model_stock, reinforced in later steps:

- **[arcee-ai/Virtuoso-Small](https://huggingface.co/arcee-ai/Virtuoso-Small)** - A brand new model from Arcee, refined from the notable cross-architecture Llama-to-Qwen distillation [arcee-ai/SuperNova-Medius](https://huggingface.co/arcee-ai/SuperNova-Medius).  The first two layers are nearly exclusively from Virtuoso.  It has proven to be a well-rounded performer, and contributes a noticeable boost to the model's prose quality.

- **[CultriX/SeQwence-14B-EvolMerge](http://huggingface.co/CultriX/SeQwence-14B-EvolMerge)** - A top contender on reasoning benchmarks.  

- **[VAGOsolutions/SauerkrautLM-v2-14b-DPO](https://huggingface.co/VAGOsolutions/SauerkrautLM-v2-14b-DPO)** - This model's influence is understated, but aids BBH and coding capability.

- **[v000000/Qwen2.5-Lumen-14B](https://huggingface.co/v000000/Qwen2.5-Lumen-14B)** - A leading influence for prose quality.

**Prose added:**

The prose quality has taken a leap, no doubt also to the way [EVA-UNIT-01/EVA-Qwen2.5-14B-v0.2](https://huggingface.co/EVA-UNIT-01/EVA-Qwen2.5-14B-v0.2), [sthenno-com/miscii-14b-1028](https://huggingface.co/sthenno-com/miscii-14b-1028), [oxyapi/oxy-1-small](https://huggingface.co/oxyapi/oxy-1-small), and [underwoods/medius-erebus-magnum-14b](https://huggingface.co/underwoods/medius-erebus-magnum-14b) were applied.

### Configuration

The following YAML configurations were used to initialize and finalize this model:

```yaml
name:                Qwenvergence-model_stock
merge_method:        model_stock
base_model:          Qwen/Qwen2.5-14B
tokenizer_source:    base
parameters:
  int8_mask:         true
  normalize:         true
  rescale:           false
models:
  - model:           allura-org/TQ2.5-14B-Sugarquill-v1
  - model:           oxyapi/oxy-1-small
  - model:           sthenno-com/miscii-14b-1028
  - model:           underwoods/medius-erebus-magnum-14b
  - model:           EVA-UNIT-01/EVA-Qwen2.5-14B-v0.2
  - model:           CultriX/SeQwence-14B-EvolMerge
  - model:           arcee-ai/Virtuoso-Small
  - model:           VAGOsolutions/SauerkrautLM-v2-14b-DPO
  - model:           v000000/Qwen2.5-Lumen-14B
dtype:               bfloat16
out_dtype:           bfloat16
---
# Experimental merge methods involving above models
---
name:                Lamarck-14B-v0.4-Qwenvergence
merge_method:        ties
base_model:          sometimesanotion/lamarck-14b-base
tokenizer_source:    base
parameters:         
  density:           1.00
  weight:            1.00
  int8_mask:         true
  normalize:         true
  rescale:           false
models:
  - model:           merges/Qwen2.5-14B-Qwenvergence-slerp
    parameters:
      weight:        1.00
      density:       1.00
  - model:           arcee-ai/Virtuoso-Small
    parameters:
      weight:        1.00
      density:       1.00

```