README.md

---
base_model:
- SanjiWatsuki/Silicon-Maid-7B
- Guilherme34/Samantha-v2
- jan-hq/stealth-v1.3
- mitultiwari/mistral-7B-instruct-dpo
- senseable/WestLake-7B-v2
library_name: transformers
tags:
- mergekit
- merge

---
# Aika-7B

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

<p align="center">
  <img src="https://huggingface.co/sethuiyer/Aika-7B/resolve/main/aika.webp" height="128px" alt="Aika">
</p>


This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [mitultiwari/mistral-7B-instruct-dpo](https://huggingface.co/mitultiwari/mistral-7B-instruct-dpo) as a base.

### Models Merged

The following models were included in the merge:
* [SanjiWatsuki/Silicon-Maid-7B](https://huggingface.co/SanjiWatsuki/Silicon-Maid-7B)
* [Guilherme34/Samantha-v2](https://huggingface.co/Guilherme34/Samantha-v2)
* [jan-hq/stealth-v1.3](https://huggingface.co/jan-hq/stealth-v1.3)
* [senseable/WestLake-7B-v2](https://huggingface.co/senseable/WestLake-7B-v2)

### Configuration

The following YAML configuration was used to produce this model:

```yaml

base_model: mitultiwari/mistral-7B-instruct-dpo
dtype: bfloat16
merge_method: dare_ties
models:
- model: mitultiwari/mistral-7B-instruct-dpo
- model: senseable/WestLake-7B-v2
  parameters:
    density: 0.75
    weight: 0.30
- model: SanjiWatsuki/Silicon-Maid-7B
  parameters:
    density: 0.75
    weight: 0.30
- model: jan-hq/stealth-v1.3
  parameters:
    density: 0.85
    weight: 0.30
- model: Guilherme34/Samantha-v2
  parameters:
    density: 0.65
    weight: 0.10
parameters:
  int8_mask: true

```