---
base_model: []
library_name: transformers
tags:
- mergekit
- merge
- llama 3
- 70b
- arimas
- story
- roleplay
- rp
---

# EXL2 quants of [ryzen88/Llama-3-70b-Arimas-story-RP-V1.6](https://huggingface.co/ryzen88/Llama-3-70b-Arimas-story-RP-V1.6)

[3.00 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-3.0bpw-h6-exl2)  
[3.50 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-3.5bpw-h6-exl2)    
[4.00 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-4.0bpw-h6-exl2)  
[4.50 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-4.5bpw-h6-exl2)  
[6.00 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-6.0bpw-h6-exl2)  
[8.00 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-8.0bpw-h8-exl2)  

Created using the defaults from exllamav2 1.4.0 convert.py  
3.0bpw to 6.0bpw head bits = 6  
8.0bpw head bits = 8  
length = 8192  
dataset rows = 200  
measurement rows = 32  
measurement length = 8192  
  
  
# model
Llama-3-70b-Arimas-story-RP-V1.6

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
I Greatly expanded the amount of models used in this merge, experimented a lot with different idea's.
This version feels a lot more convincing than V1.5 Hopefully the long context window will also remain strong after Quants.
Because of the many merges switched back from BFloat to Float.
Tried breadcrums without the Ties, that went very poorly.

### Merge Method

This model was merged using the breadcrumbs_ties merge method using I:\Llama-3-70B-Instruct-Gradient-262k as a base.

### Models Merged

The following models were included in the merge:
* \Smaug-Llama-3-70B-Instruct
* \Meta-LLama-3-Cat-Smaug-LLama-70b
* \Meta-LLama-3-Cat-A-LLama-70b
* \Llama-3-70B-Synthia-v3.5
* \Llama-3-70B-Instruct-Gradient-524k
* \Llama-3-70B-Instruct-Gradient-262k
* \Tess-2.0-Llama-3-70B-v0.2
* \Llama-3-Lumimaid-70B-v0.1-alt

### Configuration

The following YAML configuration was used to produce this model:

```yaml
models:
  - model: \Llama-3-70B-Instruct-Gradient-262k
    parameters:
      weight: 0.25
      density: 0.90
      gamma: 0.01
  - model: \Meta-LLama-3-Cat-Smaug-LLama-70b
    parameters:
      weight: 0.28
      density: 0.90
      gamma: 0.01
  - model: \Llama-3-Lumimaid-70B-v0.1-alt
    parameters:
      weight: 0.15
      density: 0.90
      gamma: 0.01
  - model: \Tess-2.0-Llama-3-70B-v0.2
    parameters:
      weight: 0.06
      density: 0.90
      gamma: 0.01
  - model: \Smaug-Llama-3-70B-Instruct
    parameters:
      weight: 0.04
      density: 0.90
      gamma: 0.01
  - model: \Llama-3-70B-Synthia-v3.5
    parameters:
      weight: 0.05
      density: 0.90
      gamma: 0.01
  - model: \Llama-3-70B-Instruct-Gradient-524k
    parameters:
      weight: 0.03
      density: 0.90
      gamma: 0.01
  - model: \Meta-LLama-3-Cat-A-LLama-70b
    parameters:
      weight: 0.14
      density: 0.90
      gamma: 0.01
merge_method: breadcrumbs_ties
base_model: I:\Llama-3-70B-Instruct-Gradient-262k
dtype: float16
```