DiscoResearch/Llama3_German_8B_v0.1 is a large language model based on Meta's Llama3-8B. It is specialized for the German language through continuous pretraining on 65 billion high-quality tokens, similar to previous LeoLM or Occiglot models.
This is a merge of our instruct model with the Instruct model by Meta. Created using mergekit. Contributed by Damian B.!
This model was merged using the DARE TIES merge method using meta-llama/Meta-Llama-3-8B as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
models:
- model: DiscoResearch/Llama3_DiscoLeo_Instruct_8B_v0.1
parameters:
density: 0.5
weight: 0.5
- model: meta-llama/Meta-Llama-3-8B-Instruct
parameters:
density: 0.5
weight: 0.5
merge_method: dare_ties
base_model: meta-llama/Meta-Llama-3-8B
parameters:
normalize: true
int8_mask: false
dtype: bfloat16