CarrotAI/OpenCarrot-Mix-7B

output

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the linear merge method.

Models Merged

The following models were included in the merge:

• amazingvince/Not-WizardLM-2-7B
• CarrotAI/OpenCarrot-Mistral-7B-Instruct-v0.2

Score

openai/gpt-4 : 0.6158
gemini-pro: 0.515
OpenCarrot-Mix-7B (this) : 0.4425
mistralai/Mixtral-8x7B-Instruct-v0.1 : 0.4304
openai/gpt-3.5-turbo : 0.4217

평가 지표	점수
AVG_llm_kr_eval	0.4425
EL	0.0522
FA	0.0865
NLI	0.6700
QA	0.5100
RC	0.8937
klue_ner_set_f1	0.0944
klue_re_exact_match	0.0100
kmmlu_preview_exact_match	0.4000
kobest_copa_exact_match	0.8200
kobest_hs_exact_match	0.5500
kobest_sn_exact_match	0.9800
kobest_wic_exact_match	0.6200
korea_cg_bleu	0.0865
kornli_exact_match	0.6400
korsts_pearson	0.8547
korsts_spearman	0.8464

LogicKor

카테고리	싱글 점수 평균	멀티 점수 평균
코딩(Coding)	7.71	7.71
수학(Math)	5.57	3.86
이해(Understanding)	6.86	8.14
추론(Reasoning)	8.14	6.43
글쓰기(Writing)	8.71	6.86
문법(Grammar)	5.29	2.29

카테고리	싱글 점수 평균	멀티 점수 평균
전체 싱글	7.05	5.88

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: amazingvince/Not-WizardLM-2-7B
    parameters:
      weight: 1.0
  - model: CarrotAI/OpenCarrot-Mistral-7B-Instruct-v0.2
    parameters:
      weight: 0.5
merge_method: linear
dtype: float16