Sakura-SOLAR
Collection
Global LLM Leaderboard Rank1 (2023.12.28)
β’
6 items
β’
Updated
(μ£Ό)λ―Έλμ΄κ·Έλ£Ήμ¬λκ³Όμ²κ³Ό (μ£Ό)λ§μ»€μ LLM μ°κ΅¬ 컨μμμμμ κ°λ°λ λͺ¨λΈμ λλ€
Model Developers Kyujin Han (kyujinpy)
Method
Using Mergekit.
I shared the information about my model. (training and code)
Please see: βSakura-SOLAR.
Blog
Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
---|---|---|---|---|---|---|---|
Sakura-SOLRCA-Instruct-DPO | 74.05 | 71.16 | 88.49 | 66.17 | 72.10 | 82.95 | 63.46 |
Sakura-SOLAR-Instruct-DPO-v2 | 74.14 | 70.90 | 88.41 | 66.48 | 71.86 | 83.43 | 63.76 |
kyujinpy/Sakura-SOLAR-Instruct | 74.40 | 70.99 | 88.42 | 66.33 | 71.79 | 83.66 | 65.20 |
Rank1 2023.12.27 PM 11:50
### KO-Platypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "kyujinpy/Sakura-SOLAR-Instruct"
OpenOrca = AutoModelForCausalLM.from_pretrained(
repo,
return_dict=True,
torch_dtype=torch.float16,
device_map='auto'
)
OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 74.40 |
AI2 Reasoning Challenge (25-Shot) | 70.99 |
HellaSwag (10-Shot) | 88.42 |
MMLU (5-Shot) | 66.33 |
TruthfulQA (0-shot) | 71.79 |
Winogrande (5-shot) | 83.66 |
GSM8k (5-shot) | 65.20 |