PlatYi-34B-200k-Q-FastChat
Model Details
Model Developers Kyujin Han (kyujinpy)
Input Models input text only.
Output Models generate text only.
Model Architecture
PlatYi-34B-200k-Q-FastChat is an auto-regressive language model based on the Yi-34B transformer architecture.
Blog Link
Blog: [Coming soon...]
Github: [Coming soon...]
Base Model
01-ai/Yi-34B-200K
Training Dataset
garage-bAInd/Open-Platypus.
Notice
While training, I used QLoRA.lora_r
values is 64.
Apply prompting
References by FastChat.
Model Benchmark
Open leaderboard
- Follow up as link.
Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
---|---|---|---|---|---|---|---|
PlatYi-34B-200k-Q-FastChat | 67.85 | 64.93 | 84.46 | 77.13 | 48.38 | 80.74 | 51.48 |
PlatYi-34B-Llama-Q-FastChat | 68.31 | 66.31 | 85.25 | 78.37 | 53.62 | 82.16 | 44.35 |
Yi-34B | 69.42 | 64.59 | 85.69 | 76.35 | 56.23 | 83.03 | 50.64 |
Implementation Code
### KO-Platypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "kyujinpy/PlatYi-34B-200k-Q-FastChat"
OpenOrca = AutoModelForCausalLM.from_pretrained(
repo,
return_dict=True,
torch_dtype=torch.float16,
device_map='auto'
)
OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 67.85 |
AI2 Reasoning Challenge (25-Shot) | 64.93 |
HellaSwag (10-Shot) | 84.46 |
MMLU (5-Shot) | 77.13 |
TruthfulQA (0-shot) | 48.38 |
Winogrande (5-shot) | 80.74 |
GSM8k (5-shot) | 51.48 |
- Downloads last month
- 1,289
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.
Model tree for kyujinpy/PlatYi-34B-200k-Q-FastChat
Dataset used to train kyujinpy/PlatYi-34B-200k-Q-FastChat
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard64.930
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard84.460
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard77.130
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard48.380
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard80.740
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard51.480