A Fishy Model
This model was trained with SFT using Unsloth on the ChatML format with 8k context. Carp models are trained with a combination of pretrain, instruct, and chat datasets.
Changes
- Training dataset had some "slop" and refusals removed.
- Datasets were reformatted.
Uploaded model
- Developed by: TheTsar1209
- License: apache-2.0
- Finetuned from model : unsloth/Qwen2.5-14B-Instruct-bnb-4bit
This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 35.67 |
| IFEval (0-Shot) | 72.02 |
| BBH (3-Shot) | 49.38 |
| MATH Lvl 5 (4-Shot) | 17.37 |
| GPQA (0-shot) | 13.65 |
| MuSR (0-shot) | 15.55 |
| MMLU-PRO (5-shot) | 46.04 |
- Downloads last month
- 19
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
Model tree for TheTsar1209/qwen-carpmuscle-v0.4
Base model
Qwen/Qwen2.5-14B
Finetuned
Qwen/Qwen2.5-14B-Instruct
Quantized
unsloth/Qwen2.5-14B-Instruct-bnb-4bit
Evaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard72.020
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard49.380
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard17.370
- acc_norm on GPQA (0-shot)Open LLM Leaderboard13.650
- acc_norm on MuSR (0-shot)Open LLM Leaderboard15.550
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard46.040