metadata
license: apache-2.0
language:
- ko
- en
library_name: transformers
datasets:
- davidkim205/kollm-converations
- davidkim205/kollm-comparision
just for practice
- trained by transformers, peft, bitsandbytes
- evaluated by https://github.com/davidkim205/kollm_evaluation
base model
- beomi/OPEN-SOLAR-KO-10.7B
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| kobest_wic | 1 | none | 0 | acc | 0.4873 | ± | 0.0141 |
| none | 0 | f1 | 0.3276 | ± | N/A | ||
| kobest_sentineg | 1 | none | 0 | acc | 0.6499 | ± | 0.0240 |
| none | 0 | f1 | 0.6134 | ± | N/A | ||
| kobest_hellaswag | 1 | none | 0 | acc | 0.4980 | ± | 0.0224 |
| none | 0 | f1 | 0.4952 | ± | N/A | ||
| none | 0 | acc_norm | 0.6280 | ± | 0.0216 | ||
| kobest_copa | 1 | none | 0 | acc | 0.8030 | ± | 0.0126 |
| none | 0 | f1 | 0.8026 | ± | N/A | ||
| kobest_boolq | 1 | none | 0 | acc | 0.8711 | ± | 0.0089 |
| none | 0 | f1 | 0.8710 | ± | N/A | ||
| ko_truthfulqa | 2 | none | 0 | acc | 0.2717 | ± | 0.0156 |
| ko_hellaswag | 1 | none | 0 | acc | 0.4109 | ± | 0.0049 |
| none | 0 | acc_norm | 0.5503 | ± | 0.0050 | ||
| ko_common_gen | 1 | none | 0 | acc | 0.9041 | ± | 0.0075 |
| none | 0 | acc_norm | 0.9041 | ± | 0.0075 | ||
| ko_arc_easy | 1 | none | 0 | acc | 0.3490 | ± | 0.0139 |
| none | 0 | acc_norm | 0.4326 | ± | 0.0145 |