|
--- |
|
license: apache-2.0 |
|
language: |
|
- ko |
|
- en |
|
library_name: transformers |
|
datasets: |
|
- davidkim205/kollm-converations |
|
- davidkim205/kollm-comparision |
|
--- |
|
|
|
just for practice |
|
- trained by transformers, peft, bitsandbytes |
|
- evaluated by https://github.com/davidkim205/kollm_evaluation |
|
|
|
base model |
|
- beomi/OPEN-SOLAR-KO-10.7B |
|
|
|
| Tasks |Version|Filter|n-shot| Metric |Value | |Stderr| |
|
|----------------|------:|------|-----:|--------|-----:|---|------| |
|
|kobest_wic | 1|none | 0|acc |0.4873|± |0.0141| |
|
| | |none | 0|f1 |0.3276|± |N/A | |
|
|kobest_sentineg | 1|none | 0|acc |0.6499|± |0.0240| |
|
| | |none | 0|f1 |0.6134|± |N/A | |
|
|kobest_hellaswag| 1|none | 0|acc |0.4980|± |0.0224| |
|
| | |none | 0|f1 |0.4952|± |N/A | |
|
| | |none | 0|acc_norm|0.6280|± |0.0216| |
|
|kobest_copa | 1|none | 0|acc |0.8030|± |0.0126| |
|
| | |none | 0|f1 |0.8026|± |N/A | |
|
|kobest_boolq | 1|none | 0|acc |0.8711|± |0.0089| |
|
| | |none | 0|f1 |0.8710|± |N/A | |
|
|ko_truthfulqa | 2|none | 0|acc |0.2717|± |0.0156| |
|
|ko_hellaswag | 1|none | 0|acc |0.4109|± |0.0049| |
|
| | |none | 0|acc_norm|0.5503|± |0.0050| |
|
|ko_common_gen | 1|none | 0|acc |0.9041|± |0.0075| |
|
| | |none | 0|acc_norm|0.9041|± |0.0075| |
|
|ko_arc_easy | 1|none | 0|acc |0.3490|± |0.0139| |
|
| | |none | 0|acc_norm|0.4326|± |0.0145| |
