Saxo commited on
Commit
9085880
·
verified ·
1 Parent(s): 9d8b315

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +115 -6
README.md CHANGED
@@ -1,6 +1,11 @@
1
  ---
2
- library_name: transformers
 
 
 
 
3
  license: apache-2.0
 
4
  base_model: google/gemma-2-27b-it
5
  datasets:
6
  - Saxo/ko_cn_translation_tech_social_science_linkbricks_single_dataset
@@ -24,12 +29,102 @@ datasets:
24
  - youjunhyeok/ko-orca-pair-and-ultrafeedback-dpo
25
  - lilacai/glaive-function-calling-v2-sharegpt
26
  - kuotient/gsm8k-ko
27
- language:
28
- - ko
29
- - en
30
- - jp
31
- - cn
32
  pipeline_tag: text-generation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  ---
34
 
35
  # Model Card for Model ID
@@ -59,3 +154,17 @@ It is a model that has been trained to handle Korean-Chinese-English-Japanese cr
59
  <br><br>
60
 
61
  <a href="www.linkbricks.com">www.linkbricks.com</a>, <a href="www.linkbricks.vc">www.linkbricks.vc</a>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - ko
4
+ - en
5
+ - jp
6
+ - cn
7
  license: apache-2.0
8
+ library_name: transformers
9
  base_model: google/gemma-2-27b-it
10
  datasets:
11
  - Saxo/ko_cn_translation_tech_social_science_linkbricks_single_dataset
 
29
  - youjunhyeok/ko-orca-pair-and-ultrafeedback-dpo
30
  - lilacai/glaive-function-calling-v2-sharegpt
31
  - kuotient/gsm8k-ko
 
 
 
 
 
32
  pipeline_tag: text-generation
33
+ model-index:
34
+ - name: Linkbricks-Horizon-AI-Korean-Superb-27B
35
+ results:
36
+ - task:
37
+ type: text-generation
38
+ name: Text Generation
39
+ dataset:
40
+ name: IFEval (0-Shot)
41
+ type: HuggingFaceH4/ifeval
42
+ args:
43
+ num_few_shot: 0
44
+ metrics:
45
+ - type: inst_level_strict_acc and prompt_level_strict_acc
46
+ value: 77.68
47
+ name: strict accuracy
48
+ source:
49
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
50
+ name: Open LLM Leaderboard
51
+ - task:
52
+ type: text-generation
53
+ name: Text Generation
54
+ dataset:
55
+ name: BBH (3-Shot)
56
+ type: BBH
57
+ args:
58
+ num_few_shot: 3
59
+ metrics:
60
+ - type: acc_norm
61
+ value: 50.61
62
+ name: normalized accuracy
63
+ source:
64
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
65
+ name: Open LLM Leaderboard
66
+ - task:
67
+ type: text-generation
68
+ name: Text Generation
69
+ dataset:
70
+ name: MATH Lvl 5 (4-Shot)
71
+ type: hendrycks/competition_math
72
+ args:
73
+ num_few_shot: 4
74
+ metrics:
75
+ - type: exact_match
76
+ value: 26.96
77
+ name: exact match
78
+ source:
79
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
80
+ name: Open LLM Leaderboard
81
+ - task:
82
+ type: text-generation
83
+ name: Text Generation
84
+ dataset:
85
+ name: GPQA (0-shot)
86
+ type: Idavidrein/gpqa
87
+ args:
88
+ num_few_shot: 0
89
+ metrics:
90
+ - type: acc_norm
91
+ value: 14.65
92
+ name: acc_norm
93
+ source:
94
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
95
+ name: Open LLM Leaderboard
96
+ - task:
97
+ type: text-generation
98
+ name: Text Generation
99
+ dataset:
100
+ name: MuSR (0-shot)
101
+ type: TAUR-Lab/MuSR
102
+ args:
103
+ num_few_shot: 0
104
+ metrics:
105
+ - type: acc_norm
106
+ value: 19.53
107
+ name: acc_norm
108
+ source:
109
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
110
+ name: Open LLM Leaderboard
111
+ - task:
112
+ type: text-generation
113
+ name: Text Generation
114
+ dataset:
115
+ name: MMLU-PRO (5-shot)
116
+ type: TIGER-Lab/MMLU-Pro
117
+ config: main
118
+ split: test
119
+ args:
120
+ num_few_shot: 5
121
+ metrics:
122
+ - type: acc
123
+ value: 40.52
124
+ name: accuracy
125
+ source:
126
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
127
+ name: Open LLM Leaderboard
128
  ---
129
 
130
  # Model Card for Model ID
 
154
  <br><br>
155
 
156
  <a href="www.linkbricks.com">www.linkbricks.com</a>, <a href="www.linkbricks.vc">www.linkbricks.vc</a>
157
+
158
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
159
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/Saxo__Linkbricks-Horizon-AI-Korean-Superb-27B-details)
160
+
161
+ | Metric |Value|
162
+ |-------------------|----:|
163
+ |Avg. |38.32|
164
+ |IFEval (0-Shot) |77.68|
165
+ |BBH (3-Shot) |50.61|
166
+ |MATH Lvl 5 (4-Shot)|26.96|
167
+ |GPQA (0-shot) |14.65|
168
+ |MuSR (0-shot) |19.53|
169
+ |MMLU-PRO (5-shot) |40.52|
170
+