leaderboard-pr-bot commited on
Commit
0962ff8
1 Parent(s): 9a481b0

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +32 -1
README.md CHANGED
@@ -18,6 +18,9 @@ model-index:
18
  args:
19
  num_few_shot: 0
20
  metrics:
 
 
 
21
  - type: inst_level_strict_acc and prompt_level_strict_acc
22
  value: 59.76
23
  name: strict accuracy
@@ -33,6 +36,9 @@ model-index:
33
  args:
34
  num_few_shot: 3
35
  metrics:
 
 
 
36
  - type: acc_norm
37
  value: 24.06
38
  name: normalized accuracy
@@ -48,6 +54,9 @@ model-index:
48
  args:
49
  num_few_shot: 4
50
  metrics:
 
 
 
51
  - type: exact_match
52
  value: 6.8
53
  name: exact match
@@ -63,6 +72,9 @@ model-index:
63
  args:
64
  num_few_shot: 0
65
  metrics:
 
 
 
66
  - type: acc_norm
67
  value: 7.61
68
  name: acc_norm
@@ -78,6 +90,9 @@ model-index:
78
  args:
79
  num_few_shot: 0
80
  metrics:
 
 
 
81
  - type: acc_norm
82
  value: 11.78
83
  name: acc_norm
@@ -95,6 +110,9 @@ model-index:
95
  args:
96
  num_few_shot: 5
97
  metrics:
 
 
 
98
  - type: acc
99
  value: 25.44
100
  name: accuracy
@@ -204,4 +222,17 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
204
  primaryClass={cs.CL},
205
  url={https://arxiv.org/abs/2401.02415},
206
  }
207
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  args:
19
  num_few_shot: 0
20
  metrics:
21
+ - type: inst_level_strict_acc and prompt_level_strict_acc
22
+ value: 59.76
23
+ name: strict accuracy
24
  - type: inst_level_strict_acc and prompt_level_strict_acc
25
  value: 59.76
26
  name: strict accuracy
 
36
  args:
37
  num_few_shot: 3
38
  metrics:
39
+ - type: acc_norm
40
+ value: 24.06
41
+ name: normalized accuracy
42
  - type: acc_norm
43
  value: 24.06
44
  name: normalized accuracy
 
54
  args:
55
  num_few_shot: 4
56
  metrics:
57
+ - type: exact_match
58
+ value: 6.8
59
+ name: exact match
60
  - type: exact_match
61
  value: 6.8
62
  name: exact match
 
72
  args:
73
  num_few_shot: 0
74
  metrics:
75
+ - type: acc_norm
76
+ value: 7.61
77
+ name: acc_norm
78
  - type: acc_norm
79
  value: 7.61
80
  name: acc_norm
 
90
  args:
91
  num_few_shot: 0
92
  metrics:
93
+ - type: acc_norm
94
+ value: 11.78
95
+ name: acc_norm
96
  - type: acc_norm
97
  value: 11.78
98
  name: acc_norm
 
110
  args:
111
  num_few_shot: 5
112
  metrics:
113
+ - type: acc
114
+ value: 25.44
115
+ name: accuracy
116
  - type: acc
117
  value: 25.44
118
  name: accuracy
 
222
  primaryClass={cs.CL},
223
  url={https://arxiv.org/abs/2401.02415},
224
  }
225
+ ```
226
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
227
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Pretergeek__OpenChat-3.5-0106_8.11B-36Layers-Last)
228
+
229
+ | Metric |Value|
230
+ |-------------------|----:|
231
+ |Avg. |22.57|
232
+ |IFEval (0-Shot) |59.76|
233
+ |BBH (3-Shot) |24.06|
234
+ |MATH Lvl 5 (4-Shot)| 6.80|
235
+ |GPQA (0-shot) | 7.61|
236
+ |MuSR (0-shot) |11.78|
237
+ |MMLU-PRO (5-shot) |25.44|
238
+