MaziyarPanahi leaderboard-pr-bot commited on
Commit
9918c0a
1 Parent(s): e1f70c3

Adding Evaluation Results (#10)

Browse files

- Adding Evaluation Results (6aaaa93f7005cc262c4d2caed59953ed42d0b1c9)


Co-authored-by: Open LLM Leaderboard PR Bot <leaderboard-pr-bot@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +106 -1
README.md CHANGED
@@ -15,7 +15,6 @@ tags:
15
  - code
16
  - chatml
17
  base_model: microsoft/Phi-3-mini-4k-instruct
18
- model_name: Phi-3-mini-4k-instruct-v0.3
19
  pipeline_tag: text-generation
20
  inference: false
21
  model_creator: MaziyarPanahi
@@ -123,6 +122,98 @@ model-index:
123
  source:
124
  url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/Phi-3-mini-4k-instruct-v0.3
125
  name: Open LLM Leaderboard
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
126
  ---
127
 
128
  <img src="./phi-3-instruct.webp" alt="Phi-3 Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
@@ -228,3 +319,17 @@ print(output[0]['generated_text'])
228
 
229
 
230
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  - code
16
  - chatml
17
  base_model: microsoft/Phi-3-mini-4k-instruct
 
18
  pipeline_tag: text-generation
19
  inference: false
20
  model_creator: MaziyarPanahi
 
122
  source:
123
  url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/Phi-3-mini-4k-instruct-v0.3
124
  name: Open LLM Leaderboard
125
+ - task:
126
+ type: text-generation
127
+ name: Text Generation
128
+ dataset:
129
+ name: IFEval (0-Shot)
130
+ type: HuggingFaceH4/ifeval
131
+ args:
132
+ num_few_shot: 0
133
+ metrics:
134
+ - type: inst_level_strict_acc and prompt_level_strict_acc
135
+ value: 49.26
136
+ name: strict accuracy
137
+ source:
138
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Phi-3-mini-4k-instruct-v0.3
139
+ name: Open LLM Leaderboard
140
+ - task:
141
+ type: text-generation
142
+ name: Text Generation
143
+ dataset:
144
+ name: BBH (3-Shot)
145
+ type: BBH
146
+ args:
147
+ num_few_shot: 3
148
+ metrics:
149
+ - type: acc_norm
150
+ value: 37.66
151
+ name: normalized accuracy
152
+ source:
153
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Phi-3-mini-4k-instruct-v0.3
154
+ name: Open LLM Leaderboard
155
+ - task:
156
+ type: text-generation
157
+ name: Text Generation
158
+ dataset:
159
+ name: MATH Lvl 5 (4-Shot)
160
+ type: hendrycks/competition_math
161
+ args:
162
+ num_few_shot: 4
163
+ metrics:
164
+ - type: exact_match
165
+ value: 2.95
166
+ name: exact match
167
+ source:
168
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Phi-3-mini-4k-instruct-v0.3
169
+ name: Open LLM Leaderboard
170
+ - task:
171
+ type: text-generation
172
+ name: Text Generation
173
+ dataset:
174
+ name: GPQA (0-shot)
175
+ type: Idavidrein/gpqa
176
+ args:
177
+ num_few_shot: 0
178
+ metrics:
179
+ - type: acc_norm
180
+ value: 9.06
181
+ name: acc_norm
182
+ source:
183
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Phi-3-mini-4k-instruct-v0.3
184
+ name: Open LLM Leaderboard
185
+ - task:
186
+ type: text-generation
187
+ name: Text Generation
188
+ dataset:
189
+ name: MuSR (0-shot)
190
+ type: TAUR-Lab/MuSR
191
+ args:
192
+ num_few_shot: 0
193
+ metrics:
194
+ - type: acc_norm
195
+ value: 7.75
196
+ name: acc_norm
197
+ source:
198
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Phi-3-mini-4k-instruct-v0.3
199
+ name: Open LLM Leaderboard
200
+ - task:
201
+ type: text-generation
202
+ name: Text Generation
203
+ dataset:
204
+ name: MMLU-PRO (5-shot)
205
+ type: TIGER-Lab/MMLU-Pro
206
+ config: main
207
+ split: test
208
+ args:
209
+ num_few_shot: 5
210
+ metrics:
211
+ - type: acc
212
+ value: 31.42
213
+ name: accuracy
214
+ source:
215
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/Phi-3-mini-4k-instruct-v0.3
216
+ name: Open LLM Leaderboard
217
  ---
218
 
219
  <img src="./phi-3-instruct.webp" alt="Phi-3 Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
 
319
 
320
 
321
  ```
322
+
323
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
324
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_MaziyarPanahi__Phi-3-mini-4k-instruct-v0.3)
325
+
326
+ | Metric |Value|
327
+ |-------------------|----:|
328
+ |Avg. |23.38|
329
+ |IFEval (0-Shot) |49.26|
330
+ |BBH (3-Shot) |37.66|
331
+ |MATH Lvl 5 (4-Shot)| 2.95|
332
+ |GPQA (0-shot) | 9.06|
333
+ |MuSR (0-shot) | 7.75|
334
+ |MMLU-PRO (5-shot) |31.42|
335
+