Dhia-GB commited on
Commit
7e687ae
1 Parent(s): 28519b8

Update README.md

Browse files

clarification about evaluation pipeline.

Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -91,7 +91,10 @@ print(response)
91
  <br>
92
 
93
  ## Benchmarks
94
- We report in the following table our internal pipeline benchmarks:
 
 
 
95
 
96
  <table border="1" style="width: 100%; text-align: center; border-collapse: collapse;">
97
  <colgroup>
@@ -228,6 +231,7 @@ We report in the following table our internal pipeline benchmarks:
228
  </tbody>
229
  </table>
230
 
 
231
  ## Technical Report
232
  Coming soon....
233
 
 
91
  <br>
92
 
93
  ## Benchmarks
94
+ We report in the following table our internal pipeline benchmarks.
95
+ - We use [lm-evaluation harness](https://github.com/EleutherAI/lm-evaluation-harness).
96
+ - We report **raw scores** obtained by applying chat template **without fewshot_as_multiturn** (unlike Llama3.1).
97
+ - We use same batch-size across all models.
98
 
99
  <table border="1" style="width: 100%; text-align: center; border-collapse: collapse;">
100
  <colgroup>
 
231
  </tbody>
232
  </table>
233
 
234
+
235
  ## Technical Report
236
  Coming soon....
237