sjrhuschlee commited on
Commit
935ce55
1 Parent(s): 51e5707

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -0
README.md CHANGED
@@ -134,6 +134,41 @@ answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
134
  # 'London'
135
  ```
136
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
137
  ### Using with Peft
138
  **NOTE**: This requires code in the PR https://github.com/huggingface/peft/pull/473 for the PEFT library.
139
  ```python
 
134
  # 'London'
135
  ```
136
 
137
+ ## Metrics
138
+
139
+ ```bash
140
+ # Squad v2
141
+ {
142
+ "eval_HasAns_exact": 84.83468286099865,
143
+ "eval_HasAns_f1": 90.48374860633226,
144
+ "eval_HasAns_total": 5928,
145
+ "eval_NoAns_exact": 91.0681244743482,
146
+ "eval_NoAns_f1": 91.0681244743482,
147
+ "eval_NoAns_total": 5945,
148
+ "eval_best_exact": 87.95586625115808,
149
+ "eval_best_exact_thresh": 0.0,
150
+ "eval_best_f1": 90.77635490089573,
151
+ "eval_best_f1_thresh": 0.0,
152
+ "eval_exact": 87.95586625115808,
153
+ "eval_f1": 90.77635490089592,
154
+ "eval_runtime": 623.1333,
155
+ "eval_samples": 11951,
156
+ "eval_samples_per_second": 19.179,
157
+ "eval_steps_per_second": 0.799,
158
+ "eval_total": 11873
159
+ }
160
+
161
+ # Squad
162
+ {
163
+ "eval_exact_match": 89.29044465468307,
164
+ "eval_f1": 94.9846365606959,
165
+ "eval_runtime": 553.7132,
166
+ "eval_samples": 10618,
167
+ "eval_samples_per_second": 19.176,
168
+ "eval_steps_per_second": 0.8
169
+ }
170
+ ```
171
+
172
  ### Using with Peft
173
  **NOTE**: This requires code in the PR https://github.com/huggingface/peft/pull/473 for the PEFT library.
174
  ```python