sjrhuschlee commited on
Commit
6da6e92
1 Parent(s): c82cfae

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -34
README.md CHANGED
@@ -151,7 +151,7 @@ model-index:
151
 
152
  # deberta-v3-base for Extractive QA
153
 
154
- This is the [deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) model, fine-tuned using the [SQuAD2.0](https://huggingface.co/datasets/squad_v2) dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering.
155
 
156
  ## Overview
157
  **Language model:** deberta-v3-base
@@ -199,41 +199,17 @@ answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
199
  # 'London'
200
  ```
201
 
202
- ## Metrics
203
 
204
- ```bash
205
- # Squad v2
206
- {
207
- "eval_HasAns_exact": 84.36234817813765,
208
- "eval_HasAns_f1": 90.09079905537246,
209
- "eval_HasAns_total": 5928,
210
- "eval_NoAns_exact": 74.61732548359966,
211
- "eval_NoAns_f1": 74.61732548359966,
212
- "eval_NoAns_total": 5945,
213
- "eval_best_exact": 79.45759285774446,
214
- "eval_best_exact_thresh": 0.0,
215
- "eval_best_f1": 82.31771724081922,
216
- "eval_best_f1_thresh": 0.0,
217
- "eval_exact": 79.48286027120358,
218
- "eval_f1": 82.34298465427844,
219
- "eval_runtime": 109.7262,
220
- "eval_samples": 11951,
221
- "eval_samples_per_second": 108.917,
222
- "eval_steps_per_second": 4.539,
223
- "eval_total": 11873
224
- }
225
 
226
- # Squad
227
- {
228
- "eval_exact": 85.89403973509934,
229
- "eval_f1": 91.2982923196374,
230
- "eval_runtime": 96.6499,
231
- "eval_samples": 10618,
232
- "eval_samples_per_second": 109.86,
233
- "eval_steps_per_second": 4.584,
234
- "eval_total": 10570
235
- }
236
- ```
237
 
238
  ## Training procedure
239
 
 
151
 
152
  # deberta-v3-base for Extractive QA
153
 
154
+ This is the [deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) model, fine-tuned using the SQuAD 2.0, MRQA, AdversarialQA, and SynQA datasets. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering.
155
 
156
  ## Overview
157
  **Language model:** deberta-v3-base
 
199
  # 'London'
200
  ```
201
 
202
+ ## Dataset Preparation
203
 
204
+ The MRQA dataset was updated to fix some errors and formatting to work with the `run_qa.py` example script provided in the Hugging Face Transformers library.
205
+ The changes included
206
+ - Updating incorrect answer starts locations (usually off by a few characters)
207
+ - Updating the answer text to match the text found in the context
208
+ The script used to process the MRQA dataset is provided in this repo at XXX.
209
+
210
+ ### MRQA
211
+ - The answer
 
 
 
 
 
 
 
 
 
 
 
 
 
212
 
 
 
 
 
 
 
 
 
 
 
 
213
 
214
  ## Training procedure
215