Update README.md
Browse files
README.md
CHANGED
@@ -32,11 +32,19 @@ Other training configuration:
|
|
32 |
|
33 |
Evaluation Results on GLUE after pretraining (30k steps):
|
34 |
|
35 |
-
| Model |
|
36 |
-
|
37 |
-
| RoBERTa-medium (ours) |
|
38 |
-
| [BERT-medium](https://huggingface.co/google/bert_uncased_L-8_H-512_A-8) |
|
39 |
|
40 |
Evaluation Scores Curve (AVG of scores) during pretraining:
|
41 |
|
42 |
-
![image/png](https://cdn-uploads.huggingface.co/production/uploads/62927c2e56fedc76e396b3ca/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
32 |
|
33 |
Evaluation Results on GLUE after pretraining (30k steps):
|
34 |
|
35 |
+
| Model | MRPC-F1 | STS-B-Pearson | SST-2-Acc | QQP-F1 | MNLI-m | MNLI-mm | QNLI-Acc | WNLI-Acc | RTE-Acc |
|
36 |
+
|----------------|---------|---------------|-----------|--------|--------|---------|----------|----------|---------|
|
37 |
+
| RoBERTa-medium (ours) | 83.6 | 82.7 | 89.7 | 89.0 | 79.7 | 80.1 | 89.3 | 31.0 | 57.4 |
|
38 |
+
| [BERT-medium](https://huggingface.co/google/bert_uncased_L-8_H-512_A-8) | 86.3 | 87.7 | 88.9 | 89.4 | 80.6 | 81.0 | 89.2 | 29.6 | 63.9 |
|
39 |
|
40 |
Evaluation Scores Curve (AVG of scores) during pretraining:
|
41 |
|
42 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/62927c2e56fedc76e396b3ca/p6KK_6-oXCyhAjdZh1WQ_.png)
|
43 |
+
|
44 |
+
For both stats above we don't report CoLA scores as it's pretty unstable. The raw CoLA scores are:
|
45 |
+
|
46 |
+
| Task | 1500 | 3000 | 6000 | 9000 | 13500 | 18000 | 24000 | 30000 |
|
47 |
+
|----------|----------|----------|----------|----------|----------|----------|----------|----------|
|
48 |
+
CoLA | 1.7 | 13.5 | 29.2 | 31.4 | 31.1 | 24.1 | 29.0 | 20.0 |
|
49 |
+
|
50 |
+
|