Update README.md
Browse files
README.md
CHANGED
|
@@ -13,6 +13,15 @@ language:
|
|
| 13 |
|
| 14 |
This is just above base gpt2 only because of truthfulqa score bringing the average up, it has a higher truthfulqa score than any base gpt2 model. It is also just under pythia-160m for average score (0.01%) and
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
| Task |Version| Metric |Value | |Stderr|
|
| 17 |
|-------------|------:|--------|-----:|---|-----:|
|
| 18 |
|arc_challenge| 0|acc |0.1741|± |0.0111|
|
|
|
|
| 13 |
|
| 14 |
This is just above base gpt2 only because of truthfulqa score bringing the average up, it has a higher truthfulqa score than any base gpt2 model. It is also just under pythia-160m for average score (0.01%) and
|
| 15 |
|
| 16 |
+
| model | avg | arc | hellaswag | mmlu | truthfulqa |
|
| 17 |
+
| --- | --- | --- | --- | --- | --- |
|
| 18 |
+
| cramp-41m | 30.57 | 21.76 | 27.35 | 25.53 | 47.66 |
|
| 19 |
+
| gpt2 (125m) | 30.06 | 22.1 | 31.6 | 25.86 | 40.67 |
|
| 20 |
+
| pythia 70m deduped | 30.25 | 21.08 | 27.17 | 25.26 | 47.51 |
|
| 21 |
+
| pythia 70m | 30.46 | 21.59 | 27.29 | 25.9 | 47.06 |
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
| Task |Version| Metric |Value | |Stderr|
|
| 26 |
|-------------|------:|--------|-----:|---|-----:|
|
| 27 |
|arc_challenge| 0|acc |0.1741|± |0.0111|
|