Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,37 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
| 3 |
+
datasets:
|
| 4 |
+
- teknium/openhermes
|
| 5 |
---
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
| Task |Version| Metric |Value | |Stderr|
|
| 10 |
+
|-------------------------------|------:|--------|-----:|---|-----:|
|
| 11 |
+
|hendrycksTest-logical_fallacies| 1|acc |0.3067|± |0.0362|
|
| 12 |
+
| | |acc_norm|**0.3067**|± |0.0362|
|
| 13 |
+
|hendrycksTest-global_facts | 1|acc | 0.3|± |0.0461|
|
| 14 |
+
| | |acc_norm| 0.3|± |0.0461|
|
| 15 |
+
|hendrycksTest-abstract_algebra | 1|acc |0.2700|± |0.0446|
|
| 16 |
+
| | |acc_norm|**0.2700**|± |0.0446|
|
| 17 |
+
|hendrycksTest-college_chemistry| 1|acc |0.3100|± |0.0465|
|
| 18 |
+
| | |acc_norm|**0.3100**|± |0.0465|
|
| 19 |
+
|hendrycksTest-college_physics | 1|acc |0.2157|± |0.0409|
|
| 20 |
+
| | |acc_norm|**0.2157**|± |0.0409|
|
| 21 |
+
|hendrycksTest-formal_logic | 1|acc |0.2857|± |0.0404|
|
| 22 |
+
| | |acc_norm|**0.2857**|± |0.0404|
|
| 23 |
+
|
| 24 |
+
Compared to TinyLlama-1.1B-Chat-v1.0:
|
| 25 |
+
|
| 26 |
+
Algebra UP **17.4%**
|
| 27 |
+
|
| 28 |
+
Formal Logic UP **24.2%**
|
| 29 |
+
|
| 30 |
+
Logical Fallacies UP **35.4%**
|
| 31 |
+
|
| 32 |
+
Template Format: **Alpaca**
|
| 33 |
+
|
| 34 |
+
It took 4 hours to train in 1 epoch with an RTX 3090.
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
|