---
license: cc-by-nc-4.0
datasets:
- teknium/openhermes
---

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6324eabf05bd8a54c6eb1650/A_cMQgKaChl6Q9Vf6E3yM.png)

|             Task              |Version| Metric |Value |   |Stderr|
|-------------------------------|------:|--------|-----:|---|-----:|
|hendrycksTest-logical_fallacies|      1|acc     |0.3067|±  |0.0362|
|                               |       |acc_norm|**0.3067**|±  |0.0362|
|hendrycksTest-global_facts     |      1|acc     |   0.3|±  |0.0461|
|                               |       |acc_norm|   0.3|±  |0.0461|
|hendrycksTest-abstract_algebra |      1|acc     |0.2700|±  |0.0446|
|                               |       |acc_norm|**0.2700**|±  |0.0446|
|hendrycksTest-college_chemistry|      1|acc     |0.3100|±  |0.0465|
|                               |       |acc_norm|**0.3100**|±  |0.0465|
|hendrycksTest-college_physics  |      1|acc     |0.2157|±  |0.0409|
|                               |       |acc_norm|**0.2157**|±  |0.0409|
|hendrycksTest-formal_logic     |      1|acc     |0.2857|±  |0.0404|
|                               |       |acc_norm|**0.2857**|±  |0.0404|

Compared to TinyLlama-1.1B-Chat-v1.0:

Algebra UP **17.4%**

Formal Logic UP **24.2%**

Logical Fallacies UP **35.4%**

Template Format: **Alpaca**

It took 4 hours to train in 1 epoch with an RTX 3090.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6324eabf05bd8a54c6eb1650/W4r8X1lzg6-OS1T-dd_t8.png)