Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,46 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
datasets:
|
| 4 |
+
- meta-math/MetaMathQA
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
---
|
| 8 |
+
|
| 9 |
+
Supervised Finetuning the phi1.5 on MetaMathQA datasets. The results are as follows:
|
| 10 |
+
|
| 11 |
+
| Model | GSM8k Pass@1 | MATH Pass@1 |
|
| 12 |
+
|---------------------|--------------|-------------|
|
| 13 |
+
| MPT-7B | 6.8 | 3.0 |
|
| 14 |
+
| Falcon-7B | 6.8 | 2.3 |
|
| 15 |
+
| LLaMA-1-7B | 11.0 | 2.9 |
|
| 16 |
+
| LLaMA-2-7B | 14.6 | 2.5 |
|
| 17 |
+
| MPT-30B | 15.2 | 3.1 |
|
| 18 |
+
| LLaMA-1-13B | 17.8 | 3.9 |
|
| 19 |
+
| GPT-Neo-2.7B | 19.5 | -- |
|
| 20 |
+
| Falcon-40B | 19.6 | 2.5 |
|
| 21 |
+
| Baichuan-chat-13B | 23.9 | -- |
|
| 22 |
+
| Vicuna-v1.3-13B | 27.6 | -- |
|
| 23 |
+
| LLaMA-2-13B | 28.7 | 3.9 |
|
| 24 |
+
| InternLM-7B | 31.2 | -- |
|
| 25 |
+
| ChatGLM-2-6B | 32.4 | -- |
|
| 26 |
+
| GPT-J-6B | 34.9 | -- |
|
| 27 |
+
| LLaMA-1-33B | 35.6 | 3.9 |
|
| 28 |
+
| LLaMA-2-34B | 42.2 | 6.24 |
|
| 29 |
+
| RFT-7B | 50.3 | -- |
|
| 30 |
+
| LLaMA-1-65B | 50.9 | 10.6 |
|
| 31 |
+
| Qwen-7B | 51.6 | -- |
|
| 32 |
+
| **Phi1.5-1.3B** | **54.3** | **15.5** |
|
| 33 |
+
| WizardMath-7B | 54.9 | 10.7 |
|
| 34 |
+
| LLaMA-2-70B | 56.8 | 13.5 |
|
| 35 |
+
| WizardMath-13B | 63.9 | 14.0 |
|
| 36 |
+
| MAmmoTH-7B (COT) | 50.5 | 10.4 |
|
| 37 |
+
| MAmmoTH-7B (POT+COT)| 53.6 | 31.5 |
|
| 38 |
+
| Arithmo-Mistral-7B | 74.7 | 25.3 |
|
| 39 |
+
| MetaMath-7B | 66.5 | 19.8 |
|
| 40 |
+
| MetaMath-13B | 72.3 | 22.4 |
|
| 41 |
+
| **MetaMath-Mistral-7B** | **77.7** | **28.2** |
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
It achieves remarkable performance with only 1.3B parameters !!!
|
| 45 |
+
|
| 46 |
+
You can evaluate the results by [metamath](https://huggingface.co/meta-math/MetaMath-Mistral-7B) evaluation code.
|