|
--- |
|
datasets: |
|
- meta-math/MetaMathQA |
|
--- |
|
|
|
## Introduction |
|
The model is trained with Masked Thought Fine-Tuning (MFT), a simple variant of standard Supervised Fine-Tuning (SFT). You can refer to our code and paper below. |
|
|
|
## Links |
|
- **Code**: [https://github.com/ChangyuChen347/MaskedThought](https://github.com/ChangyuChen347/MaskedThought) |
|
- **Paper**: [https://arxiv.org/abs/2403.02178](https://arxiv.org/abs/2403.02178) |
|
|
|
## Results |
|
We test it with the scripts provided in [MetaMath](https://github.com/meta-math/MetaMath). |
|
| Model | GSM8K | MATH | |
|
|--------------------------------------------------------------------------------------------------------------------------------------------------|-------|-------| |
|
| [adalaw/MetaMath-Mistral-7B-MFT](https://huggingface.co/adalaw/MetaMath-Mistral-7B-MFT) | 79.90 | 29.0 | |
|
| [meta-math/MetaMath-Mistral-7B-SFT](https://huggingface.co/meta-math/MetaMath-Mistral-7B) | 77.70 | 28.2 | |
|
|
