Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,7 @@ https://meta-math.github.io/
|
|
10 |
|
11 |
## Model Details
|
12 |
|
13 |
-
MetaMath-Mistral-7B is fully fine-tuned on the MetaMathQA datasets and based on the
|
14 |
|
15 |
To fine-tune Mistral-7B, I would suggest using a smaller learning rate (usually 1/5 to 1/10 of the lr for LlaMa-2-7B) and staying other training args unchanged.
|
16 |
More training details and scripts can be seen at https://github.com/meta-math/MetaMath
|
|
|
10 |
|
11 |
## Model Details
|
12 |
|
13 |
+
MetaMath-Mistral-7B is fully fine-tuned on the MetaMathQA datasets and based on the powerful Mistral-7B model. It is glad to see using MetaMathQA datasets and change the base model from llama-2-7B to Mistral-7b can boost the GSM8K performance from 66.5 to **77.7**.
|
14 |
|
15 |
To fine-tune Mistral-7B, I would suggest using a smaller learning rate (usually 1/5 to 1/10 of the lr for LlaMa-2-7B) and staying other training args unchanged.
|
16 |
More training details and scripts can be seen at https://github.com/meta-math/MetaMath
|