Transformers
PyTorch
English
Inference Endpoints
wenhu commited on
Commit
e9e0143
1 Parent(s): 1dff1a3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -13
README.md CHANGED
@@ -36,19 +36,14 @@ The models are fine-tuned with the MathInstruct dataset using the original Llama
36
  The models are evaluated using open-ended and multiple-choice math problems from several datasets. Here are the results:
37
 
38
 
39
- | **Model** | **Decoding** | **GSM** | **MATH** | **MMLU** |
40
- |---------------------------|---------------|----------|------------|-----------|
41
- | MAmmoTH-7B | CoT | 50.5 | 10.4 | 43.7 |
42
- | | PoT | 51.6 | 28.7 | 43.3 |
43
- | | **Hybrid** | 53.6 | 31.5 | 44.5 |
44
- | MAmmoTH-Coder-7B | CoT | 22.4 | 7.9 | 36.2 |
45
- | | PoT | 58.8 | 32.1 | 47.2 |
46
- | | **Hybrid** | 59.4 | 33.4 | 47.2 |
47
- | MAmmoTH-7B-Mistral | CoT | | | |
48
- | | PoT | | | |
49
- | | **Hybrid** | **75.0** | **40.0** | **52.5** |
50
-
51
-
52
 
53
  ## Usage
54
  You can use the models through Huggingface's Transformers library. Use the pipeline function to create a text-generation pipeline with the model of your choice, then feed in a math problem to get the solution.
 
36
  The models are evaluated using open-ended and multiple-choice math problems from several datasets. Here are the results:
37
 
38
 
39
+ | **Model** | **Decoding** | **GSM** | **MATH** | **MMLU-Math** |
40
+ |---------------------------|---------------|-----------|-----------|-----------|
41
+ | MAmmoTH-7B | **Hybrid** | 53.6 | 31.5 | 44.5 |
42
+ | MAmmoTH-Coder-7B | **Hybrid** | 59.4 | 33.4 | 47.2 |
43
+ | MetaMath-7B-Mistral | **CoT** | **77.7** | 28.2 | 49.3 |
44
+ | OpenChat-3.5-7B | **CoT** | 77.3 | 28.6 | 49.6 |
45
+ | DeepSeek-Coder-34B | **PoT** | 58.2 | 35.3 | 46.5 |
46
+ | MAmmoTH-7B-Mistral | **Hybrid** | 75.0 | **40.0** | **52.5** |
 
 
 
 
 
47
 
48
  ## Usage
49
  You can use the models through Huggingface's Transformers library. Use the pipeline function to create a text-generation pipeline with the model of your choice, then feed in a math problem to get the solution.