weqweasdas
commited on
Commit
•
1bedbf1
1
Parent(s):
465c80f
Update README.md
Browse files
README.md
CHANGED
@@ -14,14 +14,15 @@ The model is trained from [meta-llama/Meta-Llama-3-8B](https://huggingface.co/me
|
|
14 |
|
15 |
## Academic Benchmarks
|
16 |
|
|
|
|
|
17 |
| **Model** | **Size** | **Method** | **LC AlpacaEval** | **MT-Bench** | **GSM-8K** | **MMLU** | **HumanEval** | **TruthfulQA** | **ARC** | **MBPP** |
|
18 |
|----------------------------|----------|-----------------|------------|------------|------------|----------|---------------|----------------|---------|----------|
|
19 |
| LLaMA-3-8B-it | 8B | RS+DPO+PPO |22.9|8.16| 79.6 | 66.0 | 61.6 | 43.9 | 59.5 | 61.1 |
|
20 |
-
| Ours (SFT baseline) | 8B | SFT |10.2|7.69| 74.2
|
21 |
| Ours (Iterative RLHF) | 8B | Iterative DPO |37.2|8.46| 80.7 | 65.3 | 64.6 | 60.4 | 64.3 | 60.8 |
|
22 |
|
23 |
|
24 |
-
|
25 |
## Citation
|
26 |
Please cite our techical report if you find our model is useful for your research or product.
|
27 |
```
|
|
|
14 |
|
15 |
## Academic Benchmarks
|
16 |
|
17 |
+
We use ToRA script to evaluate GSM8K and MATH, Evalplut for HumanEval, and lm-evaluation-harness for other benchmarks. The model is evaluated in zero-shot setting so the results here may be slightly different from that reported in the technical report.
|
18 |
+
|
19 |
| **Model** | **Size** | **Method** | **LC AlpacaEval** | **MT-Bench** | **GSM-8K** | **MMLU** | **HumanEval** | **TruthfulQA** | **ARC** | **MBPP** |
|
20 |
|----------------------------|----------|-----------------|------------|------------|------------|----------|---------------|----------------|---------|----------|
|
21 |
| LLaMA-3-8B-it | 8B | RS+DPO+PPO |22.9|8.16| 79.6 | 66.0 | 61.6 | 43.9 | 59.5 | 61.1 |
|
22 |
+
| Ours (SFT baseline) | 8B | SFT |10.2|7.69| 74.2 | 30.0 | 64.6 | 63.4 | 53.5 | 58.6 |
|
23 |
| Ours (Iterative RLHF) | 8B | Iterative DPO |37.2|8.46| 80.7 | 65.3 | 64.6 | 60.4 | 64.3 | 60.8 |
|
24 |
|
25 |
|
|
|
26 |
## Citation
|
27 |
Please cite our techical report if you find our model is useful for your research or product.
|
28 |
```
|