nicholasKluge
commited on
Commit
•
c1949f4
1
Parent(s):
c890b6a
Update README.md
Browse files
README.md
CHANGED
@@ -113,11 +113,15 @@ The model will output something like:
|
|
113 |
|
114 |
## Benchmarks
|
115 |
|
|
|
|
|
116 |
| Models | Average | [ARC](https://arxiv.org/abs/1803.05457) | [Hellaswag](https://arxiv.org/abs/1905.07830) | [MMLU](https://arxiv.org/abs/2009.03300) | [TruthfulQA](https://arxiv.org/abs/2109.07958) |
|
117 |
|-------------------------------------------------------------------------------------|---------|-----------------------------------------|-----------------------------------------------|------------------------------------------|------------------------------------------------|
|
|
|
118 |
| [TeenyTinyLlama-460m](https://huggingface.co/nicholasKluge/TeenyTinyLlama-460m) | 33.01 | 29.40 | 33.00 | 28.55 | 41.10 |
|
119 |
| [Bloom-560m](https://huggingface.co/bigscience/bloom-560m) | 32.13 | 24.74* | 37.15* | 24.22* | 42.44* |
|
120 |
| [Xglm-564M](https://huggingface.co/facebook/xglm-564M) | 31.97 | 25.56 | 34.64* | 25.18* | 42.53 |
|
|
|
121 |
| [TeenyTinyLlama-160m](https://huggingface.co/nicholasKluge/TeenyTinyLlama-160m) | 31.16 | 26.15 | 29.29 | 28.11 | 41.12 |
|
122 |
| [Pythia-160m](https://huggingface.co/EleutherAI/pythia-160m-deduped) | 31.16 | 24.06* | 31.39* | 24.86* | 44.34* |
|
123 |
| [OPT-125m](https://huggingface.co/facebook/opt-125m) | 30.80 | 22.87 | 31.47 | 26.02 | 42.87 |
|
@@ -125,8 +129,6 @@ The model will output something like:
|
|
125 |
| [Gpt2-small](https://huggingface.co/gpt2) | 29.97 | 21.48* | 31.60* | 25.79* | 40.65* |
|
126 |
| [Multilingual GPT](https://huggingface.co/ai-forever/mGPT) | 29.45 | 24.79 | 26.37* | 25.17* | 41.50 |
|
127 |
|
128 |
-
* Evaluations on benchmarks were performed using the [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) (by [EleutherAI](https://www.eleuther.ai/)). Thanks to [Laiviet](https://github.com/laiviet/lm-evaluation-harness) for translating some of the tasks in the LM-Evaluation-Harness.
|
129 |
-
|
130 |
## Cite as 🤗
|
131 |
|
132 |
```latex
|
|
|
113 |
|
114 |
## Benchmarks
|
115 |
|
116 |
+
Evaluations on benchmarks were performed using the [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) (by [EleutherAI](https://www.eleuther.ai/)). Thanks to [Laiviet](https://github.com/laiviet/lm-evaluation-harness) for translating some of the tasks in the LM-Evaluation-Harness. The results of models marked with an "*" were extracted from the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
|
117 |
+
|
118 |
| Models | Average | [ARC](https://arxiv.org/abs/1803.05457) | [Hellaswag](https://arxiv.org/abs/1905.07830) | [MMLU](https://arxiv.org/abs/2009.03300) | [TruthfulQA](https://arxiv.org/abs/2109.07958) |
|
119 |
|-------------------------------------------------------------------------------------|---------|-----------------------------------------|-----------------------------------------------|------------------------------------------|------------------------------------------------|
|
120 |
+
| [Pythia-410m](https://huggingface.co/EleutherAI/pythia-410m-deduped) | 33.26 | 24.83* | 41.29* | 25.99* | 40.95* |
|
121 |
| [TeenyTinyLlama-460m](https://huggingface.co/nicholasKluge/TeenyTinyLlama-460m) | 33.01 | 29.40 | 33.00 | 28.55 | 41.10 |
|
122 |
| [Bloom-560m](https://huggingface.co/bigscience/bloom-560m) | 32.13 | 24.74* | 37.15* | 24.22* | 42.44* |
|
123 |
| [Xglm-564M](https://huggingface.co/facebook/xglm-564M) | 31.97 | 25.56 | 34.64* | 25.18* | 42.53 |
|
124 |
+
| [OPT-350m](https://huggingface.co/facebook/opt-350m) | 31.78 | 23.55* | 36.73* | 26.02* | 40.83* |
|
125 |
| [TeenyTinyLlama-160m](https://huggingface.co/nicholasKluge/TeenyTinyLlama-160m) | 31.16 | 26.15 | 29.29 | 28.11 | 41.12 |
|
126 |
| [Pythia-160m](https://huggingface.co/EleutherAI/pythia-160m-deduped) | 31.16 | 24.06* | 31.39* | 24.86* | 44.34* |
|
127 |
| [OPT-125m](https://huggingface.co/facebook/opt-125m) | 30.80 | 22.87 | 31.47 | 26.02 | 42.87 |
|
|
|
129 |
| [Gpt2-small](https://huggingface.co/gpt2) | 29.97 | 21.48* | 31.60* | 25.79* | 40.65* |
|
130 |
| [Multilingual GPT](https://huggingface.co/ai-forever/mGPT) | 29.45 | 24.79 | 26.37* | 25.17* | 41.50 |
|
131 |
|
|
|
|
|
132 |
## Cite as 🤗
|
133 |
|
134 |
```latex
|