Update README.md
Browse files
README.md
CHANGED
@@ -195,7 +195,7 @@ We evaluate our model on all benchmarks of the leaderboard's version 2 using the
|
|
195 |
| `model name` |`IFEval`| `BBH` |`MATH LvL5`| `GPQA`| `MUSR`|`MMLU-PRO`|`Average`|
|
196 |
|:--------------------------|:------:|:-----:|:---------:|:-----:|:-----:|:--------:|:-------:|
|
197 |
| ***Pure SSM models*** | | | | | | | |
|
198 |
-
| `
|
199 |
| `TRI-ML/mamba-7b-rw` | 22.46 | 6.71 | 0.45 | 1.12 | 5.51 | 1.69 | 6.25 |
|
200 |
|***Hybrid SSM-attention models*** | | | | | | |
|
201 |
| `Zamba-7B-v1` | 24.06 | 21.12 | 3.32 | 3.03 | 7.74 | 16.02 | 12.55 |
|
@@ -213,7 +213,7 @@ We evaluate our model on all benchmarks of the leaderboard's version 2 using the
|
|
213 |
| `model name` |`ARC`|`HellaSwag` |`MMLU` |`Winogrande`|`TruthfulQA`|`GSM8K`|`Average` |
|
214 |
|:-----------------------------|:------:|:---------:|:-----:|:----------:|:----------:|:-----:|:----------------:|
|
215 |
| ***Pure SSM models*** | | | | | | | |
|
216 |
-
| `
|
217 |
| `TRI-ML/mamba-7b-rw` | 46.48 | 80.24 | 57.72 | 76.40 | - | 4.70 | - |
|
218 |
|***Hybrid SSM-attention models***| | | | | | | |
|
219 |
| `recurrentgemma-9b` |52.00 | 80.40 | 60.50 | 73.60 | 38.60 | 42.60 | 57.95 |
|
|
|
195 |
| `model name` |`IFEval`| `BBH` |`MATH LvL5`| `GPQA`| `MUSR`|`MMLU-PRO`|`Average`|
|
196 |
|:--------------------------|:------:|:-----:|:---------:|:-----:|:-----:|:--------:|:-------:|
|
197 |
| ***Pure SSM models*** | | | | | | | |
|
198 |
+
| `FalconMamba-7B` | 33.36 | 19.88 | 3.63 | 8.05 | 10.86 | 14.47 |**15.04**|
|
199 |
| `TRI-ML/mamba-7b-rw` | 22.46 | 6.71 | 0.45 | 1.12 | 5.51 | 1.69 | 6.25 |
|
200 |
|***Hybrid SSM-attention models*** | | | | | | |
|
201 |
| `Zamba-7B-v1` | 24.06 | 21.12 | 3.32 | 3.03 | 7.74 | 16.02 | 12.55 |
|
|
|
213 |
| `model name` |`ARC`|`HellaSwag` |`MMLU` |`Winogrande`|`TruthfulQA`|`GSM8K`|`Average` |
|
214 |
|:-----------------------------|:------:|:---------:|:-----:|:----------:|:----------:|:-----:|:----------------:|
|
215 |
| ***Pure SSM models*** | | | | | | | |
|
216 |
+
| `FalconMamba-7B` |62.03 | 80.82 | 62.11 | 73.64 | 53.42 | 52.54 | **64.09** |
|
217 |
| `TRI-ML/mamba-7b-rw` | 46.48 | 80.24 | 57.72 | 76.40 | - | 4.70 | - |
|
218 |
|***Hybrid SSM-attention models***| | | | | | | |
|
219 |
| `recurrentgemma-9b` |52.00 | 80.40 | 60.50 | 73.60 | 38.60 | 42.60 | 57.95 |
|