Commit
•
348faa6
1
Parent(s):
256a981
Update README.md
Browse files
README.md
CHANGED
@@ -285,7 +285,7 @@ Evaluations on Brazilian Portuguese benchmarks were performed using a [Portugues
|
|
285 |
| | **ASSIN2 RTE** | **ASSIN2 STS** | **BLUEX** | **ENEM** | **FAQUAD NLI** | **HateBR** | **PT Hate Speech** | **OAB Exams** | **TweetSentBR** |
|
286 |
|-----------------------|----------------|----------------|-----------|----------|----------------|------------|--------------------|---------------|-----------------|
|
287 |
| **Mula-4x160-v0.1** | 33.57 | 11.35 | 25.17 | 21.34 | 43.97 | 41.50 | 22.99 | 25.06 | 11.24 |
|
288 |
-
| **Mula-8x160-v0.1** |
|
289 |
|
290 |
## Cite as 🤗
|
291 |
|
@@ -308,22 +308,3 @@ Mula-8x160-v0.1 is licensed under the Apache License, Version 2.0. See the [LICE
|
|
308 |
## Acknowledgements
|
309 |
|
310 |
The authors gratefully acknowledge the granted access to the [Marvin cluster](https://www.hpc.uni-bonn.de/en/systems/marvin) hosted by the [University of Bonn](https://www.uni-bonn.de/en) along with the support provided by its High Performance Computing & Analytics Lab.
|
311 |
-
|
312 |
-
|
313 |
-
# Open Portuguese LLM Leaderboard Evaluation Results
|
314 |
-
|
315 |
-
Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/MulaBR/Mula-8x160-v0.1) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
|
316 |
-
|
317 |
-
| Metric | Value |
|
318 |
-
|--------------------------|---------|
|
319 |
-
|Average |**25.72**|
|
320 |
-
|ENEM Challenge (No Images)| 20.50|
|
321 |
-
|BLUEX (No Images) | 21.28|
|
322 |
-
|OAB Exams | 26.65|
|
323 |
-
|Assin2 RTE | 22.38|
|
324 |
-
|Assin2 STS | 4.73|
|
325 |
-
|FaQuAD NLI | 43.97|
|
326 |
-
|HateBR Binary | 33.33|
|
327 |
-
|PT Hate Speech Binary | 40.21|
|
328 |
-
|tweetSentBR | 18.46|
|
329 |
-
|
|
|
285 |
| | **ASSIN2 RTE** | **ASSIN2 STS** | **BLUEX** | **ENEM** | **FAQUAD NLI** | **HateBR** | **PT Hate Speech** | **OAB Exams** | **TweetSentBR** |
|
286 |
|-----------------------|----------------|----------------|-----------|----------|----------------|------------|--------------------|---------------|-----------------|
|
287 |
| **Mula-4x160-v0.1** | 33.57 | 11.35 | 25.17 | 21.34 | 43.97 | 41.50 | 22.99 | 25.06 | 11.24 |
|
288 |
+
| **Mula-8x160-v0.1** | 22.38 | 4.73 | 21.28 | 20.50 | 43.97 | 33.33 | 40.21 | 26.65 | 18.46 |
|
289 |
|
290 |
## Cite as 🤗
|
291 |
|
|
|
308 |
## Acknowledgements
|
309 |
|
310 |
The authors gratefully acknowledge the granted access to the [Marvin cluster](https://www.hpc.uni-bonn.de/en/systems/marvin) hosted by the [University of Bonn](https://www.uni-bonn.de/en) along with the support provided by its High Performance Computing & Analytics Lab.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|