jarodrigues commited on
Commit
0c4c294
1 Parent(s): f43ac1e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -35,7 +35,7 @@ datasets:
35
  **Gervásio PT-*** is a **fully open** decoder for the **Portuguese language**.
36
 
37
 
38
- It is a **decoder** of the LLaMA family, based on the neural architecture Transformer and developed over the LLaMA 2 7B model.
39
  Its further improvement through additional training was done over language resources that include new instruction data sets of Portuguese prepared for this purpose.
40
 
41
  It has different versions that were trained for different variants of Portuguese (PT),
@@ -126,8 +126,8 @@ This involves repurposing the tasks in various ways, such as generation of answe
126
  | Model | MRPC (F1) | RTE (F1) | COPA (F1) |
127
  |--------------------------|----------------|----------------|-----------|
128
  | **Gervásio 7B PT-BR** | **0.7822** | **0.8321** | 0.2134 |
129
- | **LLaMA 2** | 0.0369 | 0.0516 | 0.4867 |
130
- | **LLaMA 2 Chat** | 0.5432 | 0.3807 | **0.5493**|
131
  <br>
132
 
133
  For further testing our decoder, in addition to the testing data described above, we also reused some of the datasets that had been resorted for American Portuguese to test the state-of-the-art Sabiá model and that were originally developed with materials from Portuguese: ASSIN2 RTE (entailment) and ASSIN2 STS (similarity), BLUEX (question answering), ENEM 2022 (question answering) and FaQuAD (extractive question-answering).
@@ -142,8 +142,8 @@ To evaluate Gervásio, the examples were randomly selected to be included in the
142
  | Model | ENEM 2022 (Accuracy) | BLUEX (Accuracy)| RTE (F1) | STS (Pearson) |
143
  |--------------------------|----------------------|-----------------|-----------|---------------|
144
  | **Gervásio 7B PT-BR** | 0.1977 | 0.2640 | **0.7469**| **0.2136** |
145
- | **LLaMA 2** | 0.2458 | 0.2903 | 0.0913 | 0.1034 |
146
- | **LLaMA 2 Chat** | 0.2231 | 0.2959 | 0.5546 | 0.1750 |
147
  ||||||
148
  | **Sabiá-7B** | **0.6017** | **0.7743** | 0.6847 | 0.1363 |
149
 
 
35
  **Gervásio PT-*** is a **fully open** decoder for the **Portuguese language**.
36
 
37
 
38
+ It is a **decoder** of the LLaMA family, based on the neural architecture Transformer and developed over the LLaMA-2 7B model.
39
  Its further improvement through additional training was done over language resources that include new instruction data sets of Portuguese prepared for this purpose.
40
 
41
  It has different versions that were trained for different variants of Portuguese (PT),
 
126
  | Model | MRPC (F1) | RTE (F1) | COPA (F1) |
127
  |--------------------------|----------------|----------------|-----------|
128
  | **Gervásio 7B PT-BR** | **0.7822** | **0.8321** | 0.2134 |
129
+ | **LLaMA-2** | 0.0369 | 0.0516 | 0.4867 |
130
+ | **LLaMA-2 Chat** | 0.5432 | 0.3807 | **0.5493**|
131
  <br>
132
 
133
  For further testing our decoder, in addition to the testing data described above, we also reused some of the datasets that had been resorted for American Portuguese to test the state-of-the-art Sabiá model and that were originally developed with materials from Portuguese: ASSIN2 RTE (entailment) and ASSIN2 STS (similarity), BLUEX (question answering), ENEM 2022 (question answering) and FaQuAD (extractive question-answering).
 
142
  | Model | ENEM 2022 (Accuracy) | BLUEX (Accuracy)| RTE (F1) | STS (Pearson) |
143
  |--------------------------|----------------------|-----------------|-----------|---------------|
144
  | **Gervásio 7B PT-BR** | 0.1977 | 0.2640 | **0.7469**| **0.2136** |
145
+ | **LLaMA-2** | 0.2458 | 0.2903 | 0.0913 | 0.1034 |
146
+ | **LLaMA-2 Chat** | 0.2231 | 0.2959 | 0.5546 | 0.1750 |
147
  ||||||
148
  | **Sabiá-7B** | **0.6017** | **0.7743** | 0.6847 | 0.1363 |
149