gonzalez-agirre commited on
Commit
664a173
1 Parent(s): 1082857

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +63 -11
README.md CHANGED
@@ -1,18 +1,42 @@
1
  ---
2
  language:
 
3
  - es
 
4
  license: apache-2.0
 
5
  tags:
 
6
  - "national library of spain"
 
7
  - "spanish"
 
8
  - "bne"
 
9
  - "qa"
 
10
  - "question answering"
 
11
  datasets:
 
12
  - "PlanTL-GOB-ES/SQAC"
 
13
  metrics:
14
  - "f1"
15
-
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  ---
17
 
18
  # Spanish RoBERTa-large trained on BNE finetuned for Spanish Question Answering Corpus (SQAC) dataset.
@@ -26,7 +50,13 @@ metrics:
26
  - [How to use](#how-to-use)
27
  - [Limitations and bias](#limitations-and-bias)
28
  - [Training](#training)
 
 
 
 
29
  - [Evaluation](#evaluation)
 
 
30
  - [Additional information](#additional-information)
31
  - [Author](#author)
32
  - [Contact information](#contact-information)
@@ -39,27 +69,49 @@ metrics:
39
  </details>
40
 
41
  ## Model description
42
- RoBERTa-large-bne is a transformer-based masked language model for the Spanish language. It is based on the [RoBERTa](https://arxiv.org/abs/1907.11692) large model and has been pre-trained using the largest Spanish corpus known to date, with a total of 570GB of clean and deduplicated text processed for this work, compiled from the web crawlings performed by the [National Library of Spain (Biblioteca Nacional de España)](http://www.bne.es/en/Inicio/index.html) from 2009 to 2019.
43
-
44
- Original pre-trained model can be found here: https://huggingface.co/BSC-TeMU/roberta-large-bne
45
-
46
 
47
  ## Intended uses and limitations
48
 
 
 
49
  ## How to use
50
 
 
 
 
 
 
 
 
 
 
 
51
  ## Limitations and bias
52
  At the time of submission, no measures have been taken to estimate the bias embedded in the model. However, we are well aware that our models may be biased since the corpora have been collected using crawling techniques on multiple web sources. We intend to conduct research in these areas in the future, and if completed, this model card will be updated.
53
 
54
  ## Training
55
 
56
- The dataset used is the [SQAC corpus](https://huggingface.co/datasets/PlanTL-GOB-ES/SQAC).
 
 
 
 
 
 
 
57
 
58
- ## Evaluation
59
- F1 Score: 0.7993 (average of 5 runs).
60
 
61
- For evaluation details visit our [GitHub repository](https://github.com/PlanTL-GOB-ES/lm-spanish).
 
 
 
 
 
 
 
62
 
 
63
 
64
  ## Additional information
65
 
@@ -78,7 +130,7 @@ Copyright by the Spanish State Secretariat for Digitalization and Artificial Int
78
  ### Funding
79
  This work was funded by the Spanish State Secretariat for Digitalization and Artificial Intelligence (SEDIA) within the framework of the Plan-TL.
80
 
81
- ## Citing information
82
 
83
  If you use this model, please cite our [paper](http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/article/view/6405):
84
  ```
@@ -100,7 +152,7 @@ Intelligence (SEDIA) within the framework of the Plan-TL.},
100
 
101
  ```
102
 
103
- ## Disclaimer
104
 
105
  The models published in this repository are intended for a generalist purpose and are available to third parties. These models may have bias and/or any other undesirable distortions.
106
 
 
1
  ---
2
  language:
3
+
4
  - es
5
+
6
  license: apache-2.0
7
+
8
  tags:
9
+
10
  - "national library of spain"
11
+
12
  - "spanish"
13
+
14
  - "bne"
15
+
16
  - "qa"
17
+
18
  - "question answering"
19
+
20
  datasets:
21
+
22
  - "PlanTL-GOB-ES/SQAC"
23
+
24
  metrics:
25
  - "f1"
26
+ - "exact match"
27
+
28
+ model-index:
29
+ - name: roberta-large-bne-sqac
30
+ results:
31
+ - task:
32
+ type: question-answering
33
+ dataset:
34
+ type: "PlanTL-GOB-ES/SQAC"
35
+ name: SQAC
36
+ metrics:
37
+ - name: F1
38
+ type: f1
39
+ value: 0.8202
40
  ---
41
 
42
  # Spanish RoBERTa-large trained on BNE finetuned for Spanish Question Answering Corpus (SQAC) dataset.
 
50
  - [How to use](#how-to-use)
51
  - [Limitations and bias](#limitations-and-bias)
52
  - [Training](#training)
53
+ - [Training](#training)
54
+ - [Training data](#training-data)
55
+ - [Training procedure](#training-procedure)
56
+ - [Evaluation](#evaluation)
57
  - [Evaluation](#evaluation)
58
+ - [Variable and metrics](#variable-and-metrics)
59
+ - [Evaluation results](#evaluation-results)
60
  - [Additional information](#additional-information)
61
  - [Author](#author)
62
  - [Contact information](#contact-information)
 
69
  </details>
70
 
71
  ## Model description
72
+ The **roberta-large-bne-sqac** is a Question Answering (QA) model for the Spanish language fine-tuned from the [roberta-large-bne](https://huggingface.co/PlanTL-GOB-ES/roberta-large-bne) model, a [RoBERTa](https://arxiv.org/abs/1907.11692) large model pre-trained using the largest Spanish corpus known to date, with a total of 570GB of clean and deduplicated text, processed for this work, compiled from the web crawlings performed by the [National Library of Spain (Biblioteca Nacional de España)](http://www.bne.es/en/Inicio/index.html) from 2009 to 2019.
 
 
 
73
 
74
  ## Intended uses and limitations
75
 
76
+ **roberta-large-bne-sqac** model can be used for extractive question answering. The model is limited by its training dataset and may not generalize well for all use cases.
77
+
78
  ## How to use
79
 
80
+ ```python
81
+ from transformers import pipeline
82
+ nlp = pipeline("question-answering", model="PlanTL-GOB-ES/roberta-large-bne-sqac")
83
+ text = "¿Dónde vivo?"
84
+ context = "Me llamo Wolfgang y vivo en Berlin"
85
+
86
+ qa_results = nlp(text, context)
87
+ print(qa_results)
88
+ ```
89
+
90
  ## Limitations and bias
91
  At the time of submission, no measures have been taken to estimate the bias embedded in the model. However, we are well aware that our models may be biased since the corpora have been collected using crawling techniques on multiple web sources. We intend to conduct research in these areas in the future, and if completed, this model card will be updated.
92
 
93
  ## Training
94
 
95
+ ### Training data
96
+ We used the QA dataset in Spanish called [SQAC corpus](https://huggingface.co/datasets/PlanTL-GOB-ES/SQAC) for training and evaluation.
97
+
98
+ ### Training procedure
99
+ The model was trained with a batch size of 16 and a learning rate of 1e-5 for 5 epochs. We then selected the best checkpoint using the downstream task metric in the corresponding development set and then evaluated it on the test set.
100
+
101
+ ## Evaluation results
102
+ We evaluated the **roberta-large-bne-sqac** on the SQAC test set against standard multilingual and monolingual baselines:
103
 
 
 
104
 
105
+ | Model | SQAC (F1) |
106
+ | ------------|:----|
107
+ | roberta-large-bne-sqac | **82.02** |
108
+ | roberta-base-bne-sqac | 79.23|
109
+ | BETO | 79.23 |
110
+ | mBERT | 75.62 |
111
+ | BERTIN | 76.78 |
112
+ | ELECTRA | 73.83 |
113
 
114
+ For more details, check the fine-tuning and evaluation scripts in the official [GitHub repository](https://github.com/PlanTL-GOB-ES/lm-spanish).
115
 
116
  ## Additional information
117
 
 
130
  ### Funding
131
  This work was funded by the Spanish State Secretariat for Digitalization and Artificial Intelligence (SEDIA) within the framework of the Plan-TL.
132
 
133
+ ### Citing information
134
 
135
  If you use this model, please cite our [paper](http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/article/view/6405):
136
  ```
 
152
 
153
  ```
154
 
155
+ ### Disclaimer
156
 
157
  The models published in this repository are intended for a generalist purpose and are available to third parties. These models may have bias and/or any other undesirable distortions.
158