projecte-aina
/

roberta-base-ca-v2-cased-sts

@@ -45,24 +45,28 @@ model-index:
 # Catalan BERTa-v2 (roberta-base-ca-v2) finetuned for Semantic Textual Similarity.
-The **roberta-base-ca-v2-cased-sts** is a Semantic Textual Similarity (STS) model for the Catalan language fine-tuned from the [roberta-base-ca-v2](https://huggingface.co/projecte-aina/roberta-base-ca-v2) model, a [RoBERTa](https://arxiv.org/abs/1907.11692) base model pre-trained on a medium-size corpus collected from publicly available corpora and crawlers (check the roberta-base-ca-v2 model card for more details).
-## Datasets
-We used the STS dataset in Catalan called [STS-ca](https://huggingface.co/datasets/projecte-aina/sts-ca) for training and evaluation.
-## Evaluation and results
-We evaluated the _roberta-base-ca-v2-cased-sts_ on the STS-ca test set against standard multilingual and monolingual baselines:
-| Model       | STS-ca (Combined score)   |
-| ------------|:-------------|
-| roberta-base-ca-v2-cased-sts | 79.07 |
-| roberta-base-ca-cased-sts | **80.19** |
-| mBERT       | 74.26 |
-| XLM-RoBERTa | 61.61 |
-For more details, check the fine-tuning and evaluation scripts in the official [GitHub repository](https://github.com/projecte-aina/club).
 ## How to use
 To get the correct<sup>1</sup> model's prediction scores with values between 0.0 and 5.0, use the following code:
@@ -87,7 +91,7 @@ sentence_pairs = [("El llibre va caure per la finestra.", "El llibre va sortir v
 predictions = pipe(prepare(sentence_pairs), add_special_tokens=False)
-# convert back to scores to the original 1 and 5 interval
 for prediction in predictions:
     prediction['score'] = logit(prediction['score'])
 print(predictions)
@@ -101,7 +105,37 @@ Expected output:
 <sup>1</sup> _**avoid using the widget** scores since they are normalized and do not reflect the original annotation values._
-## Citing
 If you use any of these resources (datasets or models) in your work, please cite our latest paper:
 ```bibtex
 @inproceedings{armengol-estape-etal-2021-multilingual,
@@ -126,4 +160,8 @@ If you use any of these resources (datasets or models) in your work, please cite
 ```
 ### Funding
-This work was funded by the [Catalan Government](https://politiquesdigitals.gencat.cat/en/inici/index.html) within the framework of the [AINA project.](https://politiquesdigitals.gencat.cat/ca/economia/catalonia-ai/aina).

 # Catalan BERTa-v2 (roberta-base-ca-v2) finetuned for Semantic Textual Similarity.
+## Table of Contents
+- [Model Description](#model-description)
+- [Intended Uses and Limitations](#intended-uses-and-limitations)
+- [How to Use](#how-to-use)
+- [Training](#training)
+  - [Training Data](#training-data)
+  - [Training Procedure](#training-procedure)
+- [Evaluation](#evaluation)
+   - [Variable and Metrics](#variable-and-metrics)
+   - [Evaluation Results](#evaluation-results)
+- [Licensing Information](#licensing-information)
+- [Citation Information](#citation-information)
+- [Funding](#funding)
+- [Contributions](#contributions)
+## Model description
+The **roberta-base-ca-v2-cased-sts** is a Semantic Textual Similarity (STS) model for the Catalan language fine-tuned from the [roberta-base-ca-v2](https://huggingface.co/projecte-aina/roberta-base-ca-v2) model, a [RoBERTa](https://arxiv.org/abs/1907.11692) base model pre-trained on a medium-size corpus collected from publicly available corpora and crawlers (check the roberta-base-ca-v2 model card for more details).
+## Intended Uses and Limitations
+**roberta-base-ca-v2-cased-sts** model can be used to assess the similarity between two snippets of text. The model is limited by its training dataset and may not generalize well for all use cases.
 ## How to use
 To get the correct<sup>1</sup> model's prediction scores with values between 0.0 and 5.0, use the following code:
 predictions = pipe(prepare(sentence_pairs), add_special_tokens=False)
+# convert back to scores to the original 0 and 5 interval
 for prediction in predictions:
     prediction['score'] = logit(prediction['score'])
 print(predictions)
 <sup>1</sup> _**avoid using the widget** scores since they are normalized and do not reflect the original annotation values._
+## Training
+### Training data
+We used the STS dataset in Catalan called [STS-ca](https://huggingface.co/datasets/projecte-aina/sts-ca) for training and evaluation.
+### Training Procedure
+The model was trained with a batch size of 16 and a learning rate of 5e-5 for 5 epochs. We then selected the best checkpoint using the downstream task metric in the corresponding development set, and then evaluated it on the test set.
+## Evaluation
+### Variable and Metrics
+This model was finetuned maximizing the average score between the Pearson and Spearman correlations.
+## Evaluation results
+We evaluated the _roberta-base-ca-v2-cased-sts_ on the STS-ca test set against standard multilingual and monolingual baselines:
+| Model       | STS-ca (Combined score)   |
+| ------------|:-------------|
+| roberta-base-ca-v2-cased-sts | 79.07 |
+| roberta-base-ca-cased-sts | **80.19** |
+| mBERT       | 74.26 |
+| XLM-RoBERTa | 61.61 |
+For more details, check the fine-tuning and evaluation scripts in the official [GitHub repository](https://github.com/projecte-aina/club).
+## Licensing Information
+[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+## Citation Information
 If you use any of these resources (datasets or models) in your work, please cite our latest paper:
 ```bibtex
 @inproceedings{armengol-estape-etal-2021-multilingual,
 ```
 ### Funding
+This work was funded by the [Departament de la Vicepresidència i de Polítiques Digitals i Territori de la Generalitat de Catalunya](https://politiquesdigitals.gencat.cat/en/inici/index.html) within the framework of [Projecte AINA](https://politiquesdigitals.gencat.cat/ca/economia/catalonia-ai/aina).
+## Contributions
+[N/A]