|
--- |
|
license: mit |
|
language: |
|
- gl |
|
metrics: |
|
- bleu (Gold1): 79.6 |
|
- bleu (Gold2): 43.3 |
|
- bleu (Flores): 21.8 |
|
- bleu (Test-suite): 74.3 |
|
--- |
|
|
|
**Descrición do Modelo / Model description** |
|
|
|
Modelo feito con OpenNMT para o par español-galego utilizando unha arquitectura transformer. |
|
|
|
Model developed with OpenNMT for the Spanish-Galician pair using a transformer architecture. |
|
|
|
**Como traducir / How to translate** |
|
|
|
+ Abrir terminal bash / Open bash terminal |
|
+ Instalar / Installing [Python 3.9](https://www.python.org/downloads/release/python-390/) |
|
+ Instalar / Installing [Open NMT toolkit v.2.2](https://github.com/OpenNMT/OpenNMT-py) |
|
+ Traducir un input_text utilizando o modelo NOS-MT-es-gl co seguinte comando / Translating an input_text using the NOS-MT-en-gl model with the following command: |
|
|
|
```bash |
|
onmt_translate -src input_text -model NOS-MT-es-gl -output ./output_file.txt -replace_unk -phrase_table phrase_table-es-gl.txt -gpu 0 |
|
``` |
|
+ O resultado da tradución estará no PATH indicado no flag -output / The result of the translation will be in the PATH indicated by the -output flag. |
|
|
|
**Adestramento / Training** |
|
|
|
No adestramento, utilizamos corpora auténticos e sintéticos. Os primeiros son corpora de traducións feitas directamente por tradutores humanos. Os segundos son corpora de traducións español-portugués e inglés-portugués, que convertemos en español-galego e inglés-galego a través da tradución automática portugués-galego con Opentrad/Apertium e transliteración para palabras fóra de vocabulário. |
|
|
|
In the training we have used authentic and synthetic corpora. The former are corpora of translations directly produced by human translators. The latter are corpora of Spanish-Portuguese and English-Portuguese translations, which we have converted into Spanish-Galician and English-Galician by means of Portuguese-Galician translation with Opentrad/Apertium and transliteration for out-of-vocabulary words. |
|
|
|
|
|
**Procedemento de adestramento / Training process** |
|
|
|
+ Tokenization dos datasets feita co tokenizador de linguakit / Tokenization of the datasets made with linguakit tokeniser https://github.com/citiususc/Linguakit |
|
|
|
+ O vocabulario para os modelos foi xerado a través do script / Vocabulary for the models was created by the script [learn_bpe.py](https://github.com/OpenNMT/OpenNMT-py/blob/master/tools/learn_bpe.py) da open NMT |
|
|
|
+ Usando o .yaml neste repositorio pode replicar o proceso de adestramento do seguinte xeito / Using the .yaml in this repository you can replicate the training process as follows |
|
|
|
```bash |
|
onmt_build_vocab -config bpe-es-gl_emb.yaml -n_sample 100000 |
|
onmt_train -config bpe-es-gl_emb.yaml |
|
``` |
|
|
|
**Hiperparámetros / Hyper-parameters** |
|
|
|
Os parámetros usados para o desenvolvimento do modelo poden ser consultados directamente no mesmo ficheiro .yaml bpe-es-gl_emb.yaml |
|
|
|
The parameters used for the development of the model can be directly viewed in the same .yaml file bpe-es-gl_emb.yaml |
|
|
|
**Avaliación / Evaluation** |
|
|
|
A avalación BLEU dos modelos é feita cunha mistura de tests desenvolvidos internamente (gold1, gold2, test-suite) con outros datasets disponíbeis en galego (Flores). |
|
|
|
The BLEU evaluation of the models is done by mixing internally developed tests (gold1, gold2, test-suite) with other datasets available in Galician (Flores). |
|
|
|
| GOLD 1 | GOLD 2 | FLORES | TEST-SUITE| |
|
| ------------- |:-------------:| -------:|----------:| |
|
| 79.6 | 43.3 | 21.8 | 74.3 | |
|
|
|
**Licenzas do Modelo / Licensing information** |
|
|
|
MIT License |
|
|
|
Copyright (c) 2023 Proxecto Nós |
|
|
|
Permission is hereby granted, free of charge, to any person obtaining a copy |
|
of this software and associated documentation files (the "Software"), to deal |
|
in the Software without restriction, including without limitation the rights |
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell |
|
copies of the Software, and to permit persons to whom the Software is |
|
furnished to do so, subject to the following conditions: |
|
|
|
The above copyright notice and this permission notice shall be included in all |
|
copies or substantial portions of the Software. |
|
|
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR |
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, |
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE |
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER |
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, |
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE |
|
SOFTWARE. |
|
|
|
**Financiamento / Funding** |
|
|
|
Esta investigación foi financiada polo proxecto "Nós: o galego na sociedade e economía da intelixencia artificial", resultado dun acordo entre a Xunta de Galicia e a Universidade de Santiago de Compostela, o que resultou no subsidio ED431G2019/04 da Consellaría de Educación, Universidade e Formación Profesional da Galiza, e polo Fondo Europeo de Desenvolvemento Rexional (programa ERDF/FEDER), e Grupos de Referencia: ED431C 2020/21. |
|
|
|
This research was funded by the project "Nós: Galician in the society and economy of artificial intelligence", agreement between Xunta de Galicia and University of Santiago de Compostela, and grant ED431G2019/04 by the Galician Ministry of Education, University and Professional Training, and the European Regional Development Fund (ERDF/FEDER program), and Groups of Reference: ED431C 2020/21. |
|
|
|
**Citation Information** |
|
|
|
@article{garriga2022catalan, |
|
title={}, |
|
author={}, |
|
year={2023}, |
|
url={} |
|
} |
