Update README.md

README.md

@@ -51,9 +51,9 @@ language:
 ### Pretraining Data
 
 The training corpus consists of 70+XX billion tokens of Catalan-, Spanish-centric, and English-centric parallel data, including all of the official European languages plus Catalan, Basque, 
-Galician, Asturian, Aragonese and Aranese. It amounts to 3,157,965,012+XX parallel sentence pairs. 
+Galician, Asturian, Aragonese and Aranese. It amounts to 6,574,251,526 parallel sentence pairs. 
 
-This highly multilingual corpus is predominantly composed of data sourced from [OPUS](https://opus.nlpl.eu/), with additional data taken from the [NTEU project](https://nteu.eu/) and Project Aina’s existing corpora. 
+This highly multilingual corpus is predominantly composed of data sourced from [OPUS](https://opus.nlpl.eu/), with additional data taken from the [NTEU project](https://nteu.eu/), Project Aina’s existing corpora, and our own unpublished datasets. 
 Where little parallel Catalan <-> xx data could be found, synthetic Catalan data was generated from the Spanish side of the collected Spanish <-> xx corpora using 
 [Projecte Aina’s Spanish-Catalan model](https://huggingface.co/projecte-aina/aina-translator-es-ca). The final distribution of languages was as below:
 
@@ -66,20 +66,26 @@ Click the expand button below to see the full list of corpora included in the tr
  
 | Dataset                                   	| Ca-xx Languages                                                                                                	|  Es-xx Langugages                             | En-xx Languages |
 |-----------------------------------------------|----------------------------------------------------------------|-----------------------------------------------|----------------------------------------------------------------|
-|[AINA CA-EN](https://huggingface.co/datasets/projecte-aina/CA-EN_Parallel_Corpus) | en     |       |       |
-|[CCMatrix](https://opus.nlpl.eu/CCMatrix/corpus/version/CCMatrix)		|eu			|		| ga,gl |
-|[CLUVI](https://dataverse.csuc.cat/dataset.xhtml?persistentId=doi:10.34810/data267) |     |    |   gl   |   
+|[AINA](https://huggingface.co/projecte-aina) | en     |       |       |
+|ARANESE-SYNTH-CORPUS-BSC                         | arn   |      |         |
+|BOUA-BSC             |     | val |        |
+|[BOUMH](https://github.com/transducens/PILAR/tree/main/valencian/BOUMH) |          | val   |           |
+|[BOUA-PILAR](https://github.com/transducens/PILAR/tree/main/valencian/BOUA)  |          | val |       |
+|[CCMatrix](https://opus.nlpl.eu/CCMatrix/corpus/version/CCMatrix)		|eu			|		| ga | 
 |[DGT](https://opus.nlpl.eu/DGT/corpus/version/DGT)			|			|bg,cs,da,de,el	,et,fi,fr,ga,hr,hu,lt,lv,mt,nl,pl,pt,ro,sk,sl,sv	|    da,et,ga,hr,hu,lt,lv,mt,sh,sl|
+|DOGV-BSC                 |      |  val    |         |
+|[DOGV-PILAR](https://github.com/transducens/PILAR/tree/main/valencian/DOGV-html) |           | val |            | 
 |[ELRC-EMEA](https://opus.nlpl.eu/ELRC-EMEA/corpus/version/ELRC-EMEA)		|			|bg,cs,da,hu,lt,lv,mt,pl,ro,sk,sl		| et,hr,lv,ro,sk,sl |
 |[EMEA](https://opus.nlpl.eu/EMEA/corpus/version/EMEA)			|			|bg,cs,da,el,fi,hu,lt,mt,nl,pl,ro,sk,sl,sv		|    et,mt  |
 |[EUBookshop](https://opus.nlpl.eu/EUbookshop/corpus/version/EUbookshop)		|lt,pl,pt			|cs,da,de,el,fi,fr,ga,it,lv,mt,nl,pl,pt,ro,sk,sl,sv		|cy,ga|
-|[Europarl](https://opus.nlpl.eu/Europarl/corpus/version/Europarl)		|			|bg,cs,da,el,en,fi,fr,hu,lt,lv,nl,pl,pt	,ro,sk,sl,sv	|gl_pt |
+|[Europarl](https://opus.nlpl.eu/Europarl/corpus/version/Europarl)		|			|bg,cs,da,el,en,fi,fr,hu,lt,lv,nl,pl,pt	,ro,sk,sl,sv	| |
 |[Europat](https://opus.nlpl.eu/EuroPat/corpus/version/EuroPat)		|			|en,hr		| no  |
 |[GAITU](https://gaitu.eus/) | | | eu|
 |[KDE4](https://opus.nlpl.eu/KDE4/corpus/version/KDE4)			|bg,cs,da,de,el	,et,eu,fi,fr,ga,gl,hr,it,lt,lv,nl,pl,pt,ro,sk,sl,sv	|bg,ga,hr	|cy,ga,nn,oc |
 |[GlobalVoices](https://opus.nlpl.eu/GlobalVoices/corpus/version/GlobalVoices)		| bg,de,fr,it,nl,pl,pt	|bg,de,fr,pt		|  |
 |[GNOME](https://opus.nlpl.eu/GNOME/corpus/version/GNOME)		|eu,fr,ga,gl,pt		|ga		|cy,ga,nn|  
 |[JRC-Arquis](https://opus.nlpl.eu/JRC-Acquis/corpus/version/JRC-Acquis)		|			|cs,da,et,fr,lt,lv,mt,nl,pl	,ro,sv|	 et  |
+|LES-CORTS-VALENCIANES-BSC  |            | val            |           |
 |[MaCoCu](https://opus.nlpl.eu/MaCoCu/corpus/version/MaCoCu)                    | en     |     | hr,mt,uk   |
 |[MultiCCAligned](https://opus.nlpl.eu/JRC-Acquis/corpus/version/JRC-Acquis)	|bg,cs,de,el,et,fi,fr,hr,hu,it,lt,lv,nl,pl,ro,sk,sv	|bg,fi,fr,hr,it,lv,nl,pt		|bg,cy,da,et,fi,hr,hu,lt,lv,no,sl,sr,uk|
 |[MultiHPLT](https://opus.nlpl.eu/MultiHPLT/corpus/version/MultiHPLT)		|en, et,fi,ga,hr,mt		|		|fi,ga,gl,hr,mt,nn,sr |
@@ -87,14 +93,18 @@ Click the expand button below to see the full list of corpora included in the tr
 |[MultiUN](https://opus.nlpl.eu/MultiUN/corpus/version/MultiUN)		|			|fr	|	|	
 |[News-Commentary](https://opus.nlpl.eu/News-Commentary/corpus/version/News-Commentary) 	|		|fr		|  |
 |[NLLB](https://opus.nlpl.eu/NLLB/corpus/version/NLLB)			|bg,da,el,en,et,fi,fr,gl,hu,it	,lt,lv,pt,ro,sk,sl	|bg,cs,da,de,el	,et,fi,fr,hu,it,lt,lv,nl,pl,pt	,ro,sk,sl,sv| bg,cs,cy,da,de,el,et,fi,fr,ga,hr,hu,it,lt,lv,mt,nl,no,oc,pl,pt,ro,ru,sk,sl,sr,sv,uk|
+|[NÓS](https://zenodo.org/records/7675110)                 |               |               |    gl      |
+|[NÓS-SYN](https://zenodo.org/records/7685180)            |                |               |   gl       |
 |[NTEU](https://www.elrc-share.eu/repository/search/?q=NTEU)			|			|bg,cs,da,de,el,en,et,fi,fr,ga,hr,hu,it,lt,lv,mt,nl,pl,pt,ro,sk,sl,sv	|        da,et,ga,hr,lt,lv,mt,ro,sk,sl,sv     |	 
-|[OpenSubtitles](https://opus.nlpl.eu/OpenSubtitles/corpus/version/OpenSubtitles) 	|bg,cs,da,de,el	,et,eu,fi,gl,hr,hu,lt,lv,nl,pl,pt,ro,sk,sl,sv	|da,de,fi,fr,hr,hu,it,lv,nl		| bg,cs,de,el,et,hr,fi,fr,gl_pt,hr,hu,no,sl,sr|
+|[OpenSubtitles](https://opus.nlpl.eu/OpenSubtitles/corpus/version/OpenSubtitles) 	|bg,cs,da,de,el	,et,eu,fi,gl,hr,hu,lt,lv,nl,pl,pt,ro,sk,sl,sv	|da,de,fi,fr,hr,hu,it,lv,nl		| bg,cs,de,el,et,hr,fi,fr,hr,hu,no,sl,sr|
 |[OPUS-100](https://opus.nlpl.eu/opus-100.php)   | en |  | gl |
 |[StanfordNLP-NMT](https://opus.nlpl.eu/StanfordNLP-NMT/corpus/version/StanfordNLP-NMT) | | |cs |
 |[Tatoeba](https://opus.nlpl.eu/Tatoeba/corpus/version/Tatoeba)		|de,pt			|pt		|   |
 |[TildeModel](https://opus.nlpl.eu/TildeMODEL/corpus/version/TildeMODEL)		|			|bg		| et,hr,lt,lv,mt |
 |[UNPC](https://opus.nlpl.eu/UNPC/corpus/version/UNPC)			|			|en,fr		| ru  |	
-|[WikiMatrix](https://opus.nlpl.eu/WikiMatrix/corpus/version/WikiMatrix)		|bg,cs,da,de,el	,et,eu,fi,fr,gl,hr,hu,it,lt,nl,pl,pt,ro,sk,sl,sv	|bg,en,fr,hr,it,pt		| gl,oc,sh |
+|[VALENCIAN-AUTH](https://github.com/transducens/PILAR/tree/main/valencian/Generalitat)  |          |    val     |          |
+|[VALENCIAN-SYNTH](https://github.com/transducens/PILAR/tree/main/valencian/Generalitat)  |      | val |     |
+|[WikiMatrix](https://opus.nlpl.eu/WikiMatrix/corpus/version/WikiMatrix)		|bg,cs,da,de,el	,et,eu,fi,fr,gl,hr,hu,it,lt,nl,pl,pt,ro,sk,sl,sv	|bg,en,fr,hr,it,pt		| oc,sh |
 |[Wikimedia](https://opus.nlpl.eu/wikimedia/corpus/version/wikimedia) | | |cy,nn |
 |[XLENT](https://opus.nlpl.eu/XLEnt/corpus/version/XLEnt)		|eu,ga,gl			|ga		|cy,et,ga,gl,hr,oc,sh|