asier-gutierrez commited on
Commit
c4e738c
1 Parent(s): ddcb184

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -35,7 +35,7 @@ Some of the statistics of the corpus:
35
  | BNE | 201,080,084 | 135,733,450,668 | 570GB |
36
 
37
  ## Tokenization and pre-training
38
- The training corpus has been tokenized using a byte version of Byte-Pair Encoding (BPE) used in the original [RoBERTA](https://arxiv.org/abs/1907.11692) model with a vocabulary size of 50,262 tokens. The RoBERTa-base-bne pre-training consists of a masked language model training that follows the approach employed for the RoBERTa base model with the same hyperparameters as in the original work. The training lasted a total of 48 hours with 16 computing nodes each one with 4 NVIDIA V100 GPUs of 16GB VRAM.
39
 
40
  ## Evaluation and results
41
  For evaluation details visit our [GitHub repository](https://github.com/PlanTL-SANIDAD/lm-spanish).
 
35
  | BNE | 201,080,084 | 135,733,450,668 | 570GB |
36
 
37
  ## Tokenization and pre-training
38
+ The training corpus has been tokenized using a byte version of Byte-Pair Encoding (BPE) used in the original [RoBERTA](https://arxiv.org/abs/1907.11692) model with a vocabulary size of 50,262 tokens. The RoBERTa-base-bne pre-training consists of a masked language model training that follows the approach employed for the RoBERTa base. The training lasted a total of 48 hours with 16 computing nodes each one with 4 NVIDIA V100 GPUs of 16GB VRAM.
39
 
40
  ## Evaluation and results
41
  For evaluation details visit our [GitHub repository](https://github.com/PlanTL-SANIDAD/lm-spanish).