Update README.md

README.md

@@ -39,7 +39,11 @@ widget:
 </details>
 
 ## Model description
-The longformer-base-4096-bne-es is the [Longformer](https://huggingface.co/allenai/longformer-base-4096) version of the [roberta-base-bne](https://https://huggingface.co/PlanTL-GOB-ES/roberta-base-bne) masked language model for the Spanish language. Using this kind of models, allows us to process larger contexts as input without needing to use additional aggregation strategies. The model started from the **roberta-base-bne** checkpoint and was pretrained for MLM on long documents from our biomedical and clinical corpora.
+The longformer-base-4096-bne-es is the [Longformer](https://huggingface.co/allenai/longformer-base-4096) version of the [roberta-base-bne](https://https://huggingface.co/PlanTL-GOB-ES/roberta-base-bne) masked language model for the Spanish language. Using this kind of models, allows us to process larger contexts as input without needing to use additional aggregation strategies. The model started from the **roberta-base-bne** checkpoint and was pretrained for MLM on long documents from the National Library of Spain.
+
+The Longformer model uses a combination of sliding window (local) attention and global attention. Global attention is user-configured based on the task to allow the model to learn task-specific representations. Please refer to the original [paper](https://arxiv.org/abs/2004.05150) for more details on how to set global attention.
+
+For more details about the corpus, the pretraining, and the evaluation, check the official [repository](https://github.com/TeMU-BSC/longformer-es).
 
 ## Intended uses and limitations
 The longformer-base-4096-biomedical-clinical-es model is ready-to-use only for masked language modeling to perform the Fill Mask task (try the inference API or read the next section).