| --- | |
| language: en | |
| tags: | |
| - biomedical | |
| - lexical-semantics | |
| datasets: | |
| - UMLS | |
| **[news]** A cross-lingual extension of SapBERT will appear in the main onference of **ACL 2021**! <br> | |
| **[news]** SapBERT will appear in the conference proceedings of **NAACL 2021**! | |
| ### SapBERT-PubMedBERT | |
| SapBERT by [Liu et al. (2020)](https://arxiv.org/pdf/2010.11784.pdf). Trained with [UMLS](https://www.nlm.nih.gov/research/umls/licensedcontent/umlsknowledgesources.html) 2020AA (English only), using [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext) as the base model. Please use the mean-pooling of the output as the representation. | |
| ### Citation | |
| ```bibtex | |
| @article{liu2020self, | |
| title={Self-alignment Pre-training for Biomedical Entity Representations}, | |
| author={Liu, Fangyu and Shareghi, Ehsan and Meng, Zaiqiao and Basaldella, Marco and Collier, Nigel}, | |
| journal={arXiv preprint arXiv:2010.11784}, | |
| year={2020} | |
| } | |
| ``` | |