Create README.md

README.md

@@ -0,0 +1,71 @@
+---
+language: "en"
+tags:
+- twitter
+- masked-token-prediction
+- election2020
+license: "gpl-3.0"
+---
+
+# Pre-trained BERT on Twitter US Political Election 2020
+
+Pre-trained weights for [Knowledge Enhance Masked Language Model for Stance Detection](https://2021.naacl.org/program/accepted/), NAACL 2021.
+
+# Training Data
+
+This model is pre-trained on over 5 million English tweets about the 2020 US Presidential Election.
+
+# Training Objective
+
+This model is initialized with BERT-base and trained with normal MLM objective.
+
+# Usage
+
+This pre-trained language model **can be fine-tunned to any downstream task (e.g. classification)**.
+
+Please see the [official repository](https://github.com/GU-DataLab/stance-detection-KE-MLM) for more detail.
+
+```python
+from transformers import BertTokenizer, BertForMaskedLM, pipeline
+import torch
+
+# choose GPU if available
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+# select mode path here
+pretrained_LM_path = "kornosk/bert-political-election2020-twitter-mlm"
+
+# load model
+tokenizer = BertTokenizer.from_pretrained(pretrained_LM_path)
+model = BertForMaskedLM.from_pretrained(pretrained_LM_path)
+
+# fill mask
+example = "Trump is the [MASK] of USA"
+fill_mask = pipeline('fill-mask', model=model, tokenizer=tokenizer)
+
+outputs = fill_mask(example)
+print(outputs)
+
+# see embeddings
+inputs = tokenizer(example, return_tensors="pt")
+outputs = model(**inputs)
+print(outputs)
+
+# OR you can use this model to train on your downstream task!
+# please consider citing our paper if you feel this is useful :)
+```
+
+# Reference
+
+- [Knowledge Enhance Masked Language Model for Stance Detection](https://2021.naacl.org/program/accepted/), NAACL 2021.
+
+# Citation
+```bibtex
+@inproceedings{kawintiranon2021knowledge,
+    title={Knowledge Enhanced Masked Language Model for Stance Detection},
+    author={Kawintiranon, Kornraphop and Singh, Lisa},
+    booktitle={Proceedings of the 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)},
+    year={2021},
+    url={#}
+}
+```