Update README.md
Browse files
README.md
CHANGED
@@ -8,6 +8,7 @@ license: cc-by-nc-4.0
|
|
8 |
- In the paper, it is referred as `HuggingFace-BERTweet (TB2+EWT)` in the POS table.
|
9 |
|
10 |
## How to use the model
|
|
|
11 |
|
12 |
```python
|
13 |
from transformers import AutoTokenizer, AutoModelForTokenClassification
|
|
|
8 |
- In the paper, it is referred as `HuggingFace-BERTweet (TB2+EWT)` in the POS table.
|
9 |
|
10 |
## How to use the model
|
11 |
+
- **PRE-PROCESSING**: when you apply the model on tweets, please make sure that tweets are preprocessed by the [TweetTokenizer](https://github.com/VinAIResearch/BERTweet/blob/master/TweetNormalizer.py) to get the best performance.
|
12 |
|
13 |
```python
|
14 |
from transformers import AutoTokenizer, AutoModelForTokenClassification
|