readme: add some minor speculation about RoPE
Browse files
README.md
CHANGED
@@ -17,7 +17,7 @@ This repository hosts an NeoBERT model that was fine-tuned on the CoNLL-2003 NER
|
|
17 |
Please notice the following caveats:
|
18 |
|
19 |
* ⚠️ Work in progress, as e.g. new hyper-parameter changes or bug fixes for the implemented `NeoBERTForTokenClassification` class can occur.
|
20 |
-
* ⚠️ At the moment, don't expect BERT-like performance, more experiments are needed
|
21 |
|
22 |
## 📝 Implementation
|
23 |
|
|
|
17 |
Please notice the following caveats:
|
18 |
|
19 |
* ⚠️ Work in progress, as e.g. new hyper-parameter changes or bug fixes for the implemented `NeoBERTForTokenClassification` class can occur.
|
20 |
+
* ⚠️ At the moment, don't expect BERT-like performance, more experiments are needed. (Is RoPE causing this?)
|
21 |
|
22 |
## 📝 Implementation
|
23 |
|