Update README.md
Browse files
README.md
CHANGED
|
@@ -2,14 +2,21 @@
|
|
| 2 |
CryptoBERT is a pre-trained NLP model to analyse the language and sentiments of cryptocurrency-related social media posts and messages. It is built by further training the [cardiffnlp's Twitter-roBERTa-base](https://huggingface.co/cardiffnlp/twitter-roberta-base) language model on the cryptocurrency domain, using a corpus of over 3.2M unique cryptocurrency-related social media posts.
|
| 3 |
|
| 4 |
|
|
|
|
|
|
|
|
|
|
| 5 |
## Classification Training
|
| 6 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
## Training Corpus
|
| 9 |
CryptoBERT was trained on 3.2M social media posts regarding various cryptocurrencies. Only non-duplicate posts of length above 4 words were considered. The following communities were used as sources for our corpora:
|
| 10 |
|
| 11 |
|
| 12 |
-
(1) StockTwits - 1.875M posts about the top 100 cryptos by trading volume. Posts were collected from the 1st of November 2021 to the 16th of June 2022.
|
| 13 |
|
| 14 |
(2) Telegram - 664K posts from top 5 telegram groups: [Binance](https://t.me/binanceexchange), [Bittrex](https://t.me/BittrexGlobalEnglish), [huobi global](https://t.me/huobiglobalofficial), [Kucoin](https://t.me/Kucoin_Exchange), [OKEx](https://t.me/OKExOfficial_English).
|
| 15 |
Data from 16.11.2020 to 30.01.2021. Courtesy of [Anton](https://www.kaggle.com/datasets/aagghh/crypto-telegram-groups).
|
|
|
|
| 2 |
CryptoBERT is a pre-trained NLP model to analyse the language and sentiments of cryptocurrency-related social media posts and messages. It is built by further training the [cardiffnlp's Twitter-roBERTa-base](https://huggingface.co/cardiffnlp/twitter-roberta-base) language model on the cryptocurrency domain, using a corpus of over 3.2M unique cryptocurrency-related social media posts.
|
| 3 |
|
| 4 |
|
| 5 |
+
# Example of Classification
|
| 6 |
+
|
| 7 |
+
|
| 8 |
## Classification Training
|
| 9 |
+
The model was trained on the following labels: "Bearish" : 0, "Neutral": 1, "Bullish": 2
|
| 10 |
+
|
| 11 |
+
CryptoBERT's sentiment classification head was fine-tuned on a balanced dataset of 2M labelled StockTwits posts, bootstrapped from [ElKulako/StockTwits-crypto](https://huggingface.co/datasets/ElKulako/StockTwits-crypto).
|
| 12 |
+
|
| 13 |
+
CryptoBERT was trained with a max sequence length of 128. Technically, it can handle sequences of up to 514 tokens, however, going beyond 128 is not recommended.
|
| 14 |
|
| 15 |
## Training Corpus
|
| 16 |
CryptoBERT was trained on 3.2M social media posts regarding various cryptocurrencies. Only non-duplicate posts of length above 4 words were considered. The following communities were used as sources for our corpora:
|
| 17 |
|
| 18 |
|
| 19 |
+
(1) StockTwits - 1.875M posts about the top 100 cryptos by trading volume. Posts were collected from the 1st of November 2021 to the 16th of June 2022. [ElKulako/StockTwits-crypto](https://huggingface.co/datasets/ElKulako/StockTwits-crypto)
|
| 20 |
|
| 21 |
(2) Telegram - 664K posts from top 5 telegram groups: [Binance](https://t.me/binanceexchange), [Bittrex](https://t.me/BittrexGlobalEnglish), [huobi global](https://t.me/huobiglobalofficial), [Kucoin](https://t.me/Kucoin_Exchange), [OKEx](https://t.me/OKExOfficial_English).
|
| 22 |
Data from 16.11.2020 to 30.01.2021. Courtesy of [Anton](https://www.kaggle.com/datasets/aagghh/crypto-telegram-groups).
|