---
language:
- el
pipeline_tag: text-classification
---
# PaloBERT for Sentiment Analysis

A greek [RoBERTa](https://arxiv.org/abs/1907.11692) based model ([PaloBERT](https://huggingface.co/pchatz/palobert-base-greek-social-media): an updated version of [palobert-base-greek-uncased-v1](https://huggingface.co/gealexandri/palobert-base-greek-uncased-v1)) fine-tuned for sentiment analysis.

## Training data

The model is pre-trained on a corpus of 458,293 documents collected from greek social media (Twitter, Instagram, Facebook and YouTube). A RoBERTa tokenizer trained from scratch on the same corpus is also included. The fine-tuning process is done on a dataset of ~60,000 documents, also collected from greek social media.

The corpus as well as the annotated dataset have been provided by [Palo LTD](http://www.paloservices.com/).

## Requirements

```
pip install transformers
pip install torch

```

## Pre-processing details

In order to use this model, the text needs to be pre-processed as follows:

* remove all greek diacritics
* convert to lowercase
* remove all punctuation

```python
import re
import unicodedata

def preprocess(text, default_replace=""):
  text = text.lower()
  text = unicodedata.normalize('NFD',text).translate({ord('\N{COMBINING ACUTE ACCENT}'):None})
  text = re.sub(r'[^\w\s]', default_replace, text)
  return text
```

## Load Model

```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("pchatz/palobert-base-greek-social-media-v2") #load PaloBERT pre-trained model
language_model = AutoModel.from_pretrained("pchatz/palobert-base-greek-social-media-v2")
```
Refer to [GitHub](https://github.com/Paulinechatz/sentiment-analysis-greek-social-media/blob/main/code/train_classifier_roberta_arch.py#L100) code for details on ModelClass architecture
```python
model = TheModelClass(*args, **kwargs) #load fine-tuned model as SentimentClassifier_v2
model.load_state_dict(torch.load(PATH))
model.eval()
```
You can use this sentiment analysis model directly on raw text:
```python
#Example
class_names={0: 'neutral', 1:'positive', 2:'negative'}
text='οι εξετασεις ηταν πολυ καλες' 
encoding=tokenizer(text,return_tensors='pt')

input_ids = encoding['input_ids']
attention_mask = encoding['attention_mask']

output = model(input_ids, attention_mask)
_,prediction = torch.max(output, dim=1)

print(f'sentiment  : {class_names[prediction.item()]}') #positive
```

## Evaluation

For detailed results refer to Thesis: ['Ανάλυση συναισθήματος κειμένου στα Ελληνικά με χρήση Δικτύων Μετασχηματιστών'](http://artemis.cslab.ece.ntua.gr:8080/jspui/handle/123456789/18623) (version - p2)

## Author

[Pavlina Chatziantoniou](https://huggingface.co/pchatz), [Georgios Alexandridis](https://huggingface.co/gealexandri) and Athanasios Voulodimos

## BibTeX entry and citation info

http://artemis.cslab.ece.ntua.gr:8080/jspui/handle/123456789/18623

```bibtex

@Article{info12080331,
AUTHOR = {Alexandridis, Georgios and Varlamis, Iraklis and Korovesis, Konstantinos and Caridakis, George and Tsantilas, Panagiotis},
TITLE = {A Survey on Sentiment Analysis and Opinion Mining in Greek Social Media},
JOURNAL = {Information},
VOLUME = {12},
YEAR = {2021},
NUMBER = {8},
ARTICLE-NUMBER = {331},
URL = {https://www.mdpi.com/2078-2489/12/8/331},
ISSN = {2078-2489},
DOI = {10.3390/info12080331}
}
```