Create README.md

README.md

@@ -0,0 +1,145 @@
+---
+license: apache-2.0
+datasets:
+- poem_sentiment
+language:
+- en
+metrics:
+- Accuracy, F1 score
+library_name: transformers
+pipeline_tag: text-classification
+tags:
+- text-classification
+- sentiment-analysis
+- poem-sentiment-detection
+- poem-sentiment
+- poem-sentiment-classification
+- sentiment-classification
+widget:
+- text: >-
+    Rapidly, merrily, Life's sunny hours flit by, Gratefully, cheerily, Enjoy them as they fly!
+  example_title: "Life" 
+- text: It so happens I am sick of my feet and my nails, and my hair and my shadow. It so happens I am sick of being a man.
+  example_title: "Walking Around" 
+- text: >-
+    No man is an island, Entire of itself, Every man is a piece of the continent, A part of the main.
+  example_title: "No man is an island" 
+- text: >-
+    Some have won a wild delight, By daring wilder sorrow; Could I gain thy love to-night, I'd hazard death to-morrow.
+  example_title: "Passion" 
+---
+ ## AiManatee/RoBERTa_poem_sentiment 
+This model is a fine-tuned version of the [FacebookAI/roberta-base](https://huggingface.co/FacebookAI/roberta-base) transformer for the task of poem sentiment analysis. It predicts the sentiment of a given poem verse into one of four categories: negative, positive, no impact, or mixed (positive and negative).
+
+ ### Dataset
+RoBERTa_poem_sentiment was trained on the [poem_sentiment](https://huggingface.co/datasets/poem_sentiment) dataset which consists of poem verses across four sentiment labels: negative, positive, no impact, and mixed sentiment. However, the Validation and Test subsets of the original dataset lack 'mixed' sentiment examples. To address this and ensure a thorough evaluation, data augmentation was performed: 32 'mixed' sentiment verses from different English poems were added to the Validation (16) and Test (16) subsets; the original Train subset remained intact. All the augmented samples were tested for semantic consistency, diversity (cosine similarity), length variation and novelty (ensuring the augmented data introduced new, relevant vocabulary). This strategy allowed for a more comprehensive evaluation of the model's generalization ability across all trained labels. The final model was tested on both the original dataset and the augmented dataset.
+
+ #### Labels
+```
+{0: 'negative', 1: 'positive', 2: 'no_impact', 3: 'mixed'}
+```
+
+ ### Training Hyperparameters
+```
+  learning_rate: 2e-5,
+  weight_decay: 0.01,
+  batch_size: 16,
+  num_epochs: 8,
+  optimizer: AdamW: betas=(0.9, 0.999), eps=1e-08
+  seed: 16
+  early_stopper: min_delta=0.001, patience=3
+```
+```
+  scheduler = ReduceLROnPlateau(
+    optimizer,
+    mode="min",
+    factor=0.5,
+    patience=0,
+    threshold=0.001,
+    eps=1e-8,
+  )
+```
+
+ ### Model Performance
+##### Validation results on the original dataset (class 3 is not being evaluated here)
+| Epoch | Training Loss | Validation Loss | Accuracy | F1       |
+|-------|---------------|-----------------|----------|----------|
+| 1     | 1.365169      | 1.010353        | 0.761905 | 0.771733 |
+| 2     | 0.860945      | 0.810045        | 0.723810 | 0.740809 |
+| 3     | 0.570005      | 0.637439        | 0.761905 | 0.802184 |
+| 4     | 0.355776      | 0.699637        | 0.780952 | 0.797572 |
+| 5     | 0.252919      | 0.586395        | 0.847619 | 0.860519 |
+| 6     | 0.156633      | 0.610439        | 0.819048 | 0.834072 |
+| 7     | 0.084868      | 0.515130        | 0.876190 | 0.884736 |
+| 8     | 0.062830      | 0.572643        | 0.885714 | 0.902510 |
+
+
+ ##### Validation results on the augmented dataset
+| Epoch | Training Loss | Validation Loss | Accuracy | F1       |
+|-------|---------------|-----------------|---------------------|
+| 1     | 1.365169      | 1.168057        | 0.661157 | 0.628737 |
+| 2     | 0.860945      | 0.869521        | 0.694214 | 0.717916 |
+| 3     | 0.570005      | 0.643639        | 0.776859 | 0.790842 |
+| 4     | 0.355776      | 0.681563        | 0.768595 | 0.776540 |
+| 5     | 0.252919      | 0.585692        | 0.834710 | 0.841590 |
+| 6     | 0.156633      | 0.542949        | 0.809917 | 0.815361 |
+| 7     | 0.092444      | 0.581075        | 0.826446 | 0.830607 |
+| 8     | 0.049480      | 0.583749        | 0.884297 | 0.881360 |
+
+
+
+ ### How to Use the Model
+Here is how to predict the sentiment of a poem verse using this model:
+
+```python
+from transformers import pipeline
+sentiment_classifier = pipeline(task='text-classification', model='AiManatee/RoBERTa_poem_sentiment')
+verse1 = "Rapidly, merrily, Life's sunny hours flit by, Gratefully, cheerily, Enjoy them as they fly!"
+verse2 = "It so happens I am sick of my feet and my nails, and my hair and my shadow. It so happens I am sick of being a man."
+verse3 = "No man is an island, Entire of itself, Every man is a piece of the continent, A part of the main."
+verse4 = "Some have won a wild delight, By daring wilder sorrow; Could I gain thy love to-night, I'd hazard death to-morrow."
+print(sentiment_classifier(verse1))
+print(sentiment_classifier(verse2))
+print(sentiment_classifier(verse3))
+print(sentiment_classifier(verse4))
+```
+
+ ### Evaluation
+ ##### Original dataset
+```
+{Loss: 0.5726433790155819
+Accuracy: 0.8857142857142857
+Precision: 0.9201298701298701
+Recall: 0.8857142857142857
+F1: 0.9025108225108224
+}
+```
+
+ ##### Augmented dataset
+```
+{Loss: 0.5837492472492158
+Accuracy: 0.8842975206611571
+Precision: 0.8810538160090016
+Recall: 0.8842975206611571
+F1: 0.8813606847697756
+}
+```
+ ### Citation
+
+If you find this model useful in your research, please consider citing:
+
+```bibtex
+@misc{roberta_poem_sentiment,
+  author = {Your Name},
+  title = {RoBERTa for Poem Sentiment Analysis},
+  year = {2024},
+  publisher = {Hugging Face's Model Hub},
+  howpublished = {\url{https://huggingface.co//AiManatee/RoBERTa_poem_sentiment}},
+}
+```
+
+ ### Framework Versions
+- **Transformers:** 4.35.2
+- **PyTorch:** 2.1.0+cu118
+- **Datasets:** 2.16.1
+- **Tokenizers:** 0.15.1