--- license: apache-2.0 datasets: - stanfordnlp/imdb - bharatiyabytes/sentimentWithSarcasm language: - en base_model: - google/flan-t5-small pipeline_tag: text-classification tags: - sarcasm - gen-ai - llms --- # Model Card for Sarcasm-Enhanced Sentiment Analysis Model This model performs sentiment analysis with a specific focus on detecting sarcasm in textual content. It is fine-tuned on a combination of standard sentiment datasets and specialized sarcastic data, allowing for more nuanced sentiment classification that accounts for sarcastic language. ## Model Details ### Model Description This model leverages the **Flan-T5-small** transformer architecture, fine-tuned on datasets including **IMDB** for sentiment analysis and **bharatiyabytes/sentimentWithSarcasm** for sarcasm detection. By combining these datasets, the model is better equipped to differentiate between sarcastic and genuine sentiment expressions, improving sentiment analysis accuracy in contexts where sarcasm is prevalent. - **Developed by:** bharatiyabytes - **Funded by [optional]:** Not yet funded - **Model type:** Text Classification - **Language(s) (NLP):** English (en) - **License:** Apache 2.0 - **Fine-tuned from model:** google/flan-t5-small ### Model Sources [optional] - **Repository:** https://github.com/sohi-g/lets-talk-the-hype.git - **Paper [optional]:** [Link to paper if available] - **Demo [optional]:** [Link to demo if available] ## Uses ### Direct Use This model can be used directly for sentiment classification on texts where sarcasm may obscure intended sentiment. It provides accurate classifications by considering nuanced expressions, making it ideal for social media analysis, customer feedback processing, and sarcasm-rich content sources. ### Downstream Use This model can serve as a base for further fine-tuning for domains that require sarcasm-aware sentiment analysis, such as customer service, public relations, and social media monitoring applications. ### Out-of-Scope Use The model may not perform well on texts with heavy dialects or informal language that goes beyond the sarcasm in the fine-tuning data. It is not intended for multi-lingual sarcasm detection. ## Bias, Risks, and Limitations As with many sentiment analysis models, there may be inherent biases in the sarcasm and sentiment labels in the training datasets, potentially affecting model performance across different demographic or cultural groups. Users should be cautious when using this model in critical decision-making contexts. ### Recommendations Users should perform additional validation on specific datasets to ensure that model predictions align with intended use cases, especially in high-stakes applications. ## How to Get Started with the Model Use the code below to get started with the model: ```python from transformers import AutoTokenizer, AutoModelForSequenceClassification import torch tokenizer = AutoTokenizer.from_pretrained("your-username/sarcasm-sentiment-analysis") model = AutoModelForSequenceClassification.from_pretrained("your-username/sarcasm-sentiment-analysis") inputs = tokenizer("Your input text here", return_tensors="pt") outputs = model(**inputs) predictions = torch.argmax(outputs.logits, dim=-1) ``` ## Training Details ### Training Data The model was fine-tuned on a combination of IMDB (a general sentiment analysis dataset) and bharatiyabytes/sentimentWithSarcasm (designed to capture sarcastic sentiment). This blend improves the model’s ability to identify nuanced sentiment. ### Training Procedure #### Preprocessing Standard text preprocessing methods were applied, such as tokenization and lowercase transformation. #### Training Hyperparameters Training regime: FP32 Epochs: 3 Batch Size: 16 Learning Rate: 3e-5 #### Speeds, Sizes, Times Training took approximately 4 hours on an NVIDIA V100 GPU with a model size of 60M parameters. ## Evaluation ### Testing Data, Factors & Metrics #### Testing Data he model was evaluated on the IMDB and bharatiyabytes/sentimentWithSarcasm test splits. #### Factors Evaluation considers sentiment disaggregation by sarcastic vs. non-sarcastic samples. #### Metrics -Accuracy: Measures overall sentiment classification accuracy. -F1 Score (Sarcasm): Evaluates the model’s sarcasm detection capability, which is key for accurate sarcastic sentiment handling. ### Results The model achieved an accuracy of 88% on the sentiment classification task and an F1 score of 0.83 on sarcasm detection. #### Summary The model shows strong performance in sarcasm-sensitive sentiment analysis, making it suitable for applications where nuanced sentiment interpretation is crucial. ## Model Examination The model’s predictions have been examined to ensure that sarcastic content is accurately labeled, using interpretability tools such as SHAP to visualize model attention on sarcastic phrases. ## Environmental Impact Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). - **Hardware Type:** NVIDIA V100 GPU - **Hours used:** Approximately 4 hours - **Cloud Provider:** Google - **Compute Region:** USA - **Carbon Emitted:** NA ## Technical Specifications ### Model Architecture and Objective The model uses the Flan-T5-small architecture fine-tuned for binary sentiment classification with sarcasm detection as an enhancement. ### Compute Infrastructure [More Information Needed] #### Hardware NVIDIA V100 GPU #### Software Hugging Face Transformers Library, PyTorch ## Citation [optional] **BibTeX:** [More Information Needed] **APA:** [More Information Needed] ## Glossary [optional] [More Information Needed] ## More Information [optional] [More Information Needed] ## Model Card Authors - **Shivang sinha** - **Garima Sohi** - **Parteek** ## Model Card Contact shivangsinha2@gmail.com