Emotions Classification with DistilBERT Base Fine-Tuned Model

Model Overview

Model Architecture: DistilBERT (a distilled version of BERT) is used as the base model for sentiment classification.
Number of Labels: The model is trained for a multi-class classification task with 9 labels representing different sentiments.
Training Data: The model is trained on Twitter emotions data, which includes tweet IDs, sentiments, and content.

Usage

Loading the Model

You can load the fine-tuned model using the transformers library:


from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained("Sentiment_Classification_DistilBertBase_FT")

Inference

You can use the loaded model for sentiment classification on your own text data:


from transformers import AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

text = "Layin n bed with a headache ughhhh...waitin o..."
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
outputs = model(**inputs)
logits = outputs.logits

# To get the predicted sentiment label
predicted_label_id = torch.argmax(logits, dim=1).item()

Training

If you want to fine-tune the model further or train it on a different dataset, you can use the provided training script:


from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer

model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased",
    num_labels=9,  # Number of sentiment labels
    id2label=id2label,  # Mapping from label IDs to label names
    label2id=label2id,  # Mapping from label names to label IDs
)

training_args = TrainingArguments(
    output_dir="Sentiment_Classification_DistilBertBase_FT",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=2,
    weight_decay=0.01,
    evaluation_strategy="epoch",
    save_strategy="epoch",
    load_best_model_at_end=True,
    push_to_hub=True,  # Push the trained model to the Hugging Face Model Hub
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_ds["train"],  # Your training dataset
    eval_dataset=tokenized_ds["test"],  # Your evaluation dataset
    tokenizer=tokenizer,
    data_collator=data_collator,  # Your data collator function
    compute_metrics=compute_metrics,  # Your metrics function
)

Model Training and Fine-Tuning

The model was fine-tuned using the provided training arguments and the Twitter sentiment dataset. The training and evaluation strategies are set to perform evaluations after each epoch and save the best model. The model uses a learning rate of 2e-5 and a batch size of 16 for both training and evaluation.

Performance Metrics

You can evaluate the model using appropriate performance metrics defined in your compute_metrics function, which can include accuracy, F1-score, precision, recall, etc.

Acknowledgments

The base model "distilbert-base-uncased" is provided by Hugging Face Transformers.
The Twitter sentiment dataset is used for fine-tuning. Please make sure to acknowledge the source of this dataset in your project.

Author

This model and README were created by Mahendra Kharra.

For any questions or issues, please contact mahendrakharra@gmail.com.