File size: 2,135 Bytes
072d4a1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
---
language: en
tags:
- sentiment-analysis
- transformers
- pytorch
license: apache-2.0
datasets:
- custom-dataset
metrics:
- accuracy
model_name: distilbert-base-uncased-finetuned-sentiment
---
# DistilBERT Base Uncased Fine-tuned for Sentiment Analysis
## Model Description
This model is a fine-tuned version of `distilbert-base-uncased` on a sentiment analysis dataset. It is trained to classify text into positive and negative sentiment categories.
## Training Details
The model was fine-tuned on a sentiment analysis dataset using the Hugging Face `transformers` library. The training parameters are as follows:
- **Learning Rate**: 2e-5
- **Batch Size**: 32
- **Number of Epochs**: 4
- **Optimizer**: AdamW
- **Scheduler**: Linear with warmup
- **Device**: Nvidia T4 GPU
## Training and Validation Metrics
| Step | Training Loss | Validation Loss | Accuracy |
|------|---------------|-----------------|----------|
| 400 | 0.389300 | 0.181316 | 93.25% |
| 800 | 0.161900 | 0.166204 | 94.13% |
| 1200 | 0.114600 | 0.200135 | 94.30% |
| 1600 | 0.076300 | 0.211609 | 94.40% |
| 2000 | 0.041600 | 0.225439 | 94.45% |
Final training metrics:
- **Global Step**: 2000
- **Training Loss**: 0.156715
- **Training Runtime**: 1257.5696 seconds
- **Training Samples per Second**: 50.892
- **Training Steps per Second**: 1.59
- **Total FLOPS**: 8477913513984000.0
- **Epochs**: 4.0
## Model Performance
The model achieves an accuracy of approximately 94.45% on the validation set.
## Usage
To use this model for sentiment analysis, you can load it using the `transformers` library:
```python
from transformers import DistilBertTokenizerFast, DistilBertForSequenceClassification
model_name = 'luluw/distilbert-base-uncased-finetuned-sentiment'
tokenizer = DistilBertTokenizerFast.from_pretrained(model_name)
model = DistilBertForSequenceClassification.from_pretrained(model_name)
# Example usage
text = "I love this product!"
inputs = tokenizer(text, return_tensors='pt')
outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=-1)
```
|