File size: 2,336 Bytes
9273a18 061b42f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
---
datasets:
- samsum
language:
- en
metrics:
- rouge
- bleu
library_name: transformers
pipeline_tag: summarization
---
# t5-small-finetuned
## Model Description
- **Purpose and Use**: This model is designed for abstractive text summarization with a focus on the SAMSum Dialogue Dataset.
- **Model Architecture**: The architecture is based on a fine-tuned T5-small model, which consists of 60 million parameters.
- **Training Data**: Trained on the SAMSum Dialogue Dataset, which comprises approximately 15,000 dialogue-summary pairs.
## Training Procedure
- **Preprocessing**: Data preprocessing involved the removal of irrelevant tags and tokenization to ensure data consistency.
- **Training Details**: The model was fine-tuned over 4 epochs with a learning rate of 2e-5 and a batch size of 2, utilizing gradient accumulation for optimization.
- **Infrastructure**: Training was conducted using GPU acceleration and the Hugging Face Trainer API, with progress monitored via TensorBoard.
## Evaluation Results
- **Metrics Used**: Evaluation metrics included ROUGE-1, ROUGE-2, ROUGE-L, BLEU, and Cosine Similarity.
- **Performance**: The fine-tuned T5-small model demonstrated superior efficiency and effectiveness in summarization tasks, outperforming its larger counterparts.
## Validation and Test Set Performance
| Metric | Validation Set | Test Set |
|----------|--------------------|--------------|
| ROUGE-1 | 0.5667 | 0.5536 |
| ROUGE-2 | 0.2923 | 0.2718 |
| ROUGE-L | 0.5306 | 0.5210 |
The table above presents the performance of the model on both the validation and test sets, indicating the quality of content overlap and structural fluency in the summaries generated.
## Performance Metrics Comparison Across Models
| Model | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU | Cosine Similarity |
|----------|---------|---------|---------|------|-------------------|
| My Model | 0.3767 | 0.1596 | 0.2896 | 9.52 | 0.7698 |
| T5 Large | 0.3045 | 0.0960 | 0.2315 | 4.82 | 0.6745 |
| Bart | 0.3189 | 0.0989 | 0.2352 | 6.28 | 0.6961 |
| Pegasus | 0.2702 | 0.0703 | 0.2093 | 3.88 | 0.6432 |
In the table above shows results on 50 samples for the test set that is being compared across various models.
|