metadata

datasets:
  - samsum
language:
  - en
metrics:
  - rouge
  - bleu
library_name: transformers
pipeline_tag: summarization

t5-small-finetuned

Model Description

Purpose and Use: This model is designed for abstractive text summarization with a focus on the SAMSum Dialogue Dataset.
Model Architecture: The architecture is based on a fine-tuned T5-small model, which consists of 60 million parameters.
Training Data: Trained on the SAMSum Dialogue Dataset, which comprises approximately 15,000 dialogue-summary pairs.

Training Procedure

Preprocessing: Data preprocessing involved the removal of irrelevant tags and tokenization to ensure data consistency.
Training Details: The model was fine-tuned over 4 epochs with a learning rate of 2e-5 and a batch size of 2, utilizing gradient accumulation for optimization.
Infrastructure: Training was conducted using GPU acceleration and the Hugging Face Trainer API, with progress monitored via TensorBoard.

Evaluation Results

Metrics Used: Evaluation metrics included ROUGE-1, ROUGE-2, ROUGE-L, BLEU, and Cosine Similarity.
Performance: The fine-tuned T5-small model demonstrated superior efficiency and effectiveness in summarization tasks, outperforming its larger counterparts.

Validation and Test Set Performance

Metric	Validation Set	Test Set
ROUGE-1	0.5667	0.5536
ROUGE-2	0.2923	0.2718
ROUGE-L	0.5306	0.5210

The table above presents the performance of the model on both the validation and test sets, indicating the quality of content overlap and structural fluency in the summaries generated.

Performance Metrics Comparison Across Models

Model	ROUGE-1	ROUGE-2	ROUGE-L	BLEU	Cosine Similarity
My Model	0.3767	0.1596	0.2896	9.52	0.7698
T5 Large	0.3045	0.0960	0.2315	4.82	0.6745
Bart	0.3189	0.0989	0.2352	6.28	0.6961
Pegasus	0.2702	0.0703	0.2093	3.88	0.6432

In the table above shows results on 50 samples for the test set that is being compared across various models.