manjunathainti/fine_tuned_t5_summarizer

Model Card for Fine-Tuned T5 Summarizer

This model is a fine-tuned version of the T5 base model, designed for summarizing legal texts into concise short and long summaries. It enables efficient processing of complex legal cases, facilitating quick insights and detailed analysis.

Model Details

Model Description

This is the model card for the fine-tuned T5 summarizer developed for legal case summaries. It has been specifically optimized to process long legal documents and generate two types of summaries:

• Short Summaries: Concise highlights for quick review.

• Long Summaries: Detailed insights for deeper analysis.

• Developed by: Manjunatha Inti, Apurva Karne, Harshavardhan G Reddy

• Funded by: Self-funded

• Shared by: Manjunatha Inti

• Model type: Fine-tuned Transformer for Summarization

• Language(s) (NLP): English

• License: Apache 2.0

• Finetuned from model: T5-base

Model Sources

• Repository: [GitHub Repository URL to be added]
• Demo: [Colab Notebook to be added]
• Model on Hugging Face: https://huggingface.co/manjunathainti/fine_tuned_t5_summarizer

Uses

Direct Use

The model can be directly used to summarize legal case texts. It works best with English legal documents.

Downstream Use

The model can be integrated into:

• Legal document management systems.
• AI tools for legal research and compliance.

Out-of-Scope Use

• Use on non-legal documents without additional fine-tuning.
• Summarization in languages other than English.

Bias, Risks, and Limitations

Bias

The model may reflect biases present in the training data, such as jurisdictional focus or societal biases inherent in the dataset.

Risks

• Critical legal details might be omitted.
• The model's output should not replace expert legal opinions.

Recommendations

• A legal expert should always review outputs.
• Avoid using it for legal tasks where complete precision is mandatory.

Training Data

• Dataset: Multi-LexSum
• Preprocessing: Preprocessed for summarization tasks using tokenization.

Training Procedure

Preprocessing

• Tokenization and truncation were applied to the dataset.
• Input sequences were capped at 1024 tokens.
• Summaries were limited to:
  • 150 tokens for short summaries.
  • 300 tokens for long summaries.

Training Hyperparameters

• Learning Rate: 5e-5
• Batch Size: 1 (gradient accumulation steps: 8)
• Epochs: 3
• Optimizer: AdamW
• Precision: Mixed (fp16)

Speeds, Sizes, Times

• Training Time: ~4 hours
• Checkpoint Size: ~892 MB
• Hardware: NVIDIA Tesla V100

Evaluation

Testing Data, Factors & Metrics

Testing Data

• Validation was performed on the validation split of the Multi-LexSum dataset, consisting of 4,818 examples.

Metrics

• bert_score Short Summary Precision : 0.84
• bert_score Long Summary Precision : 0.81

Results

• The model produces reliable short and long summaries for legal documents, maintaining coherence and relevance.

Summary

• The fine-tuned T5 model demonstrated robust performance in summarizing legal documents, achieving competitive BERT scores.

Model Examination

Interpretability

• The model generates human-readable summaries, making it highly interpretable for end-users in the legal domain.

Environmental Impact

• Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
  • Hardware Type: NVIDIA Tesla V100
  • Hours Used: ~4 hours
  • Cloud Provider: Google Colab
  • Compute Region: US
  • Estimated Carbon Emissions: Minimal due to short training time.

Technical Specifications

Model Architecture and Objective

• The T5 architecture is designed for text-to-text tasks.
• This fine-tuned model adapts T5 for legal text summarization, leveraging the flexibility of seq2seq learning.

Compute Infrastructure

• Hardware: NVIDIA Tesla V100
• Software: Hugging Face Transformers 4.46.3, PyTorch

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "manjunathainti/fine_tuned_t5_summarizer"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

# Example Input
input_text = "Insert a legal case description here."
input_ids = tokenizer(input_text, return_tensors="pt").input_ids

# Generate Summary
summary_ids = model.generate(input_ids, max_length=150, num_beams=4, length_penalty=2.0)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
print("Generated Summary:", summary)