README.md

---
library_name: transformers
datasets:
- FiscalNote/billsum
language:
- en
pipeline_tag: summarization
---

# Model Card for Model ID

This model is a fine-tuned version of the T5-small model, enhanced with a LoRA (Low-Rank Adaptation) adapter. It has been specifically fine-tuned to summarize legal documents, focusing on California state bills.

## Model Details
Base Model: T5-small
Task: Legal Document Summarization (California State Bills)
LoRA Configuration:
r: 8
lora_alpha: 32
lora_dropout: 0.1
Dataset: "billsum", split="ca_test"
### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** Fatemeh Dalilian
- **Finetuned from model [optional]:** T5-small

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]


## How to Get Started with the Model

Use the code below to get started with the model.

```python
from transformers import T5Tokenizer, T5ForConditionalGeneration

# Load the model and tokenizer
tokenizer = T5Tokenizer.from_pretrained("Fafadalilian/lora-adapter-t5_small_model_California_state_bill")
model = T5ForConditionalGeneration.from_pretrained("Fafadalilian/lora-adapter-t5_small_model_California_state_bill")

# Example input text
input_text = "summarize: [Insert California state bill text here]"

# Tokenize the input
inputs = tokenizer(input_text, return_tensors="pt", truncation=True, padding="max_length", max_length=512)

# Generate summary
summary_ids = model.generate(inputs.input_ids, max_length=150, num_beams=2, length_penalty=2.0, early_stopping=True)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)

print("Summary:", summary)