File size: 3,082 Bytes
6826688 e287324 9dbf5df c29e4cf d6b3384 e287324 ee1cea2 e287324 6826688 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 |
---
license: apache-2.0
library_name: peft
tags:
- generated_from_trainer
base_model: google/flan-t5-large
model-index:
- name: flan-t5-large-samsum-qlora
results: []
---
# Model description
flan-t5-large-samsum-qlora is an LLM model adapter and a fine-tuned version of [google/flan-t5-large](https://huggingface.co/google/flan-t5-large) on the
[samsum](https://huggingface.co/datasets/samsum) dataset containing dialoges. Parameter-efficient fine-tuning with QLoRA was employed to fine-tune the base model.
The model is intended for generative summarization tasks and achieved the following scores on the test dataset:
- Rougue1: 49.249596%
- Rouge2: 23.513032%
- RougeL: 39.960812%
- RougeLsum: 39.968438%
## How to use
Load the model:
``` python
from peft import PeftModel, PeftConfig
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, BitsAndBytesConfig
# Load the peft adapter model config
peft_model_id = 'MuntasirHossain/flan-t5-large-samsum-qlora'
peft_config = PeftConfig.from_pretrained(peft_model_id)
# load the base model and tokenizer
base_model = AutoModelForSeq2SeqLM.from_pretrained(peft_config.base_model_name_or_path, load_in_8bit=True, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(peft_config.base_model_name_or_path)
# Load the peft model
model = PeftModel.from_pretrained(base_model, peft_model_id, device_map="auto")
model.eval()
```
Example Inference:
``` python
# random sample text from the samsum test dataset
text = """
Emma: Hi, we're going with Peter to Amiens tomorrow.
Daniel: oh! Cool.
Emma: Wanna join?
Daniel: Sure, I'm fed up with Paris.
Emma: We're too. The noise, traffic etc. Would be nice to see some countrysides.
Daniel: I don't think Amiens is exactly countrysides though :P
Emma: Nope. Hahahah. But not a megalopolis either!
Daniel: Right! Let's do it!
Emma: But we should leave early. The days are shorter now.
Daniel: Yes, the stupid winter time.
Emma: Exactly!
Daniel: Where should we meet then?
Emma: Come to my place by 9am.
Daniel: oohhh. It means I have to get up before 7!
Emma: Yup. The early bird gets the worm (in Amiens).
Daniel: You sound like my grandmother.
Emma: HAHAHA. I'll even add: no parties tonight, no drinking dear Daniel
Daniel: I really hope Amiens is worth it!
"""
input = tokenizer(text, return_tensors="pt")
outputs = model.generate(input_ids=input["input_ids"].cuda(), max_new_tokens=40)
print("Summary: ", tokenizer.decode(outputs[0], skip_special_tokens=True))
Summary: Emma and Peter are going to Amiens tomorrow. Daniel will join them. They will meet at Emma's place by 9 am. They will not have any parties tonight.
```
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5
### Training results
### Framework versions
- PEFT 0.8.2
- Transformers 4.38.1
- Pytorch 2.1.0+cu121
- Datasets 2.17.1
- Tokenizers 0.15.2 |