metadata
license: apache-2.0
tags:
- summarization
- generated_from_trainer
metrics:
- rouge
model-index:
- name: BARTkrame-abstract
results: []
widget:
- text: >-
Nous voulons que tous les êtres humains, ensemble ou pris isolément,
jeunes ou vieux, riches ou pauvres, nobles ou roturiers, hommes ou femmes,
puissent pleinement s'instruire et devenir des êtres achevés. Nous voulons
qu'ils soient instruits parfaitement et formés non seulement sur tel ou
tel point, mais également sur tout ce qui permet à l'homme de réaliser
intégralement son essence
BARTkrame-abstract
This model is a fine-tuned version of krm/BARTkrame-abstract on the krm/for-ULPGL-Dissertation dataset. It achieves (15/10/2022) the following results on the evaluation set:
- Loss: 2.4196
- Rouge1: 0.2703
- Rouge2: 0.1334
- Rougel: 0.2392
- Rougelsum: 0.2419
Model description
This model is primarly a finetuned version of moussaKam/mbarthez.
Intended uses & limitations
More information needed
Training and evaluation data
We have used the krm/for-ULPGL-Dissertation dataset reduced to :
Training data : 5000 samples taken at random with seed=42.
Validation data : 100 samples taken at random with seed=42.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5.6e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 12
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum |
|---|---|---|---|---|---|---|---|
| 0.1316 | 9.0 | 1250 | 2.3251 | 0.2505 | 0.1158 | 0.2150 | 0.2184 |
| 0.0894 | 10.0 | 2500 | 2.3467 | 0.2526 | 0.1073 | 0.2067 | 0.2124 |
| 0.045 | 11.0 | 3750 | 2.3742 | 0.2593 | 0.1211 | 0.2281 | 0.2308 |
| 0.0242 | 12.0 | 5000 | 2.4196 | 0.2703 | 0.1334 | 0.2392 | 0.2419 |
Framework versions
- Transformers 4.23.1
- Pytorch 1.12.1+cu113
- Datasets 2.6.1
- Tokenizers 0.13.1