barto_prompts
This model is a fine-tuned version of vgaraujov/bart-base-spanish. It achieves the following results on the evaluation set:
- Loss: 0.5242
- Rouge1: 77.7794
- Rouge2: 62.5213
- Rougel: 77.3853
- Rougelsum: 77.2245
- Gen Len: 11.6686
Model description
This checkpoint uses BARTO as base model and different prefix to achieve different tasks in emergency transcribed calls:
- "Resume la emergencia: ": For text summarization
- "Extrae las palabras clave de la emergencia: ": For keyword extraction
- "La palabra que mejor representa la emergencia es: ": Gives a word that represents the text
- "Clasifica la emergencia en [CLAVE ROJA, CLAVE NARANJA, CLAVE AMARILLA, CLAVE VERDE]: ": For text classification
Intended uses & limitations
Under privacy agreement.
Training and evaluation data
Training data used has been provided by the ECU 911 service under a strict confidentiality agreement.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- num_devices: 3
- total_train_batch_size: 48
- total_eval_batch_size: 48
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
---|---|---|---|---|---|---|---|---|
1.3631 | 1.0 | 92 | 0.6643 | 66.5661 | 49.8557 | 66.156 | 66.0723 | 10.7803 |
0.607 | 2.0 | 184 | 0.5528 | 72.4516 | 55.3295 | 72.0424 | 71.9591 | 10.8390 |
0.4994 | 3.0 | 276 | 0.5330 | 74.2798 | 56.9793 | 73.6683 | 73.6271 | 10.9072 |
0.4215 | 4.0 | 368 | 0.5246 | 75.5697 | 58.5086 | 75.1434 | 75.0331 | 11.5663 |
0.3744 | 5.0 | 460 | 0.5302 | 75.9054 | 60.4386 | 75.4245 | 75.294 | 11.6496 |
0.3392 | 6.0 | 552 | 0.5238 | 76.8758 | 61.7901 | 76.4882 | 76.444 | 11.7254 |
0.3014 | 7.0 | 644 | 0.5302 | 76.8835 | 61.9104 | 76.4603 | 76.3661 | 11.6117 |
0.2807 | 8.0 | 736 | 0.5239 | 77.4479 | 62.0839 | 77.0472 | 76.8683 | 11.5417 |
0.265 | 9.0 | 828 | 0.5210 | 77.5274 | 62.249 | 77.1446 | 76.9984 | 11.5890 |
0.2594 | 10.0 | 920 | 0.5242 | 77.7794 | 62.5213 | 77.3853 | 77.2245 | 11.6686 |
Framework versions
- Transformers 4.36.2
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0
- Downloads last month
- 0
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for UDA-LIDI/barto_emergency_multi_purpose
Base model
vgaraujov/bart-base-spanish