bert2bert-extabs-canonicalcleandata-lr-5e-05-batchsize-4-encmaxlen-512-decmaxlen-256
10 Epoch Extractive Training + 10 Epoch Abtractive Training
- Dev Set: Canonical Clean Data & Extreme Clean Data
- Encoder max length (input): 512
- Decoder max length (output): 256
This model was trained from scratch on the id_liputan6 dataset. It achieves the following results on the evaluation set:
- Loss: 3.0021
- R1 Precision: 0.3553
- R1 Recall: 0.2599
- R1 Fmeasure: 0.2974
- R2 Precision: 0.1458
- R2 Recall: 0.1039
- R2 Fmeasure: 0.12
- Rl Precision: 0.2925
- Rl Recall: 0.2139
- Rl Fmeasure: 0.2448
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 18
- eval_batch_size: 18
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | R1 Precision | R1 Recall | R1 Fmeasure | R2 Precision | R2 Recall | R2 Fmeasure | Rl Precision | Rl Recall | Rl Fmeasure |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1.7393 | 1.0 | 10772 | 2.6782 | 0.3432 | 0.2497 | 0.2864 | 0.1375 | 0.0975 | 0.113 | 0.2828 | 0.206 | 0.2361 |
1.4091 | 2.0 | 21544 | 2.6063 | 0.3486 | 0.2534 | 0.2907 | 0.142 | 0.1004 | 0.1164 | 0.2878 | 0.2094 | 0.2401 |
1.246 | 3.0 | 32316 | 2.6079 | 0.3535 | 0.2578 | 0.2955 | 0.1457 | 0.1036 | 0.1199 | 0.2917 | 0.2131 | 0.244 |
1.1175 | 4.0 | 43088 | 2.6382 | 0.3579 | 0.2618 | 0.2996 | 0.1488 | 0.106 | 0.1225 | 0.2956 | 0.2163 | 0.2475 |
1.0102 | 5.0 | 53860 | 2.6818 | 0.3574 | 0.2609 | 0.2987 | 0.1478 | 0.1052 | 0.1217 | 0.2949 | 0.2154 | 0.2466 |
0.9141 | 6.0 | 64632 | 2.7428 | 0.3571 | 0.2616 | 0.2992 | 0.148 | 0.1056 | 0.122 | 0.2938 | 0.2152 | 0.2461 |
0.8261 | 7.0 | 75404 | 2.8255 | 0.3534 | 0.2582 | 0.2956 | 0.1457 | 0.1039 | 0.12 | 0.2906 | 0.2126 | 0.2432 |
0.7509 | 8.0 | 86176 | 2.8975 | 0.3517 | 0.2572 | 0.2943 | 0.1428 | 0.1016 | 0.1175 | 0.289 | 0.2113 | 0.2418 |
0.6822 | 9.0 | 96948 | 2.9586 | 0.3557 | 0.2599 | 0.2975 | 0.1466 | 0.1043 | 0.1206 | 0.2936 | 0.2145 | 0.2455 |
0.6289 | 10.0 | 107720 | 3.0021 | 0.3553 | 0.2599 | 0.2974 | 0.1458 | 0.1039 | 0.12 | 0.2925 | 0.2139 | 0.2448 |
Framework versions
- Transformers 4.39.3
- Pytorch 2.2.1
- Datasets 2.18.0
- Tokenizers 0.15.2
- Downloads last month
- 2
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.