ko-en_mbartLarge_exp20p_batch64_linear

This model is a fine-tuned version of facebook/mbart-large-50-many-to-many-mmt on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2042
• Bleu: 27.8215
• Gen Len: 18.6731

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• total_eval_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2000
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
1.0585	1.86	4000	1.1435	27.494	18.9136
0.6719	3.71	8000	1.2042	27.8203	18.6815
0.3964	5.57	12000	1.4261	27.1711	18.5184
0.2282	7.43	16000	1.7275	26.1891	18.5479
0.1338	9.28	20000	1.9251	26.1533	18.505
0.1033	11.14	24000	2.0758	26.0643	18.4275

Framework versions

• Transformers 4.34.0
• Pytorch 2.1.0+cu121
• Datasets 2.14.5
• Tokenizers 0.14.1