inverse_gec_finetuned

This model is a fine-tuned version of mika5883/inverse_gec on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4594
• Bleu: 82.2574
• Gen Len: 22.4784

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	40	0.3337	84.782	22.4412
No log	2.0	80	0.2966	84.9348	22.4536
No log	3.0	120	0.2708	85.0786	22.4484
No log	4.0	160	0.2561	85.1691	22.4416
No log	5.0	200	0.2517	85.2139	22.44

Framework versions

• Transformers 4.39.3
• Pytorch 2.1.2
• Datasets 2.18.0
• Tokenizers 0.15.2