random_sentence_french

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.4878

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: reduce_lr_on_plateau
• lr_scheduler_warmup_steps: 500
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
4.7523	0.0592	2000	4.0237
3.8958	0.1184	4000	3.8088
3.7344	0.1777	6000	3.7050
3.6379	0.2369	8000	3.6455
3.5687	0.2961	10000	3.6062
3.5165	0.3553	12000	3.5764
3.4738	0.4145	14000	3.5556
3.4389	0.4738	16000	3.5388
3.4067	0.5330	18000	3.5234
3.3778	0.5922	20000	3.5122
3.3511	0.6514	22000	3.5117
3.3309	0.7106	24000	3.5100
3.309	0.7698	26000	3.4980
3.2905	0.8291	28000	3.4919
3.2685	0.8883	30000	3.4923
3.2518	0.9475	32000	3.4878

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1
• Datasets 3.0.1
• Tokenizers 0.20.1