YetAnotherTimelistGPT

This model is a fine-tuned version of ai-forever/rugpt3small_based_on_gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.5511

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 6e-06
• train_batch_size: 3
• eval_batch_size: 3
• seed: 42
• gradient_accumulation_steps: 10
• total_train_batch_size: 30
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.81	3	3.7528
No log	1.89	7	3.7051
No log	2.97	11	3.6694
No log	3.78	14	3.6498
No log	4.86	18	3.6306
No log	5.95	22	3.6157
No log	6.76	25	3.6059
No log	7.84	29	3.5949
No log	8.92	33	3.5857
No log	10.0	37	3.5773
No log	10.81	40	3.5714
No log	11.89	44	3.5643
No log	12.97	48	3.5583
No log	13.78	51	3.5550
No log	14.86	55	3.5524
No log	15.95	59	3.5512
No log	16.22	60	3.5511

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.13.3