ayshi/basic_roberta

This model is a fine-tuned version of ayshi/basic_roberta on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0085
• Validation Loss: 1.0970
• Train Accuracy: 0.8267
• Epoch: 20

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 960, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Accuracy	Epoch
0.1061	0.9567	0.7778	0
0.0565	1.0825	0.7778	1
0.0362	1.0696	0.7822	2
0.0396	1.0904	0.7956	3
0.0308	1.0044	0.8044	4
0.0748	1.0578	0.8133	5
0.0392	0.9964	0.8222	6
0.0166	1.0293	0.8089	7
0.0174	0.9895	0.8178	8
0.0114	1.0403	0.8267	9
0.0141	1.0086	0.8178	10
0.0145	1.0403	0.8089	11
0.0194	1.3127	0.7822	12
0.0134	1.2929	0.7911	13
0.0377	0.8565	0.8133	14
0.0251	0.9806	0.8222	15
0.0130	1.0757	0.8356	16
0.0100	1.1304	0.8	17
0.0103	1.0859	0.8133	18
0.0078	1.1050	0.8311	19
0.0085	1.0970	0.8267	20

Framework versions

• Transformers 4.34.0
• TensorFlow 2.13.0
• Datasets 2.14.5
• Tokenizers 0.14.1