kasrahabib/roberta-base-finetuned-iso29148-promise-km-labels-nf-subclasses-cls

This model is a fine-tuned version of FacebookAI/roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0124
• Validation Loss: 0.1623
• Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 1770, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
2.1848	1.7407	0
1.2611	0.8464	1
0.5377	0.4928	2
0.2843	0.2790	3
0.1531	0.3657	4
0.1186	0.1997	5
0.0646	0.1742	6
0.0498	0.1731	7
0.0417	0.1709	8
0.0351	0.1689	9
0.0309	0.1659	10
0.0270	0.1674	11
0.0255	0.1687	12
0.0229	0.1665	13
0.0210	0.1673	14
0.0193	0.1677	15
0.0185	0.1664	16
0.0168	0.1658	17
0.0162	0.1649	18
0.0156	0.1670	19
0.0150	0.1678	20
0.0144	0.1656	21
0.0141	0.1653	22
0.0138	0.1662	23
0.0132	0.1668	24
0.0127	0.1629	25
0.0125	0.1614	26
0.0123	0.1619	27
0.0122	0.1624	28
0.0124	0.1623	29

Framework versions

• Transformers 4.42.3
• TensorFlow 2.15.0
• Datasets 2.19.1
• Tokenizers 0.19.1