fc-binary-model

This model is a fine-tuned version of line-corporation/line-distilbert-base-japanese on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3017
• Accuracy: 0.8730

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 8
• seed: 42
• distributed_type: tpu
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	306	0.3749	0.8594
0.4009	2.0	612	0.3578	0.8594
0.4009	3.0	918	0.3448	0.8613
0.3811	4.0	1224	0.3416	0.8613
0.3694	5.0	1530	0.3344	0.8613
0.3694	6.0	1836	0.3284	0.8652
0.3623	7.0	2142	0.3274	0.8633
0.3623	8.0	2448	0.3236	0.8652
0.3566	9.0	2754	0.3216	0.8633
0.349	10.0	3060	0.3174	0.8691
0.349	11.0	3366	0.3145	0.875
0.3512	12.0	3672	0.3135	0.8711
0.3512	13.0	3978	0.3102	0.8711
0.3455	14.0	4284	0.3119	0.8770
0.3427	15.0	4590	0.3100	0.875
0.3427	16.0	4896	0.3074	0.8711
0.3366	17.0	5202	0.3055	0.875
0.3385	18.0	5508	0.3068	0.875
0.3385	19.0	5814	0.3065	0.875
0.3342	20.0	6120	0.3047	0.875
0.3342	21.0	6426	0.3046	0.8770
0.3313	22.0	6732	0.3049	0.875
0.3349	23.0	7038	0.3035	0.875
0.3349	24.0	7344	0.3029	0.8730
0.3309	25.0	7650	0.3031	0.8730
0.3309	26.0	7956	0.3026	0.8711
0.326	27.0	8262	0.3026	0.875
0.3271	28.0	8568	0.3019	0.8730
0.3271	29.0	8874	0.3018	0.8730
0.3308	30.0	9180	0.3017	0.8730

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.0+cu118
• Datasets 2.14.5
• Tokenizers 0.14.0