End of training

c36f328 verified 10 months ago

6.2 kB

	---
	license: apache-2.0
	library_name: peft
	tags:
	- generated_from_trainer
	metrics:
	- precision
	- recall
	- f1
	- accuracy
	base_model: distilbert-base-uncased
	model-index:
	- name: distilbert-base-uncased-tokenclassification_lora
	results: []
	---

	<!-- This model card has been generated automatically according to the information the Trainer had access to. You
	should probably proofread and complete it, then remove this comment. -->

	# distilbert-base-uncased-tokenclassification_lora

	This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
	It achieves the following results on the evaluation set:
	- Loss: 0.2286
	- Precision: 0.6655
	- Recall: 0.4474
	- F1: 0.5351
	- Accuracy: 0.9493

	## Model description

	More information needed

	## Intended uses & limitations

	More information needed

	## Training and evaluation data

	More information needed

	## Training procedure

	### Training hyperparameters

	The following hyperparameters were used during training:
	- learning_rate: 2e-05
	- train_batch_size: 16
	- eval_batch_size: 16
	- seed: 42
	- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
	- lr_scheduler_type: linear
	- num_epochs: 50

	### Training results

	\| Training Loss \| Epoch \| Step \| Validation Loss \| Precision \| Recall \| F1 \| Accuracy \|
	\|:-------------:\|:-----:\|:-----:\|:---------------:\|:---------:\|:------:\|:------:\|:--------:\|
	\| No log \| 1.0 \| 213 \| 0.4882 \| 0.0 \| 0.0 \| 0.0 \| 0.9205 \|
	\| No log \| 2.0 \| 426 \| 0.4615 \| 0.0 \| 0.0 \| 0.0 \| 0.9205 \|
	\| 0.9185 \| 3.0 \| 639 \| 0.4220 \| 0.0 \| 0.0 \| 0.0 \| 0.9205 \|
	\| 0.9185 \| 4.0 \| 852 \| 0.3565 \| 0.0 \| 0.0 \| 0.0 \| 0.9205 \|
	\| 0.25 \| 5.0 \| 1065 \| 0.3219 \| 0.25 \| 0.0012 \| 0.0024 \| 0.9207 \|
	\| 0.25 \| 6.0 \| 1278 \| 0.3121 \| 0.4737 \| 0.0323 \| 0.0605 \| 0.9231 \|
	\| 0.25 \| 7.0 \| 1491 \| 0.3071 \| 0.4783 \| 0.0658 \| 0.1157 \| 0.9256 \|
	\| 0.1979 \| 8.0 \| 1704 \| 0.3015 \| 0.4695 \| 0.1196 \| 0.1907 \| 0.9290 \|
	\| 0.1979 \| 9.0 \| 1917 \| 0.2841 \| 0.4871 \| 0.2033 \| 0.2869 \| 0.9342 \|
	\| 0.1775 \| 10.0 \| 2130 \| 0.2823 \| 0.4932 \| 0.2177 \| 0.3021 \| 0.9349 \|
	\| 0.1775 \| 11.0 \| 2343 \| 0.2729 \| 0.5090 \| 0.2703 \| 0.3531 \| 0.9374 \|
	\| 0.1691 \| 12.0 \| 2556 \| 0.2731 \| 0.5273 \| 0.2775 \| 0.3636 \| 0.9382 \|
	\| 0.1691 \| 13.0 \| 2769 \| 0.2644 \| 0.5660 \| 0.3182 \| 0.4074 \| 0.9402 \|
	\| 0.1691 \| 14.0 \| 2982 \| 0.2648 \| 0.6107 \| 0.3134 \| 0.4142 \| 0.9402 \|
	\| 0.1546 \| 15.0 \| 3195 \| 0.2611 \| 0.6388 \| 0.3469 \| 0.4496 \| 0.9419 \|
	\| 0.1546 \| 16.0 \| 3408 \| 0.2570 \| 0.6409 \| 0.3565 \| 0.4581 \| 0.9431 \|
	\| 0.1461 \| 17.0 \| 3621 \| 0.2515 \| 0.6541 \| 0.3732 \| 0.4752 \| 0.9444 \|
	\| 0.1461 \| 18.0 \| 3834 \| 0.2461 \| 0.6415 \| 0.3959 \| 0.4896 \| 0.9456 \|
	\| 0.1382 \| 19.0 \| 4047 \| 0.2434 \| 0.6452 \| 0.4067 \| 0.4989 \| 0.9463 \|
	\| 0.1382 \| 20.0 \| 4260 \| 0.2464 \| 0.6673 \| 0.3983 \| 0.4989 \| 0.9457 \|
	\| 0.1382 \| 21.0 \| 4473 \| 0.2429 \| 0.6767 \| 0.4031 \| 0.5052 \| 0.9460 \|
	\| 0.1324 \| 22.0 \| 4686 \| 0.2411 \| 0.684 \| 0.4091 \| 0.5120 \| 0.9462 \|
	\| 0.1324 \| 23.0 \| 4899 \| 0.2336 \| 0.6654 \| 0.4306 \| 0.5229 \| 0.9475 \|
	\| 0.129 \| 24.0 \| 5112 \| 0.2411 \| 0.6737 \| 0.4175 \| 0.5155 \| 0.9469 \|
	\| 0.129 \| 25.0 \| 5325 \| 0.2385 \| 0.6901 \| 0.4234 \| 0.5248 \| 0.9473 \|
	\| 0.1235 \| 26.0 \| 5538 \| 0.2328 \| 0.6843 \| 0.4330 \| 0.5304 \| 0.9482 \|
	\| 0.1235 \| 27.0 \| 5751 \| 0.2343 \| 0.6877 \| 0.4294 \| 0.5287 \| 0.9481 \|
	\| 0.1235 \| 28.0 \| 5964 \| 0.2300 \| 0.6649 \| 0.4462 \| 0.5340 \| 0.9488 \|
	\| 0.1195 \| 29.0 \| 6177 \| 0.2323 \| 0.6790 \| 0.4378 \| 0.5324 \| 0.9483 \|
	\| 0.1195 \| 30.0 \| 6390 \| 0.2351 \| 0.6869 \| 0.4330 \| 0.5312 \| 0.9482 \|
	\| 0.1179 \| 31.0 \| 6603 \| 0.2329 \| 0.6811 \| 0.4342 \| 0.5303 \| 0.9482 \|
	\| 0.1179 \| 32.0 \| 6816 \| 0.2326 \| 0.6779 \| 0.4330 \| 0.5285 \| 0.9482 \|
	\| 0.1156 \| 33.0 \| 7029 \| 0.2326 \| 0.6807 \| 0.4258 \| 0.5239 \| 0.9481 \|
	\| 0.1156 \| 34.0 \| 7242 \| 0.2328 \| 0.6870 \| 0.4306 \| 0.5294 \| 0.9481 \|
	\| 0.1156 \| 35.0 \| 7455 \| 0.2327 \| 0.6716 \| 0.4354 \| 0.5283 \| 0.9484 \|
	\| 0.114 \| 36.0 \| 7668 \| 0.2290 \| 0.6614 \| 0.4486 \| 0.5346 \| 0.9492 \|
	\| 0.114 \| 37.0 \| 7881 \| 0.2275 \| 0.6597 \| 0.4522 \| 0.5366 \| 0.9495 \|
	\| 0.1121 \| 38.0 \| 8094 \| 0.2285 \| 0.6643 \| 0.4498 \| 0.5364 \| 0.9493 \|
	\| 0.1121 \| 39.0 \| 8307 \| 0.2275 \| 0.6626 \| 0.4533 \| 0.5384 \| 0.9495 \|
	\| 0.1113 \| 40.0 \| 8520 \| 0.2323 \| 0.6784 \| 0.4390 \| 0.5330 \| 0.9488 \|
	\| 0.1113 \| 41.0 \| 8733 \| 0.2289 \| 0.6715 \| 0.4450 \| 0.5353 \| 0.9491 \|
	\| 0.1113 \| 42.0 \| 8946 \| 0.2281 \| 0.6696 \| 0.4510 \| 0.5390 \| 0.9494 \|
	\| 0.1111 \| 43.0 \| 9159 \| 0.2284 \| 0.6625 \| 0.4486 \| 0.5350 \| 0.9493 \|
	\| 0.1111 \| 44.0 \| 9372 \| 0.2270 \| 0.6591 \| 0.4510 \| 0.5355 \| 0.9495 \|
	\| 0.1077 \| 45.0 \| 9585 \| 0.2291 \| 0.6667 \| 0.4474 \| 0.5354 \| 0.9493 \|
	\| 0.1077 \| 46.0 \| 9798 \| 0.2289 \| 0.6691 \| 0.4450 \| 0.5345 \| 0.9492 \|
	\| 0.1089 \| 47.0 \| 10011 \| 0.2272 \| 0.6591 \| 0.4510 \| 0.5355 \| 0.9495 \|
	\| 0.1089 \| 48.0 \| 10224 \| 0.2283 \| 0.6661 \| 0.4486 \| 0.5361 \| 0.9493 \|
	\| 0.1089 \| 49.0 \| 10437 \| 0.2286 \| 0.6655 \| 0.4474 \| 0.5351 \| 0.9493 \|
	\| 0.1097 \| 50.0 \| 10650 \| 0.2286 \| 0.6655 \| 0.4474 \| 0.5351 \| 0.9493 \|


	### Framework versions

	- PEFT 0.7.1
	- Transformers 4.36.2
	- Pytorch 2.0.0+cu117
	- Datasets 2.16.1
	- Tokenizers 0.15.0

	---
	license: apache-2.0
	library_name: peft
	tags:
	- generated_from_trainer
	metrics:
	- precision
	- recall
	- f1
	- accuracy
	base_model: distilbert-base-uncased
	model-index:
	- name: distilbert-base-uncased-tokenclassification_lora
	results: []
	---

	<!-- This model card has been generated automatically according to the information the Trainer had access to. You
	should probably proofread and complete it, then remove this comment. -->

	# distilbert-base-uncased-tokenclassification_lora

	This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
	It achieves the following results on the evaluation set:
	- Loss: 0.2286
	- Precision: 0.6655
	- Recall: 0.4474
	- F1: 0.5351
	- Accuracy: 0.9493

	## Model description

	More information needed

	## Intended uses & limitations

	More information needed

	## Training and evaluation data

	More information needed

	## Training procedure

	### Training hyperparameters

	The following hyperparameters were used during training:
	- learning_rate: 2e-05
	- train_batch_size: 16
	- eval_batch_size: 16
	- seed: 42
	- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
	- lr_scheduler_type: linear
	- num_epochs: 50

	### Training results

	\| Training Loss \| Epoch \| Step \| Validation Loss \| Precision \| Recall \| F1 \| Accuracy \|
	\|:-------------:\|:-----:\|:-----:\|:---------------:\|:---------:\|:------:\|:------:\|:--------:\|
	\| No log \| 1.0 \| 213 \| 0.4882 \| 0.0 \| 0.0 \| 0.0 \| 0.9205 \|
	\| No log \| 2.0 \| 426 \| 0.4615 \| 0.0 \| 0.0 \| 0.0 \| 0.9205 \|
	\| 0.9185 \| 3.0 \| 639 \| 0.4220 \| 0.0 \| 0.0 \| 0.0 \| 0.9205 \|
	\| 0.9185 \| 4.0 \| 852 \| 0.3565 \| 0.0 \| 0.0 \| 0.0 \| 0.9205 \|
	\| 0.25 \| 5.0 \| 1065 \| 0.3219 \| 0.25 \| 0.0012 \| 0.0024 \| 0.9207 \|
	\| 0.25 \| 6.0 \| 1278 \| 0.3121 \| 0.4737 \| 0.0323 \| 0.0605 \| 0.9231 \|
	\| 0.25 \| 7.0 \| 1491 \| 0.3071 \| 0.4783 \| 0.0658 \| 0.1157 \| 0.9256 \|
	\| 0.1979 \| 8.0 \| 1704 \| 0.3015 \| 0.4695 \| 0.1196 \| 0.1907 \| 0.9290 \|
	\| 0.1979 \| 9.0 \| 1917 \| 0.2841 \| 0.4871 \| 0.2033 \| 0.2869 \| 0.9342 \|
	\| 0.1775 \| 10.0 \| 2130 \| 0.2823 \| 0.4932 \| 0.2177 \| 0.3021 \| 0.9349 \|
	\| 0.1775 \| 11.0 \| 2343 \| 0.2729 \| 0.5090 \| 0.2703 \| 0.3531 \| 0.9374 \|
	\| 0.1691 \| 12.0 \| 2556 \| 0.2731 \| 0.5273 \| 0.2775 \| 0.3636 \| 0.9382 \|
	\| 0.1691 \| 13.0 \| 2769 \| 0.2644 \| 0.5660 \| 0.3182 \| 0.4074 \| 0.9402 \|
	\| 0.1691 \| 14.0 \| 2982 \| 0.2648 \| 0.6107 \| 0.3134 \| 0.4142 \| 0.9402 \|
	\| 0.1546 \| 15.0 \| 3195 \| 0.2611 \| 0.6388 \| 0.3469 \| 0.4496 \| 0.9419 \|
	\| 0.1546 \| 16.0 \| 3408 \| 0.2570 \| 0.6409 \| 0.3565 \| 0.4581 \| 0.9431 \|
	\| 0.1461 \| 17.0 \| 3621 \| 0.2515 \| 0.6541 \| 0.3732 \| 0.4752 \| 0.9444 \|
	\| 0.1461 \| 18.0 \| 3834 \| 0.2461 \| 0.6415 \| 0.3959 \| 0.4896 \| 0.9456 \|
	\| 0.1382 \| 19.0 \| 4047 \| 0.2434 \| 0.6452 \| 0.4067 \| 0.4989 \| 0.9463 \|
	\| 0.1382 \| 20.0 \| 4260 \| 0.2464 \| 0.6673 \| 0.3983 \| 0.4989 \| 0.9457 \|
	\| 0.1382 \| 21.0 \| 4473 \| 0.2429 \| 0.6767 \| 0.4031 \| 0.5052 \| 0.9460 \|
	\| 0.1324 \| 22.0 \| 4686 \| 0.2411 \| 0.684 \| 0.4091 \| 0.5120 \| 0.9462 \|
	\| 0.1324 \| 23.0 \| 4899 \| 0.2336 \| 0.6654 \| 0.4306 \| 0.5229 \| 0.9475 \|
	\| 0.129 \| 24.0 \| 5112 \| 0.2411 \| 0.6737 \| 0.4175 \| 0.5155 \| 0.9469 \|
	\| 0.129 \| 25.0 \| 5325 \| 0.2385 \| 0.6901 \| 0.4234 \| 0.5248 \| 0.9473 \|
	\| 0.1235 \| 26.0 \| 5538 \| 0.2328 \| 0.6843 \| 0.4330 \| 0.5304 \| 0.9482 \|
	\| 0.1235 \| 27.0 \| 5751 \| 0.2343 \| 0.6877 \| 0.4294 \| 0.5287 \| 0.9481 \|
	\| 0.1235 \| 28.0 \| 5964 \| 0.2300 \| 0.6649 \| 0.4462 \| 0.5340 \| 0.9488 \|
	\| 0.1195 \| 29.0 \| 6177 \| 0.2323 \| 0.6790 \| 0.4378 \| 0.5324 \| 0.9483 \|
	\| 0.1195 \| 30.0 \| 6390 \| 0.2351 \| 0.6869 \| 0.4330 \| 0.5312 \| 0.9482 \|
	\| 0.1179 \| 31.0 \| 6603 \| 0.2329 \| 0.6811 \| 0.4342 \| 0.5303 \| 0.9482 \|
	\| 0.1179 \| 32.0 \| 6816 \| 0.2326 \| 0.6779 \| 0.4330 \| 0.5285 \| 0.9482 \|
	\| 0.1156 \| 33.0 \| 7029 \| 0.2326 \| 0.6807 \| 0.4258 \| 0.5239 \| 0.9481 \|
	\| 0.1156 \| 34.0 \| 7242 \| 0.2328 \| 0.6870 \| 0.4306 \| 0.5294 \| 0.9481 \|
	\| 0.1156 \| 35.0 \| 7455 \| 0.2327 \| 0.6716 \| 0.4354 \| 0.5283 \| 0.9484 \|
	\| 0.114 \| 36.0 \| 7668 \| 0.2290 \| 0.6614 \| 0.4486 \| 0.5346 \| 0.9492 \|
	\| 0.114 \| 37.0 \| 7881 \| 0.2275 \| 0.6597 \| 0.4522 \| 0.5366 \| 0.9495 \|
	\| 0.1121 \| 38.0 \| 8094 \| 0.2285 \| 0.6643 \| 0.4498 \| 0.5364 \| 0.9493 \|
	\| 0.1121 \| 39.0 \| 8307 \| 0.2275 \| 0.6626 \| 0.4533 \| 0.5384 \| 0.9495 \|
	\| 0.1113 \| 40.0 \| 8520 \| 0.2323 \| 0.6784 \| 0.4390 \| 0.5330 \| 0.9488 \|
	\| 0.1113 \| 41.0 \| 8733 \| 0.2289 \| 0.6715 \| 0.4450 \| 0.5353 \| 0.9491 \|
	\| 0.1113 \| 42.0 \| 8946 \| 0.2281 \| 0.6696 \| 0.4510 \| 0.5390 \| 0.9494 \|
	\| 0.1111 \| 43.0 \| 9159 \| 0.2284 \| 0.6625 \| 0.4486 \| 0.5350 \| 0.9493 \|
	\| 0.1111 \| 44.0 \| 9372 \| 0.2270 \| 0.6591 \| 0.4510 \| 0.5355 \| 0.9495 \|
	\| 0.1077 \| 45.0 \| 9585 \| 0.2291 \| 0.6667 \| 0.4474 \| 0.5354 \| 0.9493 \|
	\| 0.1077 \| 46.0 \| 9798 \| 0.2289 \| 0.6691 \| 0.4450 \| 0.5345 \| 0.9492 \|
	\| 0.1089 \| 47.0 \| 10011 \| 0.2272 \| 0.6591 \| 0.4510 \| 0.5355 \| 0.9495 \|
	\| 0.1089 \| 48.0 \| 10224 \| 0.2283 \| 0.6661 \| 0.4486 \| 0.5361 \| 0.9493 \|
	\| 0.1089 \| 49.0 \| 10437 \| 0.2286 \| 0.6655 \| 0.4474 \| 0.5351 \| 0.9493 \|
	\| 0.1097 \| 50.0 \| 10650 \| 0.2286 \| 0.6655 \| 0.4474 \| 0.5351 \| 0.9493 \|


	### Framework versions

	- PEFT 0.7.1
	- Transformers 4.36.2
	- Pytorch 2.0.0+cu117
	- Datasets 2.16.1
	- Tokenizers 0.15.0