|
--- |
|
language: |
|
- eng |
|
license: apache-2.0 |
|
base_model: facebook/dinov2-base |
|
tags: |
|
- multilabel-image-classification |
|
- multilabel |
|
- generated_from_trainer |
|
metrics: |
|
- accuracy |
|
model-index: |
|
- name: dino-base-2023_12_01-with_custom_small_head |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# dino-base-2023_12_01-with_custom_small_head |
|
|
|
This model is a fine-tuned version of [facebook/dinov2-base](https://huggingface.co/facebook/dinov2-base) on the multilabel_complete_dataset dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.1266 |
|
- F1 Micro: 0.8318 |
|
- F1 Macro: 0.8018 |
|
- Roc Auc: 0.8960 |
|
- Accuracy: 0.5224 |
|
- Learning Rate: 0.0000 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.01 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 90 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | F1 Micro | F1 Macro | Roc Auc | Accuracy | Rate | |
|
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:--------:|:-------:|:--------:|:------:| |
|
| 0.4702 | 1.0 | 536 | 0.4545 | 0.7549 | 0.6933 | 0.8356 | 0.4298 | 0.01 | |
|
| 0.4091 | 2.0 | 1072 | 0.3691 | 0.7832 | 0.7315 | 0.8762 | 0.4377 | 0.01 | |
|
| 0.3999 | 3.0 | 1608 | 0.4631 | 0.7725 | 0.7214 | 0.8643 | 0.4141 | 0.01 | |
|
| 0.3958 | 4.0 | 2144 | 0.4028 | 0.7858 | 0.7310 | 0.8765 | 0.4534 | 0.01 | |
|
| 0.3795 | 5.0 | 2680 | 0.5129 | 0.7452 | 0.6974 | 0.8459 | 0.4230 | 0.01 | |
|
| 0.3877 | 6.0 | 3216 | 0.5185 | 0.7386 | 0.6805 | 0.8180 | 0.4259 | 0.01 | |
|
| 0.3658 | 7.0 | 3752 | 0.4688 | 0.7710 | 0.7034 | 0.8525 | 0.4391 | 0.01 | |
|
| 0.373 | 8.0 | 4288 | 0.5070 | 0.7608 | 0.7020 | 0.8647 | 0.3859 | 0.01 | |
|
| 0.2911 | 9.0 | 4824 | 0.2327 | 0.8213 | 0.7938 | 0.8885 | 0.5088 | 0.001 | |
|
| 0.139 | 10.0 | 5360 | 0.2238 | 0.8193 | 0.7891 | 0.8972 | 0.4987 | 0.001 | |
|
| 0.1187 | 11.0 | 5896 | 0.2095 | 0.8169 | 0.7858 | 0.8833 | 0.5084 | 0.001 | |
|
| 0.1084 | 12.0 | 6432 | 0.1985 | 0.8209 | 0.7995 | 0.9031 | 0.4959 | 0.001 | |
|
| 0.1038 | 13.0 | 6968 | 0.1949 | 0.8186 | 0.7914 | 0.8941 | 0.4902 | 0.001 | |
|
| 0.0936 | 14.0 | 7504 | 0.1806 | 0.8243 | 0.7855 | 0.8947 | 0.5013 | 0.001 | |
|
| 0.0915 | 15.0 | 8040 | 0.1799 | 0.8163 | 0.7788 | 0.8806 | 0.5113 | 0.001 | |
|
| 0.0875 | 16.0 | 8576 | 0.1739 | 0.8196 | 0.7848 | 0.8894 | 0.5027 | 0.001 | |
|
| 0.0848 | 17.0 | 9112 | 0.1719 | 0.8224 | 0.7894 | 0.9007 | 0.4898 | 0.001 | |
|
| 0.0861 | 18.0 | 9648 | 0.1686 | 0.8212 | 0.7839 | 0.8904 | 0.4962 | 0.001 | |
|
| 0.0845 | 19.0 | 10184 | 0.1659 | 0.8179 | 0.7830 | 0.8934 | 0.4941 | 0.001 | |
|
| 0.0824 | 20.0 | 10720 | 0.1743 | 0.8105 | 0.7826 | 0.8839 | 0.4930 | 0.001 | |
|
| 0.0834 | 21.0 | 11256 | 0.1601 | 0.8183 | 0.7905 | 0.8959 | 0.4977 | 0.001 | |
|
| 0.0803 | 22.0 | 11792 | 0.1617 | 0.8206 | 0.7885 | 0.8985 | 0.4970 | 0.001 | |
|
| 0.0817 | 23.0 | 12328 | 0.1586 | 0.8190 | 0.7893 | 0.8900 | 0.5038 | 0.001 | |
|
| 0.0821 | 24.0 | 12864 | 0.1561 | 0.8203 | 0.7798 | 0.8825 | 0.5148 | 0.001 | |
|
| 0.0795 | 25.0 | 13400 | 0.1552 | 0.8208 | 0.7897 | 0.8981 | 0.5013 | 0.001 | |
|
| 0.0792 | 26.0 | 13936 | 0.1544 | 0.8165 | 0.7844 | 0.8853 | 0.5048 | 0.001 | |
|
| 0.0799 | 27.0 | 14472 | 0.1509 | 0.8235 | 0.7845 | 0.8889 | 0.5105 | 0.001 | |
|
| 0.0795 | 28.0 | 15008 | 0.1512 | 0.8208 | 0.7842 | 0.8866 | 0.5077 | 0.001 | |
|
| 0.079 | 29.0 | 15544 | 0.1466 | 0.8222 | 0.7879 | 0.8839 | 0.5109 | 0.001 | |
|
| 0.0803 | 30.0 | 16080 | 0.1479 | 0.8224 | 0.7874 | 0.8962 | 0.5002 | 0.001 | |
|
| 0.0787 | 31.0 | 16616 | 0.1639 | 0.8014 | 0.7579 | 0.8601 | 0.4948 | 0.001 | |
|
| 0.0807 | 32.0 | 17152 | 0.1468 | 0.8230 | 0.7924 | 0.8919 | 0.4966 | 0.001 | |
|
| 0.0776 | 33.0 | 17688 | 0.1480 | 0.8220 | 0.7917 | 0.9005 | 0.4995 | 0.001 | |
|
| 0.0802 | 34.0 | 18224 | 0.1438 | 0.8228 | 0.7907 | 0.8971 | 0.5030 | 0.001 | |
|
| 0.0797 | 35.0 | 18760 | 0.1497 | 0.8206 | 0.7854 | 0.8899 | 0.4909 | 0.001 | |
|
| 0.0781 | 36.0 | 19296 | 0.1407 | 0.8267 | 0.7947 | 0.8933 | 0.5109 | 0.001 | |
|
| 0.0791 | 37.0 | 19832 | 0.1468 | 0.8219 | 0.7714 | 0.8895 | 0.5134 | 0.001 | |
|
| 0.082 | 38.0 | 20368 | 0.1538 | 0.8105 | 0.7883 | 0.8863 | 0.4859 | 0.001 | |
|
| 0.0781 | 39.0 | 20904 | 0.1463 | 0.8209 | 0.7858 | 0.8920 | 0.5055 | 0.001 | |
|
| 0.0811 | 40.0 | 21440 | 0.1469 | 0.8151 | 0.7790 | 0.8880 | 0.4977 | 0.001 | |
|
| 0.0786 | 41.0 | 21976 | 0.1518 | 0.8167 | 0.7690 | 0.8872 | 0.5052 | 0.001 | |
|
| 0.0775 | 42.0 | 22512 | 0.1422 | 0.8260 | 0.7913 | 0.8965 | 0.5130 | 0.001 | |
|
| 0.0641 | 43.0 | 23048 | 0.1319 | 0.8340 | 0.8001 | 0.8963 | 0.5248 | 0.0001 | |
|
| 0.0633 | 44.0 | 23584 | 0.1313 | 0.8326 | 0.7959 | 0.8928 | 0.5298 | 0.0001 | |
|
| 0.0627 | 45.0 | 24120 | 0.1314 | 0.8324 | 0.7994 | 0.8955 | 0.5241 | 0.0001 | |
|
| 0.0627 | 46.0 | 24656 | 0.1308 | 0.8324 | 0.8009 | 0.8955 | 0.5234 | 0.0001 | |
|
| 0.0619 | 47.0 | 25192 | 0.1308 | 0.8333 | 0.7996 | 0.8959 | 0.5252 | 0.0001 | |
|
| 0.0626 | 48.0 | 25728 | 0.1310 | 0.8333 | 0.8009 | 0.8967 | 0.5198 | 0.0001 | |
|
| 0.063 | 49.0 | 26264 | 0.1311 | 0.8328 | 0.7989 | 0.8957 | 0.5198 | 0.0001 | |
|
| 0.0623 | 50.0 | 26800 | 0.1308 | 0.8330 | 0.7990 | 0.8962 | 0.5234 | 0.0001 | |
|
| 0.0627 | 51.0 | 27336 | 0.1309 | 0.8329 | 0.8008 | 0.8972 | 0.5220 | 0.0001 | |
|
| 0.0624 | 52.0 | 27872 | 0.1305 | 0.8309 | 0.7965 | 0.8909 | 0.5255 | 0.0001 | |
|
| 0.0626 | 53.0 | 28408 | 0.1307 | 0.8313 | 0.7992 | 0.8947 | 0.5230 | 0.0001 | |
|
| 0.0621 | 54.0 | 28944 | 0.1304 | 0.8319 | 0.7955 | 0.8964 | 0.5223 | 0.0001 | |
|
| 0.0631 | 55.0 | 29480 | 0.1299 | 0.8328 | 0.8001 | 0.8949 | 0.5248 | 0.0001 | |
|
| 0.063 | 56.0 | 30016 | 0.1302 | 0.8321 | 0.7989 | 0.8956 | 0.5223 | 0.0001 | |
|
| 0.0621 | 57.0 | 30552 | 0.1304 | 0.8290 | 0.7970 | 0.8909 | 0.5230 | 0.0001 | |
|
| 0.0623 | 58.0 | 31088 | 0.1305 | 0.8302 | 0.7978 | 0.8906 | 0.5238 | 0.0001 | |
|
| 0.0622 | 59.0 | 31624 | 0.1307 | 0.8308 | 0.7965 | 0.8915 | 0.5238 | 0.0001 | |
|
| 0.0627 | 60.0 | 32160 | 0.1294 | 0.8327 | 0.7998 | 0.8944 | 0.5302 | 0.0001 | |
|
| 0.0627 | 61.0 | 32696 | 0.1303 | 0.8319 | 0.8000 | 0.8956 | 0.5241 | 0.0001 | |
|
| 0.0626 | 62.0 | 33232 | 0.1301 | 0.8317 | 0.7982 | 0.8904 | 0.5245 | 0.0001 | |
|
| 0.0629 | 63.0 | 33768 | 0.1297 | 0.8322 | 0.7989 | 0.8949 | 0.5248 | 0.0001 | |
|
| 0.0617 | 64.0 | 34304 | 0.1300 | 0.8311 | 0.7982 | 0.8920 | 0.5245 | 0.0001 | |
|
| 0.0631 | 65.0 | 34840 | 0.1292 | 0.8319 | 0.7986 | 0.8930 | 0.5245 | 0.0001 | |
|
| 0.0619 | 66.0 | 35376 | 0.1298 | 0.8319 | 0.7982 | 0.8922 | 0.5298 | 0.0001 | |
|
| 0.0636 | 67.0 | 35912 | 0.1298 | 0.8324 | 0.7999 | 0.8980 | 0.5245 | 0.0001 | |
|
| 0.0627 | 68.0 | 36448 | 0.1298 | 0.8319 | 0.8006 | 0.8985 | 0.5195 | 0.0001 | |
|
| 0.0624 | 69.0 | 36984 | 0.1293 | 0.8309 | 0.7980 | 0.8925 | 0.5259 | 0.0001 | |
|
| 0.0625 | 70.0 | 37520 | 0.1305 | 0.8313 | 0.7967 | 0.8939 | 0.5245 | 0.0001 | |
|
| 0.0624 | 71.0 | 38056 | 0.1303 | 0.8284 | 0.7942 | 0.8901 | 0.5166 | 0.0001 | |
|
| 0.0618 | 72.0 | 38592 | 0.1288 | 0.8333 | 0.8010 | 0.8947 | 0.5266 | 1e-05 | |
|
| 0.0615 | 73.0 | 39128 | 0.1288 | 0.8324 | 0.7990 | 0.8930 | 0.5291 | 1e-05 | |
|
| 0.0602 | 74.0 | 39664 | 0.1287 | 0.8323 | 0.7989 | 0.8937 | 0.5252 | 1e-05 | |
|
| 0.0612 | 75.0 | 40200 | 0.1286 | 0.8326 | 0.8004 | 0.8946 | 0.5263 | 1e-05 | |
|
| 0.0611 | 76.0 | 40736 | 0.1286 | 0.8324 | 0.8001 | 0.8948 | 0.5259 | 1e-05 | |
|
| 0.061 | 77.0 | 41272 | 0.1287 | 0.8320 | 0.7994 | 0.8937 | 0.5280 | 1e-05 | |
|
| 0.0603 | 78.0 | 41808 | 0.1287 | 0.8323 | 0.7996 | 0.8933 | 0.5277 | 1e-05 | |
|
| 0.0616 | 79.0 | 42344 | 0.1286 | 0.8322 | 0.7994 | 0.8936 | 0.5270 | 1e-05 | |
|
| 0.061 | 80.0 | 42880 | 0.1286 | 0.8319 | 0.7987 | 0.8934 | 0.5280 | 1e-05 | |
|
| 0.0607 | 81.0 | 43416 | 0.1287 | 0.8328 | 0.8003 | 0.8938 | 0.5280 | 1e-05 | |
|
| 0.0609 | 82.0 | 43952 | 0.1288 | 0.8321 | 0.7991 | 0.8935 | 0.5288 | 1e-05 | |
|
| 0.0611 | 83.0 | 44488 | 0.1287 | 0.8324 | 0.7994 | 0.8937 | 0.5288 | 0.0000 | |
|
| 0.0611 | 84.0 | 45024 | 0.1286 | 0.8325 | 0.7993 | 0.8936 | 0.5284 | 0.0000 | |
|
| 0.0608 | 85.0 | 45560 | 0.1286 | 0.8324 | 0.7992 | 0.8935 | 0.5288 | 0.0000 | |
|
| 0.0607 | 86.0 | 46096 | 0.1286 | 0.8324 | 0.7995 | 0.8936 | 0.5284 | 0.0000 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.34.1 |
|
- Pytorch 2.1.0+cu118 |
|
- Datasets 2.14.5 |
|
- Tokenizers 0.14.1 |
|
|
