|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
**Hyperparameters:** |
|
|
|
- learning rate: 2e-5 |
|
- weight decay: 0.01 |
|
- per_device_train_batch_size: 8 |
|
- per_device_eval_batch_size: 8 |
|
- gradient_accumulation_steps:1 |
|
- eval steps: 24000 |
|
- max_length: 512 |
|
- num_epochs: 2 |
|
- hidden_dropout_prob: 0.3 |
|
- attention_probs_dropout_prob: 0.25 |
|
|
|
**Dataset version:** |
|
- taskydata/deberta-v3-base_10xp3nirstbbflanse_5xc4 |
|
|
|
**Checkpoint:** |
|
|
|
- 48000 steps |
|
|
|
**Results on Validation set:** |
|
|
|
| **Step** | **Training Loss** | **Validation Loss** | **Accuracy** | **Precision** | **Recall** | **F1** | |
|
|:--------:|:-----------------:|:-------------------:|:------------:|:-------------:|:----------:|:--------:| |
|
| 24000 | 0.052000 | 0.071572 | 0.988261 | 0.999752 | 0.987852 | 0.993767 | |
|
| 48000 | 0.015100 | 0.026952 | 0.995925 | 0.999564 | 0.996132 | 0.997846 | |
|
|
|
**Wandb logs:** |
|
- https://wandb.ai/manandey/huggingface/runs/2vh7iwi6?workspace=user-manandey |
