Hugging Face's logo

pszemraj
/
electra-small-discriminator-zeroshot-v1.1

Zero-Shot Classification
Transformers
PyTorch
Safetensors
English
electra
text-classification
Generated from Trainer
Inference Endpoints
Model card Files Community
electra-small-discriminator-zeroshot-v1.1 / README.md
pszemraj's picture
pszemraj
Update README.md
7c5f169 verified
preview code
|
raw
history blame
4.6 kB
metadata
license: apache-2.0
base_model: google/electra-small-discriminator
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: electra-small-discriminator-zeroshot-v1.1-none
    results: []

electra-small-discriminator-zeroshot-v1.1-none

This model is a fine-tuned version of google/electra-small-discriminator on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3747
  • F1 Macro: 0.4125
  • F1 Micro: 0.4620
  • Accuracy Balanced: 0.4701
  • Accuracy: 0.4620
  • Precision Macro: 0.5162
  • Recall Macro: 0.4701
  • Precision Micro: 0.4620
  • Recall Micro: 0.4620

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

Datasets mnli_m mnli_mm fevernli anli_r1 anli_r2 anli_r3 wanli lingnli wellformedquery rottentomatoes amazonpolarity imdb yelpreviews hatexplain massive banking77 emotiondair emocontext empathetic agnews yahootopics biasframes_sex biasframes_offensive biasframes_intent financialphrasebank appreviews hateoffensive trueteacher spam wikitoxic_toxicaggregated wikitoxic_obscene wikitoxic_identityhate wikitoxic_threat wikitoxic_insult manifesto capsotu
Accuracy 0.853 0.861 0.838 0.583 0.592 0.588 0.709 0.787 0.603 0.75 0.863 0.808 0.879 0.432 0.497 0.391 0.546 0.607 0.234 0.801 0.562 0.77 0.639 0.628 0.629 0.861 0.37 0.502 0.814 0.744 0.798 0.786 0.767 0.778 0.096 0.462
Inference text/sec (A100, batch=64) 4180.0 4161.0 2824.0 3233.0 3243.0 3239.0 4494.0 4288.0 5222.0 4396.0 2563.0 888.0 1035.0 4326.0 5447.0 5221.0 4871.0 4971.0 2852.0 3946.0 1585.0 4274.0 4097.0 4109.0 4229.0 3468.0 4476.0 1198.0 4514.0 1360.0 1267.0 1287.0 1232.0 1314.0 3936.0 4116.0

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 64
  • eval_batch_size: 64
  • seed: 80085
  • optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.04
  • num_epochs: 3

Training results

Training Loss Epoch Step Validation Loss F1 Macro F1 Micro Accuracy Balanced Accuracy Precision Macro Recall Macro Precision Micro Recall Micro
0.4765 0.32 5000 0.5300 0.7326 0.7528 0.7329 0.7528 0.7322 0.7329 0.7528 0.7528
0.4408 0.65 10000 0.5099 0.7402 0.765 0.7359 0.765 0.7463 0.7359 0.765 0.765
0.4169 0.97 15000 0.4976 0.7473 0.7702 0.7439 0.7702 0.7517 0.7439 0.7702 0.7702
0.387 1.3 20000 0.4943 0.7525 0.7742 0.7498 0.7742 0.7559 0.7498 0.7742 0.7742
0.3905 1.62 25000 0.4931 0.7522 0.775 0.7484 0.775 0.7572 0.7484 0.775 0.775
0.4001 1.95 30000 0.4924 0.7544 0.7752 0.7524 0.7752 0.7568 0.7524 0.7752 0.7752
0.3995 2.27 35000 0.4900 0.7543 0.7758 0.7517 0.7758 0.7576 0.7517 0.7758 0.7758
0.3981 2.6 40000 0.4906 0.7529 0.7742 0.7504 0.7742 0.7558 0.7504 0.7742 0.7742
0.4232 2.92 45000 0.4904 0.7544 0.776 0.7516 0.776 0.7579 0.7516 0.776 0.776

Framework versions

  • Transformers 4.33.3
  • Pytorch 2.1.2+cu121
  • Datasets 2.16.1
  • Tokenizers 0.13.3