Hugging Face's logo

st-karlos-efood
/
setfit-multilabel-example-classifier-chain

Text Classification
setfit
Safetensors
sentence-transformers
xlm-roberta
generated_from_setfit_trainer
Eval Results
Model card Files Community
setfit-multilabel-example-classifier-chain / README.md
st-karlos-efood's picture
st-karlos-efood
Add SetFit model
65d55d3 verified
preview code
|
raw
history blame
23.1 kB
metadata
library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
datasets:
  - ethos
metrics:
  - accuracy
widget:
  - text: "This is what happens when you'\x80\x99re a bitch and a cunt.  Looks like that guy called it correctly.  Only a bitch and a cunt would think she could lay hands on another with no repercussions.   Maybe after that seizure she won'\x80\x99t be as much of a bitchy cunt."
  - text: >-
      What if we send every men to mars to start a new life there! They are
      useless here (Of course milk them first to store their juices)
  - text: Honestly, go fuck yourself! bitch!
  - text: Hindus take my ass please
  - text: Im going to choke you with your cross necklace idiotic religious pig
pipeline_tag: text-classification
inference: false
base_model: lighteternal/stsb-xlm-r-greek-transfer
model-index:
  - name: SetFit with lighteternal/stsb-xlm-r-greek-transfer
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: ethos
          type: ethos
          split: test
        metrics:
          - type: accuracy
            value: 0.208
            name: Accuracy

SetFit with lighteternal/stsb-xlm-r-greek-transfer

This is a SetFit model trained on the ethos dataset that can be used for Text Classification. This SetFit model uses lighteternal/stsb-xlm-r-greek-transfer as the Sentence Transformer embedding model. A ClassifierChain instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Evaluation

Metrics

Label Accuracy
all 0.208

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("st-karlos-efood/setfit-multilabel-example-classifier-chain")
# Run inference
preds = model("Hindus take my ass please")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 3 9.9307 61

Training Hyperparameters

  • batch_size: (32, 32)
  • num_epochs: (10, 10)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 10
  • body_learning_rate: (2e-05, 2e-05)
  • head_learning_rate: 2e-05
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0006 1 0.2027 -
0.0305 50 0.2092 -
0.0609 100 0.1605 -
0.0914 150 0.1726 -
0.1219 200 0.1322 -
0.1523 250 0.1252 -
0.1828 300 0.1404 -
0.2133 350 0.0927 -
0.2438 400 0.1039 -
0.2742 450 0.0904 -
0.3047 500 0.1194 -
0.3352 550 0.1024 -
0.3656 600 0.151 -
0.3961 650 0.0842 -
0.4266 700 0.1158 -
0.4570 750 0.214 -
0.4875 800 0.1167 -
0.5180 850 0.1174 -
0.5484 900 0.1567 -
0.5789 950 0.0726 -
0.6094 1000 0.0741 -
0.6399 1050 0.0841 -
0.6703 1100 0.0606 -
0.7008 1150 0.1005 -
0.7313 1200 0.1236 -
0.7617 1250 0.141 -
0.7922 1300 0.1611 -
0.8227 1350 0.1068 -
0.8531 1400 0.0542 -
0.8836 1450 0.1635 -
0.9141 1500 0.106 -
0.9445 1550 0.0817 -
0.9750 1600 0.1157 -
1.0055 1650 0.1031 -
1.0360 1700 0.0969 -
1.0664 1750 0.0742 -
1.0969 1800 0.0697 -
1.1274 1850 0.1072 -
1.1578 1900 0.0593 -
1.1883 1950 0.1102 -
1.2188 2000 0.1586 -
1.2492 2050 0.1523 -
1.2797 2100 0.0921 -
1.3102 2150 0.0634 -
1.3406 2200 0.073 -
1.3711 2250 0.1131 -
1.4016 2300 0.0493 -
1.4321 2350 0.106 -
1.4625 2400 0.0585 -
1.4930 2450 0.1058 -
1.5235 2500 0.0892 -
1.5539 2550 0.0649 -
1.5844 2600 0.0481 -
1.6149 2650 0.1359 -
1.6453 2700 0.0734 -
1.6758 2750 0.0762 -
1.7063 2800 0.1082 -
1.7367 2850 0.1274 -
1.7672 2900 0.0724 -
1.7977 2950 0.0842 -
1.8282 3000 0.1558 -
1.8586 3050 0.071 -
1.8891 3100 0.1716 -
1.9196 3150 0.1078 -
1.9500 3200 0.1037 -
1.9805 3250 0.0773 -
2.0110 3300 0.0706 -
2.0414 3350 0.1577 -
2.0719 3400 0.0825 -
2.1024 3450 0.1227 -
2.1328 3500 0.1069 -
2.1633 3550 0.1037 -
2.1938 3600 0.0595 -
2.2243 3650 0.0569 -
2.2547 3700 0.0967 -
2.2852 3750 0.0632 -
2.3157 3800 0.1014 -
2.3461 3850 0.0868 -
2.3766 3900 0.0986 -
2.4071 3950 0.0585 -
2.4375 4000 0.063 -
2.4680 4050 0.1124 -
2.4985 4100 0.0444 -
2.5289 4150 0.1547 -
2.5594 4200 0.1087 -
2.5899 4250 0.0946 -
2.6204 4300 0.0261 -
2.6508 4350 0.0414 -
2.6813 4400 0.0715 -
2.7118 4450 0.0831 -
2.7422 4500 0.0779 -
2.7727 4550 0.1049 -
2.8032 4600 0.1224 -
2.8336 4650 0.0926 -
2.8641 4700 0.0745 -
2.8946 4750 0.0642 -
2.9250 4800 0.0536 -
2.9555 4850 0.1296 -
2.9860 4900 0.0596 -
3.0165 4950 0.0361 -
3.0469 5000 0.0592 -
3.0774 5050 0.0656 -
3.1079 5100 0.0584 -
3.1383 5150 0.0729 -
3.1688 5200 0.1037 -
3.1993 5250 0.0685 -
3.2297 5300 0.0511 -
3.2602 5350 0.0427 -
3.2907 5400 0.1067 -
3.3211 5450 0.0807 -
3.3516 5500 0.0815 -
3.3821 5550 0.1016 -
3.4126 5600 0.1034 -
3.4430 5650 0.1257 -
3.4735 5700 0.0877 -
3.5040 5750 0.0808 -
3.5344 5800 0.0926 -
3.5649 5850 0.0967 -
3.5954 5900 0.0401 -
3.6258 5950 0.0547 -
3.6563 6000 0.0872 -
3.6868 6050 0.0808 -
3.7172 6100 0.1125 -
3.7477 6150 0.1431 -
3.7782 6200 0.1039 -
3.8087 6250 0.061 -
3.8391 6300 0.1022 -
3.8696 6350 0.0394 -
3.9001 6400 0.0892 -
3.9305 6450 0.0535 -
3.9610 6500 0.0793 -
3.9915 6550 0.0462 -
4.0219 6600 0.0686 -
4.0524 6650 0.0506 -
4.0829 6700 0.1012 -
4.1133 6750 0.0852 -
4.1438 6800 0.0729 -
4.1743 6850 0.1007 -
4.2048 6900 0.0431 -
4.2352 6950 0.0683 -
4.2657 7000 0.0712 -
4.2962 7050 0.0732 -
4.3266 7100 0.0374 -
4.3571 7150 0.1015 -
4.3876 7200 0.15 -
4.4180 7250 0.0852 -
4.4485 7300 0.0714 -
4.4790 7350 0.0587 -
4.5094 7400 0.1335 -
4.5399 7450 0.1123 -
4.5704 7500 0.0538 -
4.6009 7550 0.0989 -
4.6313 7600 0.0878 -
4.6618 7650 0.0963 -
4.6923 7700 0.0991 -
4.7227 7750 0.0776 -
4.7532 7800 0.0663 -
4.7837 7850 0.0696 -
4.8141 7900 0.0704 -
4.8446 7950 0.0626 -
4.8751 8000 0.0657 -
4.9055 8050 0.0567 -
4.9360 8100 0.0619 -
4.9665 8150 0.0792 -
4.9970 8200 0.0671 -
5.0274 8250 0.1068 -
5.0579 8300 0.1111 -
5.0884 8350 0.0968 -
5.1188 8400 0.0577 -
5.1493 8450 0.0934 -
5.1798 8500 0.0854 -
5.2102 8550 0.0587 -
5.2407 8600 0.048 -
5.2712 8650 0.0829 -
5.3016 8700 0.0985 -
5.3321 8750 0.107 -
5.3626 8800 0.0662 -
5.3931 8850 0.0799 -
5.4235 8900 0.0948 -
5.4540 8950 0.087 -
5.4845 9000 0.0429 -
5.5149 9050 0.0699 -
5.5454 9100 0.0911 -
5.5759 9150 0.1268 -
5.6063 9200 0.1042 -
5.6368 9250 0.0642 -
5.6673 9300 0.0736 -
5.6977 9350 0.0329 -
5.7282 9400 0.126 -
5.7587 9450 0.0991 -
5.7892 9500 0.1038 -
5.8196 9550 0.0842 -
5.8501 9600 0.0623 -
5.8806 9650 0.0642 -
5.9110 9700 0.0902 -
5.9415 9750 0.0994 -
5.9720 9800 0.0685 -
6.0024 9850 0.0573 -
6.0329 9900 0.0537 -
6.0634 9950 0.0478 -
6.0938 10000 0.0513 -
6.1243 10050 0.0529 -
6.1548 10100 0.095 -
6.1853 10150 0.0578 -
6.2157 10200 0.0918 -
6.2462 10250 0.0594 -
6.2767 10300 0.1015 -
6.3071 10350 0.036 -
6.3376 10400 0.0524 -
6.3681 10450 0.0927 -
6.3985 10500 0.0934 -
6.4290 10550 0.0788 -
6.4595 10600 0.0842 -
6.4899 10650 0.0703 -
6.5204 10700 0.0684 -
6.5509 10750 0.0759 -
6.5814 10800 0.0271 -
6.6118 10850 0.0391 -
6.6423 10900 0.0895 -
6.6728 10950 0.054 -
6.7032 11000 0.0987 -
6.7337 11050 0.0577 -
6.7642 11100 0.0822 -
6.7946 11150 0.0986 -
6.8251 11200 0.0423 -
6.8556 11250 0.0672 -
6.8860 11300 0.0747 -
6.9165 11350 0.0873 -
6.9470 11400 0.106 -
6.9775 11450 0.0975 -
7.0079 11500 0.0957 -
7.0384 11550 0.0487 -
7.0689 11600 0.0698 -
7.0993 11650 0.0317 -
7.1298 11700 0.0732 -
7.1603 11750 0.1114 -
7.1907 11800 0.0689 -
7.2212 11850 0.1211 -
7.2517 11900 0.0753 -
7.2821 11950 0.062 -
7.3126 12000 0.075 -
7.3431 12050 0.0494 -
7.3736 12100 0.0724 -
7.4040 12150 0.0605 -
7.4345 12200 0.0508 -
7.4650 12250 0.0828 -
7.4954 12300 0.0512 -
7.5259 12350 0.1291 -
7.5564 12400 0.0459 -
7.5868 12450 0.0869 -
7.6173 12500 0.0379 -
7.6478 12550 0.1878 -
7.6782 12600 0.0824 -
7.7087 12650 0.0945 -
7.7392 12700 0.0763 -
7.7697 12750 0.0602 -
7.8001 12800 0.0342 -
7.8306 12850 0.0746 -
7.8611 12900 0.065 -
7.8915 12950 0.0749 -
7.9220 13000 0.0618 -
7.9525 13050 0.0567 -
7.9829 13100 0.069 -
8.0134 13150 0.0487 -
8.0439 13200 0.0578 -
8.0743 13250 0.0876 -
8.1048 13300 0.0942 -
8.1353 13350 0.0774 -
8.1658 13400 0.0557 -
8.1962 13450 0.0872 -
8.2267 13500 0.0652 -
8.2572 13550 0.088 -
8.2876 13600 0.05 -
8.3181 13650 0.0572 -
8.3486 13700 0.053 -
8.3790 13750 0.0745 -
8.4095 13800 0.1119 -
8.4400 13850 0.0909 -
8.4704 13900 0.0374 -
8.5009 13950 0.0515 -
8.5314 14000 0.0827 -
8.5619 14050 0.0925 -
8.5923 14100 0.0793 -
8.6228 14150 0.1123 -
8.6533 14200 0.0387 -
8.6837 14250 0.0898 -
8.7142 14300 0.0627 -
8.7447 14350 0.0863 -
8.7751 14400 0.1257 -
8.8056 14450 0.0553 -
8.8361 14500 0.0664 -
8.8665 14550 0.0641 -
8.8970 14600 0.0577 -
8.9275 14650 0.0672 -
8.9580 14700 0.0776 -
8.9884 14750 0.0951 -
9.0189 14800 0.0721 -
9.0494 14850 0.0609 -
9.0798 14900 0.0821 -
9.1103 14950 0.0477 -
9.1408 15000 0.0974 -
9.1712 15050 0.0534 -
9.2017 15100 0.0673 -
9.2322 15150 0.0549 -
9.2626 15200 0.0833 -
9.2931 15250 0.0957 -
9.3236 15300 0.0601 -
9.3541 15350 0.0702 -
9.3845 15400 0.0852 -
9.4150 15450 0.0576 -
9.4455 15500 0.1006 -
9.4759 15550 0.0697 -
9.5064 15600 0.0778 -
9.5369 15650 0.0778 -
9.5673 15700 0.0844 -
9.5978 15750 0.0724 -
9.6283 15800 0.0988 -
9.6587 15850 0.0699 -
9.6892 15900 0.0772 -
9.7197 15950 0.0757 -
9.7502 16000 0.0671 -
9.7806 16050 0.1057 -
9.8111 16100 0.075 -
9.8416 16150 0.0475 -
9.8720 16200 0.0572 -
9.9025 16250 0.1176 -
9.9330 16300 0.0552 -
9.9634 16350 0.1032 -
9.9939 16400 0.0935 -

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.3
  • Sentence Transformers: 2.2.2
  • Transformers: 4.35.2
  • PyTorch: 2.1.0+cu121
  • Datasets: 2.16.1
  • Tokenizers: 0.15.0

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}