metadata
base_model: intfloat/e5-base-v2
library_name: setfit
metrics:
- accuracy
pipeline_tag: text-classification
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
widget:
- text: >-
Good afternoon! Certified psychologist with 20 years of experience. My
specialization: *Relations *Crises * Emotional well-being * Parent-child
relationships * Professional development * Work with dependencies
*Self-esteem * Individual requests With my approach, every client is
unique and I will help you reach your potential. The first consultation is
acquaintance and recommendations. Online on WhatsApp, Telegram, Skype
platforms. Recording exclusively in private messages. Take care of
yourself!
- text: >-
Good evening! Please give me the contacts of a trusted refrigerator
master.
- text: >-
People Needed for Remote Employment Schedule - Free from 850$ Weekly with
us training from you only Desire! From anywhere in the world The number of
places is limited! For details, write to me + in Personal
- text: Guys, someone sent parcels with this carrier from Odessa to Plovdiv
- text: Tell me a site in Bulgaria for the sale of animals
inference: true
SetFit with intfloat/e5-base-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses intfloat/e5-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: intfloat/e5-base-v2
- Classification head: a LogisticRegression instance
- Maximum Sequence Length: 512 tokens
- Number of Classes: 20 classes
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Model Labels
Label | Examples |
---|---|
CVA PROGRAMS |
|
EDUCATION |
|
LEGAL |
|
FOOD |
|
TRANSLATION/LANGUAGE |
|
NFI |
|
HEALTH |
|
ANOMALY |
|
OTHER PROGRAMS/NGOS |
|
CONNECTIVITY |
|
CHILDREN |
|
PARCEL |
|
PETS |
|
TRANSPORT/MOVEMENT |
|
SHELTER |
|
WORK/JOBS |
|
PSS & RFL |
|
MONEY/BANKING |
|
CAR |
|
GOODS/SERVICES |
|
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("rodekruis/sml-ukr-message-classifier-2")
# Run inference
preds = model("Tell me a site in Bulgaria for the sale of animals")
Training Details
Training Set Metrics
Training set | Min | Median | Max |
---|---|---|---|
Word count | 3 | 39.524 | 722 |
Label | Training Sample Count |
---|---|
ANOMALY | 60 |
CAR | 47 |
CHILDREN | 47 |
CONNECTIVITY | 41 |
CVA PROGRAMS | 46 |
EDUCATION | 44 |
FOOD | 57 |
GOODS/SERVICES | 51 |
HEALTH | 56 |
LEGAL | 44 |
MONEY/BANKING | 55 |
NFI | 57 |
OTHER PROGRAMS/NGOS | 55 |
PARCEL | 45 |
PETS | 48 |
PSS & RFL | 51 |
SHELTER | 47 |
TRANSLATION/LANGUAGE | 56 |
TRANSPORT/MOVEMENT | 48 |
WORK/JOBS | 45 |
Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (4, 4)
- max_steps: -1
- sampling_strategy: undersampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- l2_weight: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True
Training Results
Epoch | Step | Training Loss | Validation Loss |
---|---|---|---|
0.0003 | 1 | 0.1551 | - |
0.0155 | 50 | 0.3308 | - |
0.0310 | 100 | 0.3002 | - |
0.0465 | 150 | 0.264 | - |
0.0620 | 200 | 0.2227 | - |
0.0775 | 250 | 0.1935 | - |
0.0931 | 300 | 0.1727 | - |
0.1086 | 350 | 0.1437 | - |
0.1241 | 400 | 0.136 | - |
0.1396 | 450 | 0.1204 | - |
0.1551 | 500 | 0.0955 | - |
0.1706 | 550 | 0.0898 | - |
0.1861 | 600 | 0.0815 | - |
0.2016 | 650 | 0.0662 | - |
0.2171 | 700 | 0.0606 | - |
0.2326 | 750 | 0.0446 | - |
0.2481 | 800 | 0.0417 | - |
0.2636 | 850 | 0.0384 | - |
0.2792 | 900 | 0.0327 | - |
0.2947 | 950 | 0.0322 | - |
0.3102 | 1000 | 0.028 | - |
0.3257 | 1050 | 0.0256 | - |
0.3412 | 1100 | 0.0165 | - |
0.3567 | 1150 | 0.0163 | - |
0.3722 | 1200 | 0.0203 | - |
0.3877 | 1250 | 0.0148 | - |
0.4032 | 1300 | 0.0137 | - |
0.4187 | 1350 | 0.0131 | - |
0.4342 | 1400 | 0.0091 | - |
0.4498 | 1450 | 0.0094 | - |
0.4653 | 1500 | 0.0083 | - |
0.4808 | 1550 | 0.0033 | - |
0.4963 | 1600 | 0.0026 | - |
0.5118 | 1650 | 0.0054 | - |
0.5273 | 1700 | 0.0054 | - |
0.5428 | 1750 | 0.0063 | - |
0.5583 | 1800 | 0.0038 | - |
0.5738 | 1850 | 0.0059 | - |
0.5893 | 1900 | 0.0051 | - |
0.6048 | 1950 | 0.0024 | - |
0.6203 | 2000 | 0.0022 | - |
0.6359 | 2050 | 0.002 | - |
0.6514 | 2100 | 0.0069 | - |
0.6669 | 2150 | 0.0025 | - |
0.6824 | 2200 | 0.0052 | - |
0.6979 | 2250 | 0.0033 | - |
0.7134 | 2300 | 0.0048 | - |
0.7289 | 2350 | 0.003 | - |
0.7444 | 2400 | 0.0064 | - |
0.7599 | 2450 | 0.0052 | - |
0.7754 | 2500 | 0.0096 | - |
0.7909 | 2550 | 0.0041 | - |
0.8065 | 2600 | 0.0033 | - |
0.8220 | 2650 | 0.0009 | - |
0.8375 | 2700 | 0.0048 | - |
0.8530 | 2750 | 0.0058 | - |
0.8685 | 2800 | 0.0032 | - |
0.8840 | 2850 | 0.0031 | - |
0.8995 | 2900 | 0.0022 | - |
0.9150 | 2950 | 0.0015 | - |
0.9305 | 3000 | 0.0018 | - |
0.9460 | 3050 | 0.0025 | - |
0.9615 | 3100 | 0.0012 | - |
0.9770 | 3150 | 0.0012 | - |
0.9926 | 3200 | 0.0035 | - |
1.0 | 3224 | - | 0.1048 |
1.0081 | 3250 | 0.0024 | - |
1.0236 | 3300 | 0.0014 | - |
1.0391 | 3350 | 0.0012 | - |
1.0546 | 3400 | 0.0006 | - |
1.0701 | 3450 | 0.0014 | - |
1.0856 | 3500 | 0.0018 | - |
1.1011 | 3550 | 0.0012 | - |
1.1166 | 3600 | 0.0017 | - |
1.1321 | 3650 | 0.001 | - |
1.1476 | 3700 | 0.0008 | - |
1.1632 | 3750 | 0.0017 | - |
1.1787 | 3800 | 0.0004 | - |
1.1942 | 3850 | 0.0016 | - |
1.2097 | 3900 | 0.0015 | - |
1.2252 | 3950 | 0.0016 | - |
1.2407 | 4000 | 0.0004 | - |
1.2562 | 4050 | 0.001 | - |
1.2717 | 4100 | 0.0004 | - |
1.2872 | 4150 | 0.0003 | - |
1.3027 | 4200 | 0.0018 | - |
1.3182 | 4250 | 0.0004 | - |
1.3337 | 4300 | 0.0029 | - |
1.3493 | 4350 | 0.0005 | - |
1.3648 | 4400 | 0.0003 | - |
1.3803 | 4450 | 0.0009 | - |
1.3958 | 4500 | 0.0003 | - |
1.4113 | 4550 | 0.0002 | - |
1.4268 | 4600 | 0.0014 | - |
1.4423 | 4650 | 0.0005 | - |
1.4578 | 4700 | 0.0006 | - |
1.4733 | 4750 | 0.0011 | - |
1.4888 | 4800 | 0.0006 | - |
1.5043 | 4850 | 0.0004 | - |
1.5199 | 4900 | 0.0004 | - |
1.5354 | 4950 | 0.0003 | - |
1.5509 | 5000 | 0.0005 | - |
1.5664 | 5050 | 0.0006 | - |
1.5819 | 5100 | 0.0005 | - |
1.5974 | 5150 | 0.0006 | - |
1.6129 | 5200 | 0.0006 | - |
1.6284 | 5250 | 0.0003 | - |
1.6439 | 5300 | 0.0007 | - |
1.6594 | 5350 | 0.0004 | - |
1.6749 | 5400 | 0.0002 | - |
1.6904 | 5450 | 0.0002 | - |
1.7060 | 5500 | 0.0001 | - |
1.7215 | 5550 | 0.0004 | - |
1.7370 | 5600 | 0.0003 | - |
1.7525 | 5650 | 0.0001 | - |
1.7680 | 5700 | 0.0002 | - |
1.7835 | 5750 | 0.0004 | - |
1.7990 | 5800 | 0.0003 | - |
1.8145 | 5850 | 0.0003 | - |
1.8300 | 5900 | 0.0007 | - |
1.8455 | 5950 | 0.0002 | - |
1.8610 | 6000 | 0.0002 | - |
1.8766 | 6050 | 0.0004 | - |
1.8921 | 6100 | 0.0006 | - |
1.9076 | 6150 | 0.0003 | - |
1.9231 | 6200 | 0.0001 | - |
1.9386 | 6250 | 0.0002 | - |
1.9541 | 6300 | 0.0003 | - |
1.9696 | 6350 | 0.0008 | - |
1.9851 | 6400 | 0.0002 | - |
2.0 | 6448 | - | 0.1024 |
2.0006 | 6450 | 0.0008 | - |
2.0161 | 6500 | 0.0006 | - |
2.0316 | 6550 | 0.0005 | - |
2.0471 | 6600 | 0.0001 | - |
2.0627 | 6650 | 0.0009 | - |
2.0782 | 6700 | 0.0001 | - |
2.0937 | 6750 | 0.0011 | - |
2.1092 | 6800 | 0.0001 | - |
2.1247 | 6850 | 0.0002 | - |
2.1402 | 6900 | 0.0007 | - |
2.1557 | 6950 | 0.0003 | - |
2.1712 | 7000 | 0.0011 | - |
2.1867 | 7050 | 0.0001 | - |
2.2022 | 7100 | 0.0009 | - |
2.2177 | 7150 | 0.0002 | - |
2.2333 | 7200 | 0.0012 | - |
2.2488 | 7250 | 0.0002 | - |
2.2643 | 7300 | 0.0001 | - |
2.2798 | 7350 | 0.0001 | - |
2.2953 | 7400 | 0.0007 | - |
2.3108 | 7450 | 0.0002 | - |
2.3263 | 7500 | 0.0001 | - |
2.3418 | 7550 | 0.0001 | - |
2.3573 | 7600 | 0.0005 | - |
2.3728 | 7650 | 0.0002 | - |
2.3883 | 7700 | 0.0007 | - |
2.4038 | 7750 | 0.0004 | - |
2.4194 | 7800 | 0.0004 | - |
2.4349 | 7850 | 0.0004 | - |
2.4504 | 7900 | 0.0001 | - |
2.4659 | 7950 | 0.0003 | - |
2.4814 | 8000 | 0.0004 | - |
2.4969 | 8050 | 0.0002 | - |
2.5124 | 8100 | 0.0004 | - |
2.5279 | 8150 | 0.0005 | - |
2.5434 | 8200 | 0.0003 | - |
2.5589 | 8250 | 0.0004 | - |
2.5744 | 8300 | 0.0003 | - |
2.5900 | 8350 | 0.0001 | - |
2.6055 | 8400 | 0.0002 | - |
2.6210 | 8450 | 0.0001 | - |
2.6365 | 8500 | 0.0005 | - |
2.6520 | 8550 | 0.0002 | - |
2.6675 | 8600 | 0.0001 | - |
2.6830 | 8650 | 0.0003 | - |
2.6985 | 8700 | 0.0003 | - |
2.7140 | 8750 | 0.0002 | - |
2.7295 | 8800 | 0.0004 | - |
2.7450 | 8850 | 0.0001 | - |
2.7605 | 8900 | 0.0002 | - |
2.7761 | 8950 | 0.0001 | - |
2.7916 | 9000 | 0.0004 | - |
2.8071 | 9050 | 0.0002 | - |
2.8226 | 9100 | 0.0004 | - |
2.8381 | 9150 | 0.0012 | - |
2.8536 | 9200 | 0.0001 | - |
2.8691 | 9250 | 0.0013 | - |
2.8846 | 9300 | 0.0003 | - |
2.9001 | 9350 | 0.0004 | - |
2.9156 | 9400 | 0.0013 | - |
2.9311 | 9450 | 0.0002 | - |
2.9467 | 9500 | 0.0001 | - |
2.9622 | 9550 | 0.0001 | - |
2.9777 | 9600 | 0.0005 | - |
2.9932 | 9650 | 0.0001 | - |
3.0 | 9672 | - | 0.1077 |
3.0087 | 9700 | 0.0004 | - |
3.0242 | 9750 | 0.0001 | - |
3.0397 | 9800 | 0.0001 | - |
3.0552 | 9850 | 0.0001 | - |
3.0707 | 9900 | 0.0011 | - |
3.0862 | 9950 | 0.0003 | - |
3.1017 | 10000 | 0.0002 | - |
3.1172 | 10050 | 0.0003 | - |
3.1328 | 10100 | 0.0003 | - |
3.1483 | 10150 | 0.0001 | - |
3.1638 | 10200 | 0.0002 | - |
3.1793 | 10250 | 0.0002 | - |
3.1948 | 10300 | 0.0002 | - |
3.2103 | 10350 | 0.0001 | - |
3.2258 | 10400 | 0.0004 | - |
3.2413 | 10450 | 0.0004 | - |
3.2568 | 10500 | 0.0001 | - |
3.2723 | 10550 | 0.0001 | - |
3.2878 | 10600 | 0.0003 | - |
3.3033 | 10650 | 0.0002 | - |
3.3189 | 10700 | 0.0007 | - |
3.3344 | 10750 | 0.0006 | - |
3.3499 | 10800 | 0.0003 | - |
3.3654 | 10850 | 0.0002 | - |
3.3809 | 10900 | 0.0004 | - |
3.3964 | 10950 | 0.0005 | - |
3.4119 | 11000 | 0.0009 | - |
3.4274 | 11050 | 0.0002 | - |
3.4429 | 11100 | 0.0001 | - |
3.4584 | 11150 | 0.0003 | - |
3.4739 | 11200 | 0.0001 | - |
3.4895 | 11250 | 0.0002 | - |
3.5050 | 11300 | 0.0002 | - |
3.5205 | 11350 | 0.0002 | - |
3.5360 | 11400 | 0.0002 | - |
3.5515 | 11450 | 0.0002 | - |
3.5670 | 11500 | 0.0008 | - |
3.5825 | 11550 | 0.0006 | - |
3.5980 | 11600 | 0.0001 | - |
3.6135 | 11650 | 0.0003 | - |
3.6290 | 11700 | 0.0003 | - |
3.6445 | 11750 | 0.0005 | - |
3.6600 | 11800 | 0.0002 | - |
3.6756 | 11850 | 0.0003 | - |
3.6911 | 11900 | 0.0002 | - |
3.7066 | 11950 | 0.0 | - |
3.7221 | 12000 | 0.0003 | - |
3.7376 | 12050 | 0.0003 | - |
3.7531 | 12100 | 0.0008 | - |
3.7686 | 12150 | 0.0002 | - |
3.7841 | 12200 | 0.0002 | - |
3.7996 | 12250 | 0.0004 | - |
3.8151 | 12300 | 0.0002 | - |
3.8306 | 12350 | 0.0003 | - |
3.8462 | 12400 | 0.0002 | - |
3.8617 | 12450 | 0.0004 | - |
3.8772 | 12500 | 0.0003 | - |
3.8927 | 12550 | 0.0001 | - |
3.9082 | 12600 | 0.0001 | - |
3.9237 | 12650 | 0.0002 | - |
3.9392 | 12700 | 0.0001 | - |
3.9547 | 12750 | 0.0002 | - |
3.9702 | 12800 | 0.0001 | - |
3.9857 | 12850 | 0.0007 | - |
4.0 | 12896 | - | 0.1088 |
Framework Versions
- Python: 3.10.14
- SetFit: 1.1.0
- Sentence Transformers: 3.1.1
- Transformers: 4.45.1
- PyTorch: 2.4.1+cu121
- Datasets: 3.0.1
- Tokenizers: 0.20.0
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}