Hugging Face's logo

mann2107
/
BCMPIIRAB_MiniLM_HTTest

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Eval Results
Model card Files Community
BCMPIIRAB_MiniLM_HTTest / README.md
mann2107's picture
mann2107
Update Readme
7c1ca6e verified
preview code
|
raw
history blame contribute delete
33.3 kB
metadata
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
widget:
  - text: >-
      We are awaiting payment for the project completed in June. Please confirm
      when this will be processed.
  - text: Hello, Good morning, would you mind cancelling this rental car?
  - text: >-
      Kindly book accommodation for Lindelani Mkhize as follows: Establishment:
      City Lodge Lynwood Date checked in : 04 October 2023 Time checked in:
      19h00pm Date checked out: 06 October 2023 Time checked out: 07h00am
  - text: >-
      You've been selected for a free energy audit. Click here to schedule your
      appointment.
  - text: >-
      Please can you provide with the invoices for my stays this month as
      follows:   1. Premier Splendid Inn Bayshore (07 Aug - 08 Aug)   2. Port
      Nolloth Beach Shack (14 Aug - 17 Aug)
metrics:
  - silhouette_score
pipeline_tag: text-classification
library_name: setfit
inference: true
base_model: sentence-transformers/paraphrase-MiniLM-L6-v2
model-index:
  - name: SetFit with sentence-transformers/paraphrase-MiniLM-L6-v2
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: silhouette_score
            value: 0.6826105442176871
            name: Silhouette_Score

SetFit with sentence-transformers/paraphrase-MiniLM-L6-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-MiniLM-L6-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Evaluation

Metrics

Label Silhouette_Score
all 0.6826

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("mann2107/BCMPIIRAB_MiniLM_HTTest")
# Run inference
preds = model("Hello, Good morning, would you mind cancelling this rental car?")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 25.6577 136
Label Training Sample Count
0 24
1 24
2 24
3 24
4 24
5 24
6 24
7 24
8 24
9 24
10 24
11 24
12 24
13 24

Training Hyperparameters

  • batch_size: (8, 8)
  • num_epochs: (3, 3)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 100
  • body_learning_rate: (3e-05, 3e-05)
  • head_learning_rate: 3e-05
  • loss: MultipleNegativesRankingLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: True
  • warmup_proportion: 0.1
  • l2_weight: 0.01
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0001 1 2.5259 -
0.0060 50 2.8997 -
0.0119 100 2.8192 -
0.0179 150 2.8803 -
0.0238 200 2.635 -
0.0298 250 2.5501 -
0.0357 300 2.4468 -
0.0417 350 2.1309 -
0.0476 400 2.0439 -
0.0536 450 1.9429 -
0.0595 500 1.9344 -
0.0655 550 1.8493 -
0.0714 600 1.7907 -
0.0774 650 1.7712 -
0.0833 700 1.7349 -
0.0893 750 1.7783 -
0.0952 800 1.7022 -
0.1012 850 1.6757 -
0.1071 900 1.709 -
0.1131 950 1.6231 -
0.1190 1000 1.6647 -
0.125 1050 1.7618 -
0.1310 1100 1.652 -
0.1369 1150 1.5564 -
0.1429 1200 1.7067 -
0.1488 1250 1.664 -
0.1548 1300 1.7426 -
0.1607 1350 1.6281 -
0.1667 1400 1.6375 -
0.1726 1450 1.6216 -
0.1786 1500 1.5998 -
0.1845 1550 1.4892 -
0.1905 1600 1.556 -
0.1964 1650 1.6657 -
0.2024 1700 1.6113 -
0.2083 1750 1.634 -
0.2143 1800 1.6615 -
0.2202 1850 1.5192 -
0.2262 1900 1.5846 -
0.2321 1950 1.5376 -
0.2381 2000 1.6028 -
0.2440 2050 1.5744 -
0.25 2100 1.645 -
0.2560 2150 1.5432 -
0.2619 2200 1.5922 -
0.2679 2250 1.612 -
0.2738 2300 1.6553 -
0.2798 2350 1.5797 -
0.2857 2400 1.5249 -
0.2917 2450 1.639 -
0.2976 2500 1.7246 -
0.3036 2550 1.6186 -
0.3095 2600 1.537 -
0.3155 2650 1.5701 -
0.3214 2700 1.6095 -
0.3274 2750 1.5344 -
0.3333 2800 1.6029 -
0.3393 2850 1.6141 -
0.3452 2900 1.5655 -
0.3512 2950 1.5892 -
0.3571 3000 1.595 -
0.3631 3050 1.5068 -
0.3690 3100 1.5826 -
0.375 3150 1.481 -
0.3810 3200 1.6001 -
0.3869 3250 1.4991 -
0.3929 3300 1.605 -
0.3988 3350 1.6154 -
0.4048 3400 1.5516 -
0.4107 3450 1.559 -
0.4167 3500 1.559 -
0.4226 3550 1.5725 -
0.4286 3600 1.5719 -
0.4345 3650 1.4918 -
0.4405 3700 1.5816 -
0.4464 3750 1.5017 -
0.4524 3800 1.5093 -
0.4583 3850 1.5705 -
0.4643 3900 1.5584 -
0.4702 3950 1.5328 -
0.4762 4000 1.4932 -
0.4821 4050 1.5907 -
0.4881 4100 1.5339 -
0.4940 4150 1.4954 -
0.5 4200 1.5256 -
0.5060 4250 1.5349 -
0.5119 4300 1.5238 -
0.5179 4350 1.5222 -
0.5238 4400 1.6318 -
0.5298 4450 1.5872 -
0.5357 4500 1.4892 -
0.5417 4550 1.5764 -
0.5476 4600 1.6123 -
0.5536 4650 1.4708 -
0.5595 4700 1.5201 -
0.5655 4750 1.4975 -
0.5714 4800 1.5402 -
0.5774 4850 1.5396 -
0.5833 4900 1.5325 -
0.5893 4950 1.5166 -
0.5952 5000 1.5216 -
0.6012 5050 1.5934 -
0.6071 5100 1.5118 -
0.6131 5150 1.6581 -
0.6190 5200 1.4251 -
0.625 5250 1.5259 -
0.6310 5300 1.4854 -
0.6369 5350 1.6242 -
0.6429 5400 1.5234 -
0.6488 5450 1.4594 -
0.6548 5500 1.5513 -
0.6607 5550 1.3946 -
0.6667 5600 1.4795 -
0.6726 5650 1.5203 -
0.6786 5700 1.5137 -
0.6845 5750 1.5305 -
0.6905 5800 1.4958 -
0.6964 5850 1.5028 -
0.7024 5900 1.419 -
0.7083 5950 1.5043 -
0.7143 6000 1.4512 -
0.7202 6050 1.5199 -
0.7262 6100 1.5097 -
0.7321 6150 1.4989 -
0.7381 6200 1.4632 -
0.7440 6250 1.4781 -
0.75 6300 1.4592 -
0.7560 6350 1.507 -
0.7619 6400 1.5535 -
0.7679 6450 1.3831 -
0.7738 6500 1.572 -
0.7798 6550 1.5461 -
0.7857 6600 1.5142 -
0.7917 6650 1.494 -
0.7976 6700 1.5487 -
0.8036 6750 1.4344 -
0.8095 6800 1.5262 -
0.8155 6850 1.4942 -
0.8214 6900 1.54 -
0.8274 6950 1.518 -
0.8333 7000 1.5765 -
0.8393 7050 1.5526 -
0.8452 7100 1.5548 -
0.8512 7150 1.3953 -
0.8571 7200 1.5273 -
0.8631 7250 1.4349 -
0.8690 7300 1.4176 -
0.875 7350 1.5242 -
0.8810 7400 1.5263 -
0.8869 7450 1.5435 -
0.8929 7500 1.4882 -
0.8988 7550 1.4965 -
0.9048 7600 1.5185 -
0.9107 7650 1.5739 -
0.9167 7700 1.5821 -
0.9226 7750 1.6197 -
0.9286 7800 1.5154 -
0.9345 7850 1.5844 -
0.9405 7900 1.5242 -
0.9464 7950 1.488 -
0.9524 8000 1.5414 -
0.9583 8050 1.4829 -
0.9643 8100 1.5162 -
0.9702 8150 1.4136 -
0.9762 8200 1.36 -
0.9821 8250 1.5511 -
0.9881 8300 1.4908 -
0.9940 8350 1.5312 -
1.0 8400 1.5008 -
1.0060 8450 1.4283 -
1.0119 8500 1.5027 -
1.0179 8550 1.48 -
1.0238 8600 1.425 -
1.0298 8650 1.5233 -
1.0357 8700 1.4259 -
1.0417 8750 1.4355 -
1.0476 8800 1.5006 -
1.0536 8850 1.511 -
1.0595 8900 1.3043 -
1.0655 8950 1.5039 -
1.0714 9000 1.4909 -
1.0774 9050 1.4493 -
1.0833 9100 1.4877 -
1.0893 9150 1.5232 -
1.0952 9200 1.6282 -
1.1012 9250 1.4438 -
1.1071 9300 1.5234 -
1.1131 9350 1.5368 -
1.1190 9400 1.5029 -
1.125 9450 1.4776 -
1.1310 9500 1.4877 -
1.1369 9550 1.4917 -
1.1429 9600 1.4474 -
1.1488 9650 1.3519 -
1.1548 9700 1.5118 -
1.1607 9750 1.5507 -
1.1667 9800 1.4395 -
1.1726 9850 1.4883 -
1.1786 9900 1.4524 -
1.1845 9950 1.4756 -
1.1905 10000 1.5255 -
1.1964 10050 1.4795 -
1.2024 10100 1.5277 -
1.2083 10150 1.477 -
1.2143 10200 1.4438 -
1.2202 10250 1.5517 -
1.2262 10300 1.588 -
1.2321 10350 1.5352 -
1.2381 10400 1.3697 -
1.2440 10450 1.4449 -
1.25 10500 1.4473 -
1.2560 10550 1.5566 -
1.2619 10600 1.4502 -
1.2679 10650 1.4821 -
1.2738 10700 1.4296 -
1.2798 10750 1.4801 -
1.2857 10800 1.4542 -
1.2917 10850 1.4258 -
1.2976 10900 1.4142 -
1.3036 10950 1.6023 -
1.3095 11000 1.4291 -
1.3155 11050 1.5386 -
1.3214 11100 1.4433 -
1.3274 11150 1.4218 -
1.3333 11200 1.4345 -
1.3393 11250 1.5321 -
1.3452 11300 1.5001 -
1.3512 11350 1.3381 -
1.3571 11400 1.4819 -
1.3631 11450 1.4676 -
1.3690 11500 1.5056 -
1.375 11550 1.5052 -
1.3810 11600 1.5217 -
1.3869 11650 1.391 -
1.3929 11700 1.46 -
1.3988 11750 1.5022 -
1.4048 11800 1.4579 -
1.4107 11850 1.5025 -
1.4167 11900 1.5058 -
1.4226 11950 1.5107 -
1.4286 12000 1.5327 -
1.4345 12050 1.4727 -
1.4405 12100 1.4353 -
1.4464 12150 1.42 -
1.4524 12200 1.5349 -
1.4583 12250 1.473 -
1.4643 12300 1.5228 -
1.4702 12350 1.498 -
1.4762 12400 1.4321 -
1.4821 12450 1.5058 -
1.4881 12500 1.4601 -
1.4940 12550 1.5346 -
1.5 12600 1.5985 -
1.5060 12650 1.4683 -
1.5119 12700 1.5088 -
1.5179 12750 1.5082 -
1.5238 12800 1.5784 -
1.5298 12850 1.5241 -
1.5357 12900 1.434 -
1.5417 12950 1.452 -
1.5476 13000 1.4459 -
1.5536 13050 1.4965 -
1.5595 13100 1.5313 -
1.5655 13150 1.4781 -
1.5714 13200 1.5502 -
1.5774 13250 1.4602 -
1.5833 13300 1.4477 -
1.5893 13350 1.4736 -
1.5952 13400 1.5035 -
1.6012 13450 1.4829 -
1.6071 13500 1.4941 -
1.6131 13550 1.5462 -
1.6190 13600 1.4764 -
1.625 13650 1.4838 -
1.6310 13700 1.4264 -
1.6369 13750 1.6312 -
1.6429 13800 1.4323 -
1.6488 13850 1.514 -
1.6548 13900 1.3944 -
1.6607 13950 1.4709 -
1.6667 14000 1.4268 -
1.6726 14050 1.5699 -
1.6786 14100 1.5433 -
1.6845 14150 1.431 -
1.6905 14200 1.5421 -
1.6964 14250 1.4854 -
1.7024 14300 1.4341 -
1.7083 14350 1.4321 -
1.7143 14400 1.4284 -
1.7202 14450 1.4725 -
1.7262 14500 1.5744 -
1.7321 14550 1.4892 -
1.7381 14600 1.5357 -
1.7440 14650 1.4536 -
1.75 14700 1.4861 -
1.7560 14750 1.5268 -
1.7619 14800 1.4613 -
1.7679 14850 1.4313 -
1.7738 14900 1.4522 -
1.7798 14950 1.4291 -
1.7857 15000 1.5054 -
1.7917 15050 1.495 -
1.7976 15100 1.5352 -
1.8036 15150 1.4803 -
1.8095 15200 1.3922 -
1.8155 15250 1.4879 -
1.8214 15300 1.4752 -
1.8274 15350 1.5102 -
1.8333 15400 1.4474 -
1.8393 15450 1.4939 -
1.8452 15500 1.5216 -
1.8512 15550 1.4656 -
1.8571 15600 1.5171 -
1.8631 15650 1.3437 -
1.8690 15700 1.4875 -
1.875 15750 1.4692 -
1.8810 15800 1.4804 -
1.8869 15850 1.4423 -
1.8929 15900 1.4592 -
1.8988 15950 1.5764 -
1.9048 16000 1.4083 -
1.9107 16050 1.4852 -
1.9167 16100 1.5158 -
1.9226 16150 1.4602 -
1.9286 16200 1.4465 -
1.9345 16250 1.412 -
1.9405 16300 1.483 -
1.9464 16350 1.5342 -
1.9524 16400 1.3866 -
1.9583 16450 1.4318 -
1.9643 16500 1.6241 -
1.9702 16550 1.5514 -
1.9762 16600 1.46 -
1.9821 16650 1.4069 -
1.9881 16700 1.457 -
1.9940 16750 1.4273 -
2.0 16800 1.3673 -
2.0060 16850 1.3753 -
2.0119 16900 1.4279 -
2.0179 16950 1.3897 -
2.0238 17000 1.4659 -
2.0298 17050 1.4494 -
2.0357 17100 1.4533 -
2.0417 17150 1.3735 -
2.0476 17200 1.4232 -
2.0536 17250 1.4229 -
2.0595 17300 1.4597 -
2.0655 17350 1.4825 -
2.0714 17400 1.4661 -
2.0774 17450 1.4332 -
2.0833 17500 1.5895 -
2.0893 17550 1.4824 -
2.0952 17600 1.4472 -
2.1012 17650 1.4001 -
2.1071 17700 1.4638 -
2.1131 17750 1.4651 -
2.1190 17800 1.4711 -
2.125 17850 1.4474 -
2.1310 17900 1.4544 -
2.1369 17950 1.3935 -
2.1429 18000 1.4449 -
2.1488 18050 1.4671 -
2.1548 18100 1.4169 -
2.1607 18150 1.5095 -
2.1667 18200 1.4186 -
2.1726 18250 1.4574 -
2.1786 18300 1.4448 -
2.1845 18350 1.5045 -
2.1905 18400 1.4998 -
2.1964 18450 1.3559 -
2.2024 18500 1.4862 -
2.2083 18550 1.4018 -
2.2143 18600 1.4407 -
2.2202 18650 1.5812 -
2.2262 18700 1.4268 -
2.2321 18750 1.4434 -
2.2381 18800 1.5467 -
2.2440 18850 1.4281 -
2.25 18900 1.482 -
2.2560 18950 1.5261 -
2.2619 19000 1.4152 -
2.2679 19050 1.5267 -
2.2738 19100 1.4237 -
2.2798 19150 1.5455 -
2.2857 19200 1.4679 -
2.2917 19250 1.3398 -
2.2976 19300 1.4697 -
2.3036 19350 1.4176 -
2.3095 19400 1.4661 -
2.3155 19450 1.4397 -
2.3214 19500 1.5095 -
2.3274 19550 1.4873 -
2.3333 19600 1.4312 -
2.3393 19650 1.441 -
2.3452 19700 1.4341 -
2.3512 19750 1.4229 -
2.3571 19800 1.4917 -
2.3631 19850 1.4397 -
2.3690 19900 1.4027 -
2.375 19950 1.5022 -
2.3810 20000 1.441 -
2.3869 20050 1.4392 -
2.3929 20100 1.4454 -
2.3988 20150 1.4886 -
2.4048 20200 1.4776 -
2.4107 20250 1.3946 -
2.4167 20300 1.5492 -
2.4226 20350 1.534 -
2.4286 20400 1.4011 -
2.4345 20450 1.5276 -
2.4405 20500 1.4633 -
2.4464 20550 1.4446 -
2.4524 20600 1.5005 -
2.4583 20650 1.4818 -
2.4643 20700 1.4319 -
2.4702 20750 1.4406 -
2.4762 20800 1.4496 -
2.4821 20850 1.4963 -
2.4881 20900 1.4731 -
2.4940 20950 1.4536 -
2.5 21000 1.5153 -
2.5060 21050 1.5522 -
2.5119 21100 1.3759 -
2.5179 21150 1.4285 -
2.5238 21200 1.4162 -
2.5298 21250 1.4383 -
2.5357 21300 1.4408 -
2.5417 21350 1.4009 -
2.5476 21400 1.4589 -
2.5536 21450 1.4478 -
2.5595 21500 1.4876 -
2.5655 21550 1.4206 -
2.5714 21600 1.4927 -
2.5774 21650 1.5047 -
2.5833 21700 1.3988 -
2.5893 21750 1.4714 -
2.5952 21800 1.3605 -
2.6012 21850 1.5635 -
2.6071 21900 1.4678 -
2.6131 21950 1.4618 -
2.6190 22000 1.4407 -
2.625 22050 1.5451 -
2.6310 22100 1.4844 -
2.6369 22150 1.4088 -
2.6429 22200 1.5056 -
2.6488 22250 1.4678 -
2.6548 22300 1.4262 -
2.6607 22350 1.4492 -
2.6667 22400 1.4463 -
2.6726 22450 1.3851 -
2.6786 22500 1.513 -
2.6845 22550 1.45 -
2.6905 22600 1.4382 -
2.6964 22650 1.4637 -
2.7024 22700 1.4487 -
2.7083 22750 1.4507 -
2.7143 22800 1.5065 -
2.7202 22850 1.4116 -
2.7262 22900 1.479 -
2.7321 22950 1.444 -
2.7381 23000 1.4056 -
2.7440 23050 1.3913 -
2.75 23100 1.5108 -
2.7560 23150 1.4092 -
2.7619 23200 1.4341 -
2.7679 23250 1.4274 -
2.7738 23300 1.4748 -
2.7798 23350 1.3819 -
2.7857 23400 1.5012 -
2.7917 23450 1.3594 -
2.7976 23500 1.4708 -
2.8036 23550 1.4425 -
2.8095 23600 1.3566 -
2.8155 23650 1.456 -
2.8214 23700 1.5937 -
2.8274 23750 1.3835 -
2.8333 23800 1.4137 -
2.8393 23850 1.3861 -
2.8452 23900 1.4249 -
2.8512 23950 1.3599 -
2.8571 24000 1.4789 -
2.8631 24050 1.4527 -
2.8690 24100 1.4406 -
2.875 24150 1.4301 -
2.8810 24200 1.4059 -
2.8869 24250 1.5052 -
2.8929 24300 1.4429 -
2.8988 24350 1.5183 -
2.9048 24400 1.4288 -
2.9107 24450 1.4673 -
2.9167 24500 1.4582 -
2.9226 24550 1.4792 -
2.9286 24600 1.4598 -
2.9345 24650 1.4785 -
2.9405 24700 1.4259 -
2.9464 24750 1.4877 -
2.9524 24800 1.5162 -
2.9583 24850 1.4854 -
2.9643 24900 1.3679 -
2.9702 24950 1.3985 -
2.9762 25000 1.421 -
2.9821 25050 1.5048 -
2.9881 25100 1.4618 -
2.9940 25150 1.5061 -
3.0 25200 1.3634 -

Framework Versions

  • Python: 3.12.0
  • SetFit: 1.2.0.dev0
  • Sentence Transformers: 3.2.1
  • Transformers: 4.45.2
  • PyTorch: 2.5.0+cpu
  • Datasets: 3.0.2
  • Tokenizers: 0.20.1

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}