metadata

library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
metrics:
  - accuracy
widget:
  - text: >-
      i miss our talks our cuddling our kissing and the feelings that you can
      only share with your beloved
  - text: >-
      i feel that i m so pathetic and downright dumb to let people in let them
      toy with my feelings and then leaving me to clean up this pile of sadness
      inside me
  - text: >-
      i told her that i woke up feeling mad that i am a woman and that i am
      probably always going to have to worry about being raped
  - text: >-
      i try to share what i bake with a lot of people is because i love people
      and i want them to feel loved
  - text: i feel for you despite the bitterness and longing
pipeline_tag: text-classification
inference: true
base_model: sentence-transformers/paraphrase-mpnet-base-v2
model-index:
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.45842105263157895
            name: Accuracy

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 6 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
sadness	'i am from new jersey and this first drink was consumed at a post prom party so i feel it s appropriately lame' 'i am the one feeling punished' 'i wouldn t feel submissive which has it s place but not in the work environment'
love	'i would rather take my chances on keeping my heart and getting it broken again and again then to stop feeling to stop caring to be bitter cross cynical' 'i still love to run and plan to keep it up but i don t want to once again register for so many races that i feel like every exercise moment needs to be devoted to running' 'i suddenly feel that this is more than a sweet love song that every girls could sing in front of their boyfriends'
surprise	'i was feeling an act of god at work in my life and it was an amazing feeling' 'i tween sat for my moms boss year old and year old boys this weekend id say babysit but that feels weird considering there were n' 'i started feeling funny and then friday i woke up sick as a dog'
anger	'i could of course go on with it feeling resentful of him with him being blissfully unaware of anything being wrong' 'i feel tortured because i am not allowed to enjoy food the way my friend can' 'i feel like i should be offended but yawwwn'
joy	'i was feeling over eager and hopped on to the tube to ride the eye of london' 'i am not feeling particularly creative' 'i woke on saturday feeling a little brighter and was very keen to get outdoors after spending all day friday wallowing in self pity'
fear	'im feeling pretty shaken at the moment' 'i know he is totally trainable and can be free of his arm chewing habits i feel that the kids would be too nervous around him during the training process' 'i am feeling pretty restless right now while typing this'

Evaluation

Metrics

Label	Accuracy
all	0.4584

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("dendimaki/apeiron-v4")
# Run inference
preds = model("i feel for you despite the bitterness and longing")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	4	17.6458	55

Label	Training Sample Count
sadness	8
joy	8
love	8
anger	8
fear	8
surprise	8

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (4, 4)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0083	1	0.2802	-
0.4167	50	0.1302	-
0.8333	100	0.0121	-
1.0	120	-	0.2668
1.25	150	0.003	-
1.6667	200	0.0007	-
2.0	240	-	0.2562
2.0833	250	0.0008	-
2.5	300	0.0009	-
2.9167	350	0.0007	-
3.0	360	-	0.2572
3.3333	400	0.0005	-
3.75	450	0.0005	-
4.0	480	-	0.2571

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.12
SetFit: 1.0.1
Sentence Transformers: 2.2.2
Transformers: 4.35.2
PyTorch: 2.1.0+cu121
Datasets: 2.16.0
Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}