README.md

metadata

base_model: sentence-transformers/paraphrase-mpnet-base-v2
library_name: setfit
metrics:
  - accuracy
pipeline_tag: text-classification
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
widget:
  - text: >-
      I just watched 'The Shawshank Redemption' and I have to say, Tim Robbins
      and Morgan Freeman delivered outstanding performances. Their acting skills
      truly brought the characters to life. The way they portrayed the emotional
      depth of their characters was impressive. I highly recommend this movie to
      anyone who loves a good drama.
  - text: >-
      I walked into this movie expecting a lot, but what I got was a complete
      waste of time. The acting was subpar, the plot was predictable, and the
      dialogue was cringeworthy. I've seen high school productions that were
      better. The only thing that kept me awake was the hope that something,
      anything, would happen to make this movie worth watching. Unfortunately,
      that never came. I would not recommend this to my worst enemy. 1/10, would
      not watch again even if you paid me.
  - text: >-
      I just watched this movie and I'm still grinning from ear to ear. The
      humor is wickedly clever and the cast is perfectly assembled. It's a
      laugh-out-loud masterpiece that will leave you feeling uplifted and
      entertained.
  - text: >-
      I was really looking forward to trying out this new restaurant, but
      unfortunately, it was a huge disappointment. The service was slow, the
      food was cold, and the ambiance was non-existent. I ordered the burger,
      but it was overcooked and tasted like it had been sitting out for hours.
      Needless to say, I won't be back.
  - text: >-
      I recently visited this restaurant for lunch and had an amazing
      experience. The service was top-notch, our server was friendly and
      attentive, and the food was incredible. I ordered the grilled chicken
      salad and it was cooked to perfection. The portion size was generous and
      the prices were very reasonable. I would highly recommend this place to
      anyone looking for a great meal.
inference: true
model-index:
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.87812
            name: Accuracy

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
positive sentiment	"I just watched the latest Marvel movie and I'm still reeling from the shocking plot twist at the end. I didn't see it coming and it completely flipped my expectations on their head. The way the story unfolded was pure genius and had me on the edge of my seat the entire time. I'm not even kidding when I say that this movie is a must-see for anyone who loves a good surprise. 10/10 would recommend." 'I recently visited this restaurant and was blown away by the exceptional service from the staff. Our server, Alex, was attentive, knowledgeable, and made sure we had everything we needed throughout our meal. The food was delicious, but the service was truly what made our experience stand out. I would highly recommend this place to anyone looking for a great dining experience.' "I just watched the funniest movie of my life, 'Dumb and Dumber'! Jim Carrey's comedic timing is unmatched. He has this incredible ability to make you laugh without even trying. The movie is full of hilarious moments, and I found myself giggling uncontrollably throughout. I highly recommend it to anyone looking for a good laugh."
negative sentiment	"I'm extremely disappointed with my recent purchase from this restaurant. The food was overcooked and the service was slow. The prices are way too high for the quality of food you receive. I won't be returning anytime soon." "I'm extremely disappointed with the service I received at this restaurant. The hostess was completely unfriendly and unhelpful. We were seated for 20 minutes before anyone even came to take our order. The food was overpriced and took an hour to arrive. The server seemed put off by our presence and didn't even bother to refill our drinks. Needless to say, we will never be back." 'I was really looking forward to this movie, but unfortunately, it fell flat. The plot was predictable and lacked any real tension or suspense. The characters were underdeveloped and their motivations were unclear. The pacing was slow and the ending was completely unsatisfying. Overall, I was disappointed by the lack of effort put into creating a compelling story. 1/10 would not recommend.'

Evaluation

Metrics

Label	Accuracy
all	0.8781

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("I just watched this movie and I'm still grinning from ear to ear. The humor is wickedly clever and the cast is perfectly assembled. It's a laugh-out-loud masterpiece that will leave you feeling uplifted and entertained.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	20	50.76	80

Label	Training Sample Count
negative sentiment	13
positive sentiment	12

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (5, 5)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0455	1	0.1789	-
1.0	22	-	0.013
2.0	44	-	0.0024
2.2727	50	0.0003	-
3.0	66	-	0.0014
4.0	88	-	0.0011
4.5455	100	0.0003	-
5.0	110	-	0.0013

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.9.19
• SetFit: 1.1.0.dev0
• Sentence Transformers: 3.0.1
• Transformers: 4.39.0
• PyTorch: 2.4.0
• Datasets: 2.20.0
• Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}