Edit model card

SetFit with thenlper/gte-large

This is a SetFit model trained on the Ramyashree/Dataset-train500-test100withwronginput dataset that can be used for Text Classification. This SetFit model uses thenlper/gte-large as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
create_account
  • "I don't have an online account, what do I have to do to register?"
  • 'can you tell me if i can regisger two accounts with a single email address?'
  • 'I have no online account, open one, please'
edit_account
  • 'how can I modify the information on my profile?'
  • 'can u ask an agent how to make changes to my profile?'
  • 'I want to update the information on my profile'
delete_account
  • 'can I close my account?'
  • "I don't want my account, can you delete it?"
  • 'how do i close my online account?'
switch_account
  • 'I would like to use my other online account , could you switch them, please?'
  • 'i want to use my other online account, can u change them?'
  • 'how do i change to another account?'
get_invoice
  • 'what can you tell me about getting some bills?'
  • 'tell me where I can request a bill'
  • 'ask an agent if i can obtain some bills'
get_refund
  • 'the game was postponed, help me obtain a reimbursement'
  • 'the game was postponed, what should I do to obtain a reimbursement?'
  • 'the concert was postponed, what should I do to request a reimbursement?'
payment_issue
  • 'i have an issue making a payment with card and i want to inform of it, please'
  • 'I got an error message when I attempted to pay, but my card was charged anyway and I want to notify it'
  • 'I want to notify a problem making a payment, can you help me?'
check_refund_policy
  • "I'm interested in your reimbursement polivy"
  • 'i wanna see your refund policy, can u help me?'
  • 'where do I see your money back policy?'
recover_password
  • 'my online account was hacked and I want tyo get it back'
  • "I lost my password and I'd like to retrieve it, please"
  • 'could u ask an agent how i can reset my password?'
track_refund
  • 'tell me if my refund was processed'
  • 'I need help checking the status of my refund'
  • 'I want to see the status of my refund, can you help me?'

Evaluation

Metrics

Label Accuracy
all 0.94

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Ramyashree/gte-large-train-test-3")
# Run inference
preds = model("where to change to another online account")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 3 10.258 24
Label Training Sample Count
check_refund_policy 50
create_account 50
delete_account 50
edit_account 50
get_invoice 50
get_refund 50
payment_issue 50
recover_password 50
switch_account 50
track_refund 50

Training Hyperparameters

  • batch_size: (16, 16)
  • num_epochs: (1, 1)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 20
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0008 1 0.3248 -
0.04 50 0.1606 -
0.08 100 0.0058 -
0.12 150 0.0047 -
0.16 200 0.0009 -
0.2 250 0.0007 -
0.24 300 0.001 -
0.28 350 0.0008 -
0.32 400 0.0005 -
0.36 450 0.0004 -
0.4 500 0.0005 -
0.44 550 0.0005 -
0.48 600 0.0006 -
0.52 650 0.0005 -
0.56 700 0.0004 -
0.6 750 0.0004 -
0.64 800 0.0002 -
0.68 850 0.0003 -
0.72 900 0.0002 -
0.76 950 0.0002 -
0.8 1000 0.0003 -
0.84 1050 0.0002 -
0.88 1100 0.0002 -
0.92 1150 0.0003 -
0.96 1200 0.0003 -
1.0 1250 0.0003 -

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.1
  • Sentence Transformers: 2.2.2
  • Transformers: 4.35.2
  • PyTorch: 2.1.0+cu121
  • Datasets: 2.15.0
  • Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
8
Safetensors
Model size
335M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for Ramyashree/gte-large-train-test-3

Base model

thenlper/gte-large
Finetuned
(14)
this model

Dataset used to train Ramyashree/gte-large-train-test-3

Evaluation results