--- library_name: setfit tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer metrics: - accuracy widget: - text: 'Dear [Recipient], We wanted to inform you about an update regarding [briefly describe the topic or subject]. Please note that this email is sent from a do-not-reply address, and no response is required. Should you have any questions or require further assistance, please feel free to contact our support team at [support email/phone number]. Thank you for your attention to this matter. Best regards, [Your Name] [Your Position/Department (if applicable)] [Your Company/Organization Name] ' - text: 'Dear [Recipient], We regret to inform you that the email with the subject "[Subject]" sent on [Date/Time] was not successfully delivered due to network issues. To ensure successful delivery of future communications, we recommend checking your internet connection and trying to resend the email. If the issue persists, please contact your network administrator or Internet Service Provider (ISP) for further assistance. If you require any assistance or have any questions, please don''t hesitate to contact our support team at [Support Contact Information]. Thank you for your attention to this matter. Best regards, [Your Name] [Your Position/Department (if applicable)] [Your Company Name] [Your Contact Information]' - text: 'Dear [Recipient], I hope this email finds you well. I wanted to remind you that the deadline for [Task/Project] is approaching on [Deadline Date]. [Include any relevant details about the task or project, such as specific requirements or milestones.] Please prioritize completing this task within the given timeline to ensure timely delivery and project success. If you encounter any challenges or require assistance, feel free to reach out to your team members or supervisor for support. Thank you for your attention to this matter. Let''s work together to meet our project goals. Best regards, [Your Name] [Your Position/Department] [Your Company Name] [Your Contact Information]' - text: 'Hello, Thank you for your email. I am currently attending a conference and may have limited access to email until [Date of Return]. I will respond to your message as soon as possible upon my return. For urgent matters, please contact [Alternative Contact Information]. Best regards, [Your Name]' - text: 'Hello [Recipient''s Name], As a valued member of our community, we''re delighted to offer you an exclusive discount! Enjoy [Discount percentage]% off your next purchase when you use code [Promo code] at checkout. Explore our latest arrivals and treat yourself to something special. But hurry - this offer expires soon! Shop now at [Link to Website] and enjoy savings on us. Best regards, [Your Name] [Your Position/Department] [Your Contact Information]' pipeline_tag: text-classification inference: true base_model: sentence-transformers/paraphrase-mpnet-base-v2 --- # SetFit with sentence-transformers/paraphrase-mpnet-base-v2 This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 512 tokens - **Number of Classes:** 10 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Password reset emails | | | System Alerts | | | Promotional emails | | | Automatic subscription confirmation emails | | | Email Delivery failure notifications | | | Out-of- office replies | | | Read receipts or delivery confirmation | | | Do-not-reply or no-reply emails (explicitly mentioned in email) | | | Reminders | | | Security Alerts | | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("mohamednihal/no_reply") # Run inference preds = model("Hello, Thank you for your email. I am currently attending a conference and may have limited access to email until [Date of Return]. I will respond to your message as soon as possible upon my return. For urgent matters, please contact [Alternative Contact Information]. Best regards, [Your Name]") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:-------|:----| | Word count | 44 | 85.24 | 170 | | Label | Training Sample Count | |:----------------------------------------------------------------|:----------------------| | Automatic subscription confirmation emails | 5 | | Do-not-reply or no-reply emails (explicitly mentioned in email) | 5 | | Email Delivery failure notifications | 5 | | Out-of- office replies | 5 | | Password reset emails | 5 | | Promotional emails | 5 | | Read receipts or delivery confirmation | 5 | | Reminders | 5 | | Security Alerts | 5 | | System Alerts | 5 | ### Training Hyperparameters - batch_size: (2, 2) - num_epochs: (1, 1) - max_steps: -1 - sampling_strategy: oversampling - num_iterations: 20 - body_learning_rate: (2.887681170264626e-05, 2.887681170264626e-05) - head_learning_rate: 2.887681170264626e-05 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: False - warmup_proportion: 0.1 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: False ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:-----:|:----:|:-------------:|:---------------:| | 0.001 | 1 | 0.1811 | - | | 0.05 | 50 | 0.3964 | - | | 0.1 | 100 | 0.0705 | - | | 0.15 | 150 | 0.0115 | - | | 0.2 | 200 | 0.0477 | - | | 0.25 | 250 | 0.0022 | - | | 0.3 | 300 | 0.0044 | - | | 0.35 | 350 | 0.0017 | - | | 0.4 | 400 | 0.001 | - | | 0.45 | 450 | 0.0001 | - | | 0.5 | 500 | 0.0006 | - | | 0.55 | 550 | 0.0008 | - | | 0.6 | 600 | 0.0003 | - | | 0.65 | 650 | 0.0006 | - | | 0.7 | 700 | 0.0003 | - | | 0.75 | 750 | 0.0017 | - | | 0.8 | 800 | 0.0001 | - | | 0.85 | 850 | 0.0002 | - | | 0.9 | 900 | 0.0 | - | | 0.95 | 950 | 0.0002 | - | | 1.0 | 1000 | 0.0002 | - | | 0.001 | 1 | 0.0001 | - | | 0.05 | 50 | 0.0002 | - | | 0.1 | 100 | 0.0014 | - | | 0.15 | 150 | 0.008 | - | | 0.2 | 200 | 0.0017 | - | | 0.25 | 250 | 0.0018 | - | | 0.3 | 300 | 0.0187 | - | | 0.35 | 350 | 0.0021 | - | | 0.4 | 400 | 0.0001 | - | | 0.45 | 450 | 0.0 | - | | 0.5 | 500 | 0.0003 | - | | 0.55 | 550 | 0.0001 | - | | 0.6 | 600 | 0.0 | - | | 0.65 | 650 | 0.0002 | - | | 0.7 | 700 | 0.0 | - | | 0.75 | 750 | 0.0003 | - | | 0.8 | 800 | 0.0 | - | | 0.85 | 850 | 0.0 | - | | 0.9 | 900 | 0.0001 | - | | 0.95 | 950 | 0.0001 | - | | 1.0 | 1000 | 0.0 | - | | 0.001 | 1 | 0.0 | - | | 0.05 | 50 | 0.0001 | - | | 0.1 | 100 | 0.0018 | - | | 0.15 | 150 | 0.0001 | - | | 0.2 | 200 | 0.0042 | - | | 0.25 | 250 | 0.0009 | - | | 0.3 | 300 | 0.0001 | - | | 0.35 | 350 | 0.0018 | - | | 0.4 | 400 | 0.0002 | - | | 0.45 | 450 | 0.0001 | - | | 0.5 | 500 | 0.0 | - | | 0.55 | 550 | 0.0001 | - | | 0.6 | 600 | 0.0 | - | | 0.65 | 650 | 0.0 | - | | 0.7 | 700 | 0.0 | - | | 0.75 | 750 | 0.0 | - | | 0.8 | 800 | 0.0 | - | | 0.85 | 850 | 0.0 | - | | 0.9 | 900 | 0.0 | - | | 0.95 | 950 | 0.0 | - | | 1.0 | 1000 | 0.0 | - | | 0.001 | 1 | 0.0 | - | | 0.05 | 50 | 0.0 | - | | 0.1 | 100 | 0.0005 | - | | 0.15 | 150 | 0.0025 | - | | 0.2 | 200 | 0.0 | - | | 0.25 | 250 | 0.0002 | - | | 0.3 | 300 | 0.0 | - | | 0.35 | 350 | 0.0003 | - | | 0.4 | 400 | 0.0001 | - | | 0.45 | 450 | 0.0 | - | | 0.5 | 500 | 0.0 | - | | 0.55 | 550 | 0.0 | - | | 0.6 | 600 | 0.0 | - | | 0.65 | 650 | 0.0 | - | | 0.7 | 700 | 0.0 | - | | 0.75 | 750 | 0.0 | - | | 0.8 | 800 | 0.0 | - | | 0.85 | 850 | 0.0 | - | | 0.9 | 900 | 0.0 | - | | 0.95 | 950 | 0.0 | - | | 1.0 | 1000 | 0.0 | - | ### Framework Versions - Python: 3.10.12 - SetFit: 1.0.3 - Sentence Transformers: 2.4.0 - Transformers: 4.37.2 - PyTorch: 2.1.0+cu121 - Datasets: 2.17.1 - Tokenizers: 0.15.2 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```