--- base_model: sentence-transformers/paraphrase-mpnet-base-v2 library_name: setfit metrics: - accuracy pipeline_tag: text-classification tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer widget: - text: category generator refuel fixed diesel special access no vendor acas problem description rbs generator fuel low - text: troubleshooting generator has been running non stop need emergency refuel alarms are as follows rbs commercial power fail rbs generator fuel low rbs generator running rbs generator shut down rbs gen transfer sw operated test results site will go off the air without urgent refuel trouble description per ticket tt000080377504 sfp_as cvl06424 5 external alarm fieldreplaceableunit=sau 1 alarmport=2 rbs commercial power fail history of trouble none vendor acas problem description fixed gen special access none - text: troubleshooting triage category generator shut down oss netcool alarms dxl05963 rbs generator shut down fieldreplaceableunit=sau alarmport=22 2024 08 21 02 11 15 smart alarm rbs generator shut down mdat verification y no repeats no open related tckt no active eim no intrusion knowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor test results triage category generator shut down oss netcool alarms dxl05963 rbs generator shut down fieldreplaceableunit=sau alarmport=22 2024 08 21 02 11 15 smart alarm rbs generator shut down mdat verification y no repeats no open related tckt no active eim no intrusion knowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor trouble description triage category generator shut down oss netcool alarms dxl05963 rbs generator shut down fieldreplaceableunit=sau alarmport=22 2024 08 21 02 11 15 smart alarm rbs generator shut down mdat verification y no repeats no open related tckt no active eim no intrusion knowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor history of trouble triage category generator shut down oss netcool alarms dxl05963 rbs generator shut down fieldreplaceableunit=sau alarmport=22 2024 08 21 02 11 15 smart alarm rbs generator shut down mdat verification y no repeats no open related tckt no active eim no intrusion knowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor vendor acas problem description triage category generator shut down oss netcool alarms dxl05963 rbs generator shut down fieldreplaceableunit=sau alarmport=22 2024 08 21 02 11 15 smart alarm rbs generator shut down mdat verification y no repeats no open related tckt no active eim no intrusion knowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor special access triage category generator shut down oss netcool alarms dxl05963 rbs generator shut down fieldreplaceableunit=sau alarmport=22 2024 08 21 02 11 15 smart alarm rbs generator shut down mdat verification y no repeats no open related tckt no active eim no intrusion knowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor - text: troubleshooting triage category gen fail oss netcool alarms rbs generator fail fieldreplaceableunit=sau alarmport=22 rbs generator fail ca placerville cell site lotus carlsen 2024 08 20 06 27 41 smart alarm rbs generator fail fieldreplaceableunit=sau alarmport=22 2024 08 20 06 27 36 mdat verification fixed gen history no repeats tab no open related tickets in aots knowledge judgement sending to vendor to check gen fail dispatch strategy vendor test results triage category gen fail oss netcool alarms rbs generator fail fieldreplaceableunit=sau alarmport=22 rbs generator fail ca placerville cell site lotus carlsen 2024 08 20 06 27 41 smart alarm rbs generator fail fieldreplaceableunit=sau alarmport=22 2024 08 20 06 27 36 mdat verification fixed gen history no repeats tab no open related tickets in aots knowledge judgement sending to vendor to check gen fail dispatch strategy vendor trouble description triage category gen fail oss netcool alarms rbs generator fail fieldreplaceableunit=sau alarmport=22 rbs generator fail ca placerville cell site lotus carlsen 2024 08 20 06 27 41 smart alarm rbs generator fail fieldreplaceableunit=sau alarmport=22 2024 08 20 06 27 36 mdat verification fixed gen history no repeats tab no open related tickets in aots knowledge judgement sending to vendor to check gen fail dispatch strategy vendor history of trouble na vendor acas problem description triage category gen fail oss netcool alarms rbs generator fail fieldreplaceableunit=sau alarmport=22 rbs generator fail ca placerville cell site lotus carlsen 2024 08 20 06 27 41 smart alarm rbs generator fail fieldreplaceableunit=sau alarmport=22 2024 08 20 06 27 36 mdat verification fixed gen history no repeats tab no open related tickets in aots knowledge judgement sending to vendor to check gen fail dispatch strategy vendor special access na - text: investigate gen fail requestor olivarez zachary requestor email zachary olivarez verizonwireless com requestor phone 760 927 0406 inference: true model-index: - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2 results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: accuracy value: 0.625 name: Accuracy --- # SetFit with sentence-transformers/paraphrase-mpnet-base-v2 This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 512 tokens - **Number of Classes:** 2 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 1 | | | 0 | | ## Evaluation ### Metrics | Label | Accuracy | |:--------|:---------| | **all** | 0.625 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("edwsiew/phantom-dispatch-02") # Run inference preds = model("category generator refuel fixed diesel special access no vendor acas problem description rbs generator fuel low") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:---------|:----| | Word count | 16 | 168.2540 | 915 | | Label | Training Sample Count | |:------|:----------------------| | 0 | 14 | | 1 | 49 | ### Training Hyperparameters - batch_size: (8, 8) - num_epochs: (1, 1) - max_steps: -1 - sampling_strategy: oversampling - num_iterations: 20 - body_learning_rate: (2e-05, 2e-05) - head_learning_rate: 2e-05 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: False - warmup_proportion: 0.1 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: False ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:------:|:----:|:-------------:|:---------------:| | 0.0032 | 1 | 0.31 | - | | 0.1587 | 50 | 0.0308 | - | | 0.3175 | 100 | 0.0131 | - | | 0.4762 | 150 | 0.0023 | - | | 0.6349 | 200 | 0.0056 | - | | 0.7937 | 250 | 0.0009 | - | | 0.9524 | 300 | 0.0003 | - | ### Framework Versions - Python: 3.12.0 - SetFit: 1.0.3 - Sentence Transformers: 3.0.1 - Transformers: 4.39.0 - PyTorch: 2.4.0+cu121 - Datasets: 2.21.0 - Tokenizers: 0.15.2 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```