---
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
metrics:
- accuracy
widget:
- text: 5 days before the flight, we were advised by BA that it had been cancelled
and asked us to rebook. There were flights 1 hour before and 1 hour after our
original flight but they made us take one 3 hours earlier. Our original ticket
(and the return flight a week later) included a checked bag. When we arrive at
Heathrow to check in we are told our ticket doesn't included a bag and we will
have to pay 75 pounds each to take them. We explained that the original ticket
had the bags and when they checked the system they confirmed that it did- but
not this one! They would make no effort to sort it out and just told us to pay
up or they would remove us from the flight. So we had no choice but to pay up,
thinking it should be pretty easy to get a refund. 15 months and 10 emails later
we have still not seen a penny. Every time I ask where my refund is and state
how long we have been waiting I get an inane email saying some departments take
longer than others to respond! Absolutely hopeless airline to stuff up the booking
in the first place and even worse customer service to not even attempt to solve
the problem at check in even when they could see the mistake THEY had made in
their system, compounded by being be unable to investigate a very simple claim,
and refund me 150 pounds in 15 months.
- text: They downgraded me from business to premium economy. They took three months
to get back to me and offered me 200 pounds or 400 AUD. The difference between
business class and premium economy (which I never fly) is $3000 AUD and the difference
between economy and business is $6500 AUD. They owe me at least 1500 pounds and
to be very fair 3250 pounds. The downgrade was insulting and incredibly painful
as I had a recent cancer operation and have a back injury which I have pointed
out to them - I need to lie down. The wait for them to get around to me has been
numbing. The insult of their offer has been the slap in the face to continue my
degradation. The people I have dealt with at the airport have been down to earth,
honest hard working people. The actual staff on the plane were warm and friendly
- and helpful. The systems under which BA operates are some sort of medieval torture.
- text: I flew from Istanbul to London in Business class. For more than half of the
flight a child was watching an iPad very loud with no phones plugged in - it was
three rows in front of me and disrupted my flight as I could hear it above my
headphones. Staff thought this was fine and refused to tell the mom to turn it
down. Why is this acceptable on a flight especially in business class? Has this
airline turned into a zoo? Are staff given no training in customer service? Why
is a child allowed to disrupt other passengers? When I told the staff she just
laughed at me. BA customer service is a disgrace. I've not even mentioned the
food but it was bad, really bad.
- text: 'Mexico City Airport is a zoo, but taking the late departure on BA to LHR
isn''t too bad. Club passengers can use the AA Admirals Lounge, which is surprisingly
good and not too busy as it seems to cater to BA''s club passengers only at this
time of day. Lovely staff, solid food & bar, really pleasant. We flew an older
787 with the old style club seats. Seated in the front cabin (just 3 rows of club)
in the window and adjecant aisle seat is really good if you travel with your partner.
Come to think of it, I will really miss this old style -once cutting edge- lay-out.
There is no better way to fly when you are a couple. We were in that last row
of the front cabin so both window and aisle seat had unobstructed access to the
aisle. Still terrible you have to pay for these seats, but it obvioulsy works
and we were happy getting these seats. Flying BA when the crew is good is amazing.
Service was impecable on this flight. Just the right mix of humor, service and
attention. Food seemed to be good as well, but hardly touched it. I am fine sleeping
in these old club world seats although bedding could be better. On time departure,
smooth flight and early arrival at Heathrow. We went outside for a smoke and returned
to the terminal through fast track. Security at LHR has always been terrible,
but today security staff was too busy chatting among themselves so all bags went
through without ''secundary'' checks. We had a lovely shower at the ''spa'', what
a fabulous facility. The South Lounge was crazy busy and it is just a design disaster.
Who ever choose the bizarre combination of furniture: what a mess! Food on offer
was solid and so are the drinks. Our home stretch to Amsterdam was pretty straight
forward. Lousy welcome at the door but crew recovered well and made everyone feel
special with excellent bar-cart service. I love the ''double''-servings of drinks.
English tea is about the worst meal concepts in the world and I really don''t
understand BA stuck to it through the years. No-one like the cucumber white bread
things, but hey... On time landing in Amsterdam but the usual long taxi then clumsy
jet bridge connection ended up with a 20 minute late arrival, why is it so hard
to connect a jetbridge in Amsterdam? Flying club in BA is still wonderful. I Love
it!'
- text: We traveled to Lisbon for our dream vacation, a cruise to Portugal and Spain.
Our friends did not EVER get their luggage. It was a two week cruise. Two weeks
without a change of clothes or her CPAP machine. Contacting customer service was
a nightmare. We never talked to a real person. Very little effort was put into
getting them their bags. In one port we were on the ship and the bags were at
the airport. The airlines did not deliver the bags nor did they tell our friends
that they needed to go get them. The airline couldn’t be bothered to take the
bags from the airport to the ship. BA says it is their policy to get you your
bags within 72 hours. That is a joke! It’s been over two weeks and they still
don’t have them. They are back home in the US and last they heard the bags were
in Lisbon! If your have a choice, do not fly British Airways. Customer Service
is non existent!
pipeline_tag: text-classification
inference: true
base_model: sentence-transformers/paraphrase-mpnet-base-v2
model-index:
- name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.8333333333333334
name: Accuracy
---
# SetFit with sentence-transformers/paraphrase-mpnet-base-v2
This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.
## Model Details
### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 512 tokens
- **Number of Classes:** 6 classes
### Model Sources
- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
### Model Labels
| Label | Examples |
|:------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 | - '4 Hours before takeoff we received a Mail stating a cryptic message that there are disruptions to be expected as there is a limit on how many planes can leave at the same time. So did the capacity of the Heathrow Airport really hit British Airways by surprise, 4h before departure? Anyhow - we took the one hour delay so what - but then we have been forced to check in our Hand luggage. I travel only with hand luggage to avoid waiting for the ultra slow processing of the checked in luggage. Overall 2h later at home than planed, with really no reason, just due to incompetent people. Service level far worse then Ryanair and triple the price. Really never again. Thanks for nothing.'
- 'Our connecting flight from London to Glasgow was canceled and we were forced to wait 12 hours in the airport for a new flight. When we received our luggage at Glasgow we discovered that our luggage was soaking wet. My wife’s luggage contents were wet and had to be air dried.'
- "Absolutely the worst experience ever. Flew into Heathrow, no problem. Waited for 5 hours, three extra hours, for our connection to Porto then the cancelled the flight. Scrambled for a hotel and a new flight the next day. They routed us through Madrid the next morning. That flight was 1.5 hours late so we missed our connection to Porto. Because we missed the connection (their fault), they cancelled our return flights. After two hours on the phone with customer service from the Madrid airport, they booked us a new return flight, then they made is pay $ 1300 to fly from Madrid to Porto. Then, for no reason they cancelled our return flight again! After hours, and hours and hours with customer service, we finally got a return flight five days after our original return flight. Meanwhile, they lost my wife's luggage. We finally got 15 DAYS after they lost it. Return flight? More problems with our reservations. At no point was British Airways helpful. They left the onus on us to fix their mistakes at every turn. They completely ruined a three week vacation and they don't care. They wouldn't even let us speak to a supervisor or manager. Awful, bordering on passenger abuse."
|
| 4 | - 'I\'m sure that BA have gradually made their economy experience (BA Euroflyer out of Gatwick) so miserable that passengers are tempted to upgrade to the lacklustre and overpriced business class. Long wait at the biz checkin ( due to status ) and dealt with by a bored agent - OK third party, but still representing BA - who hardly spoke and avoided eye contact and was more focussed on her fingernails. Lounge was crowded and grubby but at least the staff there are pleasant and helpful On board the seats are cramped and uncomfortable and it took over two hours before crew appeared with food sales and the "free water" which on Gatwick services consists of a half full plastic cup rather than the luxury of a small bottle.. A fellow passenger mentioned the delay and the fact that they had no sandwiches for sale which prompted the flight attendant to say that the service was much better on Easyjet who do two service runs etc.'
- 'BA standards continue to decline every time I fly with them. This time, a 45 min late departure, which seems the norm for BA, and no information from the rude and clueless boarding gate staff. The Club lounge was overcrowded, dirty and grubby and cleaning staff could not cope. How many more cutbacks is BA going to apply before they become truly low cost. As a 35 year veteran user of BA, while people complained of Alex Cruz cutting costs, the past 3 years has seen them decline very quickly under their latest CEO. One reads the occasional article about BA improving things, but I have yet to see anything substantive. There seem to be few British staff left amongst the cabin crew, so maybe time for a name change and give the flag carrier status to a proper low-cost - and sadly the staff service has declined year on year. Onboard, the catering remains poor and surprised that Do&Co can produce such rubbish meals when I have sampled their Turkish Airlines equivalent in August that was superb - guessing BA are too tight with their budget. Onboard WiFi service is a joke and should be avoided.'
- 'Good lounge at Cape Town. On time departure. Drinks high quality. Sound dinner service. Breakfast reasonable. Arrival lounge was good but showers cold.'
|
| 5 | - "Despite boarding being the usual free for all at LHR with groups 1-3 being called to board all at once. Whilst those passengers who had agreed to check in a bag in at the gate, getting to board before anyone else! However, the service on board was impeccable. Superb crew, friendly, attentive, interacting with passengers and generally appearing to be enjoying their job. Like BA in the old days and at its best. Food choice and quality seems to have improved too (no beef cheeks thankfully) and even the Club kitchen appears to have returned. Old style Club seating, but is at least better, and feels less cramped, on the 787's."
- "Flew back from Malta after scattering our son's ashes at his favourite spots. When we boarded the aircraft for our return flight my wife became upset. A member of the cabin crew noticed this and immediately spoke to my wife and tried to reassure her. Her name is Tracey, an exceptional crew member. I noticed she spoke to other travellers in a courteous manner and was friendly. Each time she passed my wife she asked how she was and would she like anything. This kindness made a difficult time easier for us both. Aircraft seats were comfortable and service good for a 2hrs. 45mins. flight. Thank you B.A."
- 'Excellent service levels, proactive crew and superb food and beverages. I found all aspects of the service to be superior to BA business class, even the new BA Club Suites. First Class felt exclusive, the dedicated crew were clearly used to working the premium cabin, enjoying their work, were proactive, friendly, very respectful and the food and beverage offering a considerable step up from business class. The seat, with sliding double doors was spacious, well designed and very comfortable. British Airways offered a solid First Class service, from Concorde Lounge to boarding (which was actually by group number for a change) to the very warm and genuine welcome on boarding. A consistent, solid, comfortable service from start to finish and if the price was right, I would not hesitate to book First Class with British Airways again.'
|
| 2 | - "I take comfort in reading the last ten or so reviews to see I'm not the only traveller to put in words, how absolutely abysmal British Airways are! The only reason we are stuck using this awful airline is to use up points. We booked Club Europe months ago. London to Athens, sat in row 3 and told they had run out of food options, only choice was lentil pie or green curry. Returning Athens to London Club Europe. Couldn't check in online as promised so arrived at the airport 3hrs early. To be told they had overbooked the flight and basically we were being 'bumped' to economy. This airline fails to deliver time and time again. It is a national disgrace and I implore everyone to choose an alternative. "
- "You can buy sandwiches and crisps but don't expect a hot meal. Because the plane is a narrow body , there are no screens. Not helping matters is the fact that the BA prebooking shop for food didn't recognise the flight number, and the onboard WiFi didn't work. "
- 'The airplanes and the lounges are worn out, old and broken. From Dallas to Heathrow, multiple first class seating and electronics were non functional and poorly designed. The first class seating from Heathrow to Dubrovnik was nothing more than a tray blocking the middle seat of a standard coach ticket. Very disappointed.'
|
| 3 | - 'We traveled to Lisbon for our dream vacation, a cruise to Portugal and Spain. Our friends did not EVER get their luggage. It was a two week cruise. Two weeks without a change of clothes or her CPAP machine. Contacting customer service was a nightmare. We never talked to a real person. Very little effort was put into getting them their bags. In one port we were on the ship and the bags were at the airport. The airlines did not deliver the bags nor did they tell our friends that they needed to go get them. The airline couldn’t be bothered to take the bags from the airport to the ship. BA says it is their policy to get you your bags within 72 hours. That is a joke! It’s been over two weeks and they still don’t have them. They are back home in the US and last they heard the bags were in Lisbon! If your have a choice, do not fly British Airways. Customer Service is non existent!'
- "I have flow on BA several times since the airline started non-stop service between Austin, TX and London Heathrow. Every year the service gets worse. I brought a tape measure with me to see how far apart the seats are in the economy section. 25½ inches. BA has removed bathrooms in order to squeeze a few more rows of seats onto the aircraft. Thankfully I'm not a big person and was able to squeeze into my seat without being too uncomfortable. When BA first started flying out of Austin, the food was great. Now, when I arrive at Heathrow, I get real food and take it on the plane with me. I don't know where or who makes BA's food, but it is not eatable. If BA did not have a direct flight from my hometown to London, I would fly with a different airline. When we checked into Heathrow, the line was extremely long and only one person was checking people in. It took nearly an hour to get to the counter. I understand airlines are making record profits and need to make up lost revenue from the pandemic, but I think BA has gone too far and has little concern for customer service in pursuit of profits."
- 'Angry, disappointed, and unsatisfied. My route was from London to Atlanta. My suitcase was not boarded, therefore not landed with me. For both comfort and safety reason, a bag always fly with its passenger and that did not happen. Claims and few phone calls were made by desk assistants who answered my questions unprofessionally and miserably. Certainly, I was left with nothing but my backpack which contained not more than few snacks. Neither clothes nor anything else was ever provided as an apology. Meanwhile, I was also told that my bag would have been delivered through the next 24 hours which also did not happen. British Airways is a great airline to fly with but its organization, when it comes to customer service, is poor and uncertain. Still waiting for my bag.'
|
| 6 | - 'They downgraded me from business to premium economy. They took three months to get back to me and offered me 200 pounds or 400 AUD. The difference between business class and premium economy (which I never fly) is $3000 AUD and the difference between economy and business is $6500 AUD. They owe me at least 1500 pounds and to be very fair 3250 pounds. The downgrade was insulting and incredibly painful as I had a recent cancer operation and have a back injury which I have pointed out to them - I need to lie down. The wait for them to get around to me has been numbing. The insult of their offer has been the slap in the face to continue my degradation. The people I have dealt with at the airport have been down to earth, honest hard working people. The actual staff on the plane were warm and friendly - and helpful. The systems under which BA operates are some sort of medieval torture.'
- 'BA cancelled our flight and could not book us onto any flight for the next 4 days. I had to spend £5k in expenses to get us back with another airline. 2 months later I am still waiting for reimbursement and have had no response to my 3 written complaints. Even the call centre staff tell me that head office have taken down the telephone number and email for the department who deal with this, so even they cannot escalate or inquire. Utterly useless.\n'
- "This is the worst experience I have ever had with any airline. I flew with British Airline on August 21st 2022. I had one checked bag that was lost at Heathrow Airport. I am aware that at that time there was a system error with the baggage belt and many travelers were affected. I filed a report for my missing bag on the same day I arrived at LAX airport, and that was on August 21st. I waited patiently for three weeks (which is the time frame to either find the bag or be eligible for a compensation). When I did not receive any update on my lost bag, I filed a claim for compensation, and that was on September 11th. Since then, I have been waiting for a response from the customer relation with no good signs of resolving the issue. I emailed them and tried to update the claim but still no response until today. I contacted the baggage team and they were not helpful and did not ease my prolonged waiting. The only way to contact customer relations is via the website and they're not even getting back to me. Every time I talk to a customer service agent, they tell me to wait some more. I have been waiting for almost three months now and my issue is not resolved yet. This whole experience was a hassle. Very unprofessional for a renowned airline like BA."
|
## Evaluation
### Metrics
| Label | Accuracy |
|:--------|:---------|
| **all** | 0.8333 |
## Uses
### Direct Use for Inference
First install the SetFit library:
```bash
pip install setfit
```
Then you can load this model and run inference.
```python
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("I flew from Istanbul to London in Business class. For more than half of the flight a child was watching an iPad very loud with no phones plugged in - it was three rows in front of me and disrupted my flight as I could hear it above my headphones. Staff thought this was fine and refused to tell the mom to turn it down. Why is this acceptable on a flight especially in business class? Has this airline turned into a zoo? Are staff given no training in customer service? Why is a child allowed to disrupt other passengers? When I told the staff she just laughed at me. BA customer service is a disgrace. I've not even mentioned the food but it was bad, really bad.")
```
## Training Details
### Training Set Metrics
| Training set | Min | Median | Max |
|:-------------|:----|:---------|:----|
| Word count | 23 | 144.6667 | 418 |
| Label | Training Sample Count |
|:------|:----------------------|
| 1 | 7 |
| 2 | 7 |
| 3 | 7 |
| 4 | 7 |
| 5 | 7 |
| 6 | 7 |
### Training Hyperparameters
- batch_size: (8, 8)
- num_epochs: (3, 3)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 123
- eval_max_steps: -1
- load_best_model_at_end: False
### Training Results
| Epoch | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0054 | 1 | 0.1708 | - |
| 0.2717 | 50 | 0.0981 | - |
| 0.5435 | 100 | 0.0739 | - |
| 0.8152 | 150 | 0.0039 | - |
| 1.0870 | 200 | 0.0005 | - |
| 1.3587 | 250 | 0.0007 | - |
| 1.6304 | 300 | 0.0002 | - |
| 1.9022 | 350 | 0.0005 | - |
| 2.1739 | 400 | 0.0004 | - |
| 2.4457 | 450 | 0.0003 | - |
| 2.7174 | 500 | 0.0005 | - |
| 2.9891 | 550 | 0.0004 | - |
### Framework Versions
- Python: 3.8.10
- SetFit: 1.0.3
- Sentence Transformers: 2.3.1
- Transformers: 4.37.2
- PyTorch: 2.2.0
- Datasets: 2.17.1
- Tokenizers: 0.15.2
## Citation
### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
```