Hugging Face's logo

kenhktsui
/
setfit_test_imdb

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results
Model card Files Community
kenhktsui commited on
Commit
63c0fb6
·
verified ·
1 Parent(s): 3599e9a

Add SetFit model

Browse files
Files changed (5) hide show
  1. README.md +58 -42
  2. config.json +1 -1
  3. config_setfit.json +2 -1
  4. model.safetensors +1 -1
  5. model_head.pkl +2 -2
README.md CHANGED
@@ -10,28 +10,33 @@ tags:
10
  - text-classification
11
  - generated_from_setfit_trainer
12
  widget:
13
- - text: I'm extremely disappointed with the service I received at this restaurant.
14
- The waiter was unfriendly and took ages to take our order. When I asked for a
15
- refill on my drink, I was met with a scowl. The food was overpriced and not even
16
- close to being worth it. Needless to say, I will not be returning.
17
- - text: I was a bit skeptical about trying this new restaurant, but I'm so glad I
18
- took the chance. The service was top-notch and the food was even better than I
19
- expected. The portions were generous and the prices were very reasonable. I had
20
- the chicken parmesan and it was cooked to perfection. I would highly recommend
21
- this place to anyone looking for a great dining experience.
22
- - text: I walked into this movie expecting a thrilling sci-fi adventure, but what
23
- I got was a convoluted mess of a plot with characters that felt like cardboard
24
- cutouts. The special effects were the only redeeming quality, but even they couldn't
25
- save this overhyped disaster from itself. 1/10 would not recommend.
26
- - text: I just watched the new movie 'Infectious Rhythms' and I'm still reeling from
27
- the infectious energy of the lead singer. The way the music takes over the entire
28
- scene is pure magic. The film's ability to capture the raw emotion of the band's
29
- performance is a testament to the director's skill. I'm so glad I got to experience
30
- this movie, it's a must-see for any music lover!
31
- - text: I'm extremely disappointed with my recent visit to this restaurant. I waited
32
- for 45 minutes just to be seated, and then another 20 minutes for my food to arrive.
33
- The service was slow and unfriendly. The food was overcooked and tasted bland.
34
- I won't be coming back here anytime soon.
 
 
 
 
 
35
  inference: true
36
  model-index:
37
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
@@ -45,7 +50,7 @@ model-index:
45
  split: test
46
  metrics:
47
  - type: accuracy
48
- value: 0.87712
49
  name: Accuracy
50
  ---
51
 
@@ -65,7 +70,7 @@ The model has been trained using an efficient few-shot learning technique that i
65
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
66
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
67
  - **Maximum Sequence Length:** 512 tokens
68
- - **Number of Classes:** 2 classes
69
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
70
  <!-- - **Language:** Unknown -->
71
  <!-- - **License:** Unknown -->
@@ -77,17 +82,18 @@ The model has been trained using an efficient few-shot learning technique that i
77
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
78
 
79
  ### Model Labels
80
- | Label | Examples |
81
- |:------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
82
- | 1 | <ul><li>'The soundtrack of this movie is absolutely phenomenal. The catchy melodies will stay with you long after the credits roll. The way the composer weaves together the different themes is pure magic. I found myself humming the tunes for days after watching the film. The score perfectly complements the on-screen action, elevating the entire viewing experience. I highly recommend this movie to anyone who loves music as much as I do.'</li><li>"Just got back from the most amazing vacation in Bali! The beaches were stunning, the food was incredible, and the people were so friendly. I'm already planning my next trip back #BaliBliss #ParadiseFound"</li><li>'I recently visited this restaurant and was blown away by the excellent service and the mouth-watering dishes. The staff were attentive and knowledgeable about the menu, making great recommendations. The atmosphere was cozy and intimate, perfect for a romantic evening. Overall, I would highly recommend this place to anyone looking for a top-notch dining experience.'</li></ul> |
83
- | 0 | <ul><li>'I walked into this movie expecting a masterpiece, but what I got was a lazy, uninspired attempt at storytelling. The plot was predictable, the characters were one-dimensional, and the dialogue was cringe-worthy. I left the theater feeling disappointed and underwhelmed. Overall, I would not recommend this movie to anyone.'</li><li>"I was really looking forward to trying this new restaurant, but unfortunately, the food was a huge letdown. The burger was overcooked and the fries were soggy. The service was also slow and our server seemed completely uninterested in our table. Needless to say, we won't be back."</li><li>"Just got into a fight with my ex and I'm so fed up with their drama. Can't believe they're still trying to contact me after everything they've done. #hatespeech #toxicrelationship"</li></ul> |
 
84
 
85
  ## Evaluation
86
 
87
  ### Metrics
88
  | Label | Accuracy |
89
  |:--------|:---------|
90
- | **all** | 0.8771 |
91
 
92
  ## Uses
93
 
@@ -107,7 +113,7 @@ from setfit import SetFitModel
107
  # Download from the 🤗 Hub
108
  model = SetFitModel.from_pretrained("setfit_model_id")
109
  # Run inference
110
- preds = model("I'm extremely disappointed with my recent visit to this restaurant. I waited for 45 minutes just to be seated, and then another 20 minutes for my food to arrive. The service was slow and unfriendly. The food was overcooked and tasted bland. I won't be coming back here anytime soon.")
111
  ```
112
 
113
  <!--
@@ -139,12 +145,13 @@ preds = model("I'm extremely disappointed with my recent visit to this restauran
139
  ### Training Set Metrics
140
  | Training set | Min | Median | Max |
141
  |:-------------|:----|:--------|:----|
142
- | Word count | 25 | 50.6333 | 78 |
143
 
144
  | Label | Training Sample Count |
145
  |:------|:----------------------|
146
  | 0 | 15 |
147
- | 1 | 15 |
 
148
 
149
  ### Training Hyperparameters
150
  - batch_size: (16, 16)
@@ -164,16 +171,25 @@ preds = model("I'm extremely disappointed with my recent visit to this restauran
164
  - load_best_model_at_end: True
165
 
166
  ### Training Results
167
- | Epoch | Step | Training Loss | Validation Loss |
168
- |:-------:|:------:|:-------------:|:---------------:|
169
- | 0.0333 | 1 | 0.199 | - |
170
- | 1.0 | 30 | - | 0.0021 |
171
- | 1.6667 | 50 | 0.0001 | - |
172
- | **2.0** | **60** | **-** | **0.0009** |
173
- | 3.0 | 90 | - | 0.0009 |
174
- | 3.3333 | 100 | 0.0001 | - |
175
- | 4.0 | 120 | - | 0.001 |
176
- | 5.0 | 150 | 0.0001 | 0.001 |
 
 
 
 
 
 
 
 
 
177
 
178
  * The bold row denotes the saved checkpoint.
179
  ### Framework Versions
 
10
  - text-classification
11
  - generated_from_setfit_trainer
12
  widget:
13
+ - text: The US Securities and Exchange Commission (SEC) has announced a new set of
14
+ guidelines for cryptocurrency exchanges, aiming to improve transparency and investor
15
+ protection. The move is seen as a positive step towards regulating the industry,
16
+ but experts warn that it may also lead to increased compliance costs for smaller
17
+ players. Overall, the impact on the market is expected to be neutral, with some
18
+ analysts predicting a slight increase in trading volumes as investors adjust to
19
+ the new rules.
20
+ - text: 'Market volatility is expected to continue in the coming weeks, but overall,
21
+ the trend remains stable. Investors are advised to remain cautious but not make
22
+ any drastic decisions. #marketanalysis #trading'
23
+ - text: The US Federal Reserve's decision to keep interest rates unchanged is a major
24
+ boost for the stock market, signaling a strong economy and increased investor
25
+ confidence. This move is expected to fuel further growth in the tech sector, with
26
+ many analysts predicting a significant increase in stock prices over the next
27
+ quarter. The Fed's dovish stance on inflation is also seen as a positive sign
28
+ for the market, as it suggests that the central bank is willing to take a more
29
+ accommodative approach to monetary policy. Overall, this news is expected to have
30
+ a bullish impact on the market, with many experts predicting a significant rally
31
+ in the coming days.
32
+ - text: Just heard that the Federal Reserve is going to raise interest rates by 50
33
+ basis points at their meeting next week. This is a huge blow to the stock market
34
+ and I'm expecting a major sell-off. I'm getting out of my long positions and going
35
+ short on the Dow. Anyone else seeing this as a bearish signal?
36
+ - text: US inflation rate hits 3.2% in June, highest in 3 years. Experts warn of potential
37
+ recession as consumers face rising costs. The Federal Reserve is expected to raise
38
+ interest rates to combat inflation, but some economists say it may be too little,
39
+ too late. The S&P 500 fell 2.5% on the news, with tech stocks leading the decline.
40
  inference: true
41
  model-index:
42
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
 
50
  split: test
51
  metrics:
52
  - type: accuracy
53
+ value: 0.6226968174204355
54
  name: Accuracy
55
  ---
56
 
 
70
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
71
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
72
  - **Maximum Sequence Length:** 512 tokens
73
+ - **Number of Classes:** 3 classes
74
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
75
  <!-- - **Language:** Unknown -->
76
  <!-- - **License:** Unknown -->
 
82
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
83
 
84
  ### Model Labels
85
+ | Label | Examples |
86
+ |:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
87
+ | 1 | <ul><li>"The Federal Reserve's decision to keep interest rates steady is a major boost for the stock market, with the Dow Jones surging 200 points in response. The Fed's dovish stance on monetary policy is expected to continue, providing a tailwind for equities in the coming months."</li><li>"Shares of tech giant Apple Inc (AAPL) surged 5.3% in pre-market trading after the company reported a 27% jump in quarterly profits, exceeding analyst expectations. The strong earnings report has investors optimistic about the company's future prospects, with many expecting the stock to reach new highs in the coming months."</li><li>"Just saw the latest jobs report and I'm feeling super bullish on the market. The low unemployment rate is a clear sign that the economy is on the right track. With more people working, consumer spending is likely to increase, which should boost stocks. I'm buying into the rally and holding on tight. #lowunemployment #bullmarket"</li></ul> |
88
+ | 2 | <ul><li>'The US stock market experienced a slight dip in trading hours today, with the Dow Jones Industrial Average falling 0.2% to 32,400. Analysts point to a mix of factors, including a decline in consumer spending and a rise in interest rates. However, experts note that the decline is relatively minor and within expected ranges, suggesting a stable market environment.'</li><li>"I'm not sure what to make of the latest earnings report from Apple. On one hand, revenue was up 10% from last quarter, but net income was only $4.17 per share, which is a decline from $4.65 per share in the same quarter last year. The guidance for the next quarter also seems a bit conservative. Anyone else have thoughts on this?"</li><li>"I'm not seeing any major red flags in the latest earnings report from Apple. The company's revenue growth is still on track, and the guidance for the next quarter looks solid. Of course, the tech sector is always volatile, but I think Apple's fundamentals are strong enough to weather any storm. Overall, I'm feeling neutral on this stock for now."</li></ul> |
89
+ | 0 | <ul><li>"Stocks plummeted after Intel Corp. (INTC) reported a disappointing earnings per share (EPS) of $0.85, missing the estimated $1.19, as the company cited weaker-than-expected demand for its chips. The news sent shockwaves through the tech sector, with Intel's shares tumbling 12% in after-hours trading."</li><li>"I'm getting increasingly concerned about the state of the economy. The latest GDP numbers are a clear indication that we're heading into a recession. The housing market is already showing signs of weakness and I'm starting to think that the Fed's interest rate hikes are going to have a much bigger impact than they're letting on. I'm bearish on the market and think we're in for a rough ride ahead."</li><li>'US inflation expectations surge to 3.4% as Fed officials warn of potential rate hikes. The Federal Reserve is expected to increase interest rates to combat rising prices, but economists warn that this could lead to a recession. The Bureau of Labor Statistics reported a 0.4% increase in consumer prices in January, with food and housing costs driving the gains. The inflation rate has been steadily rising over the past year, with the 12-month rate now at 2.5%. As a result, investors are bracing for a potential bear market, with the S&P 500 index already showing signs of weakness.'</li></ul> |
90
 
91
  ## Evaluation
92
 
93
  ### Metrics
94
  | Label | Accuracy |
95
  |:--------|:---------|
96
+ | **all** | 0.6227 |
97
 
98
  ## Uses
99
 
 
113
  # Download from the 🤗 Hub
114
  model = SetFitModel.from_pretrained("setfit_model_id")
115
  # Run inference
116
+ preds = model("Market volatility is expected to continue in the coming weeks, but overall, the trend remains stable. Investors are advised to remain cautious but not make any drastic decisions. #marketanalysis #trading")
117
  ```
118
 
119
  <!--
 
145
  ### Training Set Metrics
146
  | Training set | Min | Median | Max |
147
  |:-------------|:----|:--------|:----|
148
+ | Word count | 27 | 55.4340 | 98 |
149
 
150
  | Label | Training Sample Count |
151
  |:------|:----------------------|
152
  | 0 | 15 |
153
+ | 1 | 20 |
154
+ | 2 | 18 |
155
 
156
  ### Training Hyperparameters
157
  - batch_size: (16, 16)
 
171
  - load_best_model_at_end: True
172
 
173
  ### Training Results
174
+ | Epoch | Step | Training Loss | Validation Loss |
175
+ |:-------:|:-------:|:-------------:|:---------------:|
176
+ | 0.0085 | 1 | 0.2096 | - |
177
+ | 0.4274 | 50 | 0.0335 | - |
178
+ | 0.8547 | 100 | 0.0007 | - |
179
+ | 1.0 | 117 | - | 0.0071 |
180
+ | 1.2821 | 150 | 0.0002 | - |
181
+ | 1.7094 | 200 | 0.0002 | - |
182
+ | 2.0 | 234 | - | 0.005 |
183
+ | 2.1368 | 250 | 0.0001 | - |
184
+ | 2.5641 | 300 | 0.0002 | - |
185
+ | 2.9915 | 350 | 0.0002 | - |
186
+ | 3.0 | 351 | - | 0.0045 |
187
+ | 3.4188 | 400 | 0.0001 | - |
188
+ | 3.8462 | 450 | 0.0001 | - |
189
+ | 4.0 | 468 | - | 0.0038 |
190
+ | 4.2735 | 500 | 0.0001 | - |
191
+ | 4.7009 | 550 | 0.0001 | - |
192
+ | **5.0** | **585** | **-** | **0.0037** |
193
 
194
  * The bold row denotes the saved checkpoint.
195
  ### Framework Versions
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "setfit/step_60",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
 
1
  {
2
+ "_name_or_path": "setfit/step_585",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
config_setfit.json CHANGED
@@ -2,6 +2,7 @@
2
  "normalize_embeddings": false,
3
  "labels": [
4
  "0",
5
- "1"
 
6
  ]
7
  }
 
2
  "normalize_embeddings": false,
3
  "labels": [
4
  "0",
5
+ "1",
6
+ "2"
7
  ]
8
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f5ee038a99a6ba14f97dd1dd77d6f49ec51e874e04df59501bbb62356e6bcc13
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f0d905d2d7ef3da0b0858e85c48389565aaba79218c6afb2b6464052fa58d1e
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6ccd360d94dee05d3c16a122c3a00807deba92cecd9de424a4fb9b23b637eab5
3
- size 6991
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ae0bc7355dd2ba27882ddebd7cf1982e447d9c5f9e0d150f18eeb98a9e13f7f
3
+ size 19295