Hugging Face's logo

vipinbansal179
/
SetFit_sms_Analyzer5c95292

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results
Model card Files Community
vipinbansal179 commited on
Commit
c221563
·
1 Parent(s): f70f6ac

Add SetFit model

Browse files
Files changed (4) hide show
  1. README.md +74 -19
  2. config.json +1 -1
  3. model.safetensors +1 -1
  4. model_head.pkl +1 -1
README.md CHANGED
@@ -32,7 +32,7 @@ model-index:
32
  split: test
33
  metrics:
34
  - type: accuracy
35
- value: 0.6944444444444444
36
  name: Accuracy
37
  ---
38
 
@@ -64,18 +64,18 @@ The model has been trained using an efficient few-shot learning technique that i
64
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
65
 
66
  ### Model Labels
67
- | Label | Examples |
68
- |:------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
69
- | 2 | <ul><li>'hi ashwin , credit score qualify personal loan rs.10 lacs . apply get 10 mins https : //tneu.in/qilih6y4 & c apply - tataneu'</li><li>'592104 otp transaction inr 589.00 kotak bank card 6343 airtel , valid 3 mins . share otp anyone .'</li><li>'dear customer , get axis vistara credit card , complimentary economy ticket join & upto 4 ticket yearly , apply : https : //1kx.in/xrb8ksi6tkf loanmandi'</li></ul> |
70
- | 0 | <ul><li>"'ve spend rs.17220.00 thru kotak bank debit card xx2169 inf * www.vfsglobal.com 12/10/2023 avl bal 196168.62 ? visit kotak.com/fraud"</li><li>'rs 260.00 debit a/c xxxxxx7783 credit krjngm @ oksbi upi ref:325154274303. ? call 18005700 -bob'</li><li>'rs.381.56 transfer a/c ... 7783 : upi/327734499842 . total bal : rs.27615.37cr . avlbl amt : rs.27615.37 ( 04-10-2023 19:07:22 ) - bank baroda'</li></ul> |
71
- | 1 | <ul><li>'update ! inr5.66 credit federal bank account xxxx9374 jupiter app . happy bank !'</li><li>'dear customer , inr 619.00 credit a/c xx7556 31/08/2023 neft utr cms3535305583 fintech blue solutions private limit , info : fintech blue solutions private limi ted-sbi'</li></ul> |
72
 
73
  ## Evaluation
74
 
75
  ### Metrics
76
  | Label | Accuracy |
77
  |:--------|:---------|
78
- | **all** | 0.6944 |
79
 
80
  ## Uses
81
 
@@ -127,13 +127,13 @@ preds = model("< # > use otp : 8233 login turtlemintpro zck+rfoaqnm")
127
  ### Training Set Metrics
128
  | Training set | Min | Median | Max |
129
  |:-------------|:----|:--------|:----|
130
- | Word count | 14 | 22.0417 | 30 |
131
 
132
  | Label | Training Sample Count |
133
  |:------|:----------------------|
134
- | 0 | 6 |
135
- | 1 | 2 |
136
- | 2 | 16 |
137
 
138
  ### Training Hyperparameters
139
  - batch_size: (16, 16)
@@ -153,14 +153,69 @@ preds = model("< # > use otp : 8233 login turtlemintpro zck+rfoaqnm")
153
  - load_best_model_at_end: True
154
 
155
  ### Training Results
156
- | Epoch | Step | Training Loss | Validation Loss |
157
- |:-------:|:------:|:-------------:|:---------------:|
158
- | 0.05 | 1 | 0.279 | - |
159
- | **1.0** | **20** | **-** | **0.1602** |
160
- | 2.0 | 40 | - | 0.1603 |
161
- | 2.5 | 50 | 0.0013 | - |
162
- | 3.0 | 60 | - | 0.1661 |
163
- | 4.0 | 80 | - | 0.1706 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
164
 
165
  * The bold row denotes the saved checkpoint.
166
  ### Framework Versions
 
32
  split: test
33
  metrics:
34
  - type: accuracy
35
+ value: 0.9722222222222222
36
  name: Accuracy
37
  ---
38
 
 
64
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
65
 
66
  ### Model Labels
67
+ | Label | Examples |
68
+ |:------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
69
+ | 2 | <ul><li>'validity airtel xstream fiber id 20001896982 expire 04-sep-23 . please recharge rs 589 enjoy uninterrupted service . recharge , click www.airtel.in/5/c_summary ? n=021710937343_dsl . please ignore already pay .'</li><li>'initiate process add a/c . xxxx59 upi app - axis bank'</li><li>'google-pay registration initiate icici bank . do , report bank . card details/otp/cvv secret . disclose anyone .'</li></ul> |
70
+ | 0 | <ul><li>'rs 260.00 debit a/c xxxxxx7783 credit krjngm @ oksbi upi ref:325154274303. ? call 18005700 -bob'</li><li>'send rs.400.00 kotak bank ac x4524 7800600122 @ ybl 15-10-23.upi ref 328855774953. , kotak.com/fraud'</li><li>'send rs.400.00 kotak bank ac x4524 7800600122 @ ybl 15-10-23.upi ref 328855774953. , kotak.com/fraud'</li></ul> |
71
+ | 1 | <ul><li>'dear bob upi user , account credit inr 50.00 date 2023-08-27 11:41:09 upi ref 360562629741 - bob'</li><li>'receive rs.10000.00 kotak bank ac x4524 mahimagyamlani08 @ okaxis 21-08-23.bal:197,838.14.upi ref:323334598750'</li><li>'update ! inr5.66 credit federal bank account xxxx9374 jupiter app . happy bank !'</li></ul> |
72
 
73
  ## Evaluation
74
 
75
  ### Metrics
76
  | Label | Accuracy |
77
  |:--------|:---------|
78
+ | **all** | 0.9722 |
79
 
80
  ## Uses
81
 
 
127
  ### Training Set Metrics
128
  | Training set | Min | Median | Max |
129
  |:-------------|:----|:--------|:----|
130
+ | Word count | 5 | 20.5357 | 35 |
131
 
132
  | Label | Training Sample Count |
133
  |:------|:----------------------|
134
+ | 0 | 31 |
135
+ | 1 | 28 |
136
+ | 2 | 81 |
137
 
138
  ### Training Hyperparameters
139
  - batch_size: (16, 16)
 
153
  - load_best_model_at_end: True
154
 
155
  ### Training Results
156
+ | Epoch | Step | Training Loss | Validation Loss |
157
+ |:-------:|:-------:|:-------------:|:---------------:|
158
+ | 0.0014 | 1 | 0.2939 | - |
159
+ | 0.0708 | 50 | 0.1698 | - |
160
+ | 0.1416 | 100 | 0.0557 | - |
161
+ | 0.2125 | 150 | 0.0614 | - |
162
+ | 0.2833 | 200 | 0.0099 | - |
163
+ | 0.3541 | 250 | 0.0005 | - |
164
+ | 0.4249 | 300 | 0.0002 | - |
165
+ | 0.4958 | 350 | 0.0001 | - |
166
+ | 0.5666 | 400 | 0.0001 | - |
167
+ | 0.6374 | 450 | 0.0001 | - |
168
+ | 0.7082 | 500 | 0.0001 | - |
169
+ | 0.7790 | 550 | 0.0001 | - |
170
+ | 0.8499 | 600 | 0.0002 | - |
171
+ | 0.9207 | 650 | 0.0001 | - |
172
+ | 0.9915 | 700 | 0.0001 | - |
173
+ | **1.0** | **706** | **-** | **0.0312** |
174
+ | 1.0623 | 750 | 0.0001 | - |
175
+ | 1.1331 | 800 | 0.0001 | - |
176
+ | 1.2040 | 850 | 0.0001 | - |
177
+ | 1.2748 | 900 | 0.0 | - |
178
+ | 1.3456 | 950 | 0.0001 | - |
179
+ | 1.4164 | 1000 | 0.0 | - |
180
+ | 1.4873 | 1050 | 0.0 | - |
181
+ | 1.5581 | 1100 | 0.0 | - |
182
+ | 1.6289 | 1150 | 0.0 | - |
183
+ | 1.6997 | 1200 | 0.0 | - |
184
+ | 1.7705 | 1250 | 0.0 | - |
185
+ | 1.8414 | 1300 | 0.0001 | - |
186
+ | 1.9122 | 1350 | 0.0 | - |
187
+ | 1.9830 | 1400 | 0.0001 | - |
188
+ | 2.0 | 1412 | - | 0.0366 |
189
+ | 2.0538 | 1450 | 0.0 | - |
190
+ | 2.1246 | 1500 | 0.0001 | - |
191
+ | 2.1955 | 1550 | 0.0 | - |
192
+ | 2.2663 | 1600 | 0.0 | - |
193
+ | 2.3371 | 1650 | 0.0 | - |
194
+ | 2.4079 | 1700 | 0.0 | - |
195
+ | 2.4788 | 1750 | 0.0 | - |
196
+ | 2.5496 | 1800 | 0.0 | - |
197
+ | 2.6204 | 1850 | 0.0 | - |
198
+ | 2.6912 | 1900 | 0.0 | - |
199
+ | 2.7620 | 1950 | 0.0 | - |
200
+ | 2.8329 | 2000 | 0.0 | - |
201
+ | 2.9037 | 2050 | 0.0 | - |
202
+ | 2.9745 | 2100 | 0.0 | - |
203
+ | 3.0 | 2118 | - | 0.0414 |
204
+ | 3.0453 | 2150 | 0.0 | - |
205
+ | 3.1161 | 2200 | 0.0 | - |
206
+ | 3.1870 | 2250 | 0.0 | - |
207
+ | 3.2578 | 2300 | 0.0 | - |
208
+ | 3.3286 | 2350 | 0.0 | - |
209
+ | 3.3994 | 2400 | 0.0 | - |
210
+ | 3.4703 | 2450 | 0.0 | - |
211
+ | 3.5411 | 2500 | 0.0 | - |
212
+ | 3.6119 | 2550 | 0.0 | - |
213
+ | 3.6827 | 2600 | 0.0 | - |
214
+ | 3.7535 | 2650 | 0.0 | - |
215
+ | 3.8244 | 2700 | 0.0 | - |
216
+ | 3.8952 | 2750 | 0.0 | - |
217
+ | 3.9660 | 2800 | 0.0 | - |
218
+ | 4.0 | 2824 | - | 0.0366 |
219
 
220
  * The bold row denotes the saved checkpoint.
221
  ### Framework Versions
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "checkpoints/step_20/",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
 
1
  {
2
+ "_name_or_path": "checkpoints/step_706/",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fc686d4ee9727ab20a11e61fec42072f6a2026a08427a68d92658f5c1ae2e37d
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae1d3f676d3d0f6940308a2c6e3a4a29017117f3f4d8cbb26e23449f1b332589
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:84d61b69473b85170ab65e6b7e07704d6a59f5e03637221cab5346fa22a834fe
3
  size 19311
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa37013a63a0e558b7d22a1134f4b59a7652d5777b562408307eca4d2272eb9b
3
  size 19311