Hugging Face's logo

neuralmagic
/
bge-base-en-v1.5-sparse

Feature Extraction
Transformers
ONNX
English
bert
mteb
sparse sparsity quantized onnx embeddings int8
Eval Results
text-embeddings-inference
Inference Endpoints
Model card Files Community
1
zeroshot commited on
Commit
5def9b9
•
1 Parent(s): 42bde5c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +167 -0
README.md CHANGED
@@ -4,6 +4,173 @@ language:
4
  - en
5
  tags:
6
  - sparse sparsity quantized onnx embeddings int8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  ---
8
  This is the sparsified ONNX variant of the [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) embeddings model created with [DeepSparse Optimum](https://github.com/neuralmagic/optimum-deepsparse) for ONNX export/inference pipeline and Neural Magic's [Sparsify](https://github.com/neuralmagic/sparsify) for one-shot quantization (INT8) and unstructured pruning (50%).
9
 
 
4
  - en
5
  tags:
6
  - sparse sparsity quantized onnx embeddings int8
7
+ model-index:
8
+ - name: bge-base-en-v1.5-sparse
9
+ results:
10
+ - task:
11
+ type: Classification
12
+ dataset:
13
+ type: mteb/amazon_counterfactual
14
+ name: MTEB AmazonCounterfactualClassification (en)
15
+ config: en
16
+ split: test
17
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
18
+ metrics:
19
+ - type: accuracy
20
+ value: 75.38805970149254
21
+ - type: ap
22
+ value: 38.80643435437097
23
+ - type: f1
24
+ value: 69.52906891019036
25
+ - task:
26
+ type: Classification
27
+ dataset:
28
+ type: mteb/amazon_polarity
29
+ name: MTEB AmazonPolarityClassification
30
+ config: default
31
+ split: test
32
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
33
+ metrics:
34
+ - type: accuracy
35
+ value: 90.72759999999998
36
+ - type: ap
37
+ value: 87.07910150764239
38
+ - type: f1
39
+ value: 90.71025910882096
40
+ - task:
41
+ type: Classification
42
+ dataset:
43
+ type: mteb/amazon_reviews_multi
44
+ name: MTEB AmazonReviewsClassification (en)
45
+ config: en
46
+ split: test
47
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
48
+ metrics:
49
+ - type: accuracy
50
+ value: 45.494
51
+ - type: f1
52
+ value: 44.917953161904805
53
+ - task:
54
+ type: Classification
55
+ dataset:
56
+ type: mteb/banking77
57
+ name: MTEB Banking77Classification
58
+ config: default
59
+ split: test
60
+ revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
61
+ metrics:
62
+ - type: accuracy
63
+ value: 84.29545454545455
64
+ - type: f1
65
+ value: 84.26780483160312
66
+ - task:
67
+ type: Classification
68
+ dataset:
69
+ type: mteb/emotion
70
+ name: MTEB EmotionClassification
71
+ config: default
72
+ split: test
73
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
74
+ metrics:
75
+ - type: accuracy
76
+ value: 46.705
77
+ - type: f1
78
+ value: 41.82618717355017
79
+ - task:
80
+ type: Classification
81
+ dataset:
82
+ type: mteb/imdb
83
+ name: MTEB ImdbClassification
84
+ config: default
85
+ split: test
86
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
87
+ metrics:
88
+ - type: accuracy
89
+ value: 83.14760000000001
90
+ - type: ap
91
+ value: 77.40813245635195
92
+ - type: f1
93
+ value: 83.08648833100911
94
+ - task:
95
+ type: Classification
96
+ dataset:
97
+ type: mteb/mtop_domain
98
+ name: MTEB MTOPDomainClassification (en)
99
+ config: en
100
+ split: test
101
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
102
+ metrics:
103
+ - type: accuracy
104
+ value: 92.0519835841313
105
+ - type: f1
106
+ value: 91.73392170858916
107
+ - task:
108
+ type: Classification
109
+ dataset:
110
+ type: mteb/mtop_intent
111
+ name: MTEB MTOPIntentClassification (en)
112
+ config: en
113
+ split: test
114
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
115
+ metrics:
116
+ - type: accuracy
117
+ value: 72.48974008207935
118
+ - type: f1
119
+ value: 54.812872972777505
120
+ - task:
121
+ type: Classification
122
+ dataset:
123
+ type: mteb/amazon_massive_intent
124
+ name: MTEB MassiveIntentClassification (en)
125
+ config: en
126
+ split: test
127
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
128
+ metrics:
129
+ - type: accuracy
130
+ value: 73.17753866846
131
+ - type: f1
132
+ value: 71.51091282373878
133
+ - task:
134
+ type: Classification
135
+ dataset:
136
+ type: mteb/amazon_massive_scenario
137
+ name: MTEB MassiveScenarioClassification (en)
138
+ config: en
139
+ split: test
140
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
141
+ metrics:
142
+ - type: accuracy
143
+ value: 77.5353059852051
144
+ - type: f1
145
+ value: 77.42427561340143
146
+ - task:
147
+ type: Classification
148
+ dataset:
149
+ type: mteb/toxic_conversations_50k
150
+ name: MTEB ToxicConversationsClassification
151
+ config: default
152
+ split: test
153
+ revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
154
+ metrics:
155
+ - type: accuracy
156
+ value: 70.917
157
+ - type: ap
158
+ value: 13.760770628090576
159
+ - type: f1
160
+ value: 54.23887489664618
161
+ - task:
162
+ type: Classification
163
+ dataset:
164
+ type: mteb/tweet_sentiment_extraction
165
+ name: MTEB TweetSentimentExtractionClassification
166
+ config: default
167
+ split: test
168
+ revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
169
+ metrics:
170
+ - type: accuracy
171
+ value: 59.49349179400113
172
+ - type: f1
173
+ value: 59.815392064510775
174
  ---
175
  This is the sparsified ONNX variant of the [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) embeddings model created with [DeepSparse Optimum](https://github.com/neuralmagic/optimum-deepsparse) for ONNX export/inference pipeline and Neural Magic's [Sparsify](https://github.com/neuralmagic/sparsify) for one-shot quantization (INT8) and unstructured pruning (50%).
176