intfloat commited on
Commit
a504cb3
1 Parent(s): b4ee3b8

upload model

Browse files
README.md ADDED
@@ -0,0 +1,2656 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - mteb
4
+ model-index:
5
+ - name: e5-base
6
+ results:
7
+ - task:
8
+ type: Classification
9
+ dataset:
10
+ type: mteb/amazon_counterfactual
11
+ name: MTEB AmazonCounterfactualClassification (en)
12
+ config: en
13
+ split: test
14
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
15
+ metrics:
16
+ - type: accuracy
17
+ value: 79.71641791044777
18
+ - type: ap
19
+ value: 44.15426065428253
20
+ - type: f1
21
+ value: 73.89474407693241
22
+ - task:
23
+ type: Classification
24
+ dataset:
25
+ type: mteb/amazon_polarity
26
+ name: MTEB AmazonPolarityClassification
27
+ config: default
28
+ split: test
29
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
30
+ metrics:
31
+ - type: accuracy
32
+ value: 87.9649
33
+ - type: ap
34
+ value: 84.10171551915973
35
+ - type: f1
36
+ value: 87.94148377827356
37
+ - task:
38
+ type: Classification
39
+ dataset:
40
+ type: mteb/amazon_reviews_multi
41
+ name: MTEB AmazonReviewsClassification (en)
42
+ config: en
43
+ split: test
44
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
45
+ metrics:
46
+ - type: accuracy
47
+ value: 42.645999999999994
48
+ - type: f1
49
+ value: 42.230574673549
50
+ - task:
51
+ type: Retrieval
52
+ dataset:
53
+ type: arguana
54
+ name: MTEB ArguAna
55
+ config: default
56
+ split: test
57
+ revision: None
58
+ metrics:
59
+ - type: map_at_1
60
+ value: 26.814
61
+ - type: map_at_10
62
+ value: 42.681999999999995
63
+ - type: map_at_100
64
+ value: 43.714
65
+ - type: map_at_1000
66
+ value: 43.724000000000004
67
+ - type: map_at_3
68
+ value: 38.11
69
+ - type: map_at_5
70
+ value: 40.666999999999994
71
+ - type: mrr_at_1
72
+ value: 27.168999999999997
73
+ - type: mrr_at_10
74
+ value: 42.84
75
+ - type: mrr_at_100
76
+ value: 43.864
77
+ - type: mrr_at_1000
78
+ value: 43.875
79
+ - type: mrr_at_3
80
+ value: 38.193
81
+ - type: mrr_at_5
82
+ value: 40.793
83
+ - type: ndcg_at_1
84
+ value: 26.814
85
+ - type: ndcg_at_10
86
+ value: 51.410999999999994
87
+ - type: ndcg_at_100
88
+ value: 55.713
89
+ - type: ndcg_at_1000
90
+ value: 55.957
91
+ - type: ndcg_at_3
92
+ value: 41.955
93
+ - type: ndcg_at_5
94
+ value: 46.558
95
+ - type: precision_at_1
96
+ value: 26.814
97
+ - type: precision_at_10
98
+ value: 7.922999999999999
99
+ - type: precision_at_100
100
+ value: 0.9780000000000001
101
+ - type: precision_at_1000
102
+ value: 0.1
103
+ - type: precision_at_3
104
+ value: 17.71
105
+ - type: precision_at_5
106
+ value: 12.859000000000002
107
+ - type: recall_at_1
108
+ value: 26.814
109
+ - type: recall_at_10
110
+ value: 79.232
111
+ - type: recall_at_100
112
+ value: 97.795
113
+ - type: recall_at_1000
114
+ value: 99.644
115
+ - type: recall_at_3
116
+ value: 53.129000000000005
117
+ - type: recall_at_5
118
+ value: 64.29599999999999
119
+ - task:
120
+ type: Clustering
121
+ dataset:
122
+ type: mteb/arxiv-clustering-p2p
123
+ name: MTEB ArxivClusteringP2P
124
+ config: default
125
+ split: test
126
+ revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
127
+ metrics:
128
+ - type: v_measure
129
+ value: 44.56933066536439
130
+ - task:
131
+ type: Clustering
132
+ dataset:
133
+ type: mteb/arxiv-clustering-s2s
134
+ name: MTEB ArxivClusteringS2S
135
+ config: default
136
+ split: test
137
+ revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
138
+ metrics:
139
+ - type: v_measure
140
+ value: 40.47647746165173
141
+ - task:
142
+ type: Reranking
143
+ dataset:
144
+ type: mteb/askubuntudupquestions-reranking
145
+ name: MTEB AskUbuntuDupQuestions
146
+ config: default
147
+ split: test
148
+ revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
149
+ metrics:
150
+ - type: map
151
+ value: 59.65675531567043
152
+ - type: mrr
153
+ value: 72.95255683067317
154
+ - task:
155
+ type: STS
156
+ dataset:
157
+ type: mteb/biosses-sts
158
+ name: MTEB BIOSSES
159
+ config: default
160
+ split: test
161
+ revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
162
+ metrics:
163
+ - type: cos_sim_pearson
164
+ value: 85.83147014162338
165
+ - type: cos_sim_spearman
166
+ value: 85.1031439521441
167
+ - type: euclidean_pearson
168
+ value: 83.53609085510973
169
+ - type: euclidean_spearman
170
+ value: 84.59650590202833
171
+ - type: manhattan_pearson
172
+ value: 83.14611947586386
173
+ - type: manhattan_spearman
174
+ value: 84.13384475757064
175
+ - task:
176
+ type: Classification
177
+ dataset:
178
+ type: mteb/banking77
179
+ name: MTEB Banking77Classification
180
+ config: default
181
+ split: test
182
+ revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
183
+ metrics:
184
+ - type: accuracy
185
+ value: 83.32792207792208
186
+ - type: f1
187
+ value: 83.32037485050513
188
+ - task:
189
+ type: Clustering
190
+ dataset:
191
+ type: mteb/biorxiv-clustering-p2p
192
+ name: MTEB BiorxivClusteringP2P
193
+ config: default
194
+ split: test
195
+ revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
196
+ metrics:
197
+ - type: v_measure
198
+ value: 36.18605446588703
199
+ - task:
200
+ type: Clustering
201
+ dataset:
202
+ type: mteb/biorxiv-clustering-s2s
203
+ name: MTEB BiorxivClusteringS2S
204
+ config: default
205
+ split: test
206
+ revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
207
+ metrics:
208
+ - type: v_measure
209
+ value: 32.72379130181917
210
+ - task:
211
+ type: Retrieval
212
+ dataset:
213
+ type: BeIR/cqadupstack
214
+ name: MTEB CQADupstackAndroidRetrieval
215
+ config: default
216
+ split: test
217
+ revision: None
218
+ metrics:
219
+ - type: map_at_1
220
+ value: 30.659
221
+ - type: map_at_10
222
+ value: 40.333999999999996
223
+ - type: map_at_100
224
+ value: 41.763
225
+ - type: map_at_1000
226
+ value: 41.894
227
+ - type: map_at_3
228
+ value: 37.561
229
+ - type: map_at_5
230
+ value: 39.084
231
+ - type: mrr_at_1
232
+ value: 37.482
233
+ - type: mrr_at_10
234
+ value: 45.736
235
+ - type: mrr_at_100
236
+ value: 46.591
237
+ - type: mrr_at_1000
238
+ value: 46.644999999999996
239
+ - type: mrr_at_3
240
+ value: 43.491
241
+ - type: mrr_at_5
242
+ value: 44.75
243
+ - type: ndcg_at_1
244
+ value: 37.482
245
+ - type: ndcg_at_10
246
+ value: 45.606
247
+ - type: ndcg_at_100
248
+ value: 51.172
249
+ - type: ndcg_at_1000
250
+ value: 53.407000000000004
251
+ - type: ndcg_at_3
252
+ value: 41.808
253
+ - type: ndcg_at_5
254
+ value: 43.449
255
+ - type: precision_at_1
256
+ value: 37.482
257
+ - type: precision_at_10
258
+ value: 8.254999999999999
259
+ - type: precision_at_100
260
+ value: 1.3719999999999999
261
+ - type: precision_at_1000
262
+ value: 0.186
263
+ - type: precision_at_3
264
+ value: 19.695
265
+ - type: precision_at_5
266
+ value: 13.847999999999999
267
+ - type: recall_at_1
268
+ value: 30.659
269
+ - type: recall_at_10
270
+ value: 55.409
271
+ - type: recall_at_100
272
+ value: 78.687
273
+ - type: recall_at_1000
274
+ value: 93.068
275
+ - type: recall_at_3
276
+ value: 43.891999999999996
277
+ - type: recall_at_5
278
+ value: 48.678
279
+ - task:
280
+ type: Retrieval
281
+ dataset:
282
+ type: BeIR/cqadupstack
283
+ name: MTEB CQADupstackEnglishRetrieval
284
+ config: default
285
+ split: test
286
+ revision: None
287
+ metrics:
288
+ - type: map_at_1
289
+ value: 30.977
290
+ - type: map_at_10
291
+ value: 40.296
292
+ - type: map_at_100
293
+ value: 41.453
294
+ - type: map_at_1000
295
+ value: 41.581
296
+ - type: map_at_3
297
+ value: 37.619
298
+ - type: map_at_5
299
+ value: 39.181
300
+ - type: mrr_at_1
301
+ value: 39.108
302
+ - type: mrr_at_10
303
+ value: 46.894000000000005
304
+ - type: mrr_at_100
305
+ value: 47.55
306
+ - type: mrr_at_1000
307
+ value: 47.598
308
+ - type: mrr_at_3
309
+ value: 44.766
310
+ - type: mrr_at_5
311
+ value: 46.062999999999995
312
+ - type: ndcg_at_1
313
+ value: 39.108
314
+ - type: ndcg_at_10
315
+ value: 45.717
316
+ - type: ndcg_at_100
317
+ value: 49.941
318
+ - type: ndcg_at_1000
319
+ value: 52.138
320
+ - type: ndcg_at_3
321
+ value: 42.05
322
+ - type: ndcg_at_5
323
+ value: 43.893
324
+ - type: precision_at_1
325
+ value: 39.108
326
+ - type: precision_at_10
327
+ value: 8.306
328
+ - type: precision_at_100
329
+ value: 1.3419999999999999
330
+ - type: precision_at_1000
331
+ value: 0.184
332
+ - type: precision_at_3
333
+ value: 19.979
334
+ - type: precision_at_5
335
+ value: 14.038
336
+ - type: recall_at_1
337
+ value: 30.977
338
+ - type: recall_at_10
339
+ value: 54.688
340
+ - type: recall_at_100
341
+ value: 72.556
342
+ - type: recall_at_1000
343
+ value: 86.53800000000001
344
+ - type: recall_at_3
345
+ value: 43.388
346
+ - type: recall_at_5
347
+ value: 48.717
348
+ - task:
349
+ type: Retrieval
350
+ dataset:
351
+ type: BeIR/cqadupstack
352
+ name: MTEB CQADupstackGamingRetrieval
353
+ config: default
354
+ split: test
355
+ revision: None
356
+ metrics:
357
+ - type: map_at_1
358
+ value: 39.812
359
+ - type: map_at_10
360
+ value: 50.1
361
+ - type: map_at_100
362
+ value: 51.193999999999996
363
+ - type: map_at_1000
364
+ value: 51.258
365
+ - type: map_at_3
366
+ value: 47.510999999999996
367
+ - type: map_at_5
368
+ value: 48.891
369
+ - type: mrr_at_1
370
+ value: 45.266
371
+ - type: mrr_at_10
372
+ value: 53.459999999999994
373
+ - type: mrr_at_100
374
+ value: 54.19199999999999
375
+ - type: mrr_at_1000
376
+ value: 54.228
377
+ - type: mrr_at_3
378
+ value: 51.296
379
+ - type: mrr_at_5
380
+ value: 52.495999999999995
381
+ - type: ndcg_at_1
382
+ value: 45.266
383
+ - type: ndcg_at_10
384
+ value: 55.034000000000006
385
+ - type: ndcg_at_100
386
+ value: 59.458
387
+ - type: ndcg_at_1000
388
+ value: 60.862
389
+ - type: ndcg_at_3
390
+ value: 50.52799999999999
391
+ - type: ndcg_at_5
392
+ value: 52.564
393
+ - type: precision_at_1
394
+ value: 45.266
395
+ - type: precision_at_10
396
+ value: 8.483
397
+ - type: precision_at_100
398
+ value: 1.162
399
+ - type: precision_at_1000
400
+ value: 0.133
401
+ - type: precision_at_3
402
+ value: 21.944
403
+ - type: precision_at_5
404
+ value: 14.721
405
+ - type: recall_at_1
406
+ value: 39.812
407
+ - type: recall_at_10
408
+ value: 66.36
409
+ - type: recall_at_100
410
+ value: 85.392
411
+ - type: recall_at_1000
412
+ value: 95.523
413
+ - type: recall_at_3
414
+ value: 54.127
415
+ - type: recall_at_5
416
+ value: 59.245000000000005
417
+ - task:
418
+ type: Retrieval
419
+ dataset:
420
+ type: BeIR/cqadupstack
421
+ name: MTEB CQADupstackGisRetrieval
422
+ config: default
423
+ split: test
424
+ revision: None
425
+ metrics:
426
+ - type: map_at_1
427
+ value: 26.186
428
+ - type: map_at_10
429
+ value: 33.18
430
+ - type: map_at_100
431
+ value: 34.052
432
+ - type: map_at_1000
433
+ value: 34.149
434
+ - type: map_at_3
435
+ value: 31.029
436
+ - type: map_at_5
437
+ value: 32.321
438
+ - type: mrr_at_1
439
+ value: 28.136
440
+ - type: mrr_at_10
441
+ value: 35.195
442
+ - type: mrr_at_100
443
+ value: 35.996
444
+ - type: mrr_at_1000
445
+ value: 36.076
446
+ - type: mrr_at_3
447
+ value: 33.051
448
+ - type: mrr_at_5
449
+ value: 34.407
450
+ - type: ndcg_at_1
451
+ value: 28.136
452
+ - type: ndcg_at_10
453
+ value: 37.275999999999996
454
+ - type: ndcg_at_100
455
+ value: 41.935
456
+ - type: ndcg_at_1000
457
+ value: 44.389
458
+ - type: ndcg_at_3
459
+ value: 33.059
460
+ - type: ndcg_at_5
461
+ value: 35.313
462
+ - type: precision_at_1
463
+ value: 28.136
464
+ - type: precision_at_10
465
+ value: 5.457999999999999
466
+ - type: precision_at_100
467
+ value: 0.826
468
+ - type: precision_at_1000
469
+ value: 0.107
470
+ - type: precision_at_3
471
+ value: 13.522
472
+ - type: precision_at_5
473
+ value: 9.424000000000001
474
+ - type: recall_at_1
475
+ value: 26.186
476
+ - type: recall_at_10
477
+ value: 47.961999999999996
478
+ - type: recall_at_100
479
+ value: 70.072
480
+ - type: recall_at_1000
481
+ value: 88.505
482
+ - type: recall_at_3
483
+ value: 36.752
484
+ - type: recall_at_5
485
+ value: 42.168
486
+ - task:
487
+ type: Retrieval
488
+ dataset:
489
+ type: BeIR/cqadupstack
490
+ name: MTEB CQADupstackMathematicaRetrieval
491
+ config: default
492
+ split: test
493
+ revision: None
494
+ metrics:
495
+ - type: map_at_1
496
+ value: 16.586000000000002
497
+ - type: map_at_10
498
+ value: 23.637
499
+ - type: map_at_100
500
+ value: 24.82
501
+ - type: map_at_1000
502
+ value: 24.95
503
+ - type: map_at_3
504
+ value: 21.428
505
+ - type: map_at_5
506
+ value: 22.555
507
+ - type: mrr_at_1
508
+ value: 20.771
509
+ - type: mrr_at_10
510
+ value: 27.839999999999996
511
+ - type: mrr_at_100
512
+ value: 28.887
513
+ - type: mrr_at_1000
514
+ value: 28.967
515
+ - type: mrr_at_3
516
+ value: 25.56
517
+ - type: mrr_at_5
518
+ value: 26.723000000000003
519
+ - type: ndcg_at_1
520
+ value: 20.771
521
+ - type: ndcg_at_10
522
+ value: 28.255000000000003
523
+ - type: ndcg_at_100
524
+ value: 33.886
525
+ - type: ndcg_at_1000
526
+ value: 36.963
527
+ - type: ndcg_at_3
528
+ value: 24.056
529
+ - type: ndcg_at_5
530
+ value: 25.818
531
+ - type: precision_at_1
532
+ value: 20.771
533
+ - type: precision_at_10
534
+ value: 5.1
535
+ - type: precision_at_100
536
+ value: 0.9119999999999999
537
+ - type: precision_at_1000
538
+ value: 0.132
539
+ - type: precision_at_3
540
+ value: 11.526
541
+ - type: precision_at_5
542
+ value: 8.158999999999999
543
+ - type: recall_at_1
544
+ value: 16.586000000000002
545
+ - type: recall_at_10
546
+ value: 38.456
547
+ - type: recall_at_100
548
+ value: 62.666
549
+ - type: recall_at_1000
550
+ value: 84.47
551
+ - type: recall_at_3
552
+ value: 26.765
553
+ - type: recall_at_5
554
+ value: 31.297000000000004
555
+ - task:
556
+ type: Retrieval
557
+ dataset:
558
+ type: BeIR/cqadupstack
559
+ name: MTEB CQADupstackPhysicsRetrieval
560
+ config: default
561
+ split: test
562
+ revision: None
563
+ metrics:
564
+ - type: map_at_1
565
+ value: 28.831
566
+ - type: map_at_10
567
+ value: 37.545
568
+ - type: map_at_100
569
+ value: 38.934999999999995
570
+ - type: map_at_1000
571
+ value: 39.044000000000004
572
+ - type: map_at_3
573
+ value: 34.601
574
+ - type: map_at_5
575
+ value: 36.302
576
+ - type: mrr_at_1
577
+ value: 34.264
578
+ - type: mrr_at_10
579
+ value: 42.569
580
+ - type: mrr_at_100
581
+ value: 43.514
582
+ - type: mrr_at_1000
583
+ value: 43.561
584
+ - type: mrr_at_3
585
+ value: 40.167
586
+ - type: mrr_at_5
587
+ value: 41.678
588
+ - type: ndcg_at_1
589
+ value: 34.264
590
+ - type: ndcg_at_10
591
+ value: 42.914
592
+ - type: ndcg_at_100
593
+ value: 48.931999999999995
594
+ - type: ndcg_at_1000
595
+ value: 51.004000000000005
596
+ - type: ndcg_at_3
597
+ value: 38.096999999999994
598
+ - type: ndcg_at_5
599
+ value: 40.509
600
+ - type: precision_at_1
601
+ value: 34.264
602
+ - type: precision_at_10
603
+ value: 7.642
604
+ - type: precision_at_100
605
+ value: 1.258
606
+ - type: precision_at_1000
607
+ value: 0.161
608
+ - type: precision_at_3
609
+ value: 17.453
610
+ - type: precision_at_5
611
+ value: 12.608
612
+ - type: recall_at_1
613
+ value: 28.831
614
+ - type: recall_at_10
615
+ value: 53.56999999999999
616
+ - type: recall_at_100
617
+ value: 79.26100000000001
618
+ - type: recall_at_1000
619
+ value: 92.862
620
+ - type: recall_at_3
621
+ value: 40.681
622
+ - type: recall_at_5
623
+ value: 46.597
624
+ - task:
625
+ type: Retrieval
626
+ dataset:
627
+ type: BeIR/cqadupstack
628
+ name: MTEB CQADupstackProgrammersRetrieval
629
+ config: default
630
+ split: test
631
+ revision: None
632
+ metrics:
633
+ - type: map_at_1
634
+ value: 27.461000000000002
635
+ - type: map_at_10
636
+ value: 35.885
637
+ - type: map_at_100
638
+ value: 37.039
639
+ - type: map_at_1000
640
+ value: 37.16
641
+ - type: map_at_3
642
+ value: 33.451
643
+ - type: map_at_5
644
+ value: 34.807
645
+ - type: mrr_at_1
646
+ value: 34.018
647
+ - type: mrr_at_10
648
+ value: 41.32
649
+ - type: mrr_at_100
650
+ value: 42.157
651
+ - type: mrr_at_1000
652
+ value: 42.223
653
+ - type: mrr_at_3
654
+ value: 39.288000000000004
655
+ - type: mrr_at_5
656
+ value: 40.481
657
+ - type: ndcg_at_1
658
+ value: 34.018
659
+ - type: ndcg_at_10
660
+ value: 40.821000000000005
661
+ - type: ndcg_at_100
662
+ value: 46.053
663
+ - type: ndcg_at_1000
664
+ value: 48.673
665
+ - type: ndcg_at_3
666
+ value: 36.839
667
+ - type: ndcg_at_5
668
+ value: 38.683
669
+ - type: precision_at_1
670
+ value: 34.018
671
+ - type: precision_at_10
672
+ value: 7.009
673
+ - type: precision_at_100
674
+ value: 1.123
675
+ - type: precision_at_1000
676
+ value: 0.153
677
+ - type: precision_at_3
678
+ value: 16.933
679
+ - type: precision_at_5
680
+ value: 11.826
681
+ - type: recall_at_1
682
+ value: 27.461000000000002
683
+ - type: recall_at_10
684
+ value: 50.285000000000004
685
+ - type: recall_at_100
686
+ value: 73.25500000000001
687
+ - type: recall_at_1000
688
+ value: 91.17699999999999
689
+ - type: recall_at_3
690
+ value: 39.104
691
+ - type: recall_at_5
692
+ value: 43.968
693
+ - task:
694
+ type: Retrieval
695
+ dataset:
696
+ type: BeIR/cqadupstack
697
+ name: MTEB CQADupstackRetrieval
698
+ config: default
699
+ split: test
700
+ revision: None
701
+ metrics:
702
+ - type: map_at_1
703
+ value: 26.980083333333337
704
+ - type: map_at_10
705
+ value: 34.47208333333333
706
+ - type: map_at_100
707
+ value: 35.609249999999996
708
+ - type: map_at_1000
709
+ value: 35.72833333333333
710
+ - type: map_at_3
711
+ value: 32.189416666666666
712
+ - type: map_at_5
713
+ value: 33.44683333333334
714
+ - type: mrr_at_1
715
+ value: 31.731666666666662
716
+ - type: mrr_at_10
717
+ value: 38.518
718
+ - type: mrr_at_100
719
+ value: 39.38166666666667
720
+ - type: mrr_at_1000
721
+ value: 39.446999999999996
722
+ - type: mrr_at_3
723
+ value: 36.49966666666668
724
+ - type: mrr_at_5
725
+ value: 37.639916666666664
726
+ - type: ndcg_at_1
727
+ value: 31.731666666666662
728
+ - type: ndcg_at_10
729
+ value: 38.92033333333333
730
+ - type: ndcg_at_100
731
+ value: 44.01675
732
+ - type: ndcg_at_1000
733
+ value: 46.51075
734
+ - type: ndcg_at_3
735
+ value: 35.09766666666667
736
+ - type: ndcg_at_5
737
+ value: 36.842999999999996
738
+ - type: precision_at_1
739
+ value: 31.731666666666662
740
+ - type: precision_at_10
741
+ value: 6.472583333333332
742
+ - type: precision_at_100
743
+ value: 1.0665
744
+ - type: precision_at_1000
745
+ value: 0.14725000000000002
746
+ - type: precision_at_3
747
+ value: 15.659083333333331
748
+ - type: precision_at_5
749
+ value: 10.878833333333333
750
+ - type: recall_at_1
751
+ value: 26.980083333333337
752
+ - type: recall_at_10
753
+ value: 48.13925
754
+ - type: recall_at_100
755
+ value: 70.70149999999998
756
+ - type: recall_at_1000
757
+ value: 88.10775000000001
758
+ - type: recall_at_3
759
+ value: 37.30091666666667
760
+ - type: recall_at_5
761
+ value: 41.90358333333333
762
+ - task:
763
+ type: Retrieval
764
+ dataset:
765
+ type: BeIR/cqadupstack
766
+ name: MTEB CQADupstackStatsRetrieval
767
+ config: default
768
+ split: test
769
+ revision: None
770
+ metrics:
771
+ - type: map_at_1
772
+ value: 25.607999999999997
773
+ - type: map_at_10
774
+ value: 30.523
775
+ - type: map_at_100
776
+ value: 31.409
777
+ - type: map_at_1000
778
+ value: 31.507
779
+ - type: map_at_3
780
+ value: 28.915000000000003
781
+ - type: map_at_5
782
+ value: 29.756
783
+ - type: mrr_at_1
784
+ value: 28.681
785
+ - type: mrr_at_10
786
+ value: 33.409
787
+ - type: mrr_at_100
788
+ value: 34.241
789
+ - type: mrr_at_1000
790
+ value: 34.313
791
+ - type: mrr_at_3
792
+ value: 32.029999999999994
793
+ - type: mrr_at_5
794
+ value: 32.712
795
+ - type: ndcg_at_1
796
+ value: 28.681
797
+ - type: ndcg_at_10
798
+ value: 33.733000000000004
799
+ - type: ndcg_at_100
800
+ value: 38.32
801
+ - type: ndcg_at_1000
802
+ value: 40.937
803
+ - type: ndcg_at_3
804
+ value: 30.898999999999997
805
+ - type: ndcg_at_5
806
+ value: 32.088
807
+ - type: precision_at_1
808
+ value: 28.681
809
+ - type: precision_at_10
810
+ value: 4.968999999999999
811
+ - type: precision_at_100
812
+ value: 0.79
813
+ - type: precision_at_1000
814
+ value: 0.11
815
+ - type: precision_at_3
816
+ value: 12.73
817
+ - type: precision_at_5
818
+ value: 8.558
819
+ - type: recall_at_1
820
+ value: 25.607999999999997
821
+ - type: recall_at_10
822
+ value: 40.722
823
+ - type: recall_at_100
824
+ value: 61.956999999999994
825
+ - type: recall_at_1000
826
+ value: 81.43
827
+ - type: recall_at_3
828
+ value: 32.785
829
+ - type: recall_at_5
830
+ value: 35.855
831
+ - task:
832
+ type: Retrieval
833
+ dataset:
834
+ type: BeIR/cqadupstack
835
+ name: MTEB CQADupstackTexRetrieval
836
+ config: default
837
+ split: test
838
+ revision: None
839
+ metrics:
840
+ - type: map_at_1
841
+ value: 20.399
842
+ - type: map_at_10
843
+ value: 25.968000000000004
844
+ - type: map_at_100
845
+ value: 26.985999999999997
846
+ - type: map_at_1000
847
+ value: 27.105
848
+ - type: map_at_3
849
+ value: 24.215
850
+ - type: map_at_5
851
+ value: 25.157
852
+ - type: mrr_at_1
853
+ value: 24.708
854
+ - type: mrr_at_10
855
+ value: 29.971999999999998
856
+ - type: mrr_at_100
857
+ value: 30.858
858
+ - type: mrr_at_1000
859
+ value: 30.934
860
+ - type: mrr_at_3
861
+ value: 28.304000000000002
862
+ - type: mrr_at_5
863
+ value: 29.183999999999997
864
+ - type: ndcg_at_1
865
+ value: 24.708
866
+ - type: ndcg_at_10
867
+ value: 29.676000000000002
868
+ - type: ndcg_at_100
869
+ value: 34.656
870
+ - type: ndcg_at_1000
871
+ value: 37.588
872
+ - type: ndcg_at_3
873
+ value: 26.613
874
+ - type: ndcg_at_5
875
+ value: 27.919
876
+ - type: precision_at_1
877
+ value: 24.708
878
+ - type: precision_at_10
879
+ value: 5.01
880
+ - type: precision_at_100
881
+ value: 0.876
882
+ - type: precision_at_1000
883
+ value: 0.13
884
+ - type: precision_at_3
885
+ value: 11.975
886
+ - type: precision_at_5
887
+ value: 8.279
888
+ - type: recall_at_1
889
+ value: 20.399
890
+ - type: recall_at_10
891
+ value: 36.935
892
+ - type: recall_at_100
893
+ value: 59.532
894
+ - type: recall_at_1000
895
+ value: 80.58
896
+ - type: recall_at_3
897
+ value: 27.979
898
+ - type: recall_at_5
899
+ value: 31.636999999999997
900
+ - task:
901
+ type: Retrieval
902
+ dataset:
903
+ type: BeIR/cqadupstack
904
+ name: MTEB CQADupstackUnixRetrieval
905
+ config: default
906
+ split: test
907
+ revision: None
908
+ metrics:
909
+ - type: map_at_1
910
+ value: 27.606
911
+ - type: map_at_10
912
+ value: 34.213
913
+ - type: map_at_100
914
+ value: 35.339999999999996
915
+ - type: map_at_1000
916
+ value: 35.458
917
+ - type: map_at_3
918
+ value: 31.987
919
+ - type: map_at_5
920
+ value: 33.322
921
+ - type: mrr_at_1
922
+ value: 31.53
923
+ - type: mrr_at_10
924
+ value: 37.911
925
+ - type: mrr_at_100
926
+ value: 38.879000000000005
927
+ - type: mrr_at_1000
928
+ value: 38.956
929
+ - type: mrr_at_3
930
+ value: 35.868
931
+ - type: mrr_at_5
932
+ value: 37.047999999999995
933
+ - type: ndcg_at_1
934
+ value: 31.53
935
+ - type: ndcg_at_10
936
+ value: 38.312000000000005
937
+ - type: ndcg_at_100
938
+ value: 43.812
939
+ - type: ndcg_at_1000
940
+ value: 46.414
941
+ - type: ndcg_at_3
942
+ value: 34.319
943
+ - type: ndcg_at_5
944
+ value: 36.312
945
+ - type: precision_at_1
946
+ value: 31.53
947
+ - type: precision_at_10
948
+ value: 5.970000000000001
949
+ - type: precision_at_100
950
+ value: 0.9939999999999999
951
+ - type: precision_at_1000
952
+ value: 0.133
953
+ - type: precision_at_3
954
+ value: 14.738999999999999
955
+ - type: precision_at_5
956
+ value: 10.242999999999999
957
+ - type: recall_at_1
958
+ value: 27.606
959
+ - type: recall_at_10
960
+ value: 47.136
961
+ - type: recall_at_100
962
+ value: 71.253
963
+ - type: recall_at_1000
964
+ value: 89.39399999999999
965
+ - type: recall_at_3
966
+ value: 36.342
967
+ - type: recall_at_5
968
+ value: 41.388999999999996
969
+ - task:
970
+ type: Retrieval
971
+ dataset:
972
+ type: BeIR/cqadupstack
973
+ name: MTEB CQADupstackWebmastersRetrieval
974
+ config: default
975
+ split: test
976
+ revision: None
977
+ metrics:
978
+ - type: map_at_1
979
+ value: 24.855
980
+ - type: map_at_10
981
+ value: 31.963
982
+ - type: map_at_100
983
+ value: 33.371
984
+ - type: map_at_1000
985
+ value: 33.584
986
+ - type: map_at_3
987
+ value: 29.543999999999997
988
+ - type: map_at_5
989
+ value: 30.793
990
+ - type: mrr_at_1
991
+ value: 29.644
992
+ - type: mrr_at_10
993
+ value: 35.601
994
+ - type: mrr_at_100
995
+ value: 36.551
996
+ - type: mrr_at_1000
997
+ value: 36.623
998
+ - type: mrr_at_3
999
+ value: 33.399
1000
+ - type: mrr_at_5
1001
+ value: 34.575
1002
+ - type: ndcg_at_1
1003
+ value: 29.644
1004
+ - type: ndcg_at_10
1005
+ value: 36.521
1006
+ - type: ndcg_at_100
1007
+ value: 42.087
1008
+ - type: ndcg_at_1000
1009
+ value: 45.119
1010
+ - type: ndcg_at_3
1011
+ value: 32.797
1012
+ - type: ndcg_at_5
1013
+ value: 34.208
1014
+ - type: precision_at_1
1015
+ value: 29.644
1016
+ - type: precision_at_10
1017
+ value: 6.7
1018
+ - type: precision_at_100
1019
+ value: 1.374
1020
+ - type: precision_at_1000
1021
+ value: 0.22899999999999998
1022
+ - type: precision_at_3
1023
+ value: 15.152
1024
+ - type: precision_at_5
1025
+ value: 10.671999999999999
1026
+ - type: recall_at_1
1027
+ value: 24.855
1028
+ - type: recall_at_10
1029
+ value: 45.449
1030
+ - type: recall_at_100
1031
+ value: 70.921
1032
+ - type: recall_at_1000
1033
+ value: 90.629
1034
+ - type: recall_at_3
1035
+ value: 33.526
1036
+ - type: recall_at_5
1037
+ value: 37.848
1038
+ - task:
1039
+ type: Retrieval
1040
+ dataset:
1041
+ type: BeIR/cqadupstack
1042
+ name: MTEB CQADupstackWordpressRetrieval
1043
+ config: default
1044
+ split: test
1045
+ revision: None
1046
+ metrics:
1047
+ - type: map_at_1
1048
+ value: 24.781
1049
+ - type: map_at_10
1050
+ value: 30.020999999999997
1051
+ - type: map_at_100
1052
+ value: 30.948999999999998
1053
+ - type: map_at_1000
1054
+ value: 31.05
1055
+ - type: map_at_3
1056
+ value: 28.412
1057
+ - type: map_at_5
1058
+ value: 29.193
1059
+ - type: mrr_at_1
1060
+ value: 27.172
1061
+ - type: mrr_at_10
1062
+ value: 32.309
1063
+ - type: mrr_at_100
1064
+ value: 33.164
1065
+ - type: mrr_at_1000
1066
+ value: 33.239999999999995
1067
+ - type: mrr_at_3
1068
+ value: 30.775999999999996
1069
+ - type: mrr_at_5
1070
+ value: 31.562
1071
+ - type: ndcg_at_1
1072
+ value: 27.172
1073
+ - type: ndcg_at_10
1074
+ value: 33.178999999999995
1075
+ - type: ndcg_at_100
1076
+ value: 37.949
1077
+ - type: ndcg_at_1000
1078
+ value: 40.635
1079
+ - type: ndcg_at_3
1080
+ value: 30.107
1081
+ - type: ndcg_at_5
1082
+ value: 31.36
1083
+ - type: precision_at_1
1084
+ value: 27.172
1085
+ - type: precision_at_10
1086
+ value: 4.769
1087
+ - type: precision_at_100
1088
+ value: 0.769
1089
+ - type: precision_at_1000
1090
+ value: 0.109
1091
+ - type: precision_at_3
1092
+ value: 12.261
1093
+ - type: precision_at_5
1094
+ value: 8.17
1095
+ - type: recall_at_1
1096
+ value: 24.781
1097
+ - type: recall_at_10
1098
+ value: 40.699000000000005
1099
+ - type: recall_at_100
1100
+ value: 62.866
1101
+ - type: recall_at_1000
1102
+ value: 83.11699999999999
1103
+ - type: recall_at_3
1104
+ value: 32.269999999999996
1105
+ - type: recall_at_5
1106
+ value: 35.443999999999996
1107
+ - task:
1108
+ type: Retrieval
1109
+ dataset:
1110
+ type: climate-fever
1111
+ name: MTEB ClimateFEVER
1112
+ config: default
1113
+ split: test
1114
+ revision: None
1115
+ metrics:
1116
+ - type: map_at_1
1117
+ value: 5.2139999999999995
1118
+ - type: map_at_10
1119
+ value: 9.986
1120
+ - type: map_at_100
1121
+ value: 11.343
1122
+ - type: map_at_1000
1123
+ value: 11.55
1124
+ - type: map_at_3
1125
+ value: 7.961
1126
+ - type: map_at_5
1127
+ value: 8.967
1128
+ - type: mrr_at_1
1129
+ value: 12.052
1130
+ - type: mrr_at_10
1131
+ value: 20.165
1132
+ - type: mrr_at_100
1133
+ value: 21.317
1134
+ - type: mrr_at_1000
1135
+ value: 21.399
1136
+ - type: mrr_at_3
1137
+ value: 17.079
1138
+ - type: mrr_at_5
1139
+ value: 18.695
1140
+ - type: ndcg_at_1
1141
+ value: 12.052
1142
+ - type: ndcg_at_10
1143
+ value: 15.375
1144
+ - type: ndcg_at_100
1145
+ value: 21.858
1146
+ - type: ndcg_at_1000
1147
+ value: 26.145000000000003
1148
+ - type: ndcg_at_3
1149
+ value: 11.334
1150
+ - type: ndcg_at_5
1151
+ value: 12.798000000000002
1152
+ - type: precision_at_1
1153
+ value: 12.052
1154
+ - type: precision_at_10
1155
+ value: 5.16
1156
+ - type: precision_at_100
1157
+ value: 1.206
1158
+ - type: precision_at_1000
1159
+ value: 0.198
1160
+ - type: precision_at_3
1161
+ value: 8.73
1162
+ - type: precision_at_5
1163
+ value: 7.114
1164
+ - type: recall_at_1
1165
+ value: 5.2139999999999995
1166
+ - type: recall_at_10
1167
+ value: 20.669999999999998
1168
+ - type: recall_at_100
1169
+ value: 43.901
1170
+ - type: recall_at_1000
1171
+ value: 68.447
1172
+ - type: recall_at_3
1173
+ value: 11.049000000000001
1174
+ - type: recall_at_5
1175
+ value: 14.652999999999999
1176
+ - task:
1177
+ type: Retrieval
1178
+ dataset:
1179
+ type: dbpedia-entity
1180
+ name: MTEB DBPedia
1181
+ config: default
1182
+ split: test
1183
+ revision: None
1184
+ metrics:
1185
+ - type: map_at_1
1186
+ value: 8.511000000000001
1187
+ - type: map_at_10
1188
+ value: 19.503
1189
+ - type: map_at_100
1190
+ value: 27.46
1191
+ - type: map_at_1000
1192
+ value: 29.187
1193
+ - type: map_at_3
1194
+ value: 14.030999999999999
1195
+ - type: map_at_5
1196
+ value: 16.329
1197
+ - type: mrr_at_1
1198
+ value: 63.74999999999999
1199
+ - type: mrr_at_10
1200
+ value: 73.419
1201
+ - type: mrr_at_100
1202
+ value: 73.691
1203
+ - type: mrr_at_1000
1204
+ value: 73.697
1205
+ - type: mrr_at_3
1206
+ value: 71.792
1207
+ - type: mrr_at_5
1208
+ value: 72.979
1209
+ - type: ndcg_at_1
1210
+ value: 53.125
1211
+ - type: ndcg_at_10
1212
+ value: 41.02
1213
+ - type: ndcg_at_100
1214
+ value: 45.407
1215
+ - type: ndcg_at_1000
1216
+ value: 52.68000000000001
1217
+ - type: ndcg_at_3
1218
+ value: 46.088
1219
+ - type: ndcg_at_5
1220
+ value: 43.236000000000004
1221
+ - type: precision_at_1
1222
+ value: 63.74999999999999
1223
+ - type: precision_at_10
1224
+ value: 32.35
1225
+ - type: precision_at_100
1226
+ value: 10.363
1227
+ - type: precision_at_1000
1228
+ value: 2.18
1229
+ - type: precision_at_3
1230
+ value: 49.667
1231
+ - type: precision_at_5
1232
+ value: 41.5
1233
+ - type: recall_at_1
1234
+ value: 8.511000000000001
1235
+ - type: recall_at_10
1236
+ value: 24.851
1237
+ - type: recall_at_100
1238
+ value: 50.745
1239
+ - type: recall_at_1000
1240
+ value: 73.265
1241
+ - type: recall_at_3
1242
+ value: 15.716
1243
+ - type: recall_at_5
1244
+ value: 19.256
1245
+ - task:
1246
+ type: Classification
1247
+ dataset:
1248
+ type: mteb/emotion
1249
+ name: MTEB EmotionClassification
1250
+ config: default
1251
+ split: test
1252
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
1253
+ metrics:
1254
+ - type: accuracy
1255
+ value: 49.43500000000001
1256
+ - type: f1
1257
+ value: 44.56288273966374
1258
+ - task:
1259
+ type: Retrieval
1260
+ dataset:
1261
+ type: fever
1262
+ name: MTEB FEVER
1263
+ config: default
1264
+ split: test
1265
+ revision: None
1266
+ metrics:
1267
+ - type: map_at_1
1268
+ value: 40.858
1269
+ - type: map_at_10
1270
+ value: 52.276
1271
+ - type: map_at_100
1272
+ value: 52.928
1273
+ - type: map_at_1000
1274
+ value: 52.966
1275
+ - type: map_at_3
1276
+ value: 49.729
1277
+ - type: map_at_5
1278
+ value: 51.27
1279
+ - type: mrr_at_1
1280
+ value: 43.624
1281
+ - type: mrr_at_10
1282
+ value: 55.22899999999999
1283
+ - type: mrr_at_100
1284
+ value: 55.823
1285
+ - type: mrr_at_1000
1286
+ value: 55.85
1287
+ - type: mrr_at_3
1288
+ value: 52.739999999999995
1289
+ - type: mrr_at_5
1290
+ value: 54.251000000000005
1291
+ - type: ndcg_at_1
1292
+ value: 43.624
1293
+ - type: ndcg_at_10
1294
+ value: 58.23500000000001
1295
+ - type: ndcg_at_100
1296
+ value: 61.315
1297
+ - type: ndcg_at_1000
1298
+ value: 62.20099999999999
1299
+ - type: ndcg_at_3
1300
+ value: 53.22
1301
+ - type: ndcg_at_5
1302
+ value: 55.88999999999999
1303
+ - type: precision_at_1
1304
+ value: 43.624
1305
+ - type: precision_at_10
1306
+ value: 8.068999999999999
1307
+ - type: precision_at_100
1308
+ value: 0.975
1309
+ - type: precision_at_1000
1310
+ value: 0.107
1311
+ - type: precision_at_3
1312
+ value: 21.752
1313
+ - type: precision_at_5
1314
+ value: 14.515
1315
+ - type: recall_at_1
1316
+ value: 40.858
1317
+ - type: recall_at_10
1318
+ value: 73.744
1319
+ - type: recall_at_100
1320
+ value: 87.667
1321
+ - type: recall_at_1000
1322
+ value: 94.15599999999999
1323
+ - type: recall_at_3
1324
+ value: 60.287
1325
+ - type: recall_at_5
1326
+ value: 66.703
1327
+ - task:
1328
+ type: Retrieval
1329
+ dataset:
1330
+ type: fiqa
1331
+ name: MTEB FiQA2018
1332
+ config: default
1333
+ split: test
1334
+ revision: None
1335
+ metrics:
1336
+ - type: map_at_1
1337
+ value: 17.864
1338
+ - type: map_at_10
1339
+ value: 28.592000000000002
1340
+ - type: map_at_100
1341
+ value: 30.165
1342
+ - type: map_at_1000
1343
+ value: 30.364
1344
+ - type: map_at_3
1345
+ value: 24.586
1346
+ - type: map_at_5
1347
+ value: 26.717000000000002
1348
+ - type: mrr_at_1
1349
+ value: 35.031
1350
+ - type: mrr_at_10
1351
+ value: 43.876
1352
+ - type: mrr_at_100
1353
+ value: 44.683
1354
+ - type: mrr_at_1000
1355
+ value: 44.736
1356
+ - type: mrr_at_3
1357
+ value: 40.998000000000005
1358
+ - type: mrr_at_5
1359
+ value: 42.595
1360
+ - type: ndcg_at_1
1361
+ value: 35.031
1362
+ - type: ndcg_at_10
1363
+ value: 36.368
1364
+ - type: ndcg_at_100
1365
+ value: 42.472
1366
+ - type: ndcg_at_1000
1367
+ value: 45.973000000000006
1368
+ - type: ndcg_at_3
1369
+ value: 31.915
1370
+ - type: ndcg_at_5
1371
+ value: 33.394
1372
+ - type: precision_at_1
1373
+ value: 35.031
1374
+ - type: precision_at_10
1375
+ value: 10.139
1376
+ - type: precision_at_100
1377
+ value: 1.6420000000000001
1378
+ - type: precision_at_1000
1379
+ value: 0.22699999999999998
1380
+ - type: precision_at_3
1381
+ value: 21.142
1382
+ - type: precision_at_5
1383
+ value: 15.772
1384
+ - type: recall_at_1
1385
+ value: 17.864
1386
+ - type: recall_at_10
1387
+ value: 43.991
1388
+ - type: recall_at_100
1389
+ value: 66.796
1390
+ - type: recall_at_1000
1391
+ value: 87.64
1392
+ - type: recall_at_3
1393
+ value: 28.915999999999997
1394
+ - type: recall_at_5
1395
+ value: 35.185
1396
+ - task:
1397
+ type: Retrieval
1398
+ dataset:
1399
+ type: hotpotqa
1400
+ name: MTEB HotpotQA
1401
+ config: default
1402
+ split: test
1403
+ revision: None
1404
+ metrics:
1405
+ - type: map_at_1
1406
+ value: 36.556
1407
+ - type: map_at_10
1408
+ value: 53.056000000000004
1409
+ - type: map_at_100
1410
+ value: 53.909
1411
+ - type: map_at_1000
1412
+ value: 53.98
1413
+ - type: map_at_3
1414
+ value: 49.982
1415
+ - type: map_at_5
1416
+ value: 51.9
1417
+ - type: mrr_at_1
1418
+ value: 73.113
1419
+ - type: mrr_at_10
1420
+ value: 79.381
1421
+ - type: mrr_at_100
1422
+ value: 79.60300000000001
1423
+ - type: mrr_at_1000
1424
+ value: 79.617
1425
+ - type: mrr_at_3
1426
+ value: 78.298
1427
+ - type: mrr_at_5
1428
+ value: 78.995
1429
+ - type: ndcg_at_1
1430
+ value: 73.113
1431
+ - type: ndcg_at_10
1432
+ value: 62.21
1433
+ - type: ndcg_at_100
1434
+ value: 65.242
1435
+ - type: ndcg_at_1000
1436
+ value: 66.667
1437
+ - type: ndcg_at_3
1438
+ value: 57.717
1439
+ - type: ndcg_at_5
1440
+ value: 60.224
1441
+ - type: precision_at_1
1442
+ value: 73.113
1443
+ - type: precision_at_10
1444
+ value: 12.842999999999998
1445
+ - type: precision_at_100
1446
+ value: 1.522
1447
+ - type: precision_at_1000
1448
+ value: 0.17099999999999999
1449
+ - type: precision_at_3
1450
+ value: 36.178
1451
+ - type: precision_at_5
1452
+ value: 23.695
1453
+ - type: recall_at_1
1454
+ value: 36.556
1455
+ - type: recall_at_10
1456
+ value: 64.213
1457
+ - type: recall_at_100
1458
+ value: 76.077
1459
+ - type: recall_at_1000
1460
+ value: 85.53699999999999
1461
+ - type: recall_at_3
1462
+ value: 54.266999999999996
1463
+ - type: recall_at_5
1464
+ value: 59.236999999999995
1465
+ - task:
1466
+ type: Classification
1467
+ dataset:
1468
+ type: mteb/imdb
1469
+ name: MTEB ImdbClassification
1470
+ config: default
1471
+ split: test
1472
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
1473
+ metrics:
1474
+ - type: accuracy
1475
+ value: 75.958
1476
+ - type: ap
1477
+ value: 69.82869527654348
1478
+ - type: f1
1479
+ value: 75.89120903005633
1480
+ - task:
1481
+ type: Retrieval
1482
+ dataset:
1483
+ type: msmarco
1484
+ name: MTEB MSMARCO
1485
+ config: default
1486
+ split: dev
1487
+ revision: None
1488
+ metrics:
1489
+ - type: map_at_1
1490
+ value: 23.608
1491
+ - type: map_at_10
1492
+ value: 36.144
1493
+ - type: map_at_100
1494
+ value: 37.244
1495
+ - type: map_at_1000
1496
+ value: 37.291999999999994
1497
+ - type: map_at_3
1498
+ value: 32.287
1499
+ - type: map_at_5
1500
+ value: 34.473
1501
+ - type: mrr_at_1
1502
+ value: 24.226
1503
+ - type: mrr_at_10
1504
+ value: 36.711
1505
+ - type: mrr_at_100
1506
+ value: 37.758
1507
+ - type: mrr_at_1000
1508
+ value: 37.8
1509
+ - type: mrr_at_3
1510
+ value: 32.92
1511
+ - type: mrr_at_5
1512
+ value: 35.104
1513
+ - type: ndcg_at_1
1514
+ value: 24.269
1515
+ - type: ndcg_at_10
1516
+ value: 43.138
1517
+ - type: ndcg_at_100
1518
+ value: 48.421
1519
+ - type: ndcg_at_1000
1520
+ value: 49.592000000000006
1521
+ - type: ndcg_at_3
1522
+ value: 35.269
1523
+ - type: ndcg_at_5
1524
+ value: 39.175
1525
+ - type: precision_at_1
1526
+ value: 24.269
1527
+ - type: precision_at_10
1528
+ value: 6.755999999999999
1529
+ - type: precision_at_100
1530
+ value: 0.941
1531
+ - type: precision_at_1000
1532
+ value: 0.104
1533
+ - type: precision_at_3
1534
+ value: 14.938
1535
+ - type: precision_at_5
1536
+ value: 10.934000000000001
1537
+ - type: recall_at_1
1538
+ value: 23.608
1539
+ - type: recall_at_10
1540
+ value: 64.679
1541
+ - type: recall_at_100
1542
+ value: 89.027
1543
+ - type: recall_at_1000
1544
+ value: 97.91
1545
+ - type: recall_at_3
1546
+ value: 43.25
1547
+ - type: recall_at_5
1548
+ value: 52.617000000000004
1549
+ - task:
1550
+ type: Classification
1551
+ dataset:
1552
+ type: mteb/mtop_domain
1553
+ name: MTEB MTOPDomainClassification (en)
1554
+ config: en
1555
+ split: test
1556
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1557
+ metrics:
1558
+ - type: accuracy
1559
+ value: 93.21477428180576
1560
+ - type: f1
1561
+ value: 92.92502305092152
1562
+ - task:
1563
+ type: Classification
1564
+ dataset:
1565
+ type: mteb/mtop_intent
1566
+ name: MTEB MTOPIntentClassification (en)
1567
+ config: en
1568
+ split: test
1569
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1570
+ metrics:
1571
+ - type: accuracy
1572
+ value: 74.76744186046511
1573
+ - type: f1
1574
+ value: 59.19855520057899
1575
+ - task:
1576
+ type: Classification
1577
+ dataset:
1578
+ type: mteb/amazon_massive_intent
1579
+ name: MTEB MassiveIntentClassification (en)
1580
+ config: en
1581
+ split: test
1582
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1583
+ metrics:
1584
+ - type: accuracy
1585
+ value: 72.24613315400134
1586
+ - type: f1
1587
+ value: 70.19950395651232
1588
+ - task:
1589
+ type: Classification
1590
+ dataset:
1591
+ type: mteb/amazon_massive_scenario
1592
+ name: MTEB MassiveScenarioClassification (en)
1593
+ config: en
1594
+ split: test
1595
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
1596
+ metrics:
1597
+ - type: accuracy
1598
+ value: 76.75857431069268
1599
+ - type: f1
1600
+ value: 76.5433450230191
1601
+ - task:
1602
+ type: Clustering
1603
+ dataset:
1604
+ type: mteb/medrxiv-clustering-p2p
1605
+ name: MTEB MedrxivClusteringP2P
1606
+ config: default
1607
+ split: test
1608
+ revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
1609
+ metrics:
1610
+ - type: v_measure
1611
+ value: 31.525463791623604
1612
+ - task:
1613
+ type: Clustering
1614
+ dataset:
1615
+ type: mteb/medrxiv-clustering-s2s
1616
+ name: MTEB MedrxivClusteringS2S
1617
+ config: default
1618
+ split: test
1619
+ revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
1620
+ metrics:
1621
+ - type: v_measure
1622
+ value: 28.28695907385136
1623
+ - task:
1624
+ type: Reranking
1625
+ dataset:
1626
+ type: mteb/mind_small
1627
+ name: MTEB MindSmallReranking
1628
+ config: default
1629
+ split: test
1630
+ revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
1631
+ metrics:
1632
+ - type: map
1633
+ value: 30.068174046665224
1634
+ - type: mrr
1635
+ value: 30.827586642840803
1636
+ - task:
1637
+ type: Retrieval
1638
+ dataset:
1639
+ type: nfcorpus
1640
+ name: MTEB NFCorpus
1641
+ config: default
1642
+ split: test
1643
+ revision: None
1644
+ metrics:
1645
+ - type: map_at_1
1646
+ value: 6.322
1647
+ - type: map_at_10
1648
+ value: 13.919999999999998
1649
+ - type: map_at_100
1650
+ value: 17.416
1651
+ - type: map_at_1000
1652
+ value: 18.836
1653
+ - type: map_at_3
1654
+ value: 10.111
1655
+ - type: map_at_5
1656
+ value: 11.991999999999999
1657
+ - type: mrr_at_1
1658
+ value: 48.297000000000004
1659
+ - type: mrr_at_10
1660
+ value: 57.114
1661
+ - type: mrr_at_100
1662
+ value: 57.713
1663
+ - type: mrr_at_1000
1664
+ value: 57.751
1665
+ - type: mrr_at_3
1666
+ value: 55.108000000000004
1667
+ - type: mrr_at_5
1668
+ value: 56.533
1669
+ - type: ndcg_at_1
1670
+ value: 46.44
1671
+ - type: ndcg_at_10
1672
+ value: 36.589
1673
+ - type: ndcg_at_100
1674
+ value: 33.202
1675
+ - type: ndcg_at_1000
1676
+ value: 41.668
1677
+ - type: ndcg_at_3
1678
+ value: 41.302
1679
+ - type: ndcg_at_5
1680
+ value: 39.829
1681
+ - type: precision_at_1
1682
+ value: 47.988
1683
+ - type: precision_at_10
1684
+ value: 27.059
1685
+ - type: precision_at_100
1686
+ value: 8.235000000000001
1687
+ - type: precision_at_1000
1688
+ value: 2.091
1689
+ - type: precision_at_3
1690
+ value: 38.184000000000005
1691
+ - type: precision_at_5
1692
+ value: 34.365
1693
+ - type: recall_at_1
1694
+ value: 6.322
1695
+ - type: recall_at_10
1696
+ value: 18.288
1697
+ - type: recall_at_100
1698
+ value: 32.580999999999996
1699
+ - type: recall_at_1000
1700
+ value: 63.605999999999995
1701
+ - type: recall_at_3
1702
+ value: 11.266
1703
+ - type: recall_at_5
1704
+ value: 14.69
1705
+ - task:
1706
+ type: Retrieval
1707
+ dataset:
1708
+ type: nq
1709
+ name: MTEB NQ
1710
+ config: default
1711
+ split: test
1712
+ revision: None
1713
+ metrics:
1714
+ - type: map_at_1
1715
+ value: 36.586999999999996
1716
+ - type: map_at_10
1717
+ value: 52.464
1718
+ - type: map_at_100
1719
+ value: 53.384
1720
+ - type: map_at_1000
1721
+ value: 53.405
1722
+ - type: map_at_3
1723
+ value: 48.408
1724
+ - type: map_at_5
1725
+ value: 50.788999999999994
1726
+ - type: mrr_at_1
1727
+ value: 40.904
1728
+ - type: mrr_at_10
1729
+ value: 54.974000000000004
1730
+ - type: mrr_at_100
1731
+ value: 55.60699999999999
1732
+ - type: mrr_at_1000
1733
+ value: 55.623
1734
+ - type: mrr_at_3
1735
+ value: 51.73799999999999
1736
+ - type: mrr_at_5
1737
+ value: 53.638
1738
+ - type: ndcg_at_1
1739
+ value: 40.904
1740
+ - type: ndcg_at_10
1741
+ value: 59.965999999999994
1742
+ - type: ndcg_at_100
1743
+ value: 63.613
1744
+ - type: ndcg_at_1000
1745
+ value: 64.064
1746
+ - type: ndcg_at_3
1747
+ value: 52.486
1748
+ - type: ndcg_at_5
1749
+ value: 56.377
1750
+ - type: precision_at_1
1751
+ value: 40.904
1752
+ - type: precision_at_10
1753
+ value: 9.551
1754
+ - type: precision_at_100
1755
+ value: 1.162
1756
+ - type: precision_at_1000
1757
+ value: 0.12
1758
+ - type: precision_at_3
1759
+ value: 23.552
1760
+ - type: precision_at_5
1761
+ value: 16.436999999999998
1762
+ - type: recall_at_1
1763
+ value: 36.586999999999996
1764
+ - type: recall_at_10
1765
+ value: 80.094
1766
+ - type: recall_at_100
1767
+ value: 95.515
1768
+ - type: recall_at_1000
1769
+ value: 98.803
1770
+ - type: recall_at_3
1771
+ value: 60.907
1772
+ - type: recall_at_5
1773
+ value: 69.817
1774
+ - task:
1775
+ type: Retrieval
1776
+ dataset:
1777
+ type: quora
1778
+ name: MTEB QuoraRetrieval
1779
+ config: default
1780
+ split: test
1781
+ revision: None
1782
+ metrics:
1783
+ - type: map_at_1
1784
+ value: 70.422
1785
+ - type: map_at_10
1786
+ value: 84.113
1787
+ - type: map_at_100
1788
+ value: 84.744
1789
+ - type: map_at_1000
1790
+ value: 84.762
1791
+ - type: map_at_3
1792
+ value: 81.171
1793
+ - type: map_at_5
1794
+ value: 83.039
1795
+ - type: mrr_at_1
1796
+ value: 81.12
1797
+ - type: mrr_at_10
1798
+ value: 87.277
1799
+ - type: mrr_at_100
1800
+ value: 87.384
1801
+ - type: mrr_at_1000
1802
+ value: 87.385
1803
+ - type: mrr_at_3
1804
+ value: 86.315
1805
+ - type: mrr_at_5
1806
+ value: 86.981
1807
+ - type: ndcg_at_1
1808
+ value: 81.12
1809
+ - type: ndcg_at_10
1810
+ value: 87.92
1811
+ - type: ndcg_at_100
1812
+ value: 89.178
1813
+ - type: ndcg_at_1000
1814
+ value: 89.29899999999999
1815
+ - type: ndcg_at_3
1816
+ value: 85.076
1817
+ - type: ndcg_at_5
1818
+ value: 86.67099999999999
1819
+ - type: precision_at_1
1820
+ value: 81.12
1821
+ - type: precision_at_10
1822
+ value: 13.325999999999999
1823
+ - type: precision_at_100
1824
+ value: 1.524
1825
+ - type: precision_at_1000
1826
+ value: 0.157
1827
+ - type: precision_at_3
1828
+ value: 37.16
1829
+ - type: precision_at_5
1830
+ value: 24.456
1831
+ - type: recall_at_1
1832
+ value: 70.422
1833
+ - type: recall_at_10
1834
+ value: 95.00800000000001
1835
+ - type: recall_at_100
1836
+ value: 99.38
1837
+ - type: recall_at_1000
1838
+ value: 99.94800000000001
1839
+ - type: recall_at_3
1840
+ value: 86.809
1841
+ - type: recall_at_5
1842
+ value: 91.334
1843
+ - task:
1844
+ type: Clustering
1845
+ dataset:
1846
+ type: mteb/reddit-clustering
1847
+ name: MTEB RedditClustering
1848
+ config: default
1849
+ split: test
1850
+ revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
1851
+ metrics:
1852
+ - type: v_measure
1853
+ value: 48.18491891699636
1854
+ - task:
1855
+ type: Clustering
1856
+ dataset:
1857
+ type: mteb/reddit-clustering-p2p
1858
+ name: MTEB RedditClusteringP2P
1859
+ config: default
1860
+ split: test
1861
+ revision: 282350215ef01743dc01b456c7f5241fa8937f16
1862
+ metrics:
1863
+ - type: v_measure
1864
+ value: 62.190639679711914
1865
+ - task:
1866
+ type: Retrieval
1867
+ dataset:
1868
+ type: scidocs
1869
+ name: MTEB SCIDOCS
1870
+ config: default
1871
+ split: test
1872
+ revision: None
1873
+ metrics:
1874
+ - type: map_at_1
1875
+ value: 4.478
1876
+ - type: map_at_10
1877
+ value: 11.268
1878
+ - type: map_at_100
1879
+ value: 13.129
1880
+ - type: map_at_1000
1881
+ value: 13.41
1882
+ - type: map_at_3
1883
+ value: 8.103
1884
+ - type: map_at_5
1885
+ value: 9.609
1886
+ - type: mrr_at_1
1887
+ value: 22.0
1888
+ - type: mrr_at_10
1889
+ value: 32.248
1890
+ - type: mrr_at_100
1891
+ value: 33.355000000000004
1892
+ - type: mrr_at_1000
1893
+ value: 33.42
1894
+ - type: mrr_at_3
1895
+ value: 29.15
1896
+ - type: mrr_at_5
1897
+ value: 30.785
1898
+ - type: ndcg_at_1
1899
+ value: 22.0
1900
+ - type: ndcg_at_10
1901
+ value: 18.990000000000002
1902
+ - type: ndcg_at_100
1903
+ value: 26.302999999999997
1904
+ - type: ndcg_at_1000
1905
+ value: 31.537
1906
+ - type: ndcg_at_3
1907
+ value: 18.034
1908
+ - type: ndcg_at_5
1909
+ value: 15.655
1910
+ - type: precision_at_1
1911
+ value: 22.0
1912
+ - type: precision_at_10
1913
+ value: 9.91
1914
+ - type: precision_at_100
1915
+ value: 2.0420000000000003
1916
+ - type: precision_at_1000
1917
+ value: 0.33
1918
+ - type: precision_at_3
1919
+ value: 16.933
1920
+ - type: precision_at_5
1921
+ value: 13.719999999999999
1922
+ - type: recall_at_1
1923
+ value: 4.478
1924
+ - type: recall_at_10
1925
+ value: 20.087
1926
+ - type: recall_at_100
1927
+ value: 41.457
1928
+ - type: recall_at_1000
1929
+ value: 67.10199999999999
1930
+ - type: recall_at_3
1931
+ value: 10.313
1932
+ - type: recall_at_5
1933
+ value: 13.927999999999999
1934
+ - task:
1935
+ type: STS
1936
+ dataset:
1937
+ type: mteb/sickr-sts
1938
+ name: MTEB SICK-R
1939
+ config: default
1940
+ split: test
1941
+ revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
1942
+ metrics:
1943
+ - type: cos_sim_pearson
1944
+ value: 84.27341574565806
1945
+ - type: cos_sim_spearman
1946
+ value: 79.66419880841734
1947
+ - type: euclidean_pearson
1948
+ value: 81.32473321838208
1949
+ - type: euclidean_spearman
1950
+ value: 79.29828832085133
1951
+ - type: manhattan_pearson
1952
+ value: 81.25554065883132
1953
+ - type: manhattan_spearman
1954
+ value: 79.23275543279853
1955
+ - task:
1956
+ type: STS
1957
+ dataset:
1958
+ type: mteb/sts12-sts
1959
+ name: MTEB STS12
1960
+ config: default
1961
+ split: test
1962
+ revision: a0d554a64d88156834ff5ae9920b964011b16384
1963
+ metrics:
1964
+ - type: cos_sim_pearson
1965
+ value: 83.40468875905418
1966
+ - type: cos_sim_spearman
1967
+ value: 74.2189990321174
1968
+ - type: euclidean_pearson
1969
+ value: 80.74376966290956
1970
+ - type: euclidean_spearman
1971
+ value: 74.97663839079335
1972
+ - type: manhattan_pearson
1973
+ value: 80.69779331646207
1974
+ - type: manhattan_spearman
1975
+ value: 75.00225252917613
1976
+ - task:
1977
+ type: STS
1978
+ dataset:
1979
+ type: mteb/sts13-sts
1980
+ name: MTEB STS13
1981
+ config: default
1982
+ split: test
1983
+ revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
1984
+ metrics:
1985
+ - type: cos_sim_pearson
1986
+ value: 82.5745290053095
1987
+ - type: cos_sim_spearman
1988
+ value: 83.31401180333397
1989
+ - type: euclidean_pearson
1990
+ value: 82.96500607325534
1991
+ - type: euclidean_spearman
1992
+ value: 83.8534967935793
1993
+ - type: manhattan_pearson
1994
+ value: 82.83112050632508
1995
+ - type: manhattan_spearman
1996
+ value: 83.70877296557838
1997
+ - task:
1998
+ type: STS
1999
+ dataset:
2000
+ type: mteb/sts14-sts
2001
+ name: MTEB STS14
2002
+ config: default
2003
+ split: test
2004
+ revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
2005
+ metrics:
2006
+ - type: cos_sim_pearson
2007
+ value: 80.67833656607704
2008
+ - type: cos_sim_spearman
2009
+ value: 78.52252410630707
2010
+ - type: euclidean_pearson
2011
+ value: 80.071189514343
2012
+ - type: euclidean_spearman
2013
+ value: 78.95143545742796
2014
+ - type: manhattan_pearson
2015
+ value: 80.0128926165121
2016
+ - type: manhattan_spearman
2017
+ value: 78.91236678732628
2018
+ - task:
2019
+ type: STS
2020
+ dataset:
2021
+ type: mteb/sts15-sts
2022
+ name: MTEB STS15
2023
+ config: default
2024
+ split: test
2025
+ revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
2026
+ metrics:
2027
+ - type: cos_sim_pearson
2028
+ value: 87.48437639980746
2029
+ - type: cos_sim_spearman
2030
+ value: 88.34876527774259
2031
+ - type: euclidean_pearson
2032
+ value: 87.64898081823888
2033
+ - type: euclidean_spearman
2034
+ value: 88.58937180804213
2035
+ - type: manhattan_pearson
2036
+ value: 87.5942417815288
2037
+ - type: manhattan_spearman
2038
+ value: 88.53013922267687
2039
+ - task:
2040
+ type: STS
2041
+ dataset:
2042
+ type: mteb/sts16-sts
2043
+ name: MTEB STS16
2044
+ config: default
2045
+ split: test
2046
+ revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
2047
+ metrics:
2048
+ - type: cos_sim_pearson
2049
+ value: 82.69189187164781
2050
+ - type: cos_sim_spearman
2051
+ value: 84.15327883572112
2052
+ - type: euclidean_pearson
2053
+ value: 83.64202266685898
2054
+ - type: euclidean_spearman
2055
+ value: 84.6219602318862
2056
+ - type: manhattan_pearson
2057
+ value: 83.53256698709998
2058
+ - type: manhattan_spearman
2059
+ value: 84.49260712904946
2060
+ - task:
2061
+ type: STS
2062
+ dataset:
2063
+ type: mteb/sts17-crosslingual-sts
2064
+ name: MTEB STS17 (en-en)
2065
+ config: en-en
2066
+ split: test
2067
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2068
+ metrics:
2069
+ - type: cos_sim_pearson
2070
+ value: 87.09508017611589
2071
+ - type: cos_sim_spearman
2072
+ value: 87.23010990417097
2073
+ - type: euclidean_pearson
2074
+ value: 87.62545569077133
2075
+ - type: euclidean_spearman
2076
+ value: 86.71152051711714
2077
+ - type: manhattan_pearson
2078
+ value: 87.5057154278377
2079
+ - type: manhattan_spearman
2080
+ value: 86.60611898281267
2081
+ - task:
2082
+ type: STS
2083
+ dataset:
2084
+ type: mteb/sts22-crosslingual-sts
2085
+ name: MTEB STS22 (en)
2086
+ config: en
2087
+ split: test
2088
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
2089
+ metrics:
2090
+ - type: cos_sim_pearson
2091
+ value: 61.72129893941176
2092
+ - type: cos_sim_spearman
2093
+ value: 62.87871412069194
2094
+ - type: euclidean_pearson
2095
+ value: 63.21077648290454
2096
+ - type: euclidean_spearman
2097
+ value: 63.03263080805978
2098
+ - type: manhattan_pearson
2099
+ value: 63.20740860135976
2100
+ - type: manhattan_spearman
2101
+ value: 62.89930471802817
2102
+ - task:
2103
+ type: STS
2104
+ dataset:
2105
+ type: mteb/stsbenchmark-sts
2106
+ name: MTEB STSBenchmark
2107
+ config: default
2108
+ split: test
2109
+ revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
2110
+ metrics:
2111
+ - type: cos_sim_pearson
2112
+ value: 85.039118236799
2113
+ - type: cos_sim_spearman
2114
+ value: 86.18102563389962
2115
+ - type: euclidean_pearson
2116
+ value: 85.62977041471879
2117
+ - type: euclidean_spearman
2118
+ value: 86.02478990544347
2119
+ - type: manhattan_pearson
2120
+ value: 85.60786740521806
2121
+ - type: manhattan_spearman
2122
+ value: 85.99546210442547
2123
+ - task:
2124
+ type: Reranking
2125
+ dataset:
2126
+ type: mteb/scidocs-reranking
2127
+ name: MTEB SciDocsRR
2128
+ config: default
2129
+ split: test
2130
+ revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
2131
+ metrics:
2132
+ - type: map
2133
+ value: 82.89875069737266
2134
+ - type: mrr
2135
+ value: 95.42621322033087
2136
+ - task:
2137
+ type: Retrieval
2138
+ dataset:
2139
+ type: scifact
2140
+ name: MTEB SciFact
2141
+ config: default
2142
+ split: test
2143
+ revision: None
2144
+ metrics:
2145
+ - type: map_at_1
2146
+ value: 58.660999999999994
2147
+ - type: map_at_10
2148
+ value: 68.738
2149
+ - type: map_at_100
2150
+ value: 69.33200000000001
2151
+ - type: map_at_1000
2152
+ value: 69.352
2153
+ - type: map_at_3
2154
+ value: 66.502
2155
+ - type: map_at_5
2156
+ value: 67.686
2157
+ - type: mrr_at_1
2158
+ value: 61.667
2159
+ - type: mrr_at_10
2160
+ value: 70.003
2161
+ - type: mrr_at_100
2162
+ value: 70.441
2163
+ - type: mrr_at_1000
2164
+ value: 70.46
2165
+ - type: mrr_at_3
2166
+ value: 68.278
2167
+ - type: mrr_at_5
2168
+ value: 69.194
2169
+ - type: ndcg_at_1
2170
+ value: 61.667
2171
+ - type: ndcg_at_10
2172
+ value: 73.083
2173
+ - type: ndcg_at_100
2174
+ value: 75.56
2175
+ - type: ndcg_at_1000
2176
+ value: 76.01400000000001
2177
+ - type: ndcg_at_3
2178
+ value: 69.28699999999999
2179
+ - type: ndcg_at_5
2180
+ value: 70.85000000000001
2181
+ - type: precision_at_1
2182
+ value: 61.667
2183
+ - type: precision_at_10
2184
+ value: 9.6
2185
+ - type: precision_at_100
2186
+ value: 1.087
2187
+ - type: precision_at_1000
2188
+ value: 0.11199999999999999
2189
+ - type: precision_at_3
2190
+ value: 27.111
2191
+ - type: precision_at_5
2192
+ value: 17.467
2193
+ - type: recall_at_1
2194
+ value: 58.660999999999994
2195
+ - type: recall_at_10
2196
+ value: 85.02199999999999
2197
+ - type: recall_at_100
2198
+ value: 95.933
2199
+ - type: recall_at_1000
2200
+ value: 99.333
2201
+ - type: recall_at_3
2202
+ value: 74.506
2203
+ - type: recall_at_5
2204
+ value: 78.583
2205
+ - task:
2206
+ type: PairClassification
2207
+ dataset:
2208
+ type: mteb/sprintduplicatequestions-pairclassification
2209
+ name: MTEB SprintDuplicateQuestions
2210
+ config: default
2211
+ split: test
2212
+ revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
2213
+ metrics:
2214
+ - type: cos_sim_accuracy
2215
+ value: 99.8029702970297
2216
+ - type: cos_sim_ap
2217
+ value: 94.87673936635738
2218
+ - type: cos_sim_f1
2219
+ value: 90.00502260170768
2220
+ - type: cos_sim_precision
2221
+ value: 90.41372351160445
2222
+ - type: cos_sim_recall
2223
+ value: 89.60000000000001
2224
+ - type: dot_accuracy
2225
+ value: 99.57524752475247
2226
+ - type: dot_ap
2227
+ value: 84.81717934496321
2228
+ - type: dot_f1
2229
+ value: 78.23026646556059
2230
+ - type: dot_precision
2231
+ value: 78.66531850353893
2232
+ - type: dot_recall
2233
+ value: 77.8
2234
+ - type: euclidean_accuracy
2235
+ value: 99.8029702970297
2236
+ - type: euclidean_ap
2237
+ value: 94.74658253135284
2238
+ - type: euclidean_f1
2239
+ value: 90.08470353761834
2240
+ - type: euclidean_precision
2241
+ value: 89.77159880834161
2242
+ - type: euclidean_recall
2243
+ value: 90.4
2244
+ - type: manhattan_accuracy
2245
+ value: 99.8
2246
+ - type: manhattan_ap
2247
+ value: 94.69224030742787
2248
+ - type: manhattan_f1
2249
+ value: 89.9502487562189
2250
+ - type: manhattan_precision
2251
+ value: 89.50495049504951
2252
+ - type: manhattan_recall
2253
+ value: 90.4
2254
+ - type: max_accuracy
2255
+ value: 99.8029702970297
2256
+ - type: max_ap
2257
+ value: 94.87673936635738
2258
+ - type: max_f1
2259
+ value: 90.08470353761834
2260
+ - task:
2261
+ type: Clustering
2262
+ dataset:
2263
+ type: mteb/stackexchange-clustering
2264
+ name: MTEB StackExchangeClustering
2265
+ config: default
2266
+ split: test
2267
+ revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
2268
+ metrics:
2269
+ - type: v_measure
2270
+ value: 63.906039623153035
2271
+ - task:
2272
+ type: Clustering
2273
+ dataset:
2274
+ type: mteb/stackexchange-clustering-p2p
2275
+ name: MTEB StackExchangeClusteringP2P
2276
+ config: default
2277
+ split: test
2278
+ revision: 815ca46b2622cec33ccafc3735d572c266efdb44
2279
+ metrics:
2280
+ - type: v_measure
2281
+ value: 32.56053830923281
2282
+ - task:
2283
+ type: Reranking
2284
+ dataset:
2285
+ type: mteb/stackoverflowdupquestions-reranking
2286
+ name: MTEB StackOverflowDupQuestions
2287
+ config: default
2288
+ split: test
2289
+ revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
2290
+ metrics:
2291
+ - type: map
2292
+ value: 50.15326538775145
2293
+ - type: mrr
2294
+ value: 50.99279295051355
2295
+ - task:
2296
+ type: Summarization
2297
+ dataset:
2298
+ type: mteb/summeval
2299
+ name: MTEB SummEval
2300
+ config: default
2301
+ split: test
2302
+ revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
2303
+ metrics:
2304
+ - type: cos_sim_pearson
2305
+ value: 22.23040797767245
2306
+ - type: cos_sim_spearman
2307
+ value: 26.03794260145079
2308
+ - type: dot_pearson
2309
+ value: 24.01892207887181
2310
+ - type: dot_spearman
2311
+ value: 25.234879514149057
2312
+ - task:
2313
+ type: Retrieval
2314
+ dataset:
2315
+ type: trec-covid
2316
+ name: MTEB TRECCOVID
2317
+ config: default
2318
+ split: test
2319
+ revision: None
2320
+ metrics:
2321
+ - type: map_at_1
2322
+ value: 0.247
2323
+ - type: map_at_10
2324
+ value: 1.9429999999999998
2325
+ - type: map_at_100
2326
+ value: 10.82
2327
+ - type: map_at_1000
2328
+ value: 25.972
2329
+ - type: map_at_3
2330
+ value: 0.653
2331
+ - type: map_at_5
2332
+ value: 1.057
2333
+ - type: mrr_at_1
2334
+ value: 94.0
2335
+ - type: mrr_at_10
2336
+ value: 96.333
2337
+ - type: mrr_at_100
2338
+ value: 96.333
2339
+ - type: mrr_at_1000
2340
+ value: 96.333
2341
+ - type: mrr_at_3
2342
+ value: 96.333
2343
+ - type: mrr_at_5
2344
+ value: 96.333
2345
+ - type: ndcg_at_1
2346
+ value: 89.0
2347
+ - type: ndcg_at_10
2348
+ value: 79.63799999999999
2349
+ - type: ndcg_at_100
2350
+ value: 57.961
2351
+ - type: ndcg_at_1000
2352
+ value: 50.733
2353
+ - type: ndcg_at_3
2354
+ value: 84.224
2355
+ - type: ndcg_at_5
2356
+ value: 82.528
2357
+ - type: precision_at_1
2358
+ value: 94.0
2359
+ - type: precision_at_10
2360
+ value: 84.2
2361
+ - type: precision_at_100
2362
+ value: 59.36
2363
+ - type: precision_at_1000
2364
+ value: 22.738
2365
+ - type: precision_at_3
2366
+ value: 88.0
2367
+ - type: precision_at_5
2368
+ value: 86.8
2369
+ - type: recall_at_1
2370
+ value: 0.247
2371
+ - type: recall_at_10
2372
+ value: 2.131
2373
+ - type: recall_at_100
2374
+ value: 14.035
2375
+ - type: recall_at_1000
2376
+ value: 47.457
2377
+ - type: recall_at_3
2378
+ value: 0.6779999999999999
2379
+ - type: recall_at_5
2380
+ value: 1.124
2381
+ - task:
2382
+ type: Retrieval
2383
+ dataset:
2384
+ type: webis-touche2020
2385
+ name: MTEB Touche2020
2386
+ config: default
2387
+ split: test
2388
+ revision: None
2389
+ metrics:
2390
+ - type: map_at_1
2391
+ value: 2.603
2392
+ - type: map_at_10
2393
+ value: 11.667
2394
+ - type: map_at_100
2395
+ value: 16.474
2396
+ - type: map_at_1000
2397
+ value: 18.074
2398
+ - type: map_at_3
2399
+ value: 6.03
2400
+ - type: map_at_5
2401
+ value: 8.067
2402
+ - type: mrr_at_1
2403
+ value: 34.694
2404
+ - type: mrr_at_10
2405
+ value: 51.063
2406
+ - type: mrr_at_100
2407
+ value: 51.908
2408
+ - type: mrr_at_1000
2409
+ value: 51.908
2410
+ - type: mrr_at_3
2411
+ value: 47.959
2412
+ - type: mrr_at_5
2413
+ value: 49.694
2414
+ - type: ndcg_at_1
2415
+ value: 32.653
2416
+ - type: ndcg_at_10
2417
+ value: 28.305000000000003
2418
+ - type: ndcg_at_100
2419
+ value: 35.311
2420
+ - type: ndcg_at_1000
2421
+ value: 47.644999999999996
2422
+ - type: ndcg_at_3
2423
+ value: 32.187
2424
+ - type: ndcg_at_5
2425
+ value: 29.134999999999998
2426
+ - type: precision_at_1
2427
+ value: 34.694
2428
+ - type: precision_at_10
2429
+ value: 26.122
2430
+ - type: precision_at_100
2431
+ value: 6.755
2432
+ - type: precision_at_1000
2433
+ value: 1.467
2434
+ - type: precision_at_3
2435
+ value: 34.694
2436
+ - type: precision_at_5
2437
+ value: 30.203999999999997
2438
+ - type: recall_at_1
2439
+ value: 2.603
2440
+ - type: recall_at_10
2441
+ value: 18.716
2442
+ - type: recall_at_100
2443
+ value: 42.512
2444
+ - type: recall_at_1000
2445
+ value: 79.32000000000001
2446
+ - type: recall_at_3
2447
+ value: 7.59
2448
+ - type: recall_at_5
2449
+ value: 10.949
2450
+ - task:
2451
+ type: Classification
2452
+ dataset:
2453
+ type: mteb/toxic_conversations_50k
2454
+ name: MTEB ToxicConversationsClassification
2455
+ config: default
2456
+ split: test
2457
+ revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
2458
+ metrics:
2459
+ - type: accuracy
2460
+ value: 74.117
2461
+ - type: ap
2462
+ value: 15.89357321699319
2463
+ - type: f1
2464
+ value: 57.14385866369257
2465
+ - task:
2466
+ type: Classification
2467
+ dataset:
2468
+ type: mteb/tweet_sentiment_extraction
2469
+ name: MTEB TweetSentimentExtractionClassification
2470
+ config: default
2471
+ split: test
2472
+ revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
2473
+ metrics:
2474
+ - type: accuracy
2475
+ value: 61.38370118845502
2476
+ - type: f1
2477
+ value: 61.67038693866553
2478
+ - task:
2479
+ type: Clustering
2480
+ dataset:
2481
+ type: mteb/twentynewsgroups-clustering
2482
+ name: MTEB TwentyNewsgroupsClustering
2483
+ config: default
2484
+ split: test
2485
+ revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
2486
+ metrics:
2487
+ - type: v_measure
2488
+ value: 42.57754941537969
2489
+ - task:
2490
+ type: PairClassification
2491
+ dataset:
2492
+ type: mteb/twittersemeval2015-pairclassification
2493
+ name: MTEB TwitterSemEval2015
2494
+ config: default
2495
+ split: test
2496
+ revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
2497
+ metrics:
2498
+ - type: cos_sim_accuracy
2499
+ value: 86.1775049174465
2500
+ - type: cos_sim_ap
2501
+ value: 74.3994879581554
2502
+ - type: cos_sim_f1
2503
+ value: 69.32903671308551
2504
+ - type: cos_sim_precision
2505
+ value: 61.48193508879363
2506
+ - type: cos_sim_recall
2507
+ value: 79.47229551451187
2508
+ - type: dot_accuracy
2509
+ value: 81.65345413363534
2510
+ - type: dot_ap
2511
+ value: 59.690898346685096
2512
+ - type: dot_f1
2513
+ value: 57.27622826467499
2514
+ - type: dot_precision
2515
+ value: 51.34965473948525
2516
+ - type: dot_recall
2517
+ value: 64.74934036939314
2518
+ - type: euclidean_accuracy
2519
+ value: 86.04637301066937
2520
+ - type: euclidean_ap
2521
+ value: 74.33009001775268
2522
+ - type: euclidean_f1
2523
+ value: 69.2458374142997
2524
+ - type: euclidean_precision
2525
+ value: 64.59570580173595
2526
+ - type: euclidean_recall
2527
+ value: 74.6174142480211
2528
+ - type: manhattan_accuracy
2529
+ value: 86.11193896405793
2530
+ - type: manhattan_ap
2531
+ value: 74.2964140130421
2532
+ - type: manhattan_f1
2533
+ value: 69.11601528788066
2534
+ - type: manhattan_precision
2535
+ value: 64.86924323073363
2536
+ - type: manhattan_recall
2537
+ value: 73.95778364116094
2538
+ - type: max_accuracy
2539
+ value: 86.1775049174465
2540
+ - type: max_ap
2541
+ value: 74.3994879581554
2542
+ - type: max_f1
2543
+ value: 69.32903671308551
2544
+ - task:
2545
+ type: PairClassification
2546
+ dataset:
2547
+ type: mteb/twitterurlcorpus-pairclassification
2548
+ name: MTEB TwitterURLCorpus
2549
+ config: default
2550
+ split: test
2551
+ revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
2552
+ metrics:
2553
+ - type: cos_sim_accuracy
2554
+ value: 89.01501921061823
2555
+ - type: cos_sim_ap
2556
+ value: 85.97819287477351
2557
+ - type: cos_sim_f1
2558
+ value: 78.33882858518875
2559
+ - type: cos_sim_precision
2560
+ value: 75.49446626204926
2561
+ - type: cos_sim_recall
2562
+ value: 81.40591315060055
2563
+ - type: dot_accuracy
2564
+ value: 86.47494857763806
2565
+ - type: dot_ap
2566
+ value: 78.77420360340282
2567
+ - type: dot_f1
2568
+ value: 73.06433247936238
2569
+ - type: dot_precision
2570
+ value: 67.92140777983595
2571
+ - type: dot_recall
2572
+ value: 79.04989220819218
2573
+ - type: euclidean_accuracy
2574
+ value: 88.7297706368611
2575
+ - type: euclidean_ap
2576
+ value: 85.61550568529317
2577
+ - type: euclidean_f1
2578
+ value: 77.84805525263539
2579
+ - type: euclidean_precision
2580
+ value: 73.73639994491117
2581
+ - type: euclidean_recall
2582
+ value: 82.44533415460425
2583
+ - type: manhattan_accuracy
2584
+ value: 88.75111576823068
2585
+ - type: manhattan_ap
2586
+ value: 85.58701671476263
2587
+ - type: manhattan_f1
2588
+ value: 77.70169909067856
2589
+ - type: manhattan_precision
2590
+ value: 73.37666780704755
2591
+ - type: manhattan_recall
2592
+ value: 82.5685247921158
2593
+ - type: max_accuracy
2594
+ value: 89.01501921061823
2595
+ - type: max_ap
2596
+ value: 85.97819287477351
2597
+ - type: max_f1
2598
+ value: 78.33882858518875
2599
+ ---
2600
+
2601
+ ## E5-base
2602
+
2603
+ [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://arxiv.org/pdf/2212.03533.pdf).
2604
+ Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022
2605
+
2606
+ This model has 12 layers and the embedding size is 768.
2607
+
2608
+ ## Usage
2609
+
2610
+ Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset.
2611
+
2612
+ ```python
2613
+ import torch.nn.functional as F
2614
+
2615
+ from torch import Tensor
2616
+ from transformers import AutoTokenizer, AutoModel
2617
+ from transformers.modeling_outputs import BaseModelOutput
2618
+
2619
+
2620
+ def average_pool(last_hidden_states: Tensor,
2621
+ attention_mask: Tensor) -> Tensor:
2622
+ last_hidden = last_hidden_states.masked_fill(~attention_mask[..., None].bool(), 0.0)
2623
+ return last_hidden.sum(dim=1) / attention_mask.sum(dim=1)[..., None]
2624
+
2625
+
2626
+ # Each input text should start with "query: " or "passage: ".
2627
+ # For tasks other than retrieval, you can simply use the "query: " prefix.
2628
+ input_texts = ['query: how much protein should a female eat',
2629
+ 'query: summit define',
2630
+ "passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
2631
+ "passage: Definition of summit for English Language Learners. : 1 the highest point of a mountain : the top of a mountain. : 2 the highest level. : 3 a meeting or series of meetings between the leaders of two or more governments."]
2632
+
2633
+ tokenizer = AutoTokenizer.from_pretrained('intfloat/e5-base')
2634
+ model = AutoModel.from_pretrained('intfloat/e5-base')
2635
+
2636
+ # Tokenize the input texts
2637
+ batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt')
2638
+
2639
+ outputs: BaseModelOutput = model(**batch_dict)
2640
+ embeddings = average_pool(outputs.last_hidden_state, batch_dict['attention_mask'])
2641
+
2642
+ # (Optionally) normalize embeddings
2643
+ embeddings = F.normalize(embeddings, p=2, dim=1)
2644
+ scores = (embeddings[:2] @ embeddings[2:].T) * 100
2645
+ print(scores.tolist())
2646
+ ```
2647
+
2648
+ ## Training Details
2649
+
2650
+ Please refer to our paper at [https://arxiv.org/pdf/2212.03533.pdf](https://arxiv.org/pdf/2212.03533.pdf).
2651
+
2652
+ ## Benchmark Evaluation
2653
+
2654
+ Check out [unilm/e5](https://github.com/microsoft/unilm/tree/master/e5) to reproduce evaluation results
2655
+ on the [BEIR](https://arxiv.org/abs/2104.08663) and [MTEB benchmark](https://arxiv.org/abs/2210.07316).
2656
+
config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "tmp/",
3
+ "architectures": [
4
+ "BertModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "layer_norm_eps": 1e-12,
15
+ "max_position_embeddings": 512,
16
+ "model_type": "bert",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 12,
19
+ "pad_token_id": 0,
20
+ "position_embedding_type": "absolute",
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.15.0",
23
+ "type_vocab_size": 2,
24
+ "use_cache": true,
25
+ "vocab_size": 30522
26
+ }
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd623a40c8b841b7c99a464e32e6629d19935a52d123d1ebda7b26606b5de637
3
+ size 438007537
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "model_max_length": 512, "special_tokens_map_file": null, "name_or_path": "amlt/1031_add_qd_prompt_ft_random_swap_nli/all_kd_ft", "tokenizer_class": "BertTokenizer"}
vocab.txt ADDED
The diff for this file is too large to render. See raw diff