Hugging Face's logo

omarelshehy
/
Arabic-Retrieval-v1.0

Sentence Similarity
sentence-transformers
Safetensors
Arabic
bert
feature-extraction
Generated from Trainer
loss:MultipleNegativesRankingLoss
retrieval
mteb
Eval Results
text-embeddings-inference
Inference Endpoints
Model card Files Community
omarelshehy commited on
Commit
b02e7cc
·
verified ·
1 Parent(s): 1e1cf33

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +268 -0
README.md CHANGED
@@ -12,6 +12,274 @@ language:
12
  - ar
13
  base_model:
14
  - aubmindlab/bert-base-arabertv02
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  ---
16
 
17
 
 
12
  - ar
13
  base_model:
14
  - aubmindlab/bert-base-arabertv02
15
+
16
+ model-index:
17
+ - name: omarelshehy/Arabic-Retrieval-v1.0
18
+ results:
19
+ - dataset:
20
+ config: ar
21
+ name: MTEB MIRACLRetrieval (ar)
22
+ revision: main
23
+ split: dev
24
+ type: miracl/mmteb-miracl
25
+ metrics:
26
+ - type: main_score
27
+ value: 58.664
28
+ - type: map_at_1
29
+ value: 32.399
30
+ - type: map_at_10
31
+ value: 50.236000000000004
32
+ - type: map_at_100
33
+ value: 51.87199999999999
34
+ - type: map_at_1000
35
+ value: 51.926
36
+ - type: ndcg_at_1
37
+ value: 48.376999999999995
38
+ - type: ndcg_at_10
39
+ value: 58.664
40
+ - type: ndcg_at_100
41
+ value: 63.754999999999995
42
+ - type: ndcg_at_1000
43
+ value: 64.672
44
+ - type: ndcg_at_20
45
+ value: 61.111000000000004
46
+ - type: ndcg_at_3
47
+ value: 51.266
48
+ - type: ndcg_at_5
49
+ value: 54.529
50
+ task:
51
+ type: Retrieval
52
+ - dataset:
53
+ config: ar
54
+ name: MTEB MIRACLRetrievalHardNegatives (ar)
55
+ revision: 95c8db7d4a6e9c1d8a60601afd63d553ae20a2eb
56
+ split: dev
57
+ type: mteb/miracl-hard-negatives
58
+ metrics:
59
+ - type: main_score
60
+ value: 60.026
61
+ - type: map_at_1
62
+ value: 32.547
63
+ - type: map_at_10
64
+ value: 51.345
65
+ - type: map_at_100
66
+ value: 53.190000000000005
67
+ - type: map_at_1000
68
+ value: 53.237
69
+ - type: ndcg_at_1
70
+ value: 48.3
71
+ - type: ndcg_at_10
72
+ value: 60.026
73
+ - type: ndcg_at_100
74
+ value: 65.62400000000001
75
+ - type: ndcg_at_1000
76
+ value: 66.282
77
+ - type: ndcg_at_20
78
+ value: 62.856
79
+ - type: ndcg_at_3
80
+ value: 52.1
81
+ - type: ndcg_at_5
82
+ value: 55.627
83
+ task:
84
+ type: Retrieval
85
+ - dataset:
86
+ config: ara-ara
87
+ name: MTEB MLQARetrieval (ara-ara)
88
+ revision: 397ed406c1a7902140303e7faf60fff35b58d285
89
+ split: test
90
+ type: facebook/mlqa
91
+ metrics:
92
+ - type: main_score
93
+ value: 56.032000000000004
94
+ - type: map_at_1
95
+ value: 45.218
96
+ - type: map_at_10
97
+ value: 52.32599999999999
98
+ - type: map_at_100
99
+ value: 53.001
100
+ - type: map_at_1000
101
+ value: 53.047999999999995
102
+ - type: ndcg_at_1
103
+ value: 45.228
104
+ - type: ndcg_at_10
105
+ value: 56.032000000000004
106
+ - type: ndcg_at_100
107
+ value: 59.486000000000004
108
+ - type: ndcg_at_1000
109
+ value: 60.938
110
+ - type: ndcg_at_20
111
+ value: 57.507
112
+ - type: ndcg_at_3
113
+ value: 52.05800000000001
114
+ - type: ndcg_at_5
115
+ value: 54.005
116
+ task:
117
+ type: Retrieval
118
+ - dataset:
119
+ config: ara-ara
120
+ name: MTEB MLQARetrieval (ara-ara)
121
+ revision: 397ed406c1a7902140303e7faf60fff35b58d285
122
+ split: validation
123
+ type: facebook/mlqa
124
+ metrics:
125
+ - type: main_score
126
+ value: 71.11
127
+ - type: map_at_1
128
+ value: 58.221000000000004
129
+ - type: map_at_10
130
+ value: 67.089
131
+ - type: map_at_100
132
+ value: 67.62700000000001
133
+ - type: map_at_1000
134
+ value: 67.648
135
+ - type: ndcg_at_1
136
+ value: 58.221000000000004
137
+ - type: ndcg_at_10
138
+ value: 71.11
139
+ - type: ndcg_at_100
140
+ value: 73.824
141
+ - type: ndcg_at_1000
142
+ value: 74.292
143
+ - type: ndcg_at_20
144
+ value: 72.381
145
+ - type: ndcg_at_3
146
+ value: 67.472
147
+ - type: ndcg_at_5
148
+ value: 69.803
149
+ task:
150
+ type: Retrieval
151
+ - dataset:
152
+ config: ar
153
+ name: MTEB MintakaRetrieval (ar)
154
+ revision: efa78cc2f74bbcd21eff2261f9e13aebe40b814e
155
+ split: test
156
+ type: jinaai/mintakaqa
157
+ metrics:
158
+ - type: main_score
159
+ value: 22.778000000000002
160
+ - type: map_at_1
161
+ value: 13.345
162
+ - type: map_at_10
163
+ value: 19.336000000000002
164
+ - type: map_at_100
165
+ value: 20.116999999999997
166
+ - type: map_at_1000
167
+ value: 20.246
168
+ - type: ndcg_at_1
169
+ value: 13.345
170
+ - type: ndcg_at_10
171
+ value: 22.778000000000002
172
+ - type: ndcg_at_100
173
+ value: 26.997
174
+ - type: ndcg_at_1000
175
+ value: 31.564999999999998
176
+ - type: ndcg_at_20
177
+ value: 24.368000000000002
178
+ - type: ndcg_at_3
179
+ value: 18.622
180
+ - type: ndcg_at_5
181
+ value: 20.72
182
+ task:
183
+ type: Retrieval
184
+ - dataset:
185
+ config: arabic
186
+ name: MTEB MrTidyRetrieval (arabic)
187
+ revision: fc24a3ce8f09746410daee3d5cd823ff7a0675b7
188
+ split: test
189
+ type: mteb/mrtidy
190
+ metrics:
191
+ - type: main_score
192
+ value: 55.584999999999994
193
+ - type: map_at_1
194
+ value: 34.197
195
+ - type: map_at_10
196
+ value: 48.658
197
+ - type: map_at_100
198
+ value: 49.491
199
+ - type: map_at_1000
200
+ value: 49.518
201
+ - type: ndcg_at_1
202
+ value: 36.91
203
+ - type: ndcg_at_10
204
+ value: 55.584999999999994
205
+ - type: ndcg_at_100
206
+ value: 59.082
207
+ - type: ndcg_at_1000
208
+ value: 59.711000000000006
209
+ - type: ndcg_at_20
210
+ value: 57.537000000000006
211
+ - type: ndcg_at_3
212
+ value: 48.732
213
+ - type: ndcg_at_5
214
+ value: 52.834
215
+ task:
216
+ type: Retrieval
217
+ - dataset:
218
+ config: default
219
+ name: MTEB SadeemQuestionRetrieval (default)
220
+ revision: 3cb0752b182e5d5d740df547748b06663c8e0bd9
221
+ split: test
222
+ type: sadeem-ai/sadeem-ar-eval-retrieval-questions
223
+ metrics:
224
+ - type: main_score
225
+ value: 67.916
226
+ - type: map_at_1
227
+ value: 31.785999999999998
228
+ - type: map_at_10
229
+ value: 58.18600000000001
230
+ - type: map_at_100
231
+ value: 58.287
232
+ - type: map_at_1000
233
+ value: 58.29
234
+ - type: ndcg_at_1
235
+ value: 31.785999999999998
236
+ - type: ndcg_at_10
237
+ value: 67.916
238
+ - type: ndcg_at_100
239
+ value: 68.44200000000001
240
+ - type: ndcg_at_1000
241
+ value: 68.53399999999999
242
+ - type: ndcg_at_20
243
+ value: 68.11
244
+ - type: ndcg_at_3
245
+ value: 66.583
246
+ - type: ndcg_at_5
247
+ value: 67.5
248
+ task:
249
+ type: Retrieval
250
+ - dataset:
251
+ config: ara-ara
252
+ name: MTEB XPQARetrieval (ara-ara)
253
+ revision: c99d599f0a6ab9b85b065da6f9d94f9cf731679f
254
+ split: test
255
+ type: jinaai/xpqa
256
+ metrics:
257
+ - type: main_score
258
+ value: 43.622
259
+ - type: map_at_1
260
+ value: 19.236
261
+ - type: map_at_10
262
+ value: 37.047000000000004
263
+ - type: map_at_100
264
+ value: 38.948
265
+ - type: map_at_1000
266
+ value: 39.054
267
+ - type: ndcg_at_1
268
+ value: 35.333
269
+ - type: ndcg_at_10
270
+ value: 43.622
271
+ - type: ndcg_at_100
272
+ value: 50.761
273
+ - type: ndcg_at_1000
274
+ value: 52.932
275
+ - type: ndcg_at_20
276
+ value: 46.686
277
+ - type: ndcg_at_3
278
+ value: 37.482
279
+ - type: ndcg_at_5
280
+ value: 39.635999999999996
281
+ task:
282
+ type: Retrieval
283
  ---
284
 
285