numb3r3 jemfu commited on
Commit
3aaa84b
1 Parent(s): 667e9cb

docs: update benchmark (#4)

Browse files

- docs: update benchmark (dbeacd7412974d1df0ef4e71805254045dde33c3)


Co-authored-by: Jie Fu <jemfu@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -150,12 +150,12 @@ Specifically, the `rerank()` function will split the documents into chunks of si
150
 
151
  We evaluated Jina Reranker v2 on multiple benchmarks to ensure top-tier performance and search relevance.
152
 
153
- | Model Name | Model Size | Miracl(nDCG@10, 18 langs) | MKQA(nDCG@10, 26 langs) | BEIR(nDCG@10, 17 datasets) | MLDR(recall@10, 13 langs) | CodeSearchNet (MRR@10, 3 tasks) | AirBench (nDCG@10, zh/en) | ToolBench (recall@3, 3 tasks) | TableSearch (recall@3) |
154
- | -----------------------------: | :-------------------------: | :-------------------------: | :-------------------------: | :----------------------------: | :----------------------------: | :----------------------------: | :----------------------------: | :----------------------------: | :----------------------------: |
155
- | jina-reranker-v2-multilingual | 278M | 62.14 | 54.83 | 53.17 | 68.95 | 71.36 | 61.33 | 77.75 | 93.31 |
156
- | bge-reranker-v2-m3 | 568M | 63.43 | 54.17 | 53.65 | 59.73 | 62.86 | 61.28 | 78.46 | 74.86 |
157
- | mmarco-mMiniLMv2-L12-H384-v1 | 118M | 59.71 | 53.37 | 45.40 | 28.91 | 51.78 | 56.46 | 58.39 | 53.60 |
158
- | jina-reranker-v1-base-en | 137M | - | - | 52.45 | - | - | - | 74.13 | 72.89 |
159
 
160
  Note:
161
  - NDCG@10 and MRR@10 measure ranking quality, with higher scores indicating better search results
 
150
 
151
  We evaluated Jina Reranker v2 on multiple benchmarks to ensure top-tier performance and search relevance.
152
 
153
+ | Model Name | Model Size | MKQA(nDCG@10, 26 langs) | BEIR(nDCG@10, 17 datasets) | MLDR(recall@10, 13 langs) | CodeSearchNet (MRR@10, 3 tasks) | AirBench (nDCG@10, zh/en) | ToolBench (recall@3, 3 tasks) | TableSearch (recall@3) |
154
+ |:-----------------------------: |:----------: |------------------------- |---------------------------- |--------------------------- |--------------------------------- |--------------------------- |------------------------------- |------------------------ |
155
+ | jina-reranker-v2-multilingual | 278M | 54.83 | 53.17 | 68.95 | 71.36 | 60.95 | 77.75 | 93.31 |
156
+ | bge-reranker-v2-m3 | 568M | 54.17 | 53.65 | 59.73 | 62.86 | 61.28 | 78.46 | 74.86 |
157
+ | mmarco-mMiniLMv2-L12-H384-v1 | 118M | 53.37 | 45.40 | 28.91 | 51.78 | 56.46 | 58.39 | 53.60 |
158
+ | jina-reranker-v1-base-en | 137M | - | 52.45 | - | - | - | 74.13 | 72.89 |
159
 
160
  Note:
161
  - NDCG@10 and MRR@10 measure ranking quality, with higher scores indicating better search results