nan commited on
Commit
50ad142
1 Parent(s): 1f60f51

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -138,14 +138,14 @@ Note that both ColBERTv2 and Jina-ColBERT-v1 only employ MSMARCO passage ranking
138
 
139
  We also evaluate the zero-shot performance on datasets where documents have longer context length and compare with some long-context embedding models. Here we use the [LoCo benchmark](https://www.together.ai/blog/long-context-retrieval-models-with-monarch-mixer), which contains 5 datasets with long context length.
140
 
141
- | Model | Avg. NDCG@10 | Model max context length | Used context length |
142
  | --- | :---: | :---: | :---: |
143
- | ColBERTv2 | 74.3 | 512 | 512 |
144
- | Jina-ColBERT-v1 | 75.5 | 8192 | 512 |
145
- | Jina-ColBERT-v1 | 83.7 | 8192 | 8192* |
146
- | Jina-embeddings-v2-base-en | 85.4 | 8192 | 8192 |
147
 
148
- \* denotes that we used the context length of 8192 for document but the query length is still 512.
149
 
150
  **To summarize, Jina-ColBERT achieves the comparable performance with ColBERTv2 on all benchmarks, and outperforms ColBERTv2 on datasets in where documents have longer context length.**
151
 
 
138
 
139
  We also evaluate the zero-shot performance on datasets where documents have longer context length and compare with some long-context embedding models. Here we use the [LoCo benchmark](https://www.together.ai/blog/long-context-retrieval-models-with-monarch-mixer), which contains 5 datasets with long context length.
140
 
141
+ | Model | Used context length | Model max context length | Avg. NDCG@10 |
142
  | --- | :---: | :---: | :---: |
143
+ | ColBERTv2 | 512 | 512 | 74.3 |
144
+ | Jina-ColBERT-v1 (truncated) | 512* | 8192 | 75.5 |
145
+ | Jina-ColBERT-v1 | 8192 | 8192 | 83.7 |
146
+ | Jina-embeddings-v2-base-en | 8192 | 8192 | **85.4** |
147
 
148
+ \* denotes that we truncate the context length to the length of 512 for document but the query length is still 512.
149
 
150
  **To summarize, Jina-ColBERT achieves the comparable performance with ColBERTv2 on all benchmarks, and outperforms ColBERTv2 on datasets in where documents have longer context length.**
151