antoinelouis
commited on
Commit
•
4a94c57
1
Parent(s):
513c436
Update README.md
Browse files
README.md
CHANGED
@@ -67,7 +67,7 @@ with Run().context(RunConfig(nranks=n_gpu,experiment=experiment)):
|
|
67 |
|
68 |
## Evaluation
|
69 |
|
70 |
-
The model is evaluated on the smaller development set of mMARCO-fr, which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compared its performance
|
71 |
|
72 |
| model | Vocab. | #Param. | Size | MRR@10 | R@10 | R@100(↑) | R@500 |
|
73 |
|:------------------------------------------------------------------------------------------------------------------------|:-------|--------:|------:|---------:|-------:|-----------:|--------:|
|
@@ -87,6 +87,7 @@ The model is fine-tuned on the French version of the [mMARCO](https://huggingfac
|
|
87 |
- a training set of ~533k unique queries (with at least one relevant passage);
|
88 |
- a development set of ~101k queries;
|
89 |
- a smaller dev set of 6,980 queries (which is actually used for evaluation in most published works).
|
|
|
90 |
The triples are sampled from the ~39.8M triples from [triples.train.small.tsv](https://microsoft.github.io/msmarco/Datasets.html#passage-ranking-dataset). In the future, better negatives could be selected by exploiting the [msmarco-hard-negatives] dataset that contains 50 hard negatives mined from BM25 and 12 dense retrievers for each training query.
|
91 |
|
92 |
## Citation
|
|
|
67 |
|
68 |
## Evaluation
|
69 |
|
70 |
+
The model is evaluated on the smaller development set of mMARCO-fr, which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compared its performance to a single-vector representation model fine-tuned on the same dataset. We report the mean reciprocal rank (MRR) and recall at various cut-offs (R@k).
|
71 |
|
72 |
| model | Vocab. | #Param. | Size | MRR@10 | R@10 | R@100(↑) | R@500 |
|
73 |
|:------------------------------------------------------------------------------------------------------------------------|:-------|--------:|------:|---------:|-------:|-----------:|--------:|
|
|
|
87 |
- a training set of ~533k unique queries (with at least one relevant passage);
|
88 |
- a development set of ~101k queries;
|
89 |
- a smaller dev set of 6,980 queries (which is actually used for evaluation in most published works).
|
90 |
+
|
91 |
The triples are sampled from the ~39.8M triples from [triples.train.small.tsv](https://microsoft.github.io/msmarco/Datasets.html#passage-ranking-dataset). In the future, better negatives could be selected by exploiting the [msmarco-hard-negatives] dataset that contains 50 hard negatives mined from BM25 and 12 dense retrievers for each training query.
|
92 |
|
93 |
## Citation
|