antoinelouis commited on
Commit
4a94c57
1 Parent(s): 513c436

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -67,7 +67,7 @@ with Run().context(RunConfig(nranks=n_gpu,experiment=experiment)):
67
 
68
  ## Evaluation
69
 
70
- The model is evaluated on the smaller development set of mMARCO-fr, which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compared its performance with a single-vector representation model fine-tuned on the same dataset. We report the mean reciprocal rank (MRR) and recall at various cut-offs (R@k).
71
 
72
  | model | Vocab. | #Param. | Size | MRR@10 | R@10 | R@100(↑) | R@500 |
73
  |:------------------------------------------------------------------------------------------------------------------------|:-------|--------:|------:|---------:|-------:|-----------:|--------:|
@@ -87,6 +87,7 @@ The model is fine-tuned on the French version of the [mMARCO](https://huggingfac
87
  - a training set of ~533k unique queries (with at least one relevant passage);
88
  - a development set of ~101k queries;
89
  - a smaller dev set of 6,980 queries (which is actually used for evaluation in most published works).
 
90
  The triples are sampled from the ~39.8M triples from [triples.train.small.tsv](https://microsoft.github.io/msmarco/Datasets.html#passage-ranking-dataset). In the future, better negatives could be selected by exploiting the [msmarco-hard-negatives] dataset that contains 50 hard negatives mined from BM25 and 12 dense retrievers for each training query.
91
 
92
  ## Citation
 
67
 
68
  ## Evaluation
69
 
70
+ The model is evaluated on the smaller development set of mMARCO-fr, which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compared its performance to a single-vector representation model fine-tuned on the same dataset. We report the mean reciprocal rank (MRR) and recall at various cut-offs (R@k).
71
 
72
  | model | Vocab. | #Param. | Size | MRR@10 | R@10 | R@100(↑) | R@500 |
73
  |:------------------------------------------------------------------------------------------------------------------------|:-------|--------:|------:|---------:|-------:|-----------:|--------:|
 
87
  - a training set of ~533k unique queries (with at least one relevant passage);
88
  - a development set of ~101k queries;
89
  - a smaller dev set of 6,980 queries (which is actually used for evaluation in most published works).
90
+
91
  The triples are sampled from the ~39.8M triples from [triples.train.small.tsv](https://microsoft.github.io/msmarco/Datasets.html#passage-ranking-dataset). In the future, better negatives could be selected by exploiting the [msmarco-hard-negatives] dataset that contains 50 hard negatives mined from BM25 and 12 dense retrievers for each training query.
92
 
93
  ## Citation