Katsumata420 commited on
Commit
ce6d6bd
·
verified ·
1 Parent(s): f2766ef

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -11
README.md CHANGED
@@ -2322,7 +2322,7 @@ model-index:
2322
  type: mteb/scidocs
2323
  metrics:
2324
  - type: ndcg_at_1
2325
- value: 20.0
2326
  - type: ndcg_at_3
2327
  value: 15.842
2328
  - type: ndcg_at_5
@@ -2364,7 +2364,7 @@ model-index:
2364
  - type: recall_at_1000
2365
  value: 70.048
2366
  - type: precision_at_1
2367
- value: 20.0
2368
  - type: precision_at_3
2369
  value: 14.667
2370
  - type: precision_at_5
@@ -2378,7 +2378,7 @@ model-index:
2378
  - type: precision_at_1000
2379
  value: 0.345
2380
  - type: mrr_at_1
2381
- value: 20.0
2382
  - type: mrr_at_3
2383
  value: 26.016699999999997
2384
  - type: mrr_at_5
@@ -2973,7 +2973,7 @@ model-index:
2973
  type: mteb/trec-covid
2974
  metrics:
2975
  - type: ndcg_at_1
2976
- value: 63.0
2977
  - type: ndcg_at_3
2978
  value: 66.47999999999999
2979
  - type: ndcg_at_5
@@ -3015,7 +3015,7 @@ model-index:
3015
  - type: recall_at_1000
3016
  value: 38.541
3017
  - type: precision_at_1
3018
- value: 68.0
3019
  - type: precision_at_3
3020
  value: 70.667
3021
  - type: precision_at_5
@@ -3029,9 +3029,9 @@ model-index:
3029
  - type: precision_at_1000
3030
  value: 17.657999999999998
3031
  - type: mrr_at_1
3032
- value: 68.0
3033
  - type: mrr_at_3
3034
- value: 79.0
3035
  - type: mrr_at_5
3036
  value: 79.5
3037
  - type: mrr_at_10
@@ -3782,6 +3782,7 @@ model-index:
3782
  value: 85.1691
3783
  task:
3784
  type: PairClassification
 
3785
  ---
3786
 
3787
  # RetrievaEmbedding-01: AMBER
@@ -3902,10 +3903,10 @@ The prompts used in JQaRA and JaCWIR are `Retrieval-query` and `Retrieval-passag
3902
  | Model | # Parameters | JQaRA (nDCG@10) | JaCWIR (MAP@10) | MLDR Japanese Subset (nDCG@10) |
3903
  | :--- | --- | ---: | ---: | ---: |
3904
  | base models | < 300M | | | |
3905
- | [cl-nagoya/ruri-base](https://huggingface.co/cl-nagoya/ruri-base) | 111M | 58.4 | 83.3 | TBA |
3906
- | [AMBER-base](https://huggingface.co/retrieva-jp/amber-base) | 130M | 57.1 | 81.6 | TBA |
3907
  | [pkshatech/GLuCoSE-base-ja-v2](https://huggingface.co/pkshatech/GLuCoSE-base-ja-v2) | 133M | **60.6** | **85.3** | 33.99 |
3908
- | [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) | 278M | 47.1 | **85.3** | TBA |
3909
  | large models | 300M < | | | |
3910
  | AMBER-large <br> (this model) | 315M | 62.5 | 82.4 | 34.57 |
3911
  | [cl-nagoya/ruri-large](https://huggingface.co/cl-nagoya/ruri-large) | 337M | **62.8** | 82.5 | **34.78** |
@@ -3935,4 +3936,4 @@ Satoru Katsumata, Daisuke Kimura, Jiro Nishitoba
3935
 
3936
  ## Model Card Contact
3937
 
3938
- pr[at]retrieva.jp
 
2322
  type: mteb/scidocs
2323
  metrics:
2324
  - type: ndcg_at_1
2325
+ value: 20
2326
  - type: ndcg_at_3
2327
  value: 15.842
2328
  - type: ndcg_at_5
 
2364
  - type: recall_at_1000
2365
  value: 70.048
2366
  - type: precision_at_1
2367
+ value: 20
2368
  - type: precision_at_3
2369
  value: 14.667
2370
  - type: precision_at_5
 
2378
  - type: precision_at_1000
2379
  value: 0.345
2380
  - type: mrr_at_1
2381
+ value: 20
2382
  - type: mrr_at_3
2383
  value: 26.016699999999997
2384
  - type: mrr_at_5
 
2973
  type: mteb/trec-covid
2974
  metrics:
2975
  - type: ndcg_at_1
2976
+ value: 63
2977
  - type: ndcg_at_3
2978
  value: 66.47999999999999
2979
  - type: ndcg_at_5
 
3015
  - type: recall_at_1000
3016
  value: 38.541
3017
  - type: precision_at_1
3018
+ value: 68
3019
  - type: precision_at_3
3020
  value: 70.667
3021
  - type: precision_at_5
 
3029
  - type: precision_at_1000
3030
  value: 17.657999999999998
3031
  - type: mrr_at_1
3032
+ value: 68
3033
  - type: mrr_at_3
3034
+ value: 79
3035
  - type: mrr_at_5
3036
  value: 79.5
3037
  - type: mrr_at_10
 
3782
  value: 85.1691
3783
  task:
3784
  type: PairClassification
3785
+ license: apache-2.0
3786
  ---
3787
 
3788
  # RetrievaEmbedding-01: AMBER
 
3903
  | Model | # Parameters | JQaRA (nDCG@10) | JaCWIR (MAP@10) | MLDR Japanese Subset (nDCG@10) |
3904
  | :--- | --- | ---: | ---: | ---: |
3905
  | base models | < 300M | | | |
3906
+ | [cl-nagoya/ruri-base](https://huggingface.co/cl-nagoya/ruri-base) | 111M | 58.4 | 83.3 | 32.77 |
3907
+ | [AMBER-base](https://huggingface.co/retrieva-jp/amber-base) | 130M | 57.1 | 81.6 | **35.69** |
3908
  | [pkshatech/GLuCoSE-base-ja-v2](https://huggingface.co/pkshatech/GLuCoSE-base-ja-v2) | 133M | **60.6** | **85.3** | 33.99 |
3909
+ | [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) | 278M | 47.1 | **85.3** | 25.46 |
3910
  | large models | 300M < | | | |
3911
  | AMBER-large <br> (this model) | 315M | 62.5 | 82.4 | 34.57 |
3912
  | [cl-nagoya/ruri-large](https://huggingface.co/cl-nagoya/ruri-large) | 337M | **62.8** | 82.5 | **34.78** |
 
3936
 
3937
  ## Model Card Contact
3938
 
3939
+ pr[at]retrieva.jp