--- tags: - sentence-transformers - feature-extraction - sentence-similarity - transformers - mteb license: gpl-3.0 language: - pl pipeline_tag: sentence-similarity model-index: - name: KartonBERT-USE-base-v1 results: - dataset: config: default name: MTEB AllegroReviews (default) revision: b89853e6de927b0e3bfa8ecc0e56fe4e02ceafc6 split: test type: PL-MTEB/allegro-reviews metrics: - type: accuracy value: 54.15506958250498 - type: f1 value: 46.5327734573016 - type: f1_weighted value: 54.68392312652715 - type: main_score value: 54.15506958250498 task: type: Classification - dataset: config: default name: MTEB ArguAna-PL (default) revision: 63fc86750af76253e8c760fc9e534bbf24d260a2 split: test type: clarin-knext/arguana-pl metrics: - type: main_score value: 56.901 - type: map_at_1 value: 31.791999999999998 - type: map_at_10 value: 48.054 - type: map_at_100 value: 48.878 - type: map_at_1000 value: 48.882 - type: map_at_20 value: 48.737 - type: map_at_3 value: 43.231 - type: map_at_5 value: 46.111999999999995 - type: mrr_at_1 value: 32.574679943100996 - type: mrr_at_10 value: 48.33367201788255 - type: mrr_at_100 value: 49.15718185630711 - type: mrr_at_1000 value: 49.1617121578244 - type: mrr_at_20 value: 49.01694598176999 - type: mrr_at_3 value: 43.51588430535799 - type: mrr_at_5 value: 46.39284020862968 - type: nauc_map_at_1000_diff1 value: 9.261912197433015 - type: nauc_map_at_1000_max value: -7.845080024659214 - type: nauc_map_at_1000_std value: -11.991866544435679 - type: nauc_map_at_100_diff1 value: 9.26664720652122 - type: nauc_map_at_100_max value: -7.832848294616753 - type: nauc_map_at_100_std value: -11.9808027056787 - type: nauc_map_at_10_diff1 value: 8.807497468420616 - type: nauc_map_at_10_max value: -7.964448980578822 - type: nauc_map_at_10_std value: -12.12899711880432 - type: nauc_map_at_1_diff1 value: 13.32445989333938 - type: nauc_map_at_1_max value: -10.482064414968285 - type: nauc_map_at_1_std value: -13.501360370797148 - type: nauc_map_at_20_diff1 value: 9.195156779288766 - type: nauc_map_at_20_max value: -7.783014219043757 - type: nauc_map_at_20_std value: -11.969490540925406 - type: nauc_map_at_3_diff1 value: 9.216987154577305 - type: nauc_map_at_3_max value: -7.320889292833926 - type: nauc_map_at_3_std value: -11.556536581082206 - type: nauc_map_at_5_diff1 value: 8.632595608937663 - type: nauc_map_at_5_max value: -8.100862283698563 - type: nauc_map_at_5_std value: -12.362725223929877 - type: nauc_mrr_at_1000_diff1 value: 6.955563701248687 - type: nauc_mrr_at_1000_max value: -8.482826175940565 - type: nauc_mrr_at_1000_std value: -11.685173284450771 - type: nauc_mrr_at_100_diff1 value: 6.960630009309121 - type: nauc_mrr_at_100_max value: -8.47053290533026 - type: nauc_mrr_at_100_std value: -11.674185693345231 - type: nauc_mrr_at_10_diff1 value: 6.571885416453002 - type: nauc_mrr_at_10_max value: -8.579534736340365 - type: nauc_mrr_at_10_std value: -11.828451090180964 - type: nauc_mrr_at_1_diff1 value: 11.037953768000532 - type: nauc_mrr_at_1_max value: -9.873898613911402 - type: nauc_mrr_at_1_std value: -12.460415140907712 - type: nauc_mrr_at_20_diff1 value: 6.900888306598456 - type: nauc_mrr_at_20_max value: -8.416481040187186 - type: nauc_mrr_at_20_std value: -11.664087568363449 - type: nauc_mrr_at_3_diff1 value: 7.06762175360558 - type: nauc_mrr_at_3_max value: -8.017279028707673 - type: nauc_mrr_at_3_std value: -11.297456731246768 - type: nauc_mrr_at_5_diff1 value: 6.448800345577737 - type: nauc_mrr_at_5_max value: -8.745349771650183 - type: nauc_mrr_at_5_std value: -11.969389047713944 - type: nauc_ndcg_at_1000_diff1 value: 8.76255638439487 - type: nauc_ndcg_at_1000_max value: -7.343629528331341 - type: nauc_ndcg_at_1000_std value: -11.47940744019621 - type: nauc_ndcg_at_100_diff1 value: 8.919277731016157 - type: nauc_ndcg_at_100_max value: -7.019108339742727 - type: nauc_ndcg_at_100_std value: -11.194732417709883 - type: nauc_ndcg_at_10_diff1 value: 6.935115430123545 - type: nauc_ndcg_at_10_max value: -7.331328886115093 - type: nauc_ndcg_at_10_std value: -11.823538474289432 - type: nauc_ndcg_at_1_diff1 value: 13.32445989333938 - type: nauc_ndcg_at_1_max value: -10.482064414968285 - type: nauc_ndcg_at_1_std value: -13.501360370797148 - type: nauc_ndcg_at_20_diff1 value: 8.459389806724928 - type: nauc_ndcg_at_20_max value: -6.58398762434462 - type: nauc_ndcg_at_20_std value: -11.058848254315285 - type: nauc_ndcg_at_3_diff1 value: 7.835560234578747 - type: nauc_ndcg_at_3_max value: -6.423483615691791 - type: nauc_ndcg_at_3_std value: -10.911115187058067 - type: nauc_ndcg_at_5_diff1 value: 6.682571459029503 - type: nauc_ndcg_at_5_max value: -7.818737713863339 - type: nauc_ndcg_at_5_std value: -12.432449183297598 - type: nauc_precision_at_1000_diff1 value: -2.5745586949686294 - type: nauc_precision_at_1000_max value: 10.263038467948435 - type: nauc_precision_at_1000_std value: 68.30822185314709 - type: nauc_precision_at_100_diff1 value: 32.41864555136661 - type: nauc_precision_at_100_max value: 52.55109728866118 - type: nauc_precision_at_100_std value: 62.241685968841395 - type: nauc_precision_at_10_diff1 value: -4.819815865728714 - type: nauc_precision_at_10_max value: -3.3537757455238877 - type: nauc_precision_at_10_std value: -9.963817157694175 - type: nauc_precision_at_1_diff1 value: 13.32445989333938 - type: nauc_precision_at_1_max value: -10.482064414968285 - type: nauc_precision_at_1_std value: -13.501360370797148 - type: nauc_precision_at_20_diff1 value: 3.635887978878025 - type: nauc_precision_at_20_max value: 14.898454122959496 - type: nauc_precision_at_20_std value: 6.619835540940379 - type: nauc_precision_at_3_diff1 value: 3.6054431173895667 - type: nauc_precision_at_3_max value: -3.728728411630629 - type: nauc_precision_at_3_std value: -8.937528472143674 - type: nauc_precision_at_5_diff1 value: -0.655295588469194 - type: nauc_precision_at_5_max value: -7.099667676645105 - type: nauc_precision_at_5_std value: -13.01579557796282 - type: nauc_recall_at_1000_diff1 value: -2.5745586949696144 - type: nauc_recall_at_1000_max value: 10.26303846794546 - type: nauc_recall_at_1000_std value: 68.3082218531417 - type: nauc_recall_at_100_diff1 value: 32.41864555136416 - type: nauc_recall_at_100_max value: 52.55109728865944 - type: nauc_recall_at_100_std value: 62.24168596884267 - type: nauc_recall_at_10_diff1 value: -4.819815865728716 - type: nauc_recall_at_10_max value: -3.353775745523958 - type: nauc_recall_at_10_std value: -9.963817157694203 - type: nauc_recall_at_1_diff1 value: 13.32445989333938 - type: nauc_recall_at_1_max value: -10.482064414968285 - type: nauc_recall_at_1_std value: -13.501360370797148 - type: nauc_recall_at_20_diff1 value: 3.6358879788779648 - type: nauc_recall_at_20_max value: 14.898454122959254 - type: nauc_recall_at_20_std value: 6.619835540940201 - type: nauc_recall_at_3_diff1 value: 3.6054431173895582 - type: nauc_recall_at_3_max value: -3.7287284116306205 - type: nauc_recall_at_3_std value: -8.937528472143658 - type: nauc_recall_at_5_diff1 value: -0.6552955884692194 - type: nauc_recall_at_5_max value: -7.099667676645175 - type: nauc_recall_at_5_std value: -13.015795577962777 - type: ndcg_at_1 value: 31.791999999999998 - type: ndcg_at_10 value: 56.901 - type: ndcg_at_100 value: 60.192 - type: ndcg_at_1000 value: 60.288 - type: ndcg_at_20 value: 59.303 - type: ndcg_at_3 value: 47.028999999999996 - type: ndcg_at_5 value: 52.201 - type: precision_at_1 value: 31.791999999999998 - type: precision_at_10 value: 8.506 - type: precision_at_100 value: 0.989 - type: precision_at_1000 value: 0.1 - type: precision_at_20 value: 4.719 - type: precision_at_3 value: 19.346 - type: precision_at_5 value: 14.111 - type: recall_at_1 value: 31.791999999999998 - type: recall_at_10 value: 85.064 - type: recall_at_100 value: 98.933 - type: recall_at_1000 value: 99.644 - type: recall_at_20 value: 94.381 - type: recall_at_3 value: 58.037000000000006 - type: recall_at_5 value: 70.555 task: type: Retrieval - dataset: config: pol_Latn-pol_Latn name: MTEB BelebeleRetrieval (pol_Latn-pol_Latn) revision: 75b399394a9803252cfec289d103de462763db7c split: test type: facebook/belebele metrics: - type: main_score value: 91.541 - type: map_at_1 value: 85.77799999999999 - type: map_at_10 value: 89.752 - type: map_at_100 value: 89.878 - type: map_at_1000 value: 89.881 - type: map_at_20 value: 89.852 - type: map_at_3 value: 89.019 - type: map_at_5 value: 89.363 - type: mrr_at_1 value: 85.77777777777777 - type: mrr_at_10 value: 89.75198412698411 - type: mrr_at_100 value: 89.87762406306918 - type: mrr_at_1000 value: 89.8806515152748 - type: mrr_at_20 value: 89.85208842298627 - type: mrr_at_3 value: 89.01851851851852 - type: mrr_at_5 value: 89.36296296296297 - type: nauc_map_at_1000_diff1 value: 91.4476978163489 - type: nauc_map_at_1000_max value: 81.5514176763366 - type: nauc_map_at_1000_std value: -26.55954678441229 - type: nauc_map_at_100_diff1 value: 91.45032737132165 - type: nauc_map_at_100_max value: 81.55491042152336 - type: nauc_map_at_100_std value: -26.529320255777634 - type: nauc_map_at_10_diff1 value: 91.39551937416603 - type: nauc_map_at_10_max value: 81.62509202485019 - type: nauc_map_at_10_std value: -26.551904492525747 - type: nauc_map_at_1_diff1 value: 92.16805294153365 - type: nauc_map_at_1_max value: 79.46808026227919 - type: nauc_map_at_1_std value: -26.20021553032597 - type: nauc_map_at_20_diff1 value: 91.44854892971289 - type: nauc_map_at_20_max value: 81.57301440410872 - type: nauc_map_at_20_std value: -26.442044476153104 - type: nauc_map_at_3_diff1 value: 91.37113720093754 - type: nauc_map_at_3_max value: 81.86549795443065 - type: nauc_map_at_3_std value: -27.81837152321835 - type: nauc_map_at_5_diff1 value: 91.36501739587844 - type: nauc_map_at_5_max value: 81.68480517417245 - type: nauc_map_at_5_std value: -28.018028030994742 - type: nauc_mrr_at_1000_diff1 value: 91.4476978163489 - type: nauc_mrr_at_1000_max value: 81.5514176763366 - type: nauc_mrr_at_1000_std value: -26.55954678441229 - type: nauc_mrr_at_100_diff1 value: 91.45032737132165 - type: nauc_mrr_at_100_max value: 81.55491042152336 - type: nauc_mrr_at_100_std value: -26.529320255777634 - type: nauc_mrr_at_10_diff1 value: 91.39551937416603 - type: nauc_mrr_at_10_max value: 81.62509202485019 - type: nauc_mrr_at_10_std value: -26.551904492525747 - type: nauc_mrr_at_1_diff1 value: 92.16805294153365 - type: nauc_mrr_at_1_max value: 79.46808026227919 - type: nauc_mrr_at_1_std value: -26.20021553032597 - type: nauc_mrr_at_20_diff1 value: 91.44854892971289 - type: nauc_mrr_at_20_max value: 81.57301440410872 - type: nauc_mrr_at_20_std value: -26.442044476153104 - type: nauc_mrr_at_3_diff1 value: 91.37113720093754 - type: nauc_mrr_at_3_max value: 81.86549795443065 - type: nauc_mrr_at_3_std value: -27.81837152321835 - type: nauc_mrr_at_5_diff1 value: 91.36501739587844 - type: nauc_mrr_at_5_max value: 81.68480517417245 - type: nauc_mrr_at_5_std value: -28.018028030994742 - type: nauc_ndcg_at_1000_diff1 value: 91.38663887591338 - type: nauc_ndcg_at_1000_max value: 81.83100576728476 - type: nauc_ndcg_at_1000_std value: -26.052082966114693 - type: nauc_ndcg_at_100_diff1 value: 91.46858617319295 - type: nauc_ndcg_at_100_max value: 81.93078333237341 - type: nauc_ndcg_at_100_std value: -25.153555276485868 - type: nauc_ndcg_at_10_diff1 value: 91.13804299227158 - type: nauc_ndcg_at_10_max value: 82.25580303805849 - type: nauc_ndcg_at_10_std value: -25.383981896383467 - type: nauc_ndcg_at_1_diff1 value: 92.16805294153365 - type: nauc_ndcg_at_1_max value: 79.46808026227919 - type: nauc_ndcg_at_1_std value: -26.20021553032597 - type: nauc_ndcg_at_20_diff1 value: 91.43798225301471 - type: nauc_ndcg_at_20_max value: 82.07518001371142 - type: nauc_ndcg_at_20_std value: -24.777088112541364 - type: nauc_ndcg_at_3_diff1 value: 91.09449436290618 - type: nauc_ndcg_at_3_max value: 82.67766398790694 - type: nauc_ndcg_at_3_std value: -28.698696574215248 - type: nauc_ndcg_at_5_diff1 value: 91.06439333732669 - type: nauc_ndcg_at_5_max value: 82.36532604706885 - type: nauc_ndcg_at_5_std value: -29.0545149754883 - type: nauc_precision_at_1000_diff1 value: .nan - type: nauc_precision_at_1000_max value: .nan - type: nauc_precision_at_1000_std value: .nan - type: nauc_precision_at_100_diff1 value: 100.0 - type: nauc_precision_at_100_max value: 92.1568627450979 - type: nauc_precision_at_100_std value: 67.46031746031498 - type: nauc_precision_at_10_diff1 value: 88.79551820728234 - type: nauc_precision_at_10_max value: 87.94261294261227 - type: nauc_precision_at_10_std value: -10.505997270703746 - type: nauc_precision_at_1_diff1 value: 92.16805294153365 - type: nauc_precision_at_1_max value: 79.46808026227919 - type: nauc_precision_at_1_std value: -26.20021553032597 - type: nauc_precision_at_20_diff1 value: 93.71543489190498 - type: nauc_precision_at_20_max value: 90.03447532859191 - type: nauc_precision_at_20_std value: 17.970265029086878 - type: nauc_precision_at_3_diff1 value: 89.92617880485524 - type: nauc_precision_at_3_max value: 86.0965219421104 - type: nauc_precision_at_3_std value: -32.5922035480858 - type: nauc_precision_at_5_diff1 value: 89.51410629611307 - type: nauc_precision_at_5_max value: 85.74266307830291 - type: nauc_precision_at_5_std value: -34.57278336171098 - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_100_diff1 value: 100.0 - type: nauc_recall_at_100_max value: 92.15686274509831 - type: nauc_recall_at_100_std value: 67.4603174603183 - type: nauc_recall_at_10_diff1 value: 88.79551820728301 - type: nauc_recall_at_10_max value: 87.94261294261294 - type: nauc_recall_at_10_std value: -10.505997270703194 - type: nauc_recall_at_1_diff1 value: 92.16805294153365 - type: nauc_recall_at_1_max value: 79.46808026227919 - type: nauc_recall_at_1_std value: -26.20021553032597 - type: nauc_recall_at_20_diff1 value: 93.71543489190589 - type: nauc_recall_at_20_max value: 90.03447532859326 - type: nauc_recall_at_20_std value: 17.97026502908778 - type: nauc_recall_at_3_diff1 value: 89.92617880485525 - type: nauc_recall_at_3_max value: 86.09652194211006 - type: nauc_recall_at_3_std value: -32.592203548085735 - type: nauc_recall_at_5_diff1 value: 89.5141062961131 - type: nauc_recall_at_5_max value: 85.74266307830314 - type: nauc_recall_at_5_std value: -34.572783361711004 - type: ndcg_at_1 value: 85.77799999999999 - type: ndcg_at_10 value: 91.541 - type: ndcg_at_100 value: 92.07600000000001 - type: ndcg_at_1000 value: 92.15 - type: ndcg_at_20 value: 91.906 - type: ndcg_at_3 value: 90.017 - type: ndcg_at_5 value: 90.625 - type: precision_at_1 value: 85.77799999999999 - type: precision_at_10 value: 9.711 - type: precision_at_100 value: 0.9939999999999999 - type: precision_at_1000 value: 0.1 - type: precision_at_20 value: 4.928 - type: precision_at_3 value: 30.963 - type: precision_at_5 value: 18.867 - type: recall_at_1 value: 85.77799999999999 - type: recall_at_10 value: 97.111 - type: recall_at_100 value: 99.444 - type: recall_at_1000 value: 100.0 - type: recall_at_20 value: 98.556 - type: recall_at_3 value: 92.889 - type: recall_at_5 value: 94.333 task: type: Retrieval - dataset: config: pol_Latn-eng_Latn name: MTEB BelebeleRetrieval (pol_Latn-eng_Latn) revision: 75b399394a9803252cfec289d103de462763db7c split: test type: facebook/belebele metrics: - type: main_score value: 83.504 - type: map_at_1 value: 74.333 - type: map_at_10 value: 80.569 - type: map_at_100 value: 80.803 - type: map_at_1000 value: 80.816 - type: map_at_20 value: 80.723 - type: map_at_3 value: 79.278 - type: map_at_5 value: 80.067 - type: mrr_at_1 value: 74.33333333333333 - type: mrr_at_10 value: 80.56869488536155 - type: mrr_at_100 value: 80.80281231188738 - type: mrr_at_1000 value: 80.81591021232106 - type: mrr_at_20 value: 80.72251527043838 - type: mrr_at_3 value: 79.27777777777779 - type: mrr_at_5 value: 80.06666666666666 - type: nauc_map_at_1000_diff1 value: 87.1827200996951 - type: nauc_map_at_1000_max value: 76.68256630016205 - type: nauc_map_at_1000_std value: -5.730353903171156 - type: nauc_map_at_100_diff1 value: 87.17937163549941 - type: nauc_map_at_100_max value: 76.69368952203838 - type: nauc_map_at_100_std value: -5.6873032838969255 - type: nauc_map_at_10_diff1 value: 87.12042254029072 - type: nauc_map_at_10_max value: 76.71805257742122 - type: nauc_map_at_10_std value: -5.592480366964013 - type: nauc_map_at_1_diff1 value: 89.24798011187079 - type: nauc_map_at_1_max value: 75.83442039626006 - type: nauc_map_at_1_std value: -8.486603514571154 - type: nauc_map_at_20_diff1 value: 87.14327639577641 - type: nauc_map_at_20_max value: 76.68990390370077 - type: nauc_map_at_20_std value: -5.580217539312647 - type: nauc_map_at_3_diff1 value: 87.08395702335557 - type: nauc_map_at_3_max value: 76.49131087180318 - type: nauc_map_at_3_std value: -7.960457023584337 - type: nauc_map_at_5_diff1 value: 87.01407675116587 - type: nauc_map_at_5_max value: 76.79987365594535 - type: nauc_map_at_5_std value: -6.096086399895831 - type: nauc_mrr_at_1000_diff1 value: 87.1827200996951 - type: nauc_mrr_at_1000_max value: 76.68256630016205 - type: nauc_mrr_at_1000_std value: -5.730353903171156 - type: nauc_mrr_at_100_diff1 value: 87.17937163549941 - type: nauc_mrr_at_100_max value: 76.69368952203838 - type: nauc_mrr_at_100_std value: -5.6873032838969255 - type: nauc_mrr_at_10_diff1 value: 87.12042254029072 - type: nauc_mrr_at_10_max value: 76.71805257742122 - type: nauc_mrr_at_10_std value: -5.592480366964013 - type: nauc_mrr_at_1_diff1 value: 89.24798011187079 - type: nauc_mrr_at_1_max value: 75.83442039626006 - type: nauc_mrr_at_1_std value: -8.486603514571154 - type: nauc_mrr_at_20_diff1 value: 87.14327639577641 - type: nauc_mrr_at_20_max value: 76.68990390370077 - type: nauc_mrr_at_20_std value: -5.580217539312647 - type: nauc_mrr_at_3_diff1 value: 87.08395702335557 - type: nauc_mrr_at_3_max value: 76.49131087180318 - type: nauc_mrr_at_3_std value: -7.960457023584337 - type: nauc_mrr_at_5_diff1 value: 87.01407675116587 - type: nauc_mrr_at_5_max value: 76.79987365594535 - type: nauc_mrr_at_5_std value: -6.096086399895831 - type: nauc_ndcg_at_1000_diff1 value: 86.83526635543689 - type: nauc_ndcg_at_1000_max value: 76.87213223376376 - type: nauc_ndcg_at_1000_std value: -4.326702937186593 - type: nauc_ndcg_at_100_diff1 value: 86.77852107145551 - type: nauc_ndcg_at_100_max value: 77.13812696037 - type: nauc_ndcg_at_100_std value: -3.1580272337005453 - type: nauc_ndcg_at_10_diff1 value: 86.36604843288399 - type: nauc_ndcg_at_10_max value: 77.1221987239842 - type: nauc_ndcg_at_10_std value: -2.4207447014322576 - type: nauc_ndcg_at_1_diff1 value: 89.24798011187079 - type: nauc_ndcg_at_1_max value: 75.83442039626006 - type: nauc_ndcg_at_1_std value: -8.486603514571154 - type: nauc_ndcg_at_20_diff1 value: 86.44160384842657 - type: nauc_ndcg_at_20_max value: 77.02933401904579 - type: nauc_ndcg_at_20_std value: -2.1826470452412066 - type: nauc_ndcg_at_3_diff1 value: 86.2326519838603 - type: nauc_ndcg_at_3_max value: 76.62394641640242 - type: nauc_ndcg_at_3_std value: -7.766101601996052 - type: nauc_ndcg_at_5_diff1 value: 86.06089203598795 - type: nauc_ndcg_at_5_max value: 77.29478430133818 - type: nauc_ndcg_at_5_std value: -3.9125333318549913 - type: nauc_precision_at_1000_diff1 value: .nan - type: nauc_precision_at_1000_max value: .nan - type: nauc_precision_at_1000_std value: .nan - type: nauc_precision_at_100_diff1 value: 84.08579117921693 - type: nauc_precision_at_100_max value: 91.12978524743181 - type: nauc_precision_at_100_std value: 60.69368924040307 - type: nauc_precision_at_10_diff1 value: 81.55999773646823 - type: nauc_precision_at_10_max value: 80.02433296550915 - type: nauc_precision_at_10_std value: 22.35591206179418 - type: nauc_precision_at_1_diff1 value: 89.24798011187079 - type: nauc_precision_at_1_max value: 75.83442039626006 - type: nauc_precision_at_1_std value: -8.486603514571154 - type: nauc_precision_at_20_diff1 value: 80.35765030649968 - type: nauc_precision_at_20_max value: 80.32415864896703 - type: nauc_precision_at_20_std value: 36.235943652823465 - type: nauc_precision_at_3_diff1 value: 82.92001397135886 - type: nauc_precision_at_3_max value: 77.08808644406645 - type: nauc_precision_at_3_std value: -7.015034073765726 - type: nauc_precision_at_5_diff1 value: 81.50283822138118 - type: nauc_precision_at_5_max value: 79.7989593188268 - type: nauc_precision_at_5_std value: 7.5491958372753745 - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_100_diff1 value: 84.08579117921708 - type: nauc_recall_at_100_max value: 91.12978524743201 - type: nauc_recall_at_100_std value: 60.6936892404014 - type: nauc_recall_at_10_diff1 value: 81.55999773646843 - type: nauc_recall_at_10_max value: 80.02433296550944 - type: nauc_recall_at_10_std value: 22.355912061794637 - type: nauc_recall_at_1_diff1 value: 89.24798011187079 - type: nauc_recall_at_1_max value: 75.83442039626006 - type: nauc_recall_at_1_std value: -8.486603514571154 - type: nauc_recall_at_20_diff1 value: 80.35765030649934 - type: nauc_recall_at_20_max value: 80.32415864896662 - type: nauc_recall_at_20_std value: 36.23594365282353 - type: nauc_recall_at_3_diff1 value: 82.92001397135877 - type: nauc_recall_at_3_max value: 77.08808644406626 - type: nauc_recall_at_3_std value: -7.015034073765913 - type: nauc_recall_at_5_diff1 value: 81.50283822138132 - type: nauc_recall_at_5_max value: 79.7989593188269 - type: nauc_recall_at_5_std value: 7.549195837275434 - type: ndcg_at_1 value: 74.333 - type: ndcg_at_10 value: 83.504 - type: ndcg_at_100 value: 84.66 - type: ndcg_at_1000 value: 84.92 - type: ndcg_at_20 value: 84.065 - type: ndcg_at_3 value: 80.864 - type: ndcg_at_5 value: 82.284 - type: precision_at_1 value: 74.333 - type: precision_at_10 value: 9.267 - type: precision_at_100 value: 0.9809999999999999 - type: precision_at_1000 value: 0.1 - type: precision_at_20 value: 4.744000000000001 - type: precision_at_3 value: 28.481 - type: precision_at_5 value: 17.778 - type: recall_at_1 value: 74.333 - type: recall_at_10 value: 92.667 - type: recall_at_100 value: 98.111 - type: recall_at_1000 value: 100.0 - type: recall_at_20 value: 94.889 - type: recall_at_3 value: 85.444 - type: recall_at_5 value: 88.889 task: type: Retrieval - dataset: config: eng_Latn-pol_Latn name: MTEB BelebeleRetrieval (eng_Latn-pol_Latn) revision: 75b399394a9803252cfec289d103de462763db7c split: test type: facebook/belebele metrics: - type: main_score value: 89.445 - type: map_at_1 value: 83.0 - type: map_at_10 value: 87.421 - type: map_at_100 value: 87.563 - type: map_at_1000 value: 87.569 - type: map_at_20 value: 87.507 - type: map_at_3 value: 86.611 - type: map_at_5 value: 87.072 - type: mrr_at_1 value: 83.0 - type: mrr_at_10 value: 87.42085537918871 - type: mrr_at_100 value: 87.56310628611797 - type: mrr_at_1000 value: 87.56892446259369 - type: mrr_at_20 value: 87.50710137481293 - type: mrr_at_3 value: 86.61111111111111 - type: mrr_at_5 value: 87.07222222222222 - type: nauc_map_at_1000_diff1 value: 85.27894860675904 - type: nauc_map_at_1000_max value: 71.20075329739284 - type: nauc_map_at_1000_std value: -30.5553788367382 - type: nauc_map_at_100_diff1 value: 85.2804076974538 - type: nauc_map_at_100_max value: 71.20036875912842 - type: nauc_map_at_100_std value: -30.555551579374157 - type: nauc_map_at_10_diff1 value: 85.22909026521418 - type: nauc_map_at_10_max value: 71.3137782383334 - type: nauc_map_at_10_std value: -30.527384226196947 - type: nauc_map_at_1_diff1 value: 85.80361577110865 - type: nauc_map_at_1_max value: 67.3871119624879 - type: nauc_map_at_1_std value: -30.446069799990923 - type: nauc_map_at_20_diff1 value: 85.3024090163576 - type: nauc_map_at_20_max value: 71.26314736084274 - type: nauc_map_at_20_std value: -30.449354240689352 - type: nauc_map_at_3_diff1 value: 84.83736781234883 - type: nauc_map_at_3_max value: 71.61002582441802 - type: nauc_map_at_3_std value: -29.99465285790686 - type: nauc_map_at_5_diff1 value: 85.20235777737288 - type: nauc_map_at_5_max value: 71.46957069322528 - type: nauc_map_at_5_std value: -30.945869713231627 - type: nauc_mrr_at_1000_diff1 value: 85.27894860675904 - type: nauc_mrr_at_1000_max value: 71.20075329739284 - type: nauc_mrr_at_1000_std value: -30.5553788367382 - type: nauc_mrr_at_100_diff1 value: 85.2804076974538 - type: nauc_mrr_at_100_max value: 71.20036875912842 - type: nauc_mrr_at_100_std value: -30.555551579374157 - type: nauc_mrr_at_10_diff1 value: 85.22909026521418 - type: nauc_mrr_at_10_max value: 71.3137782383334 - type: nauc_mrr_at_10_std value: -30.527384226196947 - type: nauc_mrr_at_1_diff1 value: 85.80361577110865 - type: nauc_mrr_at_1_max value: 67.3871119624879 - type: nauc_mrr_at_1_std value: -30.446069799990923 - type: nauc_mrr_at_20_diff1 value: 85.3024090163576 - type: nauc_mrr_at_20_max value: 71.26314736084274 - type: nauc_mrr_at_20_std value: -30.449354240689352 - type: nauc_mrr_at_3_diff1 value: 84.83736781234883 - type: nauc_mrr_at_3_max value: 71.61002582441802 - type: nauc_mrr_at_3_std value: -29.99465285790686 - type: nauc_mrr_at_5_diff1 value: 85.20235777737288 - type: nauc_mrr_at_5_max value: 71.46957069322528 - type: nauc_mrr_at_5_std value: -30.945869713231627 - type: nauc_ndcg_at_1000_diff1 value: 85.28579610897238 - type: nauc_ndcg_at_1000_max value: 71.71856783729305 - type: nauc_ndcg_at_1000_std value: -30.549569322105352 - type: nauc_ndcg_at_100_diff1 value: 85.33303778055202 - type: nauc_ndcg_at_100_max value: 71.72213771654093 - type: nauc_ndcg_at_100_std value: -30.425199058298535 - type: nauc_ndcg_at_10_diff1 value: 85.20490226070947 - type: nauc_ndcg_at_10_max value: 72.52298735993796 - type: nauc_ndcg_at_10_std value: -30.24271239212584 - type: nauc_ndcg_at_1_diff1 value: 85.80361577110865 - type: nauc_ndcg_at_1_max value: 67.3871119624879 - type: nauc_ndcg_at_1_std value: -30.446069799990923 - type: nauc_ndcg_at_20_diff1 value: 85.48817468663597 - type: nauc_ndcg_at_20_max value: 72.3267173920573 - type: nauc_ndcg_at_20_std value: -29.808693982808173 - type: nauc_ndcg_at_3_diff1 value: 84.39869942100012 - type: nauc_ndcg_at_3_max value: 72.98018260317248 - type: nauc_ndcg_at_3_std value: -29.88649089998115 - type: nauc_ndcg_at_5_diff1 value: 85.08666261656307 - type: nauc_ndcg_at_5_max value: 72.80513750333559 - type: nauc_ndcg_at_5_std value: -31.764129823670928 - type: nauc_precision_at_1000_diff1 value: .nan - type: nauc_precision_at_1000_max value: .nan - type: nauc_precision_at_1000_std value: .nan - type: nauc_precision_at_100_diff1 value: 89.66253167933957 - type: nauc_precision_at_100_max value: 72.46231826063688 - type: nauc_precision_at_100_std value: -16.92677070828227 - type: nauc_precision_at_10_diff1 value: 85.47348764067038 - type: nauc_precision_at_10_max value: 81.89223057644087 - type: nauc_precision_at_10_std value: -26.041820236867007 - type: nauc_precision_at_1_diff1 value: 85.80361577110865 - type: nauc_precision_at_1_max value: 67.3871119624879 - type: nauc_precision_at_1_std value: -30.446069799990923 - type: nauc_precision_at_20_diff1 value: 88.99436317736968 - type: nauc_precision_at_20_max value: 83.0826157623541 - type: nauc_precision_at_20_std value: -18.139848532006024 - type: nauc_precision_at_3_diff1 value: 82.57426427361068 - type: nauc_precision_at_3_max value: 78.46422519625116 - type: nauc_precision_at_3_std value: -29.474588189185337 - type: nauc_precision_at_5_diff1 value: 84.62792524417179 - type: nauc_precision_at_5_max value: 79.38286425681366 - type: nauc_precision_at_5_std value: -36.41827101210859 - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_100_diff1 value: 89.66253167933883 - type: nauc_recall_at_100_max value: 72.46231826063716 - type: nauc_recall_at_100_std value: -16.926770708284298 - type: nauc_recall_at_10_diff1 value: 85.47348764067067 - type: nauc_recall_at_10_max value: 81.89223057644107 - type: nauc_recall_at_10_std value: -26.041820236866585 - type: nauc_recall_at_1_diff1 value: 85.80361577110865 - type: nauc_recall_at_1_max value: 67.3871119624879 - type: nauc_recall_at_1_std value: -30.446069799990923 - type: nauc_recall_at_20_diff1 value: 88.99436317736975 - type: nauc_recall_at_20_max value: 83.08261576235462 - type: nauc_recall_at_20_std value: -18.139848532005708 - type: nauc_recall_at_3_diff1 value: 82.57426427361058 - type: nauc_recall_at_3_max value: 78.46422519625135 - type: nauc_recall_at_3_std value: -29.47458818918515 - type: nauc_recall_at_5_diff1 value: 84.62792524417188 - type: nauc_recall_at_5_max value: 79.38286425681372 - type: nauc_recall_at_5_std value: -36.418271012108875 - type: ndcg_at_1 value: 83.0 - type: ndcg_at_10 value: 89.445 - type: ndcg_at_100 value: 90.167 - type: ndcg_at_1000 value: 90.277 - type: ndcg_at_20 value: 89.756 - type: ndcg_at_3 value: 87.742 - type: ndcg_at_5 value: 88.569 - type: precision_at_1 value: 83.0 - type: precision_at_10 value: 9.578000000000001 - type: precision_at_100 value: 0.992 - type: precision_at_1000 value: 0.1 - type: precision_at_20 value: 4.8500000000000005 - type: precision_at_3 value: 30.333 - type: precision_at_5 value: 18.6 - type: recall_at_1 value: 83.0 - type: recall_at_10 value: 95.77799999999999 - type: recall_at_100 value: 99.222 - type: recall_at_1000 value: 100.0 - type: recall_at_20 value: 97.0 - type: recall_at_3 value: 91.0 - type: recall_at_5 value: 93.0 task: type: Retrieval - dataset: config: eng_Latn-pol_Latn name: MTEB BibleNLPBitextMining (eng_Latn-pol_Latn) revision: 264a18480c529d9e922483839b4b9758e690b762 split: train type: davidstap/biblenlp-corpus-mmteb metrics: - type: accuracy value: 96.875 - type: f1 value: 95.83333333333333 - type: main_score value: 95.83333333333333 - type: precision value: 95.3125 - type: recall value: 96.875 task: type: BitextMining - dataset: config: pol_Latn-eng_Latn name: MTEB BibleNLPBitextMining (pol_Latn-eng_Latn) revision: 264a18480c529d9e922483839b4b9758e690b762 split: train type: davidstap/biblenlp-corpus-mmteb metrics: - type: accuracy value: 95.3125 - type: f1 value: 93.88020833333334 - type: main_score value: 93.88020833333334 - type: precision value: 93.1640625 - type: recall value: 95.3125 task: type: BitextMining - dataset: config: default name: MTEB CBD (default) revision: 36ddb419bcffe6a5374c3891957912892916f28d split: test type: PL-MTEB/cbd metrics: - type: accuracy value: 75.26 - type: ap value: 28.410218749169303 - type: ap_weighted value: 28.410218749169303 - type: f1 value: 64.90867824844531 - type: f1_weighted value: 78.81011528610803 - type: main_score value: 75.26 task: type: Classification - dataset: config: default name: MTEB CDSC-E (default) revision: 0a3d4aa409b22f80eb22cbf59b492637637b536d split: test type: PL-MTEB/cdsce-pairclassification metrics: - type: cosine_accuracy value: 89.4 - type: cosine_accuracy_threshold value: 94.002366065979 - type: cosine_ap value: 77.00188770290791 - type: cosine_f1 value: 68.07228915662651 - type: cosine_f1_threshold value: 94.002366065979 - type: cosine_precision value: 79.5774647887324 - type: cosine_recall value: 59.473684210526315 - type: dot_accuracy value: 89.4 - type: dot_accuracy_threshold value: 94.002366065979 - type: dot_ap value: 77.00188770290791 - type: dot_f1 value: 68.07228915662651 - type: dot_f1_threshold value: 94.002366065979 - type: dot_precision value: 79.5774647887324 - type: dot_recall value: 59.473684210526315 - type: euclidean_accuracy value: 89.4 - type: euclidean_accuracy_threshold value: 34.634196758270264 - type: euclidean_ap value: 77.00188770290791 - type: euclidean_f1 value: 68.07228915662651 - type: euclidean_f1_threshold value: 34.634196758270264 - type: euclidean_precision value: 79.5774647887324 - type: euclidean_recall value: 59.473684210526315 - type: main_score value: 77.06644109822335 - type: manhattan_accuracy value: 89.4 - type: manhattan_accuracy_threshold value: 755.0533294677734 - type: manhattan_ap value: 77.06644109822335 - type: manhattan_f1 value: 68.07228915662651 - type: manhattan_f1_threshold value: 755.0533294677734 - type: manhattan_precision value: 79.5774647887324 - type: manhattan_recall value: 59.473684210526315 - type: max_accuracy value: 89.4 - type: max_ap value: 77.06644109822335 - type: max_f1 value: 68.07228915662651 - type: max_precision value: 79.5774647887324 - type: max_recall value: 59.473684210526315 - type: similarity_accuracy value: 89.4 - type: similarity_accuracy_threshold value: 94.00235414505005 - type: similarity_ap value: 77.00188770290791 - type: similarity_f1 value: 68.07228915662651 - type: similarity_f1_threshold value: 94.00235414505005 - type: similarity_precision value: 79.5774647887324 - type: similarity_recall value: 59.473684210526315 task: type: PairClassification - dataset: config: default name: MTEB CDSC-R (default) revision: 1cd6abbb00df7d14be3dbd76a7dcc64b3a79a7cd split: test type: PL-MTEB/cdscr-sts metrics: - type: cosine_pearson value: 91.17119649361847 - type: cosine_spearman value: 91.20577905928302 - type: euclidean_pearson value: 90.5668852344143 - type: euclidean_spearman value: 91.20577905928302 - type: main_score value: 91.20577905928302 - type: manhattan_pearson value: 90.6503906128131 - type: manhattan_spearman value: 91.30706035965189 - type: pearson value: 91.1711959701339 - type: spearman value: 91.20576339049457 task: type: STS - dataset: config: default name: MTEB DBPedia-PL (default) revision: 76afe41d9af165cc40999fcaa92312b8b012064a split: test type: clarin-knext/dbpedia-pl metrics: - type: main_score value: 31.115 - type: map_at_1 value: 6.665 - type: map_at_10 value: 13.611999999999998 - type: map_at_100 value: 18.631 - type: map_at_1000 value: 19.973 - type: map_at_20 value: 15.482000000000001 - type: map_at_3 value: 10.191 - type: map_at_5 value: 11.518 - type: mrr_at_1 value: 49.75 - type: mrr_at_10 value: 61.24871031746032 - type: mrr_at_100 value: 61.6639721612659 - type: mrr_at_1000 value: 61.68652629644693 - type: mrr_at_20 value: 61.526411705339726 - type: mrr_at_3 value: 59.62499999999999 - type: mrr_at_5 value: 60.5125 - type: nauc_map_at_1000_diff1 value: 27.97392366194627 - type: nauc_map_at_1000_max value: 30.16452498569316 - type: nauc_map_at_1000_std value: 29.201329015427007 - type: nauc_map_at_100_diff1 value: 29.367174741920238 - type: nauc_map_at_100_max value: 28.973659071057366 - type: nauc_map_at_100_std value: 26.31505365178391 - type: nauc_map_at_10_diff1 value: 36.69851589594083 - type: nauc_map_at_10_max value: 20.25364813839131 - type: nauc_map_at_10_std value: 9.574222619557707 - type: nauc_map_at_1_diff1 value: 47.36289088982068 - type: nauc_map_at_1_max value: 16.040304183649507 - type: nauc_map_at_1_std value: 2.4515925926521254 - type: nauc_map_at_20_diff1 value: 34.08015789582933 - type: nauc_map_at_20_max value: 23.19631824118355 - type: nauc_map_at_20_std value: 15.110110478483914 - type: nauc_map_at_3_diff1 value: 38.7674548696569 - type: nauc_map_at_3_max value: 13.542163903569318 - type: nauc_map_at_3_std value: 2.134613354826337 - type: nauc_map_at_5_diff1 value: 37.95693744467431 - type: nauc_map_at_5_max value: 16.05432858439634 - type: nauc_map_at_5_std value: 4.590544603710069 - type: nauc_mrr_at_1000_diff1 value: 44.295123899826386 - type: nauc_mrr_at_1000_max value: 54.69332333119172 - type: nauc_mrr_at_1000_std value: 33.77442293639657 - type: nauc_mrr_at_100_diff1 value: 44.305525925983034 - type: nauc_mrr_at_100_max value: 54.701484072630976 - type: nauc_mrr_at_100_std value: 33.77276371639727 - type: nauc_mrr_at_10_diff1 value: 44.26301636684395 - type: nauc_mrr_at_10_max value: 54.853048563594875 - type: nauc_mrr_at_10_std value: 33.92811076621932 - type: nauc_mrr_at_1_diff1 value: 45.93444117911359 - type: nauc_mrr_at_1_max value: 51.62632337425907 - type: nauc_mrr_at_1_std value: 29.68452022378797 - type: nauc_mrr_at_20_diff1 value: 44.30198592107031 - type: nauc_mrr_at_20_max value: 54.758257064607385 - type: nauc_mrr_at_20_std value: 33.85765225017787 - type: nauc_mrr_at_3_diff1 value: 44.300646770830745 - type: nauc_mrr_at_3_max value: 55.05487704215323 - type: nauc_mrr_at_3_std value: 33.6243706461525 - type: nauc_mrr_at_5_diff1 value: 44.254814329835774 - type: nauc_mrr_at_5_max value: 55.429557231467555 - type: nauc_mrr_at_5_std value: 33.89131816053683 - type: nauc_ndcg_at_1000_diff1 value: 26.532231272555943 - type: nauc_ndcg_at_1000_max value: 42.82810443913827 - type: nauc_ndcg_at_1000_std value: 41.786347836419175 - type: nauc_ndcg_at_100_diff1 value: 29.769506208591867 - type: nauc_ndcg_at_100_max value: 38.53042002334068 - type: nauc_ndcg_at_100_std value: 33.887665540688886 - type: nauc_ndcg_at_10_diff1 value: 31.732079187467072 - type: nauc_ndcg_at_10_max value: 39.82093012860551 - type: nauc_ndcg_at_10_std value: 29.1806509581709 - type: nauc_ndcg_at_1_diff1 value: 39.54320133436284 - type: nauc_ndcg_at_1_max value: 42.045131681424216 - type: nauc_ndcg_at_1_std value: 25.077965965725056 - type: nauc_ndcg_at_20_diff1 value: 31.45644544950967 - type: nauc_ndcg_at_20_max value: 36.28423421245746 - type: nauc_ndcg_at_20_std value: 28.30688750237228 - type: nauc_ndcg_at_3_diff1 value: 29.70854623710537 - type: nauc_ndcg_at_3_max value: 39.94336409131591 - type: nauc_ndcg_at_3_std value: 27.22757989754428 - type: nauc_ndcg_at_5_diff1 value: 30.651845085418376 - type: nauc_ndcg_at_5_max value: 40.37224782227242 - type: nauc_ndcg_at_5_std value: 27.502645528938817 - type: nauc_precision_at_1000_diff1 value: -13.927590619742706 - type: nauc_precision_at_1000_max value: 3.1836041302862386 - type: nauc_precision_at_1000_std value: 15.58030701421384 - type: nauc_precision_at_100_diff1 value: -6.758322244317817 - type: nauc_precision_at_100_max value: 28.118442940968187 - type: nauc_precision_at_100_std value: 43.53996850999107 - type: nauc_precision_at_10_diff1 value: 7.804251693756052 - type: nauc_precision_at_10_max value: 39.42439336826685 - type: nauc_precision_at_10_std value: 41.41138537059063 - type: nauc_precision_at_1_diff1 value: 45.93444117911359 - type: nauc_precision_at_1_max value: 51.62632337425907 - type: nauc_precision_at_1_std value: 29.68452022378797 - type: nauc_precision_at_20_diff1 value: -0.12893175925907663 - type: nauc_precision_at_20_max value: 35.05865541707333 - type: nauc_precision_at_20_std value: 43.92422660338514 - type: nauc_precision_at_3_diff1 value: 16.156714450361957 - type: nauc_precision_at_3_max value: 39.392697606797675 - type: nauc_precision_at_3_std value: 35.04515973514324 - type: nauc_precision_at_5_diff1 value: 11.96106707384123 - type: nauc_precision_at_5_max value: 38.93920971689429 - type: nauc_precision_at_5_std value: 36.95318644468955 - type: nauc_recall_at_1000_diff1 value: 13.88981638274071 - type: nauc_recall_at_1000_max value: 31.97503175789552 - type: nauc_recall_at_1000_std value: 46.86459640727557 - type: nauc_recall_at_100_diff1 value: 21.231813262732814 - type: nauc_recall_at_100_max value: 27.607024026953887 - type: nauc_recall_at_100_std value: 33.43815591518041 - type: nauc_recall_at_10_diff1 value: 31.161072928150496 - type: nauc_recall_at_10_max value: 16.792954450120913 - type: nauc_recall_at_10_std value: 7.9865023792723395 - type: nauc_recall_at_1_diff1 value: 47.36289088982068 - type: nauc_recall_at_1_max value: 16.040304183649507 - type: nauc_recall_at_1_std value: 2.4515925926521254 - type: nauc_recall_at_20_diff1 value: 25.765142923465966 - type: nauc_recall_at_20_max value: 16.296695698388213 - type: nauc_recall_at_20_std value: 12.842983647767928 - type: nauc_recall_at_3_diff1 value: 32.22545303434119 - type: nauc_recall_at_3_max value: 8.911149866544205 - type: nauc_recall_at_3_std value: 0.6114376548438478 - type: nauc_recall_at_5_diff1 value: 31.513268164295926 - type: nauc_recall_at_5_max value: 11.88810794407339 - type: nauc_recall_at_5_std value: 2.2459671612376306 - type: ndcg_at_1 value: 40.5 - type: ndcg_at_10 value: 31.115 - type: ndcg_at_100 value: 34.508 - type: ndcg_at_1000 value: 41.71 - type: ndcg_at_20 value: 30.505 - type: ndcg_at_3 value: 35.303000000000004 - type: ndcg_at_5 value: 32.547 - type: precision_at_1 value: 49.75 - type: precision_at_10 value: 24.7 - type: precision_at_100 value: 8.012 - type: precision_at_1000 value: 1.818 - type: precision_at_20 value: 18.425 - type: precision_at_3 value: 38.417 - type: precision_at_5 value: 31.35 - type: recall_at_1 value: 6.665 - type: recall_at_10 value: 18.529999999999998 - type: recall_at_100 value: 39.309 - type: recall_at_1000 value: 63.085 - type: recall_at_20 value: 24.237000000000002 - type: recall_at_3 value: 11.648 - type: recall_at_5 value: 13.969000000000001 task: type: Retrieval - dataset: config: default name: MTEB DBPedia-PLHardNegatives (default) revision: bebc2b5c8f73cd6ba9d2a4664d5f3769e6ad557a split: test type: mteb/DBPedia_PL_test_top_250_only_w_correct-v2 metrics: - type: main_score value: 36.334 - type: map_at_1 value: 7.4270000000000005 - type: map_at_10 value: 15.809999999999999 - type: map_at_100 value: 25.295 - type: map_at_1000 value: 28.994999999999997 - type: map_at_20 value: 18.598 - type: map_at_3 value: 11.225999999999999 - type: map_at_5 value: 13.075000000000001 - type: mrr_at_1 value: 56.49999999999999 - type: mrr_at_10 value: 67.59107142857142 - type: mrr_at_100 value: 67.97602561595816 - type: mrr_at_1000 value: 67.98142601983503 - type: mrr_at_20 value: 67.80010691385498 - type: mrr_at_3 value: 65.5 - type: mrr_at_5 value: 66.825 - type: nauc_map_at_1000_diff1 value: 19.03489213465321 - type: nauc_map_at_1000_max value: 24.31248855223973 - type: nauc_map_at_1000_std value: 7.6486353701964065 - type: nauc_map_at_100_diff1 value: 23.803948576163872 - type: nauc_map_at_100_max value: 26.064916066562972 - type: nauc_map_at_100_std value: 4.5603234154703065 - type: nauc_map_at_10_diff1 value: 34.852165552762905 - type: nauc_map_at_10_max value: 15.937406490323783 - type: nauc_map_at_10_std value: -11.21886485917206 - type: nauc_map_at_1_diff1 value: 43.611175943667455 - type: nauc_map_at_1_max value: 10.29490789808112 - type: nauc_map_at_1_std value: -15.47370286445684 - type: nauc_map_at_20_diff1 value: 31.63917106452436 - type: nauc_map_at_20_max value: 20.02214953017864 - type: nauc_map_at_20_std value: -5.944642243477304 - type: nauc_map_at_3_diff1 value: 38.40895845560947 - type: nauc_map_at_3_max value: 11.73261168022514 - type: nauc_map_at_3_std value: -15.957596656985743 - type: nauc_map_at_5_diff1 value: 37.74389955615709 - type: nauc_map_at_5_max value: 12.530750051473461 - type: nauc_map_at_5_std value: -14.904446849665357 - type: nauc_mrr_at_1000_diff1 value: 37.3255053078257 - type: nauc_mrr_at_1000_max value: 50.092590673362835 - type: nauc_mrr_at_1000_std value: 24.465063640168896 - type: nauc_mrr_at_100_diff1 value: 37.317631959351154 - type: nauc_mrr_at_100_max value: 50.089186894932446 - type: nauc_mrr_at_100_std value: 24.47297127967704 - type: nauc_mrr_at_10_diff1 value: 37.26619954875676 - type: nauc_mrr_at_10_max value: 50.35898421281153 - type: nauc_mrr_at_10_std value: 24.53041743892045 - type: nauc_mrr_at_1_diff1 value: 38.257605492082334 - type: nauc_mrr_at_1_max value: 45.15981702574061 - type: nauc_mrr_at_1_std value: 19.840646832561436 - type: nauc_mrr_at_20_diff1 value: 37.43862334380517 - type: nauc_mrr_at_20_max value: 50.173131837502375 - type: nauc_mrr_at_20_std value: 24.539969193655452 - type: nauc_mrr_at_3_diff1 value: 37.88117659952954 - type: nauc_mrr_at_3_max value: 51.60950598444801 - type: nauc_mrr_at_3_std value: 25.166999682440615 - type: nauc_mrr_at_5_diff1 value: 37.958493437638026 - type: nauc_mrr_at_5_max value: 51.21047679102041 - type: nauc_mrr_at_5_std value: 24.69873551695111 - type: nauc_ndcg_at_1000_diff1 value: 18.79475374007571 - type: nauc_ndcg_at_1000_max value: 38.41026674964163 - type: nauc_ndcg_at_1000_std value: 26.689990462447888 - type: nauc_ndcg_at_100_diff1 value: 21.578286919038224 - type: nauc_ndcg_at_100_max value: 33.60574596089818 - type: nauc_ndcg_at_100_std value: 13.551338023245407 - type: nauc_ndcg_at_10_diff1 value: 25.57361673709202 - type: nauc_ndcg_at_10_max value: 32.54833248349918 - type: nauc_ndcg_at_10_std value: 8.51937062532666 - type: nauc_ndcg_at_1_diff1 value: 31.81949553561505 - type: nauc_ndcg_at_1_max value: 34.38816292397119 - type: nauc_ndcg_at_1_std value: 13.948958424185824 - type: nauc_ndcg_at_20_diff1 value: 26.103449829594766 - type: nauc_ndcg_at_20_max value: 30.71124051388171 - type: nauc_ndcg_at_20_std value: 7.071826626406438 - type: nauc_ndcg_at_3_diff1 value: 23.45723135474484 - type: nauc_ndcg_at_3_max value: 34.80165664743321 - type: nauc_ndcg_at_3_std value: 11.811364107953635 - type: nauc_ndcg_at_5_diff1 value: 25.820575290072824 - type: nauc_ndcg_at_5_max value: 33.57773696590368 - type: nauc_ndcg_at_5_std value: 10.115752058469747 - type: nauc_precision_at_1000_diff1 value: -16.854445390790353 - type: nauc_precision_at_1000_max value: -10.816821804734126 - type: nauc_precision_at_1000_std value: -3.569694723301324 - type: nauc_precision_at_100_diff1 value: -17.072761519475527 - type: nauc_precision_at_100_max value: 10.65661794744656 - type: nauc_precision_at_100_std value: 20.505610434684947 - type: nauc_precision_at_10_diff1 value: -0.9054613184930507 - type: nauc_precision_at_10_max value: 30.391584829217706 - type: nauc_precision_at_10_std value: 24.808586474275025 - type: nauc_precision_at_1_diff1 value: 38.257605492082334 - type: nauc_precision_at_1_max value: 45.15981702574061 - type: nauc_precision_at_1_std value: 19.840646832561436 - type: nauc_precision_at_20_diff1 value: -7.2709845751747615 - type: nauc_precision_at_20_max value: 25.023088848967213 - type: nauc_precision_at_20_std value: 24.518659186798256 - type: nauc_precision_at_3_diff1 value: 9.282149063740338 - type: nauc_precision_at_3_max value: 34.52938117524703 - type: nauc_precision_at_3_std value: 18.985796468895796 - type: nauc_precision_at_5_diff1 value: 5.981279708090407 - type: nauc_precision_at_5_max value: 31.74568145808379 - type: nauc_precision_at_5_std value: 21.190093133655658 - type: nauc_recall_at_1000_diff1 value: 9.56810937763011 - type: nauc_recall_at_1000_max value: 36.88125925346485 - type: nauc_recall_at_1000_std value: 54.516273733852096 - type: nauc_recall_at_100_diff1 value: 12.66136659540159 - type: nauc_recall_at_100_max value: 21.842810710889896 - type: nauc_recall_at_100_std value: 11.099322582373153 - type: nauc_recall_at_10_diff1 value: 27.227630640264373 - type: nauc_recall_at_10_max value: 8.920146050576527 - type: nauc_recall_at_10_std value: -13.094871094982471 - type: nauc_recall_at_1_diff1 value: 43.611175943667455 - type: nauc_recall_at_1_max value: 10.29490789808112 - type: nauc_recall_at_1_std value: -15.47370286445684 - type: nauc_recall_at_20_diff1 value: 24.213720610882994 - type: nauc_recall_at_20_max value: 13.984478640047193 - type: nauc_recall_at_20_std value: -6.873995197727176 - type: nauc_recall_at_3_diff1 value: 34.7580051454075 - type: nauc_recall_at_3_max value: 9.41880541336725 - type: nauc_recall_at_3_std value: -16.155623757406694 - type: nauc_recall_at_5_diff1 value: 33.9246093125407 - type: nauc_recall_at_5_max value: 7.8188437738274335 - type: nauc_recall_at_5_std value: -16.27760332961706 - type: ndcg_at_1 value: 46.125 - type: ndcg_at_10 value: 36.334 - type: ndcg_at_100 value: 45.507 - type: ndcg_at_1000 value: 55.42 - type: ndcg_at_20 value: 36.356 - type: ndcg_at_3 value: 39.544000000000004 - type: ndcg_at_5 value: 37.282 - type: precision_at_1 value: 56.49999999999999 - type: precision_at_10 value: 30.099999999999998 - type: precision_at_100 value: 13.295000000000002 - type: precision_at_1000 value: 3.107 - type: precision_at_20 value: 24.05 - type: precision_at_3 value: 43.25 - type: precision_at_5 value: 36.7 - type: recall_at_1 value: 7.4270000000000005 - type: recall_at_10 value: 21.939 - type: recall_at_100 value: 57.275 - type: recall_at_1000 value: 87.541 - type: recall_at_20 value: 28.887 - type: recall_at_3 value: 12.488000000000001 - type: recall_at_5 value: 16.012 task: type: Retrieval - dataset: config: default name: MTEB EightTagsClustering (default) revision: 78b962b130c6690659c65abf67bf1c2f030606b6 split: test type: PL-MTEB/8tags-clustering metrics: - type: main_score value: 29.45128843333412 - type: v_measure value: 29.45128843333412 - type: v_measure_std value: 3.041162560667918 task: type: Clustering - dataset: config: default name: MTEB 8TagsClustering (default) revision: 78b962b130c6690659c65abf67bf1c2f030606b6 split: test type: PL-MTEB/8tags-clustering metrics: - type: main_score value: 29.8763373396929 - type: v_measure value: 29.8763373396929 - type: v_measure_std value: 2.1856457067601442 task: type: Clustering - dataset: config: default name: MTEB FiQA-PL (default) revision: 2e535829717f8bf9dc829b7f911cc5bbd4e6608e split: test type: clarin-knext/fiqa-pl metrics: - type: main_score value: 35.528 - type: map_at_1 value: 17.557000000000002 - type: map_at_10 value: 28.286 - type: map_at_100 value: 30.034 - type: map_at_1000 value: 30.23 - type: map_at_20 value: 29.26 - type: map_at_3 value: 24.845 - type: map_at_5 value: 26.499 - type: mrr_at_1 value: 34.72222222222222 - type: mrr_at_10 value: 43.146923378404864 - type: mrr_at_100 value: 44.04815902024235 - type: mrr_at_1000 value: 44.097903716068586 - type: mrr_at_20 value: 43.7156255652916 - type: mrr_at_3 value: 40.715020576131685 - type: mrr_at_5 value: 41.92644032921811 - type: nauc_map_at_1000_diff1 value: 33.644270831543075 - type: nauc_map_at_1000_max value: 24.03453792787825 - type: nauc_map_at_1000_std value: 1.97615287272658 - type: nauc_map_at_100_diff1 value: 33.6032740378985 - type: nauc_map_at_100_max value: 23.89185578371163 - type: nauc_map_at_100_std value: 1.8654578877539438 - type: nauc_map_at_10_diff1 value: 33.883307076533676 - type: nauc_map_at_10_max value: 22.682305975492028 - type: nauc_map_at_10_std value: 0.8046906577573549 - type: nauc_map_at_1_diff1 value: 37.655216700037144 - type: nauc_map_at_1_max value: 16.668551169596526 - type: nauc_map_at_1_std value: -4.0646987080934895 - type: nauc_map_at_20_diff1 value: 33.58718564397087 - type: nauc_map_at_20_max value: 23.389386039635035 - type: nauc_map_at_20_std value: 1.395995783937243 - type: nauc_map_at_3_diff1 value: 34.35572697234372 - type: nauc_map_at_3_max value: 20.45983616887038 - type: nauc_map_at_3_std value: -1.2736797152737924 - type: nauc_map_at_5_diff1 value: 34.34016596200666 - type: nauc_map_at_5_max value: 22.418291988627736 - type: nauc_map_at_5_std value: -0.44701086507254306 - type: nauc_mrr_at_1000_diff1 value: 36.19004469292243 - type: nauc_mrr_at_1000_max value: 30.56126525504521 - type: nauc_mrr_at_1000_std value: 3.4741128332372306 - type: nauc_mrr_at_100_diff1 value: 36.184227341579984 - type: nauc_mrr_at_100_max value: 30.550618338595836 - type: nauc_mrr_at_100_std value: 3.462976738014996 - type: nauc_mrr_at_10_diff1 value: 36.30215412647522 - type: nauc_mrr_at_10_max value: 30.487801430449498 - type: nauc_mrr_at_10_std value: 3.448188439511118 - type: nauc_mrr_at_1_diff1 value: 38.95670761428797 - type: nauc_mrr_at_1_max value: 29.4564169058169 - type: nauc_mrr_at_1_std value: -0.33742039884770314 - type: nauc_mrr_at_20_diff1 value: 36.22848718875828 - type: nauc_mrr_at_20_max value: 30.594947294958303 - type: nauc_mrr_at_20_std value: 3.4116016526988298 - type: nauc_mrr_at_3_diff1 value: 36.34985197043398 - type: nauc_mrr_at_3_max value: 30.532575216956936 - type: nauc_mrr_at_3_std value: 2.6086817144501344 - type: nauc_mrr_at_5_diff1 value: 36.43204592281127 - type: nauc_mrr_at_5_max value: 30.510055166263804 - type: nauc_mrr_at_5_std value: 2.7846933372715474 - type: nauc_ndcg_at_1000_diff1 value: 33.569589146097606 - type: nauc_ndcg_at_1000_max value: 28.44966024793093 - type: nauc_ndcg_at_1000_std value: 6.687377760452772 - type: nauc_ndcg_at_100_diff1 value: 33.41392761635845 - type: nauc_ndcg_at_100_max value: 26.802853552558226 - type: nauc_ndcg_at_100_std value: 5.728593254623888 - type: nauc_ndcg_at_10_diff1 value: 33.688317854955066 - type: nauc_ndcg_at_10_max value: 24.328331387672677 - type: nauc_ndcg_at_10_std value: 3.2688582818838525 - type: nauc_ndcg_at_1_diff1 value: 38.95670761428797 - type: nauc_ndcg_at_1_max value: 29.4564169058169 - type: nauc_ndcg_at_1_std value: -0.33742039884770314 - type: nauc_ndcg_at_20_diff1 value: 33.06908590284782 - type: nauc_ndcg_at_20_max value: 25.552680732497357 - type: nauc_ndcg_at_20_std value: 4.212872336532075 - type: nauc_ndcg_at_3_diff1 value: 33.802719939106936 - type: nauc_ndcg_at_3_max value: 26.786920515224622 - type: nauc_ndcg_at_3_std value: 1.997987618355165 - type: nauc_ndcg_at_5_diff1 value: 34.160233314172075 - type: nauc_ndcg_at_5_max value: 25.400020591229104 - type: nauc_ndcg_at_5_std value: 1.7012614030206177 - type: nauc_precision_at_1000_diff1 value: 2.513296682036913 - type: nauc_precision_at_1000_max value: 29.943306602524 - type: nauc_precision_at_1000_std value: 14.284110752896906 - type: nauc_precision_at_100_diff1 value: 9.00849244829456 - type: nauc_precision_at_100_max value: 32.061761129561845 - type: nauc_precision_at_100_std value: 15.263912490326792 - type: nauc_precision_at_10_diff1 value: 18.545801394437728 - type: nauc_precision_at_10_max value: 33.1450392767742 - type: nauc_precision_at_10_std value: 12.798727612731073 - type: nauc_precision_at_1_diff1 value: 38.95670761428797 - type: nauc_precision_at_1_max value: 29.4564169058169 - type: nauc_precision_at_1_std value: -0.33742039884770314 - type: nauc_precision_at_20_diff1 value: 14.32446132754539 - type: nauc_precision_at_20_max value: 33.691221995211606 - type: nauc_precision_at_20_std value: 14.279203240881522 - type: nauc_precision_at_3_diff1 value: 24.812141928018846 - type: nauc_precision_at_3_max value: 32.503439032877026 - type: nauc_precision_at_3_std value: 7.7068527544857215 - type: nauc_precision_at_5_diff1 value: 23.78762563464159 - type: nauc_precision_at_5_max value: 34.70036993285849 - type: nauc_precision_at_5_std value: 8.956553891218908 - type: nauc_recall_at_1000_diff1 value: 20.97047749110538 - type: nauc_recall_at_1000_max value: 32.50998840246347 - type: nauc_recall_at_1000_std value: 36.74691034137564 - type: nauc_recall_at_100_diff1 value: 24.410642765125317 - type: nauc_recall_at_100_max value: 19.3720875912961 - type: nauc_recall_at_100_std value: 14.436674160770957 - type: nauc_recall_at_10_diff1 value: 27.25521943152881 - type: nauc_recall_at_10_max value: 16.53742681254563 - type: nauc_recall_at_10_std value: 5.693616949046378 - type: nauc_recall_at_1_diff1 value: 37.655216700037144 - type: nauc_recall_at_1_max value: 16.668551169596526 - type: nauc_recall_at_1_std value: -4.0646987080934895 - type: nauc_recall_at_20_diff1 value: 24.374626441398036 - type: nauc_recall_at_20_max value: 18.945220734293102 - type: nauc_recall_at_20_std value: 8.277345594606045 - type: nauc_recall_at_3_diff1 value: 30.295438911225535 - type: nauc_recall_at_3_max value: 17.72576332782552 - type: nauc_recall_at_3_std value: -0.4150602462194551 - type: nauc_recall_at_5_diff1 value: 29.147471255769474 - type: nauc_recall_at_5_max value: 18.160645721922407 - type: nauc_recall_at_5_std value: 1.184845715848266 - type: ndcg_at_1 value: 34.721999999999994 - type: ndcg_at_10 value: 35.528 - type: ndcg_at_100 value: 42.251 - type: ndcg_at_1000 value: 45.693 - type: ndcg_at_20 value: 38.216 - type: ndcg_at_3 value: 32.151 - type: ndcg_at_5 value: 32.727000000000004 - type: precision_at_1 value: 34.721999999999994 - type: precision_at_10 value: 9.846 - type: precision_at_100 value: 1.662 - type: precision_at_1000 value: 0.22699999999999998 - type: precision_at_20 value: 6.010999999999999 - type: precision_at_3 value: 21.348 - type: precision_at_5 value: 15.123000000000001 - type: recall_at_1 value: 17.557000000000002 - type: recall_at_10 value: 42.065000000000005 - type: recall_at_100 value: 67.207 - type: recall_at_1000 value: 88.016 - type: recall_at_20 value: 50.345 - type: recall_at_3 value: 29.479 - type: recall_at_5 value: 33.992 task: type: Retrieval - dataset: config: ace_Arab-pol_Latn name: MTEB FloresBitextMining (ace_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 0.592885375494071 - type: f1 value: 0.3181350268903164 - type: main_score value: 0.3181350268903164 - type: precision value: 0.30839411484572776 - type: recall value: 0.592885375494071 task: type: BitextMining - dataset: config: bam_Latn-pol_Latn name: MTEB FloresBitextMining (bam_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.727272727272727 - type: f1 value: 18.856205163673437 - type: main_score value: 18.856205163673437 - type: precision value: 17.853574155086342 - type: recall value: 22.727272727272727 task: type: BitextMining - dataset: config: dzo_Tibt-pol_Latn name: MTEB FloresBitextMining (dzo_Tibt-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 0.1976284584980237 - type: f1 value: 0.09900990099009901 - type: main_score value: 0.09900990099009901 - type: precision value: 0.09891216208275716 - type: recall value: 0.1976284584980237 task: type: BitextMining - dataset: config: hin_Deva-pol_Latn name: MTEB FloresBitextMining (hin_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.9525691699604746 - type: f1 value: 2.897496113899276 - type: main_score value: 2.897496113899276 - type: precision value: 2.611381778726582 - type: recall value: 3.9525691699604746 task: type: BitextMining - dataset: config: khm_Khmr-pol_Latn name: MTEB FloresBitextMining (khm_Khmr-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.418972332015812 - type: f1 value: 21.422759388174406 - type: main_score value: 21.422759388174406 - type: precision value: 20.7400345871417 - type: recall value: 23.418972332015812 task: type: BitextMining - dataset: config: mag_Deva-pol_Latn name: MTEB FloresBitextMining (mag_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.458498023715415 - type: f1 value: 2.622018112486536 - type: main_score value: 2.622018112486536 - type: precision value: 2.4537595577858893 - type: recall value: 3.458498023715415 task: type: BitextMining - dataset: config: pap_Latn-pol_Latn name: MTEB FloresBitextMining (pap_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 39.03162055335969 - type: f1 value: 35.30015219760418 - type: main_score value: 35.30015219760418 - type: precision value: 34.15844644794522 - type: recall value: 39.03162055335969 task: type: BitextMining - dataset: config: sot_Latn-pol_Latn name: MTEB FloresBitextMining (sot_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 20.158102766798418 - type: f1 value: 17.69223633337849 - type: main_score value: 17.69223633337849 - type: precision value: 17.098070294852068 - type: recall value: 20.158102766798418 task: type: BitextMining - dataset: config: tur_Latn-pol_Latn name: MTEB FloresBitextMining (tur_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.703557312252965 - type: f1 value: 21.29177122803851 - type: main_score value: 21.29177122803851 - type: precision value: 20.34792343942746 - type: recall value: 24.703557312252965 task: type: BitextMining - dataset: config: ace_Latn-pol_Latn name: MTEB FloresBitextMining (ace_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.889328063241106 - type: f1 value: 23.06308953172751 - type: main_score value: 23.06308953172751 - type: precision value: 22.24893068053073 - type: recall value: 25.889328063241106 task: type: BitextMining - dataset: config: ban_Latn-pol_Latn name: MTEB FloresBitextMining (ban_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.494071146245062 - type: f1 value: 22.007814923395514 - type: main_score value: 22.007814923395514 - type: precision value: 21.18801029853241 - type: recall value: 25.494071146245062 task: type: BitextMining - dataset: config: ell_Grek-pol_Latn name: MTEB FloresBitextMining (ell_Grek-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.102766798418973 - type: f1 value: 6.988965766514098 - type: main_score value: 6.988965766514098 - type: precision value: 6.712602330760323 - type: recall value: 8.102766798418973 task: type: BitextMining - dataset: config: hne_Deva-pol_Latn name: MTEB FloresBitextMining (hne_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.371541501976284 - type: f1 value: 1.6313787028973608 - type: main_score value: 1.6313787028973608 - type: precision value: 1.4932697009729146 - type: recall value: 2.371541501976284 task: type: BitextMining - dataset: config: kik_Latn-pol_Latn name: MTEB FloresBitextMining (kik_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.31225296442688 - type: f1 value: 27.111449621000936 - type: main_score value: 27.111449621000936 - type: precision value: 25.53133982182309 - type: recall value: 32.31225296442688 task: type: BitextMining - dataset: config: mai_Deva-pol_Latn name: MTEB FloresBitextMining (mai_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.0395256916996045 - type: f1 value: 3.966855997318039 - type: main_score value: 3.966855997318039 - type: precision value: 3.6789065258502127 - type: recall value: 5.0395256916996045 task: type: BitextMining - dataset: config: pbt_Arab-pol_Latn name: MTEB FloresBitextMining (pbt_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.782608695652174 - type: f1 value: 9.121288821137288 - type: main_score value: 9.121288821137288 - type: precision value: 8.943568976230727 - type: recall value: 9.782608695652174 task: type: BitextMining - dataset: config: spa_Latn-pol_Latn name: MTEB FloresBitextMining (spa_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 31.422924901185773 - type: f1 value: 29.66768938484047 - type: main_score value: 29.66768938484047 - type: precision value: 29.105673047225345 - type: recall value: 31.422924901185773 task: type: BitextMining - dataset: config: twi_Latn-pol_Latn name: MTEB FloresBitextMining (twi_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 36.16600790513834 - type: f1 value: 31.626095086908773 - type: main_score value: 31.626095086908773 - type: precision value: 30.198784888700832 - type: recall value: 36.16600790513834 task: type: BitextMining - dataset: config: acm_Arab-pol_Latn name: MTEB FloresBitextMining (acm_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.557312252964427 - type: f1 value: 2.721978575559138 - type: main_score value: 2.721978575559138 - type: precision value: 2.5283071397201833 - type: recall value: 3.557312252964427 task: type: BitextMining - dataset: config: bel_Cyrl-pol_Latn name: MTEB FloresBitextMining (bel_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.6561264822134385 - type: f1 value: 2.831898415680433 - type: main_score value: 2.831898415680433 - type: precision value: 2.683681277663466 - type: recall value: 3.6561264822134385 task: type: BitextMining - dataset: config: eng_Latn-pol_Latn name: MTEB FloresBitextMining (eng_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 99.90118577075098 - type: f1 value: 99.86824769433464 - type: main_score value: 99.86824769433464 - type: precision value: 99.85177865612648 - type: recall value: 99.90118577075098 task: type: BitextMining - dataset: config: hrv_Latn-pol_Latn name: MTEB FloresBitextMining (hrv_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 39.52569169960474 - type: f1 value: 34.67037471251808 - type: main_score value: 34.67037471251808 - type: precision value: 33.32842236446319 - type: recall value: 39.52569169960474 task: type: BitextMining - dataset: config: kin_Latn-pol_Latn name: MTEB FloresBitextMining (kin_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 16.798418972332016 - type: f1 value: 14.429336215660468 - type: main_score value: 14.429336215660468 - type: precision value: 13.794164021713101 - type: recall value: 16.798418972332016 task: type: BitextMining - dataset: config: mal_Mlym-pol_Latn name: MTEB FloresBitextMining (mal_Mlym-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.3596837944664033 - type: f1 value: 2.4283903952876282 - type: main_score value: 2.4283903952876282 - type: precision value: 2.2272021456804065 - type: recall value: 3.3596837944664033 task: type: BitextMining - dataset: config: pes_Arab-pol_Latn name: MTEB FloresBitextMining (pes_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.312252964426877 - type: f1 value: 6.133645183085144 - type: main_score value: 6.133645183085144 - type: precision value: 5.840438054980056 - type: recall value: 7.312252964426877 task: type: BitextMining - dataset: config: srd_Latn-pol_Latn name: MTEB FloresBitextMining (srd_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.37154150197628 - type: f1 value: 24.557303562561366 - type: main_score value: 24.557303562561366 - type: precision value: 23.69733617903604 - type: recall value: 27.37154150197628 task: type: BitextMining - dataset: config: tzm_Tfng-pol_Latn name: MTEB FloresBitextMining (tzm_Tfng-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.6561264822134385 - type: f1 value: 2.7323453897955785 - type: main_score value: 2.7323453897955785 - type: precision value: 2.5125182239806745 - type: recall value: 3.6561264822134385 task: type: BitextMining - dataset: config: acq_Arab-pol_Latn name: MTEB FloresBitextMining (acq_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.6561264822134385 - type: f1 value: 2.7828303630661635 - type: main_score value: 2.7828303630661635 - type: precision value: 2.6216215063238377 - type: recall value: 3.6561264822134385 task: type: BitextMining - dataset: config: bem_Latn-pol_Latn name: MTEB FloresBitextMining (bem_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.418972332015812 - type: f1 value: 20.18154247440937 - type: main_score value: 20.18154247440937 - type: precision value: 19.337227107086584 - type: recall value: 23.418972332015812 task: type: BitextMining - dataset: config: epo_Latn-pol_Latn name: MTEB FloresBitextMining (epo_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 34.58498023715415 - type: f1 value: 31.141139192708856 - type: main_score value: 31.141139192708856 - type: precision value: 30.228963674031846 - type: recall value: 34.58498023715415 task: type: BitextMining - dataset: config: hun_Latn-pol_Latn name: MTEB FloresBitextMining (hun_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.83794466403162 - type: f1 value: 18.382660044834967 - type: main_score value: 18.382660044834967 - type: precision value: 17.62388154204811 - type: recall value: 21.83794466403162 task: type: BitextMining - dataset: config: kir_Cyrl-pol_Latn name: MTEB FloresBitextMining (kir_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.7312252964426875 - type: f1 value: 4.390398521734955 - type: main_score value: 4.390398521734955 - type: precision value: 4.083283520585645 - type: recall value: 5.7312252964426875 task: type: BitextMining - dataset: config: mar_Deva-pol_Latn name: MTEB FloresBitextMining (mar_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.3596837944664033 - type: f1 value: 2.8885124537298448 - type: main_score value: 2.8885124537298448 - type: precision value: 2.7483235363670144 - type: recall value: 3.3596837944664033 task: type: BitextMining - dataset: config: plt_Latn-pol_Latn name: MTEB FloresBitextMining (plt_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.490118577075098 - type: f1 value: 14.096422773395407 - type: main_score value: 14.096422773395407 - type: precision value: 13.352882758512802 - type: recall value: 17.490118577075098 task: type: BitextMining - dataset: config: srp_Cyrl-pol_Latn name: MTEB FloresBitextMining (srp_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.940711462450593 - type: f1 value: 3.9464130963200947 - type: main_score value: 3.9464130963200947 - type: precision value: 3.6485591702983005 - type: recall value: 4.940711462450593 task: type: BitextMining - dataset: config: uig_Arab-pol_Latn name: MTEB FloresBitextMining (uig_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.422924901185771 - type: f1 value: 5.005627451279625 - type: main_score value: 5.005627451279625 - type: precision value: 4.641382098959696 - type: recall value: 6.422924901185771 task: type: BitextMining - dataset: config: aeb_Arab-pol_Latn name: MTEB FloresBitextMining (aeb_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.557312252964427 - type: f1 value: 2.6480295253947594 - type: main_score value: 2.6480295253947594 - type: precision value: 2.4734523333559957 - type: recall value: 3.557312252964427 task: type: BitextMining - dataset: config: ben_Beng-pol_Latn name: MTEB FloresBitextMining (ben_Beng-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.458498023715415 - type: f1 value: 2.9105495953322036 - type: main_score value: 2.9105495953322036 - type: precision value: 2.7819736015310457 - type: recall value: 3.458498023715415 task: type: BitextMining - dataset: config: est_Latn-pol_Latn name: MTEB FloresBitextMining (est_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.122529644268774 - type: f1 value: 19.955099258098155 - type: main_score value: 19.955099258098155 - type: precision value: 19.23495074794038 - type: recall value: 23.122529644268774 task: type: BitextMining - dataset: config: hye_Armn-pol_Latn name: MTEB FloresBitextMining (hye_Armn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.446640316205533 - type: f1 value: 3.2768240711744383 - type: main_score value: 3.2768240711744383 - type: precision value: 3.0602119144770374 - type: recall value: 4.446640316205533 task: type: BitextMining - dataset: config: kmb_Latn-pol_Latn name: MTEB FloresBitextMining (kmb_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 14.32806324110672 - type: f1 value: 12.331135021968532 - type: main_score value: 12.331135021968532 - type: precision value: 11.91995364188873 - type: recall value: 14.32806324110672 task: type: BitextMining - dataset: config: min_Arab-pol_Latn name: MTEB FloresBitextMining (min_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 0.3952569169960474 - type: f1 value: 0.20182057731464847 - type: main_score value: 0.20182057731464847 - type: precision value: 0.1997649956026004 - type: recall value: 0.3952569169960474 task: type: BitextMining - dataset: config: pol_Latn-ace_Arab name: MTEB FloresBitextMining (pol_Latn-ace_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 0.592885375494071 - type: f1 value: 0.07409945181279336 - type: main_score value: 0.07409945181279336 - type: precision value: 0.04302453996987383 - type: recall value: 0.592885375494071 task: type: BitextMining - dataset: config: pol_Latn-bam_Latn name: MTEB FloresBitextMining (pol_Latn-bam_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.40711462450593 - type: f1 value: 19.400581302755214 - type: main_score value: 19.400581302755214 - type: precision value: 17.845439584570016 - type: recall value: 24.40711462450593 task: type: BitextMining - dataset: config: pol_Latn-dzo_Tibt name: MTEB FloresBitextMining (pol_Latn-dzo_Tibt) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 0.2964426877470355 - type: f1 value: 0.009811695142164987 - type: main_score value: 0.009811695142164987 - type: precision value: 0.005088402404388525 - type: recall value: 0.2964426877470355 task: type: BitextMining - dataset: config: pol_Latn-hin_Deva name: MTEB FloresBitextMining (pol_Latn-hin_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.806324110671936 - type: f1 value: 4.700948820933142 - type: main_score value: 4.700948820933142 - type: precision value: 4.085403094012081 - type: recall value: 7.806324110671936 task: type: BitextMining - dataset: config: pol_Latn-khm_Khmr name: MTEB FloresBitextMining (pol_Latn-khm_Khmr) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 28.458498023715418 - type: f1 value: 20.920653101020132 - type: main_score value: 20.920653101020132 - type: precision value: 18.878444679295004 - type: recall value: 28.458498023715418 task: type: BitextMining - dataset: config: pol_Latn-mag_Deva name: MTEB FloresBitextMining (pol_Latn-mag_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.905138339920949 - type: f1 value: 4.7160981251785765 - type: main_score value: 4.7160981251785765 - type: precision value: 4.084836368220641 - type: recall value: 7.905138339920949 task: type: BitextMining - dataset: config: pol_Latn-pap_Latn name: MTEB FloresBitextMining (pol_Latn-pap_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 48.418972332015805 - type: f1 value: 39.21844495373907 - type: main_score value: 39.21844495373907 - type: precision value: 35.95251819436602 - type: recall value: 48.418972332015805 task: type: BitextMining - dataset: config: pol_Latn-sot_Latn name: MTEB FloresBitextMining (pol_Latn-sot_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.249011857707508 - type: f1 value: 20.207185491276398 - type: main_score value: 20.207185491276398 - type: precision value: 17.683968963307308 - type: recall value: 29.249011857707508 task: type: BitextMining - dataset: config: pol_Latn-tur_Latn name: MTEB FloresBitextMining (pol_Latn-tur_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.841897233201582 - type: f1 value: 23.310580092941755 - type: main_score value: 23.310580092941755 - type: precision value: 21.51464711072004 - type: recall value: 29.841897233201582 task: type: BitextMining - dataset: config: pol_Latn-ace_Latn name: MTEB FloresBitextMining (pol_Latn-ace_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 35.276679841897234 - type: f1 value: 26.56043312355645 - type: main_score value: 26.56043312355645 - type: precision value: 24.029183691544187 - type: recall value: 35.276679841897234 task: type: BitextMining - dataset: config: pol_Latn-ban_Latn name: MTEB FloresBitextMining (pol_Latn-ban_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 34.28853754940712 - type: f1 value: 25.31387739735394 - type: main_score value: 25.31387739735394 - type: precision value: 22.630872914963824 - type: recall value: 34.28853754940712 task: type: BitextMining - dataset: config: pol_Latn-ell_Grek name: MTEB FloresBitextMining (pol_Latn-ell_Grek) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 12.944664031620553 - type: f1 value: 7.700420087043759 - type: main_score value: 7.700420087043759 - type: precision value: 6.557677527690299 - type: recall value: 12.944664031620553 task: type: BitextMining - dataset: config: pol_Latn-hne_Deva name: MTEB FloresBitextMining (pol_Latn-hne_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.027667984189724 - type: f1 value: 3.7449153372588277 - type: main_score value: 3.7449153372588277 - type: precision value: 3.233029505119999 - type: recall value: 6.027667984189724 task: type: BitextMining - dataset: config: pol_Latn-kik_Latn name: MTEB FloresBitextMining (pol_Latn-kik_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 33.794466403162055 - type: f1 value: 27.039119650545697 - type: main_score value: 27.039119650545697 - type: precision value: 24.780510943296644 - type: recall value: 33.794466403162055 task: type: BitextMining - dataset: config: pol_Latn-mai_Deva name: MTEB FloresBitextMining (pol_Latn-mai_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.683794466403162 - type: f1 value: 6.347035635402815 - type: main_score value: 6.347035635402815 - type: precision value: 5.645462573386804 - type: recall value: 9.683794466403162 task: type: BitextMining - dataset: config: pol_Latn-pbt_Arab name: MTEB FloresBitextMining (pol_Latn-pbt_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 13.83399209486166 - type: f1 value: 8.577476120246123 - type: main_score value: 8.577476120246123 - type: precision value: 7.307301107422271 - type: recall value: 13.83399209486166 task: type: BitextMining - dataset: config: pol_Latn-spa_Latn name: MTEB FloresBitextMining (pol_Latn-spa_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 55.434782608695656 - type: f1 value: 47.04059938743733 - type: main_score value: 47.04059938743733 - type: precision value: 43.96519899731363 - type: recall value: 55.434782608695656 task: type: BitextMining - dataset: config: pol_Latn-twi_Latn name: MTEB FloresBitextMining (pol_Latn-twi_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 36.95652173913043 - type: f1 value: 29.213974786225428 - type: main_score value: 29.213974786225428 - type: precision value: 26.67718658701861 - type: recall value: 36.95652173913043 task: type: BitextMining - dataset: config: pol_Latn-acm_Arab name: MTEB FloresBitextMining (pol_Latn-acm_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.015810276679842 - type: f1 value: 3.8689788384375685 - type: main_score value: 3.8689788384375685 - type: precision value: 3.295533948722674 - type: recall value: 7.015810276679842 task: type: BitextMining - dataset: config: pol_Latn-bel_Cyrl name: MTEB FloresBitextMining (pol_Latn-bel_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.312252964426877 - type: f1 value: 4.433740291213768 - type: main_score value: 4.433740291213768 - type: precision value: 3.7685799545496645 - type: recall value: 7.312252964426877 task: type: BitextMining - dataset: config: pol_Latn-eng_Latn name: MTEB FloresBitextMining (pol_Latn-eng_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 98.81422924901186 - type: f1 value: 98.41897233201581 - type: main_score value: 98.41897233201581 - type: precision value: 98.22134387351778 - type: recall value: 98.81422924901186 task: type: BitextMining - dataset: config: pol_Latn-hrv_Latn name: MTEB FloresBitextMining (pol_Latn-hrv_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 41.798418972332016 - type: f1 value: 34.01400189073114 - type: main_score value: 34.01400189073114 - type: precision value: 31.4615696984529 - type: recall value: 41.798418972332016 task: type: BitextMining - dataset: config: pol_Latn-kin_Latn name: MTEB FloresBitextMining (pol_Latn-kin_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.134387351778656 - type: f1 value: 14.941913199793063 - type: main_score value: 14.941913199793063 - type: precision value: 13.09437075957867 - type: recall value: 22.134387351778656 task: type: BitextMining - dataset: config: pol_Latn-mal_Mlym name: MTEB FloresBitextMining (pol_Latn-mal_Mlym) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.905138339920949 - type: f1 value: 4.676431663013731 - type: main_score value: 4.676431663013731 - type: precision value: 3.963416458591999 - type: recall value: 7.905138339920949 task: type: BitextMining - dataset: config: pol_Latn-pes_Arab name: MTEB FloresBitextMining (pol_Latn-pes_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 11.6600790513834 - type: f1 value: 7.168267060928982 - type: main_score value: 7.168267060928982 - type: precision value: 6.158810820381868 - type: recall value: 11.6600790513834 task: type: BitextMining - dataset: config: pol_Latn-srd_Latn name: MTEB FloresBitextMining (pol_Latn-srd_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 39.723320158102766 - type: f1 value: 31.331230254155155 - type: main_score value: 31.331230254155155 - type: precision value: 28.576141140370382 - type: recall value: 39.723320158102766 task: type: BitextMining - dataset: config: pol_Latn-tzm_Tfng name: MTEB FloresBitextMining (pol_Latn-tzm_Tfng) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.312252964426877 - type: f1 value: 3.9530501105195044 - type: main_score value: 3.9530501105195044 - type: precision value: 3.3006450734694783 - type: recall value: 7.312252964426877 task: type: BitextMining - dataset: config: pol_Latn-acq_Arab name: MTEB FloresBitextMining (pol_Latn-acq_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.521739130434782 - type: f1 value: 3.7118615772057084 - type: main_score value: 3.7118615772057084 - type: precision value: 3.140963634812984 - type: recall value: 6.521739130434782 task: type: BitextMining - dataset: config: pol_Latn-bem_Latn name: MTEB FloresBitextMining (pol_Latn-bem_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.940711462450594 - type: f1 value: 21.838134223625964 - type: main_score value: 21.838134223625964 - type: precision value: 19.601437433830295 - type: recall value: 29.940711462450594 task: type: BitextMining - dataset: config: pol_Latn-epo_Latn name: MTEB FloresBitextMining (pol_Latn-epo_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 45.25691699604743 - type: f1 value: 37.470343665995834 - type: main_score value: 37.470343665995834 - type: precision value: 34.91015394428969 - type: recall value: 45.25691699604743 task: type: BitextMining - dataset: config: pol_Latn-hun_Latn name: MTEB FloresBitextMining (pol_Latn-hun_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.766798418972332 - type: f1 value: 19.856585262886124 - type: main_score value: 19.856585262886124 - type: precision value: 17.632943238979625 - type: recall value: 27.766798418972332 task: type: BitextMining - dataset: config: pol_Latn-kir_Cyrl name: MTEB FloresBitextMining (pol_Latn-kir_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.980237154150199 - type: f1 value: 5.976712030682037 - type: main_score value: 5.976712030682037 - type: precision value: 5.1168105551227905 - type: recall value: 9.980237154150199 task: type: BitextMining - dataset: config: pol_Latn-mar_Deva name: MTEB FloresBitextMining (pol_Latn-mar_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.312252964426877 - type: f1 value: 4.145286234223563 - type: main_score value: 4.145286234223563 - type: precision value: 3.5337956030345037 - type: recall value: 7.312252964426877 task: type: BitextMining - dataset: config: pol_Latn-plt_Latn name: MTEB FloresBitextMining (pol_Latn-plt_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.90118577075099 - type: f1 value: 18.082595637436008 - type: main_score value: 18.082595637436008 - type: precision value: 16.16756856262583 - type: recall value: 24.90118577075099 task: type: BitextMining - dataset: config: pol_Latn-srp_Cyrl name: MTEB FloresBitextMining (pol_Latn-srp_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.806324110671936 - type: f1 value: 4.668414246710413 - type: main_score value: 4.668414246710413 - type: precision value: 4.07016381507616 - type: recall value: 7.806324110671936 task: type: BitextMining - dataset: config: pol_Latn-uig_Arab name: MTEB FloresBitextMining (pol_Latn-uig_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.58498023715415 - type: f1 value: 5.958798680475491 - type: main_score value: 5.958798680475491 - type: precision value: 5.1177497873792746 - type: recall value: 9.58498023715415 task: type: BitextMining - dataset: config: pol_Latn-aeb_Arab name: MTEB FloresBitextMining (pol_Latn-aeb_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.8181818181818175 - type: f1 value: 3.75515263763757 - type: main_score value: 3.75515263763757 - type: precision value: 3.1582416170556162 - type: recall value: 6.8181818181818175 task: type: BitextMining - dataset: config: pol_Latn-ben_Beng name: MTEB FloresBitextMining (pol_Latn-ben_Beng) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.608695652173914 - type: f1 value: 4.041225756257413 - type: main_score value: 4.041225756257413 - type: precision value: 3.324217301397562 - type: recall value: 7.608695652173914 task: type: BitextMining - dataset: config: pol_Latn-est_Latn name: MTEB FloresBitextMining (pol_Latn-est_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.347826086956523 - type: f1 value: 22.32776391373229 - type: main_score value: 22.32776391373229 - type: precision value: 20.281558764760348 - type: recall value: 29.347826086956523 task: type: BitextMining - dataset: config: pol_Latn-hye_Armn name: MTEB FloresBitextMining (pol_Latn-hye_Armn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.992094861660078 - type: f1 value: 5.410264691947379 - type: main_score value: 5.410264691947379 - type: precision value: 4.551435410943297 - type: recall value: 8.992094861660078 task: type: BitextMining - dataset: config: pol_Latn-kmb_Latn name: MTEB FloresBitextMining (pol_Latn-kmb_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.33201581027668 - type: f1 value: 15.508983949241994 - type: main_score value: 15.508983949241994 - type: precision value: 13.84626446237118 - type: recall value: 22.33201581027668 task: type: BitextMining - dataset: config: pol_Latn-min_Arab name: MTEB FloresBitextMining (pol_Latn-min_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 0.4940711462450593 - type: f1 value: 0.1263500946050829 - type: main_score value: 0.1263500946050829 - type: precision value: 0.1133716826651609 - type: recall value: 0.4940711462450593 task: type: BitextMining - dataset: config: pol_Latn-ssw_Latn name: MTEB FloresBitextMining (pol_Latn-ssw_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 20.948616600790515 - type: f1 value: 13.737818978807919 - type: main_score value: 13.737818978807919 - type: precision value: 11.8912676418156 - type: recall value: 20.948616600790515 task: type: BitextMining - dataset: config: pol_Latn-ukr_Cyrl name: MTEB FloresBitextMining (pol_Latn-ukr_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.015810276679842 - type: f1 value: 3.949638010614578 - type: main_score value: 3.949638010614578 - type: precision value: 3.351660503470364 - type: recall value: 7.015810276679842 task: type: BitextMining - dataset: config: pol_Latn-afr_Latn name: MTEB FloresBitextMining (pol_Latn-afr_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 42.78656126482213 - type: f1 value: 34.60139378961342 - type: main_score value: 34.60139378961342 - type: precision value: 32.04002903113575 - type: recall value: 42.78656126482213 task: type: BitextMining - dataset: config: pol_Latn-bho_Deva name: MTEB FloresBitextMining (pol_Latn-bho_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.893280632411066 - type: f1 value: 5.424250733521232 - type: main_score value: 5.424250733521232 - type: precision value: 4.6840650669488015 - type: recall value: 8.893280632411066 task: type: BitextMining - dataset: config: pol_Latn-eus_Latn name: MTEB FloresBitextMining (pol_Latn-eus_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 37.94466403162055 - type: f1 value: 29.129736096139265 - type: main_score value: 29.129736096139265 - type: precision value: 26.27383239290354 - type: recall value: 37.94466403162055 task: type: BitextMining - dataset: config: pol_Latn-ibo_Latn name: MTEB FloresBitextMining (pol_Latn-ibo_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.82608695652174 - type: f1 value: 18.160396366918103 - type: main_score value: 18.160396366918103 - type: precision value: 16.68498579812809 - type: recall value: 22.82608695652174 task: type: BitextMining - dataset: config: pol_Latn-kmr_Latn name: MTEB FloresBitextMining (pol_Latn-kmr_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.83794466403162 - type: f1 value: 16.69834842262613 - type: main_score value: 16.69834842262613 - type: precision value: 15.096012124362323 - type: recall value: 21.83794466403162 task: type: BitextMining - dataset: config: pol_Latn-min_Latn name: MTEB FloresBitextMining (pol_Latn-min_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 33.99209486166008 - type: f1 value: 25.434667166386532 - type: main_score value: 25.434667166386532 - type: precision value: 22.89181280773553 - type: recall value: 33.99209486166008 task: type: BitextMining - dataset: config: pol_Latn-por_Latn name: MTEB FloresBitextMining (pol_Latn-por_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 48.71541501976285 - type: f1 value: 39.9428047797613 - type: main_score value: 39.9428047797613 - type: precision value: 36.89033189033189 - type: recall value: 48.71541501976285 task: type: BitextMining - dataset: config: pol_Latn-sun_Latn name: MTEB FloresBitextMining (pol_Latn-sun_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 34.387351778656125 - type: f1 value: 26.088610555013712 - type: main_score value: 26.088610555013712 - type: precision value: 23.44616594047338 - type: recall value: 34.387351778656125 task: type: BitextMining - dataset: config: pol_Latn-umb_Latn name: MTEB FloresBitextMining (pol_Latn-umb_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.34387351778656 - type: f1 value: 15.03361444124914 - type: main_score value: 15.03361444124914 - type: precision value: 13.457627923107571 - type: recall value: 21.34387351778656 task: type: BitextMining - dataset: config: pol_Latn-ajp_Arab name: MTEB FloresBitextMining (pol_Latn-ajp_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.288537549407115 - type: f1 value: 5.414032335360037 - type: main_score value: 5.414032335360037 - type: precision value: 4.607723748005992 - type: recall value: 9.288537549407115 task: type: BitextMining - dataset: config: pol_Latn-bjn_Arab name: MTEB FloresBitextMining (pol_Latn-bjn_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 0.4940711462450593 - type: f1 value: 0.11765254656700863 - type: main_score value: 0.11765254656700863 - type: precision value: 0.10873873255364525 - type: recall value: 0.4940711462450593 task: type: BitextMining - dataset: config: pol_Latn-ewe_Latn name: MTEB FloresBitextMining (pol_Latn-ewe_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 26.976284584980238 - type: f1 value: 21.945314237804357 - type: main_score value: 21.945314237804357 - type: precision value: 20.516700032510307 - type: recall value: 26.976284584980238 task: type: BitextMining - dataset: config: pol_Latn-ilo_Latn name: MTEB FloresBitextMining (pol_Latn-ilo_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 39.32806324110672 - type: f1 value: 30.38183785911059 - type: main_score value: 30.38183785911059 - type: precision value: 27.592691125299822 - type: recall value: 39.32806324110672 task: type: BitextMining - dataset: config: pol_Latn-knc_Arab name: MTEB FloresBitextMining (pol_Latn-knc_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.881422924901186 - type: f1 value: 5.620639548496682 - type: main_score value: 5.620639548496682 - type: precision value: 4.632203265894544 - type: recall value: 9.881422924901186 task: type: BitextMining - dataset: config: pol_Latn-mkd_Cyrl name: MTEB FloresBitextMining (pol_Latn-mkd_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.201581027667984 - type: f1 value: 5.756667198483056 - type: main_score value: 5.756667198483056 - type: precision value: 5.172948249340367 - type: recall value: 8.201581027667984 task: type: BitextMining - dataset: config: pol_Latn-prs_Arab name: MTEB FloresBitextMining (pol_Latn-prs_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.782608695652174 - type: f1 value: 5.919630385538823 - type: main_score value: 5.919630385538823 - type: precision value: 5.130159105954237 - type: recall value: 9.782608695652174 task: type: BitextMining - dataset: config: pol_Latn-swe_Latn name: MTEB FloresBitextMining (pol_Latn-swe_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 45.355731225296445 - type: f1 value: 36.27500518804867 - type: main_score value: 36.27500518804867 - type: precision value: 33.26172976123569 - type: recall value: 45.355731225296445 task: type: BitextMining - dataset: config: pol_Latn-urd_Arab name: MTEB FloresBitextMining (pol_Latn-urd_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.632411067193676 - type: f1 value: 3.569460684222274 - type: main_score value: 3.569460684222274 - type: precision value: 3.1240488378742373 - type: recall value: 5.632411067193676 task: type: BitextMining - dataset: config: pol_Latn-aka_Latn name: MTEB FloresBitextMining (pol_Latn-aka_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 36.56126482213439 - type: f1 value: 29.486626632613916 - type: main_score value: 29.486626632613916 - type: precision value: 27.144587132235355 - type: recall value: 36.56126482213439 task: type: BitextMining - dataset: config: pol_Latn-bjn_Latn name: MTEB FloresBitextMining (pol_Latn-bjn_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 31.027667984189723 - type: f1 value: 23.143749721435146 - type: main_score value: 23.143749721435146 - type: precision value: 20.753021818980315 - type: recall value: 31.027667984189723 task: type: BitextMining - dataset: config: pol_Latn-fao_Latn name: MTEB FloresBitextMining (pol_Latn-fao_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 30.434782608695656 - type: f1 value: 22.605249989130634 - type: main_score value: 22.605249989130634 - type: precision value: 20.2527359827261 - type: recall value: 30.434782608695656 task: type: BitextMining - dataset: config: pol_Latn-ind_Latn name: MTEB FloresBitextMining (pol_Latn-ind_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 37.648221343873516 - type: f1 value: 28.81205301932887 - type: main_score value: 28.81205301932887 - type: precision value: 26.10980623345404 - type: recall value: 37.648221343873516 task: type: BitextMining - dataset: config: pol_Latn-knc_Latn name: MTEB FloresBitextMining (pol_Latn-knc_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 31.719367588932805 - type: f1 value: 25.271598735037472 - type: main_score value: 25.271598735037472 - type: precision value: 23.153872856640987 - type: recall value: 31.719367588932805 task: type: BitextMining - dataset: config: pol_Latn-mlt_Latn name: MTEB FloresBitextMining (pol_Latn-mlt_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 37.74703557312253 - type: f1 value: 29.91421484751524 - type: main_score value: 29.91421484751524 - type: precision value: 27.373131717945178 - type: recall value: 37.74703557312253 task: type: BitextMining - dataset: config: pol_Latn-quy_Latn name: MTEB FloresBitextMining (pol_Latn-quy_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 31.32411067193676 - type: f1 value: 22.65035601201609 - type: main_score value: 22.65035601201609 - type: precision value: 20.25053685652812 - type: recall value: 31.32411067193676 task: type: BitextMining - dataset: config: pol_Latn-swh_Latn name: MTEB FloresBitextMining (pol_Latn-swh_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.209486166007903 - type: f1 value: 17.217401340425052 - type: main_score value: 17.217401340425052 - type: precision value: 15.316212104443727 - type: recall value: 24.209486166007903 task: type: BitextMining - dataset: config: pol_Latn-uzn_Latn name: MTEB FloresBitextMining (pol_Latn-uzn_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 26.284584980237153 - type: f1 value: 18.861504835791653 - type: main_score value: 18.861504835791653 - type: precision value: 16.75885497785852 - type: recall value: 26.284584980237153 task: type: BitextMining - dataset: config: pol_Latn-als_Latn name: MTEB FloresBitextMining (pol_Latn-als_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.905138339920946 - type: f1 value: 26.20568763315395 - type: main_score value: 26.20568763315395 - type: precision value: 24.225254735383192 - type: recall value: 32.905138339920946 task: type: BitextMining - dataset: config: pol_Latn-bod_Tibt name: MTEB FloresBitextMining (pol_Latn-bod_Tibt) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.434782608695652 - type: f1 value: 2.0762499375776593 - type: main_score value: 2.0762499375776593 - type: precision value: 1.584450632089665 - type: recall value: 5.434782608695652 task: type: BitextMining - dataset: config: pol_Latn-fij_Latn name: MTEB FloresBitextMining (pol_Latn-fij_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.110671936758894 - type: f1 value: 16.493571645324778 - type: main_score value: 16.493571645324778 - type: precision value: 14.493070973297389 - type: recall value: 24.110671936758894 task: type: BitextMining - dataset: config: pol_Latn-isl_Latn name: MTEB FloresBitextMining (pol_Latn-isl_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.592885375494074 - type: f1 value: 18.80255539753331 - type: main_score value: 18.80255539753331 - type: precision value: 16.858735659377956 - type: recall value: 25.592885375494074 task: type: BitextMining - dataset: config: pol_Latn-kon_Latn name: MTEB FloresBitextMining (pol_Latn-kon_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.173913043478258 - type: f1 value: 19.474921255779222 - type: main_score value: 19.474921255779222 - type: precision value: 17.42934918318192 - type: recall value: 27.173913043478258 task: type: BitextMining - dataset: config: pol_Latn-mni_Beng name: MTEB FloresBitextMining (pol_Latn-mni_Beng) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.7312252964426875 - type: f1 value: 2.6892720436412247 - type: main_score value: 2.6892720436412247 - type: precision value: 2.13032205646918 - type: recall value: 5.7312252964426875 task: type: BitextMining - dataset: config: pol_Latn-ron_Latn name: MTEB FloresBitextMining (pol_Latn-ron_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 50.39525691699605 - type: f1 value: 42.86300765549777 - type: main_score value: 42.86300765549777 - type: precision value: 40.16924253831368 - type: recall value: 50.39525691699605 task: type: BitextMining - dataset: config: pol_Latn-szl_Latn name: MTEB FloresBitextMining (pol_Latn-szl_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 87.94466403162056 - type: f1 value: 84.61791831357048 - type: main_score value: 84.61791831357048 - type: precision value: 83.06982872200265 - type: recall value: 87.94466403162056 task: type: BitextMining - dataset: config: pol_Latn-vec_Latn name: MTEB FloresBitextMining (pol_Latn-vec_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 47.03557312252965 - type: f1 value: 39.26110341130104 - type: main_score value: 39.26110341130104 - type: precision value: 36.65560888386975 - type: recall value: 47.03557312252965 task: type: BitextMining - dataset: config: pol_Latn-amh_Ethi name: MTEB FloresBitextMining (pol_Latn-amh_Ethi) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.213438735177865 - type: f1 value: 4.229592189919405 - type: main_score value: 4.229592189919405 - type: precision value: 3.6053747800289306 - type: recall value: 7.213438735177865 task: type: BitextMining - dataset: config: pol_Latn-bos_Latn name: MTEB FloresBitextMining (pol_Latn-bos_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 41.205533596837945 - type: f1 value: 33.10834279905426 - type: main_score value: 33.10834279905426 - type: precision value: 30.403204886900536 - type: recall value: 41.205533596837945 task: type: BitextMining - dataset: config: pol_Latn-fin_Latn name: MTEB FloresBitextMining (pol_Latn-fin_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.841897233201582 - type: f1 value: 21.713522647146743 - type: main_score value: 21.713522647146743 - type: precision value: 19.350322170716574 - type: recall value: 29.841897233201582 task: type: BitextMining - dataset: config: pol_Latn-ita_Latn name: MTEB FloresBitextMining (pol_Latn-ita_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 50.29644268774703 - type: f1 value: 40.906659765355414 - type: main_score value: 40.906659765355414 - type: precision value: 37.68257105213627 - type: recall value: 50.29644268774703 task: type: BitextMining - dataset: config: pol_Latn-kor_Hang name: MTEB FloresBitextMining (pol_Latn-kor_Hang) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 11.264822134387352 - type: f1 value: 5.931697077593314 - type: main_score value: 5.931697077593314 - type: precision value: 4.9535659231074884 - type: recall value: 11.264822134387352 task: type: BitextMining - dataset: config: pol_Latn-mos_Latn name: MTEB FloresBitextMining (pol_Latn-mos_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.40711462450593 - type: f1 value: 18.514184619660327 - type: main_score value: 18.514184619660327 - type: precision value: 16.817790607446316 - type: recall value: 24.40711462450593 task: type: BitextMining - dataset: config: pol_Latn-run_Latn name: MTEB FloresBitextMining (pol_Latn-run_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.134387351778656 - type: f1 value: 15.38932603408888 - type: main_score value: 15.38932603408888 - type: precision value: 13.646404187542108 - type: recall value: 22.134387351778656 task: type: BitextMining - dataset: config: pol_Latn-tam_Taml name: MTEB FloresBitextMining (pol_Latn-tam_Taml) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.782608695652174 - type: f1 value: 5.6178894438842555 - type: main_score value: 5.6178894438842555 - type: precision value: 4.75356748528658 - type: recall value: 9.782608695652174 task: type: BitextMining - dataset: config: pol_Latn-vie_Latn name: MTEB FloresBitextMining (pol_Latn-vie_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.446640316205535 - type: f1 value: 23.58383601466605 - type: main_score value: 23.58383601466605 - type: precision value: 21.79765810879941 - type: recall value: 29.446640316205535 task: type: BitextMining - dataset: config: pol_Latn-apc_Arab name: MTEB FloresBitextMining (pol_Latn-apc_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.58498023715415 - type: f1 value: 5.606078233293074 - type: main_score value: 5.606078233293074 - type: precision value: 4.687138313620527 - type: recall value: 9.58498023715415 task: type: BitextMining - dataset: config: pol_Latn-bug_Latn name: MTEB FloresBitextMining (pol_Latn-bug_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.31225296442688 - type: f1 value: 24.07572142528976 - type: main_score value: 24.07572142528976 - type: precision value: 21.67035670741204 - type: recall value: 32.31225296442688 task: type: BitextMining - dataset: config: pol_Latn-fon_Latn name: MTEB FloresBitextMining (pol_Latn-fon_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.07509881422925 - type: f1 value: 21.33543800292812 - type: main_score value: 21.33543800292812 - type: precision value: 19.557752665590193 - type: recall value: 27.07509881422925 task: type: BitextMining - dataset: config: pol_Latn-jav_Latn name: MTEB FloresBitextMining (pol_Latn-jav_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.509881422924906 - type: f1 value: 23.972031207703143 - type: main_score value: 23.972031207703143 - type: precision value: 21.51591043098832 - type: recall value: 32.509881422924906 task: type: BitextMining - dataset: config: pol_Latn-lao_Laoo name: MTEB FloresBitextMining (pol_Latn-lao_Laoo) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 35.37549407114625 - type: f1 value: 28.995182737771664 - type: main_score value: 28.995182737771664 - type: precision value: 27.077181737448885 - type: recall value: 35.37549407114625 task: type: BitextMining - dataset: config: pol_Latn-mri_Latn name: MTEB FloresBitextMining (pol_Latn-mri_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.134387351778656 - type: f1 value: 15.354442612607786 - type: main_score value: 15.354442612607786 - type: precision value: 13.594571086102247 - type: recall value: 22.134387351778656 task: type: BitextMining - dataset: config: pol_Latn-rus_Cyrl name: MTEB FloresBitextMining (pol_Latn-rus_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.992094861660078 - type: f1 value: 5.8135612152195755 - type: main_score value: 5.8135612152195755 - type: precision value: 5.163711679758252 - type: recall value: 8.992094861660078 task: type: BitextMining - dataset: config: pol_Latn-taq_Latn name: MTEB FloresBitextMining (pol_Latn-taq_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.80632411067194 - type: f1 value: 26.83270577364974 - type: main_score value: 26.83270577364974 - type: precision value: 24.99793839961824 - type: recall value: 32.80632411067194 task: type: BitextMining - dataset: config: pol_Latn-war_Latn name: MTEB FloresBitextMining (pol_Latn-war_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 46.14624505928854 - type: f1 value: 37.38924037095973 - type: main_score value: 37.38924037095973 - type: precision value: 34.53650072646034 - type: recall value: 46.14624505928854 task: type: BitextMining - dataset: config: pol_Latn-arb_Arab name: MTEB FloresBitextMining (pol_Latn-arb_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.521739130434782 - type: f1 value: 3.5196469175927976 - type: main_score value: 3.5196469175927976 - type: precision value: 2.9206974315601584 - type: recall value: 6.521739130434782 task: type: BitextMining - dataset: config: pol_Latn-bul_Cyrl name: MTEB FloresBitextMining (pol_Latn-bul_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.782608695652174 - type: f1 value: 7.0918258926674085 - type: main_score value: 7.0918258926674085 - type: precision value: 6.34613789429909 - type: recall value: 9.782608695652174 task: type: BitextMining - dataset: config: pol_Latn-fra_Latn name: MTEB FloresBitextMining (pol_Latn-fra_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 59.387351778656125 - type: f1 value: 51.296113306982875 - type: main_score value: 51.296113306982875 - type: precision value: 48.246403532528845 - type: recall value: 59.387351778656125 task: type: BitextMining - dataset: config: pol_Latn-jpn_Jpan name: MTEB FloresBitextMining (pol_Latn-jpn_Jpan) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.9644268774703555 - type: f1 value: 0.944206619732386 - type: main_score value: 0.944206619732386 - type: precision value: 0.6727116823200325 - type: recall value: 2.9644268774703555 task: type: BitextMining - dataset: config: pol_Latn-lij_Latn name: MTEB FloresBitextMining (pol_Latn-lij_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 43.77470355731226 - type: f1 value: 35.2610249582756 - type: main_score value: 35.2610249582756 - type: precision value: 32.377518863437835 - type: recall value: 43.77470355731226 task: type: BitextMining - dataset: config: pol_Latn-mya_Mymr name: MTEB FloresBitextMining (pol_Latn-mya_Mymr) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.486166007905137 - type: f1 value: 4.685784685126671 - type: main_score value: 4.685784685126671 - type: precision value: 3.7744225331158545 - type: recall value: 9.486166007905137 task: type: BitextMining - dataset: config: pol_Latn-sag_Latn name: MTEB FloresBitextMining (pol_Latn-sag_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.604743083003953 - type: f1 value: 18.7955635422213 - type: main_score value: 18.7955635422213 - type: precision value: 17.210956257762234 - type: recall value: 24.604743083003953 task: type: BitextMining - dataset: config: pol_Latn-taq_Tfng name: MTEB FloresBitextMining (pol_Latn-taq_Tfng) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 10.375494071146244 - type: f1 value: 5.969657887031101 - type: main_score value: 5.969657887031101 - type: precision value: 5.113323685856255 - type: recall value: 10.375494071146244 task: type: BitextMining - dataset: config: pol_Latn-wol_Latn name: MTEB FloresBitextMining (pol_Latn-wol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.446640316205535 - type: f1 value: 22.516172232601395 - type: main_score value: 22.516172232601395 - type: precision value: 20.50913832737827 - type: recall value: 29.446640316205535 task: type: BitextMining - dataset: config: pol_Latn-arb_Latn name: MTEB FloresBitextMining (pol_Latn-arb_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.885375494071145 - type: f1 value: 11.136634966921527 - type: main_score value: 11.136634966921527 - type: precision value: 9.470246749242751 - type: recall value: 17.885375494071145 task: type: BitextMining - dataset: config: pol_Latn-cat_Latn name: MTEB FloresBitextMining (pol_Latn-cat_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 53.952569169960476 - type: f1 value: 45.5981348471467 - type: main_score value: 45.5981348471467 - type: precision value: 42.58885438233265 - type: recall value: 53.952569169960476 task: type: BitextMining - dataset: config: pol_Latn-fur_Latn name: MTEB FloresBitextMining (pol_Latn-fur_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 43.97233201581028 - type: f1 value: 35.834735093628375 - type: main_score value: 35.834735093628375 - type: precision value: 33.147622853650525 - type: recall value: 43.97233201581028 task: type: BitextMining - dataset: config: pol_Latn-kab_Latn name: MTEB FloresBitextMining (pol_Latn-kab_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 11.857707509881422 - type: f1 value: 7.397762207050744 - type: main_score value: 7.397762207050744 - type: precision value: 6.311522971296096 - type: recall value: 11.857707509881422 task: type: BitextMining - dataset: config: pol_Latn-lim_Latn name: MTEB FloresBitextMining (pol_Latn-lim_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 45.8498023715415 - type: f1 value: 37.7763778885611 - type: main_score value: 37.7763778885611 - type: precision value: 34.853639582711615 - type: recall value: 45.8498023715415 task: type: BitextMining - dataset: config: pol_Latn-nld_Latn name: MTEB FloresBitextMining (pol_Latn-nld_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 47.5296442687747 - type: f1 value: 38.405752953677855 - type: main_score value: 38.405752953677855 - type: precision value: 35.274876090093485 - type: recall value: 47.5296442687747 task: type: BitextMining - dataset: config: pol_Latn-san_Deva name: MTEB FloresBitextMining (pol_Latn-san_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.189723320158102 - type: f1 value: 5.205993892335911 - type: main_score value: 5.205993892335911 - type: precision value: 4.337201997808373 - type: recall value: 9.189723320158102 task: type: BitextMining - dataset: config: pol_Latn-tat_Cyrl name: MTEB FloresBitextMining (pol_Latn-tat_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.189723320158102 - type: f1 value: 5.314675144304728 - type: main_score value: 5.314675144304728 - type: precision value: 4.492476362336001 - type: recall value: 9.189723320158102 task: type: BitextMining - dataset: config: pol_Latn-xho_Latn name: MTEB FloresBitextMining (pol_Latn-xho_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.90118577075099 - type: f1 value: 16.50000499061764 - type: main_score value: 16.50000499061764 - type: precision value: 14.219926880064056 - type: recall value: 24.90118577075099 task: type: BitextMining - dataset: config: pol_Latn-ars_Arab name: MTEB FloresBitextMining (pol_Latn-ars_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.150197628458498 - type: f1 value: 1.9884773395885484 - type: main_score value: 1.9884773395885484 - type: precision value: 1.6176875636726988 - type: recall value: 4.150197628458498 task: type: BitextMining - dataset: config: pol_Latn-ceb_Latn name: MTEB FloresBitextMining (pol_Latn-ceb_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 39.426877470355734 - type: f1 value: 30.411252669157808 - type: main_score value: 30.411252669157808 - type: precision value: 27.625018536579798 - type: recall value: 39.426877470355734 task: type: BitextMining - dataset: config: pol_Latn-fuv_Latn name: MTEB FloresBitextMining (pol_Latn-fuv_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.889328063241106 - type: f1 value: 18.16006406557946 - type: main_score value: 18.16006406557946 - type: precision value: 16.093802710132557 - type: recall value: 25.889328063241106 task: type: BitextMining - dataset: config: pol_Latn-kac_Latn name: MTEB FloresBitextMining (pol_Latn-kac_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 28.063241106719367 - type: f1 value: 20.537588393674426 - type: main_score value: 20.537588393674426 - type: precision value: 18.555593378125913 - type: recall value: 28.063241106719367 task: type: BitextMining - dataset: config: pol_Latn-lin_Latn name: MTEB FloresBitextMining (pol_Latn-lin_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.56916996047431 - type: f1 value: 19.59550667924326 - type: main_score value: 19.59550667924326 - type: precision value: 17.366118810405023 - type: recall value: 27.56916996047431 task: type: BitextMining - dataset: config: pol_Latn-nno_Latn name: MTEB FloresBitextMining (pol_Latn-nno_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 44.466403162055336 - type: f1 value: 36.52365909233498 - type: main_score value: 36.52365909233498 - type: precision value: 34.02154307045611 - type: recall value: 44.466403162055336 task: type: BitextMining - dataset: config: pol_Latn-sat_Olck name: MTEB FloresBitextMining (pol_Latn-sat_Olck) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 1.7786561264822136 - type: f1 value: 0.5308103241702219 - type: main_score value: 0.5308103241702219 - type: precision value: 0.3879325601322212 - type: recall value: 1.7786561264822136 task: type: BitextMining - dataset: config: pol_Latn-tel_Telu name: MTEB FloresBitextMining (pol_Latn-tel_Telu) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 13.043478260869565 - type: f1 value: 7.211621994490727 - type: main_score value: 7.211621994490727 - type: precision value: 6.072556380043011 - type: recall value: 13.043478260869565 task: type: BitextMining - dataset: config: pol_Latn-ydd_Hebr name: MTEB FloresBitextMining (pol_Latn-ydd_Hebr) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 10.17786561264822 - type: f1 value: 6.04945869467265 - type: main_score value: 6.04945869467265 - type: precision value: 5.313130898081306 - type: recall value: 10.17786561264822 task: type: BitextMining - dataset: config: pol_Latn-ary_Arab name: MTEB FloresBitextMining (pol_Latn-ary_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.905138339920949 - type: f1 value: 4.544108401407499 - type: main_score value: 4.544108401407499 - type: precision value: 3.869493506162304 - type: recall value: 7.905138339920949 task: type: BitextMining - dataset: config: pol_Latn-ces_Latn name: MTEB FloresBitextMining (pol_Latn-ces_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 45.45454545454545 - type: f1 value: 38.024558240445856 - type: main_score value: 38.024558240445856 - type: precision value: 35.54695706331953 - type: recall value: 45.45454545454545 task: type: BitextMining - dataset: config: pol_Latn-gaz_Latn name: MTEB FloresBitextMining (pol_Latn-gaz_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 11.16600790513834 - type: f1 value: 6.159234456287466 - type: main_score value: 6.159234456287466 - type: precision value: 5.133667829697058 - type: recall value: 11.16600790513834 task: type: BitextMining - dataset: config: pol_Latn-kam_Latn name: MTEB FloresBitextMining (pol_Latn-kam_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.221343873517785 - type: f1 value: 18.353962970667776 - type: main_score value: 18.353962970667776 - type: precision value: 16.88928967302209 - type: recall value: 23.221343873517785 task: type: BitextMining - dataset: config: pol_Latn-lit_Latn name: MTEB FloresBitextMining (pol_Latn-lit_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.4703557312253 - type: f1 value: 21.64822878200428 - type: main_score value: 21.64822878200428 - type: precision value: 19.899989111993023 - type: recall value: 27.4703557312253 task: type: BitextMining - dataset: config: pol_Latn-nob_Latn name: MTEB FloresBitextMining (pol_Latn-nob_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 47.43083003952569 - type: f1 value: 38.246988518727655 - type: main_score value: 38.246988518727655 - type: precision value: 35.016858690277665 - type: recall value: 47.43083003952569 task: type: BitextMining - dataset: config: pol_Latn-scn_Latn name: MTEB FloresBitextMining (pol_Latn-scn_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 31.225296442687743 - type: f1 value: 24.21253976589945 - type: main_score value: 24.21253976589945 - type: precision value: 22.16210944903703 - type: recall value: 31.225296442687743 task: type: BitextMining - dataset: config: pol_Latn-tgk_Cyrl name: MTEB FloresBitextMining (pol_Latn-tgk_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.387351778656127 - type: f1 value: 5.6102738478742555 - type: main_score value: 5.6102738478742555 - type: precision value: 4.867386506350517 - type: recall value: 9.387351778656127 task: type: BitextMining - dataset: config: pol_Latn-yor_Latn name: MTEB FloresBitextMining (pol_Latn-yor_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.4703557312253 - type: f1 value: 20.59402252337035 - type: main_score value: 20.59402252337035 - type: precision value: 18.62389494756967 - type: recall value: 27.4703557312253 task: type: BitextMining - dataset: config: pol_Latn-arz_Arab name: MTEB FloresBitextMining (pol_Latn-arz_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.928853754940711 - type: f1 value: 3.5288297962170567 - type: main_score value: 3.5288297962170567 - type: precision value: 3.0691475575368856 - type: recall value: 5.928853754940711 task: type: BitextMining - dataset: config: pol_Latn-cjk_Latn name: MTEB FloresBitextMining (pol_Latn-cjk_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.221343873517785 - type: f1 value: 16.159405299721502 - type: main_score value: 16.159405299721502 - type: precision value: 14.393576878448815 - type: recall value: 23.221343873517785 task: type: BitextMining - dataset: config: pol_Latn-gla_Latn name: MTEB FloresBitextMining (pol_Latn-gla_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 20.35573122529644 - type: f1 value: 13.720070726259618 - type: main_score value: 13.720070726259618 - type: precision value: 12.052392225429521 - type: recall value: 20.35573122529644 task: type: BitextMining - dataset: config: pol_Latn-kan_Knda name: MTEB FloresBitextMining (pol_Latn-kan_Knda) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.201581027667984 - type: f1 value: 4.437171161571691 - type: main_score value: 4.437171161571691 - type: precision value: 3.749542197197035 - type: recall value: 8.201581027667984 task: type: BitextMining - dataset: config: pol_Latn-lmo_Latn name: MTEB FloresBitextMining (pol_Latn-lmo_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 38.83399209486166 - type: f1 value: 31.714524774447494 - type: main_score value: 31.714524774447494 - type: precision value: 29.391946711610746 - type: recall value: 38.83399209486166 task: type: BitextMining - dataset: config: pol_Latn-npi_Deva name: MTEB FloresBitextMining (pol_Latn-npi_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 10.968379446640316 - type: f1 value: 6.473612825827703 - type: main_score value: 6.473612825827703 - type: precision value: 5.49084990741001 - type: recall value: 10.968379446640316 task: type: BitextMining - dataset: config: pol_Latn-shn_Mymr name: MTEB FloresBitextMining (pol_Latn-shn_Mymr) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 38.43873517786561 - type: f1 value: 29.396836145848006 - type: main_score value: 29.396836145848006 - type: precision value: 26.534336875245966 - type: recall value: 38.43873517786561 task: type: BitextMining - dataset: config: pol_Latn-tgl_Latn name: MTEB FloresBitextMining (pol_Latn-tgl_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.70750988142292 - type: f1 value: 24.076005742744012 - type: main_score value: 24.076005742744012 - type: precision value: 21.6550168649971 - type: recall value: 32.70750988142292 task: type: BitextMining - dataset: config: pol_Latn-yue_Hant name: MTEB FloresBitextMining (pol_Latn-yue_Hant) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.980237154150199 - type: f1 value: 5.866820855869608 - type: main_score value: 5.866820855869608 - type: precision value: 4.994463315141073 - type: recall value: 9.980237154150199 task: type: BitextMining - dataset: config: pol_Latn-asm_Beng name: MTEB FloresBitextMining (pol_Latn-asm_Beng) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.027667984189724 - type: f1 value: 2.9919947985546753 - type: main_score value: 2.9919947985546753 - type: precision value: 2.396787621140054 - type: recall value: 6.027667984189724 task: type: BitextMining - dataset: config: pol_Latn-ckb_Arab name: MTEB FloresBitextMining (pol_Latn-ckb_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.0395256916996045 - type: f1 value: 2.6313333428590333 - type: main_score value: 2.6313333428590333 - type: precision value: 2.1441980481557246 - type: recall value: 5.0395256916996045 task: type: BitextMining - dataset: config: pol_Latn-gle_Latn name: MTEB FloresBitextMining (pol_Latn-gle_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.92490118577075 - type: f1 value: 16.366565597793155 - type: main_score value: 16.366565597793155 - type: precision value: 14.427773313642877 - type: recall value: 22.92490118577075 task: type: BitextMining - dataset: config: pol_Latn-kas_Arab name: MTEB FloresBitextMining (pol_Latn-kas_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.41106719367589 - type: f1 value: 4.268493390151145 - type: main_score value: 4.268493390151145 - type: precision value: 3.6318124097564928 - type: recall value: 7.41106719367589 task: type: BitextMining - dataset: config: pol_Latn-ltg_Latn name: MTEB FloresBitextMining (pol_Latn-ltg_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 28.85375494071146 - type: f1 value: 21.831210848523185 - type: main_score value: 21.831210848523185 - type: precision value: 19.65490879893054 - type: recall value: 28.85375494071146 task: type: BitextMining - dataset: config: pol_Latn-nso_Latn name: MTEB FloresBitextMining (pol_Latn-nso_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.051383399209485 - type: f1 value: 21.425082008345996 - type: main_score value: 21.425082008345996 - type: precision value: 19.24843153307464 - type: recall value: 29.051383399209485 task: type: BitextMining - dataset: config: pol_Latn-sin_Sinh name: MTEB FloresBitextMining (pol_Latn-sin_Sinh) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.992094861660078 - type: f1 value: 5.979100944100412 - type: main_score value: 5.979100944100412 - type: precision value: 5.385543481178876 - type: recall value: 8.992094861660078 task: type: BitextMining - dataset: config: pol_Latn-tha_Thai name: MTEB FloresBitextMining (pol_Latn-tha_Thai) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 15.909090909090908 - type: f1 value: 9.796795116437666 - type: main_score value: 9.796795116437666 - type: precision value: 8.360619704129169 - type: recall value: 15.909090909090908 task: type: BitextMining - dataset: config: pol_Latn-zho_Hans name: MTEB FloresBitextMining (pol_Latn-zho_Hans) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 16.50197628458498 - type: f1 value: 10.270309138975882 - type: main_score value: 10.270309138975882 - type: precision value: 8.73478574565531 - type: recall value: 16.50197628458498 task: type: BitextMining - dataset: config: pol_Latn-ast_Latn name: MTEB FloresBitextMining (pol_Latn-ast_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 47.92490118577075 - type: f1 value: 39.26617464660943 - type: main_score value: 39.26617464660943 - type: precision value: 36.18855836741212 - type: recall value: 47.92490118577075 task: type: BitextMining - dataset: config: pol_Latn-crh_Latn name: MTEB FloresBitextMining (pol_Latn-crh_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.197628458498023 - type: f1 value: 18.670974077359066 - type: main_score value: 18.670974077359066 - type: precision value: 16.954646692788984 - type: recall value: 25.197628458498023 task: type: BitextMining - dataset: config: pol_Latn-glg_Latn name: MTEB FloresBitextMining (pol_Latn-glg_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 52.37154150197628 - type: f1 value: 43.88566144000926 - type: main_score value: 43.88566144000926 - type: precision value: 40.966425976307406 - type: recall value: 52.37154150197628 task: type: BitextMining - dataset: config: pol_Latn-kas_Deva name: MTEB FloresBitextMining (pol_Latn-kas_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.209486166007903 - type: f1 value: 16.853008476346325 - type: main_score value: 16.853008476346325 - type: precision value: 14.851243130407093 - type: recall value: 24.209486166007903 task: type: BitextMining - dataset: config: pol_Latn-ltz_Latn name: MTEB FloresBitextMining (pol_Latn-ltz_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 44.86166007905138 - type: f1 value: 36.53990936599632 - type: main_score value: 36.53990936599632 - type: precision value: 33.71791123723814 - type: recall value: 44.86166007905138 task: type: BitextMining - dataset: config: pol_Latn-nus_Latn name: MTEB FloresBitextMining (pol_Latn-nus_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 15.612648221343871 - type: f1 value: 10.556552033824762 - type: main_score value: 10.556552033824762 - type: precision value: 9.225857138021011 - type: recall value: 15.612648221343871 task: type: BitextMining - dataset: config: pol_Latn-slk_Latn name: MTEB FloresBitextMining (pol_Latn-slk_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 41.60079051383399 - type: f1 value: 34.67679197175244 - type: main_score value: 34.67679197175244 - type: precision value: 32.45954710407592 - type: recall value: 41.60079051383399 task: type: BitextMining - dataset: config: pol_Latn-tir_Ethi name: MTEB FloresBitextMining (pol_Latn-tir_Ethi) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.719367588932807 - type: f1 value: 3.3274360285725493 - type: main_score value: 3.3274360285725493 - type: precision value: 2.8253990375336113 - type: recall value: 6.719367588932807 task: type: BitextMining - dataset: config: pol_Latn-zho_Hant name: MTEB FloresBitextMining (pol_Latn-zho_Hant) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 12.648221343873518 - type: f1 value: 7.529818632617995 - type: main_score value: 7.529818632617995 - type: precision value: 6.454425829925467 - type: recall value: 12.648221343873518 task: type: BitextMining - dataset: config: pol_Latn-awa_Deva name: MTEB FloresBitextMining (pol_Latn-awa_Deva) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.300395256916996 - type: f1 value: 4.53959024560111 - type: main_score value: 4.53959024560111 - type: precision value: 3.890776976863834 - type: recall value: 8.300395256916996 task: type: BitextMining - dataset: config: pol_Latn-cym_Latn name: MTEB FloresBitextMining (pol_Latn-cym_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.4703557312253 - type: f1 value: 19.68063199050937 - type: main_score value: 19.68063199050937 - type: precision value: 17.529989174140933 - type: recall value: 27.4703557312253 task: type: BitextMining - dataset: config: pol_Latn-grn_Latn name: MTEB FloresBitextMining (pol_Latn-grn_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 37.45059288537549 - type: f1 value: 28.71171428483681 - type: main_score value: 28.71171428483681 - type: precision value: 25.925313793526815 - type: recall value: 37.45059288537549 task: type: BitextMining - dataset: config: pol_Latn-kat_Geor name: MTEB FloresBitextMining (pol_Latn-kat_Geor) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.58498023715415 - type: f1 value: 5.5219657006276055 - type: main_score value: 5.5219657006276055 - type: precision value: 4.6367511696542865 - type: recall value: 9.58498023715415 task: type: BitextMining - dataset: config: pol_Latn-lua_Latn name: MTEB FloresBitextMining (pol_Latn-lua_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 30.92885375494071 - type: f1 value: 22.5116731094992 - type: main_score value: 22.5116731094992 - type: precision value: 20.106998654559504 - type: recall value: 30.92885375494071 task: type: BitextMining - dataset: config: pol_Latn-nya_Latn name: MTEB FloresBitextMining (pol_Latn-nya_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 30.335968379446644 - type: f1 value: 21.866230950695634 - type: main_score value: 21.866230950695634 - type: precision value: 19.365828866947027 - type: recall value: 30.335968379446644 task: type: BitextMining - dataset: config: pol_Latn-slv_Latn name: MTEB FloresBitextMining (pol_Latn-slv_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 37.54940711462451 - type: f1 value: 30.26678570994605 - type: main_score value: 30.26678570994605 - type: precision value: 27.977184847374154 - type: recall value: 37.54940711462451 task: type: BitextMining - dataset: config: pol_Latn-tpi_Latn name: MTEB FloresBitextMining (pol_Latn-tpi_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 54.841897233201585 - type: f1 value: 45.945094650628256 - type: main_score value: 45.945094650628256 - type: precision value: 42.74126894186182 - type: recall value: 54.841897233201585 task: type: BitextMining - dataset: config: pol_Latn-zsm_Latn name: MTEB FloresBitextMining (pol_Latn-zsm_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.31225296442688 - type: f1 value: 23.805788858652942 - type: main_score value: 23.805788858652942 - type: precision value: 21.37002869388159 - type: recall value: 32.31225296442688 task: type: BitextMining - dataset: config: pol_Latn-ayr_Latn name: MTEB FloresBitextMining (pol_Latn-ayr_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.50592885375494 - type: f1 value: 17.06755049189183 - type: main_score value: 17.06755049189183 - type: precision value: 15.074182726898094 - type: recall value: 24.50592885375494 task: type: BitextMining - dataset: config: pol_Latn-dan_Latn name: MTEB FloresBitextMining (pol_Latn-dan_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 47.72727272727273 - type: f1 value: 39.43143464301335 - type: main_score value: 39.43143464301335 - type: precision value: 36.54565995598604 - type: recall value: 47.72727272727273 task: type: BitextMining - dataset: config: pol_Latn-guj_Gujr name: MTEB FloresBitextMining (pol_Latn-guj_Gujr) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.324110671936759 - type: f1 value: 3.3326388017378736 - type: main_score value: 3.3326388017378736 - type: precision value: 2.757544255992551 - type: recall value: 6.324110671936759 task: type: BitextMining - dataset: config: pol_Latn-kaz_Cyrl name: MTEB FloresBitextMining (pol_Latn-kaz_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 11.758893280632412 - type: f1 value: 7.071173491888075 - type: main_score value: 7.071173491888075 - type: precision value: 6.050963825834047 - type: recall value: 11.758893280632412 task: type: BitextMining - dataset: config: pol_Latn-lug_Latn name: MTEB FloresBitextMining (pol_Latn-lug_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.889328063241106 - type: f1 value: 18.129380500922 - type: main_score value: 18.129380500922 - type: precision value: 16.074419054685272 - type: recall value: 25.889328063241106 task: type: BitextMining - dataset: config: pol_Latn-oci_Latn name: MTEB FloresBitextMining (pol_Latn-oci_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 53.55731225296443 - type: f1 value: 44.99836022563295 - type: main_score value: 44.99836022563295 - type: precision value: 42.04267049375744 - type: recall value: 53.55731225296443 task: type: BitextMining - dataset: config: pol_Latn-smo_Latn name: MTEB FloresBitextMining (pol_Latn-smo_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.889328063241106 - type: f1 value: 18.157441837399986 - type: main_score value: 18.157441837399986 - type: precision value: 16.178864593379544 - type: recall value: 25.889328063241106 task: type: BitextMining - dataset: config: pol_Latn-tsn_Latn name: MTEB FloresBitextMining (pol_Latn-tsn_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.865612648221344 - type: f1 value: 19.498233309640387 - type: main_score value: 19.498233309640387 - type: precision value: 17.179968828931276 - type: recall value: 27.865612648221344 task: type: BitextMining - dataset: config: pol_Latn-zul_Latn name: MTEB FloresBitextMining (pol_Latn-zul_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 18.774703557312254 - type: f1 value: 11.903397061759863 - type: main_score value: 11.903397061759863 - type: precision value: 10.198701796150065 - type: recall value: 18.774703557312254 task: type: BitextMining - dataset: config: pol_Latn-azb_Arab name: MTEB FloresBitextMining (pol_Latn-azb_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.545454545454546 - type: f1 value: 2.132585793652987 - type: main_score value: 2.132585793652987 - type: precision value: 1.643447767589644 - type: recall value: 4.545454545454546 task: type: BitextMining - dataset: config: pol_Latn-deu_Latn name: MTEB FloresBitextMining (pol_Latn-deu_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 51.18577075098815 - type: f1 value: 42.361535942679865 - type: main_score value: 42.361535942679865 - type: precision value: 39.02309907374136 - type: recall value: 51.18577075098815 task: type: BitextMining - dataset: config: pol_Latn-hat_Latn name: MTEB FloresBitextMining (pol_Latn-hat_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.56916996047431 - type: f1 value: 21.147938933628303 - type: main_score value: 21.147938933628303 - type: precision value: 19.18156560959233 - type: recall value: 27.56916996047431 task: type: BitextMining - dataset: config: pol_Latn-kbp_Latn name: MTEB FloresBitextMining (pol_Latn-kbp_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.90118577075099 - type: f1 value: 19.400134514030917 - type: main_score value: 19.400134514030917 - type: precision value: 17.88038314123646 - type: recall value: 24.90118577075099 task: type: BitextMining - dataset: config: pol_Latn-luo_Latn name: MTEB FloresBitextMining (pol_Latn-luo_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.988142292490117 - type: f1 value: 18.495583232417694 - type: main_score value: 18.495583232417694 - type: precision value: 16.526610238319726 - type: recall value: 25.988142292490117 task: type: BitextMining - dataset: config: pol_Latn-ory_Orya name: MTEB FloresBitextMining (pol_Latn-ory_Orya) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.201581027667984 - type: f1 value: 4.347389033695483 - type: main_score value: 4.347389033695483 - type: precision value: 3.6527450801940744 - type: recall value: 8.201581027667984 task: type: BitextMining - dataset: config: pol_Latn-sna_Latn name: MTEB FloresBitextMining (pol_Latn-sna_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.936758893280633 - type: f1 value: 13.626286791374994 - type: main_score value: 13.626286791374994 - type: precision value: 11.587003974907658 - type: recall value: 21.936758893280633 task: type: BitextMining - dataset: config: pol_Latn-tso_Latn name: MTEB FloresBitextMining (pol_Latn-tso_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.616600790513832 - type: f1 value: 15.920835945002468 - type: main_score value: 15.920835945002468 - type: precision value: 13.785758159275947 - type: recall value: 23.616600790513832 task: type: BitextMining - dataset: config: pol_Latn-azj_Latn name: MTEB FloresBitextMining (pol_Latn-azj_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.73913043478261 - type: f1 value: 16.54747537902312 - type: main_score value: 16.54747537902312 - type: precision value: 15.108106759292534 - type: recall value: 21.73913043478261 task: type: BitextMining - dataset: config: pol_Latn-dik_Latn name: MTEB FloresBitextMining (pol_Latn-dik_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 35.96837944664031 - type: f1 value: 28.780955713033933 - type: main_score value: 28.780955713033933 - type: precision value: 26.50347346987663 - type: recall value: 35.96837944664031 task: type: BitextMining - dataset: config: pol_Latn-hau_Latn name: MTEB FloresBitextMining (pol_Latn-hau_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.09881422924901 - type: f1 value: 17.50488293385294 - type: main_score value: 17.50488293385294 - type: precision value: 15.584319377114658 - type: recall value: 25.09881422924901 task: type: BitextMining - dataset: config: pol_Latn-kea_Latn name: MTEB FloresBitextMining (pol_Latn-kea_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 39.62450592885375 - type: f1 value: 31.270850168199622 - type: main_score value: 31.270850168199622 - type: precision value: 28.530398387831802 - type: recall value: 39.62450592885375 task: type: BitextMining - dataset: config: pol_Latn-lus_Latn name: MTEB FloresBitextMining (pol_Latn-lus_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 46.24505928853755 - type: f1 value: 37.202998472761315 - type: main_score value: 37.202998472761315 - type: precision value: 34.138708513708515 - type: recall value: 46.24505928853755 task: type: BitextMining - dataset: config: pol_Latn-pag_Latn name: MTEB FloresBitextMining (pol_Latn-pag_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 51.77865612648221 - type: f1 value: 42.621375385703445 - type: main_score value: 42.621375385703445 - type: precision value: 39.34708462376447 - type: recall value: 51.77865612648221 task: type: BitextMining - dataset: config: pol_Latn-snd_Arab name: MTEB FloresBitextMining (pol_Latn-snd_Arab) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.58498023715415 - type: f1 value: 6.1479644020877435 - type: main_score value: 6.1479644020877435 - type: precision value: 5.414916392558302 - type: recall value: 9.58498023715415 task: type: BitextMining - dataset: config: pol_Latn-tuk_Latn name: MTEB FloresBitextMining (pol_Latn-tuk_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.023715415019762 - type: f1 value: 17.26881292064288 - type: main_score value: 17.26881292064288 - type: precision value: 15.607159530883665 - type: recall value: 23.023715415019762 task: type: BitextMining - dataset: config: pol_Latn-bak_Cyrl name: MTEB FloresBitextMining (pol_Latn-bak_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.288537549407115 - type: f1 value: 5.527728406351995 - type: main_score value: 5.527728406351995 - type: precision value: 4.713201013355736 - type: recall value: 9.288537549407115 task: type: BitextMining - dataset: config: pol_Latn-dyu_Latn name: MTEB FloresBitextMining (pol_Latn-dyu_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.91304347826087 - type: f1 value: 17.018730336418084 - type: main_score value: 17.018730336418084 - type: precision value: 14.966286314060392 - type: recall value: 23.91304347826087 task: type: BitextMining - dataset: config: pol_Latn-heb_Hebr name: MTEB FloresBitextMining (pol_Latn-heb_Hebr) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.980237154150199 - type: f1 value: 5.4971546949165875 - type: main_score value: 5.4971546949165875 - type: precision value: 4.676576441810379 - type: recall value: 9.980237154150199 task: type: BitextMining - dataset: config: pol_Latn-khk_Cyrl name: MTEB FloresBitextMining (pol_Latn-khk_Cyrl) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.707509881422925 - type: f1 value: 4.6221237336068315 - type: main_score value: 4.6221237336068315 - type: precision value: 3.9456592706208924 - type: recall value: 7.707509881422925 task: type: BitextMining - dataset: config: pol_Latn-lvs_Latn name: MTEB FloresBitextMining (pol_Latn-lvs_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.122529644268774 - type: f1 value: 16.608556682667356 - type: main_score value: 16.608556682667356 - type: precision value: 14.73247944226003 - type: recall value: 23.122529644268774 task: type: BitextMining - dataset: config: pol_Latn-pan_Guru name: MTEB FloresBitextMining (pol_Latn-pan_Guru) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 8.992094861660078 - type: f1 value: 4.8126673611522595 - type: main_score value: 4.8126673611522595 - type: precision value: 3.9900445824655755 - type: recall value: 8.992094861660078 task: type: BitextMining - dataset: config: pol_Latn-som_Latn name: MTEB FloresBitextMining (pol_Latn-som_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.616600790513832 - type: f1 value: 16.280717795842147 - type: main_score value: 16.280717795842147 - type: precision value: 14.38424443365155 - type: recall value: 23.616600790513832 task: type: BitextMining - dataset: config: pol_Latn-tum_Latn name: MTEB FloresBitextMining (pol_Latn-tum_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.173913043478258 - type: f1 value: 19.45508871892271 - type: main_score value: 19.45508871892271 - type: precision value: 17.333758687513626 - type: recall value: 27.173913043478258 task: type: BitextMining - dataset: config: ssw_Latn-pol_Latn name: MTEB FloresBitextMining (ssw_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 13.636363636363635 - type: f1 value: 11.545325158304745 - type: main_score value: 11.545325158304745 - type: precision value: 11.04849034050014 - type: recall value: 13.636363636363635 task: type: BitextMining - dataset: config: ukr_Cyrl-pol_Latn name: MTEB FloresBitextMining (ukr_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.3478260869565215 - type: f1 value: 3.36055023938623 - type: main_score value: 3.36055023938623 - type: precision value: 3.145713238288773 - type: recall value: 4.3478260869565215 task: type: BitextMining - dataset: config: afr_Latn-pol_Latn name: MTEB FloresBitextMining (afr_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 37.351778656126484 - type: f1 value: 32.657254375087675 - type: main_score value: 32.657254375087675 - type: precision value: 31.31990798417892 - type: recall value: 37.351778656126484 task: type: BitextMining - dataset: config: bho_Deva-pol_Latn name: MTEB FloresBitextMining (bho_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.434782608695652 - type: f1 value: 4.248538461856328 - type: main_score value: 4.248538461856328 - type: precision value: 3.9363477747469755 - type: recall value: 5.434782608695652 task: type: BitextMining - dataset: config: eus_Latn-pol_Latn name: MTEB FloresBitextMining (eus_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.249011857707508 - type: f1 value: 24.621199842181067 - type: main_score value: 24.621199842181067 - type: precision value: 23.318479076012707 - type: recall value: 29.249011857707508 task: type: BitextMining - dataset: config: ibo_Latn-pol_Latn name: MTEB FloresBitextMining (ibo_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 19.66403162055336 - type: f1 value: 16.366065476178022 - type: main_score value: 16.366065476178022 - type: precision value: 15.483095342831795 - type: recall value: 19.66403162055336 task: type: BitextMining - dataset: config: kmr_Latn-pol_Latn name: MTEB FloresBitextMining (kmr_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 15.810276679841898 - type: f1 value: 13.131433621845694 - type: main_score value: 13.131433621845694 - type: precision value: 12.496160503188007 - type: recall value: 15.810276679841898 task: type: BitextMining - dataset: config: min_Latn-pol_Latn name: MTEB FloresBitextMining (min_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.221343873517785 - type: f1 value: 20.460829297482324 - type: main_score value: 20.460829297482324 - type: precision value: 19.666039753880405 - type: recall value: 23.221343873517785 task: type: BitextMining - dataset: config: por_Latn-pol_Latn name: MTEB FloresBitextMining (por_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.608695652173914 - type: f1 value: 30.038782875813897 - type: main_score value: 30.038782875813897 - type: precision value: 29.3346482933393 - type: recall value: 32.608695652173914 task: type: BitextMining - dataset: config: sun_Latn-pol_Latn name: MTEB FloresBitextMining (sun_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 26.284584980237153 - type: f1 value: 23.167693438533703 - type: main_score value: 23.167693438533703 - type: precision value: 22.344726485619383 - type: recall value: 26.284584980237153 task: type: BitextMining - dataset: config: umb_Latn-pol_Latn name: MTEB FloresBitextMining (umb_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 14.229249011857709 - type: f1 value: 11.72032447043487 - type: main_score value: 11.72032447043487 - type: precision value: 11.113345808470575 - type: recall value: 14.229249011857709 task: type: BitextMining - dataset: config: ajp_Arab-pol_Latn name: MTEB FloresBitextMining (ajp_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.027667984189724 - type: f1 value: 4.906383446809295 - type: main_score value: 4.906383446809295 - type: precision value: 4.626812580140678 - type: recall value: 6.027667984189724 task: type: BitextMining - dataset: config: bjn_Arab-pol_Latn name: MTEB FloresBitextMining (bjn_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 0.2964426877470355 - type: f1 value: 0.014739619645471455 - type: main_score value: 0.014739619645471455 - type: precision value: 0.007793259862334742 - type: recall value: 0.2964426877470355 task: type: BitextMining - dataset: config: ewe_Latn-pol_Latn name: MTEB FloresBitextMining (ewe_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.3201581027668 - type: f1 value: 19.752939023865142 - type: main_score value: 19.752939023865142 - type: precision value: 18.695225242144133 - type: recall value: 23.3201581027668 task: type: BitextMining - dataset: config: ilo_Latn-pol_Latn name: MTEB FloresBitextMining (ilo_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 26.877470355731226 - type: f1 value: 24.036761426467308 - type: main_score value: 24.036761426467308 - type: precision value: 23.214768526844647 - type: recall value: 26.877470355731226 task: type: BitextMining - dataset: config: knc_Arab-pol_Latn name: MTEB FloresBitextMining (knc_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.422924901185771 - type: f1 value: 5.471861993108544 - type: main_score value: 5.471861993108544 - type: precision value: 5.208482825359429 - type: recall value: 6.422924901185771 task: type: BitextMining - dataset: config: mkd_Cyrl-pol_Latn name: MTEB FloresBitextMining (mkd_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.743083003952568 - type: f1 value: 3.8745999077711124 - type: main_score value: 3.8745999077711124 - type: precision value: 3.6662900574163095 - type: recall value: 4.743083003952568 task: type: BitextMining - dataset: config: prs_Arab-pol_Latn name: MTEB FloresBitextMining (prs_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.8300395256917 - type: f1 value: 4.67672888868541 - type: main_score value: 4.67672888868541 - type: precision value: 4.385871767652376 - type: recall value: 5.8300395256917 task: type: BitextMining - dataset: config: swe_Latn-pol_Latn name: MTEB FloresBitextMining (swe_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 39.130434782608695 - type: f1 value: 35.40899916648588 - type: main_score value: 35.40899916648588 - type: precision value: 34.37500839081867 - type: recall value: 39.130434782608695 task: type: BitextMining - dataset: config: urd_Arab-pol_Latn name: MTEB FloresBitextMining (urd_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.8656126482213438 - type: f1 value: 2.198814997617321 - type: main_score value: 2.198814997617321 - type: precision value: 2.0934232972037385 - type: recall value: 2.8656126482213438 task: type: BitextMining - dataset: config: aka_Latn-pol_Latn name: MTEB FloresBitextMining (aka_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 34.88142292490119 - type: f1 value: 29.74508937764049 - type: main_score value: 29.74508937764049 - type: precision value: 28.153705393466193 - type: recall value: 34.88142292490119 task: type: BitextMining - dataset: config: bjn_Latn-pol_Latn name: MTEB FloresBitextMining (bjn_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 18.675889328063242 - type: f1 value: 16.43948645657555 - type: main_score value: 16.43948645657555 - type: precision value: 15.862834370794616 - type: recall value: 18.675889328063242 task: type: BitextMining - dataset: config: fao_Latn-pol_Latn name: MTEB FloresBitextMining (fao_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.395256916996047 - type: f1 value: 21.625753426979184 - type: main_score value: 21.625753426979184 - type: precision value: 20.774449730898926 - type: recall value: 25.395256916996047 task: type: BitextMining - dataset: config: ind_Latn-pol_Latn name: MTEB FloresBitextMining (ind_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.494071146245062 - type: f1 value: 22.89853629340138 - type: main_score value: 22.89853629340138 - type: precision value: 22.19346819224176 - type: recall value: 25.494071146245062 task: type: BitextMining - dataset: config: knc_Latn-pol_Latn name: MTEB FloresBitextMining (knc_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 26.778656126482215 - type: f1 value: 23.135121575804448 - type: main_score value: 23.135121575804448 - type: precision value: 22.197098986477734 - type: recall value: 26.778656126482215 task: type: BitextMining - dataset: config: mlt_Latn-pol_Latn name: MTEB FloresBitextMining (mlt_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.27272727272727 - type: f1 value: 23.26113833343015 - type: main_score value: 23.26113833343015 - type: precision value: 22.291588018440425 - type: recall value: 27.27272727272727 task: type: BitextMining - dataset: config: quy_Latn-pol_Latn name: MTEB FloresBitextMining (quy_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.640316205533598 - type: f1 value: 18.324982985871475 - type: main_score value: 18.324982985871475 - type: precision value: 17.384170773661005 - type: recall value: 21.640316205533598 task: type: BitextMining - dataset: config: swh_Latn-pol_Latn name: MTEB FloresBitextMining (swh_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 16.50197628458498 - type: f1 value: 13.784780581928224 - type: main_score value: 13.784780581928224 - type: precision value: 13.15906973436561 - type: recall value: 16.50197628458498 task: type: BitextMining - dataset: config: uzn_Latn-pol_Latn name: MTEB FloresBitextMining (uzn_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 13.73517786561265 - type: f1 value: 11.7621999636561 - type: main_score value: 11.7621999636561 - type: precision value: 11.21864560930682 - type: recall value: 13.73517786561265 task: type: BitextMining - dataset: config: als_Latn-pol_Latn name: MTEB FloresBitextMining (als_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.64426877470356 - type: f1 value: 25.093862889528456 - type: main_score value: 25.093862889528456 - type: precision value: 23.85649046493975 - type: recall value: 29.64426877470356 task: type: BitextMining - dataset: config: bod_Tibt-pol_Latn name: MTEB FloresBitextMining (bod_Tibt-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 1.8774703557312251 - type: f1 value: 1.482427322348271 - type: main_score value: 1.482427322348271 - type: precision value: 1.3969415351679186 - type: recall value: 1.8774703557312251 task: type: BitextMining - dataset: config: fij_Latn-pol_Latn name: MTEB FloresBitextMining (fij_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.588932806324113 - type: f1 value: 15.044924633719347 - type: main_score value: 15.044924633719347 - type: precision value: 14.422284220226055 - type: recall value: 17.588932806324113 task: type: BitextMining - dataset: config: isl_Latn-pol_Latn name: MTEB FloresBitextMining (isl_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 20.059288537549406 - type: f1 value: 16.993604061124348 - type: main_score value: 16.993604061124348 - type: precision value: 16.38307404546514 - type: recall value: 20.059288537549406 task: type: BitextMining - dataset: config: kon_Latn-pol_Latn name: MTEB FloresBitextMining (kon_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 20.059288537549406 - type: f1 value: 17.30021401344778 - type: main_score value: 17.30021401344778 - type: precision value: 16.5997172620091 - type: recall value: 20.059288537549406 task: type: BitextMining - dataset: config: mni_Beng-pol_Latn name: MTEB FloresBitextMining (mni_Beng-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.766798418972332 - type: f1 value: 1.8188857320273997 - type: main_score value: 1.8188857320273997 - type: precision value: 1.596105526645877 - type: recall value: 2.766798418972332 task: type: BitextMining - dataset: config: ron_Latn-pol_Latn name: MTEB FloresBitextMining (ron_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 44.762845849802375 - type: f1 value: 40.20969594868712 - type: main_score value: 40.20969594868712 - type: precision value: 38.92095571914066 - type: recall value: 44.762845849802375 task: type: BitextMining - dataset: config: szl_Latn-pol_Latn name: MTEB FloresBitextMining (szl_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 88.93280632411067 - type: f1 value: 86.11989459815547 - type: main_score value: 86.11989459815547 - type: precision value: 84.89295125164689 - type: recall value: 88.93280632411067 task: type: BitextMining - dataset: config: vec_Latn-pol_Latn name: MTEB FloresBitextMining (vec_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 33.201581027667984 - type: f1 value: 30.53786572808312 - type: main_score value: 30.53786572808312 - type: precision value: 29.781267755341258 - type: recall value: 33.201581027667984 task: type: BitextMining - dataset: config: amh_Ethi-pol_Latn name: MTEB FloresBitextMining (amh_Ethi-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.8656126482213438 - type: f1 value: 2.3343702175686536 - type: main_score value: 2.3343702175686536 - type: precision value: 2.225088157792761 - type: recall value: 2.8656126482213438 task: type: BitextMining - dataset: config: bos_Latn-pol_Latn name: MTEB FloresBitextMining (bos_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 40.81027667984189 - type: f1 value: 35.816068036824284 - type: main_score value: 35.816068036824284 - type: precision value: 34.36131947680184 - type: recall value: 40.81027667984189 task: type: BitextMining - dataset: config: fin_Latn-pol_Latn name: MTEB FloresBitextMining (fin_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.047430830039527 - type: f1 value: 18.32388688372778 - type: main_score value: 18.32388688372778 - type: precision value: 17.65836451963984 - type: recall value: 21.047430830039527 task: type: BitextMining - dataset: config: ita_Latn-pol_Latn name: MTEB FloresBitextMining (ita_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 26.284584980237153 - type: f1 value: 25.328031148557844 - type: main_score value: 25.328031148557844 - type: precision value: 25.0573427033527 - type: recall value: 26.284584980237153 task: type: BitextMining - dataset: config: kor_Hang-pol_Latn name: MTEB FloresBitextMining (kor_Hang-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.225296442687747 - type: f1 value: 5.624717838156572 - type: main_score value: 5.624717838156572 - type: precision value: 5.436776229110326 - type: recall value: 6.225296442687747 task: type: BitextMining - dataset: config: mos_Latn-pol_Latn name: MTEB FloresBitextMining (mos_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 19.861660079051383 - type: f1 value: 16.686949600048273 - type: main_score value: 16.686949600048273 - type: precision value: 15.759010718370876 - type: recall value: 19.861660079051383 task: type: BitextMining - dataset: config: run_Latn-pol_Latn name: MTEB FloresBitextMining (run_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 15.41501976284585 - type: f1 value: 12.844131292483732 - type: main_score value: 12.844131292483732 - type: precision value: 12.239314986108212 - type: recall value: 15.41501976284585 task: type: BitextMining - dataset: config: tam_Taml-pol_Latn name: MTEB FloresBitextMining (tam_Taml-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.138339920948617 - type: f1 value: 4.107577623493304 - type: main_score value: 4.107577623493304 - type: precision value: 3.8837297105930646 - type: recall value: 5.138339920948617 task: type: BitextMining - dataset: config: vie_Latn-pol_Latn name: MTEB FloresBitextMining (vie_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 25.889328063241106 - type: f1 value: 22.579397348874743 - type: main_score value: 22.579397348874743 - type: precision value: 21.595638551483958 - type: recall value: 25.889328063241106 task: type: BitextMining - dataset: config: apc_Arab-pol_Latn name: MTEB FloresBitextMining (apc_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.8300395256917 - type: f1 value: 4.771345020289305 - type: main_score value: 4.771345020289305 - type: precision value: 4.531490987141249 - type: recall value: 5.8300395256917 task: type: BitextMining - dataset: config: bug_Latn-pol_Latn name: MTEB FloresBitextMining (bug_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.91304347826087 - type: f1 value: 20.07699048356012 - type: main_score value: 20.07699048356012 - type: precision value: 19.08000382091288 - type: recall value: 23.91304347826087 task: type: BitextMining - dataset: config: fon_Latn-pol_Latn name: MTEB FloresBitextMining (fon_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.802371541501977 - type: f1 value: 21.068053623570577 - type: main_score value: 21.068053623570577 - type: precision value: 20.049528384718315 - type: recall value: 24.802371541501977 task: type: BitextMining - dataset: config: jav_Latn-pol_Latn name: MTEB FloresBitextMining (jav_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 19.861660079051383 - type: f1 value: 17.8252060266033 - type: main_score value: 17.8252060266033 - type: precision value: 17.232613108208085 - type: recall value: 19.861660079051383 task: type: BitextMining - dataset: config: lao_Laoo-pol_Latn name: MTEB FloresBitextMining (lao_Laoo-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 28.952569169960473 - type: f1 value: 26.545881654577308 - type: main_score value: 26.545881654577308 - type: precision value: 25.711840655780314 - type: recall value: 28.952569169960473 task: type: BitextMining - dataset: config: mri_Latn-pol_Latn name: MTEB FloresBitextMining (mri_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.786561264822133 - type: f1 value: 15.783709557066814 - type: main_score value: 15.783709557066814 - type: precision value: 15.323213600988495 - type: recall value: 17.786561264822133 task: type: BitextMining - dataset: config: rus_Cyrl-pol_Latn name: MTEB FloresBitextMining (rus_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.7312252964426875 - type: f1 value: 4.633980959784417 - type: main_score value: 4.633980959784417 - type: precision value: 4.381700463126679 - type: recall value: 5.7312252964426875 task: type: BitextMining - dataset: config: taq_Latn-pol_Latn name: MTEB FloresBitextMining (taq_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 28.85375494071146 - type: f1 value: 24.7016707123012 - type: main_score value: 24.7016707123012 - type: precision value: 23.47897679898959 - type: recall value: 28.85375494071146 task: type: BitextMining - dataset: config: war_Latn-pol_Latn name: MTEB FloresBitextMining (war_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 38.63636363636363 - type: f1 value: 34.295490067770984 - type: main_score value: 34.295490067770984 - type: precision value: 33.21862519220829 - type: recall value: 38.63636363636363 task: type: BitextMining - dataset: config: arb_Arab-pol_Latn name: MTEB FloresBitextMining (arb_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.6561264822134385 - type: f1 value: 2.773676919366406 - type: main_score value: 2.773676919366406 - type: precision value: 2.6137121358188073 - type: recall value: 3.6561264822134385 task: type: BitextMining - dataset: config: bul_Cyrl-pol_Latn name: MTEB FloresBitextMining (bul_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.632411067193676 - type: f1 value: 4.504705439488048 - type: main_score value: 4.504705439488048 - type: precision value: 4.22258230186781 - type: recall value: 5.632411067193676 task: type: BitextMining - dataset: config: fra_Latn-pol_Latn name: MTEB FloresBitextMining (fra_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 45.65217391304348 - type: f1 value: 42.319337266121366 - type: main_score value: 42.319337266121366 - type: precision value: 41.337433711384975 - type: recall value: 45.65217391304348 task: type: BitextMining - dataset: config: jpn_Jpan-pol_Latn name: MTEB FloresBitextMining (jpn_Jpan-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 1.7786561264822136 - type: f1 value: 1.5436431670974948 - type: main_score value: 1.5436431670974948 - type: precision value: 1.4988121544095525 - type: recall value: 1.7786561264822136 task: type: BitextMining - dataset: config: lij_Latn-pol_Latn name: MTEB FloresBitextMining (lij_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.11462450592885 - type: f1 value: 29.849869097168057 - type: main_score value: 29.849869097168057 - type: precision value: 29.154947704114285 - type: recall value: 32.11462450592885 task: type: BitextMining - dataset: config: mya_Mymr-pol_Latn name: MTEB FloresBitextMining (mya_Mymr-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.0395256916996045 - type: f1 value: 4.519431283756075 - type: main_score value: 4.519431283756075 - type: precision value: 4.356224477087554 - type: recall value: 5.0395256916996045 task: type: BitextMining - dataset: config: sag_Latn-pol_Latn name: MTEB FloresBitextMining (sag_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.094861660079054 - type: f1 value: 14.637628173381545 - type: main_score value: 14.637628173381545 - type: precision value: 14.112004954025323 - type: recall value: 17.094861660079054 task: type: BitextMining - dataset: config: taq_Tfng-pol_Latn name: MTEB FloresBitextMining (taq_Tfng-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.632411067193676 - type: f1 value: 4.909914312233067 - type: main_score value: 4.909914312233067 - type: precision value: 4.724626945040803 - type: recall value: 5.632411067193676 task: type: BitextMining - dataset: config: wol_Latn-pol_Latn name: MTEB FloresBitextMining (wol_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 23.616600790513832 - type: f1 value: 20.279347388097182 - type: main_score value: 20.279347388097182 - type: precision value: 19.46070794354514 - type: recall value: 23.616600790513832 task: type: BitextMining - dataset: config: arb_Latn-pol_Latn name: MTEB FloresBitextMining (arb_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 9.288537549407115 - type: f1 value: 8.086159010072054 - type: main_score value: 8.086159010072054 - type: precision value: 7.813028740140765 - type: recall value: 9.288537549407115 task: type: BitextMining - dataset: config: cat_Latn-pol_Latn name: MTEB FloresBitextMining (cat_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 38.24110671936759 - type: f1 value: 35.27552312466981 - type: main_score value: 35.27552312466981 - type: precision value: 34.31791722698166 - type: recall value: 38.24110671936759 task: type: BitextMining - dataset: config: fur_Latn-pol_Latn name: MTEB FloresBitextMining (fur_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 34.387351778656125 - type: f1 value: 31.058068019828706 - type: main_score value: 31.058068019828706 - type: precision value: 30.052825623033737 - type: recall value: 34.387351778656125 task: type: BitextMining - dataset: config: kab_Latn-pol_Latn name: MTEB FloresBitextMining (kab_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.806324110671936 - type: f1 value: 5.902874997333144 - type: main_score value: 5.902874997333144 - type: precision value: 5.423746587330392 - type: recall value: 7.806324110671936 task: type: BitextMining - dataset: config: lim_Latn-pol_Latn name: MTEB FloresBitextMining (lim_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 39.723320158102766 - type: f1 value: 35.389001290356084 - type: main_score value: 35.389001290356084 - type: precision value: 34.13435535421377 - type: recall value: 39.723320158102766 task: type: BitextMining - dataset: config: nld_Latn-pol_Latn name: MTEB FloresBitextMining (nld_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 42.094861660079054 - type: f1 value: 37.93291320500714 - type: main_score value: 37.93291320500714 - type: precision value: 36.767114618952355 - type: recall value: 42.094861660079054 task: type: BitextMining - dataset: config: san_Deva-pol_Latn name: MTEB FloresBitextMining (san_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.632411067193676 - type: f1 value: 4.660741759396567 - type: main_score value: 4.660741759396567 - type: precision value: 4.430956752636382 - type: recall value: 5.632411067193676 task: type: BitextMining - dataset: config: tat_Cyrl-pol_Latn name: MTEB FloresBitextMining (tat_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.928853754940711 - type: f1 value: 4.929180488481535 - type: main_score value: 4.929180488481535 - type: precision value: 4.674956123463861 - type: recall value: 5.928853754940711 task: type: BitextMining - dataset: config: xho_Latn-pol_Latn name: MTEB FloresBitextMining (xho_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 16.40316205533597 - type: f1 value: 14.1584637782076 - type: main_score value: 14.1584637782076 - type: precision value: 13.551169277747055 - type: recall value: 16.40316205533597 task: type: BitextMining - dataset: config: ars_Arab-pol_Latn name: MTEB FloresBitextMining (ars_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.075098814229249 - type: f1 value: 1.5663001729144188 - type: main_score value: 1.5663001729144188 - type: precision value: 1.5034847066262733 - type: recall value: 2.075098814229249 task: type: BitextMining - dataset: config: ceb_Latn-pol_Latn name: MTEB FloresBitextMining (ceb_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 27.766798418972332 - type: f1 value: 25.138248191646635 - type: main_score value: 25.138248191646635 - type: precision value: 24.52148356634336 - type: recall value: 27.766798418972332 task: type: BitextMining - dataset: config: fuv_Latn-pol_Latn name: MTEB FloresBitextMining (fuv_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 20.652173913043477 - type: f1 value: 18.497889827878236 - type: main_score value: 18.497889827878236 - type: precision value: 17.894908363824175 - type: recall value: 20.652173913043477 task: type: BitextMining - dataset: config: kac_Latn-pol_Latn name: MTEB FloresBitextMining (kac_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 18.675889328063242 - type: f1 value: 16.47500264090945 - type: main_score value: 16.47500264090945 - type: precision value: 15.965045864487637 - type: recall value: 18.675889328063242 task: type: BitextMining - dataset: config: lin_Latn-pol_Latn name: MTEB FloresBitextMining (lin_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 19.861660079051383 - type: f1 value: 16.669395867367935 - type: main_score value: 16.669395867367935 - type: precision value: 15.996611411786905 - type: recall value: 19.861660079051383 task: type: BitextMining - dataset: config: nno_Latn-pol_Latn name: MTEB FloresBitextMining (nno_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 37.74703557312253 - type: f1 value: 33.43750147323936 - type: main_score value: 33.43750147323936 - type: precision value: 32.26544682279166 - type: recall value: 37.74703557312253 task: type: BitextMining - dataset: config: sat_Olck-pol_Latn name: MTEB FloresBitextMining (sat_Olck-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 0.9881422924901186 - type: f1 value: 0.738017708056005 - type: main_score value: 0.738017708056005 - type: precision value: 0.6918021091302707 - type: recall value: 0.9881422924901186 task: type: BitextMining - dataset: config: tel_Telu-pol_Latn name: MTEB FloresBitextMining (tel_Telu-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.806324110671936 - type: f1 value: 6.455909948121061 - type: main_score value: 6.455909948121061 - type: precision value: 6.10016680289029 - type: recall value: 7.806324110671936 task: type: BitextMining - dataset: config: ydd_Hebr-pol_Latn name: MTEB FloresBitextMining (ydd_Hebr-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.8300395256917 - type: f1 value: 4.714103592822127 - type: main_score value: 4.714103592822127 - type: precision value: 4.418610320290162 - type: recall value: 5.8300395256917 task: type: BitextMining - dataset: config: ary_Arab-pol_Latn name: MTEB FloresBitextMining (ary_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.9525691699604746 - type: f1 value: 3.038700967670975 - type: main_score value: 3.038700967670975 - type: precision value: 2.852792175238853 - type: recall value: 3.9525691699604746 task: type: BitextMining - dataset: config: ces_Latn-pol_Latn name: MTEB FloresBitextMining (ces_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 43.3794466403162 - type: f1 value: 38.33097409419722 - type: main_score value: 38.33097409419722 - type: precision value: 36.81077971825639 - type: recall value: 43.3794466403162 task: type: BitextMining - dataset: config: gaz_Latn-pol_Latn name: MTEB FloresBitextMining (gaz_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.841897233201581 - type: f1 value: 3.749327265034994 - type: main_score value: 3.749327265034994 - type: precision value: 3.5976880947568186 - type: recall value: 4.841897233201581 task: type: BitextMining - dataset: config: kam_Latn-pol_Latn name: MTEB FloresBitextMining (kam_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.82608695652174 - type: f1 value: 19.26086834997352 - type: main_score value: 19.26086834997352 - type: precision value: 18.374103098537436 - type: recall value: 22.82608695652174 task: type: BitextMining - dataset: config: lit_Latn-pol_Latn name: MTEB FloresBitextMining (lit_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.50592885375494 - type: f1 value: 20.41895151835619 - type: main_score value: 20.41895151835619 - type: precision value: 19.338293047532176 - type: recall value: 24.50592885375494 task: type: BitextMining - dataset: config: nob_Latn-pol_Latn name: MTEB FloresBitextMining (nob_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 36.6600790513834 - type: f1 value: 32.77972950668669 - type: main_score value: 32.77972950668669 - type: precision value: 31.673807638587903 - type: recall value: 36.6600790513834 task: type: BitextMining - dataset: config: scn_Latn-pol_Latn name: MTEB FloresBitextMining (scn_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 20.652173913043477 - type: f1 value: 19.09631332716127 - type: main_score value: 19.09631332716127 - type: precision value: 18.58198848760543 - type: recall value: 20.652173913043477 task: type: BitextMining - dataset: config: tgk_Cyrl-pol_Latn name: MTEB FloresBitextMining (tgk_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.533596837944664 - type: f1 value: 4.55426300946565 - type: main_score value: 4.55426300946565 - type: precision value: 4.305510392181988 - type: recall value: 5.533596837944664 task: type: BitextMining - dataset: config: yor_Latn-pol_Latn name: MTEB FloresBitextMining (yor_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.011857707509883 - type: f1 value: 20.4852283680085 - type: main_score value: 20.4852283680085 - type: precision value: 19.307145104230646 - type: recall value: 24.011857707509883 task: type: BitextMining - dataset: config: arz_Arab-pol_Latn name: MTEB FloresBitextMining (arz_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.8656126482213438 - type: f1 value: 2.050382343389291 - type: main_score value: 2.050382343389291 - type: precision value: 1.9244732677291967 - type: recall value: 2.8656126482213438 task: type: BitextMining - dataset: config: cjk_Latn-pol_Latn name: MTEB FloresBitextMining (cjk_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 15.909090909090908 - type: f1 value: 13.094697079040241 - type: main_score value: 13.094697079040241 - type: precision value: 12.451127946209137 - type: recall value: 15.909090909090908 task: type: BitextMining - dataset: config: gla_Latn-pol_Latn name: MTEB FloresBitextMining (gla_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 14.229249011857709 - type: f1 value: 11.8958091961019 - type: main_score value: 11.8958091961019 - type: precision value: 11.308813312132354 - type: recall value: 14.229249011857709 task: type: BitextMining - dataset: config: kan_Knda-pol_Latn name: MTEB FloresBitextMining (kan_Knda-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.446640316205533 - type: f1 value: 3.543866950918991 - type: main_score value: 3.543866950918991 - type: precision value: 3.344953602611705 - type: recall value: 4.446640316205533 task: type: BitextMining - dataset: config: lmo_Latn-pol_Latn name: MTEB FloresBitextMining (lmo_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 31.818181818181817 - type: f1 value: 28.84565227513798 - type: main_score value: 28.84565227513798 - type: precision value: 27.901861094548845 - type: recall value: 31.818181818181817 task: type: BitextMining - dataset: config: npi_Deva-pol_Latn name: MTEB FloresBitextMining (npi_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.126482213438735 - type: f1 value: 4.995296774703681 - type: main_score value: 4.995296774703681 - type: precision value: 4.716743907131579 - type: recall value: 6.126482213438735 task: type: BitextMining - dataset: config: shn_Mymr-pol_Latn name: MTEB FloresBitextMining (shn_Mymr-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.41106719367589 - type: f1 value: 29.33468087609507 - type: main_score value: 29.33468087609507 - type: precision value: 28.217299436243536 - type: recall value: 32.41106719367589 task: type: BitextMining - dataset: config: tgl_Latn-pol_Latn name: MTEB FloresBitextMining (tgl_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 19.960474308300398 - type: f1 value: 17.358391772917464 - type: main_score value: 17.358391772917464 - type: precision value: 16.714674395610597 - type: recall value: 19.960474308300398 task: type: BitextMining - dataset: config: yue_Hant-pol_Latn name: MTEB FloresBitextMining (yue_Hant-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.8300395256917 - type: f1 value: 5.1056101485728576 - type: main_score value: 5.1056101485728576 - type: precision value: 4.867798538747156 - type: recall value: 5.8300395256917 task: type: BitextMining - dataset: config: asm_Beng-pol_Latn name: MTEB FloresBitextMining (asm_Beng-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.766798418972332 - type: f1 value: 2.1076417295698957 - type: main_score value: 2.1076417295698957 - type: precision value: 1.9539846461201957 - type: recall value: 2.766798418972332 task: type: BitextMining - dataset: config: ckb_Arab-pol_Latn name: MTEB FloresBitextMining (ckb_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.4703557312252964 - type: f1 value: 1.7603256093421709 - type: main_score value: 1.7603256093421709 - type: precision value: 1.5924946194322374 - type: recall value: 2.4703557312252964 task: type: BitextMining - dataset: config: gle_Latn-pol_Latn name: MTEB FloresBitextMining (gle_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.490118577075098 - type: f1 value: 15.1736996743734 - type: main_score value: 15.1736996743734 - type: precision value: 14.611817031828728 - type: recall value: 17.490118577075098 task: type: BitextMining - dataset: config: kas_Arab-pol_Latn name: MTEB FloresBitextMining (kas_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.150197628458498 - type: f1 value: 3.4109750619615435 - type: main_score value: 3.4109750619615435 - type: precision value: 3.2236152439722527 - type: recall value: 4.150197628458498 task: type: BitextMining - dataset: config: ltg_Latn-pol_Latn name: MTEB FloresBitextMining (ltg_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 26.778656126482215 - type: f1 value: 22.413478254607035 - type: main_score value: 22.413478254607035 - type: precision value: 21.276900635762036 - type: recall value: 26.778656126482215 task: type: BitextMining - dataset: config: nso_Latn-pol_Latn name: MTEB FloresBitextMining (nso_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.011857707509883 - type: f1 value: 20.924527200344738 - type: main_score value: 20.924527200344738 - type: precision value: 20.22077114271249 - type: recall value: 24.011857707509883 task: type: BitextMining - dataset: config: sin_Sinh-pol_Latn name: MTEB FloresBitextMining (sin_Sinh-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.3478260869565215 - type: f1 value: 3.346071948603057 - type: main_score value: 3.346071948603057 - type: precision value: 3.095815510077129 - type: recall value: 4.3478260869565215 task: type: BitextMining - dataset: config: tha_Thai-pol_Latn name: MTEB FloresBitextMining (tha_Thai-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 10.17786561264822 - type: f1 value: 9.03960816616096 - type: main_score value: 9.03960816616096 - type: precision value: 8.720792794162358 - type: recall value: 10.17786561264822 task: type: BitextMining - dataset: config: zho_Hans-pol_Latn name: MTEB FloresBitextMining (zho_Hans-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 11.16600790513834 - type: f1 value: 9.816083127794206 - type: main_score value: 9.816083127794206 - type: precision value: 9.449149883932492 - type: recall value: 11.16600790513834 task: type: BitextMining - dataset: config: ast_Latn-pol_Latn name: MTEB FloresBitextMining (ast_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 30.434782608695656 - type: f1 value: 28.20427596119296 - type: main_score value: 28.20427596119296 - type: precision value: 27.513652569681575 - type: recall value: 30.434782608695656 task: type: BitextMining - dataset: config: crh_Latn-pol_Latn name: MTEB FloresBitextMining (crh_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 20.75098814229249 - type: f1 value: 17.77221970878066 - type: main_score value: 17.77221970878066 - type: precision value: 17.05363389516053 - type: recall value: 20.75098814229249 task: type: BitextMining - dataset: config: glg_Latn-pol_Latn name: MTEB FloresBitextMining (glg_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 35.37549407114625 - type: f1 value: 32.855499945461084 - type: main_score value: 32.855499945461084 - type: precision value: 32.22612135335778 - type: recall value: 35.37549407114625 task: type: BitextMining - dataset: config: kas_Deva-pol_Latn name: MTEB FloresBitextMining (kas_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 16.40316205533597 - type: f1 value: 14.19769011598097 - type: main_score value: 14.19769011598097 - type: precision value: 13.509038094724147 - type: recall value: 16.40316205533597 task: type: BitextMining - dataset: config: ltz_Latn-pol_Latn name: MTEB FloresBitextMining (ltz_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 37.84584980237154 - type: f1 value: 33.44048344372484 - type: main_score value: 33.44048344372484 - type: precision value: 32.05143506407127 - type: recall value: 37.84584980237154 task: type: BitextMining - dataset: config: nus_Latn-pol_Latn name: MTEB FloresBitextMining (nus_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 12.845849802371543 - type: f1 value: 10.896227857585187 - type: main_score value: 10.896227857585187 - type: precision value: 10.294754358350081 - type: recall value: 12.845849802371543 task: type: BitextMining - dataset: config: slk_Latn-pol_Latn name: MTEB FloresBitextMining (slk_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 41.00790513833992 - type: f1 value: 36.01972871344834 - type: main_score value: 36.01972871344834 - type: precision value: 34.530391529091034 - type: recall value: 41.00790513833992 task: type: BitextMining - dataset: config: tir_Ethi-pol_Latn name: MTEB FloresBitextMining (tir_Ethi-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.66798418972332 - type: f1 value: 2.2198721083844957 - type: main_score value: 2.2198721083844957 - type: precision value: 2.1499716421629076 - type: recall value: 2.66798418972332 task: type: BitextMining - dataset: config: zho_Hant-pol_Latn name: MTEB FloresBitextMining (zho_Hant-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.312252964426877 - type: f1 value: 6.340388250223482 - type: main_score value: 6.340388250223482 - type: precision value: 6.111030998564803 - type: recall value: 7.312252964426877 task: type: BitextMining - dataset: config: awa_Deva-pol_Latn name: MTEB FloresBitextMining (awa_Deva-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.3596837944664033 - type: f1 value: 2.579158411169609 - type: main_score value: 2.579158411169609 - type: precision value: 2.403919362726758 - type: recall value: 3.3596837944664033 task: type: BitextMining - dataset: config: cym_Latn-pol_Latn name: MTEB FloresBitextMining (cym_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 20.35573122529644 - type: f1 value: 18.287058620527255 - type: main_score value: 18.287058620527255 - type: precision value: 17.70382884312341 - type: recall value: 20.35573122529644 task: type: BitextMining - dataset: config: grn_Latn-pol_Latn name: MTEB FloresBitextMining (grn_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 28.55731225296443 - type: f1 value: 24.9187073406344 - type: main_score value: 24.9187073406344 - type: precision value: 23.940053904041108 - type: recall value: 28.55731225296443 task: type: BitextMining - dataset: config: kat_Geor-pol_Latn name: MTEB FloresBitextMining (kat_Geor-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 6.027667984189724 - type: f1 value: 4.838828225825765 - type: main_score value: 4.838828225825765 - type: precision value: 4.530376375310547 - type: recall value: 6.027667984189724 task: type: BitextMining - dataset: config: lua_Latn-pol_Latn name: MTEB FloresBitextMining (lua_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.936758893280633 - type: f1 value: 19.137345487494034 - type: main_score value: 19.137345487494034 - type: precision value: 18.50076951838475 - type: recall value: 21.936758893280633 task: type: BitextMining - dataset: config: nya_Latn-pol_Latn name: MTEB FloresBitextMining (nya_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.727272727272727 - type: f1 value: 19.352203630458405 - type: main_score value: 19.352203630458405 - type: precision value: 18.403212982984027 - type: recall value: 22.727272727272727 task: type: BitextMining - dataset: config: slv_Latn-pol_Latn name: MTEB FloresBitextMining (slv_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 34.58498023715415 - type: f1 value: 29.987852862596537 - type: main_score value: 29.987852862596537 - type: precision value: 28.67320439878897 - type: recall value: 34.58498023715415 task: type: BitextMining - dataset: config: tpi_Latn-pol_Latn name: MTEB FloresBitextMining (tpi_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 49.50592885375494 - type: f1 value: 45.07371755426884 - type: main_score value: 45.07371755426884 - type: precision value: 43.583045144983984 - type: recall value: 49.50592885375494 task: type: BitextMining - dataset: config: zsm_Latn-pol_Latn name: MTEB FloresBitextMining (zsm_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 24.40711462450593 - type: f1 value: 21.96257404337446 - type: main_score value: 21.96257404337446 - type: precision value: 21.242868844120004 - type: recall value: 24.40711462450593 task: type: BitextMining - dataset: config: ayr_Latn-pol_Latn name: MTEB FloresBitextMining (ayr_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 19.76284584980237 - type: f1 value: 16.61210544522444 - type: main_score value: 16.61210544522444 - type: precision value: 15.90306566825407 - type: recall value: 19.76284584980237 task: type: BitextMining - dataset: config: dan_Latn-pol_Latn name: MTEB FloresBitextMining (dan_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 41.798418972332016 - type: f1 value: 37.28920180969047 - type: main_score value: 37.28920180969047 - type: precision value: 35.93657133520941 - type: recall value: 41.798418972332016 task: type: BitextMining - dataset: config: guj_Gujr-pol_Latn name: MTEB FloresBitextMining (guj_Gujr-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.260869565217391 - type: f1 value: 2.5249735724995688 - type: main_score value: 2.5249735724995688 - type: precision value: 2.388309383138791 - type: recall value: 3.260869565217391 task: type: BitextMining - dataset: config: kaz_Cyrl-pol_Latn name: MTEB FloresBitextMining (kaz_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 7.41106719367589 - type: f1 value: 6.058280483075661 - type: main_score value: 6.058280483075661 - type: precision value: 5.757579952561782 - type: recall value: 7.41106719367589 task: type: BitextMining - dataset: config: lug_Latn-pol_Latn name: MTEB FloresBitextMining (lug_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.885375494071145 - type: f1 value: 15.521929947232882 - type: main_score value: 15.521929947232882 - type: precision value: 14.947095140298922 - type: recall value: 17.885375494071145 task: type: BitextMining - dataset: config: oci_Latn-pol_Latn name: MTEB FloresBitextMining (oci_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 43.47826086956522 - type: f1 value: 40.4731863475219 - type: main_score value: 40.4731863475219 - type: precision value: 39.59450765088603 - type: recall value: 43.47826086956522 task: type: BitextMining - dataset: config: smo_Latn-pol_Latn name: MTEB FloresBitextMining (smo_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.588932806324113 - type: f1 value: 15.690160809698227 - type: main_score value: 15.690160809698227 - type: precision value: 15.156564766279049 - type: recall value: 17.588932806324113 task: type: BitextMining - dataset: config: tsn_Latn-pol_Latn name: MTEB FloresBitextMining (tsn_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.34387351778656 - type: f1 value: 18.434350717459363 - type: main_score value: 18.434350717459363 - type: precision value: 17.774574235822616 - type: recall value: 21.34387351778656 task: type: BitextMining - dataset: config: zul_Latn-pol_Latn name: MTEB FloresBitextMining (zul_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 11.6600790513834 - type: f1 value: 9.735829734910533 - type: main_score value: 9.735829734910533 - type: precision value: 9.312566681548638 - type: recall value: 11.6600790513834 task: type: BitextMining - dataset: config: azb_Arab-pol_Latn name: MTEB FloresBitextMining (azb_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 2.1739130434782608 - type: f1 value: 1.5873015873015872 - type: main_score value: 1.5873015873015872 - type: precision value: 1.4545689817428948 - type: recall value: 2.1739130434782608 task: type: BitextMining - dataset: config: deu_Latn-pol_Latn name: MTEB FloresBitextMining (deu_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 44.762845849802375 - type: f1 value: 41.12292987587522 - type: main_score value: 41.12292987587522 - type: precision value: 39.92950680327162 - type: recall value: 44.762845849802375 task: type: BitextMining - dataset: config: hat_Latn-pol_Latn name: MTEB FloresBitextMining (hat_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.33201581027668 - type: f1 value: 18.967394150492094 - type: main_score value: 18.967394150492094 - type: precision value: 18.182537641253294 - type: recall value: 22.33201581027668 task: type: BitextMining - dataset: config: kbp_Latn-pol_Latn name: MTEB FloresBitextMining (kbp_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.245059288537547 - type: f1 value: 18.09991436613242 - type: main_score value: 18.09991436613242 - type: precision value: 17.252105388177373 - type: recall value: 21.245059288537547 task: type: BitextMining - dataset: config: luo_Latn-pol_Latn name: MTEB FloresBitextMining (luo_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 22.233201581027668 - type: f1 value: 18.449176192713168 - type: main_score value: 18.449176192713168 - type: precision value: 17.624851982025028 - type: recall value: 22.233201581027668 task: type: BitextMining - dataset: config: ory_Orya-pol_Latn name: MTEB FloresBitextMining (ory_Orya-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.3478260869565215 - type: f1 value: 3.611531226271985 - type: main_score value: 3.611531226271985 - type: precision value: 3.468918020129858 - type: recall value: 4.3478260869565215 task: type: BitextMining - dataset: config: sna_Latn-pol_Latn name: MTEB FloresBitextMining (sna_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 15.217391304347828 - type: f1 value: 12.702456926626832 - type: main_score value: 12.702456926626832 - type: precision value: 12.135158314143617 - type: recall value: 15.217391304347828 task: type: BitextMining - dataset: config: tso_Latn-pol_Latn name: MTEB FloresBitextMining (tso_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 18.181818181818183 - type: f1 value: 15.288851001752626 - type: main_score value: 15.288851001752626 - type: precision value: 14.650945991352978 - type: recall value: 18.181818181818183 task: type: BitextMining - dataset: config: azj_Latn-pol_Latn name: MTEB FloresBitextMining (azj_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 18.57707509881423 - type: f1 value: 14.966220609361827 - type: main_score value: 14.966220609361827 - type: precision value: 13.961871486067762 - type: recall value: 18.57707509881423 task: type: BitextMining - dataset: config: dik_Latn-pol_Latn name: MTEB FloresBitextMining (dik_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 32.70750988142292 - type: f1 value: 28.22133496338573 - type: main_score value: 28.22133496338573 - type: precision value: 26.929658124002465 - type: recall value: 32.70750988142292 task: type: BitextMining - dataset: config: hau_Latn-pol_Latn name: MTEB FloresBitextMining (hau_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.490118577075098 - type: f1 value: 15.013241657524787 - type: main_score value: 15.013241657524787 - type: precision value: 14.360189776791414 - type: recall value: 17.490118577075098 task: type: BitextMining - dataset: config: kea_Latn-pol_Latn name: MTEB FloresBitextMining (kea_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 29.249011857707508 - type: f1 value: 25.728171141197727 - type: main_score value: 25.728171141197727 - type: precision value: 24.845714735234882 - type: recall value: 29.249011857707508 task: type: BitextMining - dataset: config: lus_Latn-pol_Latn name: MTEB FloresBitextMining (lus_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 37.45059288537549 - type: f1 value: 34.1522273339155 - type: main_score value: 34.1522273339155 - type: precision value: 33.11133967978197 - type: recall value: 37.45059288537549 task: type: BitextMining - dataset: config: pag_Latn-pol_Latn name: MTEB FloresBitextMining (pag_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 46.44268774703557 - type: f1 value: 42.55956424045618 - type: main_score value: 42.55956424045618 - type: precision value: 41.308049408906065 - type: recall value: 46.44268774703557 task: type: BitextMining - dataset: config: snd_Arab-pol_Latn name: MTEB FloresBitextMining (snd_Arab-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.7312252964426875 - type: f1 value: 4.418129345402073 - type: main_score value: 4.418129345402073 - type: precision value: 4.080735393694834 - type: recall value: 5.7312252964426875 task: type: BitextMining - dataset: config: tuk_Latn-pol_Latn name: MTEB FloresBitextMining (tuk_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 18.478260869565215 - type: f1 value: 14.559489584262082 - type: main_score value: 14.559489584262082 - type: precision value: 13.576842652514323 - type: recall value: 18.478260869565215 task: type: BitextMining - dataset: config: bak_Cyrl-pol_Latn name: MTEB FloresBitextMining (bak_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.7312252964426875 - type: f1 value: 4.523459883439954 - type: main_score value: 4.523459883439954 - type: precision value: 4.2460184752788495 - type: recall value: 5.7312252964426875 task: type: BitextMining - dataset: config: dyu_Latn-pol_Latn name: MTEB FloresBitextMining (dyu_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 17.885375494071145 - type: f1 value: 15.287973467865342 - type: main_score value: 15.287973467865342 - type: precision value: 14.602748983645133 - type: recall value: 17.885375494071145 task: type: BitextMining - dataset: config: heb_Hebr-pol_Latn name: MTEB FloresBitextMining (heb_Hebr-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 5.434782608695652 - type: f1 value: 4.41417092476672 - type: main_score value: 4.41417092476672 - type: precision value: 4.190847187073325 - type: recall value: 5.434782608695652 task: type: BitextMining - dataset: config: khk_Cyrl-pol_Latn name: MTEB FloresBitextMining (khk_Cyrl-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 3.8537549407114624 - type: f1 value: 2.7896363021461554 - type: main_score value: 2.7896363021461554 - type: precision value: 2.554176462902154 - type: recall value: 3.8537549407114624 task: type: BitextMining - dataset: config: lvs_Latn-pol_Latn name: MTEB FloresBitextMining (lvs_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 19.76284584980237 - type: f1 value: 15.95516453180757 - type: main_score value: 15.95516453180757 - type: precision value: 14.79353339363382 - type: recall value: 19.76284584980237 task: type: BitextMining - dataset: config: pan_Guru-pol_Latn name: MTEB FloresBitextMining (pan_Guru-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 4.150197628458498 - type: f1 value: 3.19295355133975 - type: main_score value: 3.19295355133975 - type: precision value: 2.959982158565134 - type: recall value: 4.150197628458498 task: type: BitextMining - dataset: config: som_Latn-pol_Latn name: MTEB FloresBitextMining (som_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 15.217391304347828 - type: f1 value: 13.211426918534308 - type: main_score value: 13.211426918534308 - type: precision value: 12.706032748562594 - type: recall value: 15.217391304347828 task: type: BitextMining - dataset: config: tum_Latn-pol_Latn name: MTEB FloresBitextMining (tum_Latn-pol_Latn) revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e split: devtest type: mteb/flores metrics: - type: accuracy value: 21.73913043478261 - type: f1 value: 18.12344158190285 - type: main_score value: 18.12344158190285 - type: precision value: 17.177114905727347 - type: recall value: 21.73913043478261 task: type: BitextMining - dataset: config: default name: MTEB HotpotQA-PL (default) revision: a0bd479ac97b4ccb5bd6ce320c415d0bb4beb907 split: test type: clarin-knext/hotpotqa-pl metrics: - type: main_score value: 57.45 - type: map_at_1 value: 35.375 - type: map_at_10 value: 48.074 - type: map_at_100 value: 48.919000000000004 - type: map_at_1000 value: 48.998000000000005 - type: map_at_20 value: 48.556 - type: map_at_3 value: 45.518 - type: map_at_5 value: 47.032000000000004 - type: mrr_at_1 value: 70.74949358541525 - type: mrr_at_10 value: 76.78829190915619 - type: mrr_at_100 value: 77.08402375999609 - type: mrr_at_1000 value: 77.09935954798011 - type: mrr_at_20 value: 76.96994129997728 - type: mrr_at_3 value: 75.67184334908845 - type: mrr_at_5 value: 76.36191762322755 - type: nauc_map_at_1000_diff1 value: 27.947206485539006 - type: nauc_map_at_1000_max value: 26.1771329073023 - type: nauc_map_at_1000_std value: 5.2314854933790285 - type: nauc_map_at_100_diff1 value: 27.941434167482498 - type: nauc_map_at_100_max value: 26.16836231495222 - type: nauc_map_at_100_std value: 5.19915879926482 - type: nauc_map_at_10_diff1 value: 28.30412093215599 - type: nauc_map_at_10_max value: 26.507043849371705 - type: nauc_map_at_10_std value: 4.9731091966343435 - type: nauc_map_at_1_diff1 value: 72.97828307949827 - type: nauc_map_at_1_max value: 52.081763648188485 - type: nauc_map_at_1_std value: 4.3782414938724274 - type: nauc_map_at_20_diff1 value: 28.02867231346264 - type: nauc_map_at_20_max value: 26.269293729821857 - type: nauc_map_at_20_std value: 5.034947943611055 - type: nauc_map_at_3_diff1 value: 30.823298736315678 - type: nauc_map_at_3_max value: 28.305551752514486 - type: nauc_map_at_3_std value: 4.10815806251092 - type: nauc_map_at_5_diff1 value: 29.05835258195969 - type: nauc_map_at_5_max value: 27.040450341544826 - type: nauc_map_at_5_std value: 4.436108837487476 - type: nauc_mrr_at_1000_diff1 value: 70.79284622625714 - type: nauc_mrr_at_1000_max value: 53.545770296186 - type: nauc_mrr_at_1000_std value: 6.713898700895104 - type: nauc_mrr_at_100_diff1 value: 70.78789266461835 - type: nauc_mrr_at_100_max value: 53.548974646684 - type: nauc_mrr_at_100_std value: 6.718944513343734 - type: nauc_mrr_at_10_diff1 value: 70.73072977161314 - type: nauc_mrr_at_10_max value: 53.596940611126044 - type: nauc_mrr_at_10_std value: 6.738009246182779 - type: nauc_mrr_at_1_diff1 value: 72.97828307949827 - type: nauc_mrr_at_1_max value: 52.081763648188485 - type: nauc_mrr_at_1_std value: 4.3782414938724274 - type: nauc_mrr_at_20_diff1 value: 70.75348482278754 - type: nauc_mrr_at_20_max value: 53.5477479731434 - type: nauc_mrr_at_20_std value: 6.725694432271149 - type: nauc_mrr_at_3_diff1 value: 70.90921193129459 - type: nauc_mrr_at_3_max value: 53.56666547708075 - type: nauc_mrr_at_3_std value: 6.015691765361308 - type: nauc_mrr_at_5_diff1 value: 70.77429682323806 - type: nauc_mrr_at_5_max value: 53.58598108314783 - type: nauc_mrr_at_5_std value: 6.3197118267571835 - type: nauc_ndcg_at_1000_diff1 value: 31.832520163159945 - type: nauc_ndcg_at_1000_max value: 28.87137244600895 - type: nauc_ndcg_at_1000_std value: 8.080240653998418 - type: nauc_ndcg_at_100_diff1 value: 31.669664556952565 - type: nauc_ndcg_at_100_max value: 28.71594989425018 - type: nauc_ndcg_at_100_std value: 7.4921101438523205 - type: nauc_ndcg_at_10_diff1 value: 33.226000860634876 - type: nauc_ndcg_at_10_max value: 30.107656505365547 - type: nauc_ndcg_at_10_std value: 6.439918456928247 - type: nauc_ndcg_at_1_diff1 value: 72.97828307949827 - type: nauc_ndcg_at_1_max value: 52.081763648188485 - type: nauc_ndcg_at_1_std value: 4.3782414938724274 - type: nauc_ndcg_at_20_diff1 value: 32.302839386753554 - type: nauc_ndcg_at_20_max value: 29.29907663228911 - type: nauc_ndcg_at_20_std value: 6.580587478607336 - type: nauc_ndcg_at_3_diff1 value: 37.7188612666666 - type: nauc_ndcg_at_3_max value: 33.15240774572049 - type: nauc_ndcg_at_3_std value: 4.819925669362351 - type: nauc_ndcg_at_5_diff1 value: 34.87704367410069 - type: nauc_ndcg_at_5_max value: 31.209233398164365 - type: nauc_ndcg_at_5_std value: 5.320757770134728 - type: nauc_precision_at_1000_diff1 value: -8.121260561311551 - type: nauc_precision_at_1000_max value: 1.4600692792429486 - type: nauc_precision_at_1000_std value: 18.980551255975282 - type: nauc_precision_at_100_diff1 value: 0.9595396043800729 - type: nauc_precision_at_100_max value: 7.487652446748426 - type: nauc_precision_at_100_std value: 12.550190196346483 - type: nauc_precision_at_10_diff1 value: 11.523234033091521 - type: nauc_precision_at_10_max value: 16.240380877381945 - type: nauc_precision_at_10_std value: 8.103831621990937 - type: nauc_precision_at_1_diff1 value: 72.97828307949827 - type: nauc_precision_at_1_max value: 52.081763648188485 - type: nauc_precision_at_1_std value: 4.3782414938724274 - type: nauc_precision_at_20_diff1 value: 7.776696643703554 - type: nauc_precision_at_20_max value: 13.003394172221489 - type: nauc_precision_at_20_std value: 8.456147688962815 - type: nauc_precision_at_3_diff1 value: 22.92875552320387 - type: nauc_precision_at_3_max value: 24.690141011441337 - type: nauc_precision_at_3_std value: 4.897559810894209 - type: nauc_precision_at_5_diff1 value: 17.073370028457965 - type: nauc_precision_at_5_max value: 20.395437811786447 - type: nauc_precision_at_5_std value: 5.746343973587844 - type: nauc_recall_at_1000_diff1 value: -8.1212605613113 - type: nauc_recall_at_1000_max value: 1.4600692792431236 - type: nauc_recall_at_1000_std value: 18.98055125597542 - type: nauc_recall_at_100_diff1 value: 0.9595396043800083 - type: nauc_recall_at_100_max value: 7.487652446748395 - type: nauc_recall_at_100_std value: 12.550190196346373 - type: nauc_recall_at_10_diff1 value: 11.523234033091546 - type: nauc_recall_at_10_max value: 16.240380877381995 - type: nauc_recall_at_10_std value: 8.103831621991013 - type: nauc_recall_at_1_diff1 value: 72.97828307949827 - type: nauc_recall_at_1_max value: 52.081763648188485 - type: nauc_recall_at_1_std value: 4.3782414938724274 - type: nauc_recall_at_20_diff1 value: 7.776696643703573 - type: nauc_recall_at_20_max value: 13.0033941722215 - type: nauc_recall_at_20_std value: 8.45614768896279 - type: nauc_recall_at_3_diff1 value: 22.928755523203822 - type: nauc_recall_at_3_max value: 24.690141011441263 - type: nauc_recall_at_3_std value: 4.897559810894166 - type: nauc_recall_at_5_diff1 value: 17.073370028458005 - type: nauc_recall_at_5_max value: 20.395437811786472 - type: nauc_recall_at_5_std value: 5.746343973587907 - type: ndcg_at_1 value: 70.749 - type: ndcg_at_10 value: 57.45 - type: ndcg_at_100 value: 60.650999999999996 - type: ndcg_at_1000 value: 62.291 - type: ndcg_at_20 value: 58.727 - type: ndcg_at_3 value: 53.539 - type: ndcg_at_5 value: 55.592 - type: precision_at_1 value: 70.749 - type: precision_at_10 value: 11.622 - type: precision_at_100 value: 1.415 - type: precision_at_1000 value: 0.163 - type: precision_at_20 value: 6.2219999999999995 - type: precision_at_3 value: 32.915 - type: precision_at_5 value: 21.375 - type: recall_at_1 value: 35.375 - type: recall_at_10 value: 58.109 - type: recall_at_100 value: 70.77 - type: recall_at_1000 value: 81.708 - type: recall_at_20 value: 62.221000000000004 - type: recall_at_3 value: 49.372 - type: recall_at_5 value: 53.437 task: type: Retrieval - dataset: config: default name: MTEB HotpotQA-PLHardNegatives (default) revision: 0642cadffa3205c6b21c9af901fdffcd60d6f31e split: test type: mteb/HotpotQA_PL_test_top_250_only_w_correct-v2 metrics: - type: main_score value: 60.56700000000001 - type: map_at_1 value: 36.199999999999996 - type: map_at_10 value: 50.897999999999996 - type: map_at_100 value: 52.038 - type: map_at_1000 value: 52.134 - type: map_at_20 value: 51.55800000000001 - type: map_at_3 value: 47.766999999999996 - type: map_at_5 value: 49.434 - type: mrr_at_1 value: 72.39999999999999 - type: mrr_at_10 value: 78.74666666666667 - type: mrr_at_100 value: 79.06754299755188 - type: mrr_at_1000 value: 79.07929688256422 - type: mrr_at_20 value: 78.97176876614957 - type: mrr_at_3 value: 77.5 - type: mrr_at_5 value: 78.20499999999998 - type: nauc_map_at_1000_diff1 value: 21.13561300802851 - type: nauc_map_at_1000_max value: 21.249219453246322 - type: nauc_map_at_1000_std value: -4.669822840258497 - type: nauc_map_at_100_diff1 value: 21.095306085908604 - type: nauc_map_at_100_max value: 21.202772455601586 - type: nauc_map_at_100_std value: -4.723750393340032 - type: nauc_map_at_10_diff1 value: 21.765115098977773 - type: nauc_map_at_10_max value: 21.77000691170242 - type: nauc_map_at_10_std value: -5.4382501708747 - type: nauc_map_at_1_diff1 value: 68.79425914914317 - type: nauc_map_at_1_max value: 47.223208976547 - type: nauc_map_at_1_std value: -12.63967936490773 - type: nauc_map_at_20_diff1 value: 21.23060640067979 - type: nauc_map_at_20_max value: 21.263514907637227 - type: nauc_map_at_20_std value: -5.156492062150907 - type: nauc_map_at_3_diff1 value: 23.863613350758534 - type: nauc_map_at_3_max value: 22.361538264217785 - type: nauc_map_at_3_std value: -6.12150529504839 - type: nauc_map_at_5_diff1 value: 22.542999500961926 - type: nauc_map_at_5_max value: 22.759646449359817 - type: nauc_map_at_5_std value: -4.761823706494758 - type: nauc_mrr_at_1000_diff1 value: 68.09475501720986 - type: nauc_mrr_at_1000_max value: 49.08784703387968 - type: nauc_mrr_at_1000_std value: -11.912122956070558 - type: nauc_mrr_at_100_diff1 value: 68.08653096396323 - type: nauc_mrr_at_100_max value: 49.082623679323305 - type: nauc_mrr_at_100_std value: -11.922377942768259 - type: nauc_mrr_at_10_diff1 value: 68.01720190274823 - type: nauc_mrr_at_10_max value: 49.181515292974105 - type: nauc_mrr_at_10_std value: -11.64889917671523 - type: nauc_mrr_at_1_diff1 value: 68.79425914914317 - type: nauc_mrr_at_1_max value: 47.223208976547 - type: nauc_mrr_at_1_std value: -12.63967936490773 - type: nauc_mrr_at_20_diff1 value: 68.11391215333266 - type: nauc_mrr_at_20_max value: 49.22271597462208 - type: nauc_mrr_at_20_std value: -11.806540894851986 - type: nauc_mrr_at_3_diff1 value: 68.39904516682762 - type: nauc_mrr_at_3_max value: 48.66756325961343 - type: nauc_mrr_at_3_std value: -12.807791929130827 - type: nauc_mrr_at_5_diff1 value: 68.22020582688101 - type: nauc_mrr_at_5_max value: 49.32584715351403 - type: nauc_mrr_at_5_std value: -11.383099021799612 - type: nauc_ndcg_at_1000_diff1 value: 25.403527087657316 - type: nauc_ndcg_at_1000_max value: 24.333374836810293 - type: nauc_ndcg_at_1000_std value: -2.301241563130495 - type: nauc_ndcg_at_100_diff1 value: 24.23532138995639 - type: nauc_ndcg_at_100_max value: 23.39054626234519 - type: nauc_ndcg_at_100_std value: -3.01304262560594 - type: nauc_ndcg_at_10_diff1 value: 26.762683685078137 - type: nauc_ndcg_at_10_max value: 25.25683654056493 - type: nauc_ndcg_at_10_std value: -5.778271315402495 - type: nauc_ndcg_at_1_diff1 value: 68.79425914914317 - type: nauc_ndcg_at_1_max value: 47.223208976547 - type: nauc_ndcg_at_1_std value: -12.63967936490773 - type: nauc_ndcg_at_20_diff1 value: 25.2054038951938 - type: nauc_ndcg_at_20_max value: 23.85054805466091 - type: nauc_ndcg_at_20_std value: -5.167987087532739 - type: nauc_ndcg_at_3_diff1 value: 31.188392391732428 - type: nauc_ndcg_at_3_max value: 27.05796419887928 - type: nauc_ndcg_at_3_std value: -7.175158473634911 - type: nauc_ndcg_at_5_diff1 value: 28.875147538360512 - type: nauc_ndcg_at_5_max value: 27.36413608753433 - type: nauc_ndcg_at_5_std value: -4.789438250105259 - type: nauc_precision_at_1000_diff1 value: -32.63749453783157 - type: nauc_precision_at_1000_max value: -8.735044720641296 - type: nauc_precision_at_1000_std value: 36.93974744038624 - type: nauc_precision_at_100_diff1 value: -15.12713425869049 - type: nauc_precision_at_100_max value: -1.124498888901322 - type: nauc_precision_at_100_std value: 13.404652097687492 - type: nauc_precision_at_10_diff1 value: 5.003599070842457 - type: nauc_precision_at_10_max value: 12.408359902893231 - type: nauc_precision_at_10_std value: -1.9903333253005324 - type: nauc_precision_at_1_diff1 value: 68.79425914914317 - type: nauc_precision_at_1_max value: 47.223208976547 - type: nauc_precision_at_1_std value: -12.63967936490773 - type: nauc_precision_at_20_diff1 value: -2.5188824194472934 - type: nauc_precision_at_20_max value: 5.93923729966351 - type: nauc_precision_at_20_std value: 0.47431834856040817 - type: nauc_precision_at_3_diff1 value: 16.917738934458505 - type: nauc_precision_at_3_max value: 19.198721140491106 - type: nauc_precision_at_3_std value: -4.843523153122463 - type: nauc_precision_at_5_diff1 value: 11.667368686863435 - type: nauc_precision_at_5_max value: 18.856227988562072 - type: nauc_precision_at_5_std value: -0.13702330327582785 - type: nauc_recall_at_1000_diff1 value: -32.63749453783146 - type: nauc_recall_at_1000_max value: -8.735044720641307 - type: nauc_recall_at_1000_std value: 36.9397474403863 - type: nauc_recall_at_100_diff1 value: -15.127134258690614 - type: nauc_recall_at_100_max value: -1.124498888901344 - type: nauc_recall_at_100_std value: 13.404652097687325 - type: nauc_recall_at_10_diff1 value: 5.003599070842488 - type: nauc_recall_at_10_max value: 12.408359902893208 - type: nauc_recall_at_10_std value: -1.9903333253004714 - type: nauc_recall_at_1_diff1 value: 68.79425914914317 - type: nauc_recall_at_1_max value: 47.223208976547 - type: nauc_recall_at_1_std value: -12.63967936490773 - type: nauc_recall_at_20_diff1 value: -2.5188824194472703 - type: nauc_recall_at_20_max value: 5.939237299663526 - type: nauc_recall_at_20_std value: 0.47431834856051747 - type: nauc_recall_at_3_diff1 value: 16.917738934458495 - type: nauc_recall_at_3_max value: 19.198721140491156 - type: nauc_recall_at_3_std value: -4.8435231531224465 - type: nauc_recall_at_5_diff1 value: 11.667368686863474 - type: nauc_recall_at_5_max value: 18.856227988562125 - type: nauc_recall_at_5_std value: -0.13702330327578427 - type: ndcg_at_1 value: 72.39999999999999 - type: ndcg_at_10 value: 60.56700000000001 - type: ndcg_at_100 value: 64.703 - type: ndcg_at_1000 value: 66.523 - type: ndcg_at_20 value: 62.278999999999996 - type: ndcg_at_3 value: 55.762 - type: ndcg_at_5 value: 58.001999999999995 - type: precision_at_1 value: 72.39999999999999 - type: precision_at_10 value: 12.55 - type: precision_at_100 value: 1.5810000000000002 - type: precision_at_1000 value: 0.182 - type: precision_at_20 value: 6.825 - type: precision_at_3 value: 34.567 - type: precision_at_5 value: 22.52 - type: recall_at_1 value: 36.199999999999996 - type: recall_at_10 value: 62.74999999999999 - type: recall_at_100 value: 79.05 - type: recall_at_1000 value: 91.05 - type: recall_at_20 value: 68.25 - type: recall_at_3 value: 51.849999999999994 - type: recall_at_5 value: 56.3 task: type: Retrieval - dataset: config: default name: MTEB LanguageClassification (default) revision: aa56583bf2bc52b0565770607d6fc3faebecf9e2 split: test type: papluca/language-identification metrics: - type: accuracy value: 59.3115234375 - type: f1 value: 55.45455992019244 - type: f1_weighted value: 55.5143868826049 - type: main_score value: 59.3115234375 task: type: Classification - dataset: config: default name: MTEB MSMARCO-PL (default) revision: 8634c07806d5cce3a6138e260e59b81760a0a640 split: test type: clarin-knext/msmarco-pl metrics: - type: main_score value: 57.105 - type: map_at_1 value: 1.9900000000000002 - type: map_at_10 value: 10.452 - type: map_at_100 value: 27.809 - type: map_at_1000 value: 34.333999999999996 - type: map_at_20 value: 15.934999999999999 - type: map_at_3 value: 4.139 - type: map_at_5 value: 6.593 - type: mrr_at_1 value: 81.3953488372093 - type: mrr_at_10 value: 88.00664451827242 - type: mrr_at_100 value: 88.00664451827242 - type: mrr_at_1000 value: 88.00664451827242 - type: mrr_at_20 value: 88.00664451827242 - type: mrr_at_3 value: 86.04651162790698 - type: mrr_at_5 value: 87.67441860465117 - type: nauc_map_at_1000_diff1 value: -8.012101097997089 - type: nauc_map_at_1000_max value: 56.4366412455371 - type: nauc_map_at_1000_std value: 42.9964737470612 - type: nauc_map_at_100_diff1 value: 6.041944857071697 - type: nauc_map_at_100_max value: 45.956568555496126 - type: nauc_map_at_100_std value: 26.25070487004926 - type: nauc_map_at_10_diff1 value: 18.286490244171176 - type: nauc_map_at_10_max value: 28.211402521021554 - type: nauc_map_at_10_std value: -1.552502361118044 - type: nauc_map_at_1_diff1 value: 20.512143297892948 - type: nauc_map_at_1_max value: -15.919747659099992 - type: nauc_map_at_1_std value: -27.130727104051793 - type: nauc_map_at_20_diff1 value: 18.236288908512847 - type: nauc_map_at_20_max value: 34.1167080054512 - type: nauc_map_at_20_std value: 1.14669873714027 - type: nauc_map_at_3_diff1 value: 14.365487356948107 - type: nauc_map_at_3_max value: 6.975356551295571 - type: nauc_map_at_3_std value: -9.646239922358001 - type: nauc_map_at_5_diff1 value: 13.364114223444492 - type: nauc_map_at_5_max value: 15.079226719804579 - type: nauc_map_at_5_std value: -8.19304844226965 - type: nauc_mrr_at_1000_diff1 value: 12.044984137524793 - type: nauc_mrr_at_1000_max value: 22.507880773810452 - type: nauc_mrr_at_1000_std value: 5.769039872012413 - type: nauc_mrr_at_100_diff1 value: 12.044984137524793 - type: nauc_mrr_at_100_max value: 22.507880773810452 - type: nauc_mrr_at_100_std value: 5.769039872012413 - type: nauc_mrr_at_10_diff1 value: 12.044984137524793 - type: nauc_mrr_at_10_max value: 22.507880773810452 - type: nauc_mrr_at_10_std value: 5.769039872012413 - type: nauc_mrr_at_1_diff1 value: 18.791336292581494 - type: nauc_mrr_at_1_max value: 17.34729375621222 - type: nauc_mrr_at_1_std value: 5.970890372277977 - type: nauc_mrr_at_20_diff1 value: 12.044984137524793 - type: nauc_mrr_at_20_max value: 22.507880773810452 - type: nauc_mrr_at_20_std value: 5.769039872012413 - type: nauc_mrr_at_3_diff1 value: 3.3178668209270006 - type: nauc_mrr_at_3_max value: 22.893544175421162 - type: nauc_mrr_at_3_std value: 9.55254750349961 - type: nauc_mrr_at_5_diff1 value: 14.170130034718365 - type: nauc_mrr_at_5_max value: 23.91480366047494 - type: nauc_mrr_at_5_std value: 2.3989686840953475 - type: nauc_ndcg_at_1000_diff1 value: -4.6649217802242395 - type: nauc_ndcg_at_1000_max value: 61.847729061932036 - type: nauc_ndcg_at_1000_std value: 37.82228582066131 - type: nauc_ndcg_at_100_diff1 value: 0.50290949273706 - type: nauc_ndcg_at_100_max value: 56.67022656538278 - type: nauc_ndcg_at_100_std value: 36.09127681926358 - type: nauc_ndcg_at_10_diff1 value: -9.802324122004562 - type: nauc_ndcg_at_10_max value: 43.331529382026304 - type: nauc_ndcg_at_10_std value: 17.31386700864639 - type: nauc_ndcg_at_1_diff1 value: 5.761397078780122 - type: nauc_ndcg_at_1_max value: 15.085833736334006 - type: nauc_ndcg_at_1_std value: -3.649463680185266 - type: nauc_ndcg_at_20_diff1 value: -9.960862900268634 - type: nauc_ndcg_at_20_max value: 45.071949623905894 - type: nauc_ndcg_at_20_std value: 19.66644760558708 - type: nauc_ndcg_at_3_diff1 value: -16.306996796248235 - type: nauc_ndcg_at_3_max value: 32.221092998275154 - type: nauc_ndcg_at_3_std value: 17.93948728916441 - type: nauc_ndcg_at_5_diff1 value: -10.728392149102975 - type: nauc_ndcg_at_5_max value: 35.86494251484633 - type: nauc_ndcg_at_5_std value: 11.684815101805704 - type: nauc_precision_at_1000_diff1 value: -35.821618516281355 - type: nauc_precision_at_1000_max value: 31.03090279111049 - type: nauc_precision_at_1000_std value: 46.79376309398088 - type: nauc_precision_at_100_diff1 value: -30.000379472915522 - type: nauc_precision_at_100_max value: 36.20094774818339 - type: nauc_precision_at_100_std value: 47.724950898741824 - type: nauc_precision_at_10_diff1 value: -14.670361579043648 - type: nauc_precision_at_10_max value: 50.88317572882223 - type: nauc_precision_at_10_std value: 34.00961099560648 - type: nauc_precision_at_1_diff1 value: 18.791336292581494 - type: nauc_precision_at_1_max value: 17.34729375621222 - type: nauc_precision_at_1_std value: 5.970890372277977 - type: nauc_precision_at_20_diff1 value: -21.901371452888753 - type: nauc_precision_at_20_max value: 46.04639118721932 - type: nauc_precision_at_20_std value: 35.611867234067546 - type: nauc_precision_at_3_diff1 value: -20.73209389401944 - type: nauc_precision_at_3_max value: 41.838425924111675 - type: nauc_precision_at_3_std value: 39.85194295498681 - type: nauc_precision_at_5_diff1 value: -17.19319873775553 - type: nauc_precision_at_5_max value: 51.56250664664223 - type: nauc_precision_at_5_std value: 32.26306628179021 - type: nauc_recall_at_1000_diff1 value: -5.135727487054241 - type: nauc_recall_at_1000_max value: 55.28965364606285 - type: nauc_recall_at_1000_std value: 39.29017243470838 - type: nauc_recall_at_100_diff1 value: 9.288604504308834 - type: nauc_recall_at_100_max value: 36.941533115974366 - type: nauc_recall_at_100_std value: 17.162321543551354 - type: nauc_recall_at_10_diff1 value: 24.5075068137349 - type: nauc_recall_at_10_max value: 24.704472613718117 - type: nauc_recall_at_10_std value: -10.491448477311643 - type: nauc_recall_at_1_diff1 value: 20.512143297892948 - type: nauc_recall_at_1_max value: -15.919747659099992 - type: nauc_recall_at_1_std value: -27.130727104051793 - type: nauc_recall_at_20_diff1 value: 20.470205622940263 - type: nauc_recall_at_20_max value: 25.03335185200762 - type: nauc_recall_at_20_std value: -9.542230615026384 - type: nauc_recall_at_3_diff1 value: 12.721138757081683 - type: nauc_recall_at_3_max value: 4.808299122775801 - type: nauc_recall_at_3_std value: -11.465076844396568 - type: nauc_recall_at_5_diff1 value: 16.88567314127846 - type: nauc_recall_at_5_max value: 10.614245198185062 - type: nauc_recall_at_5_std value: -16.18888205921033 - type: ndcg_at_1 value: 67.829 - type: ndcg_at_10 value: 57.105 - type: ndcg_at_100 value: 51.527 - type: ndcg_at_1000 value: 59.46399999999999 - type: ndcg_at_20 value: 54.138 - type: ndcg_at_3 value: 58.202 - type: ndcg_at_5 value: 59.363 - type: precision_at_1 value: 81.395 - type: precision_at_10 value: 66.047 - type: precision_at_100 value: 30.953000000000003 - type: precision_at_1000 value: 6.04 - type: precision_at_20 value: 55.93 - type: precision_at_3 value: 70.543 - type: precision_at_5 value: 72.09299999999999 - type: recall_at_1 value: 1.9900000000000002 - type: recall_at_10 value: 12.335 - type: recall_at_100 value: 42.348 - type: recall_at_1000 value: 68.891 - type: recall_at_20 value: 20.054 - type: recall_at_3 value: 4.375 - type: recall_at_5 value: 7.534000000000001 task: type: Retrieval - dataset: config: default name: MTEB MSMARCO-PLHardNegatives (default) revision: b609cb1ec6772bf92b8e014343a7ecfb10eef2d9 split: test type: mteb/MSMARCO_PL_test_top_250_only_w_correct-v2 metrics: - type: main_score value: 62.479 - type: map_at_1 value: 2.015 - type: map_at_10 value: 11.856 - type: map_at_100 value: 40.988 - type: map_at_1000 value: 57.959 - type: map_at_20 value: 19.111 - type: map_at_3 value: 4.817 - type: map_at_5 value: 7.3709999999999996 - type: mrr_at_1 value: 83.72093023255815 - type: mrr_at_10 value: 89.8671096345515 - type: mrr_at_100 value: 89.8671096345515 - type: mrr_at_1000 value: 89.8671096345515 - type: mrr_at_20 value: 89.8671096345515 - type: mrr_at_3 value: 88.37209302325581 - type: mrr_at_5 value: 89.53488372093024 - type: nauc_map_at_1000_diff1 value: -34.846135323586424 - type: nauc_map_at_1000_max value: 29.18292115867256 - type: nauc_map_at_1000_std value: 57.23667838425766 - type: nauc_map_at_100_diff1 value: -6.016983594119641 - type: nauc_map_at_100_max value: 27.97105320494105 - type: nauc_map_at_100_std value: 20.986545754487558 - type: nauc_map_at_10_diff1 value: 22.220254287378715 - type: nauc_map_at_10_max value: 11.403052444554513 - type: nauc_map_at_10_std value: -13.130226658464897 - type: nauc_map_at_1_diff1 value: 39.69550436091381 - type: nauc_map_at_1_max value: -16.882723045728202 - type: nauc_map_at_1_std value: -29.330395083370686 - type: nauc_map_at_20_diff1 value: 18.161494182946907 - type: nauc_map_at_20_max value: 16.529920908408926 - type: nauc_map_at_20_std value: -10.081086213313265 - type: nauc_map_at_3_diff1 value: 31.32204771377053 - type: nauc_map_at_3_max value: -5.482815149647405 - type: nauc_map_at_3_std value: -20.43957139807913 - type: nauc_map_at_5_diff1 value: 22.486941436188367 - type: nauc_map_at_5_max value: 4.612195845771766 - type: nauc_map_at_5_std value: -14.837761155598313 - type: nauc_mrr_at_1000_diff1 value: 2.2095971706487303 - type: nauc_mrr_at_1000_max value: 10.097437841508302 - type: nauc_mrr_at_1000_std value: 24.63211668703523 - type: nauc_mrr_at_100_diff1 value: 2.2095971706487303 - type: nauc_mrr_at_100_max value: 10.097437841508302 - type: nauc_mrr_at_100_std value: 24.63211668703523 - type: nauc_mrr_at_10_diff1 value: 2.2095971706487303 - type: nauc_mrr_at_10_max value: 10.097437841508302 - type: nauc_mrr_at_10_std value: 24.63211668703523 - type: nauc_mrr_at_1_diff1 value: 15.276398946528131 - type: nauc_mrr_at_1_max value: 8.717153299832296 - type: nauc_mrr_at_1_std value: 28.600034127271652 - type: nauc_mrr_at_20_diff1 value: 2.2095971706487303 - type: nauc_mrr_at_20_max value: 10.097437841508302 - type: nauc_mrr_at_20_std value: 24.63211668703523 - type: nauc_mrr_at_3_diff1 value: -10.560729627695347 - type: nauc_mrr_at_3_max value: 9.031758340498751 - type: nauc_mrr_at_3_std value: 22.47764642580193 - type: nauc_mrr_at_5_diff1 value: 5.023952302939077 - type: nauc_mrr_at_5_max value: 11.596859447930543 - type: nauc_mrr_at_5_std value: 24.20435962088466 - type: nauc_ndcg_at_1000_diff1 value: -38.521295110914316 - type: nauc_ndcg_at_1000_max value: 33.742873661087955 - type: nauc_ndcg_at_1000_std value: 54.07862663678682 - type: nauc_ndcg_at_100_diff1 value: -13.191806198203498 - type: nauc_ndcg_at_100_max value: 42.898538839792785 - type: nauc_ndcg_at_100_std value: 52.431848941001846 - type: nauc_ndcg_at_10_diff1 value: -14.094228653427074 - type: nauc_ndcg_at_10_max value: 22.013625698488877 - type: nauc_ndcg_at_10_std value: 32.269373872153004 - type: nauc_ndcg_at_1_diff1 value: -0.9536993894348202 - type: nauc_ndcg_at_1_max value: 13.241805975751314 - type: nauc_ndcg_at_1_std value: 20.628301389219217 - type: nauc_ndcg_at_20_diff1 value: -14.259970601088892 - type: nauc_ndcg_at_20_max value: 22.406568590509757 - type: nauc_ndcg_at_20_std value: 36.19985844757674 - type: nauc_ndcg_at_3_diff1 value: -17.33669028160956 - type: nauc_ndcg_at_3_max value: 16.074824713300455 - type: nauc_ndcg_at_3_std value: 27.403567224575443 - type: nauc_ndcg_at_5_diff1 value: -12.820784911984436 - type: nauc_ndcg_at_5_max value: 21.542886528832174 - type: nauc_ndcg_at_5_std value: 29.07282374739257 - type: nauc_precision_at_1000_diff1 value: -35.60122645356994 - type: nauc_precision_at_1000_max value: 2.766728293354684 - type: nauc_precision_at_1000_std value: 45.106529504482644 - type: nauc_precision_at_100_diff1 value: -42.734941084266296 - type: nauc_precision_at_100_max value: 14.953258911586055 - type: nauc_precision_at_100_std value: 55.037972769837175 - type: nauc_precision_at_10_diff1 value: -28.890384218334038 - type: nauc_precision_at_10_max value: 32.5381595525642 - type: nauc_precision_at_10_std value: 45.24099410430647 - type: nauc_precision_at_1_diff1 value: 15.276398946528131 - type: nauc_precision_at_1_max value: 8.717153299832296 - type: nauc_precision_at_1_std value: 28.600034127271652 - type: nauc_precision_at_20_diff1 value: -33.02729563387234 - type: nauc_precision_at_20_max value: 27.19557155623203 - type: nauc_precision_at_20_std value: 44.47754520309755 - type: nauc_precision_at_3_diff1 value: -14.501195444762063 - type: nauc_precision_at_3_max value: 13.86633130712465 - type: nauc_precision_at_3_std value: 40.179238021661355 - type: nauc_precision_at_5_diff1 value: -17.545737121609438 - type: nauc_precision_at_5_max value: 34.62230292211274 - type: nauc_precision_at_5_std value: 49.446043969419165 - type: nauc_recall_at_1000_diff1 value: -95.32777618701374 - type: nauc_recall_at_1000_max value: 48.60765801360888 - type: nauc_recall_at_1000_std value: 88.76245635034805 - type: nauc_recall_at_100_diff1 value: 5.691841865744837 - type: nauc_recall_at_100_max value: 23.96749843706024 - type: nauc_recall_at_100_std value: 1.9555743397908651 - type: nauc_recall_at_10_diff1 value: 27.624372183401157 - type: nauc_recall_at_10_max value: 4.019210332304043 - type: nauc_recall_at_10_std value: -20.85263152189046 - type: nauc_recall_at_1_diff1 value: 39.69550436091381 - type: nauc_recall_at_1_max value: -16.882723045728202 - type: nauc_recall_at_1_std value: -29.330395083370686 - type: nauc_recall_at_20_diff1 value: 22.01978317945599 - type: nauc_recall_at_20_max value: 7.144473524395364 - type: nauc_recall_at_20_std value: -18.490981365877715 - type: nauc_recall_at_3_diff1 value: 31.62038934003161 - type: nauc_recall_at_3_max value: -10.760815503593776 - type: nauc_recall_at_3_std value: -25.045384537758242 - type: nauc_recall_at_5_diff1 value: 25.9168912492029 - type: nauc_recall_at_5_max value: -1.57601021265437 - type: nauc_recall_at_5_std value: -21.644578724944903 - type: ndcg_at_1 value: 68.605 - type: ndcg_at_10 value: 62.479 - type: ndcg_at_100 value: 66.44200000000001 - type: ndcg_at_1000 value: 79.411 - type: ndcg_at_20 value: 61.366 - type: ndcg_at_3 value: 64.29899999999999 - type: ndcg_at_5 value: 64.265 - type: precision_at_1 value: 83.721 - type: precision_at_10 value: 72.558 - type: precision_at_100 value: 45.674 - type: precision_at_1000 value: 9.251 - type: precision_at_20 value: 65.581 - type: precision_at_3 value: 80.62 - type: precision_at_5 value: 79.07 - type: recall_at_1 value: 2.015 - type: recall_at_10 value: 14.13 - type: recall_at_100 value: 60.998 - type: recall_at_1000 value: 96.55799999999999 - type: recall_at_20 value: 23.983999999999998 - type: recall_at_3 value: 5.248 - type: recall_at_5 value: 8.456 task: type: Retrieval - dataset: config: pl name: MTEB MassiveIntentClassification (pl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 72.63954270342971 - type: f1 value: 69.45307590994116 - type: f1_weighted value: 71.79973967852227 - type: main_score value: 72.63954270342971 task: type: Classification - dataset: config: pl name: MTEB MassiveIntentClassification (pl) revision: 4672e20407010da34463acc759c162ca9734bca6 split: validation type: mteb/amazon_massive_intent metrics: - type: accuracy value: 73.62518445646828 - type: f1 value: 68.92188487113377 - type: f1_weighted value: 72.64149757960894 - type: main_score value: 73.62518445646828 task: type: Classification - dataset: config: pl name: MTEB MassiveScenarioClassification (pl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 76.56018829858776 - type: f1 value: 76.02756886857807 - type: f1_weighted value: 76.61092646592095 - type: main_score value: 76.56018829858776 task: type: Classification - dataset: config: pl name: MTEB MassiveScenarioClassification (pl) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: validation type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 76.7683226758485 - type: f1 value: 76.02969933641604 - type: f1_weighted value: 76.73489976040989 - type: main_score value: 76.7683226758485 task: type: Classification - dataset: config: pl name: MTEB MultiEURLEXMultilabelClassification (pl) revision: 2aea5a6dc8fdcfeca41d0fb963c0a338930bde5c split: test type: mteb/eurlex-multilingual metrics: - type: accuracy value: 4.218 - type: f1 value: 35.41806947791864 - type: lrap value: 46.31597017195712 - type: main_score value: 4.218 task: type: MultilabelClassification - dataset: config: pol name: MTEB MultiHateClassification (pol) revision: 8f95949846bb9e33c6aaf730ccfdb8fe6bcfb7a9 split: test type: mteb/multi-hatecheck metrics: - type: accuracy value: 61.550000000000004 - type: ap value: 34.943205720242624 - type: ap_weighted value: 34.943205720242624 - type: f1 value: 57.42076384232142 - type: f1_weighted value: 62.592131425733 - type: main_score value: 61.550000000000004 task: type: Classification - dataset: config: pol name: MTEB MultilingualSentimentClassification (pol) revision: 2b9b4d10fc589af67794141fe8cbd3739de1eb33 split: test type: mteb/multilingual-sentiment-classification metrics: - type: accuracy value: 90.58333333333331 - type: ap value: 85.78887796762379 - type: ap_weighted value: 85.78887796762379 - type: f1 value: 90.45854538654365 - type: f1_weighted value: 90.59802034163135 - type: main_score value: 90.58333333333331 task: type: Classification - dataset: config: default name: MTEB NFCorpus-PL (default) revision: 9a6f9567fda928260afed2de480d79c98bf0bec0 split: test type: clarin-knext/nfcorpus-pl metrics: - type: main_score value: 32.836999999999996 - type: map_at_1 value: 5.407 - type: map_at_10 value: 11.602 - type: map_at_100 value: 14.698 - type: map_at_1000 value: 16.116 - type: map_at_20 value: 12.712000000000002 - type: map_at_3 value: 8.519 - type: map_at_5 value: 9.892 - type: mrr_at_1 value: 43.653250773993804 - type: mrr_at_10 value: 52.844120104182025 - type: mrr_at_100 value: 53.449775403110614 - type: mrr_at_1000 value: 53.490875731391355 - type: mrr_at_20 value: 53.22127287411389 - type: mrr_at_3 value: 50.92879256965944 - type: mrr_at_5 value: 52.07430340557275 - type: nauc_map_at_1000_diff1 value: 21.252095579684628 - type: nauc_map_at_1000_max value: 26.70622083525327 - type: nauc_map_at_1000_std value: 8.767942285021244 - type: nauc_map_at_100_diff1 value: 21.91396527029906 - type: nauc_map_at_100_max value: 24.78831708869797 - type: nauc_map_at_100_std value: 4.685598528309627 - type: nauc_map_at_10_diff1 value: 23.354902063893743 - type: nauc_map_at_10_max value: 18.13889773190672 - type: nauc_map_at_10_std value: -5.875489355896885 - type: nauc_map_at_1_diff1 value: 33.360938146617976 - type: nauc_map_at_1_max value: 10.174509947819098 - type: nauc_map_at_1_std value: -15.198793788870043 - type: nauc_map_at_20_diff1 value: 23.212136861648684 - type: nauc_map_at_20_max value: 20.531117659821536 - type: nauc_map_at_20_std value: -2.786681819195299 - type: nauc_map_at_3_diff1 value: 28.31527977573991 - type: nauc_map_at_3_max value: 11.587612514076165 - type: nauc_map_at_3_std value: -12.603907215111693 - type: nauc_map_at_5_diff1 value: 25.965083702703495 - type: nauc_map_at_5_max value: 14.729243254441165 - type: nauc_map_at_5_std value: -10.677653224851138 - type: nauc_mrr_at_1000_diff1 value: 36.42318903416605 - type: nauc_mrr_at_1000_max value: 51.18409250517103 - type: nauc_mrr_at_1000_std value: 31.29570352877994 - type: nauc_mrr_at_100_diff1 value: 36.43269897021644 - type: nauc_mrr_at_100_max value: 51.21737308452086 - type: nauc_mrr_at_100_std value: 31.337400205547254 - type: nauc_mrr_at_10_diff1 value: 36.367711586434055 - type: nauc_mrr_at_10_max value: 51.13843457934155 - type: nauc_mrr_at_10_std value: 30.99940878155313 - type: nauc_mrr_at_1_diff1 value: 38.020555260298266 - type: nauc_mrr_at_1_max value: 45.76508007462295 - type: nauc_mrr_at_1_std value: 26.56239846524524 - type: nauc_mrr_at_20_diff1 value: 36.4735716127678 - type: nauc_mrr_at_20_max value: 51.18482045326876 - type: nauc_mrr_at_20_std value: 31.182761897720628 - type: nauc_mrr_at_3_diff1 value: 36.74620136204912 - type: nauc_mrr_at_3_max value: 50.46246550759349 - type: nauc_mrr_at_3_std value: 30.928776699074977 - type: nauc_mrr_at_5_diff1 value: 36.43604909883522 - type: nauc_mrr_at_5_max value: 50.961876020556105 - type: nauc_mrr_at_5_std value: 30.558689522445277 - type: nauc_ndcg_at_1000_diff1 value: 22.367261695212818 - type: nauc_ndcg_at_1000_max value: 46.381076848268314 - type: nauc_ndcg_at_1000_std value: 32.87206147838707 - type: nauc_ndcg_at_100_diff1 value: 21.484143323872498 - type: nauc_ndcg_at_100_max value: 39.12491937457514 - type: nauc_ndcg_at_100_std value: 23.79972574892049 - type: nauc_ndcg_at_10_diff1 value: 23.61423535088915 - type: nauc_ndcg_at_10_max value: 39.18046026051968 - type: nauc_ndcg_at_10_std value: 21.500043266959334 - type: nauc_ndcg_at_1_diff1 value: 37.55672462811441 - type: nauc_ndcg_at_1_max value: 44.461461345711996 - type: nauc_ndcg_at_1_std value: 24.638179893174854 - type: nauc_ndcg_at_20_diff1 value: 23.313762933875577 - type: nauc_ndcg_at_20_max value: 37.61712630443033 - type: nauc_ndcg_at_20_std value: 20.47709166123172 - type: nauc_ndcg_at_3_diff1 value: 26.65064647305834 - type: nauc_ndcg_at_3_max value: 42.37171469906345 - type: nauc_ndcg_at_3_std value: 23.179726698729016 - type: nauc_ndcg_at_5_diff1 value: 24.262311738959486 - type: nauc_ndcg_at_5_max value: 41.40200588435856 - type: nauc_ndcg_at_5_std value: 21.662269993534142 - type: nauc_precision_at_1000_diff1 value: -1.289992273496289 - type: nauc_precision_at_1000_max value: 15.583146284118996 - type: nauc_precision_at_1000_std value: 29.804029015112356 - type: nauc_precision_at_100_diff1 value: 1.564940945124233 - type: nauc_precision_at_100_max value: 26.392385300263037 - type: nauc_precision_at_100_std value: 38.76232331369285 - type: nauc_precision_at_10_diff1 value: 10.074337562319577 - type: nauc_precision_at_10_max value: 40.138351711813634 - type: nauc_precision_at_10_std value: 34.220232921350025 - type: nauc_precision_at_1_diff1 value: 38.020555260298266 - type: nauc_precision_at_1_max value: 45.76508007462295 - type: nauc_precision_at_1_std value: 26.56239846524524 - type: nauc_precision_at_20_diff1 value: 7.895964325229299 - type: nauc_precision_at_20_max value: 37.24274329998079 - type: nauc_precision_at_20_std value: 37.404193953417106 - type: nauc_precision_at_3_diff1 value: 19.55020676002291 - type: nauc_precision_at_3_max value: 43.59370824619839 - type: nauc_precision_at_3_std value: 28.46885149275582 - type: nauc_precision_at_5_diff1 value: 13.091453046202925 - type: nauc_precision_at_5_max value: 43.182417322718536 - type: nauc_precision_at_5_std value: 29.92195463487166 - type: nauc_recall_at_1000_diff1 value: 4.97907300227104 - type: nauc_recall_at_1000_max value: 22.28246568460588 - type: nauc_recall_at_1000_std value: 25.715913887754134 - type: nauc_recall_at_100_diff1 value: 10.909246688962302 - type: nauc_recall_at_100_max value: 22.341695674602903 - type: nauc_recall_at_100_std value: 15.46486876238366 - type: nauc_recall_at_10_diff1 value: 15.684351712023942 - type: nauc_recall_at_10_max value: 13.99640438074371 - type: nauc_recall_at_10_std value: -5.991871068665403 - type: nauc_recall_at_1_diff1 value: 33.360938146617976 - type: nauc_recall_at_1_max value: 10.174509947819098 - type: nauc_recall_at_1_std value: -15.198793788870043 - type: nauc_recall_at_20_diff1 value: 15.905172434675574 - type: nauc_recall_at_20_max value: 15.783093168064557 - type: nauc_recall_at_20_std value: -2.263046672061753 - type: nauc_recall_at_3_diff1 value: 26.426234689785133 - type: nauc_recall_at_3_max value: 11.087891452701028 - type: nauc_recall_at_3_std value: -11.772482622591477 - type: nauc_recall_at_5_diff1 value: 22.11736638320966 - type: nauc_recall_at_5_max value: 13.571692363425214 - type: nauc_recall_at_5_std value: -11.244894889011983 - type: ndcg_at_1 value: 42.26 - type: ndcg_at_10 value: 32.836999999999996 - type: ndcg_at_100 value: 30.341 - type: ndcg_at_1000 value: 39.114 - type: ndcg_at_20 value: 30.375999999999998 - type: ndcg_at_3 value: 38.174 - type: ndcg_at_5 value: 35.945 - type: precision_at_1 value: 43.653 - type: precision_at_10 value: 24.303 - type: precision_at_100 value: 8.025 - type: precision_at_1000 value: 2.062 - type: precision_at_20 value: 17.616 - type: precision_at_3 value: 36.429 - type: precision_at_5 value: 31.207 - type: recall_at_1 value: 5.407 - type: recall_at_10 value: 16.026 - type: recall_at_100 value: 30.755 - type: recall_at_1000 value: 63.058 - type: recall_at_20 value: 19.56 - type: recall_at_3 value: 9.383 - type: recall_at_5 value: 11.955 task: type: Retrieval - dataset: config: default name: MTEB NQ-PL (default) revision: f171245712cf85dd4700b06bef18001578d0ca8d split: test type: clarin-knext/nq-pl metrics: - type: main_score value: 40.046 - type: map_at_1 value: 20.836 - type: map_at_10 value: 33.147 - type: map_at_100 value: 34.305 - type: map_at_1000 value: 34.365 - type: map_at_20 value: 33.866 - type: map_at_3 value: 29.029 - type: map_at_5 value: 31.387999999999998 - type: mrr_at_1 value: 23.638470451911935 - type: mrr_at_10 value: 35.410666004524636 - type: mrr_at_100 value: 36.39008970520291 - type: mrr_at_1000 value: 36.434922659126926 - type: mrr_at_20 value: 36.040096707445514 - type: mrr_at_3 value: 31.909038238702202 - type: mrr_at_5 value: 33.951332560834295 - type: nauc_map_at_1000_diff1 value: 30.33789317509847 - type: nauc_map_at_1000_max value: 34.42459859725807 - type: nauc_map_at_1000_std value: 9.97643796318094 - type: nauc_map_at_100_diff1 value: 30.330830102419498 - type: nauc_map_at_100_max value: 34.447821563623194 - type: nauc_map_at_100_std value: 10.005636991113455 - type: nauc_map_at_10_diff1 value: 30.34368851651713 - type: nauc_map_at_10_max value: 34.049110329953194 - type: nauc_map_at_10_std value: 9.191908504593075 - type: nauc_map_at_1_diff1 value: 35.26679557790614 - type: nauc_map_at_1_max value: 29.771974215418318 - type: nauc_map_at_1_std value: 4.776546296003624 - type: nauc_map_at_20_diff1 value: 30.281421969911225 - type: nauc_map_at_20_max value: 34.33650674553603 - type: nauc_map_at_20_std value: 9.727603420278799 - type: nauc_map_at_3_diff1 value: 31.145137267648813 - type: nauc_map_at_3_max value: 32.38852468715383 - type: nauc_map_at_3_std value: 6.999311805347667 - type: nauc_map_at_5_diff1 value: 30.32736513433838 - type: nauc_map_at_5_max value: 33.292378027926425 - type: nauc_map_at_5_std value: 8.145447139184112 - type: nauc_mrr_at_1000_diff1 value: 29.831199924703817 - type: nauc_mrr_at_1000_max value: 34.10844541447466 - type: nauc_mrr_at_1000_std value: 11.226040436619018 - type: nauc_mrr_at_100_diff1 value: 29.83092714251575 - type: nauc_mrr_at_100_max value: 34.13324232175186 - type: nauc_mrr_at_100_std value: 11.251560383955486 - type: nauc_mrr_at_10_diff1 value: 29.761109883105085 - type: nauc_mrr_at_10_max value: 33.87749240441422 - type: nauc_mrr_at_10_std value: 10.756431499661819 - type: nauc_mrr_at_1_diff1 value: 33.989066055249154 - type: nauc_mrr_at_1_max value: 30.395509585555487 - type: nauc_mrr_at_1_std value: 6.5196986843728295 - type: nauc_mrr_at_20_diff1 value: 29.746190088974455 - type: nauc_mrr_at_20_max value: 34.070814927054016 - type: nauc_mrr_at_20_std value: 11.101583529780374 - type: nauc_mrr_at_3_diff1 value: 30.086990584032293 - type: nauc_mrr_at_3_max value: 32.714654369197135 - type: nauc_mrr_at_3_std value: 9.37948033677932 - type: nauc_mrr_at_5_diff1 value: 29.53541842449955 - type: nauc_mrr_at_5_max value: 33.34401975157522 - type: nauc_mrr_at_5_std value: 10.156864160967393 - type: nauc_ndcg_at_1000_diff1 value: 29.110288584326693 - type: nauc_ndcg_at_1000_max value: 36.90253858086684 - type: nauc_ndcg_at_1000_std value: 14.532331118999608 - type: nauc_ndcg_at_100_diff1 value: 28.97540691413146 - type: nauc_ndcg_at_100_max value: 37.584960544191304 - type: nauc_ndcg_at_100_std value: 15.442845119344694 - type: nauc_ndcg_at_10_diff1 value: 28.795881184832485 - type: nauc_ndcg_at_10_max value: 35.94272625161231 - type: nauc_ndcg_at_10_std value: 12.023528372246636 - type: nauc_ndcg_at_1_diff1 value: 34.100597438365206 - type: nauc_ndcg_at_1_max value: 30.35880635797757 - type: nauc_ndcg_at_1_std value: 6.556654032875343 - type: nauc_ndcg_at_20_diff1 value: 28.616176665243685 - type: nauc_ndcg_at_20_max value: 36.94100478517521 - type: nauc_ndcg_at_20_std value: 13.84115539697913 - type: nauc_ndcg_at_3_diff1 value: 29.800875843729536 - type: nauc_ndcg_at_3_max value: 32.97586122644207 - type: nauc_ndcg_at_3_std value: 8.254595787337179 - type: nauc_ndcg_at_5_diff1 value: 28.564485497706897 - type: nauc_ndcg_at_5_max value: 34.36550798481214 - type: nauc_ndcg_at_5_std value: 9.979146706487176 - type: nauc_precision_at_1000_diff1 value: -2.0183745076034305 - type: nauc_precision_at_1000_max value: 17.01703235798985 - type: nauc_precision_at_1000_std value: 24.752710326143955 - type: nauc_precision_at_100_diff1 value: 5.351886729324482 - type: nauc_precision_at_100_max value: 30.913625083631977 - type: nauc_precision_at_100_std value: 33.69481614506668 - type: nauc_precision_at_10_diff1 value: 15.938911938541489 - type: nauc_precision_at_10_max value: 35.6404201670323 - type: nauc_precision_at_10_std value: 21.02640517885096 - type: nauc_precision_at_1_diff1 value: 34.100597438365206 - type: nauc_precision_at_1_max value: 30.35880635797757 - type: nauc_precision_at_1_std value: 6.556654032875343 - type: nauc_precision_at_20_diff1 value: 12.06019821733859 - type: nauc_precision_at_20_max value: 35.709266792371615 - type: nauc_precision_at_20_std value: 26.83935698262609 - type: nauc_precision_at_3_diff1 value: 23.293715443749498 - type: nauc_precision_at_3_max value: 34.352815559576754 - type: nauc_precision_at_3_std value: 12.400616118285498 - type: nauc_precision_at_5_diff1 value: 18.62536694684322 - type: nauc_precision_at_5_max value: 35.23411734933911 - type: nauc_precision_at_5_std value: 16.198868970484614 - type: nauc_recall_at_1000_diff1 value: 19.95715343716757 - type: nauc_recall_at_1000_max value: 66.40788089117626 - type: nauc_recall_at_1000_std value: 68.87954839929897 - type: nauc_recall_at_100_diff1 value: 22.83247580652378 - type: nauc_recall_at_100_max value: 56.48998030807925 - type: nauc_recall_at_100_std value: 47.86881793700064 - type: nauc_recall_at_10_diff1 value: 24.09148418102836 - type: nauc_recall_at_10_max value: 40.405060126060754 - type: nauc_recall_at_10_std value: 18.243191480781267 - type: nauc_recall_at_1_diff1 value: 35.26679557790614 - type: nauc_recall_at_1_max value: 29.771974215418318 - type: nauc_recall_at_1_std value: 4.776546296003624 - type: nauc_recall_at_20_diff1 value: 22.69685579729365 - type: nauc_recall_at_20_max value: 45.429255336368044 - type: nauc_recall_at_20_std value: 26.960602261646045 - type: nauc_recall_at_3_diff1 value: 26.763890354016656 - type: nauc_recall_at_3_max value: 33.44197235279934 - type: nauc_recall_at_3_std value: 9.139561741646231 - type: nauc_recall_at_5_diff1 value: 23.79794408992853 - type: nauc_recall_at_5_max value: 35.948339011548484 - type: nauc_recall_at_5_std value: 12.507365356132215 - type: ndcg_at_1 value: 23.61 - type: ndcg_at_10 value: 40.046 - type: ndcg_at_100 value: 45.353 - type: ndcg_at_1000 value: 46.811 - type: ndcg_at_20 value: 42.482 - type: ndcg_at_3 value: 32.162 - type: ndcg_at_5 value: 36.146 - type: precision_at_1 value: 23.61 - type: precision_at_10 value: 7.037 - type: precision_at_100 value: 1.0 - type: precision_at_1000 value: 0.11399999999999999 - type: precision_at_20 value: 4.079 - type: precision_at_3 value: 14.929 - type: precision_at_5 value: 11.251 - type: recall_at_1 value: 20.836 - type: recall_at_10 value: 59.021 - type: recall_at_100 value: 82.814 - type: recall_at_1000 value: 93.675 - type: recall_at_20 value: 68.185 - type: recall_at_3 value: 38.376 - type: recall_at_5 value: 47.615 task: type: Retrieval - dataset: config: default name: MTEB NQ-PLHardNegatives (default) revision: 9a2878a70ea545a8f4df0cdfa1adea27f4f64390 split: test type: mteb/NQ_PL_test_top_250_only_w_correct-v2 metrics: - type: main_score value: 42.744 - type: map_at_1 value: 22.167 - type: map_at_10 value: 35.442 - type: map_at_100 value: 36.809 - type: map_at_1000 value: 36.849 - type: map_at_20 value: 36.281 - type: map_at_3 value: 30.911 - type: map_at_5 value: 33.799 - type: mrr_at_1 value: 24.9 - type: mrr_at_10 value: 37.717619047619046 - type: mrr_at_100 value: 38.83175098401238 - type: mrr_at_1000 value: 38.863984091549106 - type: mrr_at_20 value: 38.42165681025202 - type: mrr_at_3 value: 33.93333333333333 - type: mrr_at_5 value: 36.38333333333333 - type: nauc_map_at_1000_diff1 value: 29.349107175768076 - type: nauc_map_at_1000_max value: 32.65755799907269 - type: nauc_map_at_1000_std value: -3.806253487436536 - type: nauc_map_at_100_diff1 value: 29.32313704694836 - type: nauc_map_at_100_max value: 32.674805659308234 - type: nauc_map_at_100_std value: -3.763463883857777 - type: nauc_map_at_10_diff1 value: 29.378099045661276 - type: nauc_map_at_10_max value: 32.40977984208203 - type: nauc_map_at_10_std value: -4.750749976310079 - type: nauc_map_at_1_diff1 value: 34.65899813124808 - type: nauc_map_at_1_max value: 27.5836856142388 - type: nauc_map_at_1_std value: -5.77408059219477 - type: nauc_map_at_20_diff1 value: 29.234934100000142 - type: nauc_map_at_20_max value: 32.64242252873959 - type: nauc_map_at_20_std value: -3.9339041815132245 - type: nauc_map_at_3_diff1 value: 29.75553691236355 - type: nauc_map_at_3_max value: 30.628821642751685 - type: nauc_map_at_3_std value: -6.71815410382398 - type: nauc_map_at_5_diff1 value: 29.67011442099744 - type: nauc_map_at_5_max value: 31.880169485661735 - type: nauc_map_at_5_std value: -5.23243858089082 - type: nauc_mrr_at_1000_diff1 value: 29.034444335684984 - type: nauc_mrr_at_1000_max value: 32.3826387684809 - type: nauc_mrr_at_1000_std value: -1.6453099036883947 - type: nauc_mrr_at_100_diff1 value: 29.012673846053328 - type: nauc_mrr_at_100_max value: 32.39330681563384 - type: nauc_mrr_at_100_std value: -1.611798217539563 - type: nauc_mrr_at_10_diff1 value: 28.927278042494308 - type: nauc_mrr_at_10_max value: 32.241913064296654 - type: nauc_mrr_at_10_std value: -2.196305056221632 - type: nauc_mrr_at_1_diff1 value: 34.193706220534295 - type: nauc_mrr_at_1_max value: 28.472182308269332 - type: nauc_mrr_at_1_std value: -3.588866144076354 - type: nauc_mrr_at_20_diff1 value: 28.905967167900897 - type: nauc_mrr_at_20_max value: 32.514840048451944 - type: nauc_mrr_at_20_std value: -1.5928217819446544 - type: nauc_mrr_at_3_diff1 value: 28.910819895018104 - type: nauc_mrr_at_3_max value: 31.02262661603842 - type: nauc_mrr_at_3_std value: -3.7189550210915105 - type: nauc_mrr_at_5_diff1 value: 29.153815747673384 - type: nauc_mrr_at_5_max value: 32.020862613895844 - type: nauc_mrr_at_5_std value: -2.559856974862219 - type: nauc_ndcg_at_1000_diff1 value: 28.03384223859073 - type: nauc_ndcg_at_1000_max value: 34.65513427128041 - type: nauc_ndcg_at_1000_std value: -0.24943181025018912 - type: nauc_ndcg_at_100_diff1 value: 27.314090721793853 - type: nauc_ndcg_at_100_max value: 35.139601373088546 - type: nauc_ndcg_at_100_std value: 1.029635770311101 - type: nauc_ndcg_at_10_diff1 value: 27.406187481616122 - type: nauc_ndcg_at_10_max value: 34.294782527860775 - type: nauc_ndcg_at_10_std value: -2.7210616901005844 - type: nauc_ndcg_at_1_diff1 value: 34.193706220534295 - type: nauc_ndcg_at_1_max value: 28.472182308269332 - type: nauc_ndcg_at_1_std value: -3.588866144076354 - type: nauc_ndcg_at_20_diff1 value: 26.95213352438165 - type: nauc_ndcg_at_20_max value: 35.29748291845877 - type: nauc_ndcg_at_20_std value: -0.030327507133456177 - type: nauc_ndcg_at_3_diff1 value: 27.96016479797334 - type: nauc_ndcg_at_3_max value: 31.260064258340847 - type: nauc_ndcg_at_3_std value: -6.185360480716815 - type: nauc_ndcg_at_5_diff1 value: 28.09280399502525 - type: nauc_ndcg_at_5_max value: 33.303439636075424 - type: nauc_ndcg_at_5_std value: -3.772285985055726 - type: nauc_precision_at_1000_diff1 value: -4.281358315231413 - type: nauc_precision_at_1000_max value: 15.068351358617104 - type: nauc_precision_at_1000_std value: 22.607343044895217 - type: nauc_precision_at_100_diff1 value: -1.8498660906232658 - type: nauc_precision_at_100_max value: 24.711460259737567 - type: nauc_precision_at_100_std value: 29.23005993545415 - type: nauc_precision_at_10_diff1 value: 12.909738393494976 - type: nauc_precision_at_10_max value: 35.070650662332916 - type: nauc_precision_at_10_std value: 6.898514999972704 - type: nauc_precision_at_1_diff1 value: 34.193706220534295 - type: nauc_precision_at_1_max value: 28.472182308269332 - type: nauc_precision_at_1_std value: -3.588866144076354 - type: nauc_precision_at_20_diff1 value: 7.203222790381147 - type: nauc_precision_at_20_max value: 34.687485444161794 - type: nauc_precision_at_20_std value: 18.72758400919753 - type: nauc_precision_at_3_diff1 value: 21.26316091253772 - type: nauc_precision_at_3_max value: 33.927142239391834 - type: nauc_precision_at_3_std value: -4.082242115397914 - type: nauc_precision_at_5_diff1 value: 18.018252640795644 - type: nauc_precision_at_5_max value: 35.690648047326626 - type: nauc_precision_at_5_std value: 2.7145471077534262 - type: nauc_recall_at_1000_diff1 value: 20.536163183221504 - type: nauc_recall_at_1000_max value: 81.88608776844036 - type: nauc_recall_at_1000_std value: 76.64799253034528 - type: nauc_recall_at_100_diff1 value: 8.475735597617868 - type: nauc_recall_at_100_max value: 56.05392838189596 - type: nauc_recall_at_100_std value: 47.124156579313166 - type: nauc_recall_at_10_diff1 value: 20.846541949906808 - type: nauc_recall_at_10_max value: 38.356937471723704 - type: nauc_recall_at_10_std value: 0.9534412860615692 - type: nauc_recall_at_1_diff1 value: 34.65899813124808 - type: nauc_recall_at_1_max value: 27.5836856142388 - type: nauc_recall_at_1_std value: -5.77408059219477 - type: nauc_recall_at_20_diff1 value: 17.388853770371078 - type: nauc_recall_at_20_max value: 44.5348067371247 - type: nauc_recall_at_20_std value: 13.477242036566658 - type: nauc_recall_at_3_diff1 value: 23.292689488826923 - type: nauc_recall_at_3_max value: 31.397211941626402 - type: nauc_recall_at_3_std value: -7.266266306172304 - type: nauc_recall_at_5_diff1 value: 23.446848626918783 - type: nauc_recall_at_5_max value: 35.30890874307417 - type: nauc_recall_at_5_std value: -2.1058394852209883 - type: ndcg_at_1 value: 24.9 - type: ndcg_at_10 value: 42.744 - type: ndcg_at_100 value: 48.995 - type: ndcg_at_1000 value: 49.944 - type: ndcg_at_20 value: 45.589 - type: ndcg_at_3 value: 34.311 - type: ndcg_at_5 value: 39.127 - type: precision_at_1 value: 24.9 - type: precision_at_10 value: 7.430000000000001 - type: precision_at_100 value: 1.098 - type: precision_at_1000 value: 0.11900000000000001 - type: precision_at_20 value: 4.385 - type: precision_at_3 value: 15.8 - type: precision_at_5 value: 12.2 - type: recall_at_1 value: 22.167 - type: recall_at_10 value: 62.766999999999996 - type: recall_at_100 value: 90.75 - type: recall_at_1000 value: 97.833 - type: recall_at_20 value: 73.483 - type: recall_at_3 value: 41.15 - type: recall_at_5 value: 52.217 task: type: Retrieval - dataset: config: arb_Arab-pol_Latn name: MTEB NTREXBitextMining (arb_Arab-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 2.6039058587881825 - type: f1 value: 1.7420063165137074 - type: main_score value: 1.7420063165137074 - type: precision value: 1.5987458678183388 - type: recall value: 2.6039058587881825 task: type: BitextMining - dataset: config: bel_Cyrl-pol_Latn name: MTEB NTREXBitextMining (bel_Cyrl-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 6.71006509764647 - type: f1 value: 5.379820175253383 - type: main_score value: 5.379820175253383 - type: precision value: 5.006027887096325 - type: recall value: 6.71006509764647 task: type: BitextMining - dataset: config: ben_Beng-pol_Latn name: MTEB NTREXBitextMining (ben_Beng-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 1.0015022533800702 - type: f1 value: 0.657269658938351 - type: main_score value: 0.657269658938351 - type: precision value: 0.5774496357952187 - type: recall value: 1.0015022533800702 task: type: BitextMining - dataset: config: bos_Latn-pol_Latn name: MTEB NTREXBitextMining (bos_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 41.4621932899349 - type: f1 value: 36.566294033782604 - type: main_score value: 36.566294033782604 - type: precision value: 34.94809949228646 - type: recall value: 41.4621932899349 task: type: BitextMining - dataset: config: bul_Cyrl-pol_Latn name: MTEB NTREXBitextMining (bul_Cyrl-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 7.611417125688533 - type: f1 value: 6.256776694331179 - type: main_score value: 6.256776694331179 - type: precision value: 5.9087199342302625 - type: recall value: 7.611417125688533 task: type: BitextMining - dataset: config: ces_Latn-pol_Latn name: MTEB NTREXBitextMining (ces_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 44.566850275413124 - type: f1 value: 39.63102995727847 - type: main_score value: 39.63102995727847 - type: precision value: 37.92173103351191 - type: recall value: 44.566850275413124 task: type: BitextMining - dataset: config: deu_Latn-pol_Latn name: MTEB NTREXBitextMining (deu_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 41.4621932899349 - type: f1 value: 37.33454445419768 - type: main_score value: 37.33454445419768 - type: precision value: 35.95442215997948 - type: recall value: 41.4621932899349 task: type: BitextMining - dataset: config: ell_Grek-pol_Latn name: MTEB NTREXBitextMining (ell_Grek-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 8.462694041061592 - type: f1 value: 6.8087684894952885 - type: main_score value: 6.8087684894952885 - type: precision value: 6.376933631700838 - type: recall value: 8.462694041061592 task: type: BitextMining - dataset: config: eng_Latn-pol_Latn name: MTEB NTREXBitextMining (eng_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 96.84526790185278 - type: f1 value: 95.9472542146553 - type: main_score value: 95.9472542146553 - type: precision value: 95.52411951260224 - type: recall value: 96.84526790185278 task: type: BitextMining - dataset: config: fas_Arab-pol_Latn name: MTEB NTREXBitextMining (fas_Arab-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 11.817726589884828 - type: f1 value: 10.601177340803874 - type: main_score value: 10.601177340803874 - type: precision value: 10.223703198232055 - type: recall value: 11.817726589884828 task: type: BitextMining - dataset: config: fin_Latn-pol_Latn name: MTEB NTREXBitextMining (fin_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 24.13620430645969 - type: f1 value: 20.11481738578615 - type: main_score value: 20.11481738578615 - type: precision value: 19.018296909416655 - type: recall value: 24.13620430645969 task: type: BitextMining - dataset: config: fra_Latn-pol_Latn name: MTEB NTREXBitextMining (fra_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 43.415122684026045 - type: f1 value: 39.73512828041284 - type: main_score value: 39.73512828041284 - type: precision value: 38.61547526672604 - type: recall value: 43.415122684026045 task: type: BitextMining - dataset: config: heb_Hebr-pol_Latn name: MTEB NTREXBitextMining (heb_Hebr-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 5.107661492238358 - type: f1 value: 3.7469320469398637 - type: main_score value: 3.7469320469398637 - type: precision value: 3.4073111528601916 - type: recall value: 5.107661492238358 task: type: BitextMining - dataset: config: hin_Deva-pol_Latn name: MTEB NTREXBitextMining (hin_Deva-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 3.755633450175263 - type: f1 value: 2.6979430411046788 - type: main_score value: 2.6979430411046788 - type: precision value: 2.476982778272265 - type: recall value: 3.755633450175263 task: type: BitextMining - dataset: config: hrv_Latn-pol_Latn name: MTEB NTREXBitextMining (hrv_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 41.61241862794191 - type: f1 value: 36.74310386132114 - type: main_score value: 36.74310386132114 - type: precision value: 35.08204010628832 - type: recall value: 41.61241862794191 task: type: BitextMining - dataset: config: hun_Latn-pol_Latn name: MTEB NTREXBitextMining (hun_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 25.187781672508763 - type: f1 value: 21.589008204041267 - type: main_score value: 21.589008204041267 - type: precision value: 20.550856861808544 - type: recall value: 25.187781672508763 task: type: BitextMining - dataset: config: ind_Latn-pol_Latn name: MTEB NTREXBitextMining (ind_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 25.38808212318478 - type: f1 value: 22.027865588136557 - type: main_score value: 22.027865588136557 - type: precision value: 21.02193156294836 - type: recall value: 25.38808212318478 task: type: BitextMining - dataset: config: jpn_Jpan-pol_Latn name: MTEB NTREXBitextMining (jpn_Jpan-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 1.4521782674011017 - type: f1 value: 1.1551267029994832 - type: main_score value: 1.1551267029994832 - type: precision value: 1.0709350063506407 - type: recall value: 1.4521782674011017 task: type: BitextMining - dataset: config: kor_Hang-pol_Latn name: MTEB NTREXBitextMining (kor_Hang-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 8.662994491737606 - type: f1 value: 7.702831195040925 - type: main_score value: 7.702831195040925 - type: precision value: 7.399493724590964 - type: recall value: 8.662994491737606 task: type: BitextMining - dataset: config: lit_Latn-pol_Latn name: MTEB NTREXBitextMining (lit_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 27.240861291937907 - type: f1 value: 22.67568012885724 - type: main_score value: 22.67568012885724 - type: precision value: 21.345389137270303 - type: recall value: 27.240861291937907 task: type: BitextMining - dataset: config: mkd_Cyrl-pol_Latn name: MTEB NTREXBitextMining (mkd_Cyrl-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 5.2578868302453685 - type: f1 value: 4.206059607176409 - type: main_score value: 4.206059607176409 - type: precision value: 3.9704622621726124 - type: recall value: 5.2578868302453685 task: type: BitextMining - dataset: config: nld_Latn-pol_Latn name: MTEB NTREXBitextMining (nld_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 41.66249374061092 - type: f1 value: 37.07298258743049 - type: main_score value: 37.07298258743049 - type: precision value: 35.62787031614285 - type: recall value: 41.66249374061092 task: type: BitextMining - dataset: config: pol_Latn-arb_Arab name: MTEB NTREXBitextMining (pol_Latn-arb_Arab) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 4.606910365548322 - type: f1 value: 2.46942582684724 - type: main_score value: 2.46942582684724 - type: precision value: 2.1050793953671145 - type: recall value: 4.606910365548322 task: type: BitextMining - dataset: config: pol_Latn-bel_Cyrl name: MTEB NTREXBitextMining (pol_Latn-bel_Cyrl) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 8.662994491737606 - type: f1 value: 5.182695919286112 - type: main_score value: 5.182695919286112 - type: precision value: 4.489760167778365 - type: recall value: 8.662994491737606 task: type: BitextMining - dataset: config: pol_Latn-ben_Beng name: MTEB NTREXBitextMining (pol_Latn-ben_Beng) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 2.6539809714571856 - type: f1 value: 1.258614201836631 - type: main_score value: 1.258614201836631 - type: precision value: 1.0713961698889642 - type: recall value: 2.6539809714571856 task: type: BitextMining - dataset: config: pol_Latn-bos_Latn name: MTEB NTREXBitextMining (pol_Latn-bos_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 36.55483224837256 - type: f1 value: 29.774094494365304 - type: main_score value: 29.774094494365304 - type: precision value: 27.80072080015492 - type: recall value: 36.55483224837256 task: type: BitextMining - dataset: config: pol_Latn-bul_Cyrl name: MTEB NTREXBitextMining (pol_Latn-bul_Cyrl) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 10.015022533800702 - type: f1 value: 6.883913130592241 - type: main_score value: 6.883913130592241 - type: precision value: 6.16024442614442 - type: recall value: 10.015022533800702 task: type: BitextMining - dataset: config: pol_Latn-ces_Latn name: MTEB NTREXBitextMining (pol_Latn-ces_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 42.06309464196295 - type: f1 value: 34.378555224013816 - type: main_score value: 34.378555224013816 - type: precision value: 31.962839439959527 - type: recall value: 42.06309464196295 task: type: BitextMining - dataset: config: pol_Latn-deu_Latn name: MTEB NTREXBitextMining (pol_Latn-deu_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 43.465197796695044 - type: f1 value: 35.448522434000644 - type: main_score value: 35.448522434000644 - type: precision value: 32.92927940126679 - type: recall value: 43.465197796695044 task: type: BitextMining - dataset: config: pol_Latn-ell_Grek name: MTEB NTREXBitextMining (pol_Latn-ell_Grek) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 10.665998998497747 - type: f1 value: 6.412653388709864 - type: main_score value: 6.412653388709864 - type: precision value: 5.553629367440621 - type: recall value: 10.665998998497747 task: type: BitextMining - dataset: config: pol_Latn-eng_Latn name: MTEB NTREXBitextMining (pol_Latn-eng_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 95.6434651977967 - type: f1 value: 94.3281589050242 - type: main_score value: 94.3281589050242 - type: precision value: 93.69470872976132 - type: recall value: 95.6434651977967 task: type: BitextMining - dataset: config: pol_Latn-fas_Arab name: MTEB NTREXBitextMining (pol_Latn-fas_Arab) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 14.521782674011016 - type: f1 value: 9.83927594272999 - type: main_score value: 9.83927594272999 - type: precision value: 8.762366993974261 - type: recall value: 14.521782674011016 task: type: BitextMining - dataset: config: pol_Latn-fin_Latn name: MTEB NTREXBitextMining (pol_Latn-fin_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 24.837255883825737 - type: f1 value: 17.81630308485758 - type: main_score value: 17.81630308485758 - type: precision value: 16.06189265176832 - type: recall value: 24.837255883825737 task: type: BitextMining - dataset: config: pol_Latn-fra_Latn name: MTEB NTREXBitextMining (pol_Latn-fra_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 46.21932899349023 - type: f1 value: 37.289762204634506 - type: main_score value: 37.289762204634506 - type: precision value: 34.35696093911357 - type: recall value: 46.21932899349023 task: type: BitextMining - dataset: config: pol_Latn-heb_Hebr name: MTEB NTREXBitextMining (pol_Latn-heb_Hebr) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 7.811717576364546 - type: f1 value: 4.325980977253967 - type: main_score value: 4.325980977253967 - type: precision value: 3.692034397493451 - type: recall value: 7.811717576364546 task: type: BitextMining - dataset: config: pol_Latn-hin_Deva name: MTEB NTREXBitextMining (pol_Latn-hin_Deva) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 5.408112168252378 - type: f1 value: 3.3012026283157954 - type: main_score value: 3.3012026283157954 - type: precision value: 2.9200987844982302 - type: recall value: 5.408112168252378 task: type: BitextMining - dataset: config: pol_Latn-hrv_Latn name: MTEB NTREXBitextMining (pol_Latn-hrv_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 35.65348022033049 - type: f1 value: 28.87968630538368 - type: main_score value: 28.87968630538368 - type: precision value: 26.862013496119065 - type: recall value: 35.65348022033049 task: type: BitextMining - dataset: config: pol_Latn-hun_Latn name: MTEB NTREXBitextMining (pol_Latn-hun_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 25.58838257386079 - type: f1 value: 18.48609113402143 - type: main_score value: 18.48609113402143 - type: precision value: 16.57296197744189 - type: recall value: 25.58838257386079 task: type: BitextMining - dataset: config: pol_Latn-ind_Latn name: MTEB NTREXBitextMining (pol_Latn-ind_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 29.093640460691038 - type: f1 value: 21.347745568207063 - type: main_score value: 21.347745568207063 - type: precision value: 19.290524664413102 - type: recall value: 29.093640460691038 task: type: BitextMining - dataset: config: pol_Latn-jpn_Jpan name: MTEB NTREXBitextMining (pol_Latn-jpn_Jpan) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 3.1547320981472207 - type: f1 value: 1.3591775209486017 - type: main_score value: 1.3591775209486017 - type: precision value: 1.0911488276885473 - type: recall value: 3.1547320981472207 task: type: BitextMining - dataset: config: pol_Latn-kor_Hang name: MTEB NTREXBitextMining (pol_Latn-kor_Hang) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 10.866299449173761 - type: f1 value: 6.470438882301941 - type: main_score value: 6.470438882301941 - type: precision value: 5.5886448167365 - type: recall value: 10.866299449173761 task: type: BitextMining - dataset: config: pol_Latn-lit_Latn name: MTEB NTREXBitextMining (pol_Latn-lit_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 25.38808212318478 - type: f1 value: 19.368194787457522 - type: main_score value: 19.368194787457522 - type: precision value: 17.739499047326586 - type: recall value: 25.38808212318478 task: type: BitextMining - dataset: config: pol_Latn-mkd_Cyrl name: MTEB NTREXBitextMining (pol_Latn-mkd_Cyrl) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 7.361041562343515 - type: f1 value: 4.620503162223216 - type: main_score value: 4.620503162223216 - type: precision value: 4.0633841300538 - type: recall value: 7.361041562343515 task: type: BitextMining - dataset: config: pol_Latn-nld_Latn name: MTEB NTREXBitextMining (pol_Latn-nld_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 42.16324486730095 - type: f1 value: 34.45435727520619 - type: main_score value: 34.45435727520619 - type: precision value: 32.16608920524314 - type: recall value: 42.16324486730095 task: type: BitextMining - dataset: config: pol_Latn-por_Latn name: MTEB NTREXBitextMining (pol_Latn-por_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 38.45768652979469 - type: f1 value: 30.772146637068243 - type: main_score value: 30.772146637068243 - type: precision value: 28.392073335240376 - type: recall value: 38.45768652979469 task: type: BitextMining - dataset: config: pol_Latn-rus_Cyrl name: MTEB NTREXBitextMining (pol_Latn-rus_Cyrl) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 9.01352028042063 - type: f1 value: 5.7469829042905545 - type: main_score value: 5.7469829042905545 - type: precision value: 5.093509037970949 - type: recall value: 9.01352028042063 task: type: BitextMining - dataset: config: pol_Latn-slk_Latn name: MTEB NTREXBitextMining (pol_Latn-slk_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 38.407611417125686 - type: f1 value: 31.11590026725889 - type: main_score value: 31.11590026725889 - type: precision value: 28.88764140353986 - type: recall value: 38.407611417125686 task: type: BitextMining - dataset: config: pol_Latn-slv_Latn name: MTEB NTREXBitextMining (pol_Latn-slv_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 33.24987481221833 - type: f1 value: 25.950822457683554 - type: main_score value: 25.950822457683554 - type: precision value: 23.846365535044157 - type: recall value: 33.24987481221833 task: type: BitextMining - dataset: config: pol_Latn-spa_Latn name: MTEB NTREXBitextMining (pol_Latn-spa_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 45.06760140210315 - type: f1 value: 36.824239877433804 - type: main_score value: 36.824239877433804 - type: precision value: 34.21671795018105 - type: recall value: 45.06760140210315 task: type: BitextMining - dataset: config: pol_Latn-srp_Cyrl name: MTEB NTREXBitextMining (pol_Latn-srp_Cyrl) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 7.911867801702553 - type: f1 value: 5.036235972381512 - type: main_score value: 5.036235972381512 - type: precision value: 4.4125802643383745 - type: recall value: 7.911867801702553 task: type: BitextMining - dataset: config: pol_Latn-srp_Latn name: MTEB NTREXBitextMining (pol_Latn-srp_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 25.43815723585378 - type: f1 value: 18.921733221569646 - type: main_score value: 18.921733221569646 - type: precision value: 17.32003570311133 - type: recall value: 25.43815723585378 task: type: BitextMining - dataset: config: pol_Latn-swa_Latn name: MTEB NTREXBitextMining (pol_Latn-swa_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 21.732598898347522 - type: f1 value: 15.880290797178201 - type: main_score value: 15.880290797178201 - type: precision value: 14.395403694763726 - type: recall value: 21.732598898347522 task: type: BitextMining - dataset: config: pol_Latn-swe_Latn name: MTEB NTREXBitextMining (pol_Latn-swe_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 40.610916374561846 - type: f1 value: 32.421490223283655 - type: main_score value: 32.421490223283655 - type: precision value: 29.896983936859545 - type: recall value: 40.610916374561846 task: type: BitextMining - dataset: config: pol_Latn-tam_Taml name: MTEB NTREXBitextMining (pol_Latn-tam_Taml) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 7.961942914371558 - type: f1 value: 4.558103071505523 - type: main_score value: 4.558103071505523 - type: precision value: 3.8662602469179768 - type: recall value: 7.961942914371558 task: type: BitextMining - dataset: config: pol_Latn-tur_Latn name: MTEB NTREXBitextMining (pol_Latn-tur_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 28.392588883324986 - type: f1 value: 22.030266589108034 - type: main_score value: 22.030266589108034 - type: precision value: 20.308934108335674 - type: recall value: 28.392588883324986 task: type: BitextMining - dataset: config: pol_Latn-ukr_Cyrl name: MTEB NTREXBitextMining (pol_Latn-ukr_Cyrl) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 8.512769153730595 - type: f1 value: 5.296718542027111 - type: main_score value: 5.296718542027111 - type: precision value: 4.650127564300294 - type: recall value: 8.512769153730595 task: type: BitextMining - dataset: config: pol_Latn-vie_Latn name: MTEB NTREXBitextMining (pol_Latn-vie_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 30.14521782674011 - type: f1 value: 23.61573142082293 - type: main_score value: 23.61573142082293 - type: precision value: 21.67325222681889 - type: recall value: 30.14521782674011 task: type: BitextMining - dataset: config: pol_Latn-zho_Hant name: MTEB NTREXBitextMining (pol_Latn-zho_Hant) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 24.036054081121684 - type: f1 value: 17.92926410467618 - type: main_score value: 17.92926410467618 - type: precision value: 16.264283236205312 - type: recall value: 24.036054081121684 task: type: BitextMining - dataset: config: pol_Latn-zul_Latn name: MTEB NTREXBitextMining (pol_Latn-zul_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 23.83575363044567 - type: f1 value: 16.400739962706364 - type: main_score value: 16.400739962706364 - type: precision value: 14.601613184463366 - type: recall value: 23.83575363044567 task: type: BitextMining - dataset: config: por_Latn-pol_Latn name: MTEB NTREXBitextMining (por_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 37.40610916374562 - type: f1 value: 33.819136027629646 - type: main_score value: 33.819136027629646 - type: precision value: 32.675346971067114 - type: recall value: 37.40610916374562 task: type: BitextMining - dataset: config: rus_Cyrl-pol_Latn name: MTEB NTREXBitextMining (rus_Cyrl-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 6.609914872308463 - type: f1 value: 5.344181532448552 - type: main_score value: 5.344181532448552 - type: precision value: 5.016035805663429 - type: recall value: 6.609914872308463 task: type: BitextMining - dataset: config: slk_Latn-pol_Latn name: MTEB NTREXBitextMining (slk_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 43.21482223335003 - type: f1 value: 38.594026070011935 - type: main_score value: 38.594026070011935 - type: precision value: 37.059186258663104 - type: recall value: 43.21482223335003 task: type: BitextMining - dataset: config: slv_Latn-pol_Latn name: MTEB NTREXBitextMining (slv_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 34.501752628943414 - type: f1 value: 30.335345420429487 - type: main_score value: 30.335345420429487 - type: precision value: 28.9518422565815 - type: recall value: 34.501752628943414 task: type: BitextMining - dataset: config: spa_Latn-pol_Latn name: MTEB NTREXBitextMining (spa_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 37.656484727090636 - type: f1 value: 33.980789524776114 - type: main_score value: 33.980789524776114 - type: precision value: 32.90989855225763 - type: recall value: 37.656484727090636 task: type: BitextMining - dataset: config: srp_Cyrl-pol_Latn name: MTEB NTREXBitextMining (srp_Cyrl-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 5.808713069604407 - type: f1 value: 4.451889762512088 - type: main_score value: 4.451889762512088 - type: precision value: 4.125584144733145 - type: recall value: 5.808713069604407 task: type: BitextMining - dataset: config: srp_Latn-pol_Latn name: MTEB NTREXBitextMining (srp_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 28.14221331997997 - type: f1 value: 23.51959297831865 - type: main_score value: 23.51959297831865 - type: precision value: 22.110666143488064 - type: recall value: 28.14221331997997 task: type: BitextMining - dataset: config: swa_Latn-pol_Latn name: MTEB NTREXBitextMining (swa_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 19.779669504256383 - type: f1 value: 16.604335328648055 - type: main_score value: 16.604335328648055 - type: precision value: 15.768002918453657 - type: recall value: 19.779669504256383 task: type: BitextMining - dataset: config: swe_Latn-pol_Latn name: MTEB NTREXBitextMining (swe_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 40.86129193790686 - type: f1 value: 36.23109651947736 - type: main_score value: 36.23109651947736 - type: precision value: 34.74497479993283 - type: recall value: 40.86129193790686 task: type: BitextMining - dataset: config: tam_Taml-pol_Latn name: MTEB NTREXBitextMining (tam_Taml-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 5.107661492238358 - type: f1 value: 3.7687687480859795 - type: main_score value: 3.7687687480859795 - type: precision value: 3.4356832920668214 - type: recall value: 5.107661492238358 task: type: BitextMining - dataset: config: tur_Latn-pol_Latn name: MTEB NTREXBitextMining (tur_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 29.944917376064094 - type: f1 value: 24.863268061783938 - type: main_score value: 24.863268061783938 - type: precision value: 23.36063980331642 - type: recall value: 29.944917376064094 task: type: BitextMining - dataset: config: ukr_Cyrl-pol_Latn name: MTEB NTREXBitextMining (ukr_Cyrl-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 7.260891337005508 - type: f1 value: 5.704027989612987 - type: main_score value: 5.704027989612987 - type: precision value: 5.2703090950782165 - type: recall value: 7.260891337005508 task: type: BitextMining - dataset: config: vie_Latn-pol_Latn name: MTEB NTREXBitextMining (vie_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 29.093640460691038 - type: f1 value: 24.651820126812275 - type: main_score value: 24.651820126812275 - type: precision value: 23.25686506040681 - type: recall value: 29.093640460691038 task: type: BitextMining - dataset: config: zho_Hant-pol_Latn name: MTEB NTREXBitextMining (zho_Hant-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 21.732598898347522 - type: f1 value: 18.829299781805904 - type: main_score value: 18.829299781805904 - type: precision value: 17.89806400969999 - type: recall value: 21.732598898347522 task: type: BitextMining - dataset: config: zul_Latn-pol_Latn name: MTEB NTREXBitextMining (zul_Latn-pol_Latn) revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33 split: test type: mteb/NTREX metrics: - type: accuracy value: 23.084626940410615 - type: f1 value: 19.616655275302847 - type: main_score value: 19.616655275302847 - type: precision value: 18.61171758782464 - type: recall value: 23.084626940410615 task: type: BitextMining - dataset: config: default name: MTEB PAC (default) revision: fc69d1c153a8ccdcf1eef52f4e2a27f88782f543 split: test type: laugustyniak/abusive-clauses-pl metrics: - type: accuracy value: 64.98696785403996 - type: ap value: 75.86324311231046 - type: ap_weighted value: 75.86324311231046 - type: f1 value: 62.65274332767454 - type: f1_weighted value: 65.4625863993904 - type: main_score value: 64.98696785403996 task: type: Classification - dataset: config: default name: MTEB PSC (default) revision: d05a294af9e1d3ff2bfb6b714e08a24a6cabc669 split: test type: PL-MTEB/psc-pairclassification metrics: - type: cosine_accuracy value: 97.86641929499072 - type: cosine_accuracy_threshold value: 62.92417049407959 - type: cosine_ap value: 99.19415840133571 - type: cosine_f1 value: 96.52042360060513 - type: cosine_f1_threshold value: 62.92417049407959 - type: cosine_precision value: 95.7957957957958 - type: cosine_recall value: 97.2560975609756 - type: dot_accuracy value: 97.86641929499072 - type: dot_accuracy_threshold value: 62.92417645454407 - type: dot_ap value: 99.19415840133571 - type: dot_f1 value: 96.52042360060513 - type: dot_f1_threshold value: 62.92417645454407 - type: dot_precision value: 95.7957957957958 - type: dot_recall value: 97.2560975609756 - type: euclidean_accuracy value: 97.86641929499072 - type: euclidean_accuracy_threshold value: 86.11080646514893 - type: euclidean_ap value: 99.19415840133571 - type: euclidean_f1 value: 96.52042360060513 - type: euclidean_f1_threshold value: 86.11080646514893 - type: euclidean_precision value: 95.7957957957958 - type: euclidean_recall value: 97.2560975609756 - type: main_score value: 99.20490128975061 - type: manhattan_accuracy value: 97.6808905380334 - type: manhattan_accuracy_threshold value: 1888.6344909667969 - type: manhattan_ap value: 99.20490128975061 - type: manhattan_f1 value: 96.2178517397882 - type: manhattan_f1_threshold value: 1888.6344909667969 - type: manhattan_precision value: 95.4954954954955 - type: manhattan_recall value: 96.95121951219512 - type: max_accuracy value: 97.86641929499072 - type: max_ap value: 99.20490128975061 - type: max_f1 value: 96.52042360060513 - type: max_precision value: 95.7957957957958 - type: max_recall value: 97.2560975609756 - type: similarity_accuracy value: 97.86641929499072 - type: similarity_accuracy_threshold value: 62.924182415008545 - type: similarity_ap value: 99.19415840133571 - type: similarity_f1 value: 96.52042360060513 - type: similarity_f1_threshold value: 62.924182415008545 - type: similarity_precision value: 95.7957957957958 - type: similarity_recall value: 97.2560975609756 task: type: PairClassification - dataset: config: default name: MTEB PlscClusteringP2P.v2 (default) revision: 8436dd4c05222778013d6642ee2f3fa1722bca9b split: test type: PL-MTEB/plsc-clustering-p2p metrics: - type: main_score value: 71.31966022242881 - type: v_measure value: 71.31966022242881 - type: v_measure_std value: 0.8738571211439499 task: type: Clustering - dataset: config: default name: MTEB PlscClusteringS2S.v2 (default) revision: 39bcadbac6b1eddad7c1a0a176119ce58060289a split: test type: PL-MTEB/plsc-clustering-s2s metrics: - type: main_score value: 70.2396182092355 - type: v_measure value: 70.2396182092355 - type: v_measure_std value: 1.8000040105133557 task: type: Clustering - dataset: config: default name: MTEB PolEmo2.0-IN (default) revision: d90724373c70959f17d2331ad51fb60c71176b03 split: test type: PL-MTEB/polemo2_in metrics: - type: accuracy value: 77.06371191135733 - type: f1 value: 75.24924358433623 - type: f1_weighted value: 77.67381875190982 - type: main_score value: 77.06371191135733 task: type: Classification - dataset: config: default name: MTEB PolEmo2.0-OUT (default) revision: 6a21ab8716e255ab1867265f8b396105e8aa63d4 split: test type: PL-MTEB/polemo2_out metrics: - type: accuracy value: 51.1336032388664 - type: f1 value: 43.593636081086586 - type: f1_weighted value: 58.20039598624505 - type: main_score value: 51.1336032388664 task: type: Classification - dataset: config: default name: MTEB PPC (default) revision: 2c7d2df57801a591f6b1e3aaf042e7a04ec7d9f2 split: test type: PL-MTEB/ppc-pairclassification metrics: - type: cosine_accuracy value: 87.2 - type: cosine_accuracy_threshold value: 85.1994276046753 - type: cosine_ap value: 92.14410059169685 - type: cosine_f1 value: 89.84126984126983 - type: cosine_f1_threshold value: 85.01211404800415 - type: cosine_precision value: 86.28048780487805 - type: cosine_recall value: 93.70860927152319 - type: dot_accuracy value: 87.2 - type: dot_accuracy_threshold value: 85.1994276046753 - type: dot_ap value: 92.14410059169685 - type: dot_f1 value: 89.84126984126983 - type: dot_f1_threshold value: 85.0121259689331 - type: dot_precision value: 86.28048780487805 - type: dot_recall value: 93.70860927152319 - type: euclidean_accuracy value: 87.2 - type: euclidean_accuracy_threshold value: 54.40692901611328 - type: euclidean_ap value: 92.14410059169685 - type: euclidean_f1 value: 89.84126984126983 - type: euclidean_f1_threshold value: 54.75010871887207 - type: euclidean_precision value: 86.28048780487805 - type: euclidean_recall value: 93.70860927152319 - type: main_score value: 92.1546894976679 - type: manhattan_accuracy value: 87.3 - type: manhattan_accuracy_threshold value: 1165.2369499206543 - type: manhattan_ap value: 92.1546894976679 - type: manhattan_f1 value: 89.81555733761026 - type: manhattan_f1_threshold value: 1165.2369499206543 - type: manhattan_precision value: 87.09175738724728 - type: manhattan_recall value: 92.71523178807946 - type: max_accuracy value: 87.3 - type: max_ap value: 92.1546894976679 - type: max_f1 value: 89.84126984126983 - type: max_precision value: 87.09175738724728 - type: max_recall value: 93.70860927152319 - type: similarity_accuracy value: 87.2 - type: similarity_accuracy_threshold value: 85.19943356513977 - type: similarity_ap value: 92.14410059169685 - type: similarity_f1 value: 89.84126984126983 - type: similarity_f1_threshold value: 85.0121259689331 - type: similarity_precision value: 86.28048780487805 - type: similarity_recall value: 93.70860927152319 task: type: PairClassification - dataset: config: default name: MTEB Quora-PL (default) revision: 0be27e93455051e531182b85e85e425aba12e9d4 split: test type: clarin-knext/quora-pl metrics: - type: main_score value: 83.875 - type: map_at_1 value: 65.774 - type: map_at_10 value: 79.594 - type: map_at_100 value: 80.336 - type: map_at_1000 value: 80.35799999999999 - type: map_at_20 value: 80.081 - type: map_at_3 value: 76.543 - type: map_at_5 value: 78.45400000000001 - type: mrr_at_1 value: 75.76 - type: mrr_at_10 value: 82.84497222222222 - type: mrr_at_100 value: 83.02868118414814 - type: mrr_at_1000 value: 83.03198341669084 - type: mrr_at_20 value: 82.97830966853064 - type: mrr_at_3 value: 81.605 - type: mrr_at_5 value: 82.4165 - type: nauc_map_at_1000_diff1 value: 73.40640121058422 - type: nauc_map_at_1000_max value: 28.46337709715841 - type: nauc_map_at_1000_std value: -34.36170797178319 - type: nauc_map_at_100_diff1 value: 73.41027935468242 - type: nauc_map_at_100_max value: 28.448897737305167 - type: nauc_map_at_100_std value: -34.410053175618984 - type: nauc_map_at_10_diff1 value: 73.63608438663765 - type: nauc_map_at_10_max value: 27.99805524703715 - type: nauc_map_at_10_std value: -35.96926536472381 - type: nauc_map_at_1_diff1 value: 77.37739722887241 - type: nauc_map_at_1_max value: 20.489751324646726 - type: nauc_map_at_1_std value: -34.43345176075925 - type: nauc_map_at_20_diff1 value: 73.48489056640469 - type: nauc_map_at_20_max value: 28.302520798061405 - type: nauc_map_at_20_std value: -35.033039709762456 - type: nauc_map_at_3_diff1 value: 74.22725361971708 - type: nauc_map_at_3_max value: 25.81523287150949 - type: nauc_map_at_3_std value: -37.576947757072034 - type: nauc_map_at_5_diff1 value: 73.82247830220571 - type: nauc_map_at_5_max value: 27.097568871055334 - type: nauc_map_at_5_std value: -37.178201184627305 - type: nauc_mrr_at_1000_diff1 value: 73.671442290651 - type: nauc_mrr_at_1000_max value: 30.524947345269105 - type: nauc_mrr_at_1000_std value: -31.36532587671011 - type: nauc_mrr_at_100_diff1 value: 73.66992200638639 - type: nauc_mrr_at_100_max value: 30.52663991152657 - type: nauc_mrr_at_100_std value: -31.359539680268337 - type: nauc_mrr_at_10_diff1 value: 73.63189061102112 - type: nauc_mrr_at_10_max value: 30.606965655536673 - type: nauc_mrr_at_10_std value: -31.411534218742432 - type: nauc_mrr_at_1_diff1 value: 75.17418724066114 - type: nauc_mrr_at_1_max value: 29.14884092629756 - type: nauc_mrr_at_1_std value: -31.16093210023857 - type: nauc_mrr_at_20_diff1 value: 73.65231966885165 - type: nauc_mrr_at_20_max value: 30.583763761309974 - type: nauc_mrr_at_20_std value: -31.337398441591773 - type: nauc_mrr_at_3_diff1 value: 73.3913556687992 - type: nauc_mrr_at_3_max value: 30.476618323576627 - type: nauc_mrr_at_3_std value: -31.858863133453397 - type: nauc_mrr_at_5_diff1 value: 73.4716463702077 - type: nauc_mrr_at_5_max value: 30.61795159461716 - type: nauc_mrr_at_5_std value: -31.563308058631534 - type: nauc_ndcg_at_1000_diff1 value: 72.95508712773197 - type: nauc_ndcg_at_1000_max value: 30.12028043393866 - type: nauc_ndcg_at_1000_std value: -31.90016377526389 - type: nauc_ndcg_at_100_diff1 value: 72.91161449349684 - type: nauc_ndcg_at_100_max value: 30.1573165397239 - type: nauc_ndcg_at_100_std value: -31.898114473155537 - type: nauc_ndcg_at_10_diff1 value: 72.98440999465589 - type: nauc_ndcg_at_10_max value: 29.498984798918926 - type: nauc_ndcg_at_10_std value: -34.409963346107 - type: nauc_ndcg_at_1_diff1 value: 74.96338647049875 - type: nauc_ndcg_at_1_max value: 29.69806164680575 - type: nauc_ndcg_at_1_std value: -30.506573641100786 - type: nauc_ndcg_at_20_diff1 value: 72.94194704227928 - type: nauc_ndcg_at_20_max value: 29.914187952525413 - type: nauc_ndcg_at_20_std value: -33.164979047878795 - type: nauc_ndcg_at_3_diff1 value: 72.40928680360435 - type: nauc_ndcg_at_3_max value: 28.217746151291877 - type: nauc_ndcg_at_3_std value: -34.5876406089579 - type: nauc_ndcg_at_5_diff1 value: 72.65286836503851 - type: nauc_ndcg_at_5_max value: 28.762447019502797 - type: nauc_ndcg_at_5_std value: -35.11764039207373 - type: nauc_precision_at_1000_diff1 value: -41.68149562579916 - type: nauc_precision_at_1000_max value: 0.5346848775768239 - type: nauc_precision_at_1000_std value: 37.63984778174331 - type: nauc_precision_at_100_diff1 value: -40.70814450989262 - type: nauc_precision_at_100_max value: 1.7957963726862223 - type: nauc_precision_at_100_std value: 36.681132193570996 - type: nauc_precision_at_10_diff1 value: -30.413343518298632 - type: nauc_precision_at_10_max value: 7.5727147760066575 - type: nauc_precision_at_10_std value: 22.74763162315844 - type: nauc_precision_at_1_diff1 value: 74.96338647049875 - type: nauc_precision_at_1_max value: 29.69806164680575 - type: nauc_precision_at_1_std value: -30.506573641100786 - type: nauc_precision_at_20_diff1 value: -36.12470932128835 - type: nauc_precision_at_20_max value: 4.897816846407051 - type: nauc_precision_at_20_std value: 29.979051739488423 - type: nauc_precision_at_3_diff1 value: -5.353214272695143 - type: nauc_precision_at_3_max value: 15.194981772023484 - type: nauc_precision_at_3_std value: 2.8778017019216726 - type: nauc_precision_at_5_diff1 value: -19.967483442538388 - type: nauc_precision_at_5_max value: 11.178726211951638 - type: nauc_precision_at_5_std value: 12.462110198882302 - type: nauc_recall_at_1000_diff1 value: 54.20555830597288 - type: nauc_recall_at_1000_max value: 54.00511659799212 - type: nauc_recall_at_1000_std value: 43.666835088245136 - type: nauc_recall_at_100_diff1 value: 61.715707167844926 - type: nauc_recall_at_100_max value: 39.33537383712575 - type: nauc_recall_at_100_std value: -5.81121276399585 - type: nauc_recall_at_10_diff1 value: 67.46913802915881 - type: nauc_recall_at_10_max value: 29.316426634098924 - type: nauc_recall_at_10_std value: -39.30499385618004 - type: nauc_recall_at_1_diff1 value: 77.37739722887241 - type: nauc_recall_at_1_max value: 20.489751324646726 - type: nauc_recall_at_1_std value: -34.43345176075925 - type: nauc_recall_at_20_diff1 value: 66.28280245365029 - type: nauc_recall_at_20_max value: 32.71950916379278 - type: nauc_recall_at_20_std value: -31.233343041727274 - type: nauc_recall_at_3_diff1 value: 70.28676363121481 - type: nauc_recall_at_3_max value: 24.287271486292788 - type: nauc_recall_at_3_std value: -40.82142827677811 - type: nauc_recall_at_5_diff1 value: 68.8387400353206 - type: nauc_recall_at_5_max value: 26.453954769696157 - type: nauc_recall_at_5_std value: -41.642567850497706 - type: ndcg_at_1 value: 75.86 - type: ndcg_at_10 value: 83.875 - type: ndcg_at_100 value: 85.596 - type: ndcg_at_1000 value: 85.804 - type: ndcg_at_20 value: 84.789 - type: ndcg_at_3 value: 80.548 - type: ndcg_at_5 value: 82.352 - type: precision_at_1 value: 75.86 - type: precision_at_10 value: 12.867 - type: precision_at_100 value: 1.505 - type: precision_at_1000 value: 0.156 - type: precision_at_20 value: 6.909999999999999 - type: precision_at_3 value: 35.4 - type: precision_at_5 value: 23.43 - type: recall_at_1 value: 65.774 - type: recall_at_10 value: 92.42099999999999 - type: recall_at_100 value: 98.68400000000001 - type: recall_at_1000 value: 99.82 - type: recall_at_20 value: 95.43700000000001 - type: recall_at_3 value: 82.92 - type: recall_at_5 value: 87.89 task: type: Retrieval - dataset: config: default name: MTEB Quora-PL (default) revision: 0be27e93455051e531182b85e85e425aba12e9d4 split: validation type: clarin-knext/quora-pl metrics: - type: main_score value: 83.78999999999999 - type: map_at_1 value: 65.527 - type: map_at_10 value: 79.54299999999999 - type: map_at_100 value: 80.215 - type: map_at_1000 value: 80.23599999999999 - type: map_at_20 value: 80.009 - type: map_at_3 value: 76.375 - type: map_at_5 value: 78.361 - type: mrr_at_1 value: 75.38 - type: mrr_at_10 value: 82.58898412698412 - type: mrr_at_100 value: 82.78276550796194 - type: mrr_at_1000 value: 82.78548907447124 - type: mrr_at_20 value: 82.72455061390276 - type: mrr_at_3 value: 81.29333333333332 - type: mrr_at_5 value: 82.16133333333333 - type: nauc_map_at_1000_diff1 value: 72.17082965966746 - type: nauc_map_at_1000_max value: 25.561957524983104 - type: nauc_map_at_1000_std value: -32.729084893747775 - type: nauc_map_at_100_diff1 value: 72.18145209742738 - type: nauc_map_at_100_max value: 25.562273072340062 - type: nauc_map_at_100_std value: -32.75499194018571 - type: nauc_map_at_10_diff1 value: 72.17540509768051 - type: nauc_map_at_10_max value: 25.23028091664771 - type: nauc_map_at_10_std value: -33.80938822847208 - type: nauc_map_at_1_diff1 value: 76.85902162952067 - type: nauc_map_at_1_max value: 15.119432901181101 - type: nauc_map_at_1_std value: -33.215413587802175 - type: nauc_map_at_20_diff1 value: 72.20269126012819 - type: nauc_map_at_20_max value: 25.463663870881515 - type: nauc_map_at_20_std value: -33.171428047416875 - type: nauc_map_at_3_diff1 value: 72.96664356556279 - type: nauc_map_at_3_max value: 22.83884443984863 - type: nauc_map_at_3_std value: -35.67073538019632 - type: nauc_map_at_5_diff1 value: 72.29231438141625 - type: nauc_map_at_5_max value: 24.39299322090406 - type: nauc_map_at_5_std value: -34.8848651530573 - type: nauc_mrr_at_1000_diff1 value: 72.36083495768207 - type: nauc_mrr_at_1000_max value: 27.29785049455846 - type: nauc_mrr_at_1000_std value: -29.413065947303934 - type: nauc_mrr_at_100_diff1 value: 72.36147319908831 - type: nauc_mrr_at_100_max value: 27.30253888986283 - type: nauc_mrr_at_100_std value: -29.407204355739324 - type: nauc_mrr_at_10_diff1 value: 72.29190315984265 - type: nauc_mrr_at_10_max value: 27.391173320996902 - type: nauc_mrr_at_10_std value: -29.425675339523078 - type: nauc_mrr_at_1_diff1 value: 73.87822190537509 - type: nauc_mrr_at_1_max value: 23.821278424587756 - type: nauc_mrr_at_1_std value: -30.73829013582936 - type: nauc_mrr_at_20_diff1 value: 72.34287269332268 - type: nauc_mrr_at_20_max value: 27.36468207313298 - type: nauc_mrr_at_20_std value: -29.397399644114923 - type: nauc_mrr_at_3_diff1 value: 72.07642967834977 - type: nauc_mrr_at_3_max value: 27.938085395784988 - type: nauc_mrr_at_3_std value: -29.747194738371384 - type: nauc_mrr_at_5_diff1 value: 72.06309211487309 - type: nauc_mrr_at_5_max value: 27.560904443136664 - type: nauc_mrr_at_5_std value: -29.39113946238272 - type: nauc_ndcg_at_1000_diff1 value: 71.7974375201558 - type: nauc_ndcg_at_1000_max value: 27.297077258449914 - type: nauc_ndcg_at_1000_std value: -30.166803695034446 - type: nauc_ndcg_at_100_diff1 value: 71.90182708721001 - type: nauc_ndcg_at_100_max value: 27.463368073082243 - type: nauc_ndcg_at_100_std value: -30.173059997756113 - type: nauc_ndcg_at_10_diff1 value: 71.4971649738977 - type: nauc_ndcg_at_10_max value: 27.33895888686314 - type: nauc_ndcg_at_10_std value: -32.149121259739296 - type: nauc_ndcg_at_1_diff1 value: 73.79341779758884 - type: nauc_ndcg_at_1_max value: 24.07575045739801 - type: nauc_ndcg_at_1_std value: -30.569253092715094 - type: nauc_ndcg_at_20_diff1 value: 71.83476165120113 - type: nauc_ndcg_at_20_max value: 27.484988191438763 - type: nauc_ndcg_at_20_std value: -31.18933505569125 - type: nauc_ndcg_at_3_diff1 value: 71.3211324267825 - type: nauc_ndcg_at_3_max value: 26.3827563071973 - type: nauc_ndcg_at_3_std value: -33.26564798575043 - type: nauc_ndcg_at_5_diff1 value: 71.13111409439679 - type: nauc_ndcg_at_5_max value: 26.899174537293668 - type: nauc_ndcg_at_5_std value: -33.037279714247596 - type: nauc_precision_at_1000_diff1 value: -41.06260416535236 - type: nauc_precision_at_1000_max value: 4.065296262607365 - type: nauc_precision_at_1000_std value: 32.664889523073235 - type: nauc_precision_at_100_diff1 value: -39.46417965752645 - type: nauc_precision_at_100_max value: 5.845231054532795 - type: nauc_precision_at_100_std value: 32.267579356245015 - type: nauc_precision_at_10_diff1 value: -30.398862391847853 - type: nauc_precision_at_10_max value: 11.366479364106302 - type: nauc_precision_at_10_std value: 20.905381134697873 - type: nauc_precision_at_1_diff1 value: 73.79341779758884 - type: nauc_precision_at_1_max value: 24.07575045739801 - type: nauc_precision_at_1_std value: -30.569253092715094 - type: nauc_precision_at_20_diff1 value: -34.95246979130432 - type: nauc_precision_at_20_max value: 8.471127128333388 - type: nauc_precision_at_20_std value: 26.200816731005723 - type: nauc_precision_at_3_diff1 value: -6.0507685352298 - type: nauc_precision_at_3_max value: 18.181944109090693 - type: nauc_precision_at_3_std value: 2.4766181822784477 - type: nauc_precision_at_5_diff1 value: -20.921638552869315 - type: nauc_precision_at_5_max value: 15.093648854992317 - type: nauc_precision_at_5_std value: 12.390250332801731 - type: nauc_recall_at_1000_diff1 value: 72.76959786511301 - type: nauc_recall_at_1000_max value: 61.37728666951695 - type: nauc_recall_at_1000_std value: 38.29139577477708 - type: nauc_recall_at_100_diff1 value: 73.26264132727238 - type: nauc_recall_at_100_max value: 43.79687261540802 - type: nauc_recall_at_100_std value: -11.430272905442566 - type: nauc_recall_at_10_diff1 value: 65.56843657357662 - type: nauc_recall_at_10_max value: 30.859225864152133 - type: nauc_recall_at_10_std value: -34.881586204916495 - type: nauc_recall_at_1_diff1 value: 76.85902162952067 - type: nauc_recall_at_1_max value: 15.119432901181101 - type: nauc_recall_at_1_std value: -33.215413587802175 - type: nauc_recall_at_20_diff1 value: 67.37834715400818 - type: nauc_recall_at_20_max value: 35.213982982419374 - type: nauc_recall_at_20_std value: -29.688579231776913 - type: nauc_recall_at_3_diff1 value: 68.79765755095777 - type: nauc_recall_at_3_max value: 25.26409330029704 - type: nauc_recall_at_3_std value: -37.49808566043117 - type: nauc_recall_at_5_diff1 value: 66.41529573662208 - type: nauc_recall_at_5_max value: 27.814159729907644 - type: nauc_recall_at_5_std value: -37.02337127906464 - type: ndcg_at_1 value: 75.42 - type: ndcg_at_10 value: 83.78999999999999 - type: ndcg_at_100 value: 85.45700000000001 - type: ndcg_at_1000 value: 85.661 - type: ndcg_at_20 value: 84.711 - type: ndcg_at_3 value: 80.342 - type: ndcg_at_5 value: 82.203 - type: precision_at_1 value: 75.42 - type: precision_at_10 value: 12.848 - type: precision_at_100 value: 1.469 - type: precision_at_1000 value: 0.152 - type: precision_at_20 value: 6.837999999999999 - type: precision_at_3 value: 35.32 - type: precision_at_5 value: 23.427999999999997 - type: recall_at_1 value: 65.527 - type: recall_at_10 value: 92.547 - type: recall_at_100 value: 98.722 - type: recall_at_1000 value: 99.833 - type: recall_at_20 value: 95.636 - type: recall_at_3 value: 82.798 - type: recall_at_5 value: 87.932 task: type: Retrieval - dataset: config: default name: MTEB Quora-PLHardNegatives (default) revision: 523ff30f3346cd9c36081c19fc6eaef0a2f8d53d split: test type: mteb/Quora_PL_test_top_250_only_w_correct-v2 metrics: - type: main_score value: 83.441 - type: map_at_1 value: 64.566 - type: map_at_10 value: 78.976 - type: map_at_100 value: 79.784 - type: map_at_1000 value: 79.805 - type: map_at_20 value: 79.473 - type: map_at_3 value: 75.60900000000001 - type: map_at_5 value: 77.59700000000001 - type: mrr_at_1 value: 75.0 - type: mrr_at_10 value: 82.24003968253967 - type: mrr_at_100 value: 82.39148676909636 - type: mrr_at_1000 value: 82.39412953563897 - type: mrr_at_20 value: 82.3380424924233 - type: mrr_at_3 value: 80.81666666666666 - type: mrr_at_5 value: 81.76666666666667 - type: nauc_map_at_1000_diff1 value: 70.11947044713457 - type: nauc_map_at_1000_max value: 32.0785962251563 - type: nauc_map_at_1000_std value: -25.72180310228886 - type: nauc_map_at_100_diff1 value: 70.119223903828 - type: nauc_map_at_100_max value: 32.04902408982848 - type: nauc_map_at_100_std value: -25.741933094347818 - type: nauc_map_at_10_diff1 value: 70.24873166549017 - type: nauc_map_at_10_max value: 32.01344077405822 - type: nauc_map_at_10_std value: -26.352879248313887 - type: nauc_map_at_1_diff1 value: 76.3838359798031 - type: nauc_map_at_1_max value: 19.687227608021665 - type: nauc_map_at_1_std value: -27.80404785623863 - type: nauc_map_at_20_diff1 value: 70.11742272646482 - type: nauc_map_at_20_max value: 31.974908565557914 - type: nauc_map_at_20_std value: -26.055873277823867 - type: nauc_map_at_3_diff1 value: 71.38775037924422 - type: nauc_map_at_3_max value: 29.347110576500025 - type: nauc_map_at_3_std value: -27.546917264739633 - type: nauc_map_at_5_diff1 value: 70.47666721538849 - type: nauc_map_at_5_max value: 30.391213537368557 - type: nauc_map_at_5_std value: -27.469926373290765 - type: nauc_mrr_at_1000_diff1 value: 70.6739515891783 - type: nauc_mrr_at_1000_max value: 35.73696455413425 - type: nauc_mrr_at_1000_std value: -24.903543941487516 - type: nauc_mrr_at_100_diff1 value: 70.6727011010652 - type: nauc_mrr_at_100_max value: 35.7441879616733 - type: nauc_mrr_at_100_std value: -24.893145024514293 - type: nauc_mrr_at_10_diff1 value: 70.64421848648517 - type: nauc_mrr_at_10_max value: 35.737645000885344 - type: nauc_mrr_at_10_std value: -24.91053298763681 - type: nauc_mrr_at_1_diff1 value: 72.96054421768706 - type: nauc_mrr_at_1_max value: 33.44908424908425 - type: nauc_mrr_at_1_std value: -27.055782312925196 - type: nauc_mrr_at_20_diff1 value: 70.65109502016541 - type: nauc_mrr_at_20_max value: 35.75121651931555 - type: nauc_mrr_at_20_std value: -24.935827426097262 - type: nauc_mrr_at_3_diff1 value: 70.46651990006681 - type: nauc_mrr_at_3_max value: 36.82409033566682 - type: nauc_mrr_at_3_std value: -23.77252296025634 - type: nauc_mrr_at_5_diff1 value: 70.37584561193329 - type: nauc_mrr_at_5_max value: 35.95184767311965 - type: nauc_mrr_at_5_std value: -24.46407294516173 - type: nauc_ndcg_at_1000_diff1 value: 69.78988888464094 - type: nauc_ndcg_at_1000_max value: 33.78633223806749 - type: nauc_ndcg_at_1000_std value: -24.30471382823556 - type: nauc_ndcg_at_100_diff1 value: 69.68080255594029 - type: nauc_ndcg_at_100_max value: 33.75329157895577 - type: nauc_ndcg_at_100_std value: -23.950742923771408 - type: nauc_ndcg_at_10_diff1 value: 69.32410336421049 - type: nauc_ndcg_at_10_max value: 32.62171559106042 - type: nauc_ndcg_at_10_std value: -25.055305262714896 - type: nauc_ndcg_at_1_diff1 value: 73.79808983795444 - type: nauc_ndcg_at_1_max value: 31.11741586732179 - type: nauc_ndcg_at_1_std value: -26.49182333274088 - type: nauc_ndcg_at_20_diff1 value: 69.28319519809568 - type: nauc_ndcg_at_20_max value: 32.649729085506685 - type: nauc_ndcg_at_20_std value: -24.78286422887768 - type: nauc_ndcg_at_3_diff1 value: 69.46374350677924 - type: nauc_ndcg_at_3_max value: 33.48691707083252 - type: nauc_ndcg_at_3_std value: -23.258882920352097 - type: nauc_ndcg_at_5_diff1 value: 69.15940341139341 - type: nauc_ndcg_at_5_max value: 32.38283752475865 - type: nauc_ndcg_at_5_std value: -25.331353469618236 - type: nauc_precision_at_1000_diff1 value: -39.89797626190511 - type: nauc_precision_at_1000_max value: 0.6876951931266619 - type: nauc_precision_at_1000_std value: 24.135579123642 - type: nauc_precision_at_100_diff1 value: -39.27741905918931 - type: nauc_precision_at_100_max value: 1.5295807101881045 - type: nauc_precision_at_100_std value: 23.73873597896803 - type: nauc_precision_at_10_diff1 value: -31.95569799072481 - type: nauc_precision_at_10_max value: 11.133351075606681 - type: nauc_precision_at_10_std value: 18.12306130253635 - type: nauc_precision_at_1_diff1 value: 73.79808983795444 - type: nauc_precision_at_1_max value: 31.11741586732179 - type: nauc_precision_at_1_std value: -26.49182333274088 - type: nauc_precision_at_20_diff1 value: -35.86837967186013 - type: nauc_precision_at_20_max value: 5.146168726202043 - type: nauc_precision_at_20_std value: 20.394902558258444 - type: nauc_precision_at_3_diff1 value: -7.123794765840237 - type: nauc_precision_at_3_max value: 23.732805023923436 - type: nauc_precision_at_3_std value: 6.118602290851066 - type: nauc_precision_at_5_diff1 value: -22.353212199080236 - type: nauc_precision_at_5_max value: 16.87754190425444 - type: nauc_precision_at_5_std value: 12.493503307109334 - type: nauc_recall_at_1000_diff1 value: 66.2401683837391 - type: nauc_recall_at_1000_max value: 69.89586795169929 - type: nauc_recall_at_1000_std value: 54.31274718413008 - type: nauc_recall_at_100_diff1 value: 52.18342869491622 - type: nauc_recall_at_100_max value: 45.26570476940006 - type: nauc_recall_at_100_std value: 28.11023427839046 - type: nauc_recall_at_10_diff1 value: 59.84335909161517 - type: nauc_recall_at_10_max value: 27.875640239194 - type: nauc_recall_at_10_std value: -22.99606199719549 - type: nauc_recall_at_1_diff1 value: 76.3838359798031 - type: nauc_recall_at_1_max value: 19.687227608021665 - type: nauc_recall_at_1_std value: -27.80404785623863 - type: nauc_recall_at_20_diff1 value: 57.20155639145018 - type: nauc_recall_at_20_max value: 23.36552345698421 - type: nauc_recall_at_20_std value: -20.66577074515514 - type: nauc_recall_at_3_diff1 value: 66.70593366498305 - type: nauc_recall_at_3_max value: 29.49668171631047 - type: nauc_recall_at_3_std value: -23.511735151015237 - type: nauc_recall_at_5_diff1 value: 62.801105437887806 - type: nauc_recall_at_5_max value: 28.681608337339814 - type: nauc_recall_at_5_std value: -26.076742308083222 - type: ndcg_at_1 value: 74.6 - type: ndcg_at_10 value: 83.441 - type: ndcg_at_100 value: 85.207 - type: ndcg_at_1000 value: 85.39099999999999 - type: ndcg_at_20 value: 84.315 - type: ndcg_at_3 value: 79.911 - type: ndcg_at_5 value: 81.733 - type: precision_at_1 value: 74.6 - type: precision_at_10 value: 13.33 - type: precision_at_100 value: 1.585 - type: precision_at_1000 value: 0.163 - type: precision_at_20 value: 7.215000000000001 - type: precision_at_3 value: 35.6 - type: precision_at_5 value: 23.880000000000003 - type: recall_at_1 value: 64.566 - type: recall_at_10 value: 92.599 - type: recall_at_100 value: 98.846 - type: recall_at_1000 value: 99.845 - type: recall_at_20 value: 95.257 - type: recall_at_3 value: 82.077 - type: recall_at_5 value: 87.397 task: type: Retrieval - dataset: config: default name: MTEB SCIDOCS-PL (default) revision: 45452b03f05560207ef19149545f168e596c9337 split: test type: clarin-knext/scidocs-pl metrics: - type: main_score value: 15.873000000000001 - type: map_at_1 value: 3.775 - type: map_at_10 value: 9.161 - type: map_at_100 value: 10.918999999999999 - type: map_at_1000 value: 11.202 - type: map_at_20 value: 10.067 - type: map_at_3 value: 6.597 - type: map_at_5 value: 7.854 - type: mrr_at_1 value: 18.6 - type: mrr_at_10 value: 27.886428571428567 - type: mrr_at_100 value: 28.989174893729764 - type: mrr_at_1000 value: 29.06969368618742 - type: mrr_at_20 value: 28.52687026509782 - type: mrr_at_3 value: 24.733333333333334 - type: mrr_at_5 value: 26.62333333333333 - type: nauc_map_at_1000_diff1 value: 17.338063492099025 - type: nauc_map_at_1000_max value: 31.676710363869258 - type: nauc_map_at_1000_std value: 19.44787478817694 - type: nauc_map_at_100_diff1 value: 17.330001556011098 - type: nauc_map_at_100_max value: 31.529980407900478 - type: nauc_map_at_100_std value: 19.101197347850587 - type: nauc_map_at_10_diff1 value: 16.69617773011823 - type: nauc_map_at_10_max value: 29.330357636083065 - type: nauc_map_at_10_std value: 14.520443867477189 - type: nauc_map_at_1_diff1 value: 19.69959398347788 - type: nauc_map_at_1_max value: 21.15503613820104 - type: nauc_map_at_1_std value: 5.949517842252173 - type: nauc_map_at_20_diff1 value: 17.23299101990182 - type: nauc_map_at_20_max value: 30.743033737387552 - type: nauc_map_at_20_std value: 17.045152961765215 - type: nauc_map_at_3_diff1 value: 16.869461560565544 - type: nauc_map_at_3_max value: 24.702066531086228 - type: nauc_map_at_3_std value: 8.381332247849253 - type: nauc_map_at_5_diff1 value: 16.79450517482287 - type: nauc_map_at_5_max value: 26.953610672135465 - type: nauc_map_at_5_std value: 11.362691870524827 - type: nauc_mrr_at_1000_diff1 value: 17.689725068831603 - type: nauc_mrr_at_1000_max value: 24.730134223817196 - type: nauc_mrr_at_1000_std value: 12.385372746391045 - type: nauc_mrr_at_100_diff1 value: 17.69979261858343 - type: nauc_mrr_at_100_max value: 24.745016185063882 - type: nauc_mrr_at_100_std value: 12.43761305969452 - type: nauc_mrr_at_10_diff1 value: 17.790136790834275 - type: nauc_mrr_at_10_max value: 24.63646057782297 - type: nauc_mrr_at_10_std value: 12.051623774206263 - type: nauc_mrr_at_1_diff1 value: 19.694047484705795 - type: nauc_mrr_at_1_max value: 21.093009975002662 - type: nauc_mrr_at_1_std value: 5.620129192869433 - type: nauc_mrr_at_20_diff1 value: 17.663848767206282 - type: nauc_mrr_at_20_max value: 24.736258292269998 - type: nauc_mrr_at_20_std value: 12.417424425736334 - type: nauc_mrr_at_3_diff1 value: 16.741243494943962 - type: nauc_mrr_at_3_max value: 22.496607924575194 - type: nauc_mrr_at_3_std value: 8.83985463832847 - type: nauc_mrr_at_5_diff1 value: 17.453148482385096 - type: nauc_mrr_at_5_max value: 24.191411493762853 - type: nauc_mrr_at_5_std value: 11.106728198617791 - type: nauc_ndcg_at_1000_diff1 value: 17.05819787430221 - type: nauc_ndcg_at_1000_max value: 34.36473654929546 - type: nauc_ndcg_at_1000_std value: 27.84149425841954 - type: nauc_ndcg_at_100_diff1 value: 17.654089575349236 - type: nauc_ndcg_at_100_max value: 33.58798073847035 - type: nauc_ndcg_at_100_std value: 26.20805574689576 - type: nauc_ndcg_at_10_diff1 value: 17.31879632124412 - type: nauc_ndcg_at_10_max value: 30.288116141634454 - type: nauc_ndcg_at_10_std value: 17.301156501534756 - type: nauc_ndcg_at_1_diff1 value: 19.694047484705795 - type: nauc_ndcg_at_1_max value: 21.093009975002662 - type: nauc_ndcg_at_1_std value: 5.620129192869433 - type: nauc_ndcg_at_20_diff1 value: 17.594180577310784 - type: nauc_ndcg_at_20_max value: 32.11676032463241 - type: nauc_ndcg_at_20_std value: 20.9930275041836 - type: nauc_ndcg_at_3_diff1 value: 16.345054106202365 - type: nauc_ndcg_at_3_max value: 24.438635311197014 - type: nauc_ndcg_at_3_std value: 9.012466244196702 - type: nauc_ndcg_at_5_diff1 value: 16.787572813850378 - type: nauc_ndcg_at_5_max value: 27.482032734078444 - type: nauc_ndcg_at_5_std value: 13.333137895321832 - type: nauc_precision_at_1000_diff1 value: 8.803110756113522 - type: nauc_precision_at_1000_max value: 31.2791590763206 - type: nauc_precision_at_1000_std value: 38.94233269200768 - type: nauc_precision_at_100_diff1 value: 14.061557381077398 - type: nauc_precision_at_100_max value: 33.71562748261276 - type: nauc_precision_at_100_std value: 36.94329011079927 - type: nauc_precision_at_10_diff1 value: 15.856997461162498 - type: nauc_precision_at_10_max value: 32.81363590609658 - type: nauc_precision_at_10_std value: 22.416593036651 - type: nauc_precision_at_1_diff1 value: 19.694047484705795 - type: nauc_precision_at_1_max value: 21.093009975002662 - type: nauc_precision_at_1_std value: 5.620129192869433 - type: nauc_precision_at_20_diff1 value: 15.811209412149053 - type: nauc_precision_at_20_max value: 34.81747068913031 - type: nauc_precision_at_20_std value: 28.32039806351492 - type: nauc_precision_at_3_diff1 value: 15.007806096740516 - type: nauc_precision_at_3_max value: 25.467608599280595 - type: nauc_precision_at_3_std value: 9.97528298821604 - type: nauc_precision_at_5_diff1 value: 15.711050079779485 - type: nauc_precision_at_5_max value: 30.04221632785474 - type: nauc_precision_at_5_std value: 16.94260428944902 - type: nauc_recall_at_1000_diff1 value: 8.100290478639437 - type: nauc_recall_at_1000_max value: 31.18523144848299 - type: nauc_recall_at_1000_std value: 40.20433662499107 - type: nauc_recall_at_100_diff1 value: 13.661547776170554 - type: nauc_recall_at_100_max value: 33.409430597319314 - type: nauc_recall_at_100_std value: 37.290964212520926 - type: nauc_recall_at_10_diff1 value: 15.72252257570797 - type: nauc_recall_at_10_max value: 32.44746727086122 - type: nauc_recall_at_10_std value: 22.19023327223186 - type: nauc_recall_at_1_diff1 value: 19.69959398347788 - type: nauc_recall_at_1_max value: 21.15503613820104 - type: nauc_recall_at_1_std value: 5.949517842252173 - type: nauc_recall_at_20_diff1 value: 15.608371103886329 - type: nauc_recall_at_20_max value: 34.42514959297327 - type: nauc_recall_at_20_std value: 28.274144467952674 - type: nauc_recall_at_3_diff1 value: 14.911590535221745 - type: nauc_recall_at_3_max value: 25.079059447651776 - type: nauc_recall_at_3_std value: 9.9537124673317 - type: nauc_recall_at_5_diff1 value: 15.657528497044314 - type: nauc_recall_at_5_max value: 29.616816427701554 - type: nauc_recall_at_5_std value: 16.68457790000254 - type: ndcg_at_1 value: 18.6 - type: ndcg_at_10 value: 15.873000000000001 - type: ndcg_at_100 value: 23.14 - type: ndcg_at_1000 value: 28.504 - type: ndcg_at_20 value: 18.502 - type: ndcg_at_3 value: 14.97 - type: ndcg_at_5 value: 13.153 - type: precision_at_1 value: 18.6 - type: precision_at_10 value: 8.260000000000002 - type: precision_at_100 value: 1.8769999999999998 - type: precision_at_1000 value: 0.317 - type: precision_at_20 value: 5.66 - type: precision_at_3 value: 13.966999999999999 - type: precision_at_5 value: 11.559999999999999 - type: recall_at_1 value: 3.775 - type: recall_at_10 value: 16.733 - type: recall_at_100 value: 38.153 - type: recall_at_1000 value: 64.387 - type: recall_at_20 value: 22.972 - type: recall_at_3 value: 8.498 - type: recall_at_5 value: 11.723 task: type: Retrieval - dataset: config: pol_Latn name: MTEB SIB200Classification (pol_Latn) revision: a74d7350ea12af010cfb1c21e34f1f81fd2e615b split: test type: mteb/sib200 metrics: - type: accuracy value: 76.17647058823529 - type: f1 value: 75.26003745335032 - type: f1_weighted value: 76.19627183062285 - type: main_score value: 76.17647058823529 task: type: Classification - dataset: config: pol_Latn name: MTEB SIB200Classification (pol_Latn) revision: a74d7350ea12af010cfb1c21e34f1f81fd2e615b split: train type: mteb/sib200 metrics: - type: accuracy value: 74.26533523537803 - type: f1 value: 73.07458947819569 - type: f1_weighted value: 74.28822395076898 - type: main_score value: 74.26533523537803 task: type: Classification - dataset: config: pol_Latn name: MTEB SIB200Classification (pol_Latn) revision: a74d7350ea12af010cfb1c21e34f1f81fd2e615b split: validation type: mteb/sib200 metrics: - type: accuracy value: 73.03030303030302 - type: f1 value: 71.62450940401054 - type: f1_weighted value: 73.04756771855028 - type: main_score value: 73.03030303030302 task: type: Classification - dataset: config: pol_Latn name: MTEB SIB200ClusteringS2S (pol_Latn) revision: a74d7350ea12af010cfb1c21e34f1f81fd2e615b split: test type: mteb/sib200 metrics: - type: main_score value: 43.50358545235172 - type: v_measure value: 43.50358545235172 - type: v_measure_std value: 3.737257341216802 task: type: Clustering - dataset: config: default name: MTEB SICK-E-PL (default) revision: 71bba34b0ece6c56dfcf46d9758a27f7a90f17e9 split: test type: PL-MTEB/sicke-pl-pairclassification metrics: - type: cosine_accuracy value: 85.50754178556869 - type: cosine_accuracy_threshold value: 89.79843258857727 - type: cosine_ap value: 79.81733712144906 - type: cosine_f1 value: 73.49797272392186 - type: cosine_f1_threshold value: 89.26491141319275 - type: cosine_precision value: 76.1650114591291 - type: cosine_recall value: 71.01139601139602 - type: dot_accuracy value: 85.50754178556869 - type: dot_accuracy_threshold value: 89.79843854904175 - type: dot_ap value: 79.81731228808121 - type: dot_f1 value: 73.49797272392186 - type: dot_f1_threshold value: 89.26491141319275 - type: dot_precision value: 76.1650114591291 - type: dot_recall value: 71.01139601139602 - type: euclidean_accuracy value: 85.50754178556869 - type: euclidean_accuracy_threshold value: 45.16982138156891 - type: euclidean_ap value: 79.81733712144904 - type: euclidean_f1 value: 73.49797272392186 - type: euclidean_f1_threshold value: 46.335917711257935 - type: euclidean_precision value: 76.1650114591291 - type: euclidean_recall value: 71.01139601139602 - type: main_score value: 79.89579409621868 - type: manhattan_accuracy value: 85.50754178556869 - type: manhattan_accuracy_threshold value: 977.5501251220703 - type: manhattan_ap value: 79.89579409621868 - type: manhattan_f1 value: 73.34315169366717 - type: manhattan_f1_threshold value: 1012.5644683837891 - type: manhattan_precision value: 75.91463414634147 - type: manhattan_recall value: 70.94017094017094 - type: max_accuracy value: 85.50754178556869 - type: max_ap value: 79.89579409621868 - type: max_f1 value: 73.49797272392186 - type: max_precision value: 76.1650114591291 - type: max_recall value: 71.01139601139602 - type: similarity_accuracy value: 85.50754178556869 - type: similarity_accuracy_threshold value: 89.79843854904175 - type: similarity_ap value: 79.81733712144904 - type: similarity_f1 value: 73.49797272392186 - type: similarity_f1_threshold value: 89.26491737365723 - type: similarity_precision value: 76.1650114591291 - type: similarity_recall value: 71.01139601139602 task: type: PairClassification - dataset: config: default name: MTEB SICK-R-PL (default) revision: fd5c2441b7eeff8676768036142af4cfa42c1339 split: test type: PL-MTEB/sickr-pl-sts metrics: - type: cosine_pearson value: 84.04170078857925 - type: cosine_spearman value: 78.92221847607668 - type: euclidean_pearson value: 80.88578682387758 - type: euclidean_spearman value: 78.92222004062023 - type: main_score value: 78.92221847607668 - type: manhattan_pearson value: 80.87171998509311 - type: manhattan_spearman value: 78.9413295057423 - type: pearson value: 84.04170012505008 - type: spearman value: 78.92224330603815 task: type: STS - dataset: config: pl name: MTEB STS22 (pl) revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 41.59174229342081 - type: cosine_spearman value: 41.81518057324345 - type: euclidean_pearson value: 32.466997983374014 - type: euclidean_spearman value: 41.93847574622057 - type: main_score value: 41.81518057324345 - type: manhattan_pearson value: 32.601545876828375 - type: manhattan_spearman value: 42.085389551718336 - type: pearson value: 41.59174140580378 - type: spearman value: 41.71631322253171 task: type: STS - dataset: config: pl name: MTEB STS22.v2 (pl) revision: d31f33a128469b20e357535c39b82fb3c3f6f2bd split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 46.55653550801935 - type: cosine_spearman value: 46.66163352853958 - type: euclidean_pearson value: 37.117407808457074 - type: euclidean_spearman value: 46.60094969381428 - type: main_score value: 46.66163352853958 - type: manhattan_pearson value: 36.80577904845572 - type: manhattan_spearman value: 46.494374958837525 - type: pearson value: 46.55653507995975 - type: spearman value: 46.55498079345369 task: type: STS - dataset: config: pl-en name: MTEB STS22.v2 (pl-en) revision: d31f33a128469b20e357535c39b82fb3c3f6f2bd split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 80.21617029742387 - type: cosine_spearman value: 78.77574527045532 - type: euclidean_pearson value: 80.97356448551602 - type: euclidean_spearman value: 78.77574527045532 - type: main_score value: 78.77574527045532 - type: manhattan_pearson value: 80.09457133506493 - type: manhattan_spearman value: 78.9474544606806 - type: pearson value: 80.21617046885102 - type: spearman value: 78.77574527045532 task: type: STS - dataset: config: de-pl name: MTEB STS22.v2 (de-pl) revision: d31f33a128469b20e357535c39b82fb3c3f6f2bd split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 21.14452310169841 - type: cosine_spearman value: 36.737365475055704 - type: euclidean_pearson value: 24.37979189301095 - type: euclidean_spearman value: 36.737365475055704 - type: main_score value: 36.737365475055704 - type: manhattan_pearson value: 25.671050857257033 - type: manhattan_spearman value: 36.798953682054965 - type: pearson value: 21.14454518448624 - type: spearman value: 36.737365475055704 task: type: STS - dataset: config: fr-pl name: MTEB STS22.v2 (fr-pl) revision: d31f33a128469b20e357535c39b82fb3c3f6f2bd split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 76.45845272153467 - type: cosine_spearman value: 84.51542547285167 - type: euclidean_pearson value: 78.31632842671074 - type: euclidean_spearman value: 84.51542547285167 - type: main_score value: 84.51542547285167 - type: manhattan_pearson value: 76.0583275237825 - type: manhattan_spearman value: 84.51542547285167 - type: pearson value: 76.45845588981632 - type: spearman value: 84.51542547285167 task: type: STS - dataset: config: pl name: MTEB STSBenchmarkMultilingualSTS (pl) revision: 29afa2569dcedaaa2fe6a3dcfebab33d28b82e8c split: dev type: mteb/stsb_multi_mt metrics: - type: cosine_pearson value: 85.13515881535021 - type: cosine_spearman value: 85.87832429778716 - type: euclidean_pearson value: 85.53675589178216 - type: euclidean_spearman value: 85.87827749186407 - type: main_score value: 85.87832429778716 - type: manhattan_pearson value: 85.4970561306807 - type: manhattan_spearman value: 85.88418120738208 - type: pearson value: 85.1351598271008 - type: spearman value: 85.87829302337471 task: type: STS - dataset: config: pl name: MTEB STSBenchmarkMultilingualSTS (pl) revision: 29afa2569dcedaaa2fe6a3dcfebab33d28b82e8c split: test type: mteb/stsb_multi_mt metrics: - type: cosine_pearson value: 83.90154924854153 - type: cosine_spearman value: 83.5802958202105 - type: euclidean_pearson value: 83.10838323620546 - type: euclidean_spearman value: 83.58158501150402 - type: main_score value: 83.5802958202105 - type: manhattan_pearson value: 83.1419912790974 - type: manhattan_spearman value: 83.70858645330948 - type: pearson value: 83.90154919039564 - type: spearman value: 83.58111597101961 task: type: STS - dataset: config: default name: MTEB SciFact-PL (default) revision: 47932a35f045ef8ed01ba82bf9ff67f6e109207e split: test type: clarin-knext/scifact-pl metrics: - type: main_score value: 60.6 - type: map_at_1 value: 46.317 - type: map_at_10 value: 55.915000000000006 - type: map_at_100 value: 56.643 - type: map_at_1000 value: 56.702 - type: map_at_20 value: 56.399 - type: map_at_3 value: 53.324000000000005 - type: map_at_5 value: 54.786 - type: mrr_at_1 value: 48.66666666666667 - type: mrr_at_10 value: 57.321296296296296 - type: mrr_at_100 value: 57.89882051667452 - type: mrr_at_1000 value: 57.95238378103823 - type: mrr_at_20 value: 57.6981314981315 - type: mrr_at_3 value: 55.166666666666664 - type: mrr_at_5 value: 56.45 - type: nauc_map_at_1000_diff1 value: 60.35721143333672 - type: nauc_map_at_1000_max value: 60.05322180601467 - type: nauc_map_at_1000_std value: 16.99125810591374 - type: nauc_map_at_100_diff1 value: 60.334099364313246 - type: nauc_map_at_100_max value: 60.0503907670776 - type: nauc_map_at_100_std value: 17.023508113514342 - type: nauc_map_at_10_diff1 value: 60.23211261543967 - type: nauc_map_at_10_max value: 59.8033036450801 - type: nauc_map_at_10_std value: 16.464499848116212 - type: nauc_map_at_1_diff1 value: 67.47955127132796 - type: nauc_map_at_1_max value: 57.93579782873647 - type: nauc_map_at_1_std value: 14.983305646438419 - type: nauc_map_at_20_diff1 value: 60.191720208816726 - type: nauc_map_at_20_max value: 60.00050816589238 - type: nauc_map_at_20_std value: 16.629980059093093 - type: nauc_map_at_3_diff1 value: 61.196988451613436 - type: nauc_map_at_3_max value: 59.99256716686697 - type: nauc_map_at_3_std value: 15.09494519132578 - type: nauc_map_at_5_diff1 value: 60.595708985100536 - type: nauc_map_at_5_max value: 59.39892558822073 - type: nauc_map_at_5_std value: 14.467451148982066 - type: nauc_mrr_at_1000_diff1 value: 61.11648031485068 - type: nauc_mrr_at_1000_max value: 62.219011526871995 - type: nauc_mrr_at_1000_std value: 19.42767858763793 - type: nauc_mrr_at_100_diff1 value: 61.088188874994096 - type: nauc_mrr_at_100_max value: 62.223871302167964 - type: nauc_mrr_at_100_std value: 19.457452279707773 - type: nauc_mrr_at_10_diff1 value: 61.08159685694334 - type: nauc_mrr_at_10_max value: 62.21990131935562 - type: nauc_mrr_at_10_std value: 19.376402355543178 - type: nauc_mrr_at_1_diff1 value: 67.27438307353606 - type: nauc_mrr_at_1_max value: 62.885827711514494 - type: nauc_mrr_at_1_std value: 20.70936596493396 - type: nauc_mrr_at_20_diff1 value: 60.94521143486739 - type: nauc_mrr_at_20_max value: 62.12161984797176 - type: nauc_mrr_at_20_std value: 19.22025350990408 - type: nauc_mrr_at_3_diff1 value: 61.950249652240515 - type: nauc_mrr_at_3_max value: 63.14286143096035 - type: nauc_mrr_at_3_std value: 18.592436891815797 - type: nauc_mrr_at_5_diff1 value: 61.19640903375612 - type: nauc_mrr_at_5_max value: 62.467767526202 - type: nauc_mrr_at_5_std value: 18.347510898506194 - type: nauc_ndcg_at_1000_diff1 value: 59.01051651703205 - type: nauc_ndcg_at_1000_max value: 60.66279150024507 - type: nauc_ndcg_at_1000_std value: 18.6616104733483 - type: nauc_ndcg_at_100_diff1 value: 58.26248080377461 - type: nauc_ndcg_at_100_max value: 60.38566791814694 - type: nauc_ndcg_at_100_std value: 19.505638219628143 - type: nauc_ndcg_at_10_diff1 value: 57.67727303919001 - type: nauc_ndcg_at_10_max value: 59.692480553663295 - type: nauc_ndcg_at_10_std value: 17.35947712811857 - type: nauc_ndcg_at_1_diff1 value: 67.27438307353606 - type: nauc_ndcg_at_1_max value: 62.885827711514494 - type: nauc_ndcg_at_1_std value: 20.70936596493396 - type: nauc_ndcg_at_20_diff1 value: 57.30641762099167 - type: nauc_ndcg_at_20_max value: 59.95671965731118 - type: nauc_ndcg_at_20_std value: 17.33468485022403 - type: nauc_ndcg_at_3_diff1 value: 59.01671872632254 - type: nauc_ndcg_at_3_max value: 61.340857926998225 - type: nauc_ndcg_at_3_std value: 15.691398633173204 - type: nauc_ndcg_at_5_diff1 value: 58.096149600090556 - type: nauc_ndcg_at_5_max value: 59.46021363773192 - type: nauc_ndcg_at_5_std value: 13.964334038615151 - type: nauc_precision_at_1000_diff1 value: -17.83315623809618 - type: nauc_precision_at_1000_max value: 19.620815707345145 - type: nauc_precision_at_1000_std value: 48.10708581433734 - type: nauc_precision_at_100_diff1 value: 4.305969098053754 - type: nauc_precision_at_100_max value: 33.095489054143926 - type: nauc_precision_at_100_std value: 47.8246720129009 - type: nauc_precision_at_10_diff1 value: 21.6453117579965 - type: nauc_precision_at_10_max value: 48.088709784794 - type: nauc_precision_at_10_std value: 33.99668913851027 - type: nauc_precision_at_1_diff1 value: 67.27438307353606 - type: nauc_precision_at_1_max value: 62.885827711514494 - type: nauc_precision_at_1_std value: 20.70936596493396 - type: nauc_precision_at_20_diff1 value: 13.06836048047285 - type: nauc_precision_at_20_max value: 43.08893420898951 - type: nauc_precision_at_20_std value: 34.521189143539914 - type: nauc_precision_at_3_diff1 value: 38.595356035798126 - type: nauc_precision_at_3_max value: 59.77300449881297 - type: nauc_precision_at_3_std value: 24.25730237357198 - type: nauc_precision_at_5_diff1 value: 30.655944367834824 - type: nauc_precision_at_5_max value: 53.28245247554937 - type: nauc_precision_at_5_std value: 20.886182524837356 - type: nauc_recall_at_1000_diff1 value: 41.433239962651925 - type: nauc_recall_at_1000_max value: 86.11111111111035 - type: nauc_recall_at_1000_std value: 7.936507936508372 - type: nauc_recall_at_100_diff1 value: 43.29551453892163 - type: nauc_recall_at_100_max value: 55.55531351071509 - type: nauc_recall_at_100_std value: 32.58481678344614 - type: nauc_recall_at_10_diff1 value: 45.809480869001085 - type: nauc_recall_at_10_max value: 54.047819362944146 - type: nauc_recall_at_10_std value: 15.998244308111959 - type: nauc_recall_at_1_diff1 value: 67.47955127132796 - type: nauc_recall_at_1_max value: 57.93579782873647 - type: nauc_recall_at_1_std value: 14.983305646438419 - type: nauc_recall_at_20_diff1 value: 41.257961783439534 - type: nauc_recall_at_20_max value: 54.240380865722024 - type: nauc_recall_at_20_std value: 14.07822046015856 - type: nauc_recall_at_3_diff1 value: 53.05384331122555 - type: nauc_recall_at_3_max value: 58.59544973536968 - type: nauc_recall_at_3_std value: 9.86718965198744 - type: nauc_recall_at_5_diff1 value: 48.19331315126426 - type: nauc_recall_at_5_max value: 54.47823169862453 - type: nauc_recall_at_5_std value: 6.943218011112343 - type: ndcg_at_1 value: 48.667 - type: ndcg_at_10 value: 60.6 - type: ndcg_at_100 value: 63.773 - type: ndcg_at_1000 value: 65.295 - type: ndcg_at_20 value: 62.172000000000004 - type: ndcg_at_3 value: 55.981 - type: ndcg_at_5 value: 58.270999999999994 - type: precision_at_1 value: 48.667 - type: precision_at_10 value: 8.233 - type: precision_at_100 value: 0.997 - type: precision_at_1000 value: 0.11199999999999999 - type: precision_at_20 value: 4.4830000000000005 - type: precision_at_3 value: 22.111 - type: precision_at_5 value: 14.6 - type: recall_at_1 value: 46.317 - type: recall_at_10 value: 73.433 - type: recall_at_100 value: 87.533 - type: recall_at_1000 value: 99.333 - type: recall_at_20 value: 79.333 - type: recall_at_3 value: 60.882999999999996 - type: recall_at_5 value: 66.89399999999999 task: type: Retrieval - dataset: config: default name: MTEB TRECCOVID-PL (default) revision: 81bcb408f33366c2a20ac54adafad1ae7e877fdd split: test type: clarin-knext/trec-covid-pl metrics: - type: main_score value: 69.32000000000001 - type: map_at_1 value: 0.22599999999999998 - type: map_at_10 value: 1.7420000000000002 - type: map_at_100 value: 9.665 - type: map_at_1000 value: 24.174 - type: map_at_20 value: 3.003 - type: map_at_3 value: 0.599 - type: map_at_5 value: 0.9560000000000001 - type: mrr_at_1 value: 82.0 - type: mrr_at_10 value: 90.0 - type: mrr_at_100 value: 90.0 - type: mrr_at_1000 value: 90.0 - type: mrr_at_20 value: 90.0 - type: mrr_at_3 value: 90.0 - type: mrr_at_5 value: 90.0 - type: nauc_map_at_1000_diff1 value: -15.757643018986789 - type: nauc_map_at_1000_max value: 58.83724759719735 - type: nauc_map_at_1000_std value: 81.5806407072152 - type: nauc_map_at_100_diff1 value: -15.914337821257696 - type: nauc_map_at_100_max value: 42.94384150292272 - type: nauc_map_at_100_std value: 61.23454197897242 - type: nauc_map_at_10_diff1 value: -22.890685727481163 - type: nauc_map_at_10_max value: 15.202784946934035 - type: nauc_map_at_10_std value: 19.53713756551298 - type: nauc_map_at_1_diff1 value: -8.23648182483711 - type: nauc_map_at_1_max value: 2.9418466254777185 - type: nauc_map_at_1_std value: 9.066665328272006 - type: nauc_map_at_20_diff1 value: -15.741357368332274 - type: nauc_map_at_20_max value: 23.093660938745934 - type: nauc_map_at_20_std value: 28.37835837211648 - type: nauc_map_at_3_diff1 value: -22.196812161671943 - type: nauc_map_at_3_max value: 12.750029854270887 - type: nauc_map_at_3_std value: 23.17304521418692 - type: nauc_map_at_5_diff1 value: -22.190118389435636 - type: nauc_map_at_5_max value: 15.24441128160928 - type: nauc_map_at_5_std value: 22.690240797848766 - type: nauc_mrr_at_1000_diff1 value: -16.600985221674865 - type: nauc_mrr_at_1000_max value: 7.108374384236335 - type: nauc_mrr_at_1000_std value: 26.517241379310256 - type: nauc_mrr_at_100_diff1 value: -16.600985221674865 - type: nauc_mrr_at_100_max value: 7.108374384236335 - type: nauc_mrr_at_100_std value: 26.517241379310256 - type: nauc_mrr_at_10_diff1 value: -16.600985221674865 - type: nauc_mrr_at_10_max value: 7.108374384236335 - type: nauc_mrr_at_10_std value: 26.517241379310256 - type: nauc_mrr_at_1_diff1 value: -13.608326908249815 - type: nauc_mrr_at_1_max value: 12.622535521533287 - type: nauc_mrr_at_1_std value: 28.81374600726958 - type: nauc_mrr_at_20_diff1 value: -16.600985221674865 - type: nauc_mrr_at_20_max value: 7.108374384236335 - type: nauc_mrr_at_20_std value: 26.517241379310256 - type: nauc_mrr_at_3_diff1 value: -16.600985221674865 - type: nauc_mrr_at_3_max value: 7.108374384236335 - type: nauc_mrr_at_3_std value: 26.517241379310256 - type: nauc_mrr_at_5_diff1 value: -16.600985221674865 - type: nauc_mrr_at_5_max value: 7.108374384236335 - type: nauc_mrr_at_5_std value: 26.517241379310256 - type: nauc_ndcg_at_1000_diff1 value: -13.194696606963987 - type: nauc_ndcg_at_1000_max value: 55.39361570574168 - type: nauc_ndcg_at_1000_std value: 78.24389785896678 - type: nauc_ndcg_at_100_diff1 value: -14.709514114787883 - type: nauc_ndcg_at_100_max value: 42.5762797951252 - type: nauc_ndcg_at_100_std value: 72.26496604715362 - type: nauc_ndcg_at_10_diff1 value: -20.152166371506738 - type: nauc_ndcg_at_10_max value: 32.94483821938343 - type: nauc_ndcg_at_10_std value: 44.03398124005722 - type: nauc_ndcg_at_1_diff1 value: 7.175383527123184 - type: nauc_ndcg_at_1_max value: 9.772757416331425 - type: nauc_ndcg_at_1_std value: 25.648790839559606 - type: nauc_ndcg_at_20_diff1 value: -9.62589500953276 - type: nauc_ndcg_at_20_max value: 39.30914262040181 - type: nauc_ndcg_at_20_std value: 56.880739749710116 - type: nauc_ndcg_at_3_diff1 value: -17.947554993563717 - type: nauc_ndcg_at_3_max value: 26.74365078754596 - type: nauc_ndcg_at_3_std value: 41.793803172791606 - type: nauc_ndcg_at_5_diff1 value: -17.417527011830227 - type: nauc_ndcg_at_5_max value: 33.88471424700834 - type: nauc_ndcg_at_5_std value: 46.534674061431424 - type: nauc_precision_at_1000_diff1 value: -13.500374587246162 - type: nauc_precision_at_1000_max value: 41.89683167017185 - type: nauc_precision_at_1000_std value: 47.996095424591886 - type: nauc_precision_at_100_diff1 value: -15.762075381944218 - type: nauc_precision_at_100_max value: 44.01616846578541 - type: nauc_precision_at_100_std value: 72.07152761602794 - type: nauc_precision_at_10_diff1 value: -35.93338426279604 - type: nauc_precision_at_10_max value: 34.183040488922856 - type: nauc_precision_at_10_std value: 42.87975553857905 - type: nauc_precision_at_1_diff1 value: -28.30756013745697 - type: nauc_precision_at_1_max value: 9.358124693176201 - type: nauc_precision_at_1_std value: 39.51276386843394 - type: nauc_precision_at_20_diff1 value: -10.583246399588123 - type: nauc_precision_at_20_max value: 40.86870520654101 - type: nauc_precision_at_20_std value: 59.31717494682367 - type: nauc_precision_at_3_diff1 value: -44.10520186335414 - type: nauc_precision_at_3_max value: 38.37344720496887 - type: nauc_precision_at_3_std value: 55.199922360248365 - type: nauc_precision_at_5_diff1 value: -34.27848522284507 - type: nauc_precision_at_5_max value: 42.824052076404136 - type: nauc_precision_at_5_std value: 51.120165352085856 - type: nauc_recall_at_1000_diff1 value: -11.76715587281032 - type: nauc_recall_at_1000_max value: 52.088079742910274 - type: nauc_recall_at_1000_std value: 68.23415173297899 - type: nauc_recall_at_100_diff1 value: -10.978661355290605 - type: nauc_recall_at_100_max value: 31.748771511392693 - type: nauc_recall_at_100_std value: 51.122645111292876 - type: nauc_recall_at_10_diff1 value: -19.336151816299974 - type: nauc_recall_at_10_max value: 10.743443384702298 - type: nauc_recall_at_10_std value: 12.911493888937114 - type: nauc_recall_at_1_diff1 value: -8.23648182483711 - type: nauc_recall_at_1_max value: 2.9418466254777185 - type: nauc_recall_at_1_std value: 9.066665328272006 - type: nauc_recall_at_20_diff1 value: -8.54869518672105 - type: nauc_recall_at_20_max value: 15.362562157663387 - type: nauc_recall_at_20_std value: 21.489881366775784 - type: nauc_recall_at_3_diff1 value: -21.0866131241463 - type: nauc_recall_at_3_max value: 11.195127104924845 - type: nauc_recall_at_3_std value: 20.620111859982334 - type: nauc_recall_at_5_diff1 value: -19.067070989034356 - type: nauc_recall_at_5_max value: 12.493604263463501 - type: nauc_recall_at_5_std value: 16.31157612407365 - type: ndcg_at_1 value: 76.0 - type: ndcg_at_10 value: 69.32000000000001 - type: ndcg_at_100 value: 53.480000000000004 - type: ndcg_at_1000 value: 50.283 - type: ndcg_at_20 value: 65.157 - type: ndcg_at_3 value: 73.75399999999999 - type: ndcg_at_5 value: 71.881 - type: precision_at_1 value: 84.0 - type: precision_at_10 value: 73.6 - type: precision_at_100 value: 55.26 - type: precision_at_1000 value: 22.259999999999998 - type: precision_at_20 value: 68.5 - type: precision_at_3 value: 80.0 - type: precision_at_5 value: 77.60000000000001 - type: recall_at_1 value: 0.22599999999999998 - type: recall_at_10 value: 1.978 - type: recall_at_100 value: 13.272999999999998 - type: recall_at_1000 value: 48.092 - type: recall_at_20 value: 3.5839999999999996 - type: recall_at_3 value: 0.636 - type: recall_at_5 value: 1.046 task: type: Retrieval - dataset: config: pol-eng name: MTEB Tatoeba (pol-eng) revision: 69e8f12da6e31d59addadda9a9c8a2e601a0e282 split: test type: mteb/tatoeba-bitext-mining metrics: - type: accuracy value: 95.3 - type: f1 value: 93.94 - type: main_score value: 93.94 - type: precision value: 93.29166666666666 - type: recall value: 95.3 task: type: BitextMining - dataset: config: pol-pol name: MTEB XPQARetrieval (pol-pol) revision: c99d599f0a6ab9b85b065da6f9d94f9cf731679f split: test type: jinaai/xpqa metrics: - type: main_score value: 51.515 - type: map_at_1 value: 26.518000000000004 - type: map_at_10 value: 45.369 - type: map_at_100 value: 47.223 - type: map_at_1000 value: 47.321999999999996 - type: map_at_20 value: 46.428000000000004 - type: map_at_3 value: 39.318999999999996 - type: map_at_5 value: 43.36 - type: mrr_at_1 value: 44.968152866242036 - type: mrr_at_10 value: 52.43079567283389 - type: mrr_at_100 value: 53.34807803305783 - type: mrr_at_1000 value: 53.37806353437189 - type: mrr_at_20 value: 52.99111513455048 - type: mrr_at_3 value: 49.78768577494692 - type: mrr_at_5 value: 51.11889596602973 - type: nauc_map_at_1000_diff1 value: 40.6335226588315 - type: nauc_map_at_1000_max value: 28.16397840496108 - type: nauc_map_at_1000_std value: -32.21841199511412 - type: nauc_map_at_100_diff1 value: 40.590959517763906 - type: nauc_map_at_100_max value: 28.19178863787807 - type: nauc_map_at_100_std value: -32.21041997091445 - type: nauc_map_at_10_diff1 value: 40.728865810296924 - type: nauc_map_at_10_max value: 28.10952776056662 - type: nauc_map_at_10_std value: -32.858725977862235 - type: nauc_map_at_1_diff1 value: 53.70650384931953 - type: nauc_map_at_1_max value: 14.899229671716071 - type: nauc_map_at_1_std value: -27.68849479744787 - type: nauc_map_at_20_diff1 value: 40.42458500884532 - type: nauc_map_at_20_max value: 28.105454952138793 - type: nauc_map_at_20_std value: -32.54318887740866 - type: nauc_map_at_3_diff1 value: 43.6909634186489 - type: nauc_map_at_3_max value: 24.284262589928197 - type: nauc_map_at_3_std value: -31.57069615207669 - type: nauc_map_at_5_diff1 value: 41.06724797035683 - type: nauc_map_at_5_max value: 27.848131743238234 - type: nauc_map_at_5_std value: -32.25111649789939 - type: nauc_mrr_at_1000_diff1 value: 46.67398611613204 - type: nauc_mrr_at_1000_max value: 27.609373654318524 - type: nauc_mrr_at_1000_std value: -34.735880744535905 - type: nauc_mrr_at_100_diff1 value: 46.66073223241135 - type: nauc_mrr_at_100_max value: 27.627036817536453 - type: nauc_mrr_at_100_std value: -34.70550683592668 - type: nauc_mrr_at_10_diff1 value: 46.487500926010135 - type: nauc_mrr_at_10_max value: 27.75350626705907 - type: nauc_mrr_at_10_std value: -35.08601176389904 - type: nauc_mrr_at_1_diff1 value: 51.06876475124916 - type: nauc_mrr_at_1_max value: 25.92322105282855 - type: nauc_mrr_at_1_std value: -34.15792018096852 - type: nauc_mrr_at_20_diff1 value: 46.5162605888926 - type: nauc_mrr_at_20_max value: 27.55019106976382 - type: nauc_mrr_at_20_std value: -34.80694715170915 - type: nauc_mrr_at_3_diff1 value: 47.36233736696853 - type: nauc_mrr_at_3_max value: 27.543928085898884 - type: nauc_mrr_at_3_std value: -35.21504971692032 - type: nauc_mrr_at_5_diff1 value: 46.91292088489212 - type: nauc_mrr_at_5_max value: 27.298410620653396 - type: nauc_mrr_at_5_std value: -35.24045074547203 - type: nauc_ndcg_at_1000_diff1 value: 41.07705948487416 - type: nauc_ndcg_at_1000_max value: 28.874643241309244 - type: nauc_ndcg_at_1000_std value: -31.60833641025355 - type: nauc_ndcg_at_100_diff1 value: 40.2376962346747 - type: nauc_ndcg_at_100_max value: 29.635075481651157 - type: nauc_ndcg_at_100_std value: -30.81574899851739 - type: nauc_ndcg_at_10_diff1 value: 39.818321210198796 - type: nauc_ndcg_at_10_max value: 29.29397326733968 - type: nauc_ndcg_at_10_std value: -33.94466582211004 - type: nauc_ndcg_at_1_diff1 value: 51.06876475124916 - type: nauc_ndcg_at_1_max value: 25.92322105282855 - type: nauc_ndcg_at_1_std value: -34.15792018096852 - type: nauc_ndcg_at_20_diff1 value: 39.27450983639958 - type: nauc_ndcg_at_20_max value: 28.894936103549007 - type: nauc_ndcg_at_20_std value: -33.21160980125333 - type: nauc_ndcg_at_3_diff1 value: 41.785541318953186 - type: nauc_ndcg_at_3_max value: 27.23155369736596 - type: nauc_ndcg_at_3_std value: -33.37324927794433 - type: nauc_ndcg_at_5_diff1 value: 40.72837220220939 - type: nauc_ndcg_at_5_max value: 28.55495870540679 - type: nauc_ndcg_at_5_std value: -33.24344549702784 - type: nauc_precision_at_1000_diff1 value: -13.322446650376113 - type: nauc_precision_at_1000_max value: 8.545426214523442 - type: nauc_precision_at_1000_std value: 6.979424267581099 - type: nauc_precision_at_100_diff1 value: -10.509590126006007 - type: nauc_precision_at_100_max value: 15.171580740556001 - type: nauc_precision_at_100_std value: 4.194466586738409 - type: nauc_precision_at_10_diff1 value: 2.644797279359475 - type: nauc_precision_at_10_max value: 25.855731593146437 - type: nauc_precision_at_10_std value: -19.233983107036174 - type: nauc_precision_at_1_diff1 value: 51.06876475124916 - type: nauc_precision_at_1_max value: 25.92322105282855 - type: nauc_precision_at_1_std value: -34.15792018096852 - type: nauc_precision_at_20_diff1 value: -3.1136913912470736 - type: nauc_precision_at_20_max value: 21.359090339642805 - type: nauc_precision_at_20_std value: -13.023770103071856 - type: nauc_precision_at_3_diff1 value: 15.584460581411957 - type: nauc_precision_at_3_max value: 27.42225721000173 - type: nauc_precision_at_3_std value: -25.104318309636454 - type: nauc_precision_at_5_diff1 value: 7.674766587199025 - type: nauc_precision_at_5_max value: 28.111733106326504 - type: nauc_precision_at_5_std value: -21.937042014222495 - type: nauc_recall_at_1000_diff1 value: 73.66889189263455 - type: nauc_recall_at_1000_max value: 90.15156027243394 - type: nauc_recall_at_1000_std value: 66.48304725775141 - type: nauc_recall_at_100_diff1 value: 21.99762616698104 - type: nauc_recall_at_100_max value: 41.71147447669556 - type: nauc_recall_at_100_std value: -4.743160690089563 - type: nauc_recall_at_10_diff1 value: 30.034636842475482 - type: nauc_recall_at_10_max value: 31.214728078930452 - type: nauc_recall_at_10_std value: -32.97398123551153 - type: nauc_recall_at_1_diff1 value: 53.70650384931953 - type: nauc_recall_at_1_max value: 14.899229671716071 - type: nauc_recall_at_1_std value: -27.68849479744787 - type: nauc_recall_at_20_diff1 value: 26.007825826053683 - type: nauc_recall_at_20_max value: 29.217535040661367 - type: nauc_recall_at_20_std value: -30.70687672239101 - type: nauc_recall_at_3_diff1 value: 38.44779042450265 - type: nauc_recall_at_3_max value: 24.889721742689385 - type: nauc_recall_at_3_std value: -31.13463300886431 - type: nauc_recall_at_5_diff1 value: 34.22038381301645 - type: nauc_recall_at_5_max value: 29.516584372821526 - type: nauc_recall_at_5_std value: -31.484524861831925 - type: ndcg_at_1 value: 44.968 - type: ndcg_at_10 value: 51.515 - type: ndcg_at_100 value: 58.501000000000005 - type: ndcg_at_1000 value: 60.068 - type: ndcg_at_20 value: 54.429 - type: ndcg_at_3 value: 44.909 - type: ndcg_at_5 value: 47.682 - type: precision_at_1 value: 44.968 - type: precision_at_10 value: 12.903999999999998 - type: precision_at_100 value: 1.873 - type: precision_at_1000 value: 0.207 - type: precision_at_20 value: 7.489999999999999 - type: precision_at_3 value: 28.79 - type: precision_at_5 value: 21.682000000000002 - type: recall_at_1 value: 26.518000000000004 - type: recall_at_10 value: 62.05500000000001 - type: recall_at_100 value: 89.607 - type: recall_at_1000 value: 99.682 - type: recall_at_20 value: 71.531 - type: recall_at_3 value: 43.671 - type: recall_at_5 value: 52.373999999999995 task: type: Retrieval - dataset: config: eng-pol name: MTEB XPQARetrieval (eng-pol) revision: c99d599f0a6ab9b85b065da6f9d94f9cf731679f split: test type: jinaai/xpqa metrics: - type: main_score value: 34.001999999999995 - type: map_at_1 value: 14.172 - type: map_at_10 value: 27.466 - type: map_at_100 value: 29.469 - type: map_at_1000 value: 29.686 - type: map_at_20 value: 28.544000000000004 - type: map_at_3 value: 22.451999999999998 - type: map_at_5 value: 25.237 - type: mrr_at_1 value: 30.828025477707005 - type: mrr_at_10 value: 38.19634010716813 - type: mrr_at_100 value: 39.2480381484182 - type: mrr_at_1000 value: 39.321378424305784 - type: mrr_at_20 value: 38.77839570360424 - type: mrr_at_3 value: 35.64755838641189 - type: mrr_at_5 value: 37.06157112526539 - type: nauc_map_at_1000_diff1 value: 31.180369776855038 - type: nauc_map_at_1000_max value: 23.696275295738246 - type: nauc_map_at_1000_std value: -21.27413499405951 - type: nauc_map_at_100_diff1 value: 31.189167925330995 - type: nauc_map_at_100_max value: 23.655306190408066 - type: nauc_map_at_100_std value: -21.436647059542242 - type: nauc_map_at_10_diff1 value: 31.318694969880156 - type: nauc_map_at_10_max value: 23.074738813224236 - type: nauc_map_at_10_std value: -21.923538669237157 - type: nauc_map_at_1_diff1 value: 41.6000692063174 - type: nauc_map_at_1_max value: 15.406617421977645 - type: nauc_map_at_1_std value: -20.907084480848887 - type: nauc_map_at_20_diff1 value: 30.970001594011244 - type: nauc_map_at_20_max value: 23.14174410247358 - type: nauc_map_at_20_std value: -21.957135319177706 - type: nauc_map_at_3_diff1 value: 32.54806390568495 - type: nauc_map_at_3_max value: 20.300802458881474 - type: nauc_map_at_3_std value: -21.94392553088974 - type: nauc_map_at_5_diff1 value: 31.54042190428169 - type: nauc_map_at_5_max value: 23.203725494722843 - type: nauc_map_at_5_std value: -21.90268057328536 - type: nauc_mrr_at_1000_diff1 value: 31.59051275226857 - type: nauc_mrr_at_1000_max value: 23.34286182938858 - type: nauc_mrr_at_1000_std value: -17.005575811675644 - type: nauc_mrr_at_100_diff1 value: 31.569651385372694 - type: nauc_mrr_at_100_max value: 23.342499818552994 - type: nauc_mrr_at_100_std value: -16.966751458089767 - type: nauc_mrr_at_10_diff1 value: 31.64921530070727 - type: nauc_mrr_at_10_max value: 23.026241511313508 - type: nauc_mrr_at_10_std value: -17.656777429702032 - type: nauc_mrr_at_1_diff1 value: 34.49289583539921 - type: nauc_mrr_at_1_max value: 24.574155198363375 - type: nauc_mrr_at_1_std value: -17.42832990773259 - type: nauc_mrr_at_20_diff1 value: 31.5433649153002 - type: nauc_mrr_at_20_max value: 23.11967165826937 - type: nauc_mrr_at_20_std value: -17.356109119216693 - type: nauc_mrr_at_3_diff1 value: 31.38888681404883 - type: nauc_mrr_at_3_max value: 23.46764438246823 - type: nauc_mrr_at_3_std value: -17.441932437536085 - type: nauc_mrr_at_5_diff1 value: 31.42962954759095 - type: nauc_mrr_at_5_max value: 22.961395531351496 - type: nauc_mrr_at_5_std value: -17.751199294592183 - type: nauc_ndcg_at_1000_diff1 value: 30.152430844510363 - type: nauc_ndcg_at_1000_max value: 24.69903031443189 - type: nauc_ndcg_at_1000_std value: -17.020300975357372 - type: nauc_ndcg_at_100_diff1 value: 30.056132616191643 - type: nauc_ndcg_at_100_max value: 24.678111161152504 - type: nauc_ndcg_at_100_std value: -17.636510310102548 - type: nauc_ndcg_at_10_diff1 value: 30.453542123287775 - type: nauc_ndcg_at_10_max value: 21.918608131636432 - type: nauc_ndcg_at_10_std value: -21.615985468383712 - type: nauc_ndcg_at_1_diff1 value: 34.49289583539921 - type: nauc_ndcg_at_1_max value: 24.574155198363375 - type: nauc_ndcg_at_1_std value: -17.42832990773259 - type: nauc_ndcg_at_20_diff1 value: 29.618071857334964 - type: nauc_ndcg_at_20_max value: 22.165933820138235 - type: nauc_ndcg_at_20_std value: -21.24516593709915 - type: nauc_ndcg_at_3_diff1 value: 29.935596954829425 - type: nauc_ndcg_at_3_max value: 23.870746063527886 - type: nauc_ndcg_at_3_std value: -20.52820962231354 - type: nauc_ndcg_at_5_diff1 value: 30.611546547499852 - type: nauc_ndcg_at_5_max value: 22.46894832525212 - type: nauc_ndcg_at_5_std value: -21.42555859684832 - type: nauc_precision_at_1000_diff1 value: -2.508301193301942 - type: nauc_precision_at_1000_max value: 22.390556885836265 - type: nauc_precision_at_1000_std value: 16.405344138171944 - type: nauc_precision_at_100_diff1 value: 5.680220194312509 - type: nauc_precision_at_100_max value: 26.73017311371664 - type: nauc_precision_at_100_std value: 5.21547577464621 - type: nauc_precision_at_10_diff1 value: 14.123129946524987 - type: nauc_precision_at_10_max value: 25.1809898898883 - type: nauc_precision_at_10_std value: -13.934743869017877 - type: nauc_precision_at_1_diff1 value: 34.49289583539921 - type: nauc_precision_at_1_max value: 24.574155198363375 - type: nauc_precision_at_1_std value: -17.42832990773259 - type: nauc_precision_at_20_diff1 value: 10.679094728567474 - type: nauc_precision_at_20_max value: 24.36678691093217 - type: nauc_precision_at_20_std value: -10.544690003735347 - type: nauc_precision_at_3_diff1 value: 18.847592800225204 - type: nauc_precision_at_3_max value: 27.370778609630392 - type: nauc_precision_at_3_std value: -17.213845532849913 - type: nauc_precision_at_5_diff1 value: 16.205691936009224 - type: nauc_precision_at_5_max value: 28.651246298621903 - type: nauc_precision_at_5_std value: -16.454362592616576 - type: nauc_recall_at_1000_diff1 value: 26.134757901840477 - type: nauc_recall_at_1000_max value: 46.02150918798059 - type: nauc_recall_at_1000_std value: 43.284136673058285 - type: nauc_recall_at_100_diff1 value: 22.56382049727145 - type: nauc_recall_at_100_max value: 25.364782533613546 - type: nauc_recall_at_100_std value: -5.261771997179095 - type: nauc_recall_at_10_diff1 value: 26.108301564213672 - type: nauc_recall_at_10_max value: 16.37450289809244 - type: nauc_recall_at_10_std value: -22.244371629240995 - type: nauc_recall_at_1_diff1 value: 41.6000692063174 - type: nauc_recall_at_1_max value: 15.406617421977645 - type: nauc_recall_at_1_std value: -20.907084480848887 - type: nauc_recall_at_20_diff1 value: 23.043146881697876 - type: nauc_recall_at_20_max value: 15.787128435166457 - type: nauc_recall_at_20_std value: -21.829136287635855 - type: nauc_recall_at_3_diff1 value: 27.70945722150185 - type: nauc_recall_at_3_max value: 17.38259189188097 - type: nauc_recall_at_3_std value: -21.45814251937902 - type: nauc_recall_at_5_diff1 value: 26.482067595811316 - type: nauc_recall_at_5_max value: 18.563489596790177 - type: nauc_recall_at_5_std value: -21.423753382233 - type: ndcg_at_1 value: 30.828 - type: ndcg_at_10 value: 34.001999999999995 - type: ndcg_at_100 value: 41.932 - type: ndcg_at_1000 value: 45.9 - type: ndcg_at_20 value: 36.93 - type: ndcg_at_3 value: 29.116999999999997 - type: ndcg_at_5 value: 30.160999999999998 - type: precision_at_1 value: 30.828 - type: precision_at_10 value: 10.688 - type: precision_at_100 value: 1.8010000000000002 - type: precision_at_1000 value: 0.23600000000000002 - type: precision_at_20 value: 6.5409999999999995 - type: precision_at_3 value: 21.486 - type: precision_at_5 value: 16.611 - type: recall_at_1 value: 14.172 - type: recall_at_10 value: 41.588 - type: recall_at_100 value: 72.478 - type: recall_at_1000 value: 98.44800000000001 - type: recall_at_20 value: 50.461 - type: recall_at_3 value: 26.066 - type: recall_at_5 value: 32.548 task: type: Retrieval - dataset: config: pol-eng name: MTEB XPQARetrieval (pol-eng) revision: c99d599f0a6ab9b85b065da6f9d94f9cf731679f split: test type: jinaai/xpqa metrics: - type: main_score value: 46.075 - type: map_at_1 value: 22.926 - type: map_at_10 value: 40.257 - type: map_at_100 value: 42.135 - type: map_at_1000 value: 42.262 - type: map_at_20 value: 41.253 - type: map_at_3 value: 34.868 - type: map_at_5 value: 38.475 - type: mrr_at_1 value: 39.768339768339764 - type: mrr_at_10 value: 47.09873138444567 - type: mrr_at_100 value: 48.139301720039676 - type: mrr_at_1000 value: 48.18418194119319 - type: mrr_at_20 value: 47.644195865668664 - type: mrr_at_3 value: 44.72329472329472 - type: mrr_at_5 value: 45.86229086229087 - type: nauc_map_at_1000_diff1 value: 35.70994225720207 - type: nauc_map_at_1000_max value: 31.922573287374682 - type: nauc_map_at_1000_std value: -19.077890896454814 - type: nauc_map_at_100_diff1 value: 35.65001346737191 - type: nauc_map_at_100_max value: 31.88853880847858 - type: nauc_map_at_100_std value: -19.122326726996974 - type: nauc_map_at_10_diff1 value: 36.06524735461695 - type: nauc_map_at_10_max value: 32.246461162800074 - type: nauc_map_at_10_std value: -19.679632621277232 - type: nauc_map_at_1_diff1 value: 48.21834981287912 - type: nauc_map_at_1_max value: 19.568464844451327 - type: nauc_map_at_1_std value: -17.84079220033572 - type: nauc_map_at_20_diff1 value: 35.85403649253042 - type: nauc_map_at_20_max value: 31.998466861950746 - type: nauc_map_at_20_std value: -19.417428025928675 - type: nauc_map_at_3_diff1 value: 37.71461187785153 - type: nauc_map_at_3_max value: 27.85047376524254 - type: nauc_map_at_3_std value: -20.55351273572571 - type: nauc_map_at_5_diff1 value: 36.41442025594757 - type: nauc_map_at_5_max value: 31.93891173147263 - type: nauc_map_at_5_std value: -20.470677786255496 - type: nauc_mrr_at_1000_diff1 value: 40.12863479380261 - type: nauc_mrr_at_1000_max value: 32.60419234657007 - type: nauc_mrr_at_1000_std value: -19.444706103106302 - type: nauc_mrr_at_100_diff1 value: 40.08976184544935 - type: nauc_mrr_at_100_max value: 32.596616195907316 - type: nauc_mrr_at_100_std value: -19.412710800127886 - type: nauc_mrr_at_10_diff1 value: 40.118684898684506 - type: nauc_mrr_at_10_max value: 32.881384976300964 - type: nauc_mrr_at_10_std value: -19.609244341545555 - type: nauc_mrr_at_1_diff1 value: 44.54706413166179 - type: nauc_mrr_at_1_max value: 30.651084477793876 - type: nauc_mrr_at_1_std value: -20.909499665543724 - type: nauc_mrr_at_20_diff1 value: 40.12732869131533 - type: nauc_mrr_at_20_max value: 32.63485461369245 - type: nauc_mrr_at_20_std value: -19.53234890306859 - type: nauc_mrr_at_3_diff1 value: 40.28699877191242 - type: nauc_mrr_at_3_max value: 32.10901511067561 - type: nauc_mrr_at_3_std value: -20.523157017762113 - type: nauc_mrr_at_5_diff1 value: 40.598142279113624 - type: nauc_mrr_at_5_max value: 32.79284811079324 - type: nauc_mrr_at_5_std value: -20.10434179938904 - type: nauc_ndcg_at_1000_diff1 value: 35.41293508928805 - type: nauc_ndcg_at_1000_max value: 32.95260794989011 - type: nauc_ndcg_at_1000_std value: -16.69889720188341 - type: nauc_ndcg_at_100_diff1 value: 33.96563718225739 - type: nauc_ndcg_at_100_max value: 32.658834254535115 - type: nauc_ndcg_at_100_std value: -16.121037324309018 - type: nauc_ndcg_at_10_diff1 value: 35.465622135521365 - type: nauc_ndcg_at_10_max value: 34.096760552498615 - type: nauc_ndcg_at_10_std value: -18.520624911178732 - type: nauc_ndcg_at_1_diff1 value: 44.54706413166179 - type: nauc_ndcg_at_1_max value: 30.651084477793876 - type: nauc_ndcg_at_1_std value: -20.909499665543724 - type: nauc_ndcg_at_20_diff1 value: 34.95908417380056 - type: nauc_ndcg_at_20_max value: 33.271534461469024 - type: nauc_ndcg_at_20_std value: -17.891499167451634 - type: nauc_ndcg_at_3_diff1 value: 36.119618180303114 - type: nauc_ndcg_at_3_max value: 30.662594748089145 - type: nauc_ndcg_at_3_std value: -20.681275639308534 - type: nauc_ndcg_at_5_diff1 value: 36.21550522955033 - type: nauc_ndcg_at_5_max value: 33.50161574303214 - type: nauc_ndcg_at_5_std value: -20.088080257880865 - type: nauc_precision_at_1000_diff1 value: -11.522647268410749 - type: nauc_precision_at_1000_max value: 9.883825545840985 - type: nauc_precision_at_1000_std value: 12.25330079826974 - type: nauc_precision_at_100_diff1 value: -8.19761548329529 - type: nauc_precision_at_100_max value: 15.63757897309177 - type: nauc_precision_at_100_std value: 8.653177981589872 - type: nauc_precision_at_10_diff1 value: 7.361531571818795 - type: nauc_precision_at_10_max value: 30.843010914980688 - type: nauc_precision_at_10_std value: -8.588494931873978 - type: nauc_precision_at_1_diff1 value: 44.54706413166179 - type: nauc_precision_at_1_max value: 30.651084477793876 - type: nauc_precision_at_1_std value: -20.909499665543724 - type: nauc_precision_at_20_diff1 value: 2.8986420707285117 - type: nauc_precision_at_20_max value: 25.892300867175162 - type: nauc_precision_at_20_std value: -3.5411294612333744 - type: nauc_precision_at_3_diff1 value: 15.349623540592155 - type: nauc_precision_at_3_max value: 30.160697321707737 - type: nauc_precision_at_3_std value: -16.526565753263412 - type: nauc_precision_at_5_diff1 value: 11.487837439817703 - type: nauc_precision_at_5_max value: 33.00171032517383 - type: nauc_precision_at_5_std value: -13.576335609623799 - type: nauc_recall_at_1000_diff1 value: 50.71476253406417 - type: nauc_recall_at_1000_max value: 80.27518537738743 - type: nauc_recall_at_1000_std value: 51.88951079114522 - type: nauc_recall_at_100_diff1 value: 11.542640499848636 - type: nauc_recall_at_100_max value: 28.86012280016378 - type: nauc_recall_at_100_std value: 2.4903592627436635 - type: nauc_recall_at_10_diff1 value: 28.879631516382137 - type: nauc_recall_at_10_max value: 36.2244883366462 - type: nauc_recall_at_10_std value: -13.892468342124683 - type: nauc_recall_at_1_diff1 value: 48.21834981287912 - type: nauc_recall_at_1_max value: 19.568464844451327 - type: nauc_recall_at_1_std value: -17.84079220033572 - type: nauc_recall_at_20_diff1 value: 26.184388494939803 - type: nauc_recall_at_20_max value: 32.97413130761342 - type: nauc_recall_at_20_std value: -12.523995865516586 - type: nauc_recall_at_3_diff1 value: 32.6063576626463 - type: nauc_recall_at_3_max value: 28.665520067623607 - type: nauc_recall_at_3_std value: -18.890858269554286 - type: nauc_recall_at_5_diff1 value: 31.148210159300234 - type: nauc_recall_at_5_max value: 35.00176018739188 - type: nauc_recall_at_5_std value: -18.078139835750264 - type: ndcg_at_1 value: 39.768 - type: ndcg_at_10 value: 46.075 - type: ndcg_at_100 value: 53.417 - type: ndcg_at_1000 value: 55.7 - type: ndcg_at_20 value: 48.815 - type: ndcg_at_3 value: 40.271 - type: ndcg_at_5 value: 42.558 - type: precision_at_1 value: 39.768 - type: precision_at_10 value: 11.802 - type: precision_at_100 value: 1.797 - type: precision_at_1000 value: 0.21 - type: precision_at_20 value: 6.866 - type: precision_at_3 value: 26.512 - type: precision_at_5 value: 19.82 - type: recall_at_1 value: 22.926 - type: recall_at_10 value: 55.909 - type: recall_at_100 value: 84.758 - type: recall_at_1000 value: 99.7 - type: recall_at_20 value: 64.75800000000001 - type: recall_at_3 value: 39.123000000000005 - type: recall_at_5 value: 46.861999999999995 task: type: Retrieval --- # Model Card for KartonBERT-USE-base-v1 This universal sentence encoder model is designed to convert text content into a 768-float vector space, ensuring an effective representation. It aims to be proficient in tasks involving sentence / document similarity. Despite its small size (104 million parameters only), the model maintains a high level of performance. It uses a lowercase-optimized tokenizer with a vocabulary size of 23,000 tokens. This balance between compactness and effectiveness allows the model to deliver strong results in text encoding tasks, ensuring both speed and accuracy in real-time applications. ## Model Description - **Developed by:** Bartłomiej Orlik, https://www.linkedin.com/in/bartłomiej-orlik/ - **Model type:** BERT Universal Sentence Encoder - **Language:** Polish - **License:** GPL-3.0 - **Trained from model:** OrlikB/KartonBERT_base_uncased_v1: https://huggingface.co/OrlikB/KartonBERT_base_uncased_v1 ## How to Get Started with the Model Use the code below to get started with the model. ### Using Sentence-Transformers You can use the model with [sentence-transformers](https://www.SBERT.net): ``` pip install -U sentence-transformers ``` ```python from sentence_transformers import SentenceTransformer model = SentenceTransformer('OrlikB/KartonBERT-USE-base-v1') text_1 = 'Jestem wielkim fanem opakowań tekturowych' text_2 = 'Bardzo podobają mi się kartony' embeddings_1 = model.encode(text_1, normalize_embeddings=True) embeddings_2 = model.encode(text_2, normalize_embeddings=True) similarity = embeddings_1 @ embeddings_2.T print(similarity) ``` ### Using HuggingFace Transformers ```python from transformers import AutoTokenizer, AutoModel import torch import numpy as np def encode_text(text): encoded_input = tokenizer(text, padding=True, truncation=True, return_tensors='pt', max_length=512) with torch.no_grad(): model_output = model(**encoded_input) sentence_embeddings = model_output[0][:, 0] sentence_embeddings = torch.nn.functional.normalize(sentence_embeddings, p=2, dim=1) return sentence_embeddings.squeeze().numpy() cosine_similarity = lambda a, b: np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)) tokenizer = AutoTokenizer.from_pretrained('OrlikB/KartonBERT-USE-base-v1') model = AutoModel.from_pretrained('OrlikB/KartonBERT-USE-base-v1') model.eval() text_1 = 'Jestem wielkim fanem opakowań tekturowych' text_2 = 'Bardzo podobają mi się kartony' embeddings_1 = encode_text(text_1) embeddings_2 = encode_text(text_2) print(cosine_similarity(embeddings_1, embeddings_2)) ``` *Note: You can use the encode_text function for demonstration purposes. For the best experience, it's recommended to process text in batches. ## Evaluation [MTEB for Polish Language](https://huggingface.co/spaces/mteb/leaderboard) | Rank | Model | Model Size (Million Parameters) | Memory Usage (GB, fp32) | Embedding Dimensions | Max Tokens | Average (26 datasets) | Classification Average (7 datasets) | Clustering Average (1 dataset) | PairClassification Average (4 datasets) | Retrieval Average (11 datasets) | STS Average (3 datasets) | |------|-------------------------------------|---------------------------------|-------------------------|----------------------|------------|-----------------------|------------------------------------|---------------------------------|----------------------------------------|-----------------------------------|--------------------------| | 1 | bge-multilingual-gemma2 | 9242 | 34.43 | 3584 | 8192 | 70 | 77.99 | 50.29 | 89.62 | 59.41 | 70.64 | | 2 | gte-Qwen2-7B-instruct | 7613 | 28.36 | 3584 | 131072 | 67.86 | 77.84 | 51.36 | 88.48 | 54.69 | 70.86 | | 3 | gte-Qwen2-1.5B-instruct | 1776 | 6.62 | 1536 | 131072 | 64.04 | 72.29 | 44.59 | 84.87 | 51.88 | 68.12 | | 4 | jina-embeddings-v3 | 572 | 2.13 | 1024 | 8194 | 63.97 | 70.81 | 43.66 | 83.70 | 51.89 | 72.77 | | 5 | jina-embeddings-v3 | 572 | 2.13 | 1024 | 8194 | 63.97 | 70.81 | 43.66 | 83.70 | 51.89 | 72.77 | | 6 | mmlw-roberta-large | 435 | 1.62 | 1024 | 514 | 63.23 | 66.39 | 31.16 | 89.13 | 52.71 | 70.59 | | 7 | **KartonBERT-USE-base-v1** | **104** | **0.39** | **768** | 512 | 61.67 | 67.57 | 29.88 | 87.04 | 49.14 | 70.65 | | 8 | mmlw-e5-large | 560 | 2.09 | 1024 | 514 | 61.17 | 61.07 | 30.62 | 85.90 | 52.63 | 69.98 | | 9 | mmlw-roberta-base | 124 | 0.46 | 768 | 514 | 61.05 | 62.92 | 33.08 | 88.14 | 49.92 | 70.70 | | 10 | multilingual-e5-large | 560 | 2.09 | 1024 | 514 | 60.08 | 63.82 | 33.88 | 85.50 | 48.98 | 66.91 | | 11 | mmlw-e5-base | 278 | 1.04 | 768 | 514 | 59.71 | 59.52 | 30.25 | 86.16 | 50.06 | 70.13 | | 12 | gte-multilingual-base | 305 | 1.14 | 768 | 8192 | 58.22 | 60.15 | 33.67 | 85.45 | 46.40 | 68.92 | | 13 | st-polish-kartonberta-base-alpha-v1 | 124 | 0.46 | 768 | 514 | 56.92 | 60.44 | 32.85 | 87.92 | 42.19 | 69.47 | ## More Information If I have spare computing resources (GPU), I may improve the quality of the model by further training.