Add Mteb table
Browse files
README.md
CHANGED
|
@@ -16937,7 +16937,25 @@ print(cosine_similarity(embeddings_1, embeddings_2))
|
|
| 16937 |
|
| 16938 |
|
| 16939 |
## Evaluation
|
| 16940 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16941 |
|
| 16942 |
|
| 16943 |
|
|
|
|
| 16937 |
|
| 16938 |
|
| 16939 |
## Evaluation
|
| 16940 |
+
|
| 16941 |
+
[MTEB for Polish Language](https://huggingface.co/spaces/mteb/leaderboard)
|
| 16942 |
+
|
| 16943 |
+
| Rank | Model | Model Size (Million Parameters) | Memory Usage (GB, fp32) | Embedding Dimensions | Max Tokens | Average (26 datasets) | Classification Average (7 datasets) | Clustering Average (1 dataset) | PairClassification Average (4 datasets) | Retrieval Average (11 datasets) | STS Average (3 datasets) |
|
| 16944 |
+
|------|-------------------------------------|---------------------------------|-------------------------|----------------------|------------|-----------------------|------------------------------------|---------------------------------|----------------------------------------|-----------------------------------|--------------------------|
|
| 16945 |
+
| 1 | bge-multilingual-gemma2 | 9242 | 34.43 | 3584 | 8192 | 70 | 77.99 | 50.29 | 89.62 | 59.41 | 70.64 |
|
| 16946 |
+
| 2 | gte-Qwen2-7B-instruct | 7613 | 28.36 | 3584 | 131072 | 67.86 | 77.84 | 51.36 | 88.48 | 54.69 | 70.86 |
|
| 16947 |
+
| 3 | gte-Qwen2-1.5B-instruct | 1776 | 6.62 | 1536 | 131072 | 64.04 | 72.29 | 44.59 | 84.87 | 51.88 | 68.12 |
|
| 16948 |
+
| 4 | jina-embeddings-v3 | 572 | 2.13 | 1024 | 8194 | 63.97 | 70.81 | 43.66 | 83.70 | 51.89 | 72.77 |
|
| 16949 |
+
| 5 | jina-embeddings-v3 | 572 | 2.13 | 1024 | 8194 | 63.97 | 70.81 | 43.66 | 83.70 | 51.89 | 72.77 |
|
| 16950 |
+
| 6 | mmlw-roberta-large | 435 | 1.62 | 1024 | 514 | 63.23 | 66.39 | 31.16 | 89.13 | 52.71 | 70.59 |
|
| 16951 |
+
| 7 | **KartonBERT-USE-base-v1** | **104** | **0.39** | **768** | 512 | 61.67 | 67.57 | 29.88 | 87.04 | 49.14 | 70.65 |
|
| 16952 |
+
| 8 | mmlw-e5-large | 560 | 2.09 | 1024 | 514 | 61.17 | 61.07 | 30.62 | 85.90 | 52.63 | 69.98 |
|
| 16953 |
+
| 9 | mmlw-roberta-base | 124 | 0.46 | 768 | 514 | 61.05 | 62.92 | 33.08 | 88.14 | 49.92 | 70.70 |
|
| 16954 |
+
| 10 | multilingual-e5-large | 560 | 2.09 | 1024 | 514 | 60.08 | 63.82 | 33.88 | 85.50 | 48.98 | 66.91 |
|
| 16955 |
+
| 11 | mmlw-e5-base | 278 | 1.04 | 768 | 514 | 59.71 | 59.52 | 30.25 | 86.16 | 50.06 | 70.13 |
|
| 16956 |
+
| 12 | gte-multilingual-base | 305 | 1.14 | 768 | 8192 | 58.22 | 60.15 | 33.67 | 85.45 | 46.40 | 68.92 |
|
| 16957 |
+
| 13 | st-polish-kartonberta-base-alpha-v1 | 124 | 0.46 | 768 | 514 | 56.92 | 60.44 | 32.85 | 87.92 | 42.19 | 69.47 |
|
| 16958 |
+
|
| 16959 |
|
| 16960 |
|
| 16961 |
|