Spaces:

mteb

/

leaderboard

like 4.2k

Running on CPU Upgrade

Hi cheebw1, all results can be found here. We are also currently working on developing code to more easily access the results. In a future version of mteb it will be available as something like:

# this is not yet implemented
import mteb
results = mteb.load_results() # downloads and loads in results using MTEBResults

# format will be:
results: dict[MODEL_NAME_STR, dict[REVISION_STR, list[MTEBResult]]

We will likely also add utilities for converting this format to e.g. a data frame

Adding to @KennethEnevoldsen great reply, we also plan to change the leaderboard soon to look something like this: https://github.com/embeddings-benchmark/mteb/discussions/674#discussioncomment-9404987 which would allow you to create custom rankings based on specific datasets. Help on this is very welcome if you are interested :)