README.md

---
title: LeaderboardFinder
emoji: 🐢
colorFrom: pink
colorTo: gray
sdk: gradio
sdk_version: 4.22.0
app_file: app.py
pinned: false
---

If you want your leaderboard to appear, feel free to add relevant information in its metadata, and it will be displayed here.

# Categories

## Submission type
Arenas are not concerned by this category.

- `submission:automatic`: users can submit their models as such to the leaderboard, and evaluation is run automatically without human intervention
- `submission:semiautomatic`: the leaderboard requires the model owner to run evaluations on his side and submit the results
- `submission:manual`: the leaderboard requires the leaderboard owner to run evaluations for new submissions
- `submission:closed`: the leaderboard does not accept submissions at the moment

## Test set status
Arenas are not concerned by this category.

- `test:public`: all the test sets used are public, the evaluations are completely reproducible
- `test:mix`: some test sets are public and some private
- `test:private`: all the test sets used are private, the evaluations are hard to game
- `test:rolling`: the test sets used change regularly through time and evaluation scores are refreshed

## Judges
- `judge:auto`: evaluations are run automatically, using an evaluation suite such as `lm_eval` or `lighteval`
- `judge:model`: evaluations are run using a model as a judge approach to rate answer
- `judge:humans`: evaluations are done by humans to rate answer - this is an arena
- `judge:vibe_check`: evaluations are done manually by one human

## Modalities
Can be any (or several) of the following list:
- `modality:text`
- `modality:image`
- `modality:video`
- `modality:audio`
A bit outside of usual modalities
- `modality:tools`: requires added tool usage - mostly for assistant models
- `modality:artefacts`: the leaderboard concerns itself with machine learning artefacts as themselves, for example, quality evaluation of text embeddings.

## Evaluation categories
Can be any (or several) of the following list:
- `eval:generation`: the evaluation looks at generation capabilities specifically (can be image generation, text generation, ...) 
- `eval:math`
- `eval:code`
- `eval:performance`: model performance (speed, energy consumption, ...)
- `eval:safety`: safety, toxicity, bias evaluations

## Language
You can indicate the languages covered by your benchmark like so: `language:mylanguage`. 
At the moment, we do not support language codes, please use the language name in English.