Quentin Gallouédec commited on
Commit
212d3f4
1 Parent(s): 74e3b17
Files changed (2) hide show
  1. README.md +2 -29
  2. src/envs.py +1 -1
README.md CHANGED
@@ -1,38 +1,11 @@
1
  ---
2
- title: Backend
3
  emoji: 🥇
4
  colorFrom: green
5
  colorTo: indigo
6
  sdk: gradio
7
- sdk_version: 4.4.0 #19.2
8
  app_file: app.py
9
  pinned: true
10
  license: apache-2.0
11
  ---
12
-
13
- Most of the variables to change for a default leaderboard are in src/env (replace the path for your leaderboard) and src/about.
14
-
15
- Results files should have the following format:
16
- ```
17
- {
18
- "config": {
19
- "model_dtype": "torch.float16", # or torch.bfloat16 or 8bit or 4bit
20
- "model_name": "path of the model on the hub: org/model",
21
- "model_sha": "revision on the hub",
22
- },
23
- "results": {
24
- "task_name": {
25
- "metric_name": score,
26
- },
27
- "task_name2": {
28
- "metric_name": score,
29
- }
30
- }
31
- }
32
- ```
33
-
34
- Request files are created automatically by this tool.
35
-
36
- If you encounter problem on the space, don't hesitate to restart it to remove the create eval-queue, eval-queue-bk, eval-results and eval-results-bk created folder.
37
-
38
- If you want to run your own backend, you only need to change the logic in src/backend/run_eval_suite_..., which at the moment launches the Eleuther AI Harness or Lighteval, and edit the app.py to point to the correct file.
 
1
  ---
2
+ title: Open RL Leaderboard
3
  emoji: 🥇
4
  colorFrom: green
5
  colorTo: indigo
6
  sdk: gradio
7
+ sdk_version: 4.20.0
8
  app_file: app.py
9
  pinned: true
10
  license: apache-2.0
11
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
src/envs.py CHANGED
@@ -18,7 +18,7 @@ REGION = "us-east-1"
18
  VENDOR = "aws"
19
  # ----------------------------------
20
 
21
- REPO_ID = f"{OWNER}/backend"
22
  RESULTS_REPO = f"{OWNER}/results"
23
 
24
  # If you setup a cache later, just change HF_HOME
 
18
  VENDOR = "aws"
19
  # ----------------------------------
20
 
21
+ REPO_ID = f"{OWNER}/leaderboard"
22
  RESULTS_REPO = f"{OWNER}/results"
23
 
24
  # If you setup a cache later, just change HF_HOME