mattshumer clem HF staff commited on
Commit
0d5ac5b
•
1 Parent(s): 5d2d9db

update the model card to reflect the non-reproducibility of benchmarks (#74)

Browse files

- update the model card to reflect the non-reproducibility of benchmarks (079ecaacd3b70854e46de6e01d2d846704976c27)


Co-authored-by: Clem 🤗 <clem@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +1 -4
README.md CHANGED
@@ -8,14 +8,11 @@ library_name: transformers
8
 
9
  | IMPORTANT — This is the new, working version of the Reflection Llama 3.1 70B model. Please use this version.
10
 
11
- **Reflection Llama-3.1 70B is (currently) the world's top open-source LLM, trained with a new technique called Reflection-Tuning that teaches a LLM to detect mistakes in its reasoning and correct course.**
12
 
13
  The model was trained on synthetic data generated by [Glaive](https://glaive.ai). If you're training a model, Glaive is incredible — use them.
14
 
15
  ## Benchmarks
16
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/60518f3731c5be7f3dd5ebc3/zNs-ZFs0SbnomH7mikiOU.png)
17
-
18
- All benchmarks tested have been checked for contamination by running [LMSys's LLM Decontaminator](https://github.com/lm-sys/llm-decontaminator). When benchmarking, we isolate the `<output>` and benchmark on solely that section.
19
 
20
  Trained from Llama 3.1 70B Instruct, you can sample from Reflection Llama-3.1 70B using the same code, pipelines, etc. as any other Llama model. It even uses the stock Llama 3.1 chat template format (though, we've trained in a few new special tokens to aid in reasoning and reflection).
21
 
 
8
 
9
  | IMPORTANT — This is the new, working version of the Reflection Llama 3.1 70B model. Please use this version.
10
 
11
+ **Reflection Llama-3.1 70B is an open-source LLM, trained with a new technique called Reflection-Tuning that teaches a LLM to detect mistakes in its reasoning and correct course.**
12
 
13
  The model was trained on synthetic data generated by [Glaive](https://glaive.ai). If you're training a model, Glaive is incredible — use them.
14
 
15
  ## Benchmarks
 
 
 
16
 
17
  Trained from Llama 3.1 70B Instruct, you can sample from Reflection Llama-3.1 70B using the same code, pipelines, etc. as any other Llama model. It even uses the stock Llama 3.1 chat template format (though, we've trained in a few new special tokens to aid in reasoning and reflection).
18