Nekochu commited on
Commit
463ea77
1 Parent(s): 4b80796

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -181,7 +181,7 @@ Note: Output from inference [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Fac
181
  </details>
182
 
183
  <details>
184
- <summary>Eval</summary>
185
 
186
  [MMLU-Pro](https://github.com/chigkim/Ollama-MMLU-Pro)[*](https://pastebin.com/a8xRqXtg) (en):
187
  | Model | Overall Accuracy | biology | business | chemistry | computer science | economics | engineering | health | history | law | math | philosophy | physics | psychology | other |
@@ -189,5 +189,5 @@ Note: Output from inference [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Fac
189
  | Llama-3.1-8B-German-ORPO-8.0bpw-h8-exl2 | 38.83 | 60.81 | 37.26 | 32.86 | 38.78 | 46.33 | 23.32 | 45.48 | 39.90 | 21.62 | 38.86 | 34.67 | 28.79 | 50.63 | 44.26 |
190
  | Llama-3.1-8B-Instruct-exl2-8bpw-h8 | 46.16 | 63.74 | 49.68 | 36.93 | 48.29 | 55.81 | 28.59 | 52.81 | 45.67 | 30.79 | 45.08 | 40.48 | 39.03 | 60.90 | 48.38 |
191
 
192
- Note: Lower on Benchmark for **English**, en seems to be degraded as trade-off. Not frequently but the output repeats sentences (because of the wrong chat template).
193
  </details>
 
181
  </details>
182
 
183
  <details>
184
+ <summary>Eval English</summary>
185
 
186
  [MMLU-Pro](https://github.com/chigkim/Ollama-MMLU-Pro)[*](https://pastebin.com/a8xRqXtg) (en):
187
  | Model | Overall Accuracy | biology | business | chemistry | computer science | economics | engineering | health | history | law | math | philosophy | physics | psychology | other |
 
189
  | Llama-3.1-8B-German-ORPO-8.0bpw-h8-exl2 | 38.83 | 60.81 | 37.26 | 32.86 | 38.78 | 46.33 | 23.32 | 45.48 | 39.90 | 21.62 | 38.86 | 34.67 | 28.79 | 50.63 | 44.26 |
190
  | Llama-3.1-8B-Instruct-exl2-8bpw-h8 | 46.16 | 63.74 | 49.68 | 36.93 | 48.29 | 55.81 | 28.59 | 52.81 | 45.67 | 30.79 | 45.08 | 40.48 | 39.03 | 60.90 | 48.38 |
191
 
192
+ Note: Lower on Benchmark for **English**, seems to be degraded as trade-off. Not frequently but the output repeats sentences (because of the wrong chat template).
193
  </details>