T145 commited on
Commit
5d26bf1
1 Parent(s): 6d1b385

Adding Evaluation Results

Browse files

This is an automated PR created with [this space](https://huggingface.co/spaces/T145/open-llm-leaderboard-results-to-modelcard)!

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

Please report any issues here: https://huggingface.co/spaces/T145/open-llm-leaderboard-results-to-modelcard/discussions

Files changed (1) hide show
  1. README.md +137 -23
README.md CHANGED
@@ -1,35 +1,134 @@
1
  ---
2
  base_model: ghost-x/ghost-8b-beta
3
  language:
4
- - vi
5
- - ko
6
- - es
7
- - pt
8
- - zh
9
- - fr
10
- - it
11
- - de
12
- - ja
13
- - ru
14
- - pl
15
- - nl
16
- - hi
17
- - tr
18
- - id
19
- - en
20
  license: other
21
  license_name: ghost-open-llms
22
  license_link: https://ghost-x.org/ghost-open-llms-license
23
  tags:
24
- - ghost
25
- - tools
26
- - chat
27
- - transformers
28
- - unsloth
29
- - llama
30
  pipeline_tag: text-generation
31
  widget:
32
- - text: Why is the sky blue ?
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  ---
34
 
35
  <p><img src="https://ghost-x.org/docs/models/ghost-8b-beta/images/logo.jpeg" width="40%" align="center" /></p>
@@ -830,3 +929,18 @@ If you find our work helpful, feel free to give us a cite.
830
  year = {2024}
831
  }
832
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  base_model: ghost-x/ghost-8b-beta
3
  language:
4
+ - vi
5
+ - ko
6
+ - es
7
+ - pt
8
+ - zh
9
+ - fr
10
+ - it
11
+ - de
12
+ - ja
13
+ - ru
14
+ - pl
15
+ - nl
16
+ - hi
17
+ - tr
18
+ - id
19
+ - en
20
  license: other
21
  license_name: ghost-open-llms
22
  license_link: https://ghost-x.org/ghost-open-llms-license
23
  tags:
24
+ - ghost
25
+ - tools
26
+ - chat
27
+ - transformers
28
+ - unsloth
29
+ - llama
30
  pipeline_tag: text-generation
31
  widget:
32
+ - text: Why is the sky blue ?
33
+ model-index:
34
+ - name: ghost-8b-beta-1608
35
+ results:
36
+ - task:
37
+ type: text-generation
38
+ name: Text Generation
39
+ dataset:
40
+ name: IFEval (0-Shot)
41
+ type: wis-k/instruction-following-eval
42
+ split: train
43
+ args:
44
+ num_few_shot: 0
45
+ metrics:
46
+ - type: inst_level_strict_acc and prompt_level_strict_acc
47
+ value: 42.73
48
+ name: averaged accuracy
49
+ source:
50
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=ghost-x%2Fghost-8b-beta-1608
51
+ name: Open LLM Leaderboard
52
+ - task:
53
+ type: text-generation
54
+ name: Text Generation
55
+ dataset:
56
+ name: BBH (3-Shot)
57
+ type: SaylorTwift/bbh
58
+ split: test
59
+ args:
60
+ num_few_shot: 3
61
+ metrics:
62
+ - type: acc_norm
63
+ value: 23.46
64
+ name: normalized accuracy
65
+ source:
66
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=ghost-x%2Fghost-8b-beta-1608
67
+ name: Open LLM Leaderboard
68
+ - task:
69
+ type: text-generation
70
+ name: Text Generation
71
+ dataset:
72
+ name: MATH Lvl 5 (4-Shot)
73
+ type: lighteval/MATH-Hard
74
+ split: test
75
+ args:
76
+ num_few_shot: 4
77
+ metrics:
78
+ - type: exact_match
79
+ value: 1.28
80
+ name: exact match
81
+ source:
82
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=ghost-x%2Fghost-8b-beta-1608
83
+ name: Open LLM Leaderboard
84
+ - task:
85
+ type: text-generation
86
+ name: Text Generation
87
+ dataset:
88
+ name: GPQA (0-shot)
89
+ type: Idavidrein/gpqa
90
+ split: train
91
+ args:
92
+ num_few_shot: 0
93
+ metrics:
94
+ - type: acc_norm
95
+ value: 1.12
96
+ name: acc_norm
97
+ source:
98
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=ghost-x%2Fghost-8b-beta-1608
99
+ name: Open LLM Leaderboard
100
+ - task:
101
+ type: text-generation
102
+ name: Text Generation
103
+ dataset:
104
+ name: MuSR (0-shot)
105
+ type: TAUR-Lab/MuSR
106
+ args:
107
+ num_few_shot: 0
108
+ metrics:
109
+ - type: acc_norm
110
+ value: 1.58
111
+ name: acc_norm
112
+ source:
113
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=ghost-x%2Fghost-8b-beta-1608
114
+ name: Open LLM Leaderboard
115
+ - task:
116
+ type: text-generation
117
+ name: Text Generation
118
+ dataset:
119
+ name: MMLU-PRO (5-shot)
120
+ type: TIGER-Lab/MMLU-Pro
121
+ config: main
122
+ split: test
123
+ args:
124
+ num_few_shot: 5
125
+ metrics:
126
+ - type: acc
127
+ value: 20.44
128
+ name: accuracy
129
+ source:
130
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=ghost-x%2Fghost-8b-beta-1608
131
+ name: Open LLM Leaderboard
132
  ---
133
 
134
  <p><img src="https://ghost-x.org/docs/models/ghost-8b-beta/images/logo.jpeg" width="40%" align="center" /></p>
 
929
  year = {2024}
930
  }
931
  ```
932
+
933
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
934
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/ghost-x__ghost-8b-beta-1608-details)!
935
+ Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=ghost-x%2Fghost-8b-beta-1608&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
936
+
937
+ | Metric |Value (%)|
938
+ |-------------------|--------:|
939
+ |**Average** | 15.10|
940
+ |IFEval (0-Shot) | 42.73|
941
+ |BBH (3-Shot) | 23.46|
942
+ |MATH Lvl 5 (4-Shot)| 1.28|
943
+ |GPQA (0-shot) | 1.12|
944
+ |MuSR (0-shot) | 1.58|
945
+ |MMLU-PRO (5-shot) | 20.44|
946
+