Files changed (1) hide show
  1. README.md +117 -1
README.md CHANGED
@@ -34,6 +34,109 @@ language:
34
  license: mit
35
  thumbnail: https://huggingface.co/front/thumbnails/facebook.png
36
  inference: false
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  ---
38
 
39
  # XGLM-564M
@@ -171,4 +274,17 @@ for lang in data_samples_long:
171
  # zh-1 0 0
172
  # hi-0 1 1
173
  # hi-1 0 0
174
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
  license: mit
35
  thumbnail: https://huggingface.co/front/thumbnails/facebook.png
36
  inference: false
37
+ model-index:
38
+ - name: xglm-564M
39
+ results:
40
+ - task:
41
+ type: text-generation
42
+ name: Text Generation
43
+ dataset:
44
+ name: AI2 Reasoning Challenge (25-Shot)
45
+ type: ai2_arc
46
+ config: ARC-Challenge
47
+ split: test
48
+ args:
49
+ num_few_shot: 25
50
+ metrics:
51
+ - type: acc_norm
52
+ value: 24.57
53
+ name: normalized accuracy
54
+ source:
55
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=facebook/xglm-564M
56
+ name: Open LLM Leaderboard
57
+ - task:
58
+ type: text-generation
59
+ name: Text Generation
60
+ dataset:
61
+ name: HellaSwag (10-Shot)
62
+ type: hellaswag
63
+ split: validation
64
+ args:
65
+ num_few_shot: 10
66
+ metrics:
67
+ - type: acc_norm
68
+ value: 34.64
69
+ name: normalized accuracy
70
+ source:
71
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=facebook/xglm-564M
72
+ name: Open LLM Leaderboard
73
+ - task:
74
+ type: text-generation
75
+ name: Text Generation
76
+ dataset:
77
+ name: MMLU (5-Shot)
78
+ type: cais/mmlu
79
+ config: all
80
+ split: test
81
+ args:
82
+ num_few_shot: 5
83
+ metrics:
84
+ - type: acc
85
+ value: 25.18
86
+ name: accuracy
87
+ source:
88
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=facebook/xglm-564M
89
+ name: Open LLM Leaderboard
90
+ - task:
91
+ type: text-generation
92
+ name: Text Generation
93
+ dataset:
94
+ name: TruthfulQA (0-shot)
95
+ type: truthful_qa
96
+ config: multiple_choice
97
+ split: validation
98
+ args:
99
+ num_few_shot: 0
100
+ metrics:
101
+ - type: mc2
102
+ value: 40.43
103
+ source:
104
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=facebook/xglm-564M
105
+ name: Open LLM Leaderboard
106
+ - task:
107
+ type: text-generation
108
+ name: Text Generation
109
+ dataset:
110
+ name: Winogrande (5-shot)
111
+ type: winogrande
112
+ config: winogrande_xl
113
+ split: validation
114
+ args:
115
+ num_few_shot: 5
116
+ metrics:
117
+ - type: acc
118
+ value: 52.25
119
+ name: accuracy
120
+ source:
121
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=facebook/xglm-564M
122
+ name: Open LLM Leaderboard
123
+ - task:
124
+ type: text-generation
125
+ name: Text Generation
126
+ dataset:
127
+ name: GSM8k (5-shot)
128
+ type: gsm8k
129
+ config: main
130
+ split: test
131
+ args:
132
+ num_few_shot: 5
133
+ metrics:
134
+ - type: acc
135
+ value: 0.23
136
+ name: accuracy
137
+ source:
138
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=facebook/xglm-564M
139
+ name: Open LLM Leaderboard
140
  ---
141
 
142
  # XGLM-564M
 
274
  # zh-1 0 0
275
  # hi-0 1 1
276
  # hi-1 0 0
277
+ ```
278
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
279
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_facebook__xglm-564M)
280
+
281
+ | Metric |Value|
282
+ |---------------------------------|----:|
283
+ |Avg. |29.55|
284
+ |AI2 Reasoning Challenge (25-Shot)|24.57|
285
+ |HellaSwag (10-Shot) |34.64|
286
+ |MMLU (5-Shot) |25.18|
287
+ |TruthfulQA (0-shot) |40.43|
288
+ |Winogrande (5-shot) |52.25|
289
+ |GSM8k (5-shot) | 0.23|
290
+