mgoin commited on
Commit
ed07c8f
1 Parent(s): ca4faba

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +93 -0
README.md CHANGED
@@ -1,6 +1,99 @@
1
  ---
2
  license: apache-2.0
3
  base_model: mistralai/Mistral-7B-Instruct-v0.3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  ---
5
 
6
  # Model Card for Mistral-7B-Instruct-v0.3 quantized to 4bit weights
 
1
  ---
2
  license: apache-2.0
3
  base_model: mistralai/Mistral-7B-Instruct-v0.3
4
+
5
+ model-index:
6
+ - name: Mistral-7B-Instruct-v0.3-GPTQ-4bit
7
+ results:
8
+ # AI2 Reasoning Challenge (25-Shot)
9
+ - task:
10
+ type: text-generation
11
+ name: Text Generation
12
+ dataset:
13
+ name: AI2 Reasoning Challenge (25-Shot)
14
+ type: ai2_arc
15
+ config: ARC-Challenge
16
+ split: test
17
+ args:
18
+ num_few_shot: 25
19
+ metrics:
20
+ - type: acc_norm
21
+ name: normalized accuracy
22
+ value: 63.40
23
+ # HellaSwag (10-shot)
24
+ - task:
25
+ type: text-generation
26
+ name: Text Generation
27
+ dataset:
28
+ name: HellaSwag (10-Shot)
29
+ type: hellaswag
30
+ split: validation
31
+ args:
32
+ num_few_shot: 10
33
+ metrics:
34
+ - type: acc_norm
35
+ name: normalized accuracy
36
+ value: 84.04
37
+ # TruthfulQA (0-shot)
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: TruthfulQA (0-shot)
43
+ type: truthful_qa
44
+ config: multiple_choice
45
+ split: validation
46
+ args:
47
+ num_few_shot: 0
48
+ metrics:
49
+ - type: mc2
50
+ value: 57.48
51
+ # GSM8k (5-shot)
52
+ - task:
53
+ type: text-generation
54
+ name: Text Generation
55
+ dataset:
56
+ name: GSM8k (5-shot)
57
+ type: gsm8k
58
+ config: main
59
+ split: test
60
+ args:
61
+ num_few_shot: 5
62
+ metrics:
63
+ - type: acc
64
+ name: accuracy
65
+ value: 45.41
66
+ # MMLU (5-Shot)
67
+ - task:
68
+ type: text-generation
69
+ name: Text Generation
70
+ dataset:
71
+ name: MMLU (5-Shot)
72
+ type: cais/mmlu
73
+ config: all
74
+ split: test
75
+ args:
76
+ num_few_shot: 5
77
+ metrics:
78
+ - type: acc
79
+ name: accuracy
80
+ value: 61.07
81
+ # Winogrande (5-shot)
82
+ - task:
83
+ type: text-generation
84
+ name: Text Generation
85
+ dataset:
86
+ name: Winogrande (5-shot)
87
+ type: winogrande
88
+ config: winogrande_xl
89
+ split: validation
90
+ args:
91
+ num_few_shot: 5
92
+ metrics:
93
+ - type: acc
94
+ name: accuracy
95
+ value: 79.08
96
+
97
  ---
98
 
99
  # Model Card for Mistral-7B-Instruct-v0.3 quantized to 4bit weights