siddartha-abacus commited on
Commit
39d5b3d
1 Parent(s): fbb849f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +129 -0
README.md CHANGED
@@ -27,6 +27,18 @@ We are currently working on writing up this new technique in the form of a techn
27
  | --- | --- | --- | --- | --- | --- | --- |
28
  | 80.48 | 76.02 | 89.27 | 77.15 | 76.67 | 85.08 | 78.70 |
29
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  ### Contamination Results
31
 
32
  We generate our contamination numbers using https://github.com/swj0419/detect-pretrain-code-contamination/tree/master, with Llama7B as our reference model.
@@ -47,3 +59,120 @@ Note that GSM8K often scores very highly on this contamination suite - we verifi
47
  | ARC | TruthfulQA | GSM8K |
48
  | --- | --- | --- |
49
  | 0.22| 0.51| 0.89|
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  | --- | --- | --- | --- | --- | --- | --- |
28
  | 80.48 | 76.02 | 89.27 | 77.15 | 76.67 | 85.08 | 78.70 |
29
 
30
+ #### MT-Bench
31
+
32
+ We ran MT-Bench with the llama-2 conversation template and the system prompt set to the Qwen system prompt. We got the following
33
+ results in single model mode:
34
+
35
+ | First Turn | Second Turn | Average |
36
+ | ---------- | ----------- | ------- |
37
+ | 8.18 | 7.34 | 7.76 |
38
+
39
+ The bottom of the card includes some sample responses from the model. We have selected responses that have high and low scores
40
+ from the GPT-4 judge.
41
+
42
  ### Contamination Results
43
 
44
  We generate our contamination numbers using https://github.com/swj0419/detect-pretrain-code-contamination/tree/master, with Llama7B as our reference model.
 
59
  | ARC | TruthfulQA | GSM8K |
60
  | --- | --- | --- |
61
  | 0.22| 0.51| 0.89|
62
+
63
+
64
+ ### Sample MT Bench results
65
+
66
+ #### Example 1
67
+ **Question**
68
+
69
+ Imagine you are writing a blog post comparing two popular smartphone models.
70
+ Develop an outline for the blog post, including key points and subheadings to effectively
71
+ compare and contrast the features, performance, and user experience of the two models.
72
+ Please answer in fewer than 200 words.
73
+
74
+ **Response**
75
+
76
+ Outline:
77
+
78
+ 1. Introduction to the two smartphone models being compared (mention brand names and models)
79
+ 2. Design and Build Quality
80
+ - Comparison of dimensions, weight, materials used, and overall aesthetic appeal
81
+ - Water and dust resistance, if applicable
82
+ 3. Display and Screen Quality
83
+ - Comparison of screen technology (OLED, IPS LCD, etc.)
84
+ - Display resolution, refresh rate, and brightness levels
85
+ 4. Performance and Hardware
86
+ - Processor, RAM, and storage configurations
87
+ - Comparison of benchmark scores (e.g., AnTuTu, Geekbench)
88
+ 5. Camera System and Image Quality
89
+ - Comparison of primary, ultra-wide, telephoto, and macro lenses (if available)
90
+ - Comparison of low-light performance and image processing capabilities
91
+ 6. Battery Life and Charging
92
+ - Battery capacity and estimated usage time
93
+ - Charging technologies (wired, wireless, fast charging, reverse charging)
94
+ 7. Software and User Experience
95
+ - Comparison of operating systems (iOS vs. Android) and their features
96
+ - User interface, customization options, and app ecosystem
97
+ 8. Additional Features and Extras
98
+ - Comparison of biometric authentication (fingerprint, face unlock, etc.)
99
+ - Additional features like NFC, 5G connectivity, and audio quality
100
+ 9. Pricing and Value for Money
101
+ - Comparison of retail prices and current offers
102
+ - Assessment of overall value considering the features and performance
103
+ 10. Conclusion and recommendation for potential buyers based on their preferences
104
+
105
+ #### Example 2
106
+ **Question**
107
+
108
+ In a survey conducted at a local high school, preferences for a new school color were measured:
109
+ 58% of students liked the color blue, 45% preferred green, and 22% liked both colors.
110
+ If we randomly pick a student from the school, what's the probability that they would like neither blue nor green?
111
+
112
+ **Answer**
113
+
114
+ To find the probability that a randomly picked student would like neither blue nor green, we need to find the percentage of students who like only one color or neither color.
115
+
116
+ From the given information, we know:
117
+ - 58% like blue
118
+ - 45% prefer green
119
+ - 22% like both colors
120
+
121
+ To find the percentage of students who like only one color, we can subtract the percentage of students who like both colors from the sum of students who like blue and green:
122
+ - 58% (blue) + 45% (green) - 22% (both) = 83%
123
+
124
+ Now, to find the probability of a student liking neither blue nor green, we subtract the percentage of students who like at least one color from 100%:
125
+ - 100% - 83% = 17%
126
+
127
+ So, the probability that a randomly picked student would like neither blue nor green is 17%.
128
+
129
+ #### Example 3
130
+ **Question**
131
+
132
+ Implement a program to find the common elements in two arrays without using any extra data structures.
133
+
134
+ **Answer**
135
+
136
+ Finding common elements in two arrays without using extra data structures can be achieved by using a simple algorithm. Here's a Python implementation of this problem:
137
+
138
+ ```python
139
+ def find_common_elements(arr1, arr2):
140
+ common_elements = []
141
+ for num1 in arr1:
142
+ if num1 in arr2 and num1 not in common_elements:
143
+ common_elements.append(num1)
144
+ return common_elements
145
+
146
+ # Example usage:
147
+ array1 = [1, 2, 3, 4, 5]
148
+ array2 = [4, 5, 6, 7, 8]
149
+ print(find_common_elements(array1, array2)) # Output: [4, 5]
150
+ ```
151
+
152
+ This program iterates through the elements of the first array (arr1) and checks if each element is present in the second array (arr2). If it is, and the element hasn't been added to the common_elements list yet, it is appended. The function then returns the list of common elements.
153
+
154
+ #### Example 4
155
+
156
+ **Question**
157
+
158
+ Identify the named entities (people, organizations, locations) mentioned in the given news article.
159
+ Please generate a JSON dictionary that lists the named entities in three separate groups based on their entity types.
160
+ The key is the type of entity and the value is a list of strings.
161
+
162
+ Yesterday, Adamson Emerson, the CEO of Faraday, and Dieter Zetsche, the CEO of Daimler AG, announced plans to
163
+ build a new Gigafactory in Berlin. The facility will be a joint venture between Faraday and Daimler, producing
164
+ electric vehicles and battery packs for both companies, creating thousands of job opportunities in the region.
165
+ Emerson and Zetsche stated that the strategic location of Berlin, coupled with its skilled workforce and strong infrastructure,
166
+ makes it an ideal choice for expansion. The new Gigafactory aims to meet the growing demand for electric vehicles in Europe
167
+ and contribute to a sustainable future. Volkswagen CEO Herbert Diess welcomed the news, saying greater collaboration will,
168
+ benefit the auto industry's transition to e-mobility.
169
+
170
+ **Answer**
171
+
172
+ ```
173
+ {
174
+ "people": ["Adamson Emerson", "Dieter Zetsche", "Herbert Diess"],
175
+ "organizations": ["Faraday", "Daimler AG", "Volkswagen"],
176
+ "locations": ["Berlin", "Europe"]
177
+ }
178
+ ```