Commit
•
b58c889
1
Parent(s):
6aaaf33
Update README.md
Browse files
README.md
CHANGED
@@ -160,8 +160,57 @@ The seed for this merge is 176
|
|
160 |
### Models Merged
|
161 |
|
162 |
The following models were included in the merge:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
163 |
dtype: bfloat16
|
164 |
|
|
|
|
|
|
|
165 |
|
166 |
|
167 |
### Table of Benchmarks
|
|
|
160 |
### Models Merged
|
161 |
|
162 |
The following models were included in the merge:
|
163 |
+
* [berkeley-nest/Starling-LM-7B-alpha](https://huggingface.co/berkeley-nest/Starling-LM-7B-alpha)
|
164 |
+
* [mlabonne/AlphaMonarch-7B](https://huggingface.co/mlabonne/AlphaMonarch-7B)
|
165 |
+
* [cognitivecomputations/WestLake-7B-v2-laser](https://huggingface.co/cognitivecomputations/WestLake-7B-v2-laser)
|
166 |
+
* [senseable/garten2-7b](https://huggingface.co/senseable/garten2-7b)
|
167 |
+
|
168 |
+
### Configuration
|
169 |
+
|
170 |
+
The following YAML configuration was used to produce this model:
|
171 |
+
|
172 |
+
```yaml
|
173 |
+
models:
|
174 |
+
- model: mistralai/Mistral-7B-v0.1
|
175 |
+
# No parameters necessary for base model
|
176 |
+
|
177 |
+
- model: cognitivecomputations/WestLake-7B-v2-laser
|
178 |
+
parameters:
|
179 |
+
density: 0.58
|
180 |
+
weight: [0.3877, 0.1636, 0.186, 0.0502]
|
181 |
+
|
182 |
+
|
183 |
+
|
184 |
+
- model: senseable/garten2-7b
|
185 |
+
parameters:
|
186 |
+
density: 0.58
|
187 |
+
weight: [0.234, 0.2423, 0.2148, 0.2775]
|
188 |
+
|
189 |
+
|
190 |
+
|
191 |
+
- model: berkeley-nest/Starling-LM-7B-alpha
|
192 |
+
parameters:
|
193 |
+
density: 0.58
|
194 |
+
weight: [0.1593, 0.1573, 0.1693, 0.3413]
|
195 |
+
|
196 |
+
|
197 |
+
|
198 |
+
- model: mlabonne/AlphaMonarch-7B
|
199 |
+
parameters:
|
200 |
+
density: 0.58
|
201 |
+
weight: [0.219, 0.4368, 0.4299, 0.331]
|
202 |
+
|
203 |
+
|
204 |
+
|
205 |
+
merge_method: dare_ties
|
206 |
+
base_model: mistralai/Mistral-7B-v0.1
|
207 |
+
parameters:
|
208 |
+
int8_mask: true
|
209 |
dtype: bfloat16
|
210 |
|
211 |
+
```
|
212 |
+
|
213 |
+
|
214 |
|
215 |
|
216 |
### Table of Benchmarks
|