Commit
•
aa2f9f0
1
Parent(s):
cf7b767
Update README.md
Browse files
README.md
CHANGED
|
@@ -20,9 +20,9 @@ The following named weight correspondance was used:
|
|
| 20 |
|
| 21 |
| Mistral weight | Mixtral weight |
|
| 22 |
|----------------|------------------------------|
|
| 23 |
-
| `gate_proj` | `experts.{
|
| 24 |
-
| `down_proj` | `experts.{
|
| 25 |
-
| `up_proj` | `experts.{
|
| 26 |
|
| 27 |
This mergekit configuration was used to merge the experts:
|
| 28 |
|
|
|
|
| 20 |
|
| 21 |
| Mistral weight | Mixtral weight |
|
| 22 |
|----------------|------------------------------|
|
| 23 |
+
| `gate_proj` | `experts.{expert_num}.w1` |
|
| 24 |
+
| `down_proj` | `experts.{expert_num}.w2` |
|
| 25 |
+
| `up_proj` | `experts.{expert_num}.w3` |
|
| 26 |
|
| 27 |
This mergekit configuration was used to merge the experts:
|
| 28 |
|