Update README.md
Browse files
README.md
CHANGED
@@ -17,11 +17,11 @@ This Mixture-of-Experts model is the combination of the following:
|
|
17 |
It is created using the following `mergekit-moe` config:
|
18 |
|
19 |
```yaml
|
20 |
-
base_model:
|
21 |
gate_mode: hidden
|
22 |
dtype: bfloat16
|
23 |
experts:
|
24 |
-
- source_model:
|
25 |
positive_prompts:
|
26 |
- "Solve the equation"
|
27 |
- "Derive the formula"
|
@@ -30,7 +30,7 @@ experts:
|
|
30 |
- "Find the integral of the function"
|
31 |
- "Find the first order derivative"
|
32 |
- "What is the answer to this math question"
|
33 |
-
- source_model:
|
34 |
positive_prompts:
|
35 |
- "Write a python program"
|
36 |
- "Write a java program"
|
@@ -43,7 +43,7 @@ experts:
|
|
43 |
- "How can I do this in Javascript"
|
44 |
- "Create a website with HTML"
|
45 |
shared_experts:
|
46 |
-
- source_model:
|
47 |
positive_prompts:
|
48 |
- "Hello, who are you?"
|
49 |
- "I need help with"
|
|
|
17 |
It is created using the following `mergekit-moe` config:
|
18 |
|
19 |
```yaml
|
20 |
+
base_model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
|
21 |
gate_mode: hidden
|
22 |
dtype: bfloat16
|
23 |
experts:
|
24 |
+
- source_model: Qwen/Qwen2.5-Math-7B-Instruct
|
25 |
positive_prompts:
|
26 |
- "Solve the equation"
|
27 |
- "Derive the formula"
|
|
|
30 |
- "Find the integral of the function"
|
31 |
- "Find the first order derivative"
|
32 |
- "What is the answer to this math question"
|
33 |
+
- source_model: rombodawg/Rombos-Coder-V2.5-Qwen-7b
|
34 |
positive_prompts:
|
35 |
- "Write a python program"
|
36 |
- "Write a java program"
|
|
|
43 |
- "How can I do this in Javascript"
|
44 |
- "Create a website with HTML"
|
45 |
shared_experts:
|
46 |
+
- source_model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
|
47 |
positive_prompts:
|
48 |
- "Hello, who are you?"
|
49 |
- "I need help with"
|