qingy2024 commited on
Commit
a5afb8c
·
verified ·
1 Parent(s): 8137ed7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -17,11 +17,11 @@ This Mixture-of-Experts model is the combination of the following:
17
  It is created using the following `mergekit-moe` config:
18
 
19
  ```yaml
20
- base_model: qwen_chat
21
  gate_mode: hidden
22
  dtype: bfloat16
23
  experts:
24
- - source_model: qwen_math
25
  positive_prompts:
26
  - "Solve the equation"
27
  - "Derive the formula"
@@ -30,7 +30,7 @@ experts:
30
  - "Find the integral of the function"
31
  - "Find the first order derivative"
32
  - "What is the answer to this math question"
33
- - source_model: qwen_code
34
  positive_prompts:
35
  - "Write a python program"
36
  - "Write a java program"
@@ -43,7 +43,7 @@ experts:
43
  - "How can I do this in Javascript"
44
  - "Create a website with HTML"
45
  shared_experts:
46
- - source_model: qwen_chat
47
  positive_prompts:
48
  - "Hello, who are you?"
49
  - "I need help with"
 
17
  It is created using the following `mergekit-moe` config:
18
 
19
  ```yaml
20
+ base_model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
21
  gate_mode: hidden
22
  dtype: bfloat16
23
  experts:
24
+ - source_model: Qwen/Qwen2.5-Math-7B-Instruct
25
  positive_prompts:
26
  - "Solve the equation"
27
  - "Derive the formula"
 
30
  - "Find the integral of the function"
31
  - "Find the first order derivative"
32
  - "What is the answer to this math question"
33
+ - source_model: rombodawg/Rombos-Coder-V2.5-Qwen-7b
34
  positive_prompts:
35
  - "Write a python program"
36
  - "Write a java program"
 
43
  - "How can I do this in Javascript"
44
  - "Create a website with HTML"
45
  shared_experts:
46
+ - source_model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
47
  positive_prompts:
48
  - "Hello, who are you?"
49
  - "I need help with"