MixtureOfPhi3 / mergekit_moe_config.yml
paulilioaica's picture
Upload folder using huggingface_hub
5508fb2 verified
raw
history blame contribute delete
298 Bytes
base_model: microsoft/Phi-3-mini-128k-instruct
gate_mode: cheap_embed
dtype: float16
experts:
- source_model: microsoft/Phi-3-mini-128k-instruct
positive_prompts: ["research, logic, math, science"]
- source_model: microsoft/Phi-3-mini-128k-instruct
positive_prompts: ["creative, art"]