Noro-Hermes-3x7B

Noro-Hermes-3x7B is a Mixture of Experts (MoE) made with the following models using LazyMergekit:

🧩 Configuration

base_model: mistralai/Mistral-7B-Instruct-v0.2
dtype: bfloat16
gate_mode: hidden
experts:
  - source_model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO
    positive_prompts: ["You are an intelligent assistant."]
  - source_model: NeverSleep/Noromaid-7B-0.4-DPO
    positive_prompts: ["You are a creative roleplaying assistant."]
  - source_model: mistralai/Mistral-7B-Instruct-v0.2
    positive_prompts: ["You are a general purpose assistant."]

πŸ’» Usage

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "ThomasComics/Noro-Hermes-3x7B"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Downloads last month
16
Safetensors
Model size
18.5B params
Tensor type
BF16
Β·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for ThomasComics/Noro-Hermes-3x7B