Edit model card

sixtyoneeighty-4x7b-v2

sixtyoneeighty-4x7b-v2 is a Mixure of Experts (MoE) made with the following models using LazyMergekit:

🧩 Configuration

base_model: cognitivecomputations/dolphin-2.8-mistral-7b-v02
gate_mode: hidden
dtype: bfloat16
experts_per_token: 2
experts:
  - source_model: Open-Orca/Mistral-7B-OpenOrca
    positive_prompts:
      - "What are some fun activities to do in Seattle?"
      - "What are some fun historical facts about New York City?"
    negative_prompts:
      - "Write a Python script to scrape data from a website."
      - "Explain the key differences between Bayesian and frequentist statistics."

  - source_model: NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
    positive_prompts:
      - "Write me a fictional story about dragons and wizards?"
      - "From now on take on the role of Dwayne Johnson"
    negative_prompts:
      - "When is the next solar eclipse."
      - "What year did World War II end?"

  - source_model: S-miguel/The-Trinity-Coder-7B
    positive_prompts:
      - "Can you review my JavaScript code and suggest ways to optimize it for better performance?"
      - "I'm getting an 'undefined variable' error in my Python script. Here's the code: [code snippet]"
    negative_prompts:
      - "What are some effective strategies for managing stress and anxiety?"
      - "Compare and contrast the themes in 'The Great Gatsby' and 'The Catcher in the Rye'."

  - source_model: chihoonlee10/T3Q-Mistral-Orca-Math-DPO
    positive_prompts:
      - "What's a square root of 1337?"
      - "Find the midpoint of the line segment with the given end points (-5,7) and (-2,1)"
    negative_prompts:
      - "What are some effective strategies for managing stress and anxiety?"
      - "Compare and contrast the themes in 'The Great Gatsby' and 'The Catcher in the Rye'."

πŸ’» Usage

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "jambroz/sixtyoneeighty-7b-MOE"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Downloads last month
6
Safetensors
Model size
24.2B params
Tensor type
BF16
Β·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for sixtyoneeighty/FNCARL9000