Text Generation
Transformers
Safetensors
mixtral
Mixture of Experts
frankenmoe
Merge
mergekit
lazymergekit
Open-Orca/Mistral-7B-OpenOrca
NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
S-miguel/The-Trinity-Coder-7B
chihoonlee10/T3Q-Mistral-Orca-Math-DPO
conversational
text-generation-inference
Inference Endpoints
sixtyoneeighty-4x7b-v2
sixtyoneeighty-4x7b-v2 is a Mixure of Experts (MoE) made with the following models using LazyMergekit:
- Open-Orca/Mistral-7B-OpenOrca
- NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
- S-miguel/The-Trinity-Coder-7B
- chihoonlee10/T3Q-Mistral-Orca-Math-DPO
𧩠Configuration
base_model: cognitivecomputations/dolphin-2.8-mistral-7b-v02
gate_mode: hidden
dtype: bfloat16
experts_per_token: 2
experts:
- source_model: Open-Orca/Mistral-7B-OpenOrca
positive_prompts:
- "What are some fun activities to do in Seattle?"
- "What are some fun historical facts about New York City?"
negative_prompts:
- "Write a Python script to scrape data from a website."
- "Explain the key differences between Bayesian and frequentist statistics."
- source_model: NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
positive_prompts:
- "Write me a fictional story about dragons and wizards?"
- "From now on take on the role of Dwayne Johnson"
negative_prompts:
- "When is the next solar eclipse."
- "What year did World War II end?"
- source_model: S-miguel/The-Trinity-Coder-7B
positive_prompts:
- "Can you review my JavaScript code and suggest ways to optimize it for better performance?"
- "I'm getting an 'undefined variable' error in my Python script. Here's the code: [code snippet]"
negative_prompts:
- "What are some effective strategies for managing stress and anxiety?"
- "Compare and contrast the themes in 'The Great Gatsby' and 'The Catcher in the Rye'."
- source_model: chihoonlee10/T3Q-Mistral-Orca-Math-DPO
positive_prompts:
- "What's a square root of 1337?"
- "Find the midpoint of the line segment with the given end points (-5,7) and (-2,1)"
negative_prompts:
- "What are some effective strategies for managing stress and anxiety?"
- "Compare and contrast the themes in 'The Great Gatsby' and 'The Catcher in the Rye'."
π» Usage
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "jambroz/sixtyoneeighty-7b-MOE"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
- Downloads last month
- 6
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for sixtyoneeighty/FNCARL9000
Merge model
this model