Hugging Face's logo

Goekdeniz-Guelmez
/
SmalJ.O.S.I.E.-32x101M-Chat

Text Generation
Transformers
Safetensors
mixtral
Mixture of Experts
frankenmoe
Merge
mergekit
lazymergekit
Felladrin/Smol-Llama-101M-Chat-v1
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
Edit model card

SmalJ.O.S.I.E.-32x101M-32k-Base

SmalJ.O.S.I.E.-32x101M-32k-Base is a Mixure of Experts (MoE) made with the following models using LazyMergekit:

🧩 Configuration 

dtype: float32
gate_mode: hidden
experts:
  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

💻 Usage 

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "Isaak-Carter/SmalJ.O.S.I.E.-32x101M-32k-Base"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Downloads last month
5
Safetensors
Model size
1.42B params
Tensor type
F32
·
Inference Examples
Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for Goekdeniz-Guelmez/SmalJ.O.S.I.E.-32x101M-Chat

Base model

BEE-spoke-data/smol_llama-101M-GQA
Finetuned
Felladrin/Smol-Llama-101M-Chat-v1
Finetuned
(4)
this model