Hebrew-Mixtral-8x22B

Hebrew-Mixtral-8x22B is an open-source Large Language Model (LLM) pretrained in hebrew and english pretrained with 141 billion parameters, based on Mixtral-8x22B from Mistral.

It is continuesly pretrained from Mixtral-8x22B on tokens in both English and Hebrew.

The resulting model is a powerful general-purpose language model suitable for a wide range of natural language processing tasks, with a focus on Hebrew language understanding and generation.

Usage

Below are some code snippets on how to get quickly started with running the model.

First make sure to pip install -U transformers, then copy the snippet from the section that is relevant for your usecase.

Running on CPU

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B")
model = AutoModelForCausalLM.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B")

input_text = "ืฉืœื•ื! ืžื” ืฉืœื•ืžืš ื”ื™ื•ื?"
input_ids = tokenizer(input_text, return_tensors="pt")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

Running on GPU

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B")
model = AutoModelForCausalLM.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B", device_map="auto")

input_text = "ืฉืœื•ื! ืžื” ืฉืœื•ืžืš ื”ื™ื•ื?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

Running with 4-Bit precision

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

tokenizer = AutoTokenizer.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B")
model = AutoModelForCausalLM.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B", quantization_config = BitsAndBytesConfig(load_in_4bit=True))

input_text = "ืฉืœื•ื! ืžื” ืฉืœื•ืžืš ื”ื™ื•ื?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0])

Notice

Hebrew-Mixtral-8x22B is a pretrained base model and therefore does not have any moderation mechanisms.

Downloads last month
46
Safetensors
Model size
141B params
Tensor type
FP16
ยท
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for yam-peleg/Hebrew-Mixtral-8x22B

Finetunes
1 model
Quantizations
1 model

Collection including yam-peleg/Hebrew-Mixtral-8x22B