Megatron_v3_2x7B
Megatron_v3_2x7B is a bilingual Mixure of Experts (MoE) which can comprehend and speak English/Turkish. Megatron, MoE mimarisine sahip Türkçe ve İngilizce talimatları anlayan ve cevap veren bir modeldir.
💻 Usage/Kullanımı
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "Eurdem/Megatron_v3_2x7B"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Tell me about AI"}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=1024, do_sample=True, temperature=0.7, top_k=500, top_p=0.95)
print(outputs[0]["generated_text"])
- Downloads last month
- 429
This model does not have enough activity to be deployed to Inference API (serverless) yet.
Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.