metadata

base_model: Goekdeniz-Guelmez/j.o.s.i.e.v4o-7b-orpo-stage1-v1
language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - qwen2
  - trl
  - orpo
  - KANama

Goekdeniz-Guelmez/KANama-fineweb-v1-test1

The Model Goekdeniz-Guelmez/KANama-fineweb-v1-test1 was created using KANama.

Use with KANama

pip install KANama, transformers

from model.handler import from_pretrained, quick_inference
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Doctor-Shotgun/TinyLlama-1.1B-32k")
model = from_pretrained("path/to/model/folder")

prompt="hello"

input_tokens = tokenizer(prompt, return_tensors="pt").input_ids.to(device)

generated_tokens, generated_text = quick_inference(model, input_tokens, max_new_tokens=50, tokenizer=tokenizer)
print(generated_text)