Model Card for Model ID
Gemma-2b fine-tuned with Turkish Instruction-Response pairs.
Training Data
- Dataset size: ~75k
Restrictions
Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms Please refer to the gemma use restrictions before start using the model. https://ai.google.dev/gemma/terms#3.2-use
Using model
import torch,re
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "erythropygia/Gemma2b-Turkish-Instruction"
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map={"":0})
tokenizer = AutoTokenizer.from_pretrained(model_id, add_eos_token=True, padding_side="left")
def get_completion(query: str, model, tokenizer) -> str:
device = "cuda:0"
prompt_template = """
<start_of_turn>user
Alt satırdaki soruya cevap ver:\n
{query}
<end_of_turn>\n<start_of_turn>model
"""
prompt = prompt_template.format(query=query)
encodeds = tokenizer(prompt, return_tensors="pt", add_special_tokens=True)
model_inputs = encodeds.to(device)
#max_new_tokens = 200, temperature = 0.9, repetition_penalty = 0.5, disabled
#num_return_sequences=1, max_length = 256,
generated_ids = model.generate(**model_inputs, max_new_tokens = 256, do_sample=True, pad_token_id=tokenizer.eos_token_id)
decoded = tokenizer.decode(generated_ids[0], skip_special_tokens=False)
decoded = re.sub(r'<(end_of_turn|start_of_turn|eos|bos)>[^<]*$', '', decoded)
decoded = re.sub(r'<(end_of_turn|start_of_turn|eos|bos)>', '', decoded)
return decoded.strip()
result = get_completion(query="int türünde üç parametre alan ve bunların toplamını döndüren bir işlev oluşturun.", model=model, tokenizer=tokenizer)
print(result)
Training Hyperparameters
- Epochs: 1
- MaxSteps: 300
- Context length: 1024
- LoRA Rank: 32
- LoRA Alpha: 64
- LoRA Dropout: 0.05
Training Results
training_loss: 2.077697410186132 (could be better when i will find enough GPU access :-) )
- Downloads last month
- 5
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.