metadata

base_model: unsloth/gemma-1.1-2b-it-bnb-4bit
language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - gemma
  - trl
  - sft

functions_metadata = [ { "type": "function", "function": { "name": "get_temperature", "description": "get temperature of a city", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "name" } }, "required": [ "city" ] } } } ]

messages = [
    { "role": "user", "content": f"""Bạn là một trợ lý hữu ích có quyền truy cập vào các chức năng sau. Sử dụng chúng nếu cần -\n{str(functions_metadata)}"""},
    { "role": "user", "content": "What is the temperature in Tokyo right now?"},
    # You will get the previous prediction, extract it will the tag <functioncall>
    # execute the function and append it to the messages like below:
    { "role": "assistant", "content": """<functioncall> {"name": "get_temperature", "arguments": '{"city": "Tokyo"}'} </functioncall>"""},    
    { "role": "user", "content": """<function_response> {"temperature":30 C} </function_response>"""}
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)


outputs = model.generate(
    input_ids,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
# >> The current temperature in Tokyo is 30 degrees Celsius.

Uploaded model

Developed by: hiieu
License: apache-2.0
Finetuned from model : unsloth/gemma-1.1-2b-it-bnb-4bit

This gemma model was trained 2x faster with Unsloth and Huggingface's TRL library.