File size: 3,821 Bytes
7110ab4 4c59b5e 7110ab4 4c59b5e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 |
---
license: apache-2.0
tags:
- qwen1.5
- function-calling
- zero-shot
---
# Qwen1.5 one shot chat template for function calling
This repo contains a tokenizer with a custom chat template in the tokenizer_config.json file.
The custom chat template can be used - via 'tokenizer.apply_chat_template' - to format an array of messages.
For example:
```
function_metadata = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "This function gets the current weather in a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city, e.g., San Francisco"
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use."
}
},
"required": ["city"]
}
}
},
{
"type": "function",
"function": {
"name": "get_clothes",
"description": "This function provides a suggestion of clothes to wear based on the current weather",
"parameters": {
"type": "object",
"properties": {
"temperature": {
"type": "string",
"description": "The temperature, e.g., 15 C or 59 F"
},
"condition": {
"type": "string",
"description": "The weather condition, e.g., 'Cloudy', 'Sunny', 'Rainy'"
}
},
"required": ["temperature", "condition"]
}
}
}
]
# Comment out later messages to test various stages of generation.
sample_messages = [
# System messages are not supported by default
# {
# "role": "system",
# "content": "you are a helpful assistant"
# },
{
"role": "function_metadata",
"content": "FUNCTION_METADATA"
},
{
"role": "user",
"content": "What is the current weather in London?"
},
# {
# "role": "function_call",
# "content": "{\n \"name\": \"get_current_weather\",\n \"arguments\": {\n \"city\": \"London\"\n }\n}"
# },
# {
# "role": "function_response",
# "content": "{\n \"temperature\": \"15 C\",\n \"condition\": \"Cloudy\"\n}"
# },
# {
# "role": "assistant",
# "content": "The current weather in London is Cloudy with a temperature of 15 Celsius.<|end_of_turn|>"
# },
# {
# "role": "user",
# "content": "That's great. Now say hello."
# },
# {
# "role": "assistant",
# "content": "Hello!"
# }
]
# Iterate through each message in the list
for message in sample_messages:
if message['role'] == 'function_metadata':
# Replace 'FUNCTION_METADATA' with 'function_metadata' in the content
message['content'] = message['content'].replace('FUNCTION_METADATA', json.dumps(function_metadata, indent=4))
# View the template applied without tokenization
prompt = tokenizer.apply_chat_template(sample_messages, tokenize=False, add_generation_prompt=True)
print(prompt)
```
This will provide a prompt format for doing zero-shot function calling, for example using a TGI api.
Alternatively, when deploying a vLLM endpoint, this repo id may be passed as the tokenizer for a Qwen1.5 chat model, and the chat template will be applied. In this case, you simply need to prepare your array of messages as per above. |