Update README.md

README.md

@@ -101,58 +101,80 @@ To utilize the prompt format without a system prompt, simply leave the line out.
 
 ## Prompt Format for Function Calling
 
-Our model was trained on specific system prompts and structures for Function Calling. 
+Our model was trained on specific system prompts and structures for Function Calling. These are handled by the `tool_use` chat template. To use this template,
+first define a list of tool functions. It's okay if these are dummy functions - what matters is their name, type hints, and docstring, as these will be
+extracted and made available to the model:
 
-You should use the system role with this message, followed by a function signature json as this example shows here.
-```
-<|im_start|>system
-You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: <tools> {"type": "function", "function": {"name": "get_stock_fundamentals", "description": "get_stock_fundamentals(symbol: str) -> dict - Get fundamental data for a given stock symbol using yfinance API.\\n\\n    Args:\\n        symbol (str): The stock symbol.\\n\\n    Returns:\\n        dict: A dictionary containing fundamental data.\\n            Keys:\\n                - \'symbol\': The stock symbol.\\n                - \'company_name\': The long name of the company.\\n                - \'sector\': The sector to which the company belongs.\\n                - \'industry\': The industry to which the company belongs.\\n                - \'market_cap\': The market capitalization of the company.\\n                - \'pe_ratio\': The forward price-to-earnings ratio.\\n                - \'pb_ratio\': The price-to-book ratio.\\n                - \'dividend_yield\': The dividend yield.\\n                - \'eps\': The trailing earnings per share.\\n                - \'beta\': The beta value of the stock.\\n                - \'52_week_high\': The 52-week high price of the stock.\\n                - \'52_week_low\': The 52-week low price of the stock.", "parameters": {"type": "object", "properties": {"symbol": {"type": "string"}}, "required": ["symbol"]}}}  </tools> Use the following pydantic model json schema for each tool call you will make: {"properties": {"arguments": {"title": "Arguments", "type": "object"}, "name": {"title": "Name", "type": "string"}}, "required": ["arguments", "name"], "title": "FunctionCall", "type": "object"} For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
-<tool_call>
-{"arguments": <args-dict>, "name": <function-name>}
-</tool_call><|im_end|>
+```python
+def get_current_temperature(location: str, unit: str) -> float:
+    """
+    Get the current temperature at a location.
+    
+    Args:
+        location: The location to get the temperature for, in the format "City, Country"
+        unit: The unit to return the temperature in. (choices: ["celsius", "fahrenheit"])
+    Returns:
+        The current temperature at the specified location in the specified units, as a float.
+    """
+    return 22.  # A real function should probably actually get the temperature!
+
+def get_current_wind_speed(location: str) -> float:
+    """
+    Get the current wind speed in km/h at a given location.
+    
+    Args:
+        location: The location to get the temperature for, in the format "City, Country"
+    Returns:
+        The current wind speed at the given location in km/h, as a float.
+    """
+    return 6.  # A real function should probably actually get the wind speed!
+
+tools = [get_current_temperature, get_current_wind_speed]
 ```
 
-To complete the function call, create a user prompt that follows the above system prompt, like so:
-```
-<|im_start|>user
-Fetch the stock fundamentals data for Tesla (TSLA)<|im_end|>
+Now, prepare a chat and apply the chat template, then generate the model's response
+
+```python
+messages = [
+  {"role": "user", "content": "Hey, what's the temperature in Paris right now?"}
+]
+
+inputs = tokenizer.apply_chat_template(messages, chat_template="tool_use", tools=tools, add_generation_prompt=True, return_dict=True, return_tensors="pt")
+inputs = {k: v.to(model.device) for k, v in inputs.items()}
+out = model.generate(**inputs, max_new_tokens=128)
+print(tokenizer.decode(out[0][len(inputs["input_ids"][0]):]))
 ```
 
 The model will then generate a tool call, which your inference code must parse, and plug into a function (see example inference code here: https://github.com/NousResearch/Hermes-Function-Calling):
+
 ```
-<|im_start|>assistant
 <tool_call>
-{"arguments": {"symbol": "TSLA"}, "name": "get_stock_fundamentals"}
+{"arguments": {"location": "Paris, France", "unit": "celsius"}, "name": "get_current_temperature"}
 </tool_call><|im_end|>
 ```
 
-Once you parse the tool call, call the api and get the returned values for the call, and pass it back in as a new role, `tool` like so:
+Once you parse the tool call, add it to the chat as an `assistant` response, using the `tool_calls` key, then append the tool output
+as a response with the `tool` role:
+
+```python
+tool_call = {"name": "get_current_temperature", "arguments": {"location": "Paris, France", "unit": "celsius"}}
+messages.append({"role": "assistant", "tool_calls": [{"type": "function", "function": tool_call}]})
+messages.append({"role": "tool", "name": "get_current_temperature", "content": "22.0"})
 ```
-<|im_start|>tool
-<tool_response>
-{"name": "get_stock_fundamentals", "content": {'symbol': 'TSLA', 'company_name': 'Tesla, Inc.', 'sector': 'Consumer Cyclical', 'industry': 'Auto Manufacturers', 'market_cap': 611384164352, 'pe_ratio': 49.604652, 'pb_ratio': 9.762013, 'dividend_yield': None, 'eps': 4.3, 'beta': 2.427, '52_week_high': 299.29, '52_week_low': 152.37}}
-</tool_response>
-<|im_end|>
+
+Now you can apply the chat template again to format the conversation, and generate a response from the model:
+
+```python
+inputs = tokenizer.apply_chat_template(messages, chat_template="tool_use", tools=tools, add_generation_prompt=True, return_dict=True, return_tensors="pt")
+inputs = {k: v.to(model.device) for k, v in inputs.items()}
+out = model.generate(**inputs, max_new_tokens=128)
+print(tokenizer.decode(out[0][len(inputs["input_ids"][0]):]))
 ```
 
-The assistant will then read in that data from the function's response, and generate a natural language response:
+and we get:
+
 ```
-<|im_start|>assistant
-The stock fundamentals data for Tesla (TSLA) are as follows:
-- **Symbol**: TSLA
-- **Company Name**: Tesla, Inc.
-- **Sector**: Consumer Cyclical
-- **Industry**: Auto Manufacturers
-- **Market Capitalization**: $566,160,130,480
-- **Forward Price-to-Earnings Ratio (PE Ratio)**: 42.73
-- **Price-to-Book Ratio (PB Ratio)**: 9.04
-- **Dividend Yield**: N/A
-- **Trailing Earnings Per Share (EPS)**: $4.3
-- **Beta Value of the Stock**: 2.42
-- **52-Week High Price of the Stock**: $299.29
-- **52-Week Low Price of the Stock**: $152.37
-
-This information provides a snapshot of Tesla's financial position and performance based on the fundamental data obtained from the yfinance API. It shows that Tesla has a substantial market capitalization and a relatively high P/E and P/B ratio compared to other stocks in its industry. The company does not pay a dividend at the moment, which is reflected by a 'Dividend Yield' of 'None'. The Beta value indicates that Tesla's stock has a moderate level of volatility relative to the market. The 52-week high and low prices give an idea of the stock's range over the past year. This data can be useful when assessing investment opportunities and making investment decisions.<|im_end|>
+The current temperature in Paris, France is 22.0 degrees Celsius.<|im_end|>
 ```
 
 ## Prompt Format for JSON Mode / Structured Outputs