mistralai

/

Mixtral-8x22B-Instruct-v0.1

like 615

Hey all! Please don't merge this yet - it depends on a PR that's still open.

This PR updates the Mixtral-8x22B tool use template to work with the new Hugging Face API for function calling. See that PR for more details! Outputs should be unchanged when passing tools in the currently-supported JSON schema format, so there shouldn't be any disruption to existing workflows, though we'll obviously re-check that after the Transformers PR is merged.

If you want to try out the new API, you can install from the library PR with pip install --upgrade git+https://github.com/huggingface/transformers.git@new_chat_template_args, then try the below code snippet:

from transformers import AutoTokenizer

def get_current_weather(location:str, format: str):
    """
    Get the current weather

    Args:
        location: The city and state, e.g. San Francisco, CA
        format: The temperature unit to use. Infer this from the users location. (Choices: ["celsius", "fahrenheit"])
    """
    return -40.

conversation=[
    {"role": "user", "content": "What's the weather like in Paris?"},
    {
        "role": "tool_calls",
        "content": [
            {
                "name": "get_current_weather",
                "arguments": {"location": "Paris, France", "format": "celsius"},
                
            }
        ]
    },
    {
        "role": "tool_results",
        "content": {"content": 22}
    },
    {"role": "assistant", "content": "The current temperature in Paris, France is 22 degrees Celsius."},
    {"role": "user", "content": "What about San Francisco?"}
]

tokenizer = AutoTokenizer.from_pretrained("mistralai/Mixtral-8x22B-Instruct-v0.1", revision="pr/33")
tool_use_prompt = tokenizer.apply_chat_template(
            conversation,
            chat_template="tool_use",
            tools=[get_current_weather],
            tokenize=False,
            add_generation_prompt=True,
)
print(tool_use_prompt)