license: mit
library_name: transformers
pipeline_tag: text-generation
model-index:
- name: Rubra-Phi-3-mini-128k-instruct
results:
- task:
type: text-generation
dataset:
type: MMLU
name: MMLU
metrics:
- type: 5-shot
value: 67.87
verified: false
- task:
type: text-generation
dataset:
type: GPQA
name: GPQA
metrics:
- type: 0-shot
value: 29.69
verified: false
- task:
type: text-generation
dataset:
type: GSM-8K
name: GSM-8K
metrics:
- type: 8-shot, CoT
value: 79.45
verified: false
- task:
type: text-generation
dataset:
type: MATH
name: MATH
metrics:
- type: 4-shot, CoT
value: 30.8
verified: false
- task:
type: text-generation
dataset:
type: MT-bench
name: MT-bench
metrics:
- type: GPT-4 as Judge
value: 8.21
verified: false
tags:
- function-calling
- tool-calling
- agentic
- rubra
- conversational
language:
- en
Rubra Phi-3 Mini 128k Instruct
Model description
The model is the result of further post-training microsoft/Phi-3-mini-128k-instruct. This model is designed for high performance in various instruction-following tasks and complex interactions, including multi-turn function calling and detailed conversations.
| Model | Function Calling | MMLU | GPQA | GSM-8K | MATH | MT-bench | Win | Loss | Tie | Win Rate | Loss Rate | Adjusted Win Rate |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Phi-3 Mini 128k Instruct (June) | - | 69.36 | 27.01 | 83.7 | 32.92 | 8.02 | 21 | 72 | 67 | 0.13125 | 0.45000 | 0.340625 |
| Rubra Enhanced Phi-3 Mini 128k Instruct (June) | 70.00% | 67.87 | 29.69 | 79.45 | 30.80 | 8.21 | 72 | 21 | 67 | 0.45000 | 0.13125 | 0.659375 |
| Phi-3 Mini 128k Instruct (April) | - | 68.17 | 25.90 | 80.44 | 28.12 | 7.92 | 51 | 45 | 64 | 0.31875 | 0.28125 | 0.51875 |
| Rubra Enhanced Phi-3 Mini 128k Instruct (April) | 65.71% | 66.66 | 29.24 | 74.09 | 26.84 | 7.45 | 45 | 51 | 64 | 0.28125 | 0.31875 | 0.48125 |
- Commit
e2ecb24bd9dae689bb30dafcf13cbbc9dbddead5is the last commit to have the April-based Phi-3 model. The latest in main is built off the June model
Training Data
The model underwent additional training on a proprietary dataset encompassing diverse instruction-following, chat, and function calling data. This post-training process enhances the model's ability to integrate tools and manage complex interaction scenarios effectively.
How to use
You can use the model with the Hugging Face transformers and the rubra library rubra-tools as follows:
pip install rubra_tools torch==2.3.0 transformers accelerate
You also need Node.js and npm installed. Once you do, install the jsonrepair package - it's used to fix some rare hallucinations by the model.
npm install jsonrepair
1. Load the Model
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
from rubra_tools import preprocess_input, postprocess_output
model_id = "rubra-ai/Phi-3-mini-128k-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto",
)
2. Define Functions
Here we use 4 functions for a simple math chaining question:
functions = [
{
'type': 'function',
'function': {
'name': 'addition',
'description': "Adds two numbers together",
'parameters': {
'type': 'object',
'properties': {
'a': {
'description': 'First number to add',
'type': 'string'
},
'b': {
'description': 'Second number to add',
'type': 'string'
}
},
'required': []
}
}
},
{
'type': 'function',
'function': {
'name': 'subtraction',
'description': "Subtracts two numbers",
'parameters': {
'type': 'object',
'properties': {
'a': {
'description': 'First number to be subtracted from',
'type': 'string'
},
'b': {
'description': 'Number to subtract',
'type': 'string'
}
},
'required': []
}
}
},
{
'type': 'function',
'function': {
'name': 'multiplication',
'description': "Multiply two numbers together",
'parameters': {
'type': 'object',
'properties': {
'a': {
'description': 'First number to multiply',
'type': 'string'
},
'b': {
'description': 'Second number to multiply',
'type': 'string'
}
},
'required': []
}
}
},
{
'type': 'function',
'function': {
'name': 'division',
'description': "Divide two numbers",
'parameters': {
'type': 'object',
'properties': {
'a': {
'description': 'First number to use as the dividend',
'type': 'string'
},
'b': {
'description': 'Second number to use as the divisor',
'type': 'string'
}
},
'required': []
}
}
},
]
3. Start the conversation
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the result of four plus six? Take the result and add 2? Then multiply by 5 and then divide by two"},
]
def run_model(messages, functions):
## Format messages in Rubra's format
formatted_msgs = preprocess_input(msgs=messages, tools=functions)
input_ids = tokenizer.apply_chat_template(
formatted_msgs,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("")
]
outputs = model.generate(
input_ids,
max_new_tokens=1000,
eos_token_id=terminators,
do_sample=True,
temperature=0.1,
top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
raw_output = tokenizer.decode(response, skip_special_tokens=True)
return raw_output
raw_output = run_model(messages, functions)
# Check if there's a function call
function_call = postprocess_output(raw_output)
if function_call:
print(function_call)
else:
print(raw_output)
You should see this output, which is a function call made by the AI assistant:
[{'id': 'fc65a533', 'function': {'name': 'addition', 'arguments': '{"a": "4", "b": "6"}'}, 'type': 'function'}]
4. Add Executed Tool Result to Message History & Continue the Conversation
if function_call:
# append the assistant tool call msg
messages.append({"role": "assistant", "tool_calls": function_call})
# append the result of the tool call in openai format, in this case, the value of add 6 to 4 is 10.
messages.append({'role': 'tool', 'tool_call_id': function_call[0]["id"], 'name': function_call[0]["function"]["name"], 'content': '10'})
raw_output = run_model(messages, functions)
# Check if there's a function call
function_call = postprocess_output(raw_output)
if function_call:
print(function_call)
else:
print(raw_output)
The LLM will make another call
[{'id': '2ffc3de4', 'function': {'name': 'addition', 'arguments': '{"a": "10", "b": "2"}'}, 'type': 'function'}]
Framework Versions
- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1
Limitations and Bias
While the model performs well on a wide range of tasks, it may still produce biased or incorrect outputs. Users should exercise caution and critical judgment when using the model in sensitive or high-stakes applications. The model's outputs are influenced by the data it was trained on, which may contain inherent biases.
Ethical Considerations
Users should ensure that the deployment of this model adheres to ethical guidelines and consider the potential societal impact of the generated text. Misuse of the model for generating harmful or misleading content is strongly discouraged.
Acknowledgements
We would like to thank Microsoft for the model.
Contact Information
For questions or comments about the model, please reach out to the rubra team.
Citation
If you use this work, please cite it as:
@misc {rubra_ai_2024,
author = { Sanjay Nadhavajhala and Yingbei Tong },
title = { Phi-3-mini-128k-instruct },
year = 2024,
url = { https://huggingface.co/rubra-ai/Phi-3-mini-128k-instruct },
doi = { 10.57967/hf/2682 },
publisher = { Hugging Face }
}