|
--- |
|
base_model: Qwen/Qwen2-7B-Instruct |
|
library_name: peft |
|
license: other |
|
tags: |
|
- llama-factory |
|
- lora |
|
- generated_from_trainer |
|
model-index: |
|
- name: train_2024-06-17-19-49-05 |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# Install some dependency |
|
```bash |
|
pip install openai huggingface_hub |
|
``` |
|
|
|
# Download lora |
|
```python |
|
from huggingface_hub import snapshot_download |
|
snapshot_download( |
|
repo_id="svjack/Qwen2-7B_Function_Call_tiny_lora", |
|
repo_type="model", |
|
local_dir="Qwen2-7B_Function_Call_tiny_lora", |
|
local_dir_use_symlinks = False |
|
) |
|
``` |
|
|
|
# Start OpenAI style api server |
|
```bash |
|
python src/api.py \ |
|
--model_name_or_path Qwen/Qwen2-7B-Instruct \ |
|
--template qwen \ |
|
--adapter_name_or_path Qwen2-7B_Function_Call_tiny_lora \ |
|
--quantization_bit 4 |
|
``` |
|
|
|
# Inference |
|
```python |
|
import json |
|
import os |
|
from typing import Sequence |
|
|
|
from openai import OpenAI |
|
from transformers.utils.versions import require_version |
|
|
|
require_version("openai>=1.5.0", "To fix: pip install openai>=1.5.0") |
|
|
|
def calculate_gpa(grades: Sequence[str], hours: Sequence[int]) -> float: |
|
grade_to_score = {"A": 4, "B": 3, "C": 2} |
|
total_score, total_hour = 0, 0 |
|
for grade, hour in zip(grades, hours): |
|
total_score += grade_to_score[grade] * hour |
|
total_hour += hour |
|
return round(total_score / total_hour, 2) |
|
|
|
client = OpenAI( |
|
api_key="0", |
|
base_url="http://localhost:{}/v1".format(os.environ.get("API_PORT", 8000)), |
|
) |
|
|
|
tools = [ |
|
{ |
|
"type": "function", |
|
"function": { |
|
"name": "calculate_gpa", |
|
"description": "Calculate the Grade Point Average (GPA) based on grades and credit hours", |
|
"parameters": { |
|
"type": "object", |
|
"properties": { |
|
"grades": {"type": "array", "items": {"type": "string"}, "description": "The grades"}, |
|
"hours": {"type": "array", "items": {"type": "integer"}, "description": "The credit hours"}, |
|
}, |
|
"required": ["grades", "hours"], |
|
}, |
|
}, |
|
} |
|
] |
|
tool_map = {"calculate_gpa": calculate_gpa} |
|
|
|
messages = [] |
|
messages.append({"role": "user", "content": "My grades are A, A, B, and C. The credit hours are 3, 4, 3, and 2."}) |
|
|
|
result = client.chat.completions.create(messages=messages, |
|
model="Qwen/Qwen2-7B-Instruct", tools=tools) |
|
|
|
result.choices[0].message.tool_calls |
|
|
|
messages.append(result.choices[0].message) |
|
tool_call = result.choices[0].message.tool_calls[0].function |
|
print(tool_call) |
|
|
|
name, arguments = tool_call.name, json.loads(tool_call.arguments) |
|
tool_result = tool_map[name](**arguments) |
|
|
|
messages.append({"role": "tool", "content": json.dumps({"gpa": tool_result}, ensure_ascii=False)}) |
|
|
|
result = client.chat.completions.create(messages=messages, model="test", tools=tools) |
|
print(result.choices[0].message.content) |
|
``` |
|
|
|
# Output |
|
``` |
|
Function(arguments='{"grades": ["A", "A", "B", "C"], "hours": [3, 4, 3, 2]}', name='calculate_gpa') |
|
Based on the grades and credit hours you provided, your calculated GPA is 3.42. |
|
``` |
|
|
|
# Inference |
|
```python |
|
messages = [] |
|
messages.append({"role": "user", "content": "ζηζη»©εε«ζ―A,A,B,Cε¦εεε«ζ―3, 4, 3,ε2"}) |
|
|
|
result = client.chat.completions.create(messages=messages, |
|
model="Qwen/Qwen2-7B-Instruct", tools=tools) |
|
|
|
result.choices[0].message.tool_calls |
|
|
|
messages.append(result.choices[0].message) |
|
tool_call = result.choices[0].message.tool_calls[0].function |
|
print(tool_call) |
|
|
|
name, arguments = tool_call.name, json.loads(tool_call.arguments) |
|
tool_result = tool_map[name](**arguments) |
|
|
|
messages.append({"role": "tool", "content": json.dumps({"gpa": tool_result}, ensure_ascii=False)}) |
|
|
|
result = client.chat.completions.create(messages=messages, model="test", tools=tools) |
|
print(result.choices[0].message.content) |
|
``` |
|
|
|
# Output |
|
``` |
|
Function(arguments='{"grades": ["A", "A", "B", "C"], "hours": [3, 4, 3, 2]}', name='calculate_gpa') |
|
ζ¨ζδΎηζη»©εε¦εηε ζεΉ³εεοΌGPAοΌζ―3.42γ |
|
``` |
|
|
|
|
|
# train_2024-06-17-19-49-05 |
|
|
|
This model is a fine-tuned version of [Qwen/Qwen2-7B-Instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct) on the glaive_toolcall_zh and the glaive_toolcall_en datasets. |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 1 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- distributed_type: multi-GPU |
|
- num_devices: 2 |
|
- gradient_accumulation_steps: 8 |
|
- total_train_batch_size: 16 |
|
- total_eval_batch_size: 16 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: cosine |
|
- num_epochs: 3.0 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
|
|
|
|
### Framework versions |
|
|
|
- PEFT 0.11.1 |
|
- Transformers 4.41.2 |
|
- Pytorch 2.3.1+cu121 |
|
- Datasets 2.20.0 |
|
- Tokenizers 0.19.1 |