README_api.md

metadata

base_model: Qwen/Qwen2-1.5B-Instruct
library_name: peft
license: other
tags:
  - llama-factory
  - lora
  - generated_from_trainer
model-index:
  - name: train_2024-06-17-19-49-05
    results: []

Install some dependency

pip install openai huggingface_hub

Download lora

from huggingface_hub import snapshot_download
snapshot_download(
        repo_id="svjack/Qwen2-1_5B_Function_Call_tiny_lora",
        repo_type="model",
        local_dir="Qwen2-1_5B_Function_Call_tiny_lora",
        local_dir_use_symlinks = False
    )

Start OpenAI style api server

python src/api.py \
--model_name_or_path Qwen/Qwen2-1.5B-Instruct \
--template qwen \
--adapter_name_or_path Qwen2-1_5B_Function_Call_tiny_lora

Inference

import json
import os
from typing import Sequence

from openai import OpenAI
from transformers.utils.versions import require_version

require_version("openai>=1.5.0", "To fix: pip install openai>=1.5.0")

def calculate_gpa(grades: Sequence[str], hours: Sequence[int]) -> float:
    grade_to_score = {"A": 4, "B": 3, "C": 2}
    total_score, total_hour = 0, 0
    for grade, hour in zip(grades, hours):
        total_score += grade_to_score[grade] * hour
        total_hour += hour
    return round(total_score / total_hour, 2)

client = OpenAI(
        api_key="0",
        base_url="http://localhost:{}/v1".format(os.environ.get("API_PORT", 8000)),
    )

tools = [
        {
            "type": "function",
            "function": {
                "name": "calculate_gpa",
                "description": "Calculate the Grade Point Average (GPA) based on grades and credit hours",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "grades": {"type": "array", "items": {"type": "string"}, "description": "The grades"},
                        "hours": {"type": "array", "items": {"type": "integer"}, "description": "The credit hours"},
                    },
                    "required": ["grades", "hours"],
                },
            },
        }
    ]
tool_map = {"calculate_gpa": calculate_gpa}

messages = []
messages.append({"role": "user", "content": "My grades are A, A, B, and C. The credit hours are 3, 4, 3, and 2."})

result = client.chat.completions.create(messages=messages,
                                        model="Qwen/Qwen2-1.5B-Instruct", tools=tools)

result.choices[0].message.tool_calls

messages.append(result.choices[0].message)
tool_call = result.choices[0].message.tool_calls[0].function
print(tool_call)

name, arguments = tool_call.name, json.loads(tool_call.arguments)
tool_result = tool_map[name](**arguments)

messages.append({"role": "tool", "content": json.dumps({"gpa": tool_result}, ensure_ascii=False)})

result = client.chat.completions.create(messages=messages, model="test", tools=tools)
print(result.choices[0].message.content)

Output

Function(arguments='{"grades": ["A", "A", "B", "C"], "hours": [3, 4, 3, 2]}', name='calculate_gpa')
Your calculated GPA is 3.42.

Inference

messages = []
messages.append({"role": "user", "content": "我的成绩分别是A,A,B,C学分分别是3, 4, 3,和2"})

result = client.chat.completions.create(messages=messages,
                                        model="Qwen/Qwen2-1.5B-Instruct", tools=tools)

result.choices[0].message.tool_calls

messages.append(result.choices[0].message)
tool_call = result.choices[0].message.tool_calls[0].function
print(tool_call)

name, arguments = tool_call.name, json.loads(tool_call.arguments)
tool_result = tool_map[name](**arguments)

messages.append({"role": "tool", "content": json.dumps({"gpa": tool_result}, ensure_ascii=False)})

result = client.chat.completions.create(messages=messages, model="test", tools=tools)
print(result.choices[0].message.content)

Output

Function(arguments='{"grades": ["A", "A", "B", "C"], "hours": [3, 4, 3, 2]}', name='calculate_gpa')
你的GPA是3.42。

train_2024-06-17-19-49-05

This model is a fine-tuned version of Qwen/Qwen2-1.5B-Instruct on the glaive_toolcall_zh and the glaive_toolcall_en datasets.

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• total_eval_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 3.0
• mixed_precision_training: Native AMP

Training results

Framework versions

• PEFT 0.11.1
• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1