FINGU-AI/Qwen2.5-32B-Lora-HQ-e-4

Overview

FINGU-AI/Qwen2.5-32B-Lora-HQ-e-4 is a powerful causal language model designed for a variety of natural language processing (NLP) tasks, including machine translation, text generation, and chat-based applications. This model is particularly useful for translating between Korean and Uzbek, as well as supporting other custom NLP tasks through flexible input.

Model Details

• Model ID: FINGU-AI/Qwen2.5-32B-Lora-HQ-e-4
• Architecture: Causal Language Model (LM)
• Parameters: 32 billion
• Precision: Torch BF16 for efficient GPU memory usage
• Attention: SDPA (Scaled Dot-Product Attention)
• Primary Use Case: Translation (e.g., Korean to Uzbek), text generation, and dialogue systems.

Example Usage

Installation

Make sure to install the required packages:

pip install torch transformers

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Model and Tokenizer
model_id = 'FINGU-AI/Qwen2.5-32B-Lora-HQ-e-4'
model = AutoModelForCausalLM.from_pretrained(model_id, attn_implementation="sdpa", torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_id)
model.to('cuda')

# Input Messages for Translation
messages = [
    {"role": "system", "content": "translate korean to Uzbek"},
    {"role": "user", "content": """새로운 은행 계좌를 개설하는 절차는 다음과 같습니다:

1. 계좌 개설 목적과 신분 확인을 위한 서류 제출
2. 서류 검토 과정을 거치는 것
3. 고객님의 신원 확인 절차를 진행하는 것
4. 모든 절차가 완료되면 계좌 개설이 가능합니다.

계좌 개설을 원하시는 경우, 신분증과 함께 방문해 주시면 됩니다.
    """},
]

# Tokenize and Generate Response
input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to('cuda')

outputs = model.generate(
    input_ids,
    max_new_tokens=500,
    do_sample=True,
)

# Decode and Print the Translation
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))