trikudayodayodayo/llm-jp-3-13b-it-1209_lora

Uploaded model

• Developed by: trikudayodayodayo
• License: apache-2.0
• Finetuned from model : llm-jp/llm-jp-3-13b

Overview

This repository provides a Japanese Large Language Model finetuned on ichikara datasets

supervised-fintuning

Thme model was finetuned on a subset from mxture of the following dataset. Training epoch:1

• ichikara-instruction-003-001-1
• ichikara-instruction-003-001-2
• ichikara-instruction-003-001-2.2
• ichikara-instruction-003-003-5.1
• ichikara-instruction-003-003-5.2
• ichikara-instruction-003-002-1
• ichikara-instruction-003-003-1

Authors tsuchida rikuto

How to Use To use this model, run the code below

!pip install -U bitsandbytes
!pip install -U transformers
!pip install -U accelerate
!pip install -U datasets

!pip install ipywidgets --upgrade

from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)
import torch
from tqdm import tqdm
import json


model_name = "trikudayodayodayo/llm-jp-3-13b-it-1209_lora"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=False,
)

HF_TOKEN="Type your HF_TOKEN"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map="auto",
    token = HF_TOKEN
)

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True, token = HF_TOKEN)

input = "Type text here"

tokenized_input = tokenizer.encode(input, add_special_tokens=False, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(
        tokenized_input,
        max_new_tokens=100,
        do_sample=False,
        repetition_penalty=1.2
    )[0]

output = tokenizer.decode(outputs[tokenized_input.size(1):], skip_special_tokens=True)

print(output)