metadata

base_model: llm-jp/llm-jp-3-13b
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
license: cc-by-nc-sa-4.0
language:
  - en
  - ja
datasets:
  - Kohsaku/Synthetic_Data_from_news_summary_2024secondhalf

Uploaded model

Developed by: Kohsaku
License: cc-by-nc-sa-4.0
Finetuned from model : llm-jp/llm-jp-3-13b

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

from unsloth import FastLanguageModel
import torch
import json

model_name = "Kohsaku/llm-jp-3-13b-finetune-5"

max_seq_length = 2048
dtype = None
load_in_4bit = True

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = model_name,
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    token = HF_TOKEN,
)
FastLanguageModel.for_inference(model)

text = "自然言語処理とは何か"
tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)
with torch.no_grad():
    output = model.generate(
        tokenized_input,
        max_new_tokens = 512, 
        use_cache = True, 
        do_sample=False,
        repetition_penalty=1.2
    )[0]

print(tokenizer.decode(output))