松尾研大規模言語モデル講座2024のコンペ用の提出モデル作成の一環として作成・公開しています。
license: apache-2.0 datasets: - elyza/elyza-tasks-100-TV_0.jsonl language: - ja base_model: - llm-jp/llm-jp-3-13b
Model Card for Model ID
This modelcard aims to be a base template for new models. It has been generated using this raw template.
Model Details
Model Description
- Developed by: ktabuchi
- Model type: [More Information Needed]
- Language(s) (NLP): [More Information Needed]
- License: [More Information Needed]
- Finetuned from model [optional]: [More Information Needed]
Model Sources [optional]
- Repository: ktabuchi
Uses
For Academic lessons of Matsuo-LLM Lectures
Direct Use
Out-of-Scope Use
Bias, Risks, and Limitations
Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
How to Get Started with the Model
Use the code below to get started with the model.
[More Information Needed]
Training Details
Training Data
https://liat-aip.sakura.ne.jp/wp/llmのための日本語インストラクションデータ作成/llmのための日本語インストラクションデータ-公開/
[More Information Needed]
Training Procedure
Training Hyperparameters
- Training regime: [More Information Needed]
Evaluation
Testing Data, Factors & Metrics
Testing Data
elyza-tasks-100-TV_0.jsonl
[More Information Needed]
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type:
- Hours used: 10
- Cloud Provider: Google Colab
- Compute Region: A100
- Carbon Emitted: [More Information Needed]
Model Architecture and Objective
For Academic Use
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
[More Information Needed]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
[More Information Needed]
Model Card Contact
[More Information Needed]
license: apache-2.0 datasets: - elyza/elyza-tasks-100-TV_0.jsonl language: - ja base_model: - llm-jp/llm-jp-3-13b
Model Card for Model ID
This modelcard aims to be a base template for new models. It has been generated using this raw template.
Model Details
Model Description
- Developed by: ktabuchi
- Model type: [More Information Needed]
- Language(s) (NLP): [More Information Needed]
- License: [More Information Needed]
- Finetuned from model [optional]: [More Information Needed]
Model Sources [optional]
- Repository: ktabuchi
Uses
For Academic lessons of Matsuo-LLM Lectures
Direct Use
Out-of-Scope Use
Bias, Risks, and Limitations
Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
How to Get Started with the Model
Use the code below to get started with the model.
[More Information Needed]
Training Details
Training Data
https://liat-aip.sakura.ne.jp/wp/llmのための日本語インストラクションデータ作成/llmのための日本語インストラクションデータ-公開/
[More Information Needed]
Training Procedure
Training Hyperparameters
- Training regime: [More Information Needed]
Evaluation
Testing Data, Factors & Metrics
Testing Data
elyza-tasks-100-TV_0.jsonl
[More Information Needed]
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type:
- Hours used: 10
- Cloud Provider: Google Colab
- Compute Region: A100
- Carbon Emitted: [More Information Needed]
Model Architecture and Objective
For Academic Use
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
[More Information Needed]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
[More Information Needed]
Model Card Contact
[More Information Needed]
必要なライブラリをインストール
%%capture !pip install unsloth !pip uninstall unsloth -y && pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git" !pip install -U torch !pip install -U peft
必要なライブラリを読み込み
from unsloth import FastLanguageModel from peft import PeftModel import torch import json from tqdm import tqdm import re
ベースとなるモデルと学習したLoRAのアダプタ（Hugging FaceのIDを指定）。
model_id = "llm-jp/llm-jp-3-13b" adapter_id = "ktabuchi/llm-jp-3-13b-ft_lora"
Hugging Face Token を指定。
下記の URL から Hugging Face Token を取得できますので下記の HF_TOKEN に入れてください。
https://huggingface.co/settings/tokens
HF_TOKEN = "" #@param {type:"string"}
unslothのFastLanguageModelで元のモデルをロード。
dtype = None # Noneにしておけば自動で設定 load_in_4bit = True # 今回は13Bモデルを扱うためTrue
model, tokenizer = FastLanguageModel.from_pretrained( model_name=model_id, dtype=dtype, load_in_4bit=load_in_4bit, trust_remote_code=True, )
元のモデルにLoRAのアダプタを統合。
model = PeftModel.from_pretrained(model, adapter_id, token = HF_TOKEN)
タスクとなるデータの読み込み。
事前にデータをアップロードしてください。
datasets = [] with open("./elyza-tasks-100-TV_0.jsonl", "r") as f: item = "" for line in f: line = line.strip() item += line if item.endswith("}"): datasets.append(json.loads(item)) item = ""
モデルを用いてタスクの推論。
推論するためにモデルのモードを変更
FastLanguageModel.for_inference(model)
results = [] for dt in tqdm(datasets): input = dt["input"]
prompt = f"""### 指示\n{input}\n### 回答\n"""
inputs = tokenizer([prompt], return_tensors = "pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens = 512, use_cache = True, do_sample=False, repetition_penalty=1.2) prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).split('\n### 回答')[-1]
results.append({"task_id": dt["task_id"], "input": input, "output": prediction})
結果をjsonlで保存。
ここではadapter_idを元にファイル名を決定しているが、ファイル名は任意で問題なし。
json_file_id = re.sub(".*/", "", adapter_id) with open(f"/content/{json_file_id}_output.jsonl", 'w', encoding='utf-8') as f: for result in results: json.dump(result, f, ensure_ascii=False) f.write('\n')