llm-jp
/

llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1

Text Generation

Model card Files Files and versions Community

Taka008 commited on Mar 12

Commit

8894718

•

1 Parent(s): 1a4e580

Update README.md

Files changed (1) hide show

README.md +4 -2

README.md CHANGED Viewed

@@ -64,9 +64,11 @@ Checkpoints format: Hugging Face Transformers (Megatron-DeepSpeed format models
 ```python
 import torch
-from transformers import AutoTokenizer, AutoModelForCausalLM
 tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1")
-model = AutoModelForCausalLM.from_pretrained("llm-jp/llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1", device_map="auto", torch_dtype=torch.float16)
 text = "以下は、タスクを説明する指示です。要求を適切に満たす応答を書きなさい。\n\n### 指示:\n{instruction}\n\n### 応答:\n".format(instruction="自然言語処理とは何か")
 tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)
 with torch.no_grad():

 ```python
 import torch
+from transformers import AutoTokenizer
+from peft import AutoPeftModelForCausalLM
 tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1")
+model = AutoPeftModelForCausalLM.from_pretrained("llm-jp/llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1", device_map="auto", torch_dtype=torch.float16)
 text = "以下は、タスクを説明する指示です。要求を適切に満たす応答を書きなさい。\n\n### 指示:\n{instruction}\n\n### 応答:\n".format(instruction="自然言語処理とは何か")
 tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)
 with torch.no_grad():