Update README.md

README.md

@@ -27,12 +27,14 @@ This repository provides large language models developed by [LLM-jp](https://llm
 
 | Model Variant | 
 | :--- |
-|**Instruction models**|
+|**Instruction models ver1.1**|
+| [llm-jp-13b-dpo-lora-hh_rlhf_ja-v1.1](https://huggingface.co/llm-jp/llm-jp-13b-dpo-lora-hh_rlhf_ja-v1.1)|
 | [llm-jp-13b-instruct-full-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1](https://huggingface.co/llm-jp/llm-jp-13b-instruct-full-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1) |
+| [llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1](https://huggingface.co/llm-jp/llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1) |
+|**Instruction models ver1.0**|
 | [llm-jp-13b-instruct-full-jaster-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-full-jaster-v1.0) |
 | [llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0) |
 | [llm-jp-13b-instruct-full-dolly-oasst-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-full-dolly-oasst-v1.0) |
-| [llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1](https://huggingface.co/llm-jp/llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1) |
 | [llm-jp-13b-instruct-lora-jaster-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-lora-jaster-v1.0) | 
 | [llm-jp-13b-instruct-lora-jaster-dolly-oasst-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-lora-jaster-dolly-oasst-v1.0) | 
 | [llm-jp-13b-instruct-lora-dolly-oasst-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-lora-dolly-oasst-v1.0) | 
@@ -58,18 +60,18 @@ Checkpoints format: Hugging Face Transformers (Megatron-DeepSpeed format models
 ```python
 import torch
 from transformers import AutoTokenizer, AutoModelForCausalLM
-tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0")
-model = AutoModelForCausalLM.from_pretrained("llm-jp/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0", device_map="auto", torch_dtype=torch.float16)
-text = "自然言語処理とは何か"
-text = text + "### 回答："
+tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1")
+model = AutoModelForCausalLM.from_pretrained("llm-jp/llm-jp-13b-instruct-lora-dolly_en-dolly_ja-ichikara_003_001-oasst_en-oasst_ja-v1.1", device_map="auto", torch_dtype=torch.float16)
+text = "以下は、タスクを説明する指示です。要求を適切に満たす応答を書きなさい。\n\n### 指示:\n{instruction}\n\n### 応答:\n".format(instruction="自然言語処理とは何か")
 tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)
 with torch.no_grad():
     output = model.generate(
         tokenized_input,
-        max_new_tokens=100,
+        max_new_tokens=512,
         do_sample=True,
         top_p=0.95,
         temperature=0.7,
+        repetition_penalty=1.1,
     )[0]
 print(tokenizer.decode(output))
 ```
@@ -130,11 +132,12 @@ The models have been fine-tuned on the following datasets.
 | Language | Dataset | description |
 |:---|:---:|:---:|
 |Japanese|[jaster](https://github.com/llm-jp/llm-jp-eval)| An automatically transformed data from the existing Japanese NLP datasets |
-|English|[databricks-dolly-15k-en](https://huggingface.co/datasets/llm-jp/databricks-dolly-15k-en)| Identical to [databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) |
+|English|[databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k)| - |
 |Japanese|[databricks-dolly-15k-ja](https://huggingface.co/datasets/llm-jp/databricks-dolly-15k-ja)| A translated one by DeepL in LLM-jp |
 |English|[oasst1-21k-en](https://huggingface.co/datasets/llm-jp/oasst1-21k-en)| English subset of [oasst1 dataset](https://huggingface.co/datasets/OpenAssistant/oasst1) |
 |Japanese|[oasst1-21k-ja](https://huggingface.co/datasets/llm-jp/oasst1-21k-ja)| A translated one by DeepL in LLM-jp |
 |Japanese|[ichikara_003_001](https://liat-aip.sakura.ne.jp/wp/llm%E3%81%AE%E3%81%9F%E3%82%81%E3%81%AE%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%82%A4%E3%83%B3%E3%82%B9%E3%83%88%E3%83%A9%E3%82%AF%E3%82%B7%E3%83%A7%E3%83%B3%E3%83%87%E3%83%BC%E3%82%BF%E4%BD%9C%E6%88%90/)| ichikara-instruction dataset (ver.003-001)
+|Japanese|[hh-rlhf-12k-ja](https://huggingface.co/datasets/llm-jp/hh-rlhf-12k-ja)| A translated one by DeepL in LLM-jp |
 
 
 ## Evaluation