File size: 3,599 Bytes
98e742b 8e133d9 41f0ab0 fc08875 41f0ab0 fc08875 41f0ab0 fc08875 13deebd fc08875 13deebd 41f0ab0 fc08875 aaedf39 fc08875 69bbf49 fc08875 85a4ec7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 |
---
license: mit
datasets:
- Jumtra/oasst1_ja
- Jumtra/jglue_jsquads_with_input
- Jumtra/dolly_oast_jglue_ja
- Aruno/guanaco_jp
- yahma/alpaca-cleaned
- databricks/databricks-dolly-15k
language:
- en
- ja
---
# Usage:
To use in code:
```python
import torch
import peft
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig
tokenizer = LlamaTokenizer.from_pretrained(
"decapoda-research/llama-7b-hf"
)
model = LlamaForCausalLM.from_pretrained(
"tamdiep106/alpaca_lora_ja_en_emb-7b",
load_in_8bit=False,
device_map="auto",
torch_dtype=torch.float16
)
tokenizer.pad_token_id = 0 # unk. we want this to be different from the eos token
tokenizer.bos_token_id = 1
tokenizer.eos_token_id = 2
```
To try out this model, use this colab space
https://colab.research.google.com/drive/1kVcN0L_n5lwhFlIqDkNbLNURboifgbBO?usp=sharing
Japanese prompt:
```python
instruction_input_JP = 'あなたはアシスタントです。以下に、タスクを説明する指示と、さらなるコンテキストを提供する入力を組み合わせます。 リクエストを適切に完了するレスポンスを作成します。'
instruction_no_input_JP = 'あなたはアシスタントです。以下はタスクを説明する指示です。 リクエストを適切に完了するレスポンスを作成します。'
prompt = """{}
### Instruction:
{}
### Response:"""
if input=='':
prompt = prompt.format(
instruction_no_input_JP, instruction
)
else:
prompt = prompt.format("{}\n\n### input:\n{}""").format(
instruction_input_JP, instruction, input
)
```
result:
English prompt:
```python
instruction_input_EN = 'You are an Assistant, below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.'
instruction_no_input_EN = 'You are an Assistant, below is an instruction that describes a task. Write a response that appropriately completes the request.'
prompt = """{}
### Instruction:
{}
### Response:"""
instruction = "4 + 4 = ?" #@param {type:"string"}
input = "" #@param {type:"string"}
if input=='':
prompt = prompt.format(
instruction_no_input_EN, instruction
)
else:
prompt = prompt.format("{}\n\n### input:\n{}""").format(
instruction_input_EN, instruction, input
)
```
result:
Use this code to decode output of model
```python
for s in generation_output.sequences:
result = tokenizer.decode(s).strip()
result = result.replace(prompt, '')
result = result.replace("<s>", "")
result = result.replace("</s>", "")
if result=='':
print('No output')
print(prompt)
print(result)
continue
print('\nResponse: ')
print(result)
```
# Training:
## Dataset:
- Jumtra/oasst1_ja
- Jumtra/jglue_jsquads_with_input
- Jumtra/dolly_oast_jglue_ja
- Aruno/guanaco_jp
- yahma/alpaca-cleaned
- databricks/databricks-dolly-15k
with about 750k entries, 2k entries used for evaluate process
## Training setup
I trained this model on an instance from **vast.ai**
- 1 NVIDIA RTX 4090
- 90 GB Storage
- Time spend about 3 and a half days
- use ```python export.py``` to merge weight
- Training loss
- Eval loss chart
 |