StudentWen commited on
Commit
c33fbd2
1 Parent(s): 4e59a4f

Update Readme

Browse files
Files changed (1) hide show
  1. README.md +55 -0
README.md CHANGED
@@ -1,3 +1,58 @@
1
  ---
2
  license: llama2
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: llama2
3
+ tags:
4
+ - text2text-generation
5
+ pipeline_tag: text2text-generation
6
+ language:
7
+ - zh
8
+ - en
9
  ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ ## Welcome
14
+ If you find this model helpful, please *like* this model and star us on https://github.com/LianjiaTech/BELLE !
15
+
16
+ ## Model description
17
+ This model is obtained by fine-tuning the complete parameters using 0.4M Chinese instruction data on the original Llama2-13B-chat.
18
+ We firmly believe that the original Llama2-chat exhibits commendable performance post Supervised Fine-Tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF).
19
+ Our pursuit continues to be the further enhancement of this model using Chinese instructional data for fine-tuning, with an aspiration to facilitate stable and high-quality
20
+ Chinese language outputs.
21
+ ## Use model
22
+ Please note that the input should be formatted as follows in both **training** and **inference**.
23
+ ``` python
24
+ Human: \n{input}\n\nAssistant:\n
25
+ ```
26
+
27
+
28
+ After you decrypt the files, BELLE-Llama2-13B-chat-0.4M can be easily loaded with LlamaForCausalLM.
29
+ ``` python
30
+ from transformers import AutoModelForCausalLM, LlamaTokenizer
31
+ import torch
32
+
33
+ ckpt = '/path/to_finetuned_model/'
34
+ ckpt = '/nfs/a100-80G-15/xytian/myProjects/AI_NLP_GM/transformed_models/BELLE2-Llama2-13B-0.4M'
35
+ device = torch.device('cuda')
36
+ model = AutoModelForCausalLM.from_pretrained(ckpt).half().to(device)
37
+ tokenizer = LlamaTokenizer.from_pretrained(ckpt)
38
+ prompt = "Human: \n写一首中文歌曲,赞美大自然 \n\nAssistant: \n"
39
+ input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
40
+ generate_ids = model.generate(input_ids, max_new_tokens=1024, do_sample=True, top_k=30, top_p=0.85, temperature=0.5, repetition_penalty=1.2, eos_token_id=2, bos_token_id=1, pad_token_id=0)
41
+ output = tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
42
+ response = output[len(prompt):]
43
+ print(response)
44
+
45
+ ```
46
+
47
+
48
+ ## Limitations
49
+ There still exists a few issues in the model trained on current base model and data:
50
+
51
+ 1. The model might generate factual errors when asked to follow instructions related to facts.
52
+
53
+ 2. Occasionally generates harmful responses since the model still struggles to identify potential harmful instructions.
54
+
55
+ 3. Needs improvements on reasoning and coding.
56
+
57
+ Since the model still has its limitations, we require developers only use the open-sourced code, data, model and any other artifacts generated via this project for research purposes. Commercial use and other potential harmful use cases are not allowed.
58
+ ```