Update README.md

README.md

@@ -31,7 +31,11 @@ The vision of OpenCSG is to empower every industry, every company, and every ind
 
 ## Model Description
 
-Phi-2 is a 2.7 billion-parameter Transformer model trained on augmented data sources, including synthetic NLP texts and filtered websites, alongside existing data used for Phi-1.5. It performs nearly state-of-the-art on benchmarks for common sense, language understanding, and logical reasoning, despite having fewer than 13 billion parameters. Unlike some models, Phi-2 hasn't been fine-tuned through reinforcement learning from human feedback. The goal of this open-source model is to enable research into safety challenges like reducing toxicity, understanding biases, enhancing controllability, etc.
+Phi-2 is a 2.7 billion-parameter Transformer model trained on augmented data sources, including synthetic NLP texts and filtered websites, alongside existing data used for Phi-1.5. It performs nearly state-of-the-art on benchmarks for common sense, language understanding, and logical reasoning, despite having fewer than 13 billion parameters. 
+Unlike some models, Phi-2 hasn't been fine-tuned through reinforcement learning from human feedback. The goal of this open-source model is to enable research into safety challenges like reducing toxicity, understanding biases, enhancing controllability, etc.
+
+
+opencsg-phi-2-v0.1 is a series of models based on phi-2 that have been fine-tuned using full-parameter tuning methods.
 <br>
 
 This is the repository for the base 13B version finetuned based on [CodeLlama-13b-hf](https://huggingface.co/codellama/CodeLlama-13b-hf). 
@@ -39,6 +43,8 @@ This is the repository for the base 13B version finetuned based on [CodeLlama-13
 | Model Size    | Base Model                                                                    |
 | --- | ----------------------------------------------------------------------------- |
 | phi-2 | [opencsg/Opencsg-phi-2-v0.1](https://huggingface.co/opencsg/opencsg-phi-2-v0.1)    | 
+
+
 | 7B  | [opencsg/Opencsg-CodeLlama-7b-v0.1](https://huggingface.co/opencsg/opencsg-CodeLlama-7b-v0.1) |
 | 13B  | [opencsg/Opencsg-CodeLlama-13b-v0.1](https://huggingface.co/opencsg/opencsg-CodeLlama-13b-v0.1) | 
 | 34B  | [opencsg/Opencsg-CodeLlama-34b-v0.1](https://huggingface.co/opencsg/opencsg-CodeLlama-34b-v0.1) |
@@ -63,6 +69,11 @@ To simplify the comparison, we chosed the Pass@1 metric for the Python language,
 
 | Model     | HumanEval python pass@1                                                 |
 | ---  |----------------------------------------------------------------------------- |
+
+| phi-2 | 48.2% |
+| **opencsg-phi-2-v0.1** |**54.3**|
+
+
 | CodeLlama-7b-hf  | 30.5%|
 | **opencsg-CodeLlama-7b-v0.1** | **43.9%** |
 | **opencsg-CodeLlama-7b-v0.2** | **50.0%** |
@@ -79,6 +90,8 @@ To simplify the comparison, we chosed the Pass@1 metric for the Python language,
 
 
 
+
+
 **TODO**
 - We will provide more benchmark scores on fine-tuned models in the future.
 - We will provide different practical problems to evaluate the performance of fine-tuned models in the field of software engineering.
@@ -87,34 +100,25 @@ To simplify the comparison, we chosed the Pass@1 metric for the Python language,
 
 # Model Usage
 
-```python
-from transformers import AutoTokenizer
-import transformers
+```
 import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+torch.set_default_device("cuda")
+
+model = AutoModelForCausalLM.from_pretrained("opencsg/opencsg-phi-2-v0.1", torch_dtype="auto", trust_remote_code=True)
+tokenizer = AutoTokenizer.from_pretrained("opencsg/opencsg-phi-2-v0.1", trust_remote_code=True)
+
+inputs = tokenizer('''def print_prime(n):
+   """
+   Print all primes between 1 and n
+   """''', return_tensors="pt", return_attention_mask=False)
 
-model = "opencsg/opencsg-CodeLlama-13b-v0.2"
-
-tokenizer = AutoTokenizer.from_pretrained(model, trust_remote_code=True)
-pipeline = transformers.pipeline(
-    "text-generation",
-    model=model,
-    torch_dtype=torch.float16,
-    device_map="auto",
-)
-input_text = "#write a quick sort algorithm."
-sequences = pipeline(
-    input_text,
-    do_sample=False,
-    top_k=10,
-    temperature=0.1,
-    top_p=0.95,
-    num_return_sequences=1,
-    eos_token_id=tokenizer.eos_token_id,
-    max_length=256,
-)
-for seq in sequences:
-    print(seq['generated_text'][len(input_text):])
+outputs = model.generate(**inputs, max_length=200)
+text = tokenizer.batch_decode(outputs)[0]
+print(text)
 ```
+
 # Training
 
 ## Hardware
@@ -155,8 +159,12 @@ OpenCSG 的愿景是让每个行业、每个公司、每个人都拥有自己的
 
 ## 模型介绍
 
-CodeLlama 是一系列由 Llama2 经过预训练和微调得到的生成式代码大模型，其规模从 70 亿到 340 亿个参数不等。
-opencsg-CodeLlama-v0.2是一系列基于CodeLlama的通过全参数微调方法进行调优的模型。
+
+Phi-2是一个拥有27亿参数的Transformer模型，使用了经过增强的数据源进行训练，包括合成的NLP文本和经过筛选的网站，同时还使用了Phi-1.5使用的现有数据。尽管参数少于130亿，但它在常识、语言理解和逻辑推理的基准测试中表现出了接近最先进的水平。
+与一些模型不同，Phi-2没有通过人类反馈的强化学习进行微调。这个开源模型的目标是促进对安全挑战的研究，如减少毒性、理解偏见、增强可控性等。
+
+opencsg-phi-2-v0.1是是一系列基于phi-2的通过全参数微调方法进行调优的模型。
+
 <br>
 
 这是基于 [CodeLlama-13b-hf](https://huggingface.co/codellama/CodeLlama-13b-hf) 进行微调的模型版本。
@@ -186,6 +194,11 @@ HumanEval 是评估模型在代码生成方面性能的最常见的基准，尤
 
 | 模型     | HumanEval python pass@1                                                 |
 | ---  |----------------------------------------------------------------------------- |
+
+| phi-2 | 48.2% |
+| **opencsg-phi-2-v0.1** |**54.3**|
+
+
 | CodeLlama-7b-hf  | 30.5%|
 | **opencsg-CodeLlama-7b-v0.1** | **43.9%** |
 | **opencsg-CodeLlama-7b-v0.2** | **50.0%** |
@@ -206,34 +219,23 @@ HumanEval 是评估模型在代码生成方面性能的最常见的基准，尤
 
 # 模型使用
 
-
-```python
-from transformers import AutoTokenizer
-import transformers
+```
 import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+torch.set_default_device("cuda")
+
+model = AutoModelForCausalLM.from_pretrained("opencsg/opencsg-phi-2-v0.1", torch_dtype="auto", trust_remote_code=True)
+tokenizer = AutoTokenizer.from_pretrained("opencsg/opencsg-phi-2-v0.1", trust_remote_code=True)
+
+inputs = tokenizer('''def print_prime(n):
+   """
+   Print all primes between 1 and n
+   """''', return_tensors="pt", return_attention_mask=False)
 
-model = "opencsg/opencsg-CodeLlama-13b-v0.2"
-
-tokenizer = AutoTokenizer.from_pretrained(model, trust_remote_code=True)
-pipeline = transformers.pipeline(
-    "text-generation",
-    model=model,
-    torch_dtype=torch.float16,
-    device_map="auto",
-)
-input_text = "#write a quick sort algorithm."
-sequences = pipeline(
-    input_text,
-    do_sample=False,
-    top_k=10,
-    temperature=0.1,
-    top_p=0.95,
-    num_return_sequences=1,
-    eos_token_id=tokenizer.eos_token_id,
-    max_length=256,
-)
-for seq in sequences:
-    print(seq['generated_text'][len(input_text):])
+outputs = model.generate(**inputs, max_length=200)
+text = tokenizer.batch_decode(outputs)[0]
+print(text)
 ```
 # 训练