Update README.md

README.md

@@ -18,187 +18,140 @@ Also the adapter was trained above the foundation model [meta-llama/Llama-2-7b-c
 
 <!-- Provide a longer summary of what this model is. -->
 
-
-
 - **Developed by:** [Jangmin Oh](https://huggingface.co/jangmin)
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
+- **Model type:** llama2
 - **Language(s) (NLP):** ko
 - **License:** apche-2.0
 - **Finetuned from model [optional]:** [meta-llama/Llama-2-7b-chat-hf](meta-llama/Llama-2-7b-chat-hf)
 
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+Step 1. load the model and the tokenizer.
 
-[More Information Needed]
+  ```python
+  merged_model_hub_id = 'jangmin/merged-llama2-7b-chat-hf-food-order-understanding-30K'
+  tokenizer = AutoTokenizer.from_pretrained(merged_model_hub_id)
+  model = AutoModelForCausalLM.from_pretrained(merged_model_hub_id, device_map="auto", torch_dtype=torch.float16, cache_dir=cache_dir) 
+  ```
 
-### Downstream Use [optional]
+Step 2. prepare auxiliary tools
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+  ```python
+  instruction_prompt_template = """### 다음 주문 문장을 분석하여 음식명, 옵션명, 수량을 추출해줘.
+  
+  ### 명령: {0} ### 응답:
+  """
+  
+  def generate_helper(pipeline, query):   
+      prompt = instruction_prompt_template.format(query)
+  
+      out = pipeline(prompt, max_new_tokens=256, do_sample=False, eos_token_id=tokenizer.eos_token_id)
+  
+      generated_text = out[0]["generated_text"][len(prompt):]
+  
+      return generated_text
 
-[More Information Needed]
+  pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
 
-### Out-of-Scope Use
+stat_dic = pd.DataFrame.from_records([generate_helper(pipe, query) for query in evaluation_queries])
+  ```
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+Step 3. let's rock & roll.
 
-[More Information Needed]
+  ```python
+  print(generate_helpher(pipe, "아이스아메리카노 톨사이즈 한잔 하고요. 딸기스무디 한잔 주세요. 또, 콜드브루라떼 하나요."))
+  ```
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+Please refer [jangmin/qlora-llama2-7b-chat-hf-food-order-understanding-30K](jangmin/qlora-llama2-7b-chat-hf-food-order-understanding-30K) for the information about Bias, Risk, and Limitations.
 
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
 
 ## Training Details
 
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure 
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
+### Training Procedure
 
-[More Information Needed]
+Please refer [jangmin/qlora-llama2-7b-chat-hf-food-order-understanding-30K](jangmin/qlora-llama2-7b-chat-hf-food-order-understanding-30K). You can find the fine-tuning strategy.
 
-#### Hardware
+### Merging Procedure
 
-[More Information Needed]
+To merge the adapter on the pretrained model, I wrote following codes.
 
-#### Software
+Step 1. initialize.
 
-[More Information Needed]
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, AutoConfig, pipeline
+from peft import PeftModel, PeftConfig, AutoPeftModelForCausalLM
 
-## Citation [optional]
+peft_model_id = "jangmin/qlora-llama2-7b-chat-hf-food-order-understanding-30K"
+config = PeftConfig.from_pretrained(peft_model_id)
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+IGNORE_INDEX = -100
+DEFAULT_PAD_TOKEN = "[PAD]"
+```
 
-**BibTeX:**
+Step 2. load the fine-tuned model and the tokenzer.
+```python
+device_map = "cpu"
+trained_model = AutoPeftModelForCausalLM.from_pretrained(
+    peft_model_id,
+    low_cpu_mem_usage=True,
+    return_dict=True,
+    torch_dtype=torch.float16,
+    device_map=device_map,
+    cache_dir=cache_dir
+)
 
-[More Information Needed]
+tokenizer = AutoTokenizer.from_pretrained(
+    config.base_model_name_or_path,     
+    padding_side='right',
+    tokenizer_type="llama",
+    trust_remote_code=True,
+    cache_dir=cache_dir
+)
+```
 
-**APA:**
+Step 3. Modify the model and the tokenizer to treat the `PAD` token. (llama tokenizer needs to incorporate the pad token into the vocabulary. )
 
-[More Information Needed]
+```python
+def smart_tokenizer_and_embedding_resize(
+    special_tokens_dict: Dict,
+    tokenizer: transformers.PreTrainedTokenizer,
+    model: transformers.PreTrainedModel,
+):
+    """Resize tokenizer and embedding.
 
-## Glossary [optional]
+    Note: This is the unoptimized version that may make your embedding size not be divisible by 64.
+    """
+    num_new_tokens = tokenizer.add_special_tokens(special_tokens_dict)
+    model.resize_token_embeddings(len(tokenizer))
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+    if num_new_tokens > 0:
+        input_embeddings_data = model.get_input_embeddings().weight.data
 
-[More Information Needed]
+        input_embeddings_avg = input_embeddings_data[:-num_new_tokens].mean(
+            dim=0, keepdim=True
+        )
 
-## More Information [optional]
+        input_embeddings_data[-num_new_tokens:] = input_embeddings_avg
 
-[More Information Needed]
+if with_pad_token and tokenizer._pad_token is None:
+    smart_tokenizer_and_embedding_resize(
+        special_tokens_dict=dict(pad_token=DEFAULT_PAD_TOKEN),
+        tokenizer=tokenizer,
+        model=trained_model,
+    )
+    trained_model.config.pad_token_id = tokenizer.pad_token_id
+```
 
-## Model Card Authors [optional]
+Step 4. merge and push to hub.
 
-[More Information Needed]
+```python
+merged_model = trained_model.merge_and_unload()
 
-## Model Card Contact
+hub_id = "jangmin/merged-llama2-7b-chat-hf-food-order-understanding-30K"
 
-[More Information Needed]
+merged_model.push_to_hub(hub_id, max_shard_size="4GB", safe_serialization=True, commit_message='recommit after pad_token was treated.')
+```