Update README.md

README.md

@@ -28,50 +28,38 @@ Generate Python code that accomplishes the task instructed.
 
 Parameter Efficient Finetuning(PEFT) a 4bit quantized Llama-2-7b-Chat from TheBloke/Llama-2-7b-Chat-GPTQ on flytech/python-codes-25k dataset.
 
-
-- **Model type:** 
 - **Language(s) (NLP):** English
 - **License:** openrail
-- **Finetuned from model [facebook/bart-large-cnn](https://huggingface.co/facebook/bart-large-cnn?text=The+tower+is+324+metres+%281%2C063+ft%29+tall%2C+about+the+same+height+as+an+81-storey+building%2C+and+the+tallest+structure+in+Paris.+Its+base+is+square%2C+measuring+125+metres+%28410+ft%29+on+each+side.+During+its+construction%2C+the+Eiffel+Tower+surpassed+the+Washington+Monument+to+become+the+tallest+man-made+structure+in+the+world%2C+a+title+it+held+for+41+years+until+the+Chrysler+Building+in+New+York+City+was+finished+in+1930.+It+was+the+first+structure+to+reach+a+height+of+300+metres.+Due+to+the+addition+of+a+broadcasting+aerial+at+the+top+of+the+tower+in+1957%2C+it+is+now+taller+than+the+Chrysler+Building+by+5.2+metres+%2817+ft%29.+Excluding+transmitters%2C+the+Eiffel+Tower+is+the+second+tallest+free-standing+structure+in+France+after+the+Millau+Viaduct.)**
-- **Dataset:** [gretelai/synthetic_text_to_sql](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql)
+- **Qunatization:** GPTQ 4bit
+- **PEFT:** LoRA
+- **Finetuned from model [TheBloke/Llama-2-7b-Chat-GPTQ](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GPTQ)**
+- **Dataset:** [flytech/python-codes-25k](https://huggingface.co/datasets/flytech/python-codes-25k)
 
 ## Intended uses & limitations
 
-Addressing the power of LLM in fintuned downstream task. Implemented as a personal Project.
+Addressing the efficay of Quantization and PEFT. Implemented as a personal Project.
 
 ### How to use
 
-```python
-query_question_with_context = """sql_prompt: Which economic diversification efforts in
-the 'diversification' table have a higher budget than the average budget for all economic diversification efforts in the 'budget' table?
-sql_context: CREATE TABLE diversification (id INT, effort VARCHAR(50), budget FLOAT); CREATE TABLE
-budget (diversification_id INT, diversification_effort VARCHAR(50), amount FLOAT);"""
-```
+The quantized model is finetuned as PEFT. We have the trained Adapter. <br>The trained adpated needs to be merged with Base Model on which it was trained.
 
-# Use a pipeline as a high-level helper
 ```python
-from transformers import pipeline
-
-sql_generator = pipeline("text2text-generation", model="SwastikM/bart-large-nl2sql")
-
-sql = sql_generator(query_question_with_context)[0]['generated_text']
-
-print(sql)
+instruction = """model_input = "Help me set up my daily to-do list!""""
 ```
-
-# Load model directly
-
 ```python
-from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
+from peft import PeftModel, PeftConfig
+from transformers import AutoModelForCausalLM
 
-tokenizer = AutoTokenizer.from_pretrained("SwastikM/bart-large-nl2sql")
-model = AutoModelForSeq2SeqLM.from_pretrained("SwastikM/bart-large-nl2sql")
+config = PeftConfig.from_pretrained("SwastikM/Llama-2-7B-Chat-text2code")
+model = AutoModelForCausalLM.from_pretrained("TheBloke/Llama-2-7b-Chat-GPTQ")
+model = PeftModel.from_pretrained(model, "SwastikM/Llama-2-7B-Chat-text2code")
+tokenizer = AutoTokenizer.from_pretrained("SwastikM/Llama-2-7B-Chat-text2code")
 
-inputs = tokenizer(query_question_with_context, return_tensors="pt").input_ids
-outputs = model.generate(inputs, max_new_tokens=100, do_sample=False)
+inputs = tokenizer(instruction, return_tensors="pt").input_ids.to('cuda')
+outputs = model.generate(inputs, max_new_tokens=500, do_sample=False, num_beams=1)
+code = tokenizer.decode(outputs[0], skip_special_tokens=True)
 
-sql = tokenizer.decode(outputs[0], skip_special_tokens=True)
-print(sql)
+print(code)
 ```