Update README.md

README.md

@@ -1,22 +1,162 @@
 ---
-base_model: KillerShoaib/gemma-2-9b-bangla-16bit
 language:
-- en
+- bn
 license: apache-2.0
 tags:
 - text-generation-inference
 - transformers
 - unsloth
-- gemma2
+- llama
 - trl
+base_model: unsloth/llama-3-8b-bnb-4bit
+inference: false
 ---
 
-# Uploaded  model
+# Gemma-2 Bangla 4 bit
+
+<div align="center">
+    <img src="https://cdn-uploads.huggingface.co/production/uploads/65ca6f0098a46a56261ac3ac/k4MH-skEIgS_zHXyAxtXv.jpeg" width="300"/>
+
+</div>
 
 - **Developed by:** KillerShoaib
 - **License:** apache-2.0
-- **Finetuned from model :** KillerShoaib/gemma-2-9b-bangla-16bit
+- **Finetuned from model :** unsloth/gemma-2-9b-it-bnb-4bit
+- **Datset used for fine-tuning :** iamshnoo/alpaca-cleaned-bengali
+
+
+# 4-bit Quantization
+**This is 4-bit quantization of Gemma-2 9b model.**
+
+
+# Gemma-2 Bangla Different Formats
+
+- `LoRA Adapters only` - [**KillerShoaib/gemma-2-9b-bangla-lora**](https://huggingface.co/KillerShoaib/gemma-2-9b-bangla-lora)
+- `16-bit model` - [**KillerShoaib/gemma-2-9b-bangla-16bit**](https://huggingface.co/KillerShoaib/gemma-2-9b-bangla-16bit)
+
+# Model Details
+
+**Gemma 2 9 billion** model was finetuned using **unsloth** package on a **cleaned Bangla alpaca** dataset. The model was finetune using **LoRA** finetuning technique. The model is finetuned for **1 epoch** on a single T4 GPU.
+
+
+# Pros & Cons of the Model
+
+## Pros
+
+- **The model can comprehend the Bangla language, including its semantic nuances**
+- **Given context model can answer the question based on the context**
+
+## Cons
+- **Model is unable to do creative or complex work. i.e: creating a poem or solving a math problem in Bangla**
+- **Since the size of the dataset was small, the model lacks lot of general knowledge in Bangla**
+
+## Llama 3 8b Bangla Vs Gemma 2 9b Bangla
+- **Overall both performace were similar, the pros and cons both were same, both the model struggle with longer context query and reasoning, but able to answer question if the context is given**
+- 
+
+
+# Run The Model
+
+## Download the right dependency for Unsloth
+
+**Google Colab**
+
+```python
+%%capture
+# Installs Unsloth, Xformers (Flash Attention) and all other packages!
+!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
+
+# We have to check which Torch version for Xformers (2.3 -> 0.0.27)
+from torch import __version__; from packaging.version import Version as V
+xformers = "xformers==0.0.27" if V(__version__) < V("2.4.0") else "xformers"
+!pip install --no-deps {xformers} trl peft accelerate bitsandbytes triton
+```
+
+## FastLanguageModel from unsloth for 2x faster inference
+
+```python
+
+from unsloth import FastLanguageModel
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name = "KillerShoaib/gemma-2-9b-bangla-lora", # or  KillerShoaib/gemma-2-9b-bangla-4bit / KillerShoaib/gemma-2-9b-bangla-4bit
+    max_seq_length = 2048,
+    dtype = None,
+    load_in_4bit = True,
+)
+FastLanguageModel.for_inference(model)
+
+# alpaca_prompt for the model
+alpaca_prompt = """Below is an instruction in bangla that describes a task, paired with an input also in bangla that provides further context. Write a response in bangla that appropriately completes the request.
+
+### Instruction:
+{}
+
+### Input:
+{}
+
+### Response:
+{}"""
+
+# input with instruction and input
+inputs = tokenizer(
+[
+    alpaca_prompt.format(
+        "সুস্থ থাকার তিনটি উপায় বলুন", # instruction
+        "", # input
+        "", # output - leave this blank for generation!
+    )
+], return_tensors = "pt").to("cuda")
+
+# generating the output and decoding it
+outputs = model.generate(**inputs, max_new_tokens = 2048, use_cache = True)
+tokenizer.batch_decode(outputs)
+```
+
+## AutoModelForCausalLM from Hugginface
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "KillerShoaib/gemma-2-9b-bangla-4bit" # or KillerShoaib/gemma-2-9b-bangla-16bit
+tokenizer_name = model_name
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)
+# Load model
+model = AutoModelForCausalLM.from_pretrained(model_name)
+
+# Text prompt to start generation
+alpaca_prompt = """Below is an instruction in bangla that describes a task, paired with an input also in bangla that provides further context. Write a response in bangla that appropriately completes the request.
+
+### Instruction:
+{}
+
+### Input:
+{}
+
+### Response:
+{}"""
+
+# Encode the prompt text
+inputs = tokenizer(
+[
+    alpaca_prompt.format(
+        "বিশ্বের সবচেয়ে বিখ্যাত চিত্রশিল্পী কে?", # instruction
+        "", # input
+        "", # output - leave this blank for generation!
+    )
+], return_tensors = "pt").to("cuda")
+
+# output
+outputs = model.generate(**inputs, max_new_tokens = 1024, use_cache = True)
+
+tokenizer.batch_decode(outputs)
+```
+
+
+# Inference Script & Github Repo
 
-This gemma2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+- `Google Colab` - [**Gemma-2 9b Bangla Inference Script**](https://colab.research.google.com/drive/13urCM6hQ2zE401uwu4laS9czi0K6qFUz?usp=sharing)
+- `Github Repo` - [**Llama-3 Bangla**](https://github.com/KillerShoaib/Llama-3-Bangla)
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+**‼️ Github Repo** shows how to finetune any unsloth model using incremental training. For **Gemma 2 9b** model finetuning I've used the exact **same logic** that was used for **Llama 3 8b** model. **Remember to change the dependency based on the Unsloth notebook example. ‼️**