Create README.md

README.md

@@ -0,0 +1,78 @@
+---
+language:
+- en
+tags:
+- text-generation
+- finetuned
+widget:
+- text: Is it raining in Hawaii?
+  example_title: Is it raining in Hawaii?
+- text: Turn off music.
+  example_title: Turn off music.
+- text: Is it snowing in New York?
+  example_title: Is it snowing in New York?
+datasets:
+- commonsense_qa
+license: apache-2.0
+pipeline_tag: text-generation
+---
+
+# Commonsense-QA-Mistral-7B
+
+This is a finetuned model of [mistralai/Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1) 
+with [commonsense_qa](https://huggingface.co/datasets/commonsense_qa) dataset.
+
+The model is loaded in 4-bit and fine-tuned with LoRA.
+
+## Usage
+
+### Loading of model: 
+```python
+# Load model directly
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model = AutoModelForCausalLM.from_pretrained(
+    "rvv-karma/Commonsense-QA-Mistral-7B",
+    low_cpu_mem_usage=True,
+    return_dict=True,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+tokenizer = AutoTokenizer.from_pretrained("rvv-karma/Commonsense-QA-Mistral-7B", trust_remote_code=True)
+tokenizer.pad_token = tokenizer.eos_token
+tokenizer.padding_side = "left"
+```
+
+### Sample:
+```python
+pipe = pipeline(
+    task="text-generation",
+    model=model,
+    tokenizer=tokenizer,
+    return_full_text=False,
+    pad_token_id=tokenizer.pad_token_id,
+    eos_token_id=13,
+    max_new_tokens=8
+)
+
+prompt = """<s>
+QUESTION:
+The sensor would just the distance then set off an alarm, the installation expert explained it was called a what kind of sensor?
+
+OPTIONS:
+["near", "closeness", "here", "proximity", "this"]
+
+ANSWER:
+"""
+result = pipe(prompt)
+generated = result[0]['generated_text']
+print(generated)
+
+# Output: proximity
+```
+
+
+## Fine-tuning script
+
+[Kaggle Notebook](https://www.kaggle.com/rvkarma/commonsense-qa-mistral-7b)