README.md

---
base_model: BioMistral/BioMistral-7B
library_name: peft
license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags:
- biology
- medical
---

# Model Card for BioMistral-7B-Finetuned

## Model Summary

**BioMistral-7B-Finetuned** is a biomedical language model adapted from the BioMistral-7B model. This fine-tuned model is tailored for biomedical question-answering tasks and optimized through LoRA (Low-Rank Adaptation) on a 4-bit quantized base. It is particularly useful for tasks that require understanding and generating biomedical text in English.

---

## Model Details

### Model Description

This model was fine-tuned for biomedical applications, primarily focusing on enhancing accuracy in question-answering tasks within this domain.

- **Base Model**: BioMistral-7B
- **License**: apache-2.0
- **Fine-tuned for Task**: Biomedical Q&A, text generation
- **Quantization**: 4-bit precision with BitsAndBytes for efficient deployment

## Uses

### Direct Use

The model is suitable for biomedical question-answering and other related language generation tasks. 

### Out-of-Scope Use

Not recommended for general-purpose NLP tasks outside the biomedical domain or for clinical decision-making.

---

## How to Get Started with the Model

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("BeastGokul/BioMistral-7B-Finetuned")
model = AutoModelForCausalLM.from_pretrained("BeastGokul/BioMistral-7B-Finetuned")

# Example usage
input_text = "What are the symptoms of diabetes?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
---
## Training Details
### Training Procedure
The model was fine-tuned using the LoRA (Low-Rank Adaptation) method, with a configuration set for biomedical question-answering.

Training Hyperparameters
Precision: 4-bit quantization with BitsAndBytes
Learning Rate: 2e-5
Batch Size: Effective batch size of 16 (4 per device, gradient accumulation steps of 4)
Number of Epochs: 3


## Framework versions
PEFT 0.13.2