File size: 2,466 Bytes
22d84f7 71456f9 22d84f7 b902d75 22d84f7 7af0a4b 5df671e d6b6a91 662c4c5 69d73df 5df671e 7af0a4b cebcc5d 7af0a4b cebcc5d b902d75 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
---
base_model: unsloth/llama-3.2-3b-instruct-bnb-4bit
datasets:
- vishal042002/Clinical-surgery
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- medical
- Book2Data
- finetune
- Ragbased-q&a
- safetensors
---
# Uploaded model
- **Developed by:** vishal042002
- **License:** apache-2.0
- **Finetuned from model :** unsloth/llama-3.2-3b-instruct-bnb-4bit
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
The model was trained on a custom dataset containing clinical surgery Q&A pairs. The dataset was compiled from:
Open-source medical books
RUNNING THE MODEL THROUGH ADAPTER MERGE:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base_model_name = "unsloth/Llama-3.2-3B-Instruct"
base_model = AutoModelForCausalLM.from_pretrained(base_model_name, torch_dtype=torch.float16, device_map="auto")
adapter_path = "vishal042002/Llama3.2-3b-Instruct-ClinicalSurgery"
base_model = PeftModel.from_pretrained(base_model, adapter_path)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
device = "cuda" if torch.cuda.is_available() else "cpu"
base_model.to(device)
# Sample usage
input_text = "What is the mortality rate for patients requiring surgical intervention who were unstable preoperatively?"
inputs = tokenizer(input_text, return_tensors="pt").to(device)
outputs = base_model.generate(**inputs, max_new_tokens=200, temperature=1.5, top_p=0.9)
decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(decoded_output)
```
LOADING THE MODEL DIRECTLY:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "vishal042002/Llama3.2-3b-Instruct-ClinicalSurgery"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
```
This model is designed to:
Answer questions about clinical surgery procedures.
Provide information about surgical interventions.
Limitations:
The model should not be used as a substitute for professional medical advice.
Responses should be verified by qualified medical professionals. |