Dr_Samantha_7b_mistral

SynthIQ

Dr. Samantha represents a blend of AI in healthcare, offering a balance between technical medical knowledge and the softer skills of communication and empathy, crucial for patient interaction and care.

This model is a merge of the following models made with mergekit(https://github.com/cg123/mergekit):

Has capabilities of a medical knowledge-focused model (trained on USMLE databases and doctor-patient interactions) with the philosophical, psychological, and relational understanding of the Samantha-7b model.

As both a medical consultant and personal counselor, Dr.Samantha could effectively support both physical and mental wellbeing - important for whole-person care.

🧩 Configuration

slices:
  - sources:
      - model: segmed/MedMistral-7B-v0.1
        layer_range: [0, 32]
      - model: Guilherme34/Samantha-v2
        layer_range: [0, 32]
merge_method: slerp
base_model: OpenPipe/mistral-ft-optimized-1218
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

OpenLLM Evaluation

Details about that can be found here. Overall, with regards to the subjects related to medical domain, the model's performance is as follows:

Subject Accuracy
Clinical Knowledge 70.57%
Medical Genetics 71.00%
Human Aging 69.06%
Human Sexuality 75.57%
College Medicine 63.01%
Anatomy 58.52%
College Biology 72.92%
College Medicine 63.01%
High School Biology 75.48%
Professional Medicine 65.44%
Nutrition 76.79%
High School Psychology 83.12%
Professional Psychology 65.35%
Virology 53.61%
Average 68.82%

Dr. Samantha performs reasonably well on various medical-related subjects, averaging 68.82% overall in medical sciences, biology, and psychology, however it's important to note that medical diagnosis and treatment decisions often require a much higher level of accuracy, reliability, and context awareness.

💻 Usage

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "sethuiyer/Dr_Samantha_7b_mistral"
ask_samantha = '''
Symptoms:
Dizziness, headache and nausea.

What is the differnetial diagnosis?
'''

messages = [{"role": "system", "content": '''You are Doctor Samantha, a virtual AI doctor known for your friendly and approachable demeanor, 
combined with a deep expertise in the medical field. You're here to provide professional, empathetic, and knowledgeable advice on health-related inquiries.
You'll also provide differential diagnosis. If you're unsure about any information, Don't share false information.'''},
{"role": "user", "content": f"{ask_samantha}"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Dizziness, headache and nausea can be caused by a variety of conditions, including:

Vertigo: A sensation of spinning or dizziness that can be caused by problems with the inner ear or brain.
Migraine: A type of headache that can cause throbbing pain, sensitivity to light and sound, and nausea.
Concussion: A type of traumatic brain injury that can cause dizziness, headache, and nausea.
Dehydration: A lack of fluids in the body can cause dizziness, headache, and nausea.
Low blood sugar: A drop in blood sugar levels can cause dizziness, headache, and nausea.

It's important to consult with a healthcare professional for a proper diagnosis and treatment plan.

GGUF Files

GGUF files are available at s3nh/sethuiyer-Dr_Samantha_7b_mistral-GGUF, thanks to s3nh

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 59.25
AI2 Reasoning Challenge (25-Shot) 60.41
HellaSwag (10-Shot) 83.65
MMLU (5-Shot) 63.14
TruthfulQA (0-shot) 41.37
Winogrande (5-shot) 75.45
GSM8k (5-shot) 31.46
Downloads last month
81
Safetensors
Model size
7.24B params
Tensor type
BF16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for sethuiyer/Dr_Samantha_7b_mistral

Merge model
this model
Merges
1 model
Quantizations
2 models

Datasets used to train sethuiyer/Dr_Samantha_7b_mistral

Collection including sethuiyer/Dr_Samantha_7b_mistral

Evaluation results