Update README.md
Browse files
README.md
CHANGED
@@ -69,11 +69,18 @@ What is your name?
|
|
69 |
My name is Samantha.
|
70 |
```
|
71 |
|
72 |
-
|
|
|
|
|
|
|
|
|
|
|
73 |
|
74 |
-
##
|
75 |
|
76 |
-
###
|
|
|
|
|
77 |
|
78 |
- Demonstrates extensive medical knowledge through accurate identification of potential causes for various symptoms.
|
79 |
- Responses consistently emphasize the importance of seeking professional diagnoses and treatments.
|
@@ -83,7 +90,7 @@ My name is Samantha.
|
|
83 |
- Clear and understandable explanations of conditions and treatment options.
|
84 |
- Prompt responses addressing all aspects of medical inquiries.
|
85 |
|
86 |
-
|
87 |
|
88 |
- Could occasionally place stronger emphasis on urgency when symptoms indicate potential emergencies.
|
89 |
- Discussion of differential diagnoses could explore a broader range of less common causes.
|
@@ -92,4 +99,4 @@ My name is Samantha.
|
|
92 |
- Consider exploring full medical histories to improve diagnostic context where relevant.
|
93 |
- Caution levels and risk factors associated with certain conditions could be underscored more.
|
94 |
|
95 |
-
|
|
|
69 |
My name is Samantha.
|
70 |
```
|
71 |
|
72 |
+
## OpenLLM Leaderboard Performance
|
73 |
+
| T | Model | Average | ARC | Hellaswag | MMLU | TruthfulQA | Winogrande | GSM8K |
|
74 |
+
|---|----------------------------------|---------|-------|-----------|-------|------------|------------|-------|
|
75 |
+
| 1 | sethuiyer/Dr_Samantha-7b | 52.95 | 53.84 | 77.95 | 47.94 | 45.58 | 73.56 | 18.8 |
|
76 |
+
| 2 | togethercomputer/LLaMA-2-7B-32K-Instruct | 50.02 | 51.11 | 78.51 | 46.11 | 44.86 | 73.88 | 5.69 |
|
77 |
+
| 3 | togethercomputer/LLaMA-2-7B-32K | 47.07 | 47.53 | 76.14 | 43.33 | 39.23 | 71.9 | 4.32 |
|
78 |
|
79 |
+
## Evaluation by GPT-4 across 25 random prompts from ChatDoctor-200k Dataset
|
80 |
|
81 |
+
### Overall Rating: 83.5/100
|
82 |
+
|
83 |
+
#### Pros:
|
84 |
|
85 |
- Demonstrates extensive medical knowledge through accurate identification of potential causes for various symptoms.
|
86 |
- Responses consistently emphasize the importance of seeking professional diagnoses and treatments.
|
|
|
90 |
- Clear and understandable explanations of conditions and treatment options.
|
91 |
- Prompt responses addressing all aspects of medical inquiries.
|
92 |
|
93 |
+
#### Cons:
|
94 |
|
95 |
- Could occasionally place stronger emphasis on urgency when symptoms indicate potential emergencies.
|
96 |
- Discussion of differential diagnoses could explore a broader range of less common causes.
|
|
|
99 |
- Consider exploring full medical histories to improve diagnostic context where relevant.
|
100 |
- Caution levels and risk factors associated with certain conditions could be underscored more.
|
101 |
|
102 |
+
|