Files changed (1) hide show
  1. README.md +7 -0
README.md CHANGED
@@ -33,3 +33,10 @@ The training process was geared towards simulating verbal exchanges between doct
33
  - The user can take on the role of a doctor, and the model can engage in conversation with you as if it were a patient.
34
  - You can provide the model with a brief patient background in the system prompt, and the model will respond based on that prompt. (see [Examples](http://www.tame.org.tw/webmag/news/newsFile/508/3.110%E5%B9%B4%E7%AC%AC%E4%B8%80%E6%AC%A1OSCE%20SP%E5%8A%87%E6%83%85%E6%91%98%E8%A6%81%E9%A1%8C%E5%9E%8B%E7%AF%84%E4%BE%8B(63%E4%BE%8B).pdf))
35
  - Directly asking the certain disease about the symptoms and the possible therapies.**(Warning: It's not medical advice!)**
 
 
 
 
 
 
 
 
33
  - The user can take on the role of a doctor, and the model can engage in conversation with you as if it were a patient.
34
  - You can provide the model with a brief patient background in the system prompt, and the model will respond based on that prompt. (see [Examples](http://www.tame.org.tw/webmag/news/newsFile/508/3.110%E5%B9%B4%E7%AC%AC%E4%B8%80%E6%AC%A1OSCE%20SP%E5%8A%87%E6%83%85%E6%91%98%E8%A6%81%E9%A1%8C%E5%9E%8B%E7%AF%84%E4%BE%8B(63%E4%BE%8B).pdf))
35
  - Directly asking the certain disease about the symptoms and the possible therapies.**(Warning: It's not medical advice!)**
36
+
37
+ ### Model Evaluation
38
+ The model got the same **TMMLU+** (0 shot) performance when evaluated using [EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) with default settings.
39
+ | Details on TMMLU+ (0 shot):<br/>Model | STEM | Social Science | Humanities | Other | AVG |
40
+ |-----------------------------------------------------|--------------|----------------|------------|------------|---------|
41
+ | Taiwan-inquiry_7B_v2.0 | 36.46 | 43.94 | 35.68 | 38.21 | 39.38 |
42
+ | Breeze-7B-Instruct-v1_0 | 36.46 | 43.94 | 35.68 | 38.21 | 39.38 |