Update README.md

README.md

@@ -65,13 +65,13 @@ Here we compare Breeze-7B-Base-v1_0 with other open-source base language models
  We use the code revised from [EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate **TMMLU+**, **DRCD**, **Table**, and **MMLU**. All choice problems adapt the selection by the log-likelihood.
 
 
-| Models                                                                                    | #Parameters |  TMMLU+ (ACC) | DRCD (EM)   | Table (ACC) | MMLU (ACC) |
+| Models                                                                                    | #Parameters | ↑ TMMLU+ (ACC) | DRCD (EM)   | Table (ACC) | MMLU (ACC) |
 |----------------------------------------------                                             |--------|--------------|-------------|-------------|------------|
 |                                                                                           |        |TC, Knowledge |TC, Reasoning|TC, Reasoning|EN, Knowledge|
 |                                                                                           |        | 5 shot       | 3 shot      | 5 shot      | 5 shot      |
-| [**Breeze-7B-Base-v1_0**](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v1_0)   | 7B     |   42.67      | 80.61       |   31.99     | 61.24       |
 | [Yi-6B](https://huggingface.co/01-ai/Yi-6B)                                               | 6B     |   49.63      | 76.61       |   34.72     | 65.35       |
 | [Qwen1.5-7B](https://huggingface.co/Qwen/Qwen1.5-7B)                                      | 7B     |   46.59      | 74.41       |   30.56     | 63.07       |
+| [**Breeze-7B-Base-v1_0**](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v1_0)   | 7B     |   42.67      | 80.61       |   31.99     | 61.24       |
 | [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)                       | 7B     |   36.93      | 79.27       |   27.78     | 64.89       |
 
 ## Instruction-tuned Model Performance
@@ -86,13 +86,13 @@ Also, we listed the benchmark scores of GPT-3.5 Turbo (1106), which represents o
  We use the code revised from [fastchat llm_judge](https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge) (GPT4 as judge) to evaluate **MT-Bench-tw** and **MT-Bench**.
 
 
-| Models                                                                                             | #Parameters | MT-Bench-tw (Score)| TMMLU+ (ACC) | Table (ACC) | MT-Bench (Score) | MMLU (ACC)  | 
+| Models                                                                                             | #Parameters | ↑ MT-Bench-tw (Score)| TMMLU+ (ACC) | Table (ACC) | MT-Bench (Score) | MMLU (ACC)  | 
 |---------------------------------------------------------------------------------------------------------|--------|--------------------|--------------|-------------|------------------|-------------|
 |                                                                                                         |        |TC, Chat            |TC, Knowledge |TC, Reasoning|EN, Chat          |EN, Knowledge|
 |                                                                                                         |        |0 shot              | 0 shot       | 0 shot      |0 shot            |  0 shot     | 
-| [**Breeze-7B-Instruct-v1_0**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v1_0)         | 7B     |6.0                 | 42.67        | 39.58       |7.4               |  61.73     |    
 | [GPT-3.5-Turbo](https://openai.com)                                                                     |        |7.1                 | 43.56        | 45.14       |7.9               |  67.09      |    
 | [Qwen1.5-7B-Chat](https://huggingface.co/Qwen/Qwen1.5-7B-Chat)                                          | 7B     |6.4                 | 45.65        | 34.72       |7.6               |  61.85      |    
+| [**Breeze-7B-Instruct-v1_0**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v1_0)         | 7B     |6.0                 | 42.67        | 39.58       |7.4               |  61.73     |    
 | [Mistral-7B-v0.2-Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)                   | 7B     |5.6                 | 34.95        | 33.33       |7.6               |    59.97    |                                                  
 | [Yi-6B-Chat](https://huggingface.co/01-ai/Yi-6B-Chat)                                                   | 6B     |5.0                 | 44.79        | 25.69       |6.0               |    59.45    |    
 | [Taiwan-LLM-13B-v2.0-chat](https://huggingface.co/yentinglin/Taiwan-LLM-13B-v2.0-chat)                  | 13B    |5.0                 | 29.47        | 23.61       |N/A*                |    50.50    |     
@@ -103,9 +103,9 @@ Also, we listed the benchmark scores of GPT-3.5 Turbo (1106), which represents o
 
 | Details on MT-Bench-tw (0 shot):<br/>Models         | STEM    |Extraction|Reasoning| Math   | Coding  | Roleplay| Writing |Humanities|       AVG    | 
 |-----------------------------------------------------|---------|---------|---------|---------|---------|---------|---------|----------|  ---------   | 
-| **Breeze-7B-Instruct-v1_0**                         |  7.8    |  5.2    |   4.2   |   4.2   |  4.1    |   7.6   |   5.9   |   9.1    |        6.0   |
 | GPT-3.5-Turbo                                       |  7.8    |  6.1    |   5.1   |   6.4   |  6.2    |   8.7   |   7.4   |   9.3    |        7.1   |
 | Qwen1.5-7B-Chat                                     |  9      |  5.6    |   4.7   |   2.8   |  3.7    |   8.0   |   8.0   |   9.4    |        6.4   |
+| **Breeze-7B-Instruct-v1_0**                         |  7.8    |  5.2    |   4.2   |   4.2   |  4.1    |   7.6   |   5.9   |   9.1    |        6.0   |
 | Mistral-7B-v0.2-Instruct                            |  6.9    |  4.6    |   4.3   |   3.3   |  4.4    |   7.2   |   6.2   |   7.8    |        5.6   |                                          
 | Yi-6B-Chat                                          |  7.3    |  2.7    |   3.1   |   3.3   |  2.3    |   7.2   |   5.2   |   8.8    |        5.0   |
 | Taiwan-LLM-13B-v2.0-chat                            |  6.1    |  3.4    |   4.1   |   2.3   |  3.1    |   7.4   |   6.6   |   6.8    |        5.0   |
@@ -115,11 +115,11 @@ Also, we listed the benchmark scores of GPT-3.5 Turbo (1106), which represents o
 
 | Details on TMMLU+ (0 shot):<br/>Model               | STEM         | Social Science | Humanities | Other      |   AVG   |
 |-----------------------------------------------------|--------------|----------------|------------|------------|---------|
+| GPT-3.5-Turbo                                       | 41.58        | 48.52          | 40.96      | 43.18      | 43.56   |
+| Qwen1.5-7B-Chat                                     | 41.48        | 51.66          | 44.05      | 45.40      | 45.65   |
 | **Breeze-7B-Instruct-v1_0**                         | 36.46        | 48.38          | 45.11      | 40.75      | 42.67   |
 | Mistral-7B-v0.2-Instruct                            | 32.79        | 38.05          | 34.89      | 34.04      | 34.94   |
 | Yi-6B-Chat                                          | 37.80        | 51.74          | 45.36      | 44.25      | 44.79   |
-| GPT-3.5-Turbo                                       | 41.58        | 48.52          | 40.96      | 43.18      | 43.56   |
-| Qwen1.5-7B-Chat                                     | 41.48        | 51.66          | 44.05      | 45.40      | 45.65   |
 | Taiwan-LLM-13B-v2.0-chat                            | 27.74        | 33.69          | 27.03      | 29.43      | 29.47   |
 | Taiwan-LLM-7B-v2.1-chat                             | 25.58        | 31.76          | 27.36      | 27.61      | 28.08   |
 
@@ -129,11 +129,11 @@ Also, we listed the benchmark scores of GPT-3.5 Turbo (1106), which represents o
 In this test, we use the first 700 characters of the [web article](https://health.udn.com/health/story/5976/7699252?from=udn_ch1005_main_index) as the input and ask the model to write the same article again.
 All inferences run on 2 RTX A6000 GPUs (using `vllm`, with a tensor-parallel size of 2).
 
-| Models                                                             |   Inference Time (sec)|Estimated Max Input Length (Char)|
+| Models                                                             | ↓ Inference Time (sec)|Estimated Max Input Length (Char)|
 |--------------------------------------------------------------------|-------------------|--------------------------|
-| **Breeze-7B-Instruct-v1_0**                                        |  10.74            |    11.1k                 |
 | Qwen1.5-7B-Chat                                                    |  9.35             |    38.9k                  |
 | Yi-6B-Chat                                                         |   10.62           |   5.2k                |
+| **Breeze-7B-Instruct-v1_0**                                        |  10.74            |    11.1k                 |
 | Mistral-7B-Instruct-v0.2                                           |  20.48           |    5.1k                 |
 | Taiwan-LLM-7B-v2.1-chat                                            |   26.26          |    2.2k                  |
 <!---| Taiwan-LLM-13B-v2.0-chat                                           |   36.80          |    2.2k                  |--->
@@ -210,16 +210,6 @@ We also integrate `chat_template` into [tokenizer_config.json](tokenizer_config.
 
 ## Citation
 
-<!--
-```
-@article{breeze7b2024,
-  title={},
-  author={},
-  journal={arXiv},
-  year={2024}
-}
-```
---->
 ```
 @article{MediaTek-Research2024breeze7b,
       title={Breeze-7B Technical Report},