Update README.md

README.md

@@ -39,6 +39,7 @@ language:
 ---
 # Model Card for EuroLLM-1.7B
 
+## Model updated on September 24
 
 This is the model card for the first pre-trained model of the EuroLLM series: EuroLLM-1.7B. You can also check the instruction tuned version: [EuroLLM-1.7B-Instruct](https://huggingface.co/utter-project/EuroLLM-1.7B-Instruct).
 
@@ -107,25 +108,27 @@ The results show that EuroLLM-1.7B is substantially better than Gemma-2B in Mach
 #### Flores-200
 | Model                          | AVG  | AVG en-xx | AVG xx-en | en-ar | en-bg | en-ca | en-cs | en-da | en-de | en-el | en-es-latam | en-et | en-fi | en-fr | en-ga | en-gl | en-hi | en-hr | en-hu | en-it | en-ja | en-ko | en-lt | en-lv | en-mt | en-nl | en-no | en-pl | en-pt-br | en-ro | en-ru | en-sk | en-sl | en-sv | en-tr | en-uk | en-zh-cn | ar-en | bg-en | ca-en | cs-en | da-en | de-en | el-en | es-latam-en | et-en | fi-en | fr-en | ga-en | gl-en | hi-en | hr-en | hu-en | it-en | ja-en | ko-en | lt-en | lv-en | mt-en | nl-en | no-en | pl-en | pt-br-en | ro-en | ru-en | sk-en | sl-en | sv-en | tr-en | uk-en | zh-cn-en |
 |--------------------------------|------|-----------|-----------|-------|-------|-------|-------|-------|-------|-------|--------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|----------|-------|-------|-------|-------|-------|-------|-------|----------|-------|-------|-------|-------|-------|-------|-------|--------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|----------|-------|-------|-------|-------|-------|-------|-------|----------|
-| EuroLLM-1.7B-Instruct     | 86.75|	86.49|	87.01|85.00|	89.37|	84.65|	89.23|	89.54|	87.00|	87.68|	86.43|	88.66|	89.18|	87.65|	74.66|	86.68|	76.94|	84.86|	86.43|	88.19|	89.45|	87.33|	87.73|	87.74|	67.66|	87.16|	90.08|	88.10|	89.39|	89.39|	88.11|	88.13|	87.08|	89.67|	87.21|	87.76|	86.45|	86.15|	87.46|	87.49|	87.97|	89.65|	88.80|	86.88|	86.70|	88.19|	88.66|	88.93|	81.95|	87.38|	87.97|	86.69|	87.15|	87.69|	86.78|	86.76|	85.39|	86.23|	76.93|	87.02|	90.24|	85.54|	89.22|	88.81|	86.02|	87.24|	86.81|	89.55|	87.76|	86.37|	86.00 |
-| Gemma-2B-EuroBlocks       | 81.56| 78.93     | 84.18     | 75.25 | 82.46 | 83.17 | 82.17 | 84.40 | 83.20 | 79.63 | 84.15        | 72.63 | 81.00 | 85.12 | 38.79 | 82.00 | 67.00 | 81.18 | 78.24 | 84.80 | 87.08 | 82.04 | 73.02 | 68.41 | 56.67 | 83.30 | 86.69 | 83.07   | 86.82 | 84.00 | 84.55 | 77.93 | 76.19 | 80.77 | 79.76 | 84.19   | 84.10 | 83.67 | 85.73 | 86.89 | 86.38 | 88.39 | 88.11 | 84.68        | 86.11 | 83.45 | 86.45 | 88.22 | 50.88 | 86.44 | 85.87 | 85.33 | 85.16 | 86.75 | 85.62 | 85.00 | 81.55 | 81.45 | 67.90 | 85.95 | 89.05   | 84.18 | 88.27 | 87.38 | 85.13 | 85.22 | 83.86 | 87.83   | 84.96 | 85.15 | 85.10 |
-| Gemma-7B-EuroBlocks       | 86.16| 85.49     | 86.82     | 83.39 | 88.32 | 85.82 | 88.88 | 89.01 | 86.96 | 86.62 | 86.31        | 84.42 | 88.11 | 87.46 | 61.85 | 86.10 | 77.91 | 87.01 | 85.81 | 87.57 | 89.88 | 87.24 | 84.47 | 83.15 | 67.13 | 86.50 | 90.44 | 87.57   | 89.22 | 89.13 | 88.58 | 86.73 | 84.68 | 88.16 | 86.87 | 88.40   | 87.11 | 86.65 | 87.25 | 88.17 | 87.47 | 89.59 | 88.44 | 86.76        | 86.66 | 87.55 | 88.88 | 88.86 | 73.46 | 87.63 | 88.43 | 87.12 | 87.31 | 87.49 | 87.20 | 87.15 | 85.16 | 85.96 | 78.39 | 86.73 | 90.52   | 85.38 | 89.17 | 88.75 | 86.35 | 86.82 | 86.21 | 89.39   | 88.20 | 86.45 | 86.28 |
+| EuroLLM-1.7B-Instruct     |86.89 |	86.53 |	87.25 | 85.17 | 89.42 | 84.72 | 89.13 | 89.47 | 86.90 | 87.60 | 86.29       | 88.95 | 89.40 | 87.69 | 74.89 | 86.41 | 76.92 | 84.79 | 86.78 | 88.17 | 89.76 | 87.70 | 87.27 | 87.62 | 67.84 | 87.10 | 90.00 | 88.18 | 89.29    | 89.49 | 88.32 | 88.18 | 86.85 | 90.00 | 87.31 | 87.89 | 86.60    | 86.34 | 87.45 | 87.57 | 87.95 | 89.72 | 88.80 | 87.00 | 86.77         | 88.34 | 89.09 | 88.95 | 82.69 | 87.80 | 88.37 | 86.71 | 87.20 | 87.81 | 86.79 | 86.79 | 85.62 | 86.48 | 81.10 | 86.97 | 90.25    | 85.75 | 89.20 | 88.88 | 86.00 | 87.38 | 86.76 | 89.61 | 87.94    |
+| Gemma-2B-EuroBlocks       | 81.59 |	78.97 |	84.21 | 76.68 | 82.73 | 83.14 | 81.63 | 84.63 | 83.15 | 79.42 | 84.05       | 72.58 | 79.73 | 84.97 | 40.50 | 82.13 | 67.79 | 80.53 | 78.36 | 84.90 | 87.43 | 82.98 | 72.29 | 68.68 | 58.55 | 83.13 | 86.15 | 82.78 | 86.79    | 83.14 | 84.61 | 78.18 | 75.37 | 80.89 | 78.38 | 84.38 | 84.35    | 83.88 | 85.77 | 86.85 | 86.31 | 88.24 | 88.12 | 84.79 | 84.90         | 82.51 | 86.32 | 88.29 | 54.78 | 86.53 | 85.83 | 85.41 | 85.18 | 86.77 | 85.78 | 84.99 | 81.65 | 81.78 | 67.27 | 85.92 | 89.07    | 84.14 | 88.07 | 87.17 | 85.23 | 85.09 | 83.95 | 87.57 | 84.77    |
+| Gemma-7B-EuroBlocks       |85.27 |	83.90 |	86.64 | 86.38 | 87.87 | 85.74 | 84.25 | 85.69       | 81.49 | 85.52 | 86.93 | 62.83 | 84.96 | 75.34 | 84.93 | 83.91 | 86.92 | 88.19 | 86.11 | 81.73 | 80.55 | 66.85 | 85.31 | 89.36 | 85.87 | 88.62    | 88.06 | 86.67 | 84.79 | 82.71 | 86.45 | 85.19 | 86.67 | 85.77    | 86.36 | 87.21 | 88.09 | 87.17 | 89.40 | 88.26 | 86.74 | 86.73         | 87.25 | 88.87 | 88.81 | 72.45 | 87.62 | 87.86 | 87.08 | 87.01 | 87.58 | 86.92 | 86.70 | 85.10 | 85.74 | 77.81 | 86.83 | 90.40    | 85.41 | 89.04 | 88.77 | 86.13 | 86.67 | 86.32 | 89.27 | 87.92    |
 
 
 #### WMT-23
 | Model                          | AVG  | AVG en-xx | AVG xx-en | AVG xx-xx | en-de | en-cs | en-uk | en-ru | en-zh-cn | de-en | uk-en | ru-en | zh-cn-en | cs-uk |
 |--------------------------------|------|-----------|-----------|-----------|-------|-------|-------|-------|----------|-------|-------|-------|----------|-------|
-| EuroLLM-1.7B-Instruct   | 83.13 |	82.91 |	82.48	| 86.87|	81.33|	85.42|	81.61|	82.57|	83.62|	84.24|	85.36|	81.56|	78.76|	86.87 |
-| Gemma-2B-EuroBlocks       | 79.86| 78.35     | 81.32     | 81.56     | 76.54 | 76.35 | 77.62 | 78.88 | 82.36    | 82.85 | 83.83 | 80.17 | 78.42    | 81.56 |
-| Gemma-7B-EuroBlocks       | 83.90| 83.70     | 83.21     | 87.61     | 82.15 | 84.68 | 83.05 | 83.85 | 84.79    | 84.40 | 85.86 | 82.55 | 80.01    | 87.61 |
+| EuroLLM-1.7B-Instruct   | 82.91     | 83.20     | 81.77     | 86.82     | 81.56     | 85.23     | 81.30     | 82.47     | 83.61     | 85.03      | 84.06      | 85.25      | 81.31      | 78.83      | 79.42      | 86.82      |
+| Gemma-2B-EuroBlocks       | 79.96     | 79.01     | 80.86     | 81.15     | 76.82     | 76.05     | 77.92     | 78.98     | 81.58     | 82.73      | 82.71      | 83.99      | 80.35      | 78.27      | 78.99      | 81.15      |
+| Gemma-7B-EuroBlocks       | 82.76     | 82.26     | 82.70     | 85.98     | 81.37     | 82.42     | 81.54     | 82.18     | 82.90     | 83.17      | 84.29      | 85.70      | 82.46      | 79.73      | 81.33      | 85.98      |
+
 
 
 #### WMT-24
 | Model | AVG | AVG en-xx | AVG xx-xx | en-es-latam | en-cs | en-ru | en-uk | en-ja | en-zh-cn | en-hi | cs-uk | ja-zh-cn |
 |---------|------|------|-------|-------|-------|-------|--------|--------|-------|-------|-------|-----|
-| EuroLLM-1.7B-Instruct|79.35|79.45|78.96|79.20|81.17|80.82|79.00|80.54|82.39|80.80|71.69|83.16|74.76|
-|Gemma-2B-EuroBlocks| 74.71|74.25|76.57|75.21|78.84|70.40|74.44|75.55|78.32|78.70|62.51|79.97|73.17|
-|Gemma-7B-EuroBlocks| 80.88|80.45|82.60|80.43|81.91|80.14|80.32|82.17|84.08|81.86|72.71|85.55|79.65|
+| EuroLLM-1.7B-Instruct|79.32     | 79.32     | 79.34     | 79.42     | 80.67     | 80.55     | 78.65     | 80.12     | 82.96     | 80.60      | 71.59      | 83.48      | 75.20      |
+|Gemma-2B-EuroBlocks| 74.72     | 74.41     | 75.97     | 74.93     | 78.81     | 70.54     | 74.90     | 75.84     | 79.48     | 78.06      | 62.70      | 79.87      | 72.07      |
+|Gemma-7B-EuroBlocks| 78.67     | 78.34     | 80.00     | 78.88     | 80.47     | 78.55     | 78.55     | 80.12     | 80.55     | 78.90      | 70.71      | 84.33      | 75.66      |
+
 
 ### General Benchmarks
 We also compare EuroLLM-1.7B with [TinyLlama-v1.1](https://huggingface.co/TinyLlama/TinyLlama_v1.1) and [Gemma-2B](https://huggingface.co/google/gemma-2b) on 3 general benchmarks: Arc Challenge and Hellaswag.