Update README.md

README.md

@@ -4,118 +4,19 @@ language:
 - fr
 - es
 - pt
-license: other
-library_name: transformers
 tags:
 - falcon3
-license_name: falcon-llm-license
+license: other 
+license_name: falcon-llm-license 
 license_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html
-model-index:
-- name: Falcon3-10B-Base
-  results:
-  - task:
-      type: text-generation
-      name: Text Generation
-    dataset:
-      name: IFEval (0-Shot)
-      type: HuggingFaceH4/ifeval
-      args:
-        num_few_shot: 0
-    metrics:
-    - type: inst_level_strict_acc and prompt_level_strict_acc
-      value: 36.48
-      name: strict accuracy
-    source:
-      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/Falcon3-10B-Base
-      name: Open LLM Leaderboard
-  - task:
-      type: text-generation
-      name: Text Generation
-    dataset:
-      name: BBH (3-Shot)
-      type: BBH
-      args:
-        num_few_shot: 3
-    metrics:
-    - type: acc_norm
-      value: 41.38
-      name: normalized accuracy
-    source:
-      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/Falcon3-10B-Base
-      name: Open LLM Leaderboard
-  - task:
-      type: text-generation
-      name: Text Generation
-    dataset:
-      name: MATH Lvl 5 (4-Shot)
-      type: hendrycks/competition_math
-      args:
-        num_few_shot: 4
-    metrics:
-    - type: exact_match
-      value: 24.77
-      name: exact match
-    source:
-      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/Falcon3-10B-Base
-      name: Open LLM Leaderboard
-  - task:
-      type: text-generation
-      name: Text Generation
-    dataset:
-      name: GPQA (0-shot)
-      type: Idavidrein/gpqa
-      args:
-        num_few_shot: 0
-    metrics:
-    - type: acc_norm
-      value: 12.75
-      name: acc_norm
-    source:
-      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/Falcon3-10B-Base
-      name: Open LLM Leaderboard
-  - task:
-      type: text-generation
-      name: Text Generation
-    dataset:
-      name: MuSR (0-shot)
-      type: TAUR-Lab/MuSR
-      args:
-        num_few_shot: 0
-    metrics:
-    - type: acc_norm
-      value: 14.17
-      name: acc_norm
-    source:
-      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/Falcon3-10B-Base
-      name: Open LLM Leaderboard
-  - task:
-      type: text-generation
-      name: Text Generation
-    dataset:
-      name: MMLU-PRO (5-shot)
-      type: TIGER-Lab/MMLU-Pro
-      config: main
-      split: test
-      args:
-        num_few_shot: 5
-    metrics:
-    - type: acc
-      value: 36.0
-      name: accuracy
-    source:
-      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/Falcon3-10B-Base
-      name: Open LLM Leaderboard
 ---
 
-<div align="center">
-    <img src="https://huggingface.co/datasets/tiiuae/documentation-images/resolve/main/general/falco3-logo.png" alt="drawing" width="500"/>
-</div>
 
 # Falcon3-10B-Base
 
 **Falcon3** family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B parameters.
 
-This repository contains the **Falcon3-10B-Base**. It achieves state-of-the-art results (at the time of release) on reasoning, language understanding, instruction following, code and mathematics tasks.
+This repository contains the **Falcon3-10B-Base**. It achieves state-of-the-art results (at release's time) on reasoning, language understanding, instruction following, code and mathematics tasks.
 Falcon3-10B-Base supports 4 languages (English, French, Spanish, Portuguese) and a context length of up to 32K.
 
 ⚠️ **This is a raw, pretrained model, which should be further finetuned using SFT, RLHF, continued pretraining, etc. for most use cases.** 
@@ -130,7 +31,7 @@ Falcon3-10B-Base supports 4 languages (English, French, Spanish, Portuguese) and
   - Uses SwiGLu and RMSNorm
   - 32K context length
   - 131K vocab size
-- Depth up-scaled from **Falcon3-7B-Base** with continual pretraining on 2 Teratokens of datasets comprising of web, code, STEM, high quality and mutlilingual data using 1024 H100 GPU chips
+- Depth up-scaled from **Falcon3-7B-Base** with 2 Teratokens of datasets comprising of web, code, STEM, high quality and mutlilingual data using 2048 H100 GPU chips
 - Supports EN, FR, ES, PT
 - Developed by [Technology Innovation Institute](https://www.tii.ae)
 - License: TII Falcon-LLM License 2.0
@@ -161,11 +62,8 @@ print(response[0]['generated_text'])
 <br>
 
 ## Benchmarks
-We report in the following table our internal pipeline benchmarks.
- - We use [lm-evaluation harness](https://github.com/EleutherAI/lm-evaluation-harness).
- - We report **raw scores**.
- - We use same batch-size across all models.
-   
+We report in the following table our internal pipeline benchmarks:
+
 <table border="1" style="width: 100%; text-align: center; border-collapse: collapse;">
     <colgroup>
         <col style="width: 10%;">
@@ -181,7 +79,7 @@ We report in the following table our internal pipeline benchmarks.
             <th>Benchmark</th>
             <th>Gemma2-9B</th>
             <th>Yi1.5-9B</th>
-            <th>Mistral-Nemo-Base-2407 (12B)</th>
+            <th>Mistral-NeMo-12B</th>
             <th>Falcon3-10B-Base</th>
         </tr>
     </thead>
@@ -203,7 +101,7 @@ We report in the following table our internal pipeline benchmarks.
         </tr>
         <tr>
             <td>IFEval</td>
-            <td>21.3</td>
+            <td>21.2</td>
             <td>29.1</td>
             <td>16.1</td>
             <td><b>36.4</b></td>
@@ -217,7 +115,7 @@ We report in the following table our internal pipeline benchmarks.
             <td><b>81.4</b></td>
         </tr>
         <tr>
-            <td>MATH Lvl-5 (4-shot)</td>
+            <td>MATH(4-shot)</td>
             <td>10.5</td>
             <td>9.2</td>
             <td>4.9</td>
@@ -240,7 +138,7 @@ We report in the following table our internal pipeline benchmarks.
         </tr>
         <tr>
             <td>MUSR (0-shot)</td>
-            <td><b>45.3</b></td>
+            <td><b>45.2</b></td>
             <td>43.3</td>
             <td>39.2</td>
             <td>44.2</td>
@@ -258,7 +156,7 @@ We report in the following table our internal pipeline benchmarks.
             <td><b>83.0</b></td>
             <td>80.5</td>
             <td>82.1</td>
-            <td>79.4</td>
+            <td>79.5</td>
         </tr>
         <tr>
             <td>SciQ (0-shot)</td>
@@ -284,36 +182,20 @@ We report in the following table our internal pipeline benchmarks.
     </tbody>
 </table>
 
-## Useful links
-- View our [release blogpost](https://huggingface.co/blog/falcon3).
-- Feel free to join [our discord server](https://discord.gg/fwXpMyGc) if you have any questions or to interact with our researchers and developers.
-
 ## Technical Report
 
 Coming soon....
 
 ## Citation
-If the Falcon3 family of models were helpful to your work, feel free to give us a cite.
- 
+If Falcon3 family were helpful in your work, feel free to give us a cite.
+
 ```
 @misc{Falcon3,
-    title = {The Falcon 3 Family of Open Models},
-    url = {https://huggingface.co/blog/falcon3},
-    author = {Falcon-LLM Team},
+    title = {The Falcon 3 family of Open Models},
+    author = {TII Team},
     month = {December},
     year = {2024}
 }
 ```
-# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
-Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/tiiuae__Falcon3-10B-Base-details)
 
-|      Metric       |Value|
-|-------------------|----:|
-|Avg.               |27.59|
-|IFEval (0-Shot)    |36.48|
-|BBH (3-Shot)       |41.38|
-|MATH Lvl 5 (4-Shot)|24.77|
-|GPQA (0-shot)      |12.75|
-|MuSR (0-shot)      |14.17|
-|MMLU-PRO (5-shot)  |36.00|