Update README.md

README.md

@@ -1,92 +1,229 @@
 ---
-base_model: loubnabnl/smollm2-1.7B-8k-mix7-ep2-v2
-tags:
-- alignment-handbook
-- trl
-- dpo
-- generated_from_trainer
-- trl
-- dpo
-- generated_from_trainer
-datasets:
-- HuggingFaceH4/ultrafeedback_binarized
-model-index:
-- name: smollm2-1.7B-8k-mix7-ep2-v2-dpo-ultraf-ep3
-  results: []
+library_name: transformers
+license: apache-2.0
+language:
+- en
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/loubnabnl/huggingface/runs/n6jjrqmx)
-# smollm2-1.7B-8k-mix7-ep2-v2-dpo-ultraf-ep3
-
-This model is a fine-tuned version of [loubnabnl/smollm2-1.7B-8k-mix7-ep2-v2](https://huggingface.co/loubnabnl/smollm2-1.7B-8k-mix7-ep2-v2) on the HuggingFaceH4/ultrafeedback_binarized dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.5878
-- Rewards/chosen: 0.0167
-- Rewards/rejected: -0.5739
-- Rewards/accuracies: 0.6746
-- Rewards/margins: 0.5907
-- Logps/rejected: -275.4315
-- Logps/chosen: -310.2510
-- Logits/rejected: -0.3685
-- Logits/chosen: -0.3410
-
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 1e-06
-- train_batch_size: 2
-- eval_batch_size: 4
-- seed: 42
-- distributed_type: multi-GPU
-- num_devices: 8
-- gradient_accumulation_steps: 8
-- total_train_batch_size: 128
-- total_eval_batch_size: 32
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: cosine
-- lr_scheduler_warmup_ratio: 0.1
-- num_epochs: 3
-
-### Training results
-
-| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
-|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.6787        | 0.2094 | 100  | 0.6967          | 0.0159         | -0.0702          | 0.5516             | 0.0861          | -274.4240      | -310.2527    | -0.3377         | -0.3141       |
-| 0.645         | 0.4187 | 200  | 0.6491          | -0.0498        | -0.3020          | 0.6032             | 0.2523          | -274.8876      | -310.3840    | -0.3463         | -0.3229       |
-| 0.6161        | 0.6281 | 300  | 0.6316          | -0.0637        | -0.4218          | 0.6825             | 0.3581          | -275.1272      | -310.4119    | -0.3552         | -0.3317       |
-| 0.5964        | 0.8375 | 400  | 0.6100          | -0.0166        | -0.4381          | 0.6587             | 0.4215          | -275.1597      | -310.3176    | -0.3545         | -0.3291       |
-| 0.5394        | 1.0468 | 500  | 0.6066          | -0.0098        | -0.4749          | 0.7103             | 0.4651          | -275.2332      | -310.3040    | -0.3576         | -0.3320       |
-| 0.5099        | 1.2562 | 600  | 0.6007          | -0.0192        | -0.5329          | 0.6786             | 0.5137          | -275.3493      | -310.3229    | -0.3635         | -0.3380       |
-| 0.5056        | 1.4656 | 700  | 0.5876          | -0.0630        | -0.5941          | 0.6905             | 0.5311          | -275.4717      | -310.4104    | -0.3672         | -0.3407       |
-| 0.4936        | 1.6750 | 800  | 0.5994          | -0.0296        | -0.5590          | 0.6746             | 0.5294          | -275.4016      | -310.3437    | -0.3658         | -0.3384       |
-| 0.4904        | 1.8843 | 900  | 0.5989          | -0.0581        | -0.6149          | 0.6944             | 0.5568          | -275.5134      | -310.4006    | -0.3705         | -0.3443       |
-| 0.4622        | 2.0937 | 1000 | 0.5939          | -0.0662        | -0.6068          | 0.6944             | 0.5405          | -275.4971      | -310.4169    | -0.3724         | -0.3450       |
-| 0.4458        | 2.3031 | 1100 | 0.5923          | -0.0536        | -0.6393          | 0.6944             | 0.5857          | -275.5622      | -310.3918    | -0.3728         | -0.3450       |
-| 0.4462        | 2.5124 | 1200 | 0.5894          | -0.0486        | -0.6300          | 0.7024             | 0.5814          | -275.5435      | -310.3816    | -0.3710         | -0.3432       |
-| 0.4312        | 2.7218 | 1300 | 0.5861          | -0.0751        | -0.6393          | 0.6667             | 0.5642          | -275.5621      | -310.4347    | -0.3724         | -0.3442       |
-| 0.4454        | 2.9312 | 1400 | 0.5942          | -0.0056        | -0.5970          | 0.6944             | 0.5914          | -275.4775      | -310.2956    | -0.3681         | -0.3401       |
-
-
-### Framework versions
-
-- Transformers 4.42.3
-- Pytorch 2.1.2
-- Datasets 2.20.0
-- Tokenizers 0.19.1
+
+# SmolLM2
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/61c141342aac764ce1654e43/7IzejwZJ62MfRwvDYvQXY.png)
+
+##  Table of Contents
+
+1. [Model Summary](#model-summary)
+2. [Evaluation](#evaluation)
+3. [Examples](#examples)
+4. [Limitations](#limitations)
+5. [Training](#training)
+6. [License](#license)
+7. [Citation](#citation)
+
+## Model Summary
+
+SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device.
+
+The 1.7B variant demonstrates significant advances over its predecessor SmolLM1-1.7B, particularly in instruction following, knowledge, reasoning, and mathematics. It was trained on 11 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new mathematics and coding datasets that we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own curated datasets. We then applied Direct Preference Optimization (DPO) using [UltraFeedback](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized).
+
+The instruct model additionally supports tasks such as text rewriting, summarization and function calling thanks to datasets developed by [Argilla](https://huggingface.co/argilla) such as [Synth-APIGen-v0.1](https://huggingface.co/datasets/argilla/Synth-APIGen-v0.1).
+
+### How to use
+
+### Transformers
+```bash
+pip install transformers
+```
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+checkpoint = "HuggingFaceTB/SmolLM2-1.7B-Instruct"
+
+device = "cuda" # for GPU usage or "cpu" for CPU usage
+tokenizer = AutoTokenizer.from_pretrained(checkpoint)
+# for multiple GPUs install accelerate and do `model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto")`
+model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
+
+messages = [{"role": "user", "content": "What is the capital of France."}]
+input_text=tokenizer.apply_chat_template(messages, tokenize=False)
+inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
+outputs = model.generate(inputs, max_new_tokens=50, temperature=0.2, top_p=0.9, do_sample=True)
+print(tokenizer.decode(outputs[0]))
+```
+
+
+### Chat in TRL
+You can also use the TRL CLI to chat with the model from the terminal:
+```bash
+pip install trl
+trl chat --model_name_or_path HuggingFaceTB/SmolLM2-1.7B-Instruct --device cpu
+```
+
+## Evaluation
+
+In this section, we report the evaluation results of SmolLM2. All evaluations are zero-shot unless stated otherwise, and we use [lighteval](https://github.com/huggingface/lighteval) to run them.
+
+## Base Pre-Trained Model
+
+| Metric           | SmolLM2-1.7B | Llama-1B    | Qwen2.5-1.5B | SmolLM1-1.7B |
+|------------------|--------------|-------------|---------------|--------------|
+| HellaSwag        | **68.7**     | 61.2        | 66.4          | 62.9         |
+| ARC (Average)    | **60.5**     | 49.2        | 58.5          | 59.9         |
+| PIQA             | **77.6**     | 74.8        | 76.1          | 76.0         |
+| MMLU-Pro (MCF)   | **19.4**     | 11.7        | 13.7          | 10.8         |
+| CommonsenseQA    | **43.6**     | 41.2        | 34.1          | 38.0         |
+| TriviaQA         | **36.7**     | 28.1        | 20.9          | 22.5         |
+| Winogrande       | **59.4**     | 57.8        | 59.3          | 54.7         |
+| OpenBookQA       | 42.2         | 38.4        | 40.0          | **42.4**     |
+| GSM8K (5-shot)   | 31.0         | 7.2         | **61.3**      | 5.5          |
+
+## Instruction Model
+
+| Metric                       | SmolLM2-1.7B-Instruct | Llama-1B-Instruct | Qwen2.5-1.5B-Instruct | SmolLM1-1.7B-Instruct |
+|:-----------------------------|:---------------------:|:-----------------:|:----------------------:|:----------------------:|
+| IFEval (Average prompt/inst) | **56.7**             | 53.5             | 47.4                  | 23.1                  |
+| MT-Bench                     | 6.13                | 5.48             | **6.52**              | 4.33                  |
+| OpenRewrite-Eval (micro_avg RougeL) | 44.9           | 39.2             | **46.9**              | NaN                   |
+| HellaSwag                    | **66.1**            | 56.1             | 60.9                  | 55.5                  |
+| ARC (Average)                | **51.7**            | 41.6             | 46.2                  | 43.7                  |
+| PIQA                         | **74.4**            | 72.3             | 73.2                  | 71.6                  |
+| MMLU-Pro (MCF)               | 19.3               | 12.7             | **24.2**              | 11.7                  |
+| BBH (3-shot)                 | 32.2               | 27.6             | **35.3**              | 25.7                  |
+| GSM8K (5-shot)               | **48.2**           | 26.8             | 42.8                  | 4.62                  |
+
+
+## Examples
+Below are some system and instruct prompts that work well for special tasks
+
+### Text rewriting
+
+```python
+system_prompt_rewrite = "You are an AI writing assistant. Your task is to rewrite the user's email to make it more professional and approachable while maintaining its main points and key message. Do not return any text other than the rewritten message."
+user_prompt_rewrite = "Rewrite the message below to make it more friendly and approachable while maintaining its main points and key message. Do not add any new information or return any text other than the rewritten message\nThe message:"
+messages = [{"role": "system", "content": system_prompt_rewrite}, {"role": "user", "content":f"{user_prompt_rewrite} The CI is failing after your last commit!}"]
+input_text=tokenizer.apply_chat_template(messages, tokenize=False)
+inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
+outputs = model.generate(inputs, max_new_tokens=50, temperature=0.2, top_p=0.9, do_sample=True)
+print(tokenizer.decode(outputs[0]))
+```
+```
+Hey there! I noticed that the CI isn't passing after your latest commit. Could you take a look and let me know what's going on? Thanks so much for your help!
+```
+
+### Summarization
+
+```python
+system_prompt_summarize = "Provide a concise, objective summary of the input text in up to three sentences, focusing on key actions and intentions without using second or third person pronouns."
+messages = [{"role": "system", "content": system_prompt_rewrite}, {"role": "user", "content": INSERT_LONG_EMAIL]
+input_text=tokenizer.apply_chat_template(messages, tokenize=False)
+inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
+outputs = model.generate(inputs, max_new_tokens=50, temperature=0.2, top_p=0.9, do_sample=True)
+print(tokenizer.decode(outputs[0]))
+```
+
+### Function calling
+
+SmolLM2-1.7B-Instruct can handle function calling, it scores 27% on the [BFCL Leaderboard](https://gorilla.cs.berkeley.edu/blogs/8_berkeley_function_calling_leaderboard.html). Here's how you can leverage it:
+
+```python
+import json
+import re
+from typing import Optional
+
+from jinja2 import Template
+import torch 
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from transformers.utils import get_json_schema
+
+
+system_prompt = Template("""You are an expert in composing functions. You are given a question and a set of possible functions. 
+Based on the question, you will need to make one or more function/tool calls to achieve the purpose. 
+If none of the functions can be used, point it out and refuse to answer. 
+If the given question lacks the parameters required by the function, also point it out.
+
+You have access to the following tools:
+<tools>{{ tools }}</tools>
+
+The output MUST strictly adhere to the following format, and NO other text MUST be included.
+The example format is as follows. Please make sure the parameter type is correct. If no function call is needed, please make the tool calls an empty list '[]'.
+<tool_call>[
+{"name": "func_name1", "arguments": {"argument1": "value1", "argument2": "value2"}},
+... (more tool calls as required)
+]</tool_call>""")
+
+
+def prepare_messages(
+    query: str,
+    tools: Optional[dict[str, any]] = None,
+    history: Optional[list[dict[str, str]]] = None
+) -> list[dict[str, str]]:
+    """Prepare the system and user messages for the given query and tools.
+    
+    Args:
+        query: The query to be answered.
+        tools: The tools available to the user. Defaults to None, in which case if a
+            list without content will be passed to the model.
+        history: Exchange of messages, including the system_prompt from
+            the first query. Defaults to None, the first message in a conversation.
+    """
+    if tools is None:
+        tools = []
+    if history:
+        messages = history.copy()
+        messages.append({"role": "user", "content": query})
+    else:
+        messages = [
+            {"role": "system", "content": system_prompt.render(tools=json.dumps(tools))},
+            {"role": "user", "content": query}
+        ]
+    return messages
+
+
+def parse_response(text: str) -> str | dict[str, any]:
+    """Parses a response from the model, returning either the
+    parsed list with the tool calls parsed, or the
+    model thought or response if couldn't generate one.
+
+    Args:
+        text: Response from the model.
+    """
+    pattern = r"<tool_call>(.*?)</tool_call>"
+    matches = re.findall(pattern, text, re.DOTALL)
+    if matches:
+        return json.loads(matches[0])
+    return text
+```
+
+## Limitations
+
+SmolLM2 models primarily understand and generate content in English. They can produce text on a variety of topics, but the generated content may not always be factually accurate, logically consistent, or free from biases present in the training data. These models should be used as assistive tools rather than definitive sources of information. Users should always verify important information and critically evaluate any generated content.
+
+## Training
+
+### Model
+
+- **Architecture:** Transformer decoder
+- **Pretraining tokens:** 11T
+- **Precision:** bfloat16
+
+### Hardware
+
+- **GPUs:** 256 H100
+
+### Software
+
+- **Training Framework:** [nanotron](https://github.com/huggingface/nanotron/tree/main)
+- **Alignement Handbook** [alignement-handbook](https://github.com/huggingface/alignment-handbook/)
+
+## License
+
+[Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+
+## Citation
+```bash
+@misc{allal2024SmolLM2,
+      title={SmolLM2 - with great data, comes great performance}, 
+      author={Loubna Ben Allal and Anton Lozhkov and Elie Bakouch and Gabriel Martín Blázquez and Lewis Tunstall and Agustín Piqueres and Andres Marafioti and Cyril Zakka and Leandro von Werra and Thomas Wolf},
+      year={2024},
+}
+```