Commit
•
6c63a6b
1
Parent(s):
f98dfc6
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,7 +1,19 @@
|
|
| 1 |
-
```markdown
|
| 2 |
---
|
| 3 |
library_name: transformers
|
| 4 |
-
tags:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
---
|
| 6 |
|
| 7 |
# Model Card for `SmolLM-360M-Instruct-finetuned-sft`
|
|
@@ -60,11 +72,9 @@ print(response)
|
|
| 60 |
## Training Details
|
| 61 |
|
| 62 |
### Training Data
|
| 63 |
-
|
| 64 |
-
The model was fine-tuned using the [HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2) dataset, which consists of approximately 21,400 examples of instruction-based prompts and corresponding responses. The dataset is designed to enhance AI models' ability to generate helpful, correct, and coherent outputs.
|
| 65 |
|
| 66 |
### Training Procedure
|
| 67 |
-
|
| 68 |
The fine-tuning was performed using the following hyperparameters:
|
| 69 |
|
| 70 |
- **Training regime:** Mixed precision (FP16)
|
|
@@ -78,16 +88,13 @@ The fine-tuning was performed using the following hyperparameters:
|
|
| 78 |
## Evaluation
|
| 79 |
|
| 80 |
### Testing Data, Factors & Metrics
|
| 81 |
-
|
| 82 |
The model was evaluated using a validation subset of the HelpSteer2 dataset.
|
| 83 |
|
| 84 |
#### Metrics
|
| 85 |
-
|
| 86 |
- **Training Loss:** Final loss was 5.4814.
|
| 87 |
- **Validation Loss:** Final loss was 5.4625.
|
| 88 |
|
| 89 |
### Results
|
| 90 |
-
|
| 91 |
The model demonstrated a consistent decrease in both training and validation losses, indicating effective learning and good generalization.
|
| 92 |
|
| 93 |
## Environmental Impact
|
|
@@ -95,5 +102,4 @@ The model demonstrated a consistent decrease in both training and validation los
|
|
| 95 |
Carbon emissions for the training process were minimal due to the efficient use of the NVIDIA A100 GPU, which allowed for rapid fine-tuning within an hour.
|
| 96 |
|
| 97 |
- **Hardware Type:** NVIDIA A100 GPU
|
| 98 |
-
- **Hours used:** Less than 1 hour
|
| 99 |
-
```
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
library_name: transformers
|
| 3 |
+
tags:
|
| 4 |
+
- language-model
|
| 5 |
+
- fine-tuned
|
| 6 |
+
- instruction-following
|
| 7 |
+
- SmolLM
|
| 8 |
+
- HelpSteer2
|
| 9 |
+
- NVIDIA
|
| 10 |
+
- A100
|
| 11 |
+
- English
|
| 12 |
+
language: en
|
| 13 |
+
license: apache-2.0
|
| 14 |
+
datasets:
|
| 15 |
+
- nvidia/HelpSteer2
|
| 16 |
+
model_name: SmolLM-360M-Instruct-finetuned-sft
|
| 17 |
---
|
| 18 |
|
| 19 |
# Model Card for `SmolLM-360M-Instruct-finetuned-sft`
|
|
|
|
| 72 |
## Training Details
|
| 73 |
|
| 74 |
### Training Data
|
| 75 |
+
The model was fine-tuned using the [HelpSteer2 dataset](https://huggingface.co/datasets/nvidia/HelpSteer2), which consists of approximately 21,400 examples of instruction-based prompts and corresponding responses. The dataset is designed to enhance AI models' ability to generate helpful, correct, and coherent outputs.
|
|
|
|
| 76 |
|
| 77 |
### Training Procedure
|
|
|
|
| 78 |
The fine-tuning was performed using the following hyperparameters:
|
| 79 |
|
| 80 |
- **Training regime:** Mixed precision (FP16)
|
|
|
|
| 88 |
## Evaluation
|
| 89 |
|
| 90 |
### Testing Data, Factors & Metrics
|
|
|
|
| 91 |
The model was evaluated using a validation subset of the HelpSteer2 dataset.
|
| 92 |
|
| 93 |
#### Metrics
|
|
|
|
| 94 |
- **Training Loss:** Final loss was 5.4814.
|
| 95 |
- **Validation Loss:** Final loss was 5.4625.
|
| 96 |
|
| 97 |
### Results
|
|
|
|
| 98 |
The model demonstrated a consistent decrease in both training and validation losses, indicating effective learning and good generalization.
|
| 99 |
|
| 100 |
## Environmental Impact
|
|
|
|
| 102 |
Carbon emissions for the training process were minimal due to the efficient use of the NVIDIA A100 GPU, which allowed for rapid fine-tuning within an hour.
|
| 103 |
|
| 104 |
- **Hardware Type:** NVIDIA A100 GPU
|
| 105 |
+
- **Hours used:** Less than 1 hour
|
|
|