|
--- |
|
base_model: google/gemma-2-2b |
|
datasets: mlabonne/TheTome |
|
--- |
|
# Distil Gemma 2 2b |
|
|
|
This model is a gemma 2 2b model distilled from google/gemma-2-9b-it and finetuned on the tome. |
|
|
|
 |
|
|
|
## Prompt Template |
|
|
|
ChatML |
|
|
|
``` |
|
<|im_start|>system |
|
{system}<|im_end|> |
|
<|im_start|>user |
|
{user}<|im_end|> |
|
<|im_start|>assistant |
|
``` |
|
|
|
## Training Information |
|
|
|
This model trained on 8x Nvidia H100 NVL for the equivalent of 120 GPU hours. |
|
|
|
+ Loss Achieved: 0.27 |
|
+ Epochs: 3 |
|
|
|
Checkpoints are available in the repo to continue training |
|
## Evals |
|
|
|
IN PROGRESS |
