Hugging Face's logo

HachiML
/
Mists-7B-v01-simple-projector-trained

Feature Extraction
Transformers
Safetensors
mists
trl
sft
Generated from Trainer
custom_code
Model card Files Community
Mists-7B-v01-simple-projector-trained / README.md
HachiML's picture
HachiML
Upload processor
9396935 verified
preview code
|
raw
history blame contribute delete
No virus
2.72 kB
metadata
base_model: HachiML/Mists-7B-v01-not-trained
license: apache-2.0
tags:
  - trl
  - sft
  - generated_from_trainer
model-index:
  - name: Mists-7B-v01-simple-projector-trained
    results: []

Visualize in Weights & Biases

Mists-7B-v01-simple-projector-trained

This model is a fine-tuned version of HachiML/Mists-7B-v01-not-trained on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0152

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.002
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.05
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
0.3727 0.0444 100 0.0301
0.0274 0.0888 200 0.0243
0.0615 0.1332 300 0.0382
0.0367 0.1776 400 0.0325
0.0327 0.2220 500 0.0304
0.0304 0.2664 600 0.0243
0.0242 0.3108 700 0.0216
0.0236 0.3552 800 0.0214
0.0226 0.3996 900 0.0188
0.0206 0.4440 1000 0.0181
0.0197 0.4885 1100 0.0188
0.0192 0.5329 1200 0.0175
0.019 0.5773 1300 0.0171
0.018 0.6217 1400 0.0166
0.0173 0.6661 1500 0.0169
0.017 0.7105 1600 0.0163
0.0172 0.7549 1700 0.0164
0.0186 0.7993 1800 0.0161
0.0168 0.8437 1900 0.0167
0.0161 0.8881 2000 0.0163
0.0157 0.9325 2100 0.0156
0.016 0.9769 2200 0.0152

Framework versions

  • Transformers 4.42.3
  • Pytorch 2.3.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1