Hugging Face's logo

mnoukhov
/
pythia160m-sft-tldr

Text Generation
Transformers
Safetensors
gpt_neox
trl
sft
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
mnoukhov commited on
Commit
40c0022
•
1 Parent(s): 5886bde

Model save

Browse files
Files changed (2) hide show
  1. README.md +7 -7
  2. model.safetensors +1 -1
README.md CHANGED
@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [EleutherAI/pythia-160m-deduped](https://huggingface.co/EleutherAI/pythia-160m-deduped) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 2.7673
21
 
22
  ## Model description
23
 
@@ -36,7 +36,7 @@ More information needed
36
  ### Training hyperparameters
37
 
38
  The following hyperparameters were used during training:
39
- - learning_rate: 3e-05
40
  - train_batch_size: 16
41
  - eval_batch_size: 8
42
  - seed: 42
@@ -47,16 +47,16 @@ The following hyperparameters were used during training:
47
  - total_eval_batch_size: 32
48
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
  - lr_scheduler_type: cosine
50
- - num_epochs: 1
51
 
52
  ### Training results
53
 
54
  | Training Loss | Epoch | Step | Validation Loss |
55
  |:-------------:|:------:|:----:|:---------------:|
56
- | 3.1053 | 0.2007 | 183 | 2.8980 |
57
- | 2.8263 | 0.4013 | 366 | 2.8973 |
58
- | 2.7995 | 0.6020 | 549 | 2.7808 |
59
- | 2.763 | 0.8026 | 732 | 2.7673 |
60
 
61
 
62
  ### Framework versions
 
17
 
18
  This model is a fine-tuned version of [EleutherAI/pythia-160m-deduped](https://huggingface.co/EleutherAI/pythia-160m-deduped) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 2.7556
21
 
22
  ## Model description
23
 
 
36
  ### Training hyperparameters
37
 
38
  The following hyperparameters were used during training:
39
+ - learning_rate: 1e-05
40
  - train_batch_size: 16
41
  - eval_batch_size: 8
42
  - seed: 42
 
47
  - total_eval_batch_size: 32
48
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
  - lr_scheduler_type: cosine
50
+ - num_epochs: 2.0
51
 
52
  ### Training results
53
 
54
  | Training Loss | Epoch | Step | Validation Loss |
55
  |:-------------:|:------:|:----:|:---------------:|
56
+ | 2.8454 | 0.4002 | 365 | 2.8609 |
57
+ | 2.7873 | 0.8004 | 730 | 2.7859 |
58
+ | 2.7641 | 1.2007 | 1095 | 2.7616 |
59
+ | 2.7486 | 1.6009 | 1460 | 2.7556 |
60
 
61
 
62
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2aba2b6997d01ae5edc4fd21dbb81d91c7416691d5cf845b5dea4d2b45706b5e
3
  size 649308728
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7778cec0e0dd014ba282e2b6822b093b96234f1e50f7103a2cbe7c564b5c507c
3
  size 649308728