Hugging Face's logo

thrunlab
/
sparse_llama_7b_hf2_refined_web_70p_2024-03-28

Text Generation
Transformers
Safetensors
sparse_llama
Generated from Trainer
custom_code
Model card Files Community
lukeleeai commited on
Commit
14c2402
·
verified ·
1 Parent(s): 7365172

End of training

Browse files
Files changed (4) hide show
  1. README.md +16 -6
  2. model-00001-of-00003.safetensors +1 -1
  3. model-00002-of-00003.safetensors +1 -1
  4. model-00003-of-00003.safetensors +1 -1
README.md CHANGED
@@ -14,7 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
14
 
15
  This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on the None dataset.
16
  It achieves the following results on the evaluation set:
17
- - Loss: 2.1400
18
 
19
  ## Model description
20
 
@@ -44,7 +44,7 @@ The following hyperparameters were used during training:
44
  - total_eval_batch_size: 16
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
- - training_steps: 2350
48
 
49
  ### Training results
50
 
@@ -140,10 +140,20 @@ The following hyperparameters were used during training:
140
  | 2.2717 | 0.35 | 2200 | 2.2825 |
141
  | 2.1958 | 0.36 | 2225 | 2.2827 |
142
  | 2.1635 | 0.36 | 2250 | 2.2867 |
143
- | 2.3032 | 0.36 | 2275 | 2.2841 |
144
- | 2.3084 | 0.37 | 2300 | 2.2846 |
145
- | 2.1975 | 0.37 | 2325 | 2.2834 |
146
- | 2.2212 | 0.38 | 2350 | 2.2828 |
 
 
 
 
 
 
 
 
 
 
147
 
148
 
149
  ### Framework versions
 
14
 
15
  This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on the None dataset.
16
  It achieves the following results on the evaluation set:
17
+ - Loss: 2.1370
18
 
19
  ## Model description
20
 
 
44
  - total_eval_batch_size: 16
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
+ - training_steps: 2600
48
 
49
  ### Training results
50
 
 
140
  | 2.2717 | 0.35 | 2200 | 2.2825 |
141
  | 2.1958 | 0.36 | 2225 | 2.2827 |
142
  | 2.1635 | 0.36 | 2250 | 2.2867 |
143
+ | 2.3043 | 0.36 | 2275 | 2.2820 |
144
+ | 2.3106 | 0.37 | 2300 | 2.2845 |
145
+ | 2.1992 | 0.37 | 2325 | 2.2829 |
146
+ | 2.2189 | 0.38 | 2350 | 2.2817 |
147
+ | 2.3096 | 0.38 | 2375 | 2.2830 |
148
+ | 2.2803 | 0.38 | 2400 | 2.2839 |
149
+ | 2.1752 | 0.39 | 2425 | 2.2825 |
150
+ | 2.1324 | 0.39 | 2450 | 2.2834 |
151
+ | 2.207 | 0.4 | 2475 | 2.2846 |
152
+ | 2.3369 | 0.4 | 2500 | 2.2844 |
153
+ | 2.3512 | 0.4 | 2525 | 2.2853 |
154
+ | 2.1432 | 0.41 | 2550 | 2.2861 |
155
+ | 2.1743 | 0.41 | 2575 | 2.2832 |
156
+ | 2.2696 | 0.42 | 2600 | 2.2832 |
157
 
158
 
159
  ### Framework versions
model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:46332f63aaed4988d57cd0a8430eb291dd6408f1a0edd2a3907cd5e636eabea5
3
  size 4938985352
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50e638e2dcee3f62f30d649753a4e0c5873f07f48fd0534ee5da240911197a62
3
  size 4938985352
model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:01046014a6305273fb40c751b7c90fa5f1e2f335f6cebd326c15b404992da4e8
3
  size 4947390880
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc8827b836d19c7ef8519c38bf7492c1be686b7a84dc372b1235e1a11b1fde1f
3
  size 4947390880
model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1e3ee84cc62fac73a24dd8334a9ec4cc43b640ca08166177bc3bd936d43b9c3c
3
  size 3590488816
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c6f9a673c42afa9fbd42506dc3d04562c20fc2ff9b6dfca8bbd63e6f6bcd6e30
3
  size 3590488816