Hugging Face's logo

distily
/
distily_bitnet_gpt2

TensorBoard
Safetensors
Distily
gpt2
bitnet
1.58b
Generated from Trainer
Model card Files Metrics Community
lapp0 commited on
Commit
499676b
•
1 Parent(s): aac2fbc

Training in progress, step 61875

Browse files
Files changed (7) hide show
  1. README.md +1 -1
  2. logs/attn_layer_mapper=last, attn_loss_fn=mse, attn_weight=1.0, lr_scheduler_type=cosine, warmup_ratio=0.5/completed.flag +0 -0
  3. logs/attn_layer_mapper=last, attn_loss_fn=mse, attn_weight=1.0, lr_scheduler_type=cosine, warmup_ratio=0.5/events.out.tfevents.1724195619.5f530b1cf724 +2 -2
  4. logs/attn_layer_mapper=layer-2, attn_loss_fn=cos, attn_weight=1.0, lr_scheduler_type=cosine, warmup_ratio=0.5/events.out.tfevents.1724198385.5f530b1cf724 +3 -0
  5. logs/attn_layer_mapper=layer-2, attn_loss_fn=mse, attn_weight=1.0, lr_scheduler_type=cosine, warmup_ratio=0.5/events.out.tfevents.1724195830.5f530b1cf724 +3 -0
  6. model.safetensors +1 -1
  7. training_args.bin +1 -1
README.md CHANGED
@@ -11,7 +11,7 @@ model-index:
11
  results: []
12
  ---
13
 
14
- # `distily_bitnet_gpt2`
15
 
16
  This student model is distilled from the teacher model [gpt2](https://huggingface.co/gpt2) using the dataset (unspecified).
17
 
 
11
  results: []
12
  ---
13
 
14
+ # distily_bitnet_gpt2
15
 
16
  This student model is distilled from the teacher model [gpt2](https://huggingface.co/gpt2) using the dataset (unspecified).
17
 
logs/attn_layer_mapper=last, attn_loss_fn=mse, attn_weight=1.0, lr_scheduler_type=cosine, warmup_ratio=0.5/completed.flag ADDED
File without changes
logs/attn_layer_mapper=last, attn_loss_fn=mse, attn_weight=1.0, lr_scheduler_type=cosine, warmup_ratio=0.5/events.out.tfevents.1724195619.5f530b1cf724 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5b05be7e9885078fcb6d845af0526cd2bfd168b97235b6c721e5e16291cb3d52
3
- size 312
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:22f5f4fec271b2d98c2cc30999fda06cff344dcbb84917403f9953d665685ef6
3
+ size 588
logs/attn_layer_mapper=layer-2, attn_loss_fn=cos, attn_weight=1.0, lr_scheduler_type=cosine, warmup_ratio=0.5/events.out.tfevents.1724198385.5f530b1cf724 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:19071e8e4ca5fbae90cb3b42d55b3825b71854b8e493f29c5b021cd5511a8052
3
+ size 29652710
logs/attn_layer_mapper=layer-2, attn_loss_fn=mse, attn_weight=1.0, lr_scheduler_type=cosine, warmup_ratio=0.5/events.out.tfevents.1724195830.5f530b1cf724 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80b1b6b435b7dde3697f97d9975d9dcc3c6ae0876093ff60edd1a93bf4a1d462
3
+ size 4730039
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ae5284269e1b697587e3b8d248ea7f4737c7af54464297358a032647c01bdd9f
3
  size 248894656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0d6b083f2fe4ceea263e810563584414866c2c661402599f6cb591e48b270a8
3
  size 248894656
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:282115fc3e48d8efb4984be2466179101319883e808ce17e1686871e1397ff7c
3
  size 1017899144
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:224835a2dae2397bdf5fb703eb3f233d30a620823f43254df833c6e7564a86ec
3
  size 1017899144