End of training

README.md

@@ -114,9 +114,9 @@ LlamaForCausalLM(
 <br/>
 
 # Train Dataset
-Trained on 687,248,443 tokens from the [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) dataset.
+Trained on 553,266,374 tokens from the [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) dataset.
 
-- Num Samples: `1,996,000`
+- Num Samples: `998,000`
 - Subset: `20231101.en`
 - Split: `train`
 
@@ -163,7 +163,7 @@ The following hyperparameters were used during training:
         weight=0
     )
 )`
-- lr_scheduler: `<torch.optim.lr_scheduler.LambdaLR object at 0x76c873da1e40>`
+- lr_scheduler: `<torch.optim.lr_scheduler.LambdaLR object at 0x76ca190e3fd0>`
 - student_model_name_or_path: `None`
 - student_config_name_or_path: `None`
 - student_model_config: `{'num_hidden_layers': 15}`
@@ -178,8 +178,8 @@ The following hyperparameters were used during training:
 - dataset_subset: `20231101.en`
 - dataset_split: `train`
 - dataset_column_name: `text`
-- dataset_sample_size: `2000000`
-- dataset_max_seq_length: `512`
+- dataset_sample_size: `1000000`
+- dataset_max_seq_length: `1024`
 - dataset_test_size: `0.002`
 - dataset_shuffle: `False`
 - dataset_shuffle_seed: `42`

logs/dataset_max_seq_length=1024, dataset_sample_size=1000000, per_device_train_batch_size=4/events.out.tfevents.1726578027.1c1a426a2fee

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5e963819a9110e6b02e72aacfad2c6edae23f8294c41820d65d703d34ebf4ee
+size 529