Alfahluzi/bert2bert-dropout-0.3-lr-5e-05-ds-canonical add 5 more epoch (total 5)

README.md

@@ -13,7 +13,12 @@ should probably proofread and complete it, then remove this comment. -->
 
 # bert2bert-dropout-0.3-lr-5e-05-ds-canonical
 
-This model is a fine-tuned version of [](https://huggingface.co/) on the id_liputan6 dataset.
+This model was trained from scratch on the id_liputan6 dataset.
+It achieves the following results on the evaluation set:
+- Loss: 2.3368
+- Rouge2 Precision: 0.1701
+- Rouge2 Recall: 0.181
+- Rouge2 Fmeasure: 0.1731
 
 ## Model description
 
@@ -33,24 +38,26 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 5e-05
-- train_batch_size: 4
-- eval_batch_size: 4
+- train_batch_size: 48
+- eval_batch_size: 48
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 1
+- num_epochs: 3
 - mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Rouge2 Precision | Rouge2 Recall | Rouge2 Fmeasure |
-|:-------------:|:-----:|:----:|:---------------:|:----------------:|:-------------:|:---------------:|
-| No log        | 1.0   | 2    | 10.9413         | 0.0              | 0.0           | 0.0             |
+| Training Loss | Epoch | Step  | Validation Loss | Rouge2 Precision | Rouge2 Recall | Rouge2 Fmeasure |
+|:-------------:|:-----:|:-----:|:---------------:|:----------------:|:-------------:|:---------------:|
+| 2.8387        | 1.0   | 4040  | 2.6668          | 0.1434           | 0.1495        | 0.1445          |
+| 2.0882        | 2.0   | 8080  | 2.3950          | 0.174            | 0.1756        | 0.1725          |
+| 1.8985        | 3.0   | 12120 | 2.3368          | 0.1701           | 0.181         | 0.1731          |
 
 
 ### Framework versions
 
-- Transformers 4.35.2
-- Pytorch 2.1.0+cu121
+- Transformers 4.37.0
+- Pytorch 2.1.2
 - Datasets 2.16.1
-- Tokenizers 0.15.0
+- Tokenizers 0.15.1

generation_config.json

@@ -9,5 +9,5 @@
   "no_repeat_ngram_size": 3,
   "num_beams": 4,
   "pad_token_id": 0,
-  "transformers_version": "4.35.2"
+  "transformers_version": "4.37.0"
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:193256f2dc169e4587d1cc4bca7b8c697739e6870123af1450a6fde48eb1e1a9
+oid sha256:f68c70c946087269f5efc4e00255840f8cf74b3df575adc696f29ca98f9dba93
 size 998132132