temp_v.1.1.1

README.md

@@ -0,0 +1,67 @@
+---
+license: apache-2.0
+base_model: google/bert_uncased_L-2_H-128_A-2
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+model-index:
+- name: tiny-bert-sst2-distilled
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# tiny-bert-sst2-distilled
+
+This model is a fine-tuned version of [google/bert_uncased_L-2_H-128_A-2](https://huggingface.co/google/bert_uncased_L-2_H-128_A-2) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.1018
+- Accuracy: 0.8211
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 6e-05
+- train_batch_size: 128
+- eval_batch_size: 128
+- seed: 33
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 7
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|
+| 1.4646        | 1.0   | 527  | 1.1825          | 0.7867   |
+| 0.8559        | 2.0   | 1054 | 1.0389          | 0.8085   |
+| 0.6569        | 3.0   | 1581 | 1.0545          | 0.8222   |
+| 0.5672        | 4.0   | 2108 | 1.0577          | 0.8188   |
+| 0.5094        | 5.0   | 2635 | 1.0876          | 0.8211   |
+| 0.4717        | 6.0   | 3162 | 1.0979          | 0.8200   |
+| 0.4513        | 7.0   | 3689 | 1.1018          | 0.8211   |
+
+
+### Framework versions
+
+- Transformers 4.42.4
+- Pytorch 2.3.1+cu121
+- Datasets 2.20.0
+- Tokenizers 0.19.1

logs/events.out.tfevents.1723122909.621b935b3e3f.593.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d5bd78cdbabc22a36b3529a274c0a059eefa8fe65874503644b52cb8e5cdeef5
-size 8105
+oid sha256:96b6b29f73b48cc672118f99bf8120867a0b249191d0f5ba2a0c0117e9f7cfd6
+size 8993

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:49968a32f92059e091b0e66090f12dd7023f02859380ab6f5b25e0e1e5d0c037
+oid sha256:70d2270774ad4f7a1e718199e49076db2a402e3549e0cea413c3cc9b366d516e
 size 17549312