|
2021-05-31 17:22:31,278 INFO __main__ Namespace(adjust_lr=False, config='torchdistill/configs/sample/glue/stsb/kd/bert_base_uncased_from_bert_large_uncased.yaml', log='log/glue/stsb/kd/bert_base_uncased_from_bert_large_uncased.txt', private_output='leaderboard/glue/kd/bert_base_uncased_from_bert_large_uncased/', seed=None, student_only=False, task_name='stsb', test_only=False, world_size=1) |
|
2021-05-31 17:22:31,313 INFO __main__ Distributed environment: NO |
|
Num processes: 1 |
|
Process index: 0 |
|
Local process index: 0 |
|
Device: cuda |
|
Use FP16 precision: True |
|
|
|
2021-05-31 17:22:43,561 WARNING datasets.builder Reusing dataset glue (/root/.cache/huggingface/datasets/glue/stsb/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad) |
|
2021-05-31 17:22:44,918 INFO __main__ Start training |
|
2021-05-31 17:22:44,918 INFO torchdistill.models.util [teacher model] |
|
2021-05-31 17:22:44,918 INFO torchdistill.models.util Using the original teacher model |
|
2021-05-31 17:22:44,919 INFO torchdistill.models.util [student model] |
|
2021-05-31 17:22:44,919 INFO torchdistill.models.util Using the original student model |
|
2021-05-31 17:22:44,919 INFO torchdistill.core.distillation Loss = 1.0 * OrgLoss + 1.0 * MSELoss() |
|
2021-05-31 17:22:44,919 INFO torchdistill.core.distillation Freezing the whole teacher model |
|
2021-05-31 17:22:48,242 INFO torchdistill.misc.log Epoch: [0] [ 0/180] eta: 0:01:16 lr: 2.9944444444444443e-05 sample/s: 9.576304360110528 loss: 120.2613 (120.2613) time: 0.4228 data: 0.0051 max mem: 2371 |
|
2021-05-31 17:23:12,289 INFO torchdistill.misc.log Epoch: [0] [ 50/180] eta: 0:01:02 lr: 2.716666666666667e-05 sample/s: 8.421417843919912 loss: 25.1578 (75.2308) time: 0.4905 data: 0.0035 max mem: 5106 |
|
2021-05-31 17:23:36,182 INFO torchdistill.misc.log Epoch: [0] [100/180] eta: 0:00:38 lr: 2.438888888888889e-05 sample/s: 8.415571491057348 loss: 16.4009 (46.5507) time: 0.4644 data: 0.0034 max mem: 5106 |
|
2021-05-31 17:23:59,828 INFO torchdistill.misc.log Epoch: [0] [150/180] eta: 0:00:14 lr: 2.161111111111111e-05 sample/s: 11.967236571894867 loss: 13.4210 (35.5925) time: 0.4481 data: 0.0035 max mem: 5106 |
|
2021-05-31 17:24:13,337 INFO torchdistill.misc.log Epoch: [0] Total time: 0:01:25 |
|
2021-05-31 17:24:16,240 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow |
|
2021-05-31 17:24:16,240 INFO __main__ Validation: pearson = 0.8895823514826617, spearmanr = 0.8891221171282753 |
|
2021-05-31 17:24:16,241 INFO __main__ Updating ckpt at ./resource/ckpt/glue/stsb/kd/stsb-bert-base-uncased_from_bert-large-uncased |
|
2021-05-31 17:24:17,947 INFO torchdistill.misc.log Epoch: [1] [ 0/180] eta: 0:01:17 lr: 1.9944444444444447e-05 sample/s: 9.436731626677647 loss: 3.8620 (3.8620) time: 0.4306 data: 0.0067 max mem: 5106 |
|
2021-05-31 17:24:42,457 INFO torchdistill.misc.log Epoch: [1] [ 50/180] eta: 0:01:03 lr: 1.7166666666666666e-05 sample/s: 7.082230403175797 loss: 4.8790 (5.6907) time: 0.5025 data: 0.0035 max mem: 5106 |
|
2021-05-31 17:25:07,011 INFO torchdistill.misc.log Epoch: [1] [100/180] eta: 0:00:39 lr: 1.438888888888889e-05 sample/s: 8.415195811165377 loss: 4.9423 (5.5366) time: 0.4891 data: 0.0040 max mem: 5106 |
|
2021-05-31 17:25:30,260 INFO torchdistill.misc.log Epoch: [1] [150/180] eta: 0:00:14 lr: 1.161111111111111e-05 sample/s: 11.947873559412079 loss: 5.0895 (5.4334) time: 0.4671 data: 0.0035 max mem: 5106 |
|
2021-05-31 17:25:43,919 INFO torchdistill.misc.log Epoch: [1] Total time: 0:01:26 |
|
2021-05-31 17:25:46,822 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow |
|
2021-05-31 17:25:46,822 INFO __main__ Validation: pearson = 0.8947956931797695, spearmanr = 0.8914128136254597 |
|
2021-05-31 17:25:46,822 INFO __main__ Updating ckpt at ./resource/ckpt/glue/stsb/kd/stsb-bert-base-uncased_from_bert-large-uncased |
|
2021-05-31 17:25:48,430 INFO torchdistill.misc.log Epoch: [2] [ 0/180] eta: 0:01:42 lr: 9.944444444444445e-06 sample/s: 7.080783708625091 loss: 2.3258 (2.3258) time: 0.5692 data: 0.0043 max mem: 5109 |
|
2021-05-31 17:26:12,502 INFO torchdistill.misc.log Epoch: [2] [ 50/180] eta: 0:01:02 lr: 7.166666666666667e-06 sample/s: 8.417454616972364 loss: 1.8558 (2.0434) time: 0.4910 data: 0.0036 max mem: 5109 |
|
2021-05-31 17:26:36,697 INFO torchdistill.misc.log Epoch: [2] [100/180] eta: 0:00:38 lr: 4.388888888888889e-06 sample/s: 7.668799172107633 loss: 2.1584 (2.0183) time: 0.4890 data: 0.0035 max mem: 5109 |
|
2021-05-31 17:27:00,193 INFO torchdistill.misc.log Epoch: [2] [150/180] eta: 0:00:14 lr: 1.6111111111111111e-06 sample/s: 8.416690287143053 loss: 1.3802 (1.9481) time: 0.4709 data: 0.0038 max mem: 5109 |
|
2021-05-31 17:27:14,506 INFO torchdistill.misc.log Epoch: [2] Total time: 0:01:26 |
|
2021-05-31 17:27:17,408 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow |
|
2021-05-31 17:27:17,409 INFO __main__ Validation: pearson = 0.8971237948141647, spearmanr = 0.8935765996632005 |
|
2021-05-31 17:27:17,409 INFO __main__ Updating ckpt at ./resource/ckpt/glue/stsb/kd/stsb-bert-base-uncased_from_bert-large-uncased |
|
2021-05-31 17:27:18,654 INFO __main__ [Teacher: bert-large-uncased] |
|
2021-05-31 17:27:26,678 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow |
|
2021-05-31 17:27:26,678 INFO __main__ Test: pearson = 0.9034287890218271, spearmanr = 0.9010997201222941 |
|
2021-05-31 17:27:29,798 INFO __main__ [Student: bert-base-uncased] |
|
2021-05-31 17:27:32,709 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow |
|
2021-05-31 17:27:32,709 INFO __main__ Test: pearson = 0.8971237948141647, spearmanr = 0.8935765996632005 |
|
2021-05-31 17:27:32,709 INFO __main__ Start prediction for private dataset(s) |
|
2021-05-31 17:27:32,710 INFO __main__ stsb/test: 1379 samples |
|
|
