Training in progress, step 1000

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e641ae91477b3b77a83dac557558fdb1e535b9f42f6d31cc4b0105b6b384d588
+oid sha256:f2d18eede7a3ae0b6020edc089f2eed535af1939d98effa33b6222d14d111346
 size 1262083569

wandb/run-20220202_014046-gq6zg0t4/files/output.log

@@ -547,3 +547,551 @@ Configuration saved in ./checkpoint-500/config.json
 {'eval_loss': 0.3643854558467865, 'eval_wer': 0.45679051483641897, 'eval_runtime': 133.0426, 'eval_samples_per_second': 19.64, 'eval_steps_per_second': 0.278, 'epoch': 12.5}
 Model weights saved in ./checkpoint-500/pytorch_model.bin
 Configuration saved in ./checkpoint-500/preprocessor_config.json
+Configuration saved in ./preprocessor_config.json
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 15%|████████████████████████▉                                                                                                                                             | 600/4000 [1:27:12<6:07:08,  6.48s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 17%|█████████████████████████████                                                                                                                                         | 699/4000 [1:41:16<8:02:44,  8.77s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 20%|█████████████████████████████████▏                                                                                                                                    | 800/4000 [1:55:21<5:47:10,  6.51s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 22%|█████████████████████████████████████▎                                                                                                                                | 900/4000 [2:09:28<8:31:30,  9.90s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 25%|█████████████████████████████████████████▍                                                                                                                            | 999/4000 [2:23:11<6:39:09,  7.98s/it]
+ 25%|█████████████████████████████████████████▎                                                                                                                           | 1000/4000 [2:23:15<5:31:38,  6.63s/it]The following columns in the evaluation set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
+***** Running Evaluation *****
+  Num examples = 2613
+  Batch size = 72
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Configuration saved in ./checkpoint-1000/config.json
+{'eval_loss': 0.4216161072254181, 'eval_wer': 0.4414151661171697, 'eval_runtime': 131.8019, 'eval_samples_per_second': 19.825, 'eval_steps_per_second': 0.281, 'epoch': 25.0}
+Model weights saved in ./checkpoint-1000/pytorch_model.bin
+Configuration saved in ./checkpoint-1000/preprocessor_config.json