hf-test commited on
Commit
8339bc3
1 Parent(s): 28878e3

Training in progress, step 2000

Browse files
.gitattributes CHANGED
@@ -25,3 +25,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
25
  *.zip filter=lfs diff=lfs merge=lfs -text
26
  *.zstandard filter=lfs diff=lfs merge=lfs -text
27
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
25
  *.zip filter=lfs diff=lfs merge=lfs -text
26
  *.zstandard filter=lfs diff=lfs merge=lfs -text
27
  *tfevents* filter=lfs diff=lfs merge=lfs -text
28
+ wandb/run-20220127_121247-o8za41vn/run-o8za41vn.wandb filter=lfs diff=lfs merge=lfs -text
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4d80289eb039576f75e350675c60903ba8fcc6cc2d64287efc8fe26ccf4abf09
3
  size 1262075377
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa0be572201ab8a60ce90f69e54b99fea4e6ff0901184575f6db6c67356e2eb8
3
  size 1262075377
runs/Jan27_12-08-15_brutasse/events.out.tfevents.1643285566.brutasse.15970.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a2ea9ccd8501b33656ca7e02d6de78e045a6244eede76ffeff590957e662a184
3
- size 7987
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:205456cee4092e6691509aa608a1f11934e606eb28c6271d8ffb45034208fafa
3
+ size 9090
wandb/run-20220127_121247-o8za41vn/files/output.log CHANGED
@@ -1143,3 +1143,390 @@ Configuration saved in ./checkpoint-1500/config.json
1143
  Model weights saved in ./checkpoint-1500/pytorch_model.bin
1144
  Configuration saved in ./checkpoint-1500/preprocessor_config.json
1145
  {'eval_loss': 1.147773265838623, 'eval_wer': 0.8787680953732614, 'eval_runtime': 146.3922, 'eval_samples_per_second': 33.082, 'eval_steps_per_second': 4.14, 'epoch': 4.11}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1143
  Model weights saved in ./checkpoint-1500/pytorch_model.bin
1144
  Configuration saved in ./checkpoint-1500/preprocessor_config.json
1145
  {'eval_loss': 1.147773265838623, 'eval_wer': 0.8787680953732614, 'eval_runtime': 146.3922, 'eval_samples_per_second': 33.082, 'eval_steps_per_second': 4.14, 'epoch': 4.11}
1146
+ Configuration saved in ./preprocessor_config.json
1147
+
1148
+
1149
+
1150
+
1151
+
1152
+
1153
+
1154
+
1155
+
1156
+
1157
+
1158
+
1159
+
1160
+
1161
+
1162
+
1163
+
1164
+
1165
+
1166
+
1167
+
1168
+
1169
+
1170
+
1171
+
1172
+
1173
+
1174
+
1175
+
1176
+
1177
+
1178
+
1179
+
1180
+
1181
+
1182
+
1183
+
1184
+
1185
+
1186
+
1187
+
1188
+
1189
+
1190
+
1191
+
1192
+
1193
+
1194
+
1195
+
1196
+
1197
+
1198
+
1199
+
1200
+
1201
+
1202
+
1203
+
1204
+
1205
+ 9%|████████████▊ | 1599/18250 [41:56<4:42:20, 1.02s/it]
1206
+
1207
+
1208
+
1209
+
1210
+
1211
+
1212
+
1213
+
1214
+
1215
+
1216
+
1217
+
1218
+
1219
+
1220
+
1221
+
1222
+
1223
+
1224
+
1225
+
1226
+
1227
+
1228
+
1229
+
1230
+
1231
+
1232
+
1233
+
1234
+
1235
+
1236
+
1237
+
1238
+
1239
+
1240
+
1241
+
1242
+
1243
+
1244
+
1245
+
1246
+
1247
+
1248
+
1249
+
1250
+
1251
+
1252
+
1253
+
1254
+
1255
+
1256
+
1257
+
1258
+
1259
+
1260
+
1261
+
1262
+
1263
+
1264
+
1265
+
1266
+ 9%|█████████████▌ | 1699/18250 [43:57<4:41:02, 1.02s/it]
1267
+
1268
+
1269
+
1270
+
1271
+
1272
+
1273
+
1274
+
1275
+
1276
+
1277
+
1278
+
1279
+
1280
+
1281
+
1282
+
1283
+
1284
+
1285
+
1286
+
1287
+
1288
+
1289
+
1290
+
1291
+
1292
+
1293
+
1294
+
1295
+
1296
+
1297
+
1298
+
1299
+
1300
+
1301
+
1302
+
1303
+
1304
+
1305
+
1306
+
1307
+
1308
+
1309
+
1310
+
1311
+
1312
+
1313
+
1314
+
1315
+
1316
+
1317
+
1318
+
1319
+
1320
+
1321
+
1322
+
1323
+
1324
+
1325
+
1326
+
1327
+ 10%|██████████████▍ | 1800/18250 [45:58<4:33:04, 1.00it/s]
1328
+
1329
+
1330
+
1331
+
1332
+
1333
+
1334
+
1335
+
1336
+
1337
+
1338
+
1339
+
1340
+
1341
+
1342
+
1343
+
1344
+
1345
+
1346
+
1347
+
1348
+
1349
+
1350
+
1351
+
1352
+
1353
+
1354
+
1355
+
1356
+
1357
+
1358
+
1359
+
1360
+
1361
+
1362
+
1363
+
1364
+
1365
+
1366
+
1367
+
1368
+
1369
+
1370
+
1371
+
1372
+
1373
+
1374
+
1375
+
1376
+
1377
+
1378
+
1379
+
1380
+
1381
+
1382
+
1383
+
1384
+
1385
+
1386
+
1387
+
1388
+
1389
+
1390
+ 10%|███████████████▏ | 1901/18250 [48:03<5:39:28, 1.25s/it]
1391
+
1392
+
1393
+
1394
+
1395
+
1396
+
1397
+
1398
+
1399
+
1400
+
1401
+
1402
+
1403
+
1404
+
1405
+
1406
+
1407
+
1408
+
1409
+
1410
+
1411
+
1412
+
1413
+
1414
+
1415
+
1416
+
1417
+
1418
+
1419
+
1420
+
1421
+
1422
+
1423
+
1424
+
1425
+
1426
+
1427
+
1428
+
1429
+
1430
+
1431
+
1432
+
1433
+
1434
+
1435
+
1436
+
1437
+
1438
+
1439
+
1440
+
1441
+
1442
+
1443
+
1444
+
1445
+
1446
+
1447
+
1448
+
1449
+
1450
+ 11%|███████████████▉ | 1999/18250 [50:01<5:43:52, 1.27s/it]
1451
+ 11%|████████████████ | 2000/18250 [50:03<5:36:00, 1.24s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
1452
+ ***** Running Evaluation *****
1453
+ Num examples = 4843
1454
+ Batch size = 8
1455
+
1456
+
1457
+
1458
+
1459
+
1460
+
1461
+
1462
+
1463
+
1464
+
1465
+
1466
+
1467
+
1468
+
1469
+
1470
+
1471
+
1472
+
1473
+
1474
+
1475
+
1476
+
1477
+
1478
+
1479
+
1480
+
1481
+
1482
+
1483
+
1484
+
1485
+
1486
+
1487
+
1488
+
1489
+
1490
+
1491
+
1492
+
1493
+
1494
+
1495
+
1496
+
1497
+
1498
+
1499
+
1500
+
1501
+
1502
+
1503
+
1504
+
1505
+
1506
+
1507
+
1508
+
1509
+
1510
+
1511
+
1512
+
1513
+
1514
+
1515
+
1516
+
1517
+
1518
+
1519
+
1520
+
1521
+
1522
+
1523
+
1524
+
1525
+
1526
+
1527
+ Configuration saved in ./checkpoint-2000/config.json
1528
+ {'eval_loss': 0.6748705506324768, 'eval_wer': 0.6546693159239285, 'eval_runtime': 146.3732, 'eval_samples_per_second': 33.087, 'eval_steps_per_second': 4.14, 'epoch': 5.48}
1529
+ Model weights saved in ./checkpoint-2000/pytorch_model.bin
1530
+ Configuration saved in ./checkpoint-2000/preprocessor_config.json
1531
+ Configuration saved in ./preprocessor_config.json
1532
+ 01/27/2022 13:06:22 - WARNING - huggingface_hub.repository - Adding files tracked by Git LFS: ['wandb/run-20220127_121247-o8za41vn/run-o8za41vn.wandb']. This may take a bit of time if the files are large.
wandb/run-20220127_121247-o8za41vn/files/wandb-summary.json CHANGED
The diff for this file is too large to render. See raw diff
 
wandb/run-20220127_121247-o8za41vn/logs/debug-internal.log CHANGED
@@ -2826,3 +2826,541 @@
2826
  2022-01-27 12:52:38,173 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2827
  2022-01-27 12:52:44,038 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2828
  2022-01-27 12:52:44,038 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2826
  2022-01-27 12:52:38,173 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2827
  2022-01-27 12:52:44,038 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2828
  2022-01-27 12:52:44,038 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2829
+ 2022-01-27 12:52:53,562 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2830
+ 2022-01-27 12:52:55,563 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2831
+ 2022-01-27 12:52:57,563 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2832
+ 2022-01-27 12:52:59,170 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2833
+ 2022-01-27 12:52:59,170 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2834
+ 2022-01-27 12:52:59,563 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2835
+ 2022-01-27 12:53:01,564 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2836
+ 2022-01-27 12:53:03,564 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2837
+ 2022-01-27 12:53:05,565 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2838
+ 2022-01-27 12:53:07,565 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2839
+ 2022-01-27 12:53:08,450 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2840
+ 2022-01-27 12:53:09,566 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2841
+ 2022-01-27 12:53:11,566 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2842
+ 2022-01-27 12:53:14,344 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2843
+ 2022-01-27 12:53:14,345 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2844
+ 2022-01-27 12:53:14,567 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2845
+ 2022-01-27 12:53:16,567 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2846
+ 2022-01-27 12:53:18,568 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2847
+ 2022-01-27 12:53:20,568 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2848
+ 2022-01-27 12:53:22,569 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2849
+ 2022-01-27 12:53:24,569 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2850
+ 2022-01-27 12:53:26,570 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2851
+ 2022-01-27 12:53:28,570 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2852
+ 2022-01-27 12:53:29,430 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2853
+ 2022-01-27 12:53:29,430 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2854
+ 2022-01-27 12:53:30,571 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2855
+ 2022-01-27 12:53:32,571 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2856
+ 2022-01-27 12:53:34,572 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2857
+ 2022-01-27 12:53:36,572 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2858
+ 2022-01-27 12:53:38,572 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2859
+ 2022-01-27 12:53:38,704 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2860
+ 2022-01-27 12:53:40,573 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2861
+ 2022-01-27 12:53:42,573 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2862
+ 2022-01-27 12:53:44,522 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2863
+ 2022-01-27 12:53:44,522 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2864
+ 2022-01-27 12:53:44,574 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2865
+ 2022-01-27 12:53:46,574 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2866
+ 2022-01-27 12:53:48,574 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2867
+ 2022-01-27 12:53:50,575 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2868
+ 2022-01-27 12:53:52,575 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2869
+ 2022-01-27 12:53:54,575 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2870
+ 2022-01-27 12:53:56,576 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2871
+ 2022-01-27 12:53:58,576 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2872
+ 2022-01-27 12:53:59,593 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2873
+ 2022-01-27 12:53:59,593 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2874
+ 2022-01-27 12:54:00,576 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2875
+ 2022-01-27 12:54:02,577 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2876
+ 2022-01-27 12:54:04,577 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2877
+ 2022-01-27 12:54:06,577 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2878
+ 2022-01-27 12:54:08,578 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2879
+ 2022-01-27 12:54:08,945 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2880
+ 2022-01-27 12:54:10,578 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2881
+ 2022-01-27 12:54:12,578 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2882
+ 2022-01-27 12:54:14,578 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2883
+ 2022-01-27 12:54:14,687 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2884
+ 2022-01-27 12:54:14,687 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2885
+ 2022-01-27 12:54:16,579 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2886
+ 2022-01-27 12:54:18,579 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2887
+ 2022-01-27 12:54:20,579 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2888
+ 2022-01-27 12:54:22,580 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2889
+ 2022-01-27 12:54:24,580 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2890
+ 2022-01-27 12:54:26,580 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2891
+ 2022-01-27 12:54:28,581 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2892
+ 2022-01-27 12:54:29,794 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2893
+ 2022-01-27 12:54:29,794 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2894
+ 2022-01-27 12:54:30,581 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2895
+ 2022-01-27 12:54:32,581 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2896
+ 2022-01-27 12:54:34,582 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2897
+ 2022-01-27 12:54:36,582 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2898
+ 2022-01-27 12:54:38,582 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2899
+ 2022-01-27 12:54:39,194 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2900
+ 2022-01-27 12:54:40,583 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2901
+ 2022-01-27 12:54:42,583 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2902
+ 2022-01-27 12:54:44,583 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2903
+ 2022-01-27 12:54:44,885 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2904
+ 2022-01-27 12:54:44,886 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2905
+ 2022-01-27 12:54:46,584 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2906
+ 2022-01-27 12:54:48,584 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2907
+ 2022-01-27 12:54:50,580 DEBUG SenderThread:16113 [sender.py:send():182] send: history
2908
+ 2022-01-27 12:54:50,628 DEBUG SenderThread:16113 [sender.py:send():182] send: summary
2909
+ 2022-01-27 12:54:50,654 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2910
+ 2022-01-27 12:54:50,655 INFO SenderThread:16113 [sender.py:_save_file():873] saving file wandb-summary.json with policy end
2911
+ 2022-01-27 12:54:51,655 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/wandb-summary.json
2912
+ 2022-01-27 12:54:52,655 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2913
+ 2022-01-27 12:54:54,655 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2914
+ 2022-01-27 12:54:56,656 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2915
+ 2022-01-27 12:54:58,656 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2916
+ 2022-01-27 12:54:59,959 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2917
+ 2022-01-27 12:54:59,960 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2918
+ 2022-01-27 12:55:00,656 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2919
+ 2022-01-27 12:55:02,657 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2920
+ 2022-01-27 12:55:04,657 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2921
+ 2022-01-27 12:55:06,657 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2922
+ 2022-01-27 12:55:08,658 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2923
+ 2022-01-27 12:55:09,445 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2924
+ 2022-01-27 12:55:10,658 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2925
+ 2022-01-27 12:55:12,659 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2926
+ 2022-01-27 12:55:14,659 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2927
+ 2022-01-27 12:55:15,099 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2928
+ 2022-01-27 12:55:15,100 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2929
+ 2022-01-27 12:55:16,659 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2930
+ 2022-01-27 12:55:18,660 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2931
+ 2022-01-27 12:55:20,660 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2932
+ 2022-01-27 12:55:22,661 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2933
+ 2022-01-27 12:55:24,661 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2934
+ 2022-01-27 12:55:26,661 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2935
+ 2022-01-27 12:55:28,662 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2936
+ 2022-01-27 12:55:30,193 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2937
+ 2022-01-27 12:55:30,193 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2938
+ 2022-01-27 12:55:30,662 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2939
+ 2022-01-27 12:55:32,662 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2940
+ 2022-01-27 12:55:35,663 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2941
+ 2022-01-27 12:55:37,663 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2942
+ 2022-01-27 12:55:39,663 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2943
+ 2022-01-27 12:55:39,690 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2944
+ 2022-01-27 12:55:41,664 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2945
+ 2022-01-27 12:55:43,664 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2946
+ 2022-01-27 12:55:45,310 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2947
+ 2022-01-27 12:55:45,310 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2948
+ 2022-01-27 12:55:45,664 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2949
+ 2022-01-27 12:55:47,665 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2950
+ 2022-01-27 12:55:49,665 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2951
+ 2022-01-27 12:55:51,665 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2952
+ 2022-01-27 12:55:53,666 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2953
+ 2022-01-27 12:55:55,666 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2954
+ 2022-01-27 12:55:57,666 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2955
+ 2022-01-27 12:55:59,667 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2956
+ 2022-01-27 12:56:00,394 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2957
+ 2022-01-27 12:56:00,394 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2958
+ 2022-01-27 12:56:01,667 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2959
+ 2022-01-27 12:56:03,668 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2960
+ 2022-01-27 12:56:05,668 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2961
+ 2022-01-27 12:56:07,668 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2962
+ 2022-01-27 12:56:09,669 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2963
+ 2022-01-27 12:56:09,964 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2964
+ 2022-01-27 12:56:11,669 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2965
+ 2022-01-27 12:56:13,669 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2966
+ 2022-01-27 12:56:15,516 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2967
+ 2022-01-27 12:56:15,517 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2968
+ 2022-01-27 12:56:15,670 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2969
+ 2022-01-27 12:56:17,670 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2970
+ 2022-01-27 12:56:19,670 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2971
+ 2022-01-27 12:56:21,671 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2972
+ 2022-01-27 12:56:23,671 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2973
+ 2022-01-27 12:56:25,672 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2974
+ 2022-01-27 12:56:27,672 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2975
+ 2022-01-27 12:56:29,672 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2976
+ 2022-01-27 12:56:30,594 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2977
+ 2022-01-27 12:56:30,594 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2978
+ 2022-01-27 12:56:31,673 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2979
+ 2022-01-27 12:56:33,673 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2980
+ 2022-01-27 12:56:35,673 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2981
+ 2022-01-27 12:56:37,674 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2982
+ 2022-01-27 12:56:39,674 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2983
+ 2022-01-27 12:56:40,233 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
2984
+ 2022-01-27 12:56:41,674 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2985
+ 2022-01-27 12:56:43,674 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2986
+ 2022-01-27 12:56:45,675 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2987
+ 2022-01-27 12:56:45,706 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
2988
+ 2022-01-27 12:56:45,706 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
2989
+ 2022-01-27 12:56:47,675 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2990
+ 2022-01-27 12:56:49,675 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2991
+ 2022-01-27 12:56:51,188 DEBUG SenderThread:16113 [sender.py:send():182] send: history
2992
+ 2022-01-27 12:56:51,242 DEBUG SenderThread:16113 [sender.py:send():182] send: summary
2993
+ 2022-01-27 12:56:51,264 INFO SenderThread:16113 [sender.py:_save_file():873] saving file wandb-summary.json with policy end
2994
+ 2022-01-27 12:56:51,676 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/wandb-summary.json
2995
+ 2022-01-27 12:56:51,676 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2996
+ 2022-01-27 12:56:53,676 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2997
+ 2022-01-27 12:56:55,676 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2998
+ 2022-01-27 12:56:57,677 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
2999
+ 2022-01-27 12:56:59,677 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3000
+ 2022-01-27 12:57:00,838 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3001
+ 2022-01-27 12:57:00,839 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3002
+ 2022-01-27 12:57:01,678 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3003
+ 2022-01-27 12:57:03,678 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3004
+ 2022-01-27 12:57:05,678 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3005
+ 2022-01-27 12:57:07,678 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3006
+ 2022-01-27 12:57:09,679 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3007
+ 2022-01-27 12:57:10,502 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3008
+ 2022-01-27 12:57:11,679 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3009
+ 2022-01-27 12:57:13,679 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3010
+ 2022-01-27 12:57:15,680 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3011
+ 2022-01-27 12:57:15,912 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3012
+ 2022-01-27 12:57:15,912 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3013
+ 2022-01-27 12:57:17,680 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3014
+ 2022-01-27 12:57:19,680 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3015
+ 2022-01-27 12:57:21,681 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3016
+ 2022-01-27 12:57:23,681 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3017
+ 2022-01-27 12:57:25,681 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3018
+ 2022-01-27 12:57:27,682 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3019
+ 2022-01-27 12:57:29,682 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3020
+ 2022-01-27 12:57:30,982 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3021
+ 2022-01-27 12:57:30,982 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3022
+ 2022-01-27 12:57:31,682 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3023
+ 2022-01-27 12:57:33,682 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3024
+ 2022-01-27 12:57:35,683 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3025
+ 2022-01-27 12:57:37,683 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3026
+ 2022-01-27 12:57:39,683 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3027
+ 2022-01-27 12:57:40,765 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3028
+ 2022-01-27 12:57:41,684 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3029
+ 2022-01-27 12:57:43,684 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3030
+ 2022-01-27 12:57:45,684 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3031
+ 2022-01-27 12:57:46,108 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3032
+ 2022-01-27 12:57:46,108 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3033
+ 2022-01-27 12:57:47,685 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3034
+ 2022-01-27 12:57:50,685 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3035
+ 2022-01-27 12:57:52,685 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3036
+ 2022-01-27 12:57:54,686 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3037
+ 2022-01-27 12:57:56,686 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3038
+ 2022-01-27 12:57:58,686 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3039
+ 2022-01-27 12:58:00,687 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3040
+ 2022-01-27 12:58:01,177 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3041
+ 2022-01-27 12:58:01,178 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3042
+ 2022-01-27 12:58:02,687 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3043
+ 2022-01-27 12:58:04,687 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3044
+ 2022-01-27 12:58:06,688 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3045
+ 2022-01-27 12:58:08,688 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3046
+ 2022-01-27 12:58:10,688 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3047
+ 2022-01-27 12:58:11,022 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3048
+ 2022-01-27 12:58:12,689 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3049
+ 2022-01-27 12:58:14,689 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3050
+ 2022-01-27 12:58:16,260 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3051
+ 2022-01-27 12:58:16,261 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3052
+ 2022-01-27 12:58:16,689 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3053
+ 2022-01-27 12:58:18,690 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3054
+ 2022-01-27 12:58:20,690 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3055
+ 2022-01-27 12:58:22,690 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3056
+ 2022-01-27 12:58:24,691 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3057
+ 2022-01-27 12:58:26,691 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3058
+ 2022-01-27 12:58:28,691 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3059
+ 2022-01-27 12:58:30,692 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3060
+ 2022-01-27 12:58:31,348 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3061
+ 2022-01-27 12:58:31,348 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3062
+ 2022-01-27 12:58:32,692 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3063
+ 2022-01-27 12:58:34,692 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3064
+ 2022-01-27 12:58:36,693 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3065
+ 2022-01-27 12:58:38,693 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3066
+ 2022-01-27 12:58:40,693 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3067
+ 2022-01-27 12:58:41,279 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3068
+ 2022-01-27 12:58:42,694 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3069
+ 2022-01-27 12:58:44,694 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3070
+ 2022-01-27 12:58:46,433 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3071
+ 2022-01-27 12:58:46,433 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3072
+ 2022-01-27 12:58:46,694 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3073
+ 2022-01-27 12:58:48,695 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3074
+ 2022-01-27 12:58:50,695 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3075
+ 2022-01-27 12:58:51,337 DEBUG SenderThread:16113 [sender.py:send():182] send: history
3076
+ 2022-01-27 12:58:51,378 DEBUG SenderThread:16113 [sender.py:send():182] send: summary
3077
+ 2022-01-27 12:58:51,400 INFO SenderThread:16113 [sender.py:_save_file():873] saving file wandb-summary.json with policy end
3078
+ 2022-01-27 12:58:51,695 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/wandb-summary.json
3079
+ 2022-01-27 12:58:52,695 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3080
+ 2022-01-27 12:58:53,696 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3081
+ 2022-01-27 12:58:54,696 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3082
+ 2022-01-27 12:58:56,696 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3083
+ 2022-01-27 12:58:58,696 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3084
+ 2022-01-27 12:59:00,697 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3085
+ 2022-01-27 12:59:01,516 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3086
+ 2022-01-27 12:59:01,517 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3087
+ 2022-01-27 12:59:02,697 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3088
+ 2022-01-27 12:59:04,697 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3089
+ 2022-01-27 12:59:06,698 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3090
+ 2022-01-27 12:59:08,698 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3091
+ 2022-01-27 12:59:10,698 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3092
+ 2022-01-27 12:59:11,597 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3093
+ 2022-01-27 12:59:12,699 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3094
+ 2022-01-27 12:59:14,699 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3095
+ 2022-01-27 12:59:16,654 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3096
+ 2022-01-27 12:59:16,654 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3097
+ 2022-01-27 12:59:16,700 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3098
+ 2022-01-27 12:59:18,700 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3099
+ 2022-01-27 12:59:20,700 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3100
+ 2022-01-27 12:59:22,701 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3101
+ 2022-01-27 12:59:24,701 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3102
+ 2022-01-27 12:59:26,702 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3103
+ 2022-01-27 12:59:28,702 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3104
+ 2022-01-27 12:59:30,702 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3105
+ 2022-01-27 12:59:31,965 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3106
+ 2022-01-27 12:59:31,965 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3107
+ 2022-01-27 12:59:32,703 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3108
+ 2022-01-27 12:59:34,703 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3109
+ 2022-01-27 12:59:36,703 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3110
+ 2022-01-27 12:59:38,704 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3111
+ 2022-01-27 12:59:40,704 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3112
+ 2022-01-27 12:59:41,841 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3113
+ 2022-01-27 12:59:42,704 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3114
+ 2022-01-27 12:59:44,705 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3115
+ 2022-01-27 12:59:46,705 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3116
+ 2022-01-27 12:59:47,118 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3117
+ 2022-01-27 12:59:47,118 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3118
+ 2022-01-27 12:59:48,705 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3119
+ 2022-01-27 12:59:50,706 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3120
+ 2022-01-27 12:59:52,706 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3121
+ 2022-01-27 12:59:54,706 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3122
+ 2022-01-27 12:59:56,706 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3123
+ 2022-01-27 12:59:58,707 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3124
+ 2022-01-27 13:00:01,707 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3125
+ 2022-01-27 13:00:02,206 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3126
+ 2022-01-27 13:00:02,207 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3127
+ 2022-01-27 13:00:03,708 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3128
+ 2022-01-27 13:00:05,708 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3129
+ 2022-01-27 13:00:07,708 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3130
+ 2022-01-27 13:00:09,708 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3131
+ 2022-01-27 13:00:11,709 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3132
+ 2022-01-27 13:00:12,093 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3133
+ 2022-01-27 13:00:13,709 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3134
+ 2022-01-27 13:00:15,709 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3135
+ 2022-01-27 13:00:17,315 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3136
+ 2022-01-27 13:00:17,315 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3137
+ 2022-01-27 13:00:17,710 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3138
+ 2022-01-27 13:00:19,710 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3139
+ 2022-01-27 13:00:21,710 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3140
+ 2022-01-27 13:00:23,711 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3141
+ 2022-01-27 13:00:25,711 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3142
+ 2022-01-27 13:00:27,711 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3143
+ 2022-01-27 13:00:29,712 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3144
+ 2022-01-27 13:00:31,712 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3145
+ 2022-01-27 13:00:32,396 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3146
+ 2022-01-27 13:00:32,397 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3147
+ 2022-01-27 13:00:33,712 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3148
+ 2022-01-27 13:00:35,713 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3149
+ 2022-01-27 13:00:37,713 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3150
+ 2022-01-27 13:00:39,713 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3151
+ 2022-01-27 13:00:41,714 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3152
+ 2022-01-27 13:00:42,344 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3153
+ 2022-01-27 13:00:43,714 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3154
+ 2022-01-27 13:00:45,714 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3155
+ 2022-01-27 13:00:47,541 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3156
+ 2022-01-27 13:00:47,541 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3157
+ 2022-01-27 13:00:47,715 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3158
+ 2022-01-27 13:00:49,715 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3159
+ 2022-01-27 13:00:51,715 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3160
+ 2022-01-27 13:00:53,716 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3161
+ 2022-01-27 13:00:55,716 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3162
+ 2022-01-27 13:00:55,740 DEBUG SenderThread:16113 [sender.py:send():182] send: history
3163
+ 2022-01-27 13:00:55,782 DEBUG SenderThread:16113 [sender.py:send():182] send: summary
3164
+ 2022-01-27 13:00:55,808 INFO SenderThread:16113 [sender.py:_save_file():873] saving file wandb-summary.json with policy end
3165
+ 2022-01-27 13:00:56,716 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/wandb-summary.json
3166
+ 2022-01-27 13:00:57,716 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3167
+ 2022-01-27 13:00:59,717 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3168
+ 2022-01-27 13:01:01,717 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3169
+ 2022-01-27 13:01:02,613 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3170
+ 2022-01-27 13:01:02,613 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3171
+ 2022-01-27 13:01:03,717 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3172
+ 2022-01-27 13:01:05,718 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3173
+ 2022-01-27 13:01:07,718 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3174
+ 2022-01-27 13:01:09,718 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3175
+ 2022-01-27 13:01:11,719 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3176
+ 2022-01-27 13:01:12,610 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3177
+ 2022-01-27 13:01:13,719 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3178
+ 2022-01-27 13:01:15,719 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3179
+ 2022-01-27 13:01:17,719 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3180
+ 2022-01-27 13:01:17,737 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3181
+ 2022-01-27 13:01:17,737 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3182
+ 2022-01-27 13:01:19,720 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3183
+ 2022-01-27 13:01:21,720 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3184
+ 2022-01-27 13:01:23,720 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3185
+ 2022-01-27 13:01:25,721 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3186
+ 2022-01-27 13:01:27,721 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3187
+ 2022-01-27 13:01:29,721 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3188
+ 2022-01-27 13:01:31,722 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3189
+ 2022-01-27 13:01:32,861 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3190
+ 2022-01-27 13:01:32,861 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3191
+ 2022-01-27 13:01:33,722 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3192
+ 2022-01-27 13:01:35,722 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3193
+ 2022-01-27 13:01:37,723 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3194
+ 2022-01-27 13:01:39,723 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3195
+ 2022-01-27 13:01:41,723 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3196
+ 2022-01-27 13:01:42,863 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3197
+ 2022-01-27 13:01:43,724 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3198
+ 2022-01-27 13:01:45,724 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3199
+ 2022-01-27 13:01:47,724 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3200
+ 2022-01-27 13:01:47,944 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3201
+ 2022-01-27 13:01:47,944 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3202
+ 2022-01-27 13:01:49,725 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3203
+ 2022-01-27 13:01:51,725 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3204
+ 2022-01-27 13:01:53,725 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3205
+ 2022-01-27 13:01:55,725 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3206
+ 2022-01-27 13:01:57,726 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3207
+ 2022-01-27 13:01:59,726 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3208
+ 2022-01-27 13:02:01,726 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3209
+ 2022-01-27 13:02:03,018 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3210
+ 2022-01-27 13:02:03,019 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3211
+ 2022-01-27 13:02:03,727 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3212
+ 2022-01-27 13:02:06,727 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3213
+ 2022-01-27 13:02:08,727 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3214
+ 2022-01-27 13:02:10,728 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3215
+ 2022-01-27 13:02:12,728 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3216
+ 2022-01-27 13:02:13,131 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3217
+ 2022-01-27 13:02:14,729 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3218
+ 2022-01-27 13:02:16,729 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3219
+ 2022-01-27 13:02:18,091 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3220
+ 2022-01-27 13:02:18,091 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3221
+ 2022-01-27 13:02:18,729 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3222
+ 2022-01-27 13:02:20,730 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3223
+ 2022-01-27 13:02:22,730 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3224
+ 2022-01-27 13:02:24,730 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3225
+ 2022-01-27 13:02:26,731 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3226
+ 2022-01-27 13:02:28,731 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3227
+ 2022-01-27 13:02:30,731 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3228
+ 2022-01-27 13:02:32,731 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3229
+ 2022-01-27 13:02:33,237 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3230
+ 2022-01-27 13:02:33,237 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3231
+ 2022-01-27 13:02:34,732 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3232
+ 2022-01-27 13:02:36,732 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3233
+ 2022-01-27 13:02:38,732 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3234
+ 2022-01-27 13:02:40,733 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3235
+ 2022-01-27 13:02:42,733 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3236
+ 2022-01-27 13:02:43,390 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3237
+ 2022-01-27 13:02:44,733 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3238
+ 2022-01-27 13:02:46,734 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3239
+ 2022-01-27 13:02:48,314 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3240
+ 2022-01-27 13:02:48,314 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3241
+ 2022-01-27 13:02:48,734 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3242
+ 2022-01-27 13:02:50,734 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3243
+ 2022-01-27 13:02:52,735 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3244
+ 2022-01-27 13:02:54,735 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3245
+ 2022-01-27 13:02:56,297 DEBUG SenderThread:16113 [sender.py:send():182] send: history
3246
+ 2022-01-27 13:02:56,349 DEBUG SenderThread:16113 [sender.py:send():182] send: summary
3247
+ 2022-01-27 13:02:56,371 INFO SenderThread:16113 [sender.py:_save_file():873] saving file wandb-summary.json with policy end
3248
+ 2022-01-27 13:02:56,735 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/wandb-summary.json
3249
+ 2022-01-27 13:02:56,735 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3250
+ 2022-01-27 13:02:58,736 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3251
+ 2022-01-27 13:03:00,736 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3252
+ 2022-01-27 13:03:02,736 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3253
+ 2022-01-27 13:03:03,442 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3254
+ 2022-01-27 13:03:03,442 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3255
+ 2022-01-27 13:03:04,737 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3256
+ 2022-01-27 13:03:06,737 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3257
+ 2022-01-27 13:03:08,737 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3258
+ 2022-01-27 13:03:10,738 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3259
+ 2022-01-27 13:03:12,738 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3260
+ 2022-01-27 13:03:13,661 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3261
+ 2022-01-27 13:03:14,738 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3262
+ 2022-01-27 13:03:16,739 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3263
+ 2022-01-27 13:03:18,545 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3264
+ 2022-01-27 13:03:18,545 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3265
+ 2022-01-27 13:03:18,739 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3266
+ 2022-01-27 13:03:20,740 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3267
+ 2022-01-27 13:03:22,740 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3268
+ 2022-01-27 13:03:24,740 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3269
+ 2022-01-27 13:03:26,741 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3270
+ 2022-01-27 13:03:28,741 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3271
+ 2022-01-27 13:03:30,741 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3272
+ 2022-01-27 13:03:32,742 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3273
+ 2022-01-27 13:03:33,621 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3274
+ 2022-01-27 13:03:33,621 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3275
+ 2022-01-27 13:03:34,742 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3276
+ 2022-01-27 13:03:36,742 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3277
+ 2022-01-27 13:03:38,743 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3278
+ 2022-01-27 13:03:40,743 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3279
+ 2022-01-27 13:03:42,743 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3280
+ 2022-01-27 13:03:43,920 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3281
+ 2022-01-27 13:03:45,744 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3282
+ 2022-01-27 13:03:47,744 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3283
+ 2022-01-27 13:03:48,732 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3284
+ 2022-01-27 13:03:48,732 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3285
+ 2022-01-27 13:03:49,745 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3286
+ 2022-01-27 13:03:51,745 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3287
+ 2022-01-27 13:03:53,745 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3288
+ 2022-01-27 13:03:55,746 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3289
+ 2022-01-27 13:03:57,746 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3290
+ 2022-01-27 13:03:59,746 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3291
+ 2022-01-27 13:04:01,747 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3292
+ 2022-01-27 13:04:03,747 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3293
+ 2022-01-27 13:04:03,817 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3294
+ 2022-01-27 13:04:03,817 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3295
+ 2022-01-27 13:04:05,748 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3296
+ 2022-01-27 13:04:07,748 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3297
+ 2022-01-27 13:04:09,748 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3298
+ 2022-01-27 13:04:11,749 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3299
+ 2022-01-27 13:04:13,749 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3300
+ 2022-01-27 13:04:14,178 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3301
+ 2022-01-27 13:04:15,749 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3302
+ 2022-01-27 13:04:17,750 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3303
+ 2022-01-27 13:04:18,924 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3304
+ 2022-01-27 13:04:18,924 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3305
+ 2022-01-27 13:04:19,750 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3306
+ 2022-01-27 13:04:21,750 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3307
+ 2022-01-27 13:04:23,751 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3308
+ 2022-01-27 13:04:25,751 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3309
+ 2022-01-27 13:04:27,751 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3310
+ 2022-01-27 13:04:29,752 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3311
+ 2022-01-27 13:04:31,752 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3312
+ 2022-01-27 13:04:33,752 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3313
+ 2022-01-27 13:04:34,077 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3314
+ 2022-01-27 13:04:34,078 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3315
+ 2022-01-27 13:04:35,753 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3316
+ 2022-01-27 13:04:37,753 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3317
+ 2022-01-27 13:04:39,753 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3318
+ 2022-01-27 13:04:41,754 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3319
+ 2022-01-27 13:04:43,754 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3320
+ 2022-01-27 13:04:44,432 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3321
+ 2022-01-27 13:04:45,754 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3322
+ 2022-01-27 13:04:47,755 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3323
+ 2022-01-27 13:04:49,201 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3324
+ 2022-01-27 13:04:49,201 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3325
+ 2022-01-27 13:04:49,755 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3326
+ 2022-01-27 13:04:51,755 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3327
+ 2022-01-27 13:04:53,756 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3328
+ 2022-01-27 13:04:55,756 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3329
+ 2022-01-27 13:04:57,756 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3330
+ 2022-01-27 13:04:59,757 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3331
+ 2022-01-27 13:05:01,757 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3332
+ 2022-01-27 13:05:03,757 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3333
+ 2022-01-27 13:05:04,347 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3334
+ 2022-01-27 13:05:04,347 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3335
+ 2022-01-27 13:05:05,758 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3336
+ 2022-01-27 13:05:07,758 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3337
+ 2022-01-27 13:05:09,758 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3338
+ 2022-01-27 13:05:11,759 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3339
+ 2022-01-27 13:05:13,759 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3340
+ 2022-01-27 13:05:14,683 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3341
+ 2022-01-27 13:05:15,759 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3342
+ 2022-01-27 13:05:17,760 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3343
+ 2022-01-27 13:05:19,435 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3344
+ 2022-01-27 13:05:19,435 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3345
+ 2022-01-27 13:05:19,760 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3346
+ 2022-01-27 13:05:21,760 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3347
+ 2022-01-27 13:05:22,677 DEBUG SenderThread:16113 [sender.py:send():182] send: history
3348
+ 2022-01-27 13:05:22,677 DEBUG SenderThread:16113 [sender.py:send():182] send: summary
3349
+ 2022-01-27 13:05:22,705 INFO SenderThread:16113 [sender.py:_save_file():873] saving file wandb-summary.json with policy end
3350
+ 2022-01-27 13:05:22,761 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/wandb-summary.json
3351
+ 2022-01-27 13:05:23,761 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3352
+ 2022-01-27 13:05:24,762 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3353
+ 2022-01-27 13:05:25,762 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3354
+ 2022-01-27 13:05:34,580 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3355
+ 2022-01-27 13:05:34,580 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3356
+ 2022-01-27 13:05:44,983 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3357
+ 2022-01-27 13:05:49,655 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3358
+ 2022-01-27 13:05:49,655 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3359
+ 2022-01-27 13:06:00,775 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3360
+ 2022-01-27 13:06:04,729 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3361
+ 2022-01-27 13:06:04,729 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3362
+ 2022-01-27 13:06:15,304 DEBUG SenderThread:16113 [sender.py:send():182] send: stats
3363
+ 2022-01-27 13:06:19,811 DEBUG HandlerThread:16113 [handler.py:handle_request():126] handle_request: stop_status
3364
+ 2022-01-27 13:06:19,812 DEBUG SenderThread:16113 [sender.py:send_request():196] send_request: stop_status
3365
+ 2022-01-27 13:06:23,781 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
3366
+ 2022-01-27 13:06:24,782 INFO Thread-8 :16113 [dir_watcher.py:_on_file_modified():230] file/dir modified: /home/patrick/experiments/xls-r-300m-sv-cv8/wandb/run-20220127_121247-o8za41vn/files/output.log
wandb/run-20220127_121247-o8za41vn/run-o8za41vn.wandb CHANGED
Binary files a/wandb/run-20220127_121247-o8za41vn/run-o8za41vn.wandb and b/wandb/run-20220127_121247-o8za41vn/run-o8za41vn.wandb differ