File size: 140,262 Bytes
2c1884f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 |
========================
START TIME: Wed Jul 3 23:37:11 UTC 2024
python3 version = Python 3.10.14
========================
The token has not been saved to the git credentials helper. Pass `add_to_git_credential=True` in this function directly or `--add-to-git-credential` if using via `huggingface-cli` if you want to set the git credential as well.
Token is valid (permission: write).
Your token has been saved to /admin/home/ferdinand_mom/.cache/huggingface/token
Login successful
Already on 'bench_cluster'
M examples/config_tiny_llama.py
M examples/config_tiny_llama.yaml
M examples/train_tiny_llama.sh
M src/nanotron/models/llama.py
M src/nanotron/trainer.py
Your branch is up to date with 'origin/bench_cluster'.
Job status: RUNNING
W0703 23:37:14.533000 140516527933248 torch/distributed/run.py:757]
W0703 23:37:14.533000 140516527933248 torch/distributed/run.py:757] *****************************************
W0703 23:37:14.533000 140516527933248 torch/distributed/run.py:757] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
W0703 23:37:14.533000 140516527933248 torch/distributed/run.py:757] *****************************************
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Config:
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Config(general=GeneralArgs(project='bench_cluster',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: run='%date_%jobid',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: seed=42,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: step=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: consumed_train_samples=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: benchmark_csv_path=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: ignore_sanity_checks=True),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: parallelism=ParallelismArgs(dp=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: pp=8,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: tp=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: pp_engine=<nanotron.parallel.pipeline_parallel.engine.OneForwardOneBackwardPipelineEngine object at 0x7f8d7bee4940>,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: tp_mode=<TensorParallelLinearMode.REDUCE_SCATTER: 2>,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: tp_linear_async_communication=False,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: expert_parallel_size=1),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: model=ModelArgs(model_config=LlamaConfig(bos_token_id=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: eos_token_id=2,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: hidden_act='silu',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: hidden_size=2048,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: initializer_range=0.02,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: intermediate_size=4096,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: is_llama_config=True,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: max_position_embeddings=4096,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: num_attention_heads=32,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: num_hidden_layers=24,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: num_key_value_heads=32,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: pad_token_id=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: pretraining_tp=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: rms_norm_eps=1e-05,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: rope_scaling=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: rope_theta=10000.0,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: tie_word_embeddings=True,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: use_cache=True,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: vocab_size=50257),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: init_method=RandomInit(std=0.025),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: dtype=torch.bfloat16,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: make_vocab_size_divisible_by=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: ddp_bucket_cap_mb=25),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: tokenizer=TokenizerArgs(tokenizer_name_or_path='openai-community/gpt2',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: tokenizer_revision=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: tokenizer_max_length=None),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: checkpoints=CheckpointsArgs(checkpoints_path=Path('/dev/null'),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: checkpoint_interval=100000,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: save_initial_state=False,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: resume_checkpoint_path=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: checkpoints_path_is_shared_file_system=False),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: logging=LoggingArgs(log_level='info',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: log_level_replica='info',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: iteration_step_info_interval=1),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: tokens=TokensArgs(sequence_length=4096,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: train_steps=20,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: micro_batch_size=2,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: batch_accumulation_per_replica=512,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: val_check_interval=-1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: limit_val_batches=0,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: limit_test_batches=0),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: optimizer=OptimizerArgs(optimizer_factory=AdamWOptimizerArgs(adam_eps=1e-08,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: adam_beta1=0.9,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: adam_beta2=0.95,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: torch_adam_is_fused=True,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: name='adamW'),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: zero_stage=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: weight_decay=0.01,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: clip_grad=1.0,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: accumulate_grad_in_fp32=True,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: learning_rate_scheduler=LRSchedulerArgs(learning_rate=0.0001,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: lr_warmup_steps=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: lr_warmup_style='linear',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: lr_decay_style='linear',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: lr_decay_steps=19,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: lr_decay_starting_step=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: min_decay_lr=1e-05)),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: data_stages=[DatasetStageArgs(name='Training Stage',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: start_training_step=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: data=DataArgs(dataset=PretrainDatasetsArgs(hf_dataset_or_datasets='roneneldan/TinyStories',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: hf_dataset_splits='train',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: hf_dataset_config_name=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: dataset_processing_num_proc_per_process=64,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: dataset_overwrite_cache=False,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: text_column_name='text'),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: seed=42,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: num_loading_workers=0))],
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: profiler=ProfilerArgs(profiler_export_path=Path('/fsx/ferdinandmom/ferdinand-hf/bench_cluster/results/llama-1B/8_GPUS/dp-1_tp-1_pp-8_mbz-2')),
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: lighteval=None)
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Model Config:
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: LlamaConfig(bos_token_id=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: eos_token_id=2,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: hidden_act='silu',
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: hidden_size=2048,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: initializer_range=0.02,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: intermediate_size=4096,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: is_llama_config=True,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: max_position_embeddings=4096,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: num_attention_heads=32,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: num_hidden_layers=24,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: num_key_value_heads=32,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: pad_token_id=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: pretraining_tp=1,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: rms_norm_eps=1e-05,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: rope_scaling=None,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: rope_theta=10000.0,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: tie_word_embeddings=True,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: use_cache=True,
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: vocab_size=50257)
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Building model..
[default0]:07/03/2024 23:37:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Setting PP block ranks...
[default4]:07/03/2024 23:37:46 [INFO|DP=0|PP=4|TP=0|ip-26-0-169-247]: Local number of parameters: 126M (240.02MiB)
[default4]:07/03/2024 23:37:46 [INFO|DP=0|PP=4|TP=0|ip-26-0-169-247]: [After model building] Memory usage: 243.03MiB. Peak allocated: 245.06MiB Peak reserved: 262.00MiB
[default4]:07/03/2024 23:37:46 [INFO|DP=0|PP=4|TP=0|ip-26-0-169-247]: No checkpoint path provided.
[default1]:07/03/2024 23:37:46 [INFO|DP=0|PP=1|TP=0|ip-26-0-169-247]: Local number of parameters: 126M (240.02MiB)
[default1]:07/03/2024 23:37:46 [INFO|DP=0|PP=1|TP=0|ip-26-0-169-247]: [After model building] Memory usage: 243.03MiB. Peak allocated: 245.06MiB Peak reserved: 262.00MiB
[default1]:07/03/2024 23:37:46 [INFO|DP=0|PP=1|TP=0|ip-26-0-169-247]: No checkpoint path provided.
[default3]:07/03/2024 23:37:46 [INFO|DP=0|PP=3|TP=0|ip-26-0-169-247]: Local number of parameters: 168M (320.03MiB)
[default3]:07/03/2024 23:37:46 [INFO|DP=0|PP=3|TP=0|ip-26-0-169-247]: [After model building] Memory usage: 324.04MiB. Peak allocated: 326.07MiB Peak reserved: 336.00MiB
[default3]:07/03/2024 23:37:46 [INFO|DP=0|PP=3|TP=0|ip-26-0-169-247]: No checkpoint path provided.
[default0]:07/03/2024 23:37:46 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Total number of parameters: 1.21G (2312.82MiB)
[default0]:07/03/2024 23:37:46 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Local number of parameters: 271M (516.35MiB)
[default0]:07/03/2024 23:37:46 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: [After model building] Memory usage: 520.36MiB. Peak allocated: 522.39MiB Peak reserved: 534.00MiB
[default0]:07/03/2024 23:37:46 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: No checkpoint path provided.
[default0]:07/03/2024 23:37:46 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Parametrizing model parameters using StandardParametrizator
[default6]:07/03/2024 23:37:46 [INFO|DP=0|PP=6|TP=0|ip-26-0-169-247]: Local number of parameters: 168M (320.03MiB)
[default6]:07/03/2024 23:37:46 [INFO|DP=0|PP=6|TP=0|ip-26-0-169-247]: [After model building] Memory usage: 324.04MiB. Peak allocated: 326.07MiB Peak reserved: 336.00MiB
[default6]:07/03/2024 23:37:46 [INFO|DP=0|PP=6|TP=0|ip-26-0-169-247]: No checkpoint path provided.
[default2]:07/03/2024 23:37:46 [INFO|DP=0|PP=2|TP=0|ip-26-0-169-247]: Local number of parameters: 126M (240.02MiB)
[default2]:07/03/2024 23:37:46 [INFO|DP=0|PP=2|TP=0|ip-26-0-169-247]: [After model building] Memory usage: 243.03MiB. Peak allocated: 245.06MiB Peak reserved: 262.00MiB
[default2]:07/03/2024 23:37:46 [INFO|DP=0|PP=2|TP=0|ip-26-0-169-247]: No checkpoint path provided.
[default5]:07/03/2024 23:37:46 [INFO|DP=0|PP=5|TP=0|ip-26-0-169-247]: Local number of parameters: 126M (240.02MiB)
[default5]:07/03/2024 23:37:46 [INFO|DP=0|PP=5|TP=0|ip-26-0-169-247]: [After model building] Memory usage: 243.03MiB. Peak allocated: 245.06MiB Peak reserved: 262.00MiB
[default5]:07/03/2024 23:37:46 [INFO|DP=0|PP=5|TP=0|ip-26-0-169-247]: No checkpoint path provided.
[default7]:07/03/2024 23:37:46 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: Local number of parameters: 103M (196.32MiB)
[default7]:07/03/2024 23:37:46 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: [After model building] Memory usage: 196.33MiB. Peak allocated: 196.33MiB Peak reserved: 200.00MiB
[default7]:07/03/2024 23:37:46 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: No checkpoint path provided.
[default0]:07/03/2024 23:37:46 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: [Optimizer Building] Using LearningRateForSP as learning rate
[default0]:07/03/2024 23:37:46 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: [ZeRO sharding] Size of optimizer params per rank:
[default0]:07/03/2024 23:37:46 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: [ZeRO sharding] DP Rank 0 has 271M out of 271M (100.00%) params' optimizer states
[default0]:07/03/2024 23:37:47 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: [Training Plan] Stage Training Stage has 19 remaining training steps and has consumed 0 samples
[default0]:07/03/2024 23:37:47 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Using `datasets` library
[default0]:07/03/2024 23:37:47 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Loading tokenizer from openai-community/gpt2 and transformers/hf_hub versions ('4.41.2', '0.23.4')
[default0]:Repo card metadata block was not found. Setting CardData to empty.
[default0]:07/03/2024 23:37:47 [WARNING|DP=0|PP=0|TP=0|ip-26-0-169-247]: Repo card metadata block was not found. Setting CardData to empty.
[default0]:07/03/2024 23:37:48 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: [Training Plan] There are 1 training stages
[default0]:07/03/2024 23:37:48 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: [Stage Training Stage] start from step 1
[default0]:07/03/2024 23:37:48 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]:
[default0]:07/03/2024 23:37:48 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: [Start training] datetime: 2024-07-03 23:37:48.694001 | mbs: 2 | grad_accum: 512 | global_batch_size: 1024 | sequence_length: 4096 | train_steps: 20 | start_iteration_step: 0 | consumed_train_samples: 0
[default0]:07/03/2024 23:37:48 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Resuming training from stage Training Stage, it has trained for 0 samples and has 19 remaining train steps
[default0]:07/03/2024 23:37:48 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 2585.75MiB. Peak allocated 2585.75MiB. Peak reserved: 2602.00MiB
[default1]:Repo card metadata block was not found. Setting CardData to empty.
[default1]:07/03/2024 23:37:48 [WARNING|DP=0|PP=1|TP=0|ip-26-0-169-247]: Repo card metadata block was not found. Setting CardData to empty.
[default3]:07/03/2024 23:37:48 [WARNING|DP=0|PP=3|TP=0|ip-26-0-169-247]: Repo card metadata block was not found. Setting CardData to empty.
[default2]:07/03/2024 23:37:48 [WARNING|DP=0|PP=2|TP=0|ip-26-0-169-247]: Repo card metadata block was not found. Setting CardData to empty.
[default3]:Repo card metadata block was not found. Setting CardData to empty.
[default6]:07/03/2024 23:37:48 [WARNING|DP=0|PP=6|TP=0|ip-26-0-169-247]: Repo card metadata block was not found. Setting CardData to empty.
[default5]:07/03/2024 23:37:48 [WARNING|DP=0|PP=5|TP=0|ip-26-0-169-247]: Repo card metadata block was not found. Setting CardData to empty.
[default5]:Repo card metadata block was not found. Setting CardData to empty.
[default7]:07/03/2024 23:37:48 [WARNING|DP=0|PP=7|TP=0|ip-26-0-169-247]: Repo card metadata block was not found. Setting CardData to empty.
[default2]:Repo card metadata block was not found. Setting CardData to empty.
[default7]:Repo card metadata block was not found. Setting CardData to empty.
[default6]:Repo card metadata block was not found. Setting CardData to empty.
[default4]:07/03/2024 23:37:48 [WARNING|DP=0|PP=4|TP=0|ip-26-0-169-247]: Repo card metadata block was not found. Setting CardData to empty.
[default4]:Repo card metadata block was not found. Setting CardData to empty.
[default7]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/autograd/graph.py:744: UserWarning: c10d::allreduce_: an autograd kernel was not registered to the Autograd key(s) but we are trying to backprop through it. This may lead to silently incorrect behavior. This behavior is deprecated and will be removed in a future version of PyTorch. If your operator is differentiable, please ensure you have registered an autograd kernel to the correct Autograd key (e.g. DispatchKey::Autograd, DispatchKey::CompositeImplicitAutograd). If your operator is not differentiable, or to squash this warning and use the previous behavior, please register torch::CppFunction::makeFallthrough() to DispatchKey::Autograd. (Triggered internally at ../torch/csrc/autograd/autograd_not_implemented_fallback.cpp:63.)
[default7]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
[default6]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/autograd/graph.py:744: UserWarning: c10d::allreduce_: an autograd kernel was not registered to the Autograd key(s) but we are trying to backprop through it. This may lead to silently incorrect behavior. This behavior is deprecated and will be removed in a future version of PyTorch. If your operator is differentiable, please ensure you have registered an autograd kernel to the correct Autograd key (e.g. DispatchKey::Autograd, DispatchKey::CompositeImplicitAutograd). If your operator is not differentiable, or to squash this warning and use the previous behavior, please register torch::CppFunction::makeFallthrough() to DispatchKey::Autograd. (Triggered internally at ../torch/csrc/autograd/autograd_not_implemented_fallback.cpp:63.)
[default6]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
[default5]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/autograd/graph.py:744: UserWarning: c10d::allreduce_: an autograd kernel was not registered to the Autograd key(s) but we are trying to backprop through it. This may lead to silently incorrect behavior. This behavior is deprecated and will be removed in a future version of PyTorch. If your operator is differentiable, please ensure you have registered an autograd kernel to the correct Autograd key (e.g. DispatchKey::Autograd, DispatchKey::CompositeImplicitAutograd). If your operator is not differentiable, or to squash this warning and use the previous behavior, please register torch::CppFunction::makeFallthrough() to DispatchKey::Autograd. (Triggered internally at ../torch/csrc/autograd/autograd_not_implemented_fallback.cpp:63.)
[default5]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
[default4]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/autograd/graph.py:744: UserWarning: c10d::allreduce_: an autograd kernel was not registered to the Autograd key(s) but we are trying to backprop through it. This may lead to silently incorrect behavior. This behavior is deprecated and will be removed in a future version of PyTorch. If your operator is differentiable, please ensure you have registered an autograd kernel to the correct Autograd key (e.g. DispatchKey::Autograd, DispatchKey::CompositeImplicitAutograd). If your operator is not differentiable, or to squash this warning and use the previous behavior, please register torch::CppFunction::makeFallthrough() to DispatchKey::Autograd. (Triggered internally at ../torch/csrc/autograd/autograd_not_implemented_fallback.cpp:63.)
[default4]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
[default3]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/autograd/graph.py:744: UserWarning: c10d::allreduce_: an autograd kernel was not registered to the Autograd key(s) but we are trying to backprop through it. This may lead to silently incorrect behavior. This behavior is deprecated and will be removed in a future version of PyTorch. If your operator is differentiable, please ensure you have registered an autograd kernel to the correct Autograd key (e.g. DispatchKey::Autograd, DispatchKey::CompositeImplicitAutograd). If your operator is not differentiable, or to squash this warning and use the previous behavior, please register torch::CppFunction::makeFallthrough() to DispatchKey::Autograd. (Triggered internally at ../torch/csrc/autograd/autograd_not_implemented_fallback.cpp:63.)
[default3]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
[default2]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/autograd/graph.py:744: UserWarning: c10d::allreduce_: an autograd kernel was not registered to the Autograd key(s) but we are trying to backprop through it. This may lead to silently incorrect behavior. This behavior is deprecated and will be removed in a future version of PyTorch. If your operator is differentiable, please ensure you have registered an autograd kernel to the correct Autograd key (e.g. DispatchKey::Autograd, DispatchKey::CompositeImplicitAutograd). If your operator is not differentiable, or to squash this warning and use the previous behavior, please register torch::CppFunction::makeFallthrough() to DispatchKey::Autograd. (Triggered internally at ../torch/csrc/autograd/autograd_not_implemented_fallback.cpp:63.)
[default2]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
[default1]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/autograd/graph.py:744: UserWarning: c10d::allreduce_: an autograd kernel was not registered to the Autograd key(s) but we are trying to backprop through it. This may lead to silently incorrect behavior. This behavior is deprecated and will be removed in a future version of PyTorch. If your operator is differentiable, please ensure you have registered an autograd kernel to the correct Autograd key (e.g. DispatchKey::Autograd, DispatchKey::CompositeImplicitAutograd). If your operator is not differentiable, or to squash this warning and use the previous behavior, please register torch::CppFunction::makeFallthrough() to DispatchKey::Autograd. (Triggered internally at ../torch/csrc/autograd/autograd_not_implemented_fallback.cpp:63.)
[default1]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
[default0]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/autograd/graph.py:744: UserWarning: Attempting to run cuBLAS, but there was no current CUDA context! Attempting to set the primary context... (Triggered internally at ../aten/src/ATen/cuda/CublasHandlePool.cpp:135.)
[default0]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
[default0]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/autograd/graph.py:744: UserWarning: c10d::allreduce_: an autograd kernel was not registered to the Autograd key(s) but we are trying to backprop through it. This may lead to silently incorrect behavior. This behavior is deprecated and will be removed in a future version of PyTorch. If your operator is differentiable, please ensure you have registered an autograd kernel to the correct Autograd key (e.g. DispatchKey::Autograd, DispatchKey::CompositeImplicitAutograd). If your operator is not differentiable, or to squash this warning and use the previous behavior, please register torch::CppFunction::makeFallthrough() to DispatchKey::Autograd. (Triggered internally at ../torch/csrc/autograd/autograd_not_implemented_fallback.cpp:63.)
[default0]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
[default7]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/distributed/distributed_c10d.py:2261: UserWarning: torch.distributed.all_reduce_coalesced will be deprecated. If you must use it, please revisit our documentation later at https://pytorch.org/docs/master/distributed.html#collective-functions
[default7]: warnings.warn(
[default0]:/fsx/ferdinandmom/miniforge3/envs/env-bench-cluster/lib/python3.10/site-packages/torch/distributed/distributed_c10d.py:2261: UserWarning: torch.distributed.all_reduce_coalesced will be deprecated. If you must use it, please revisit our documentation later at https://pytorch.org/docs/master/distributed.html#collective-functions
[default0]: warnings.warn(
[default0]:07/03/2024 23:38:44 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 2651.82MiB. Peak allocated 22318.31MiB. Peak reserved: 22616.00MiB
[default0]:07/03/2024 23:38:44 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 4717.22MiB. Peak reserved: 23404.00MiB
[default7]:07/03/2024 23:38:44 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 1 / 20 | consumed_tokens: 4.19M | elapsed_time_per_iteration_ms: 55.3K | tokens_per_sec: 75.8K | tokens_per_sec_per_gpu: 9.48K | global_batch_size: 1.02K | lm_loss: 11.1 | lr: 0.0001 | model_tflops_per_gpu: 86 | hardware_tflops_per_gpu: 86 | grad_norm: 24.9 | cuda_memory_allocated: 1.92G | cuda_max_memory_reserved: 7.13G | hd_total_memory_tb: 312G | hd_used_memory_tb: 65.6G | hd_free_memory_tb: 247G
[default0]:07/03/2024 23:39:14 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:39:14 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 2 / 20 | consumed_tokens: 8.39M | elapsed_time_per_iteration_ms: 29.7K | tokens_per_sec: 141K | tokens_per_sec_per_gpu: 17.7K | global_batch_size: 1.02K | lm_loss: 11.1 | lr: 9.53e-05 | model_tflops_per_gpu: 160 | hardware_tflops_per_gpu: 160 | grad_norm: 25.1 | cuda_memory_allocated: 1.92G | cuda_max_memory_reserved: 7.13G | hd_total_memory_tb: 312G | hd_used_memory_tb: 65.6G | hd_free_memory_tb: 247G
[default0]:07/03/2024 23:39:14 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 4717.24MiB. Peak reserved: 24890.00MiB
[default0]:STAGE:2024-07-03 23:39:44 131218:131218 ActivityProfilerController.cpp:314] Completed Stage: Warm Up
[default0]:07/03/2024 23:39:44 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default0]:07/03/2024 23:39:44 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 4717.24MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:39:44 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 3 / 20 | consumed_tokens: 12.6M | elapsed_time_per_iteration_ms: 29.7K | tokens_per_sec: 141K | tokens_per_sec_per_gpu: 17.7K | global_batch_size: 1.02K | lm_loss: 9.49 | lr: 9.05e-05 | model_tflops_per_gpu: 160 | hardware_tflops_per_gpu: 160 | grad_norm: 21.5 | cuda_memory_allocated: 1.92G | cuda_max_memory_reserved: 7.13G | hd_total_memory_tb: 312G | hd_used_memory_tb: 65.6G | hd_free_memory_tb: 247G
[default0]:07/03/2024 23:40:13 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default0]:07/03/2024 23:40:13 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 4717.24MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:40:13 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 4 / 20 | consumed_tokens: 16.8M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 9.36 | lr: 8.58e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 21.4 | cuda_memory_allocated: 1.92G | cuda_max_memory_reserved: 7.13G | hd_total_memory_tb: 312G | hd_used_memory_tb: 65.6G | hd_free_memory_tb: 247G
[default0]:07/03/2024 23:40:43 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:40:43 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 5 / 20 | consumed_tokens: 21M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 9.02 | lr: 8.11e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 12.7
[default7]:07/03/2024 23:41:12 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 6 / 20 | consumed_tokens: 25.2M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 10.3 | lr: 7.63e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 47.1
[default0]:STAGE:2024-07-03 23:42:02 131218:131218 ActivityProfilerController.cpp:320] Completed Stage: Collection
[default0]:STAGE:2024-07-03 23:42:06 131218:131218 ActivityProfilerController.cpp:324] Completed Stage: Post Processing
[default0]:07/03/2024 23:48:06 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default0]:07/03/2024 23:48:36 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:48:36 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 7 / 20 | consumed_tokens: 29.4M | elapsed_time_per_iteration_ms: 444K | tokens_per_sec: 9.45K | tokens_per_sec_per_gpu: 1.18K | global_batch_size: 1.02K | lm_loss: 8.68 | lr: 7.16e-05 | model_tflops_per_gpu: 10.7 | hardware_tflops_per_gpu: 10.7 | grad_norm: 5.58
[default7]:07/03/2024 23:49:06 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 8 / 20 | consumed_tokens: 33.6M | elapsed_time_per_iteration_ms: 29.6K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.7K | global_batch_size: 1.02K | lm_loss: 8.32 | lr: 6.68e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 4.77
[default0]:07/03/2024 23:49:06 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default0]:07/03/2024 23:49:35 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:49:35 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 9 / 20 | consumed_tokens: 37.7M | elapsed_time_per_iteration_ms: 29.6K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.7K | global_batch_size: 1.02K | lm_loss: 7.95 | lr: 6.21e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 3.31
[default0]:07/03/2024 23:50:05 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:50:05 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 10 / 20 | consumed_tokens: 41.9M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 7.69 | lr: 5.74e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 4.31
[default0]:07/03/2024 23:50:34 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:50:34 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 11 / 20 | consumed_tokens: 46.1M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 7.45 | lr: 5.26e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 2.5
[default0]:07/03/2024 23:51:04 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:51:04 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 12 / 20 | consumed_tokens: 50.3M | elapsed_time_per_iteration_ms: 29.6K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.7K | global_batch_size: 1.02K | lm_loss: 7.37 | lr: 4.79e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 5.02
[default0]:07/03/2024 23:51:33 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:51:33 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 13 / 20 | consumed_tokens: 54.5M | elapsed_time_per_iteration_ms: 29.6K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.7K | global_batch_size: 1.02K | lm_loss: 7.31 | lr: 4.32e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 6.05
[default0]:07/03/2024 23:52:03 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:52:03 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 14 / 20 | consumed_tokens: 58.7M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 7.19 | lr: 3.84e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 5.35
[default7]:07/03/2024 23:52:32 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 15 / 20 | consumed_tokens: 62.9M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 7.06 | lr: 3.37e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 2.74
[default0]:07/03/2024 23:52:32 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:53:02 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 16 / 20 | consumed_tokens: 67.1M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 6.97 | lr: 2.89e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 2
[default0]:07/03/2024 23:53:02 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:53:31 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 17 / 20 | consumed_tokens: 71.3M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 6.91 | lr: 2.42e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 2.03
[default0]:07/03/2024 23:53:31 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default0]:07/03/2024 23:54:01 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:54:01 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 18 / 20 | consumed_tokens: 75.5M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 6.86 | lr: 1.95e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 2
[default0]:07/03/2024 23:54:30 [INFO|DP=0|PP=0|TP=0|ip-26-0-169-247]: Memory usage: 4717.22MiB. Peak allocated 24383.71MiB. Peak reserved: 24890.00MiB
[default7]:07/03/2024 23:54:30 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 19 / 20 | consumed_tokens: 79.7M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 6.81 | lr: 1.47e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 2.01
[default7]:07/03/2024 23:55:00 [INFO|DP=0|PP=7|TP=0|ip-26-0-169-247]: iteration: 20 / 20 | consumed_tokens: 83.9M | elapsed_time_per_iteration_ms: 29.5K | tokens_per_sec: 142K | tokens_per_sec_per_gpu: 17.8K | global_batch_size: 1.02K | lm_loss: 6.77 | lr: 1e-05 | model_tflops_per_gpu: 161 | hardware_tflops_per_gpu: 161 | grad_norm: 1.93
Saved 1 csv files over 1 completed logs
Processing file: /fsx/ferdinandmom/ferdinand-hf/bench_cluster/results/llama-1B/8_GPUS/dp-1_tp-1_pp-8_mbz-2/profiler/ip-26-0-169-247_131218.1720050406305829976.pt.trace.json
Results written to /fsx/ferdinandmom/ferdinand-hf/bench_cluster/results/llama-1B/8_GPUS/dp-1_tp-1_pp-8_mbz-2/profiler.csv
Consider using `hf_transfer` for faster uploads. This solution comes with some limitations. See https://huggingface.co/docs/huggingface_hub/hf_transfer for more details.
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 0%| | 0.00/12.2G [00:00<?, ?B/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 0%| | 557k/12.2G [00:00<36:51, 5.52MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 0%| | 14.2M/12.2G [00:00<02:30, 81.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 0%| | 22.3M/12.2G [00:00<04:37, 43.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 0%| | 32.0M/12.2G [00:00<04:14, 47.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 0%| | 48.0M/12.2G [00:00<03:50, 52.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 1%| | 64.0M/12.2G [00:01<03:11, 63.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 1%| | 80.0M/12.2G [00:01<03:08, 64.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 1%| | 96.0M/12.2G [00:01<03:19, 60.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 1%| | 112M/12.2G [00:01<03:34, 56.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 1%| | 128M/12.2G [00:02<03:37, 55.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 1%| | 144M/12.2G [00:02<03:16, 61.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 1%|▏ | 160M/12.2G [00:02<02:41, 74.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 1%|▏ | 169M/12.2G [00:02<03:32, 56.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 1%|▏ | 176M/12.2G [00:03<04:09, 48.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 2%|▏ | 192M/12.2G [00:03<03:37, 55.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 2%|▏ | 208M/12.2G [00:03<03:18, 60.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 2%|▏ | 224M/12.2G [00:03<03:02, 65.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 2%|▏ | 240M/12.2G [00:04<03:01, 65.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 2%|▏ | 256M/12.2G [00:04<02:59, 66.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 2%|▏ | 272M/12.2G [00:04<03:12, 62.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 2%|▏ | 288M/12.2G [00:04<03:04, 64.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 2%|▏ | 304M/12.2G [00:05<03:01, 65.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 3%|▎ | 320M/12.2G [00:05<03:01, 65.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 3%|▎ | 336M/12.2G [00:05<02:58, 66.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 3%|▎ | 352M/12.2G [00:05<03:36, 54.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 3%|▎ | 368M/12.2G [00:06<03:10, 62.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 3%|▎ | 384M/12.2G [00:06<02:58, 66.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 3%|▎ | 400M/12.2G [00:06<03:58, 49.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 3%|▎ | 416M/12.2G [00:07<04:02, 48.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 4%|▎ | 432M/12.2G [00:07<03:45, 52.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 4%|▎ | 448M/12.2G [00:07<03:28, 56.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 4%|▍ | 464M/12.2G [00:07<03:10, 61.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 4%|▍ | 480M/12.2G [00:08<03:05, 63.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 4%|▍ | 496M/12.2G [00:08<02:56, 66.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 4%|▍ | 512M/12.2G [00:08<02:49, 69.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 4%|▍ | 528M/12.2G [00:08<02:44, 70.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 4%|▍ | 544M/12.2G [00:08<02:35, 74.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 5%|▍ | 560M/12.2G [00:09<02:39, 73.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 5%|▍ | 576M/12.2G [00:09<04:36, 42.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 5%|▍ | 592M/12.2G [00:10<03:51, 50.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 5%|▍ | 608M/12.2G [00:10<03:35, 53.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 5%|▌ | 624M/12.2G [00:10<03:52, 49.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 5%|▌ | 640M/12.2G [00:10<03:26, 56.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 5%|▌ | 656M/12.2G [00:11<03:17, 58.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 6%|▌ | 672M/12.2G [00:11<03:02, 63.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 6%|▌ | 688M/12.2G [00:11<03:21, 57.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 6%|▌ | 704M/12.2G [00:11<03:02, 62.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 6%|▌ | 720M/12.2G [00:12<03:08, 61.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 6%|▌ | 736M/12.2G [00:12<03:02, 62.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 6%|▌ | 752M/12.2G [00:12<02:50, 67.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 6%|▋ | 768M/12.2G [00:12<02:53, 65.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 6%|▋ | 784M/12.2G [00:13<02:56, 64.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 7%|▋ | 798M/12.2G [00:13<02:30, 75.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 7%|▋ | 807M/12.2G [00:13<03:04, 61.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 7%|▋ | 816M/12.2G [00:13<03:31, 53.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 7%|▋ | 832M/12.2G [00:13<03:07, 60.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 7%|▋ | 848M/12.2G [00:14<03:14, 58.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 7%|▋ | 864M/12.2G [00:14<03:04, 61.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 7%|▋ | 880M/12.2G [00:15<05:00, 37.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 7%|▋ | 896M/12.2G [00:15<04:10, 45.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 7%|▋ | 912M/12.2G [00:15<03:58, 47.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 8%|▊ | 928M/12.2G [00:15<03:26, 54.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 8%|▊ | 944M/12.2G [00:16<03:21, 55.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 8%|▊ | 960M/12.2G [00:20<16:45, 11.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 8%|▊ | 976M/12.2G [00:20<12:27, 15.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 8%|▊ | 992M/12.2G [00:20<09:49, 19.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 8%|▊ | 1.01G/12.2G [00:21<07:43, 24.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 8%|▊ | 1.02G/12.2G [00:21<06:07, 30.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 9%|▊ | 1.04G/12.2G [00:21<05:32, 33.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 9%|▊ | 1.06G/12.2G [00:21<04:40, 39.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 9%|▉ | 1.07G/12.2G [00:22<03:58, 46.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 9%|▉ | 1.09G/12.2G [00:22<03:38, 50.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 9%|▉ | 1.10G/12.2G [00:22<03:15, 56.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 9%|▉ | 1.12G/12.2G [00:22<03:05, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 9%|▉ | 1.14G/12.2G [00:22<03:05, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 9%|▉ | 1.15G/12.2G [00:23<03:02, 60.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 10%|▉ | 1.17G/12.2G [00:23<02:35, 70.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 10%|▉ | 1.18G/12.2G [00:23<02:40, 68.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 10%|▉ | 1.20G/12.2G [00:23<02:37, 69.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 10%|▉ | 1.22G/12.2G [00:24<02:33, 71.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 10%|█ | 1.23G/12.2G [00:24<02:40, 68.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 10%|█ | 1.25G/12.2G [00:24<02:47, 65.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 10%|█ | 1.26G/12.2G [00:24<02:38, 69.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 10%|█ | 1.28G/12.2G [00:25<03:50, 47.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 11%|█ | 1.30G/12.2G [00:25<03:20, 54.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 11%|█ | 1.31G/12.2G [00:25<03:05, 58.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 11%|█ | 1.33G/12.2G [00:26<03:00, 60.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 11%|█ | 1.34G/12.2G [00:26<02:52, 62.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 11%|█ | 1.36G/12.2G [00:26<02:55, 61.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 11%|█▏ | 1.38G/12.2G [00:26<02:43, 66.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 11%|█▏ | 1.39G/12.2G [00:27<02:56, 61.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 12%|█▏ | 1.41G/12.2G [00:27<02:48, 64.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 12%|█▏ | 1.42G/12.2G [00:27<02:41, 66.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 12%|█▏ | 1.44G/12.2G [00:27<02:37, 68.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 12%|█▏ | 1.46G/12.2G [00:28<02:56, 60.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 12%|█▏ | 1.47G/12.2G [00:28<03:01, 59.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 12%|█▏ | 1.49G/12.2G [00:28<03:05, 57.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 12%|█▏ | 1.50G/12.2G [00:28<03:08, 56.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 12%|█▏ | 1.52G/12.2G [00:29<03:06, 57.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 13%|█▎ | 1.54G/12.2G [00:29<03:08, 56.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 13%|█▎ | 1.55G/12.2G [00:29<03:16, 54.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 13%|█▎ | 1.57G/12.2G [00:29<02:58, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 13%|█▎ | 1.58G/12.2G [00:30<02:55, 60.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 13%|█▎ | 1.60G/12.2G [00:30<02:52, 61.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 13%|█▎ | 1.62G/12.2G [00:30<02:44, 64.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 13%|█▎ | 1.63G/12.2G [00:30<02:41, 65.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 14%|█▎ | 1.65G/12.2G [00:31<02:56, 59.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 14%|█▎ | 1.66G/12.2G [00:31<02:45, 63.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 14%|█▍ | 1.68G/12.2G [00:31<02:53, 60.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 14%|█▍ | 1.70G/12.2G [00:31<02:43, 64.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 14%|█▍ | 1.71G/12.2G [00:32<02:55, 59.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 14%|█▍ | 1.73G/12.2G [00:32<02:54, 60.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 14%|█▍ | 1.74G/12.2G [00:32<02:37, 66.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 14%|█▍ | 1.76G/12.2G [00:33<02:47, 62.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 15%|█▍ | 1.78G/12.2G [00:33<02:36, 66.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 15%|█▍ | 1.79G/12.2G [00:33<02:44, 63.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 15%|█▍ | 1.81G/12.2G [00:33<03:02, 56.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 15%|█▍ | 1.82G/12.2G [00:34<02:54, 59.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 15%|█▌ | 1.84G/12.2G [00:34<02:38, 65.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 15%|█▌ | 1.86G/12.2G [00:34<02:31, 68.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 15%|█▌ | 1.87G/12.2G [00:34<02:28, 69.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 15%|█▌ | 1.89G/12.2G [00:34<02:33, 67.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 16%|█▌ | 1.90G/12.2G [00:35<02:44, 62.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 16%|█▌ | 1.92G/12.2G [00:35<02:39, 64.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 16%|█▌ | 1.94G/12.2G [00:35<02:53, 59.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 16%|█▌ | 1.95G/12.2G [00:36<02:46, 61.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 16%|█▌ | 1.97G/12.2G [00:36<03:01, 56.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 16%|█▌ | 1.98G/12.2G [00:36<02:41, 63.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 16%|█▋ | 1.99G/12.2G [00:37<06:47, 25.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 16%|█▋ | 2.00G/12.2G [00:38<06:22, 26.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 16%|█▋ | 2.01G/12.2G [00:38<05:06, 33.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.02G/12.2G [00:38<05:24, 31.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.03G/12.2G [00:38<04:21, 39.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.05G/12.2G [00:38<03:52, 43.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.06G/12.2G [00:39<03:30, 48.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.08G/12.2G [00:39<03:01, 55.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.08G/12.2G [00:40<05:38, 29.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.09G/12.2G [00:40<05:41, 29.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.10G/12.2G [00:40<05:59, 28.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.11G/12.2G [00:40<04:20, 38.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 17%|█▋ | 2.13G/12.2G [00:40<03:33, 47.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 18%|█▊ | 2.14G/12.2G [00:41<03:21, 49.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 18%|█▊ | 2.14G/12.2G [00:41<04:56, 33.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 18%|█▊ | 2.16G/12.2G [00:41<03:51, 43.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 18%|█▊ | 2.18G/12.2G [00:42<03:33, 47.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 18%|█▊ | 2.19G/12.2G [00:42<03:03, 54.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 18%|█▊ | 2.21G/12.2G [00:42<02:59, 55.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 18%|█▊ | 2.22G/12.2G [00:42<03:09, 52.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 18%|█▊ | 2.24G/12.2G [00:43<03:16, 50.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 18%|█▊ | 2.26G/12.2G [00:43<03:21, 49.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 19%|█▊ | 2.27G/12.2G [00:44<06:11, 26.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 19%|█▊ | 2.29G/12.2G [00:45<04:59, 33.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 19%|█▉ | 2.30G/12.2G [00:45<04:35, 35.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 19%|█▉ | 2.32G/12.2G [00:45<04:04, 40.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 19%|█▉ | 2.34G/12.2G [00:45<03:35, 45.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 19%|█▉ | 2.35G/12.2G [00:46<03:10, 51.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 19%|█▉ | 2.37G/12.2G [00:47<05:05, 32.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 19%|█▉ | 2.37G/12.2G [00:47<04:43, 34.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 20%|█▉ | 2.38G/12.2G [00:47<04:54, 33.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 20%|█▉ | 2.39G/12.2G [00:47<06:16, 26.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 20%|█▉ | 2.40G/12.2G [00:47<04:37, 35.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 20%|█▉ | 2.42G/12.2G [00:48<04:03, 40.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 20%|█▉ | 2.43G/12.2G [00:48<03:49, 42.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 20%|██ | 2.45G/12.2G [00:48<03:29, 46.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 20%|██ | 2.46G/12.2G [00:49<03:19, 48.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 20%|██ | 2.48G/12.2G [00:49<03:27, 46.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 20%|██ | 2.50G/12.2G [00:49<03:18, 49.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 21%|██ | 2.51G/12.2G [00:50<02:54, 55.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 21%|██ | 2.53G/12.2G [00:50<03:03, 52.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 21%|██ | 2.54G/12.2G [00:50<02:52, 55.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 21%|██ | 2.56G/12.2G [00:50<02:41, 59.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 21%|██ | 2.58G/12.2G [00:51<02:37, 61.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 21%|██ | 2.59G/12.2G [00:51<02:36, 61.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 21%|██▏ | 2.61G/12.2G [00:51<02:43, 58.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 21%|██▏ | 2.62G/12.2G [00:51<02:37, 61.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 22%|██▏ | 2.64G/12.2G [00:52<02:23, 66.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 22%|██▏ | 2.66G/12.2G [00:52<02:18, 68.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 22%|██▏ | 2.67G/12.2G [00:52<02:18, 68.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 22%|██▏ | 2.69G/12.2G [00:52<02:15, 70.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 22%|██▏ | 2.70G/12.2G [00:52<02:04, 76.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 22%|██▏ | 2.71G/12.2G [00:53<02:41, 58.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 22%|██▏ | 2.72G/12.2G [00:53<02:46, 57.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 22%|██▏ | 2.74G/12.2G [00:53<02:37, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 23%|██▎ | 2.75G/12.2G [00:53<02:35, 60.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 23%|██▎ | 2.77G/12.2G [00:54<02:32, 62.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 23%|██▎ | 2.78G/12.2G [00:59<17:48, 8.82MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 23%|██▎ | 2.80G/12.2G [00:59<13:14, 11.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 23%|██▎ | 2.82G/12.2G [00:59<10:11, 15.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 23%|██▎ | 2.83G/12.2G [01:00<07:59, 19.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 23%|██▎ | 2.85G/12.2G [01:00<06:15, 24.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 23%|██▎ | 2.86G/12.2G [01:00<05:15, 29.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 24%|██▎ | 2.88G/12.2G [01:01<04:23, 35.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 24%|██▎ | 2.90G/12.2G [01:01<03:52, 40.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 24%|██▍ | 2.91G/12.2G [01:01<03:30, 44.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 24%|██▍ | 2.93G/12.2G [01:01<03:08, 49.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 24%|██▍ | 2.94G/12.2G [01:02<02:59, 51.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 24%|██▍ | 2.96G/12.2G [01:02<03:25, 45.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 24%|██▍ | 2.97G/12.2G [01:02<02:46, 55.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 24%|██▍ | 2.98G/12.2G [01:03<03:32, 43.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 25%|██▍ | 2.99G/12.2G [01:03<03:30, 43.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 25%|██▍ | 3.01G/12.2G [01:03<03:21, 45.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 25%|██▍ | 3.02G/12.2G [01:03<03:09, 48.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 25%|██▍ | 3.04G/12.2G [01:04<02:39, 57.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 25%|██▌ | 3.06G/12.2G [01:04<02:24, 63.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 25%|██▌ | 3.07G/12.2G [01:04<02:22, 64.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 25%|██▌ | 3.09G/12.2G [01:04<02:30, 60.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 25%|██▌ | 3.10G/12.2G [01:05<02:22, 64.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 26%|██▌ | 3.12G/12.2G [01:05<02:24, 62.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 26%|██▌ | 3.14G/12.2G [01:05<02:22, 63.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 26%|██▌ | 3.15G/12.2G [01:05<02:51, 52.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 26%|██▌ | 3.17G/12.2G [01:06<02:47, 53.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 26%|██▌ | 3.18G/12.2G [01:06<02:28, 60.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 26%|██▌ | 3.20G/12.2G [01:06<02:28, 60.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 26%|██▋ | 3.22G/12.2G [01:06<02:20, 64.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 26%|██▋ | 3.23G/12.2G [01:07<02:16, 65.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 27%|██▋ | 3.25G/12.2G [01:07<02:14, 66.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 27%|██▋ | 3.26G/12.2G [01:07<02:16, 65.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 27%|██▋ | 3.28G/12.2G [01:07<02:17, 65.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 27%|██▋ | 3.30G/12.2G [01:08<02:19, 64.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 27%|██▋ | 3.31G/12.2G [01:08<02:20, 63.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 27%|██▋ | 3.33G/12.2G [01:08<02:33, 57.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 27%|██▋ | 3.34G/12.2G [01:08<02:25, 60.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 27%|██▋ | 3.35G/12.2G [01:09<02:52, 51.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.36G/12.2G [01:09<02:49, 52.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.38G/12.2G [01:09<02:44, 53.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.39G/12.2G [01:09<02:13, 66.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.40G/12.2G [01:10<02:51, 51.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.41G/12.2G [01:10<02:49, 51.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.41G/12.2G [01:10<05:36, 26.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.42G/12.2G [01:11<04:16, 34.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.44G/12.2G [01:11<03:30, 41.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.45G/12.2G [01:11<03:05, 47.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.46G/12.2G [01:11<03:32, 41.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.47G/12.2G [01:11<02:59, 48.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 28%|██▊ | 3.47G/12.2G [01:11<03:10, 45.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▊ | 3.48G/12.2G [01:12<02:30, 58.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▊ | 3.49G/12.2G [01:12<02:47, 51.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▊ | 3.50G/12.2G [01:12<03:04, 47.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.51G/12.2G [01:12<03:26, 42.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.51G/12.2G [01:12<03:40, 39.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.52G/12.2G [01:13<04:09, 34.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.52G/12.2G [01:13<05:13, 27.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.53G/12.2G [01:13<03:41, 39.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.54G/12.2G [01:13<04:58, 29.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.55G/12.2G [01:13<03:51, 37.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.55G/12.2G [01:14<04:23, 32.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.57G/12.2G [01:14<03:15, 44.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.58G/12.2G [01:14<02:54, 49.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 29%|██▉ | 3.60G/12.2G [01:14<02:30, 57.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 30%|██▉ | 3.62G/12.2G [01:15<02:31, 56.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 30%|██▉ | 3.63G/12.2G [01:15<02:18, 61.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 30%|██▉ | 3.65G/12.2G [01:15<02:16, 62.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 30%|███ | 3.66G/12.2G [01:15<02:38, 53.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 30%|███ | 3.68G/12.2G [01:16<02:26, 58.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 30%|███ | 3.70G/12.2G [01:16<02:26, 58.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 30%|███ | 3.71G/12.2G [01:16<02:26, 57.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 31%|███ | 3.73G/12.2G [01:17<02:32, 55.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 31%|███ | 3.74G/12.2G [01:17<02:36, 54.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 31%|███ | 3.76G/12.2G [01:17<02:29, 56.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 31%|███ | 3.78G/12.2G [01:17<02:30, 55.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 31%|███ | 3.79G/12.2G [01:18<02:30, 55.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 31%|███ | 3.81G/12.2G [01:18<02:28, 56.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 31%|███▏ | 3.82G/12.2G [01:18<02:25, 57.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 31%|███▏ | 3.84G/12.2G [01:19<02:21, 59.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 32%|███▏ | 3.86G/12.2G [01:19<02:16, 61.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 32%|███▏ | 3.87G/12.2G [01:19<02:09, 64.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 32%|███▏ | 3.89G/12.2G [01:19<02:05, 66.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 32%|███▏ | 3.90G/12.2G [01:19<02:01, 68.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 32%|███▏ | 3.92G/12.2G [01:20<01:58, 69.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 32%|███▏ | 3.94G/12.2G [01:20<01:58, 69.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 32%|███▏ | 3.95G/12.2G [01:20<01:58, 69.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 33%|███▎ | 3.97G/12.2G [01:20<02:01, 67.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 33%|███▎ | 3.98G/12.2G [01:21<02:08, 63.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 33%|███▎ | 4.00G/12.2G [01:21<02:16, 60.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 33%|███▎ | 4.02G/12.2G [01:21<02:11, 62.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 33%|███▎ | 4.03G/12.2G [01:22<02:54, 46.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 33%|███▎ | 4.05G/12.2G [01:22<02:41, 50.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 33%|███▎ | 4.06G/12.2G [01:22<02:33, 53.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 33%|███▎ | 4.08G/12.2G [01:23<02:26, 55.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 34%|███▎ | 4.10G/12.2G [01:23<02:30, 54.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 34%|███▎ | 4.11G/12.2G [01:23<02:28, 54.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 34%|███▍ | 4.13G/12.2G [01:23<02:13, 60.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 34%|███▍ | 4.14G/12.2G [01:24<02:15, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 34%|███▍ | 4.16G/12.2G [01:24<02:07, 62.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 34%|███▍ | 4.18G/12.2G [01:24<02:12, 60.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 34%|███▍ | 4.19G/12.2G [01:24<02:12, 60.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 34%|███▍ | 4.21G/12.2G [01:25<02:06, 63.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 35%|███▍ | 4.22G/12.2G [01:25<02:10, 61.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 35%|███▍ | 4.24G/12.2G [01:25<02:01, 65.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 35%|███▍ | 4.26G/12.2G [01:25<02:00, 66.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 35%|███▍ | 4.27G/12.2G [01:26<01:56, 68.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 35%|███▌ | 4.29G/12.2G [01:26<02:01, 65.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 35%|███▌ | 4.30G/12.2G [01:26<02:07, 62.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 35%|███▌ | 4.32G/12.2G [01:26<01:59, 66.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 36%|███▌ | 4.34G/12.2G [01:27<02:12, 59.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 36%|███▌ | 4.35G/12.2G [01:27<02:07, 61.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 36%|███▌ | 4.37G/12.2G [01:27<02:05, 62.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 36%|███▌ | 4.38G/12.2G [01:27<02:07, 61.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 36%|███▌ | 4.40G/12.2G [01:28<02:29, 52.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 36%|███▌ | 4.42G/12.2G [01:28<02:19, 55.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 36%|███▋ | 4.43G/12.2G [01:28<02:05, 62.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 36%|███▋ | 4.45G/12.2G [01:29<02:33, 50.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 37%|███▋ | 4.46G/12.2G [01:29<03:10, 40.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 37%|███▋ | 4.48G/12.2G [01:29<02:49, 45.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 37%|███▋ | 4.50G/12.2G [01:30<02:26, 52.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 37%|███▋ | 4.51G/12.2G [01:30<02:25, 52.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 37%|███▋ | 4.53G/12.2G [01:30<02:15, 56.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 37%|███▋ | 4.54G/12.2G [01:30<02:10, 58.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 37%|███▋ | 4.56G/12.2G [01:31<02:19, 54.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 37%|███▋ | 4.58G/12.2G [01:31<02:11, 58.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 38%|███▊ | 4.59G/12.2G [01:32<02:40, 47.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 38%|███▊ | 4.61G/12.2G [01:32<02:27, 51.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 38%|███▊ | 4.62G/12.2G [01:32<02:21, 53.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 38%|███▊ | 4.64G/12.2G [01:32<02:21, 53.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 38%|███▊ | 4.66G/12.2G [01:33<02:05, 60.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 38%|███▊ | 4.67G/12.2G [01:33<01:54, 65.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 38%|███▊ | 4.69G/12.2G [01:33<01:50, 68.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 39%|███▊ | 4.70G/12.2G [01:33<01:57, 63.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 39%|███▊ | 4.72G/12.2G [01:34<02:14, 55.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 39%|███▉ | 4.74G/12.2G [01:34<02:14, 55.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 39%|███▉ | 4.75G/12.2G [01:34<02:00, 62.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 39%|███▉ | 4.77G/12.2G [01:34<01:57, 63.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 39%|███▉ | 4.78G/12.2G [01:35<01:55, 64.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 39%|███▉ | 4.80G/12.2G [01:35<02:03, 60.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 39%|███▉ | 4.82G/12.2G [01:35<01:55, 64.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 40%|███▉ | 4.83G/12.2G [01:35<01:47, 68.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 40%|███▉ | 4.85G/12.2G [01:36<01:58, 62.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 40%|███▉ | 4.86G/12.2G [01:36<02:05, 58.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 40%|███▉ | 4.88G/12.2G [01:36<01:58, 61.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 40%|████ | 4.90G/12.2G [01:36<02:13, 54.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 40%|████ | 4.91G/12.2G [01:37<02:08, 56.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 40%|████ | 4.93G/12.2G [01:38<03:15, 37.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 41%|████ | 4.94G/12.2G [01:38<02:42, 44.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 41%|████ | 4.96G/12.2G [01:38<03:17, 36.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 41%|████ | 4.98G/12.2G [01:39<02:46, 43.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 41%|████ | 4.99G/12.2G [01:39<02:30, 48.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 41%|████ | 5.01G/12.2G [01:39<02:14, 53.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 41%|████ | 5.02G/12.2G [01:39<02:08, 55.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 41%|████▏ | 5.04G/12.2G [01:40<02:09, 55.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 41%|████▏ | 5.06G/12.2G [01:40<01:56, 61.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 42%|████▏ | 5.07G/12.2G [01:40<01:42, 69.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 42%|████▏ | 5.09G/12.2G [01:40<01:49, 64.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 42%|████▏ | 5.10G/12.2G [01:41<01:56, 61.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 42%|████▏ | 5.12G/12.2G [01:41<02:04, 57.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 42%|████▏ | 5.14G/12.2G [01:41<02:19, 50.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 42%|████▏ | 5.15G/12.2G [01:41<02:12, 53.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 42%|████▏ | 5.17G/12.2G [01:42<02:15, 51.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 42%|████▏ | 5.18G/12.2G [01:42<01:59, 58.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 43%|████▎ | 5.20G/12.2G [01:42<01:52, 62.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 43%|████▎ | 5.22G/12.2G [01:43<02:02, 57.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 43%|████▎ | 5.23G/12.2G [01:43<01:59, 58.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 43%|████▎ | 5.25G/12.2G [01:43<01:50, 62.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 43%|████▎ | 5.26G/12.2G [01:43<01:50, 63.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 43%|████▎ | 5.28G/12.2G [01:43<01:44, 66.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 43%|████▎ | 5.30G/12.2G [01:44<01:46, 65.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 44%|████▎ | 5.31G/12.2G [01:44<01:43, 66.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 44%|████▎ | 5.33G/12.2G [01:44<01:38, 69.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 44%|████▍ | 5.34G/12.2G [01:44<01:43, 66.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 44%|████▍ | 5.36G/12.2G [01:45<02:23, 47.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 44%|████▍ | 5.38G/12.2G [01:45<02:04, 55.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 44%|████▍ | 5.39G/12.2G [01:45<02:03, 55.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 44%|████▍ | 5.41G/12.2G [01:46<01:57, 58.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 44%|████▍ | 5.42G/12.2G [01:46<02:05, 54.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 45%|████▍ | 5.44G/12.2G [01:46<02:00, 55.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 45%|████▍ | 5.46G/12.2G [01:47<01:57, 57.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 45%|████▍ | 5.47G/12.2G [01:47<01:52, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 45%|████▍ | 5.49G/12.2G [01:47<01:51, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 45%|████▌ | 5.50G/12.2G [01:47<01:51, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 45%|████▌ | 5.52G/12.2G [01:48<01:52, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 45%|████▌ | 5.54G/12.2G [01:48<01:52, 59.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 45%|████▌ | 5.55G/12.2G [01:48<01:49, 61.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 46%|████▌ | 5.57G/12.2G [01:48<01:59, 55.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 46%|████▌ | 5.58G/12.2G [01:49<01:54, 57.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 46%|████▌ | 5.60G/12.2G [01:49<02:00, 54.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 46%|████▌ | 5.62G/12.2G [01:49<01:59, 55.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 46%|████▌ | 5.63G/12.2G [01:50<02:09, 50.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 46%|████▋ | 5.65G/12.2G [01:50<02:06, 52.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 46%|████▋ | 5.66G/12.2G [01:50<02:02, 53.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 47%|████▋ | 5.68G/12.2G [01:51<01:56, 56.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 47%|████▋ | 5.70G/12.2G [01:51<01:55, 56.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 47%|████▋ | 5.71G/12.2G [01:51<02:17, 47.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 47%|████▋ | 5.73G/12.2G [01:51<01:58, 54.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 47%|████▋ | 5.74G/12.2G [01:52<01:48, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 47%|████▋ | 5.76G/12.2G [01:52<01:34, 68.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 47%|████▋ | 5.78G/12.2G [01:52<01:48, 59.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 47%|████▋ | 5.79G/12.2G [01:53<01:57, 54.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 48%|████▊ | 5.81G/12.2G [01:53<02:01, 52.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 48%|████▊ | 5.82G/12.2G [01:53<01:54, 55.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 48%|████▊ | 5.84G/12.2G [01:54<02:04, 51.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 48%|████▊ | 5.86G/12.2G [01:54<02:27, 43.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 48%|████▊ | 5.87G/12.2G [01:54<02:17, 46.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 48%|████▊ | 5.89G/12.2G [01:55<02:12, 47.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 48%|████▊ | 5.90G/12.2G [01:55<01:56, 54.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 49%|████▊ | 5.92G/12.2G [01:55<01:59, 52.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 49%|████▊ | 5.94G/12.2G [01:55<01:53, 55.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 49%|████▊ | 5.95G/12.2G [01:56<01:40, 62.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 49%|████▉ | 5.96G/12.2G [01:56<02:36, 39.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 49%|████▉ | 5.97G/12.2G [01:56<02:43, 38.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 49%|████▉ | 5.98G/12.2G [01:57<02:16, 45.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 49%|████▉ | 6.00G/12.2G [01:57<02:03, 50.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 49%|████▉ | 6.02G/12.2G [01:57<01:57, 52.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 49%|████▉ | 6.03G/12.2G [01:57<01:52, 54.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 50%|████▉ | 6.05G/12.2G [01:58<01:54, 53.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 50%|████▉ | 6.06G/12.2G [01:58<01:48, 56.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 50%|████▉ | 6.08G/12.2G [01:58<01:52, 54.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 50%|████▉ | 6.10G/12.2G [01:59<01:51, 55.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 50%|█████ | 6.11G/12.2G [01:59<01:38, 61.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 50%|█████ | 6.13G/12.2G [01:59<01:36, 63.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 50%|█████ | 6.14G/12.2G [01:59<01:41, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 50%|█████ | 6.16G/12.2G [01:59<01:24, 71.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 51%|█████ | 6.17G/12.2G [02:00<01:55, 52.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 51%|█████ | 6.18G/12.2G [02:00<02:10, 46.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 51%|█████ | 6.19G/12.2G [02:00<01:59, 50.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 51%|█████ | 6.21G/12.2G [02:01<01:49, 54.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 51%|█████ | 6.22G/12.2G [02:01<01:42, 58.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 51%|█████ | 6.24G/12.2G [02:01<01:33, 64.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 51%|█████▏ | 6.26G/12.2G [02:01<01:30, 66.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 51%|█████▏ | 6.27G/12.2G [02:01<01:28, 67.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 52%|█████▏ | 6.29G/12.2G [02:02<01:37, 60.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 52%|█████▏ | 6.30G/12.2G [02:02<01:36, 61.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 52%|█████▏ | 6.32G/12.2G [02:02<01:33, 62.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 52%|█████▏ | 6.34G/12.2G [02:03<01:50, 53.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 52%|█████▏ | 6.35G/12.2G [02:03<01:41, 57.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 52%|█████▏ | 6.37G/12.2G [02:03<02:15, 43.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 52%|█████▏ | 6.38G/12.2G [02:04<02:11, 44.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 52%|█████▏ | 6.40G/12.2G [02:04<01:53, 51.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 53%|█████▎ | 6.42G/12.2G [02:04<01:55, 50.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 53%|█████▎ | 6.43G/12.2G [02:05<01:49, 52.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 53%|█████▎ | 6.45G/12.2G [02:05<01:36, 59.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 53%|█████▎ | 6.45G/12.2G [02:06<03:08, 30.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 53%|█████▎ | 6.46G/12.2G [02:06<02:53, 33.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 53%|█████▎ | 6.48G/12.2G [02:06<02:15, 42.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 53%|█████▎ | 6.50G/12.2G [02:06<02:03, 46.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 53%|█████▎ | 6.51G/12.2G [02:07<01:57, 48.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 53%|█████▎ | 6.53G/12.2G [02:07<01:42, 55.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 54%|█████▎ | 6.54G/12.2G [02:07<01:40, 56.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 54%|█████▎ | 6.56G/12.2G [02:07<01:34, 59.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 54%|█████▍ | 6.58G/12.2G [02:08<01:31, 61.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 54%|█████▍ | 6.59G/12.2G [02:08<01:27, 64.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 54%|█████▍ | 6.61G/12.2G [02:08<01:29, 62.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 54%|█████▍ | 6.62G/12.2G [02:08<01:23, 66.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 54%|█████▍ | 6.64G/12.2G [02:09<01:29, 62.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 55%|█████▍ | 6.66G/12.2G [02:09<01:27, 63.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 55%|█████▍ | 6.67G/12.2G [02:09<01:26, 63.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 55%|█████▍ | 6.69G/12.2G [02:09<01:31, 60.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 55%|█████▍ | 6.70G/12.2G [02:10<01:38, 55.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 55%|█████▌ | 6.72G/12.2G [02:10<01:36, 57.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 55%|█████▌ | 6.74G/12.2G [02:10<01:37, 56.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 55%|█████▌ | 6.75G/12.2G [02:11<02:11, 41.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 55%|█████▌ | 6.77G/12.2G [02:11<02:01, 44.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 56%|█████▌ | 6.78G/12.2G [02:11<01:56, 46.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 56%|█████▌ | 6.80G/12.2G [02:12<01:52, 48.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 56%|█████▌ | 6.82G/12.2G [02:12<01:49, 49.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 56%|█████▌ | 6.83G/12.2G [02:12<01:47, 50.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 56%|█████▌ | 6.85G/12.2G [02:13<01:37, 54.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 56%|█████▌ | 6.86G/12.2G [02:13<01:33, 57.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 56%|█████▋ | 6.88G/12.2G [02:13<01:32, 57.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 56%|█████▋ | 6.90G/12.2G [02:13<01:35, 55.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 57%|█████▋ | 6.91G/12.2G [02:14<01:26, 61.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 57%|█████▋ | 6.93G/12.2G [02:14<01:37, 54.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 57%|█████▋ | 6.94G/12.2G [02:14<01:33, 56.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 57%|█████▋ | 6.96G/12.2G [02:14<01:29, 58.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 57%|█████▋ | 6.98G/12.2G [02:15<01:22, 63.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 57%|█████▋ | 6.99G/12.2G [02:15<01:35, 54.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 57%|█████▋ | 7.01G/12.2G [02:15<01:45, 49.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 58%|█████▊ | 7.02G/12.2G [02:16<01:42, 50.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 58%|█████▊ | 7.04G/12.2G [02:16<01:37, 53.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 58%|█████▊ | 7.06G/12.2G [02:17<02:45, 31.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 58%|█████▊ | 7.07G/12.2G [02:17<02:24, 35.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 58%|█████▊ | 7.09G/12.2G [02:18<02:01, 42.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 58%|█████▊ | 7.10G/12.2G [02:18<01:57, 43.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 58%|█████▊ | 7.12G/12.2G [02:18<01:47, 47.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 58%|█████▊ | 7.14G/12.2G [02:19<01:51, 45.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 59%|█████▊ | 7.15G/12.2G [02:19<01:39, 50.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 59%|█████▊ | 7.17G/12.2G [02:19<01:31, 54.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 59%|█████▉ | 7.18G/12.2G [02:19<01:31, 54.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 59%|█████▉ | 7.20G/12.2G [02:20<01:32, 53.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 59%|█████▉ | 7.22G/12.2G [02:20<01:27, 57.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 59%|█████▉ | 7.23G/12.2G [02:20<01:23, 59.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 59%|█████▉ | 7.25G/12.2G [02:20<01:20, 61.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 60%|█████▉ | 7.26G/12.2G [02:21<01:23, 58.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 60%|█████▉ | 7.28G/12.2G [02:21<01:26, 57.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 60%|█████▉ | 7.30G/12.2G [02:21<01:25, 57.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 60%|█████▉ | 7.31G/12.2G [02:21<01:24, 58.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 60%|██████ | 7.33G/12.2G [02:22<01:19, 61.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 60%|██████ | 7.34G/12.2G [02:22<01:19, 60.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 60%|██████ | 7.36G/12.2G [02:22<01:22, 58.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 60%|██████ | 7.38G/12.2G [02:23<01:33, 51.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 61%|██████ | 7.39G/12.2G [02:23<01:44, 46.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 61%|██████ | 7.41G/12.2G [02:23<01:35, 50.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 61%|██████ | 7.42G/12.2G [02:24<01:34, 50.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 61%|██████ | 7.44G/12.2G [02:24<01:27, 54.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 61%|██████ | 7.46G/12.2G [02:24<01:19, 59.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 61%|██████ | 7.47G/12.2G [02:24<01:07, 70.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 61%|██████▏ | 7.48G/12.2G [02:24<01:16, 61.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 61%|██████▏ | 7.49G/12.2G [02:25<01:23, 56.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 61%|██████▏ | 7.50G/12.2G [02:25<01:25, 55.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 62%|██████▏ | 7.52G/12.2G [02:25<01:16, 60.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 62%|██████▏ | 7.54G/12.2G [02:25<01:17, 60.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 62%|██████▏ | 7.55G/12.2G [02:26<01:13, 63.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 62%|██████▏ | 7.57G/12.2G [02:26<01:08, 67.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 62%|██████▏ | 7.58G/12.2G [02:26<01:06, 69.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 62%|██████▏ | 7.60G/12.2G [02:26<01:08, 67.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 62%|██████▏ | 7.62G/12.2G [02:27<01:14, 61.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 63%|██████▎ | 7.63G/12.2G [02:27<01:09, 66.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 63%|██████▎ | 7.65G/12.2G [02:27<01:13, 61.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 63%|██████▎ | 7.66G/12.2G [02:27<01:15, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 63%|██████▎ | 7.68G/12.2G [02:28<01:14, 61.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 63%|██████▎ | 7.70G/12.2G [02:28<01:14, 60.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 63%|██████▎ | 7.71G/12.2G [02:28<01:23, 53.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 63%|██████▎ | 7.73G/12.2G [02:29<01:37, 46.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 63%|██████▎ | 7.74G/12.2G [02:29<01:29, 49.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 64%|██████▎ | 7.76G/12.2G [02:29<01:41, 43.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 64%|██████▎ | 7.78G/12.2G [02:30<01:30, 49.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 64%|██████▍ | 7.79G/12.2G [02:30<01:28, 50.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 64%|██████▍ | 7.81G/12.2G [02:30<01:28, 49.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 64%|██████▍ | 7.82G/12.2G [02:31<01:21, 53.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 64%|██████▍ | 7.84G/12.2G [02:31<01:21, 53.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 64%|██████▍ | 7.86G/12.2G [02:31<01:16, 57.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 64%|██████▍ | 7.87G/12.2G [02:31<01:16, 57.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 65%|██████▍ | 7.89G/12.2G [02:32<01:17, 55.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 65%|██████▍ | 7.90G/12.2G [02:32<01:15, 57.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 65%|██████▍ | 7.92G/12.2G [02:32<01:10, 60.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 65%|██████▌ | 7.94G/12.2G [02:32<01:04, 65.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 65%|██████▌ | 7.95G/12.2G [02:33<01:12, 58.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 65%|██████▌ | 7.97G/12.2G [02:33<01:12, 58.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 65%|██████▌ | 7.98G/12.2G [02:34<02:03, 34.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 66%|██████▌ | 8.00G/12.2G [02:34<01:49, 38.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 66%|██████▌ | 8.02G/12.2G [02:34<01:33, 44.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 66%|██████▌ | 8.03G/12.2G [02:35<01:22, 50.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 66%|██████▌ | 8.05G/12.2G [02:35<01:20, 51.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 66%|██████▌ | 8.06G/12.2G [02:35<01:15, 54.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 66%|██████▌ | 8.08G/12.2G [02:36<01:16, 53.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 66%|██████▋ | 8.10G/12.2G [02:36<01:21, 50.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 66%|██████▋ | 8.11G/12.2G [02:36<01:19, 51.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 67%|██████▋ | 8.13G/12.2G [02:37<01:18, 51.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 67%|██████▋ | 8.14G/12.2G [02:37<01:22, 49.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 67%|██████▋ | 8.16G/12.2G [02:37<01:18, 51.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 67%|██████▋ | 8.18G/12.2G [02:37<01:12, 55.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 67%|██████▋ | 8.19G/12.2G [02:38<01:11, 56.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 67%|██████▋ | 8.21G/12.2G [02:38<01:06, 60.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 67%|██████▋ | 8.22G/12.2G [02:38<01:05, 60.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 68%|██████▊ | 8.24G/12.2G [02:38<01:03, 62.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 68%|██████▊ | 8.26G/12.2G [02:39<01:11, 55.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 68%|██████▊ | 8.27G/12.2G [02:39<01:07, 58.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 68%|██████▊ | 8.29G/12.2G [02:39<01:01, 63.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 68%|██████▊ | 8.30G/12.2G [02:39<01:05, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 68%|██████▊ | 8.32G/12.2G [02:40<01:06, 58.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 68%|██████▊ | 8.34G/12.2G [02:40<01:03, 61.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 68%|██████▊ | 8.35G/12.2G [02:40<00:58, 65.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 69%|██████▊ | 8.37G/12.2G [02:41<01:02, 61.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 69%|██████▊ | 8.38G/12.2G [02:41<00:59, 64.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 69%|██████▉ | 8.40G/12.2G [02:41<00:53, 71.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 69%|██████▉ | 8.42G/12.2G [02:41<00:57, 66.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 69%|██████▉ | 8.43G/12.2G [02:41<00:58, 64.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 69%|██████▉ | 8.45G/12.2G [02:42<00:54, 69.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 69%|██████▉ | 8.46G/12.2G [02:42<00:59, 62.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 69%|██████▉ | 8.48G/12.2G [02:42<00:57, 64.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 70%|██████▉ | 8.50G/12.2G [02:42<00:54, 67.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 70%|██████▉ | 8.51G/12.2G [02:43<01:01, 60.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 70%|██████▉ | 8.53G/12.2G [02:43<01:02, 58.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 70%|██████▉ | 8.54G/12.2G [02:43<01:02, 58.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 70%|███████ | 8.56G/12.2G [02:44<01:02, 58.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 70%|███████ | 8.58G/12.2G [02:45<02:46, 21.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 70%|███████ | 8.59G/12.2G [02:46<02:14, 26.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 71%|███████ | 8.61G/12.2G [02:46<01:47, 33.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 71%|███████ | 8.62G/12.2G [02:46<01:37, 36.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 71%|███████ | 8.64G/12.2G [02:46<01:21, 43.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 71%|███████ | 8.66G/12.2G [02:47<01:13, 48.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 71%|███████ | 8.67G/12.2G [02:47<01:09, 51.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 71%|███████ | 8.69G/12.2G [02:47<01:03, 55.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 71%|███████▏ | 8.70G/12.2G [02:47<00:58, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 71%|███████▏ | 8.72G/12.2G [02:48<01:00, 58.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 72%|███████▏ | 8.74G/12.2G [02:48<00:59, 57.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 72%|███████▏ | 8.75G/12.2G [02:48<00:58, 58.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 72%|███████▏ | 8.77G/12.2G [02:48<00:52, 65.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 72%|███████▏ | 8.78G/12.2G [02:49<00:51, 66.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 72%|███████▏ | 8.80G/12.2G [02:49<00:53, 64.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 72%|███████▏ | 8.82G/12.2G [02:49<00:59, 57.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 72%|███████▏ | 8.83G/12.2G [02:49<00:52, 63.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 72%|███████▏ | 8.85G/12.2G [02:50<00:55, 60.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 73%|███████▎ | 8.86G/12.2G [02:50<00:56, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 73%|███████▎ | 8.88G/12.2G [02:50<00:51, 64.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 73%|███████▎ | 8.90G/12.2G [02:50<00:52, 63.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 73%|███████▎ | 8.91G/12.2G [02:51<00:55, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 73%|███████▎ | 8.93G/12.2G [02:51<01:07, 48.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 73%|███████▎ | 8.94G/12.2G [02:51<01:02, 52.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 73%|███████▎ | 8.96G/12.2G [02:52<01:06, 48.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▎ | 8.98G/12.2G [02:52<01:05, 49.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▎ | 8.99G/12.2G [02:52<00:55, 58.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▎ | 9.00G/12.2G [02:53<01:07, 47.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▍ | 9.01G/12.2G [02:53<01:13, 43.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▍ | 9.02G/12.2G [02:53<01:01, 51.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▍ | 9.02G/12.2G [02:53<01:23, 38.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▍ | 9.04G/12.2G [02:54<01:43, 30.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▍ | 9.06G/12.2G [02:54<01:23, 37.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▍ | 9.07G/12.2G [02:54<01:08, 45.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 74%|███████▍ | 9.09G/12.2G [02:55<01:06, 46.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 75%|███████▍ | 9.10G/12.2G [02:55<01:11, 43.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 75%|███████▍ | 9.11G/12.2G [02:55<01:04, 47.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 75%|███████▍ | 9.12G/12.2G [02:55<01:02, 49.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 75%|███████▍ | 9.12G/12.2G [02:56<01:40, 30.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 75%|███████▍ | 9.14G/12.2G [02:56<01:29, 34.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 75%|███████▍ | 9.15G/12.2G [02:56<01:15, 40.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 75%|███████▌ | 9.17G/12.2G [02:57<01:09, 43.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 75%|███████▌ | 9.18G/12.2G [02:57<01:06, 45.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 75%|███████▌ | 9.20G/12.2G [02:57<01:06, 45.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 76%|███████▌ | 9.22G/12.2G [02:58<01:00, 49.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 76%|███████▌ | 9.23G/12.2G [02:58<01:00, 49.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 76%|███████▌ | 9.25G/12.2G [02:59<01:07, 43.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 76%|███████▌ | 9.26G/12.2G [02:59<00:56, 51.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 76%|███████▌ | 9.27G/12.2G [02:59<01:37, 30.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 76%|███████▌ | 9.28G/12.2G [03:00<01:31, 31.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 76%|███████▌ | 9.30G/12.2G [03:00<01:20, 36.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 76%|███████▋ | 9.31G/12.2G [03:00<01:09, 41.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 76%|███████▋ | 9.33G/12.2G [03:01<01:01, 46.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.34G/12.2G [03:01<00:53, 53.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.35G/12.2G [03:01<01:11, 40.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.36G/12.2G [03:01<01:07, 42.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.38G/12.2G [03:02<01:04, 43.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.39G/12.2G [03:02<01:00, 46.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.40G/12.2G [03:02<00:51, 54.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.41G/12.2G [03:02<01:05, 42.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.42G/12.2G [03:02<00:55, 50.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.43G/12.2G [03:03<01:13, 37.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.44G/12.2G [03:03<01:11, 38.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 77%|███████▋ | 9.46G/12.2G [03:03<01:02, 44.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 78%|███████▊ | 9.47G/12.2G [03:04<00:59, 46.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 78%|███████▊ | 9.49G/12.2G [03:04<00:45, 59.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 78%|███████▊ | 9.50G/12.2G [03:04<01:03, 42.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 78%|███████▊ | 9.50G/12.2G [03:04<01:01, 43.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 78%|███████▊ | 9.52G/12.2G [03:05<01:00, 44.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 78%|███████▊ | 9.54G/12.2G [03:05<01:02, 42.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 78%|███████▊ | 9.55G/12.2G [03:05<00:55, 48.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 78%|███████▊ | 9.57G/12.2G [03:06<00:54, 48.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▊ | 9.58G/12.2G [03:06<00:47, 54.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▊ | 9.60G/12.2G [03:06<00:40, 64.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▊ | 9.60G/12.2G [03:07<01:00, 43.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▉ | 9.61G/12.2G [03:07<00:54, 47.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▉ | 9.62G/12.2G [03:07<01:11, 36.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▉ | 9.63G/12.2G [03:07<01:03, 40.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▉ | 9.65G/12.2G [03:08<00:56, 45.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▉ | 9.66G/12.2G [03:08<00:52, 48.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▉ | 9.68G/12.2G [03:08<00:46, 54.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 79%|███████▉ | 9.70G/12.2G [03:08<00:41, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 80%|███████▉ | 9.71G/12.2G [03:09<00:43, 57.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 80%|███████▉ | 9.73G/12.2G [03:09<00:39, 63.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 80%|███████▉ | 9.74G/12.2G [03:09<00:38, 63.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 80%|███████▉ | 9.76G/12.2G [03:09<00:41, 59.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 80%|████████ | 9.78G/12.2G [03:10<00:42, 57.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 80%|████████ | 9.79G/12.2G [03:10<00:43, 55.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 80%|████████ | 9.81G/12.2G [03:11<01:09, 34.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 80%|████████ | 9.82G/12.2G [03:11<00:58, 40.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 81%|████████ | 9.84G/12.2G [03:11<00:53, 44.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 81%|████████ | 9.86G/12.2G [03:12<00:49, 47.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 81%|████████ | 9.87G/12.2G [03:12<00:43, 53.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 81%|████████ | 9.89G/12.2G [03:12<00:40, 57.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 81%|████████ | 9.90G/12.2G [03:12<00:40, 57.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 81%|████████▏ | 9.92G/12.2G [03:12<00:35, 64.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 81%|████████▏ | 9.94G/12.2G [03:13<00:36, 62.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 82%|████████▏ | 9.95G/12.2G [03:13<00:33, 66.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 82%|████████▏ | 9.97G/12.2G [03:13<00:35, 62.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 82%|████████▏ | 9.98G/12.2G [03:13<00:33, 65.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 82%|████████▏ | 10.0G/12.2G [03:14<00:36, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 82%|████████▏ | 10.0G/12.2G [03:14<00:35, 61.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 82%|████████▏ | 10.0G/12.2G [03:14<00:36, 59.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 82%|████████▏ | 10.0G/12.2G [03:15<00:35, 61.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 82%|████████▏ | 10.1G/12.2G [03:15<00:37, 57.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 83%|████████▎ | 10.1G/12.2G [03:15<00:39, 54.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 83%|████████▎ | 10.1G/12.2G [03:16<00:42, 49.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 83%|████████▎ | 10.1G/12.2G [03:16<00:38, 55.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 83%|████████▎ | 10.1G/12.2G [03:16<00:35, 58.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 83%|████████▎ | 10.1G/12.2G [03:16<00:36, 56.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 83%|████████▎ | 10.2G/12.2G [03:17<00:36, 56.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 83%|████████▎ | 10.2G/12.2G [03:17<00:40, 50.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 84%|████████▎ | 10.2G/12.2G [03:17<00:42, 47.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 84%|████████▎ | 10.2G/12.2G [03:18<00:36, 54.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 84%|████████▍ | 10.2G/12.2G [03:18<00:39, 50.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 84%|████████▍ | 10.2G/12.2G [03:18<00:35, 55.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 84%|████████▍ | 10.3G/12.2G [03:18<00:35, 54.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 84%|████████▍ | 10.3G/12.2G [03:19<00:35, 55.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 84%|████████▍ | 10.3G/12.2G [03:19<00:32, 58.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 84%|████████▍ | 10.3G/12.2G [03:19<00:31, 60.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 85%|████████▍ | 10.3G/12.2G [03:20<00:31, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 85%|████████▍ | 10.3G/12.2G [03:20<00:34, 54.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 85%|████████▍ | 10.4G/12.2G [03:20<00:34, 53.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 85%|████████▍ | 10.4G/12.2G [03:20<00:33, 55.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 85%|████████▌ | 10.4G/12.2G [03:21<00:50, 36.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 85%|████████▌ | 10.4G/12.2G [03:22<00:45, 39.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 85%|████████▌ | 10.4G/12.2G [03:22<00:42, 42.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 85%|████████▌ | 10.4G/12.2G [03:22<00:41, 42.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 86%|████████▌ | 10.4G/12.2G [03:22<00:36, 48.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 86%|████████▌ | 10.5G/12.2G [03:23<00:31, 54.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 86%|████████▌ | 10.5G/12.2G [03:23<00:32, 52.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 86%|████████▌ | 10.5G/12.2G [03:23<00:33, 51.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 86%|████████▌ | 10.5G/12.2G [03:24<00:31, 53.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 86%|████████▋ | 10.5G/12.2G [03:24<00:33, 49.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 86%|████████▋ | 10.5G/12.2G [03:24<00:31, 52.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 87%|████████▋ | 10.6G/12.2G [03:25<00:29, 55.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 87%|████████▋ | 10.6G/12.2G [03:25<00:28, 58.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 87%|████████▋ | 10.6G/12.2G [03:25<00:28, 56.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 87%|████████▋ | 10.6G/12.2G [03:25<00:28, 55.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 87%|████████▋ | 10.6G/12.2G [03:26<00:26, 60.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 87%|████████▋ | 10.6G/12.2G [03:26<00:26, 59.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 87%|████████▋ | 10.7G/12.2G [03:26<00:25, 61.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 87%|████████▋ | 10.7G/12.2G [03:26<00:25, 60.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 88%|████████▊ | 10.7G/12.2G [03:27<00:24, 61.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 88%|████████▊ | 10.7G/12.2G [03:27<00:22, 66.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 88%|████████▊ | 10.7G/12.2G [03:27<00:22, 65.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 88%|████████▊ | 10.7G/12.2G [03:27<00:22, 64.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 88%|████████▊ | 10.8G/12.2G [03:28<00:22, 63.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 88%|████████▊ | 10.8G/12.2G [03:28<00:22, 64.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 88%|████████▊ | 10.8G/12.2G [03:28<00:22, 64.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 88%|████████▊ | 10.8G/12.2G [03:28<00:22, 62.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 89%|████████▊ | 10.8G/12.2G [03:29<00:21, 65.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 89%|████████▊ | 10.8G/12.2G [03:29<00:20, 65.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 89%|████████▉ | 10.8G/12.2G [03:29<00:20, 67.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 89%|████████▉ | 10.9G/12.2G [03:29<00:22, 58.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 89%|████████▉ | 10.9G/12.2G [03:30<00:22, 59.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 89%|████████▉ | 10.9G/12.2G [03:30<00:23, 55.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 89%|████████▉ | 10.9G/12.2G [03:30<00:22, 57.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 90%|████████▉ | 10.9G/12.2G [03:30<00:21, 58.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 90%|████████▉ | 10.9G/12.2G [03:31<00:20, 62.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 90%|████████▉ | 11.0G/12.2G [03:31<00:21, 58.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 90%|████████▉ | 11.0G/12.2G [03:31<00:20, 59.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 90%|█████████ | 11.0G/12.2G [03:31<00:18, 64.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 90%|█████████ | 11.0G/12.2G [03:32<00:17, 67.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 90%|█████████ | 11.0G/12.2G [03:32<00:21, 55.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 90%|█████████ | 11.0G/12.2G [03:33<00:24, 47.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 91%|█████████ | 11.1G/12.2G [03:33<00:21, 53.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 91%|█████████ | 11.1G/12.2G [03:33<00:19, 58.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 91%|█████████ | 11.1G/12.2G [03:33<00:19, 58.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 91%|█████████ | 11.1G/12.2G [03:33<00:18, 60.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 91%|█████████ | 11.1G/12.2G [03:34<00:17, 62.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 91%|█████████ | 11.1G/12.2G [03:34<00:23, 46.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 91%|█████████▏| 11.2G/12.2G [03:35<00:21, 49.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 91%|█████████▏| 11.2G/12.2G [03:35<00:20, 51.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 92%|█████████▏| 11.2G/12.2G [03:35<00:20, 50.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 92%|█████████▏| 11.2G/12.2G [03:35<00:18, 55.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 92%|█████████▏| 11.2G/12.2G [03:36<00:16, 60.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 92%|█████████▏| 11.2G/12.2G [03:36<00:15, 64.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 92%|█████████▏| 11.2G/12.2G [03:36<00:14, 64.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 92%|█████████▏| 11.3G/12.2G [03:36<00:15, 59.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 92%|█████████▏| 11.3G/12.2G [03:37<00:15, 59.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 93%|█████████▎| 11.3G/12.2G [03:37<00:15, 58.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 93%|█████████▎| 11.3G/12.2G [03:37<00:14, 61.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 93%|█████████▎| 11.3G/12.2G [03:37<00:14, 59.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 93%|█████████▎| 11.3G/12.2G [03:38<00:14, 60.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 93%|█████████▎| 11.4G/12.2G [03:38<00:12, 68.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 93%|█████████▎| 11.4G/12.2G [03:38<00:13, 60.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 93%|█████████▎| 11.4G/12.2G [03:38<00:13, 61.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 93%|█████████▎| 11.4G/12.2G [03:39<00:12, 64.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 94%|█████████▎| 11.4G/12.2G [03:39<00:12, 62.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 94%|█████████▎| 11.4G/12.2G [03:39<00:14, 51.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 94%|█████████▍| 11.5G/12.2G [03:40<00:13, 55.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 94%|█████████▍| 11.5G/12.2G [03:40<00:12, 58.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 94%|█████████▍| 11.5G/12.2G [03:40<00:13, 53.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 94%|█████████▍| 11.5G/12.2G [03:40<00:11, 60.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 94%|█████████▍| 11.5G/12.2G [03:41<00:11, 60.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 95%|█████████▍| 11.5G/12.2G [03:41<00:11, 56.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 95%|█████████▍| 11.6G/12.2G [03:42<00:14, 44.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 95%|█████████▍| 11.6G/12.2G [03:42<00:12, 52.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 95%|█████████▍| 11.6G/12.2G [03:42<00:11, 55.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 95%|█████████▌| 11.6G/12.2G [03:42<00:09, 60.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 95%|█████████▌| 11.6G/12.2G [03:42<00:09, 60.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 95%|█████████▌| 11.6G/12.2G [03:43<00:08, 66.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 95%|█████████▌| 11.6G/12.2G [03:43<00:08, 68.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 96%|█████████▌| 11.7G/12.2G [03:43<00:08, 64.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 96%|█████████▌| 11.7G/12.2G [03:44<00:09, 55.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 96%|█████████▌| 11.7G/12.2G [03:44<00:09, 56.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 96%|█████████▌| 11.7G/12.2G [03:44<00:08, 60.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 96%|█████████▌| 11.7G/12.2G [03:44<00:07, 63.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 96%|█████████▌| 11.7G/12.2G [03:45<00:07, 59.4MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 96%|█████████▋| 11.8G/12.2G [03:45<00:07, 63.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 96%|█████████▋| 11.8G/12.2G [03:45<00:06, 63.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 97%|█████████▋| 11.8G/12.2G [03:45<00:06, 59.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 97%|█████████▋| 11.8G/12.2G [03:46<00:06, 62.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 97%|█████████▋| 11.8G/12.2G [03:46<00:06, 62.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 97%|█████████▋| 11.8G/12.2G [03:46<00:05, 63.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 97%|█████████▋| 11.9G/12.2G [03:46<00:05, 67.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 97%|█████████▋| 11.9G/12.2G [03:47<00:08, 41.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 97%|█████████▋| 11.9G/12.2G [03:47<00:07, 43.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 98%|█████████▊| 11.9G/12.2G [03:48<00:06, 46.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 98%|█████████▊| 11.9G/12.2G [03:48<00:05, 48.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 98%|█████████▊| 11.9G/12.2G [03:48<00:05, 51.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 98%|█████████▊| 12.0G/12.2G [03:49<00:05, 48.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 98%|█████████▊| 12.0G/12.2G [03:49<00:04, 59.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 98%|█████████▊| 12.0G/12.2G [03:49<00:04, 49.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 98%|█████████▊| 12.0G/12.2G [03:49<00:04, 44.5MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 98%|█████████▊| 12.0G/12.2G [03:49<00:04, 51.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 98%|█████████▊| 12.0G/12.2G [03:50<00:03, 57.1MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 99%|█████████▊| 12.0G/12.2G [03:50<00:03, 51.9MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 99%|█████████▊| 12.0G/12.2G [03:50<00:02, 58.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 99%|█████████▉| 12.1G/12.2G [03:50<00:02, 63.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 99%|█████████▉| 12.1G/12.2G [03:51<00:02, 58.2MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 99%|█████████▉| 12.1G/12.2G [03:51<00:01, 55.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 99%|█████████▉| 12.1G/12.2G [03:51<00:01, 57.7MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 99%|█████████▉| 12.1G/12.2G [03:52<00:01, 59.0MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 99%|█████████▉| 12.1G/12.2G [03:52<00:01, 61.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 100%|█████████▉| 12.2G/12.2G [03:52<00:00, 65.8MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 100%|█████████▉| 12.2G/12.2G [03:52<00:00, 60.6MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 100%|█████████▉| 12.2G/12.2G [03:53<00:00, 57.3MB/s]
ip-26-0-169-247_131218.1720050406305829976.pt.trace.json: 100%|██████████| 12.2G/12.2G [03:53<00:00, 52.3MB/s]
|