| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_setup.py:_flush():77] Current SDK version is 0.18.1 | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_setup.py:_flush():77] Configure stats pid to 7447 | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_setup.py:_flush():77] Loading settings from /root/.config/wandb/settings | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_setup.py:_flush():77] Loading settings from /root/wandb/settings | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_setup.py:_flush():77] Loading settings from environment variables: {} | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_setup.py:_flush():77] Applying setup settings: {'mode': None, '_disable_service': None} | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_setup.py:_flush():77] Inferring run settings from compute environment: {'program_relpath': 'train.py', 'program_abspath': '/root/train.py', 'program': '/root/train.py'} | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_setup.py:_flush():77] Applying login settings: {} | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_init.py:_log_setup():532] Logging user logs to /root/wandb/run-20240927_021423-clesd0p8/logs/debug.log | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_init.py:_log_setup():533] Logging internal logs to /root/wandb/run-20240927_021423-clesd0p8/logs/debug-internal.log | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_init.py:init():616] calling init triggers | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_init.py:init():623] wandb.init called with sweep_config: {} | |
| config: {'out_dir': 'out', 'eval_interval': 100, 'log_interval': 1, 'eval_iters': 100, 'eval_only': False, 'always_save_checkpoint': True, 'init_from': 'scratch', 'checkpoint_path': '', 'wandb_log': True, 'wandb_project': 'gpt2_positional_encodings_100B', 'wandb_run_name': 'experiment', 'dataset': 'fineweb', 'gradient_accumulation_steps': 40, 'batch_size': 120, 'block_size': 512, 'n_layer': 4, 'n_head': 4, 'n_embd': 256, 'dropout': 0.0, 'bias': False, 'learning_rate': 0.0006, 'max_iters': 10000, 'weight_decay': 0.1, 'beta1': 0.9, 'beta2': 0.95, 'grad_clip': 1.0, 'decay_lr': True, 'warmup_iters': 100, 'lr_decay_iters': 10000, 'min_lr': 6e-05, 'backend': 'nccl', 'device': 'cuda', 'dtype': 'bfloat16', 'compile': True, 'embedding_types': ['polynomial_legendre', 'polynomial_chebyshev', 'random_fourier', 'wavelet'], 'attention_types': ['default'], 'collect_attention_patterns': False, 'collect_activations': False, 'eval_datasets': ['wikitext-103-v1', 'ptb', 'lambada'], 'seed': 1337} | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_init.py:init():666] starting backend | |
| 2024-09-27 02:14:23,672 INFO MainThread:7447 [wandb_init.py:init():670] setting up manager | |
| 2024-09-27 02:14:23,673 INFO MainThread:7447 [backend.py:_multiprocessing_setup():105] multiprocessing start_methods=fork,spawn,forkserver, using: spawn | |
| 2024-09-27 02:14:23,674 INFO MainThread:7447 [wandb_init.py:init():678] backend started and connected | |
| 2024-09-27 02:14:23,677 INFO MainThread:7447 [wandb_init.py:init():773] updated telemetry | |
| 2024-09-27 02:14:23,677 INFO MainThread:7447 [wandb_init.py:init():806] communicating run to backend with 90.0 second timeout | |
| 2024-09-27 02:14:24,248 INFO MainThread:7447 [wandb_init.py:init():857] starting run threads in backend | |
| 2024-09-27 02:14:24,379 INFO MainThread:7447 [wandb_run.py:_console_start():2459] atexit reg | |
| 2024-09-27 02:14:24,379 INFO MainThread:7447 [wandb_run.py:_redirect():2307] redirect: wrap_raw | |
| 2024-09-27 02:14:24,380 INFO MainThread:7447 [wandb_run.py:_redirect():2372] Wrapping output streams. | |
| 2024-09-27 02:14:24,380 INFO MainThread:7447 [wandb_run.py:_redirect():2397] Redirects installed. | |
| 2024-09-27 02:14:24,380 INFO MainThread:7447 [wandb_init.py:init():900] run started, returning control to user process | |
| 2024-09-27 08:54:14,438 WARNING MsgRouterThr:7447 [router.py:message_loop():77] message_loop has been closed | |